Commit Graph

17673 Commits

Author SHA1 Message Date
Taus Brock-Nannestad
1251bc57f5 Python: Fix bad join in TObject::literal_instantiation
Here, `context.appliesTo(n)` was being distributed across all of the
disjuncts, which caused poor performance.

The new helper predicate, `literal_node_class` should be fairly small,
since it only applies to a subset of `ControlFlowNode`s, and only
assigns a limited set of `ClassObjectInternal`s to these nodes.
2020-11-05 16:40:29 +01:00
Taus Brock-Nannestad
35a63e2411 Python: Fix bad join in regex::used_as_regex
Since the number of relevant attributes in the `re` module is fairly
small, it made sense to factor this out in a separate predicate, and
the join order also became more sensible.
2020-11-05 16:33:59 +01:00
Taus Brock-Nannestad
035e747ad5 Python: Fix slow use of regexCapture in Builtin::strValue
This is only _really_ expensive when there are a _lot_ of strings in
the database, but for this case, where we're always extracting the
same substring of the string, it's easier -- and faster -- to just
make a substring operation directly.
2020-11-05 16:33:33 +01:00
Taus Brock-Nannestad
83ba8c9bf5 Python: Add LocalSourceNode and flowsTo
This fixes the major performance problem with type tracking on
some (pathological) databases.

The interface could probably be improved a bit. In particular, I'm
thinking that we might want to have `DataFlow::exprNode` return a
`LocalSourceNode` so that a cast isn't necessary in order to use
`flowsTo`.

I have added two `cached` annotations. The one on `flowsTo` is
crucial, as performance regresses without it. The one on
`simpleLocalFlowStep` may not be needed, but Java has a similar
annotation, and to me it makes sense to have this relation cached.
2020-11-05 16:26:03 +01:00
yoff
104ff5d217 Merge pull request #4596 from RasmusWL/python-import-customizations
Python: Import Customizations into python
2020-11-03 11:49:51 +01:00
Jonas Jensen
76fd710946 Merge pull request #4571 from MathiasVP/better-syntax-for-false-positives-and-negatives-inline-expectation
C++/Python: Better syntax for false positives and negatives in inline expectations
2020-11-03 11:05:21 +01:00
Rasmus Wriedt Larsen
cac336d053 Python: Import Customizations into python
Using the pattern from JS and Java to make this the _first_ import in `<lang>.qll`
2020-11-03 10:23:05 +01:00
Jonas Jensen
5680b2df13 Merge remote-tracking branch 'upstream/main' into better-syntax-for-false-positives-and-negatives-inline-expectation
Required fixing up semantic conflicts in tests.

Conflicts:
	python/ql/test/experimental/library-tests/frameworks/stdlib/Decoding.py
2020-11-03 09:47:26 +01:00
Anders Schack-Mulligen
89361a3b75 Merge pull request #3812 from luchua-bc/java-android-remote-source
Java: Add remote source of Android intent extra
2020-11-03 09:35:40 +01:00
Taus
82f37e97c8 Merge pull request #4595 from tausbn/python-promote-dataflow-libraries
Python: Promote experimental data-flow libraries
2020-11-02 23:55:05 +01:00
Taus Brock-Nannestad
69d2d714a2 Python: Update identical-files.json 2020-11-02 23:25:58 +01:00
Taus Brock-Nannestad
5dadb0f476 Python: Fix imports in tests 2020-11-02 23:02:29 +01:00
Taus Brock-Nannestad
8752b1af1e Python: Fix up remaining data-flow library copies 2020-11-02 23:02:04 +01:00
Taus Brock-Nannestad
b7773849d7 Python: Fix up some comments 2020-11-02 22:57:40 +01:00
Taus Brock-Nannestad
d8c554ed4f Python: Add redirects to old data-flow libraries 2020-11-02 22:20:16 +01:00
Taus Brock-Nannestad
a5121babc8 Python: The one with changes that don't look like renames anymore 2020-11-02 22:19:15 +01:00
Taus Brock-Nannestad
fb6a02e060 Python: More import fixups 2020-11-02 22:17:42 +01:00
Taus Brock-Nannestad
5156bf756d Python: Promote data-flow libraries
Step 1: Moving stuff around. Also includes a bit of import renaming.
2020-11-02 22:15:38 +01:00
Taus
25e88ed585 Merge pull request #4588 from yoff/python-pep-249
Python: Model PEP 249
2020-11-02 18:57:15 +01:00
yoff
1535ce1f5d Apply suggestions from code review
Co-authored-by: Taus <tausbn@github.com>
2020-11-02 18:26:37 +01:00
Rasmus Lerchedahl Petersen
ea74c7f12b Python: add tests 2020-11-02 17:59:51 +01:00
Rasmus Lerchedahl Petersen
cb47b57440 Python: fix typos 2020-11-02 17:22:01 +01:00
yoff
222a2f7f5d Update python/ql/src/experimental/semmle/python/frameworks/PEP249.qll
Co-authored-by: Taus <tausbn@github.com>
2020-11-02 17:19:21 +01:00
yoff
ca34219ec5 Update python/ql/src/experimental/semmle/python/frameworks/PEP249.qll
Co-authored-by: Taus <tausbn@github.com>
2020-11-02 17:19:10 +01:00
yoff
1c83d67513 Update python/ql/src/experimental/semmle/python/frameworks/PEP249.qll
Co-authored-by: Taus <tausbn@github.com>
2020-11-02 17:18:18 +01:00
Rasmus Lerchedahl Petersen
6a81987284 Python: Rename and add docs 2020-11-02 17:17:12 +01:00
Taus
2dfffdbab8 Merge pull request #4590 from RasmusWL/python-model-base64
Python: Model encoding/decoding with base64 module
2020-11-02 17:00:21 +01:00
Rasmus Lerchedahl Petersen
6d850b2e0c Merge branch 'main' of github.com:github/codeql into python-pep-249 2020-11-02 16:58:31 +01:00
yoff
ac85a77ac5 Update python/ql/src/experimental/semmle/python/Frameworks.qll
Co-authored-by: Rasmus Wriedt Larsen <rasmuswriedtlarsen@gmail.com>
2020-11-02 16:40:22 +01:00
Calum Grant
cb527cae73 Merge pull request #4583 from tausbn/python-test-2
Python: Promote experimental queries
2020-11-02 14:42:27 +00:00
luchua-bc
864411b4b9 Updates to Android stub classes 2020-11-02 14:06:44 +00:00
Rasmus Wriedt Larsen
247fd4f5f3 Python: Make encoding/decoding preserve taint automatically
With the way we have set things up, there is no way to opt out of this behavior.
2020-11-02 14:53:30 +01:00
Rasmus Lerchedahl Petersen
36e364d6ef Python: Django use PEP 249 2020-11-02 14:49:34 +01:00
Rasmus Wriedt Larsen
66f5d0d9d5 Python: Model encoding/decoding with base64 module 2020-11-02 14:44:53 +01:00
Rasmus Wriedt Larsen
eff244db71 Python: Add Encoding concept
I wasn't able to find a good opposite of "parsing", so left that out of the list
of intended purposes.
2020-11-02 14:19:20 +01:00
Taus Brock-Nannestad
8147ad4e0b Python: Remove irrelevant files 2020-11-02 14:08:59 +01:00
Taus Brock-Nannestad
f84ab2fa99 Python: Remove old data-flow tests 2020-11-02 14:07:04 +01:00
Taus Brock-Nannestad
9d6c07c8df Python: Add copy of old queries 2020-11-02 13:35:20 +01:00
luchua-bc
8da9b9d3ea Add documentation to new library method and use the singular form 2020-11-02 10:53:46 +00:00
Taus Brock-Nannestad
b620b9b7c6 Python: Fixup CWE-022 tests
This was a bit of a mess, since there was crosstalk between the
TarSlip and PathInjection queries. (Also one of these needs the
`options` file to be in one way, and the other not). To fix this, I
split these out into separate directories.
2020-11-02 11:46:28 +01:00
Taus Brock-Nannestad
af7626a6b3 Python: Fixup CWE-079 tests 2020-11-02 11:46:02 +01:00
Taus Brock-Nannestad
57b51090ef Python: Fixup CWE-094 tests 2020-11-02 11:45:44 +01:00
Taus Brock-Nannestad
ebb593466d Python: Fixup CWE-089 tests 2020-11-02 11:45:14 +01:00
Taus Brock-Nannestad
7a395bf7c8 Python: Fixup CWE-078 tests. 2020-11-02 11:44:42 +01:00
Taus Brock-Nannestad
52dc905037 Python: Fixup CWE-502 tests. 2020-11-02 11:44:00 +01:00
Rasmus Lerchedahl Petersen
d35bf8f446 Python: Update comments on PEP 249 module 2020-11-02 11:22:51 +01:00
Tom Hvitved
b78234fe83 Merge pull request #4586 from hvitved/csharp/dataflow/common-sub-type-join-fix
C#: Fix bad join-order in `commonSubTypeGeneral/2`
2020-11-02 09:40:54 +01:00
Tom Hvitved
e89a463b25 C#: Fix bad join-order in commonSubTypeGeneral/2 2020-11-01 20:08:14 +01:00
Rasmus Lerchedahl Petersen
0240670d62 Python: import frameworks 2020-11-01 18:02:36 +01:00
Rasmus Lerchedahl Petersen
babcf7acd9 Python: add two implementations of PEP249 2020-11-01 16:01:05 +01:00