Commit Graph

689 Commits

Author SHA1 Message Date
Rasmus Lerchedahl Petersen
5a4e661e60 Merge branch 'main' of github.com:github/codeql into python-support-pathlib 2021-04-22 15:04:21 +02:00
Rasmus Wriedt Larsen
bf6f5074c2 Python: Port cryptodome tests to crypto
I don't know if this is really a smart test-setup... I feel a bit stupid when
doing this xD
2021-04-22 14:51:19 +02:00
Rasmus Wriedt Larsen
f8254381f3 Python: Add MISSING: CryptographicOperationAlgorithm annotations
For RSA it's unclear what the algorithm name should even be. Signatures based on
RSA private keys with PSS scheme is ok, but with pkcs#1 v1.5 they are
weak/vulnerable. So clearly just putting RSA as the algorithm name is not enough
information...

and that problem is also why I wanted to do this commit separetely (to call
extra atten to this).
2021-04-22 14:51:18 +02:00
Rasmus Wriedt Larsen
23140dfb76 Python: Add CryptographicOperation modeling for Cryptodome 2021-04-22 14:51:17 +02:00
Rasmus Wriedt Larsen
1b2ed9d99a Python: Align cryptodome tests 2021-04-22 14:51:16 +02:00
Rasmus Wriedt Larsen
5a9e27c6fc Merge branch 'main' into django-3.2 2021-04-21 17:15:47 +02:00
CodeQL CI
30d7f0dc98 Merge pull request #5687 from RasmusWL/inline-taint-tests
Approved by yoff
2021-04-21 06:24:12 -07:00
Rasmus Wriedt Larsen
59c6f76457 Python: Add test for new response.headers in Django
See https://docs.djangoproject.com/en/3.2/ref/request-response/#setting-header-fields
2021-04-21 13:55:22 +02:00
Rasmus Wriedt Larsen
2302c8d5fa Python: Model new alias method on django QuerySets 2021-04-21 13:52:38 +02:00
Rasmus Wriedt Larsen
63a2657aef Merge branch 'main' into inline-taint-tests 2021-04-21 10:02:55 +02:00
Taus
31bd701bd5 Python: Final LocalSourceNode fixes 2021-04-20 12:59:33 +00:00
Rasmus Wriedt Larsen
3e7dc12246 Python: Port taint tests to use inline expectations
The meat of this PR is described in the new python/ql/test/experimental/meta/InlineTaintTest.qll file:

> Defines a InlineExpectationsTest for checking whether any arguments in
> `ensure_tainted` and `ensure_not_tainted` calls are tainted.
>
> Also defines query predicates to ensure that:
> - if any arguments to `ensure_not_tainted` are tainted, their annotation is marked with `SPURIOUS`.
> - if any arguments to `ensure_tainted` are not tainted, their annotation is marked with `MISSING`.
>
> The functionality of this module is tested in `ql/test/experimental/meta/inline-taint-test-demo`.
2021-04-15 18:00:33 +02:00
Rasmus Wriedt Larsen
b359205d17 Python: Add taint tests for .get() in flask 2021-04-15 14:53:44 +02:00
Rasmus Lerchedahl Petersen
02e41d8018 Python: update annotations
This because `resolve` accesses the file system,
I am open to not include that fact in the modelling.
2021-04-15 10:49:22 +02:00
Rasmus Lerchedahl Petersen
52a9040d73 Python update tests 2021-04-15 09:46:53 +02:00
Rasmus Lerchedahl Petersen
8489403051 Python: Add some tests for pathlib 2021-04-15 09:40:23 +02:00
CodeQL CI
bc56d16c18 Merge pull request #5485 from RasmusWL/django-queryset-chains
Approved by tausbn
2021-04-12 08:49:31 -07:00
Rasmus Wriedt Larsen
bc49bc7095 Python: Add variable with underscore to __all__ tests 2021-04-06 11:54:25 +02:00
Rasmus Wriedt Larsen
224d3790b5 Python: Highlight all_indirect.py is not super important
At least not in my mind
2021-04-06 11:50:04 +02:00
Rasmus Wriedt Larsen
b11703cc74 Python: all_dybamic2 => all_indirect 2021-04-06 11:49:55 +02:00
Rasmus Wriedt Larsen
95ac2c8edd Python: Add another dynamic __all__ test 2021-03-31 17:31:55 +02:00
Rasmus Wriedt Larsen
ab3edf37d7 Python: Handle __all__ assigned to a tuple
Examples where this is used in real code:

- 76c0b32f82/django/core/files/temp.py (L24)
- 76c0b32f82/django/contrib/gis/gdal/__init__.py (L44-L49)
2021-03-31 17:25:19 +02:00
Rasmus Wriedt Larsen
43306f4700 Python: Add tests for Module.declaredInAll 2021-03-31 17:24:17 +02:00
Calum Grant
c26d05b1d5 Merge pull request #5532 from RasmusWL/python-cleanup
Python: Delete filter queries, code duplication library, and precision tag from metric queries
2021-03-29 17:16:43 +01:00
Rasmus Wriedt Larsen
bd4934380a Python: Remove code duplication library 2021-03-25 15:27:55 +01:00
Rasmus Wriedt Larsen
70974ea197 Python: Fix grammar in QLDoc
Co-authored-by: yoff <lerchedahl@gmail.com>
2021-03-24 14:06:06 +01:00
Rasmus Wriedt Larsen
8d0f6086af Python: Model django forms/fields
I'm not feeling 100% confident about `SelfRefMixin`, but since I needed it for
both DjangoViewClass and DjangoFormClass, I wanted to avoid copy-pasting this
code around. However, I'm not so opitimistic about it that I want to add it to a
sharable utility qll file :D
2021-03-23 13:57:38 +01:00
Rasmus Wriedt Larsen
c8a6e837b5 Python: Model QuerySet chains in django 2021-03-22 14:38:54 +01:00
Rasmus Wriedt Larsen
3a83ecf067 Python: Add test for taint in django forms/fields 2021-03-22 10:03:32 +01:00
Rasmus Wriedt Larsen
701b935564 Python: Add example of QuerySet chain (django) 2021-03-22 00:57:43 +01:00
Rasmus Wriedt Larsen
d9079e34e3 Python: Move framework tests out of experimental
Since they are not experimental anymore 😄
2021-03-19 15:51:54 +01:00
Taus Brock-Nannestad
404649d5f1 Python: Get rid of superfluous options file 2021-02-24 11:24:43 +01:00
Taus Brock-Nannestad
af644a0adb Python: Decrease import depth in regex tests
These were increased because of the indirection needed to get to the
regex flags, but as we no longer rely on this, we can make do with a
smaller import depth.
2021-02-24 10:23:01 +01:00
Taus Brock-Nannestad
e812eb777d Python: Port URL sanitisation queries to API graphs
Really, this boils down to "Port `re` library model to use API graphs
instead of points-to", which is what this PR actually does.

Instead of using points-to to track flags, we use a type tracker. To
handle multiple flags at the same time, we add additional flow from

`x` to `x | y` and `y | x`

and, as an added bonus, the above with `+` instead of `|`, neatly
fixing https://github.com/github/codeql/issues/4707

I had to modify the `Qualified.ql` test slightly, as it now had a
result stemming from the standard library (in `warnings.py`) that
points-to previously ignored.

It might be possible to implement this as a type tracker on
`LocalSourceNode`s, but with the added steps for the above operations,
this was not obvious to me, and so I opted for the simpler
"`smallstep`" variant.
2021-02-23 22:02:35 +01:00
Rasmus Wriedt Larsen
014fbfa86b Python: Add regex FP with + for flags
Notice that there is no new results for line 54

I also added a test for the short-named version of a flag, just since I didn't
see any of those already. That just works out of the box (due to points-to).
2020-11-27 14:57:11 +01:00
CodeQL CI
d3cded330e Merge pull request #4693 from RasmusWL/python-add-import-test-shadowing-stdlib-v2
Approved by tausbn
2020-11-27 10:32:21 +00:00
Rasmus Wriedt Larsen
7b4e890e7b Python: Fix grammar
Co-authored-by: Taus <tausbn@github.com>
2020-11-27 11:00:30 +01:00
Rasmus Lerchedahl Petersen
88643da01f Python: Use default getEnclosingCallable
for `RetrunNode`
2020-11-25 08:19:07 +01:00
Rasmus Wriedt Larsen
7e407d43d2 Python: Change (single) test to match codeql database create 2020-11-19 14:56:18 +01:00
Rasmus Wriedt Larsen
8ffcff0824 Python: Add example of top-level module shadowing stdlib
Although this test is added under the `wrong` folder, the current results from
this CodeQL test is actually correct (compared with the Python
interpreter). However, they don't match what the extractor does when invoked
with `codeql database create`.

Since I deemed it "more than an easy fix" to change the extractor behavior for
`codeql database create` to match the real python behavior, and it turned out to
be quite a challenge to change the extractor behavior for all tests, I'm just
going to make THIS ONE test-case behave like the extractor will with `codeql
database create`...

This is a first commit, to show how the extractor works with qltest by default.

Inspired by the debugging in https://github.com/github/codeql/issues/4640
2020-11-19 14:56:17 +01:00
Rasmus Lerchedahl Petersen
109d55eb25 Python: Make ParameterNode a CfgNode
Add a step from that `CfgNode` to the corresponding `EssaNode`.
The intended effect is seen in `ImpliesDataflow.expected`.
The efeect seen in other `.expected`-files is that parameter nodes
change type, that the extra steps are seen, and that flow from
`EssaVar`s is mirrored in flow from `CfgNode`s.
There is one surprise, which is the `.0` node in
`coverage/localFlow.expected`.
2020-11-10 11:35:50 +01:00
yoff
45317bcec9 Update python/ql/test/library-tests/PointsTo/new/code/w_function_values.py
Co-authored-by: Rasmus Wriedt Larsen <rasmuswriedtlarsen@gmail.com>
2020-11-06 15:03:20 +01:00
Rasmus Lerchedahl Petersen
fe186bf854 Python: Add test 2020-11-06 13:30:11 +01:00
Rasmus Lerchedahl Petersen
64b9e9150e Python: only show results in extracted files 2020-11-06 12:01:16 +01:00
Rasmus Lerchedahl Petersen
6cecd3ba83 Python: Move and rename query 2020-11-05 11:49:39 +01:00
Erik Krogh Kristensen
e89e99deaa Merge pull request #4461 from erik-krogh/pyPrint
Python: implement printAst for Python
2020-10-22 09:37:10 +02:00
Rasmus Wriedt Larsen
09a2a6cdfd Python: Fix django re_path kwarg test
Since it was using the wrong keyword argument name before :D
2020-10-16 11:12:12 +02:00
Erik Krogh Kristensen
1d2ca42d49 update expected output 2020-10-15 13:42:13 +02:00
Erik Krogh Kristensen
9b7c59f4b4 implement printAst for Python 2020-10-12 21:17:46 +02:00
Taus Brock-Nannestad
b5d05f99c9 Python: Fix test output 2020-10-02 12:04:43 +02:00