Commit Graph

769 Commits

Author SHA1 Message Date
Rasmus Wriedt Larsen
95ac2c8edd Python: Add another dynamic __all__ test 2021-03-31 17:31:55 +02:00
Rasmus Wriedt Larsen
ab3edf37d7 Python: Handle __all__ assigned to a tuple
Examples where this is used in real code:

- 76c0b32f82/django/core/files/temp.py (L24)
- 76c0b32f82/django/contrib/gis/gdal/__init__.py (L44-L49)
2021-03-31 17:25:19 +02:00
Rasmus Wriedt Larsen
43306f4700 Python: Add tests for Module.declaredInAll 2021-03-31 17:24:17 +02:00
Calum Grant
c26d05b1d5 Merge pull request #5532 from RasmusWL/python-cleanup
Python: Delete filter queries, code duplication library, and precision tag from metric queries
2021-03-29 17:16:43 +01:00
Rasmus Wriedt Larsen
bd4934380a Python: Remove code duplication library 2021-03-25 15:27:55 +01:00
Rasmus Wriedt Larsen
70974ea197 Python: Fix grammar in QLDoc
Co-authored-by: yoff <lerchedahl@gmail.com>
2021-03-24 14:06:06 +01:00
Rasmus Wriedt Larsen
8d0f6086af Python: Model django forms/fields
I'm not feeling 100% confident about `SelfRefMixin`, but since I needed it for
both DjangoViewClass and DjangoFormClass, I wanted to avoid copy-pasting this
code around. However, I'm not so opitimistic about it that I want to add it to a
sharable utility qll file :D
2021-03-23 13:57:38 +01:00
Rasmus Wriedt Larsen
c8a6e837b5 Python: Model QuerySet chains in django 2021-03-22 14:38:54 +01:00
Rasmus Wriedt Larsen
3a83ecf067 Python: Add test for taint in django forms/fields 2021-03-22 10:03:32 +01:00
Rasmus Wriedt Larsen
701b935564 Python: Add example of QuerySet chain (django) 2021-03-22 00:57:43 +01:00
Rasmus Wriedt Larsen
d9079e34e3 Python: Move framework tests out of experimental
Since they are not experimental anymore 😄
2021-03-19 15:51:54 +01:00
Taus Brock-Nannestad
404649d5f1 Python: Get rid of superfluous options file 2021-02-24 11:24:43 +01:00
Taus Brock-Nannestad
af644a0adb Python: Decrease import depth in regex tests
These were increased because of the indirection needed to get to the
regex flags, but as we no longer rely on this, we can make do with a
smaller import depth.
2021-02-24 10:23:01 +01:00
Taus Brock-Nannestad
e812eb777d Python: Port URL sanitisation queries to API graphs
Really, this boils down to "Port `re` library model to use API graphs
instead of points-to", which is what this PR actually does.

Instead of using points-to to track flags, we use a type tracker. To
handle multiple flags at the same time, we add additional flow from

`x` to `x | y` and `y | x`

and, as an added bonus, the above with `+` instead of `|`, neatly
fixing https://github.com/github/codeql/issues/4707

I had to modify the `Qualified.ql` test slightly, as it now had a
result stemming from the standard library (in `warnings.py`) that
points-to previously ignored.

It might be possible to implement this as a type tracker on
`LocalSourceNode`s, but with the added steps for the above operations,
this was not obvious to me, and so I opted for the simpler
"`smallstep`" variant.
2021-02-23 22:02:35 +01:00
Rasmus Wriedt Larsen
014fbfa86b Python: Add regex FP with + for flags
Notice that there is no new results for line 54

I also added a test for the short-named version of a flag, just since I didn't
see any of those already. That just works out of the box (due to points-to).
2020-11-27 14:57:11 +01:00
CodeQL CI
d3cded330e Merge pull request #4693 from RasmusWL/python-add-import-test-shadowing-stdlib-v2
Approved by tausbn
2020-11-27 10:32:21 +00:00
Rasmus Wriedt Larsen
7b4e890e7b Python: Fix grammar
Co-authored-by: Taus <tausbn@github.com>
2020-11-27 11:00:30 +01:00
Rasmus Lerchedahl Petersen
88643da01f Python: Use default getEnclosingCallable
for `RetrunNode`
2020-11-25 08:19:07 +01:00
Rasmus Wriedt Larsen
7e407d43d2 Python: Change (single) test to match codeql database create 2020-11-19 14:56:18 +01:00
Rasmus Wriedt Larsen
8ffcff0824 Python: Add example of top-level module shadowing stdlib
Although this test is added under the `wrong` folder, the current results from
this CodeQL test is actually correct (compared with the Python
interpreter). However, they don't match what the extractor does when invoked
with `codeql database create`.

Since I deemed it "more than an easy fix" to change the extractor behavior for
`codeql database create` to match the real python behavior, and it turned out to
be quite a challenge to change the extractor behavior for all tests, I'm just
going to make THIS ONE test-case behave like the extractor will with `codeql
database create`...

This is a first commit, to show how the extractor works with qltest by default.

Inspired by the debugging in https://github.com/github/codeql/issues/4640
2020-11-19 14:56:17 +01:00
Rasmus Lerchedahl Petersen
109d55eb25 Python: Make ParameterNode a CfgNode
Add a step from that `CfgNode` to the corresponding `EssaNode`.
The intended effect is seen in `ImpliesDataflow.expected`.
The efeect seen in other `.expected`-files is that parameter nodes
change type, that the extra steps are seen, and that flow from
`EssaVar`s is mirrored in flow from `CfgNode`s.
There is one surprise, which is the `.0` node in
`coverage/localFlow.expected`.
2020-11-10 11:35:50 +01:00
yoff
45317bcec9 Update python/ql/test/library-tests/PointsTo/new/code/w_function_values.py
Co-authored-by: Rasmus Wriedt Larsen <rasmuswriedtlarsen@gmail.com>
2020-11-06 15:03:20 +01:00
Rasmus Lerchedahl Petersen
fe186bf854 Python: Add test 2020-11-06 13:30:11 +01:00
Rasmus Lerchedahl Petersen
64b9e9150e Python: only show results in extracted files 2020-11-06 12:01:16 +01:00
Rasmus Lerchedahl Petersen
6cecd3ba83 Python: Move and rename query 2020-11-05 11:49:39 +01:00
Erik Krogh Kristensen
e89e99deaa Merge pull request #4461 from erik-krogh/pyPrint
Python: implement printAst for Python
2020-10-22 09:37:10 +02:00
Rasmus Wriedt Larsen
09a2a6cdfd Python: Fix django re_path kwarg test
Since it was using the wrong keyword argument name before :D
2020-10-16 11:12:12 +02:00
Erik Krogh Kristensen
1d2ca42d49 update expected output 2020-10-15 13:42:13 +02:00
Erik Krogh Kristensen
9b7c59f4b4 implement printAst for Python 2020-10-12 21:17:46 +02:00
Taus Brock-Nannestad
b5d05f99c9 Python: Fix test output 2020-10-02 12:04:43 +02:00
Taus Brock-Nannestad
45eccb2521 Python: Fix test failures. 2020-07-07 17:01:17 +02:00
Taus Brock-Nannestad
f07a7bf8cf Python: Autoformat everything using qlformat.
Will need subsequent PRs fixing up test failures (due to deprecated
methods moving around), but other than that everything should be
straight-forward.
2020-07-07 15:43:52 +02:00
Rasmus Wriedt Larsen
67be45f045 Merge branch 'master' into python-fix-django-taint-sinks 2020-07-02 11:55:42 +02:00
Rasmus Lerchedahl Petersen
f6c59abcd9 Merge branch 'master' of github.com:github/codeql into UnmatchableDollar
to make CodeScan happy
2020-06-24 11:04:07 +02:00
Taus Brock-Nannestad
48e3e9c0b4 Python: Do all the renames. 2020-06-19 17:02:47 +02:00
Taus Brock-Nannestad
06d6913a20 Python: Change "sanity" to "consistency". 2020-06-19 16:55:59 +02:00
Rasmus Wriedt Larsen
c0043eb9db Python: Don't treat re.escape(...) as a regex
Fixes https://github.com/github/codeql/issues/3712
2020-06-15 11:54:14 +02:00
Rasmus Wriedt Larsen
7601bd497e Python: Add tests for re.escape FP 2020-06-15 11:34:42 +02:00
semmle-qlci
f7c6b1364b Merge pull request #3640 from RasmusWL/python-handle-3.8-enum-convert
Approved by tausbn
2020-06-10 17:19:22 +01:00
Rasmus Wriedt Larsen
baa415fec8 Python: Add points-to regression for metaclass 2020-06-08 15:03:46 +02:00
Rasmus Wriedt Larsen
1ff369f62d Python: Update test results for fabric.api.execute 2020-06-04 16:30:03 +02:00
Rasmus Wriedt Larsen
6cba2fe4f8 Python: Model Django response sinks that are not vuln to XSS
Since HttpResponse is not *only* used for XSS, it is still valuable to know the
content is send as part of the response.

The *proper* solution to this problem of not all HttpResponses being vulnerable
to XSS is probably to define a new abstract class in Http.qll called
HttpResponseXSSVulnerableSink (or similar). I would like to model a few more
libraries/frameworks before fully comitting to an approach though.
2020-05-26 16:45:46 +02:00
Rasmus Wriedt Larsen
9c75a39b81 Python: Extend command-injection to handle fabric.api.execute 2020-05-26 10:22:27 +02:00
Rasmus Wriedt Larsen
e04d1ffcd2 Python: Add test for fabric.api.execute 2020-05-26 10:20:22 +02:00
semmle-qlci
8146073c74 Merge pull request #3553 from RasmusWL/python-fix-tainttracking-import
Approved by tausbn
2020-05-25 14:18:54 +01:00
Rasmus Wriedt Larsen
f602f3e1c7 Python: Use proper import for semmle.python.dataflow.TaintTracking
It was moved in 637677d515, but imports were not
updated.
2020-05-25 13:45:49 +02:00
semmle-qlci
ac1a338390 Merge pull request #3407 from RasmusWL/python-add-BoundMethodValue-v2
Approved by tausbn
2020-05-25 12:00:45 +01:00
Rasmus Wriedt Larsen
49d7e12acd Python: Remove unnecessary restriction from getNamedArgumentForCall
As agreed in https://github.com/github/codeql/pull/3407
2020-05-25 10:17:37 +02:00
Rasmus Wriedt Larsen
4fc3cae646 Python: Add test for how arguments to *args and **kwargs are handled 2020-05-25 10:16:10 +02:00
Rasmus Lerchedahl Petersen
4d6ad32f04 Python: Update test expectations.
As ar as I can tell, all these are improvements
2020-05-20 08:11:03 +02:00