Commit Graph

1688 Commits

Author SHA1 Message Date
Rasmus Wriedt Larsen
5a9e27c6fc Merge branch 'main' into django-3.2 2021-04-21 17:15:47 +02:00
CodeQL CI
30d7f0dc98 Merge pull request #5687 from RasmusWL/inline-taint-tests
Approved by yoff
2021-04-21 06:24:12 -07:00
Rasmus Wriedt Larsen
59c6f76457 Python: Add test for new response.headers in Django
See https://docs.djangoproject.com/en/3.2/ref/request-response/#setting-header-fields
2021-04-21 13:55:22 +02:00
Rasmus Wriedt Larsen
2302c8d5fa Python: Model new alias method on django QuerySets 2021-04-21 13:52:38 +02:00
Rasmus Wriedt Larsen
63a2657aef Merge branch 'main' into inline-taint-tests 2021-04-21 10:02:55 +02:00
Rasmus Lerchedahl Petersen
9c893cb0f4 Merge branch 'main' of github.com:github/codeql into python-port-insecure-protocol 2021-04-20 16:33:03 +02:00
Taus
a55b43b67e Python: Use LocalSourceNode throughout step
This commit does a lot of stuff all at once, so here are the main
highlights:

In `TypeTracker.qll`, we change `StepSummary::step` to step only between
source nodes. Because reads and writes of global variables happen in two
different (jump) steps, this requires the intermediate
`ModuleVariableNode` to _also_ be a `LocalSourceNode`, and we therefore
modify the charpred for that class accordingly. (This also means
changing a few of the tests to account for these new source nodes.)

In addition, we change `TypeTracker::step` to likewise step between
local source nodes.

Next, to enable the use of the `track` convenience method on nodes, we
add some pragmas to `TypeTracker::step` that prevent bad joins from
occurring. With this, we can eliminate all of the manual type tracker
join predicates.

Next, we observe that because `StepSummary::step` now uses `flowsTo`, it
automatically encapsulates all local-flow steps. In particular this
means we do not have to use `typePreservingStep` in `smallstep`, but can
use `jumpStep` directly. A similar observation applies to
`TypeTracker::smallstep`.

Having done this, we no longer need `typePreservingStep`, so we get rid
of it.
2021-04-20 12:59:33 +00:00
Taus
31bd701bd5 Python: Final LocalSourceNode fixes 2021-04-20 12:59:33 +00:00
Rasmus Wriedt Larsen
d607c13ab6 Python: Taint tests: include elment for forgotten MISSING 2021-04-19 15:01:42 +02:00
Rasmus Wriedt Larsen
9585390941 Python: Taint tests, report error location first
To better match the standard output from inline expectation tests
2021-04-19 14:59:47 +02:00
Rasmus Wriedt Larsen
b2cb284ff2 Python: Add more examples of what is ok with new taint tests 2021-04-19 14:56:20 +02:00
Rasmus Lerchedahl Petersen
a8280f9b12 Python: update test expectation 2021-04-16 08:25:29 +02:00
Rasmus Wriedt Larsen
3e7dc12246 Python: Port taint tests to use inline expectations
The meat of this PR is described in the new python/ql/test/experimental/meta/InlineTaintTest.qll file:

> Defines a InlineExpectationsTest for checking whether any arguments in
> `ensure_tainted` and `ensure_not_tainted` calls are tainted.
>
> Also defines query predicates to ensure that:
> - if any arguments to `ensure_not_tainted` are tainted, their annotation is marked with `SPURIOUS`.
> - if any arguments to `ensure_tainted` are not tainted, their annotation is marked with `MISSING`.
>
> The functionality of this module is tested in `ql/test/experimental/meta/inline-taint-test-demo`.
2021-04-15 18:00:33 +02:00
Rasmus Wriedt Larsen
b359205d17 Python: Add taint tests for .get() in flask 2021-04-15 14:53:44 +02:00
Rasmus Lerchedahl Petersen
02e41d8018 Python: update annotations
This because `resolve` accesses the file system,
I am open to not include that fact in the modelling.
2021-04-15 10:49:22 +02:00
Rasmus Lerchedahl Petersen
3eb1813584 Python: update test expectations 2021-04-15 10:47:49 +02:00
Rasmus Lerchedahl Petersen
52a9040d73 Python update tests 2021-04-15 09:46:53 +02:00
Rasmus Lerchedahl Petersen
8489403051 Python: Add some tests for pathlib 2021-04-15 09:40:23 +02:00
Taus
6d4ddc0329 Merge pull request #5614 from tausbn/python-allow-absolute-imports-from-source-directory
Python: Allow absolute imports from source directory
2021-04-12 18:02:00 +02:00
CodeQL CI
bc56d16c18 Merge pull request #5485 from RasmusWL/django-queryset-chains
Approved by tausbn
2021-04-12 08:49:31 -07:00
Rasmus Wriedt Larsen
364d48948f Merge pull request #3810 from dilanbhalla/syntaxpython
Python: Function/Class Naming Convention (Syntax)
2021-04-12 10:42:17 +02:00
Rasmus Lerchedahl Petersen
9f91dde76f Python: Update test expectation after comment 2021-04-12 09:58:06 +02:00
jorgectf
7819d1a30b Generate .expected 2021-04-08 23:45:26 +02:00
jorgectf
4f85de87de Add qlref 2021-04-08 23:45:12 +02:00
jorgectf
95bfdc4955 Move tests to /test 2021-04-08 23:45:03 +02:00
yoff
38daeb4df2 Apply suggestions from code review
Co-authored-by: Rasmus Wriedt Larsen <rasmuswriedtlarsen@gmail.com>
2021-04-07 15:50:51 +02:00
Rasmus Lerchedahl Petersen
a0e3e3afaf Python: adjust test expectations 2021-04-07 08:22:36 +02:00
Rasmus Lerchedahl Petersen
094d2f3b7d Python: clean up tests 2021-04-06 22:59:58 +02:00
Taus
b44db460f6 Python: Only track modules that are imported 2021-04-06 19:55:43 +00:00
Taus Brock-Nannestad
8e11abca40 Revert "Merge pull request #5552 from RasmusWL/revert-import-change"
This reverts commit 49d1937dc4, reversing
changes made to d4877a9038.
2021-04-06 17:39:41 +02:00
Rasmus Wriedt Larsen
bc49bc7095 Python: Add variable with underscore to __all__ tests 2021-04-06 11:54:25 +02:00
Rasmus Wriedt Larsen
224d3790b5 Python: Highlight all_indirect.py is not super important
At least not in my mind
2021-04-06 11:50:04 +02:00
Rasmus Wriedt Larsen
b11703cc74 Python: all_dybamic2 => all_indirect 2021-04-06 11:49:55 +02:00
Rasmus Wriedt Larsen
95ac2c8edd Python: Add another dynamic __all__ test 2021-03-31 17:31:55 +02:00
Rasmus Wriedt Larsen
ab3edf37d7 Python: Handle __all__ assigned to a tuple
Examples where this is used in real code:

- 76c0b32f82/django/core/files/temp.py (L24)
- 76c0b32f82/django/contrib/gis/gdal/__init__.py (L44-L49)
2021-03-31 17:25:19 +02:00
Rasmus Wriedt Larsen
43306f4700 Python: Add tests for Module.declaredInAll 2021-03-31 17:24:17 +02:00
Rasmus Wriedt Larsen
51c27de049 Merge branch 'main' into revert-import-change 2021-03-30 21:51:53 +02:00
Calum Grant
c26d05b1d5 Merge pull request #5532 from RasmusWL/python-cleanup
Python: Delete filter queries, code duplication library, and precision tag from metric queries
2021-03-29 17:16:43 +01:00
Rasmus Wriedt Larsen
92e0e195a4 Revert "Merge pull request #5506 from tausbn/python-allow-absolute-imports-from-source-directory"
This reverts commit 8d15680af4, reversing
changes made to 63831cc62b.

This PR caused performance problems, so reverting now to clear up immediate
problems.
2021-03-27 18:08:20 +01:00
Rasmus Lerchedahl Petersen
1be2be843d Python: update test expectations 2021-03-26 13:08:23 +01:00
Rasmus Lerchedahl Petersen
c93e0c08fd Merge branch 'python-port-insecure-protocol' of github.com:yoff/codeql into python-port-insecure-protocol 2021-03-26 00:26:33 +01:00
yoff
54dad57cf4 Update python/ql/test/query-tests/Security/CWE-327/pyOpenSSL_fluent.py
Co-authored-by: Rasmus Wriedt Larsen <rasmuswriedtlarsen@gmail.com>
2021-03-26 00:25:40 +01:00
Rasmus Lerchedahl Petersen
2b257318f1 Python: more precise comment 2021-03-25 23:22:24 +01:00
yoff
208d5157fa Merge pull request #5500 from RasmusWL/django-forms
Python: Model RemoteFlowSources on Django forms/fields
2021-03-25 20:43:19 +01:00
Rasmus Wriedt Larsen
bd4934380a Python: Remove code duplication library 2021-03-25 15:27:55 +01:00
yoff
8d15680af4 Merge pull request #5506 from tausbn/python-allow-absolute-imports-from-source-directory
Python: Allow absolute imports in directories with scripts
2021-03-24 14:42:14 +01:00
Rasmus Wriedt Larsen
70974ea197 Python: Fix grammar in QLDoc
Co-authored-by: yoff <lerchedahl@gmail.com>
2021-03-24 14:06:06 +01:00
Taus Brock-Nannestad
8d30ee5c3c Python: Include unmarked Python file in snapshot
Sadly, it seems we're not interpreting this as Python code, even if we
explicitly ask to have it included.
2021-03-24 14:01:13 +01:00
Taus Brock-Nannestad
6d86239929 Python: Test all cases
Note that the test in `no_py_extension` isn't complete, since we're
not extracting the `main` file there.
2021-03-24 13:15:59 +01:00
Taus Brock-Nannestad
17d1768259 Python: Allow absolute imports in directories with scripts
Fixes the import logic to account for absolute imports.

We do this by classifying which files and folders may serve as the
entry point for execution, based on a few simple heuristics. If the
file `module.py` is in the same folder as a file `main.py` that may be
executed directly, then we allow `module` to be a valid name for
`module.py` so that `import module` will work as expected.
2021-03-23 18:32:17 +01:00