Commit Graph

9108 Commits

Author SHA1 Message Date
amammad
5795c72a99 added inline tests 2023-12-07 14:04:33 +01:00
amammad
6ebdae3bab Merge branch 'main' into amammad-python-bombs 2023-12-07 13:50:20 +01:00
amammad
2d0067d618 fix some qldocs, change Sink extenstion model, deduct some not necessarily checks :) 2023-12-07 13:45:28 +01:00
github-actions[bot]
92af5f5386 Post-release preparation for codeql-cli-2.15.4 2023-12-06 22:59:22 +00:00
Rasmus Lerchedahl Petersen
263c0aade7 Python: adjust test expectations
mostly removing of nodes from the graph.
One result lost:
```
check("submodule.submodule_attr", submodule.submodule_attr, "submodule_attr", globals()) #$ MISSING:prints=submodule_attr
```
2023-12-06 23:00:51 +01:00
github-actions[bot]
c04457e9e7 Release preparation for version 2.15.4 2023-12-06 21:11:50 +00:00
Rasmus Lerchedahl Petersen
8c5ca3f564 Python: remove control flow nodes
for module entry definitions from the dataflow graph.
2023-12-06 21:47:03 +01:00
Rasmus Lerchedahl Petersen
9e1c818db6 Python: address review comments 2023-12-04 17:49:26 +01:00
yoff
f5c176bd12 Apply suggestions from code review
Co-authored-by: Taus <tausbn@github.com>
2023-12-04 17:41:00 +01:00
Rasmus Wriedt Larsen
c952f6a648 Python: Update rest of tests to new dataflow lib
I had missed these originally, since I had just fixed the ones that were
highlighted in the actions logs, thinking they had covered everything :(
2023-12-04 14:49:40 +01:00
Rasmus Lerchedahl Petersen
e091ae84ab Merge branch 'main' of https://github.com/github/codeql into python/remove-ssa-nodes-from-dataflow-graph 2023-12-04 14:05:40 +01:00
Rasmus Wriedt Larsen
2fed0adde7 Merge pull request #8457 from RasmusWL/add-dataflow-consistency-query
Python: Add dataflow consistency query
2023-12-04 12:50:46 +01:00
Rasmus Wriedt Larsen
4dd3ea3798 Python: Update tests to new dataflow lib
Avoids some deprecation warnings :)
2023-12-04 12:36:57 +01:00
Anders Schack-Mulligen
67f0529cda Dataflow: Sync. 2023-12-04 12:36:57 +01:00
Taus
4ef1fe49e3 Merge pull request #14918 from github/tausbn/python-support-tarslip-extraction-filters
Python: Add support for extraction filters
2023-11-30 22:55:09 +01:00
Taus
6e279183d9 Python: Remove unused unsafeFilter predicates 2023-11-28 13:54:17 +00:00
Taus
91643ad08f Python: Update hasUnsafeFilter to use API graph
This will probably break the tests in the short run. I'll fix the remaining issues in a follow-up commit.

Co-authored-by: Rasmus Wriedt Larsen <rasmuswl@github.com>
2023-11-28 14:48:26 +01:00
Rasmus Wriedt Larsen
2c10160ad4 Python: Highlight we actually want post-update nodes for *args and **kwargs arguments 2023-11-28 14:07:03 +01:00
Rasmus Wriedt Larsen
02f2031239 Python: Ensure other call for super().foo 2023-11-28 14:04:51 +01:00
Rasmus Wriedt Larsen
3c82653b63 Python: Highlight missing post-update flow for *args and **kwargs 2023-11-28 10:59:48 +01:00
Taus
ad1a86879e Python: Add change note 2023-11-27 14:39:32 +00:00
Taus
95e9284d08 Python: Add support for extraction filters
Adds support for extraction filters as defined in
https://peps.python.org/pep-0706/
and implemented in Python 3.12.

By my reading, setting the filter to `'data'` or `'tar'` is probably
safe, whereas `'fully_trusted'` or the default (which is the same as
`None`) is not.

For now, I have just added this modelling to the tarslip query. We could
also share it with the modelling of `shutil.unpack_archive` (which has also
gained a `filter` argument), but it was unclear to me where we should put
this modelling in that case. Perhaps the best solution would be to merge
the experimental `py/tarslip-extended` query into the existing query (in
which case the current location is perhaps not too bad).
2023-11-27 14:11:17 +00:00
Rasmus Wriedt Larsen
4e0cca9a41 Merge pull request #14353 from GeekMasher/py-restframework
Python: support `*args` and `**kwargs` in request handlers
2023-11-23 14:04:36 +01:00
Rasmus Wriedt Larsen
d056706af5 Merge pull request #14725 from RasmusWL/re-modeling
Python: Add taint-flow modeling for `re` module
2023-11-23 11:35:36 +01:00
Rasmus Wriedt Larsen
3d46129bbf Python: Remove intermediary steps from taint-test
These were leftovers from old way of propagating taint
2023-11-23 10:40:25 +01:00
Rasmus Wriedt Larsen
4a98ed903e Python: Fix consistency for bound-methods used in list-comp 2023-11-22 14:07:40 +01:00
Rasmus Wriedt Larsen
67b1414177 Python: Highlight even more cases for multipleArgumentCallExclude 2023-11-22 11:25:23 +01:00
Rasmus Wriedt Larsen
30891ca4aa Merge pull request #14861 from yoff/python/demonstrate-def-use-explosion
Python: test demonstrating the need for phi nodes
2023-11-22 09:57:10 +01:00
yoff
4785048076 Apply suggestions from code review
Co-authored-by: Rasmus Wriedt Larsen <rasmuswriedtlarsen@gmail.com>
2023-11-22 09:29:07 +01:00
Rasmus Lerchedahl Petersen
d288c4a709 Python: rename folder 2023-11-21 16:48:41 +01:00
yoff
4f7fde7b87 Merge pull request #14858 from yoff/python/demonstrate-use-use-explosion
Python: Test demonstrating the need for phi-read-nodes
2023-11-21 16:44:11 +01:00
Rasmus Wriedt Larsen
63fcaca82f Python: add change-note 2023-11-21 16:02:41 +01:00
Rasmus Wriedt Larsen
a0867b4f66 Python: More HTTP request handler *args/**kwargs modeling
I looked through all `override Parameter getARoutedParameter() {` in our
codebase, and we now modeling *args/**kwargs for all of them 👍
2023-11-21 16:02:40 +01:00
Rasmus Wriedt Larsen
f9d7becd04 Python: Make multipleArgumentCallExclude more specific 2023-11-21 15:57:12 +01:00
Rasmus Lerchedahl Petersen
c552bc5eb1 Python: fix test output 2023-11-21 15:48:22 +01:00
Rasmus Lerchedahl Petersen
077e51c6c6 Python: fix test output 2023-11-21 15:47:18 +01:00
Rasmus Lerchedahl Petersen
4857960f72 Python: test demonstrating the need for phi nodes
or a dataflow node playing that role, at least.
2023-11-21 15:40:05 +01:00
Rasmus Lerchedahl Petersen
f138fc0d2d Python: Test demonstrating need for phi-read-nodes
Or for a data flow node filling that role, at least.
2023-11-21 13:54:02 +01:00
Rasmus Wriedt Larsen
37d03ee0f3 Python: Accept .expected changes
Note that in this case, since there is a known `django.urls.path`
route-setup, we know that the request-handler will only be passed
keyword arguments, so it is not a mistake that `*args` is not considered
a routed-parameter here (although it certainly wouldn't have hurt us if
we did consider it a routed-parameter either).
2023-11-21 13:46:55 +01:00
Rasmus Wriedt Larsen
1bc8a6de61 Python: Fixup mistaken modelling 2023-11-21 13:46:23 +01:00
Rasmus Wriedt Larsen
36a846ee32 Python: Fix django regex path handling 2023-11-21 13:08:45 +01:00
Rasmus Wriedt Larsen
c51c15ae74 Python: Add test of routed parameters to *args
Also move the **kwargs and *args test to a more appropriate file
2023-11-21 13:01:01 +01:00
Rasmus Wriedt Larsen
2ec1822e9c Python: Accept consistency-errors in django-orm 2023-11-21 12:44:42 +01:00
Rasmus Wriedt Larsen
5f26790b90 Merge branch 'main' into py-restframework 2023-11-21 11:57:48 +01:00
Rasmus Wriedt Larsen
df9fb141b8 Python: Remove old manual consistency query tests 2023-11-21 11:50:23 +01:00
Rasmus Wriedt Larsen
b6df6b7c99 Python: Add dataflow consistency query 2023-11-21 11:33:28 +01:00
Rasmus Wriedt Larsen
71c017f053 Python: Apply suggestions from code review
Co-authored-by: yoff <lerchedahl@gmail.com>
2023-11-21 10:07:42 +01:00
Rasmus Lerchedahl Petersen
c8b87f71c5 Python: add change note 2023-11-20 21:44:16 +01:00
Rasmus Lerchedahl Petersen
421d4f3497 Python: filter more sinks in stdlib
Rename variable to reflect larger scope

We had test results inside `os.py`, I suppose we have found a little extra flow.
2023-11-20 21:35:52 +01:00
Rasmus Lerchedahl Petersen
11c71fdd18 Python: remove EssaNodes
This commit removes SSA nodes from the data flow graph. Specifically, for a definition and use such as
```python
  x = expr
  y = x + 2
```
we used to have flow from `expr` to an SSA variable representing x and from that SSA variable to the use of `x` in the definition of `y`. Now we instead have flow from `expr` to the control flow node for `x` at line 1 and from there to the control flow node for `x` at line 2.

Specific changes:
- `EssaNode` from the data flow layer no longer exists.
- Several glue steps between `EssaNode`s and `CfgNode`s have been deleted.
- Entry nodes are now admitted as `CfgNodes` in the data flow layer (they were filtered out before).
- Entry nodes now have a new `toString` taking into account that the module name may be ambigous.
- Some tests have been rewritten to accomodate the changes, but only `python/ql/test/experimental/dataflow/basic/maximalFlowsConfig.qll` should have semantic changes.
- Comments have been updated
- Test output has been updated, but apart from `python/ql/test/experimental/dataflow/basic/maximalFlows.expected` only `python/ql/test/experimental/dataflow/typetracking-summaries/summaries.py` should have a semantic change. This is a bonus fix, probably meaning that something was never connected up correctly.
2023-11-20 21:35:32 +01:00