Commit Graph

269 Commits

Author SHA1 Message Date
Rasmus Lerchedahl Petersen
58bf70d61b Python: filter self steps from use-use flow
Factor out use-use flow in order to do this.
Also improve names and comments.

I also wanted to change the types in `difinitionFlowStep`, but
that broke the module instantiation.
2023-11-02 09:31:28 +01:00
Rasmus Lerchedahl Petersen
613831b2e1 Python: add test for post-update loop flow 2023-11-02 09:31:28 +01:00
Taus
8e1bb4b364 Python: Accept moved consistency test results
Co-authored-by: Rasmus Lerchedahl Petersen <yoff@github.com>
2023-10-10 09:22:36 +00:00
Taus
e8ac258994 Python: Add missing flow for AssignmentExpr nodes
Also extend the tests surrounding this construct to be a bit more comprehensive.

Co-authored-by: Rasmus Lerchedahl Petersen <yoff@github.com>
2023-10-09 14:16:03 +00:00
Tom Hvitved
d3558f8579 Python: Update expected test output 2023-09-12 21:18:31 +02:00
Jeroen Ketema
abe06e5b95 Python: Update remaining inline expectation tests to use the paramterized module 2023-07-03 10:22:35 +02:00
Jeroen Ketema
dba4460526 Python: Update more inline expectation tests to use the paramterized module 2023-06-20 10:16:15 +02:00
Rasmus Lerchedahl Petersen
4b4b9bf9da python: add missing summaries
For append/add:
The new results in the experimental tar slip query
show that we do not recognize the sanitisers.
2023-06-13 20:22:21 +02:00
Rasmus Wriedt Larsen
2b7fc94aef Python: Fix validTest.py expectation 2023-06-13 12:11:28 +02:00
Rasmus Lerchedahl Petersen
b709ed47e1 python: add test 2023-06-13 11:20:15 +02:00
Rasmus Lerchedahl Petersen
9cb83fcdc9 python: add summaries for
copy, pop, get, getitem, setdefault

Also add read steps to taint tracking.

Reading from a tainted collection can be done in two situations:
1. There is an acces path
    In this case a read step (possibly from a flow summary)
    gives rise to a taint step.
2. There is no access path
    In this case an explicit taint step (possibly via a flow
    summary) should exist.
2023-05-26 14:04:15 +02:00
Rasmus Lerchedahl Petersen
144df9a39e python: remove explicit dataflow steps 2023-05-26 13:24:22 +02:00
Rasmus Lerchedahl Petersen
5b4f98d6c4 python: Add summaries for container constructors
Also:
- turn on flow summaries for taint
- do not restrict node type
  (as now we need summary nodes)
2023-05-16 14:38:51 +02:00
Rasmus Lerchedahl Petersen
81adf5aad4 python: remember to adjust annotation 2023-05-12 14:28:41 +02:00
Rasmus Lerchedahl Petersen
1b848bb510 python: fix tests 2023-05-12 13:51:50 +02:00
yoff
62b60f490c Apply suggestions from code review
Co-authored-by: Rasmus Wriedt Larsen <rasmuswriedtlarsen@gmail.com>
2023-05-12 12:54:17 +02:00
Rasmus Lerchedahl Petersen
0a9515dbcd python: add tests for built-in collections
- constructors: list, tuple, set, dict
- methods:
  - general: copy, pop
  - list: append
  - set: add
  - dict: keys, values, items, get, popitem
- functions: sorted, reversed, iter, next
2023-05-10 18:10:05 +02:00
yoff
1a57f81aca Merge pull request #12537 from yoff/python/captured-variables-for-typetracking
Python: Captured variables for type tracking and the API graph
2023-05-09 12:34:22 +02:00
Mathias Vorreiter Pedersen
e650df810d Python: Accept consistency changes. 2023-05-03 20:33:00 +01:00
Rasmus Lerchedahl Petersen
a25c7f7549 Merge branch 'main' of https://github.com/github/codeql into python/captured-variables-for-typetracking 2023-04-24 11:50:32 +02:00
Rasmus Wriedt Larsen
2ee09cc5d1 Merge branch 'main' into import-refined 2023-03-20 15:42:01 +01:00
Rasmus Wriedt Larsen
93c9f59e86 Python: Extract version specific coverage/classes.py tests
Since we can analyze operator.py from Python3, but not in Python 2
(since it's implemented in C), we get a difference for the index tests.

note: `operator.length_hint` is only available in Python 3.4 and later,
so would always fail under Python 2.
2023-03-20 15:39:20 +01:00
Rasmus Lerchedahl Petersen
4713ba1e12 python: more results no longer missing
Adjusted `tracked.ql`
- no need to annotate results on line 0
  this could happen for global SSA variables
- no need to annotate scope entry definitons
  they look a bit weird, as the annotation goes on the
  line of the function definition.
2023-03-16 12:55:58 +01:00
Tom Hvitved
404ead8a18 Python: Update expected test output 2023-03-16 08:40:53 +01:00
Rasmus Wriedt Larsen
dda29e99b2 Python: Add test of keyword argument with same name as positional-only parameter
This is a bit of an edge case, but allowed. Since we currently don't
provide information on positional only arguments, we can't do much to
solve it right now.
2023-03-07 13:28:48 +01:00
Rasmus Wriedt Larsen
2cc8fbaa50 Python: Accept changes due to better import resolution of operator.py 2023-03-06 14:48:48 +01:00
Rasmus Wriedt Larsen
9e2eb56032 Python: Remove support for late *args arguments
I found this to cause bad performance, so the implementation of this has
to be thought out more carefully.
2023-02-15 09:42:11 +01:00
Rasmus Wriedt Larsen
cef933f813 Python: Add comment explaining SINK3_F(kwargs["c"]) test
Co-authored-by: yoff <yoff@github.com>
2023-01-27 15:48:59 +01:00
Rasmus Wriedt Larsen
02b3a1b515 Python: At most one **kwargs ParameterNode per callable
Similar to the Ruby changes from
https://github.com/github/codeql/pull/11461

I feel the change to `DataFlowFunciton.getParameter` where we use
`not exists(func.getArgByName(_))` is not very great, but I was not allowed
to use `not exists(this.getParameter(any(ParameterPosition _).isKeyword(_)))`
because of negative recursion.
2023-01-27 11:14:42 +01:00
Rasmus Wriedt Larsen
63b2bd0871 Python: Fixup test_only_starargs addition
validTest.py did not pass, since we use `SINK3_F`.

I initially tried swapping the order

```
args = (arg1, arg2) # $ arg1 arg2 func=starargs_only
more_args = (arg4, arg3)
starargs_only(*args, *more_args)
```

But then asked myself, what is it _actually_ we're testing here? and it
seems to be the way we handle multiple *args arguments in the same call,
so I converted the test to be that instead! (and it matches what we do
in test_stararg_mixed)
2023-01-25 09:37:07 +01:00
Rasmus Wriedt Larsen
d9fbe58ad5 Python: Expand starargs_only test 2023-01-20 16:34:59 +01:00
Rasmus Wriedt Larsen
dad6221b61 Python: Accept dataflow-consistency.expected changes for now
As highlighted in the configuration file, there are some things to catch
up on, and we also need to apply the same fix as Ruby for **kwargs
handling.
2023-01-17 13:58:40 +01:00
Rasmus Wriedt Larsen
61151d4aa7 Merge branch 'main' into call-graph-code 2023-01-16 13:39:15 +01:00
yoff
5f0cde5be7 Merge branch 'main' into python/support-grouped-exceptions 2022-12-19 13:38:25 +01:00
Rasmus Lerchedahl Petersen
8e8d36f35e python: this also works in 3.11 2022-12-15 12:54:14 +01:00
Tom Hvitved
39fea378b8 Python: Update expected test output 2022-12-13 09:53:01 +01:00
Rasmus Wriedt Larsen
a826c4f48b Merge branch 'main' into call-graph-code 2022-12-08 11:39:30 +01:00
Tom Hvitved
8f701cf1cb Python: Update expected test output 2022-12-05 14:33:06 +01:00
Rasmus Wriedt Larsen
c0ad870949 Python: Exclude synthetic generator functions from DataFlowCallable 2022-11-22 14:46:33 +01:00
Rasmus Wriedt Larsen
5e5bab5a7c Python: Don't pass synthetic class instance to __new__ on class calls 2022-11-22 14:46:31 +01:00
Rasmus Wriedt Larsen
98a849405f Python: Add support for late *args arguments 2022-11-22 14:46:30 +01:00
Rasmus Wriedt Larsen
035d083515 Python: Support flow to *args param from positional arg 2022-11-22 14:46:30 +01:00
Rasmus Wriedt Larsen
db921ac036 Python: Add basic support for *args 2022-11-22 14:46:30 +01:00
Rasmus Wriedt Larsen
b6314dd19d Python: Add *args tests 2022-11-22 14:46:30 +01:00
Rasmus Wriedt Larsen
c687df4ddc Python: Support flow to keyword param from **kwargs arg
When resolving merge conflict after flow-summaries was merged, this is
the original commit where I introduced ParameterNodeImpl, so this is the
commit where differences in that implementation was committed...

I removed TParameterNode, since I could not see we we gain anything from
having it.
2022-11-22 14:46:30 +01:00
Rasmus Wriedt Larsen
215a03d948 Python: Support flow to **kwargs param from keyword arg 2022-11-22 14:46:30 +01:00
Rasmus Wriedt Larsen
503ad544e9 Python: Remove impossible flow for **kwargs params 2022-11-22 14:46:30 +01:00
Rasmus Wriedt Larsen
eb600f07b7 Python: Use config for dataflow-consistency.ql
And ignore post-update nodes for `**kwargs` arguments
2022-11-22 14:46:30 +01:00
Rasmus Wriedt Larsen
5722d231bd Python: Add basic support for **kwargs
For now this is JUST from `**kwargs` in arguments, to `**kwargs`
parameters, and this part is based on field-flow

Note that dataflow-library complains about missing post update nodes for
these. This needs to be ignored, since post update nodes for `**kwargs`
arguments doesn't make sense, it's not possible to alter the dictionary
inside the method.
2022-11-22 14:46:30 +01:00
Rasmus Wriedt Larsen
9b2663034d Python: Change test .ql to also show bad argument flow 2022-11-22 14:46:30 +01:00