Commit Graph

56 Commits

Author SHA1 Message Date
Michael Nebel
2321ca59f6 Python: Update all test util paths to point to the new location. 2024-12-12 13:54:30 +01:00
yoff
81c8a702ff Merge pull request #18112 from github/tausbn/add-api-graph-support-for-parameter-annotations 2024-12-05 15:05:27 +01:00
Jeroen Ketema
c3ea883b11 Python: Update expected test results 2024-12-03 19:18:57 +01:00
Taus
2734377e5d Python: Add API graph support for parameter annotations
Adds API graph support for observing that in
```python
def foo(x : Bar): ...
```
The variable `x` is likely to be an instance of the type `Bar` inside
this function.
In particular, we add `getInstanceFromAnnotation` as a predicate on API
graph nodes that tracks this step (corresponding to a new edge type
labeled with "annotation" in the API graph), and extend the existing
`getAnInstance` predicate to also include instances arising from type
annotations.

A more complete solution would also add support for annotated
assignments (`x : Foo = ...` or just `x : Foo`) as well as track types
through type aliases (`type Foo = Bar`). This turns out to be
non-trivial, however, as these type constructs don't have any CFG nodes
(and so no data-flow nodes by default either). In order to not have
perfect be the enemy of good, this commit is only targeting the type
parameter case (which is also likely to be the most common use case
anyway).

The tests for API graphs have been extended accordingly, including tests
for the kinds of type ascriptions that we _don't_ currently model in API
graphs (marked with `MISSING:` in the inline tests).
2024-11-26 13:03:06 +00:00
Taus
1c68c987b0 Python: Change all remaining occurrences of StrConst
Done using
```
git grep StrConst | xargs sed -i 's/StrConst/StringLiteral/g'
```
2024-04-22 12:00:09 +00:00
Rasmus Lerchedahl Petersen
169d7a3c98 Python: Add scope entry definition nodes
otherwise we confuse captured variables
in the single scope entry cfg node. Now
we have one for each defined variable.
2023-12-20 12:09:00 +01:00
Rasmus Lerchedahl Petersen
3b7e29bed6 Python: add test for crosstalk 2023-12-20 12:08:05 +01:00
Rasmus Wriedt Larsen
df9fb141b8 Python: Remove old manual consistency query tests 2023-11-21 11:50:23 +01:00
Rasmus Wriedt Larsen
55f5b26ba6 Python: Accept new ordering of query predicates in .expected 2023-11-15 10:09:54 +01:00
Rasmus Wriedt Larsen
9f43108ba8 Python: Fix DataFlowCall.getEnclosingCallable
Now it is aligned with the implementation of DataFlow::Node

See 4bc4e0845d/python/ql/lib/semmle/python/dataflow/new/internal/DataFlowPublic.qll (L134-L138)
2023-11-07 11:29:23 +01:00
Rasmus Lerchedahl Petersen
58bf70d61b Python: filter self steps from use-use flow
Factor out use-use flow in order to do this.
Also improve names and comments.

I also wanted to change the types in `difinitionFlowStep`, but
that broke the module instantiation.
2023-11-02 09:31:28 +01:00
Rasmus Wriedt Larsen
72d0dcdaba Python: Workaround for module level items from import * not being LocalSourceNodes 2023-10-10 17:45:11 +02:00
Rasmus Wriedt Larsen
6521e5165c Python: Extend import * with plain use
(no calls or anything)
2023-10-10 17:45:11 +02:00
Tom Hvitved
d3558f8579 Python: Update expected test output 2023-09-12 21:18:31 +02:00
Jeroen Ketema
8f599faf85 Python: Rewrite inline expectation tests to use parameterized module 2023-06-09 10:42:29 +02:00
yoff
1a57f81aca Merge pull request #12537 from yoff/python/captured-variables-for-typetracking
Python: Captured variables for type tracking and the API graph
2023-05-09 12:34:22 +02:00
Mathias Vorreiter Pedersen
09ba9a74ce Merge pull request #12959 from MathiasVP/identity-consistency-check
DataFlow: Add an "identity-step" consistency check
2023-05-05 10:03:20 +01:00
yoff
42090b55fa Merge branch 'main' into python/captured-variables-for-typetracking 2023-05-04 13:52:23 +02:00
yoff
d1206ea620 Update python/ql/test/library-tests/ApiGraphs/py3/test_captured_inheritance.py
Co-authored-by: Rasmus Wriedt Larsen <rasmuswriedtlarsen@gmail.com>
2023-05-04 13:52:08 +02:00
Kasper Svendsen
d9f29a85d6 Python: Enable implicit this warnings 2023-05-04 10:16:52 +02:00
Mathias Vorreiter Pedersen
e650df810d Python: Accept consistency changes. 2023-05-03 20:33:00 +01:00
Rasmus Lerchedahl Petersen
6d9fd24f1b python: update comments 2023-05-03 18:10:15 +02:00
Rasmus Lerchedahl Petersen
b71306104e python: add test for inheritance 2023-04-26 13:50:12 +02:00
yoff
9e3d57d442 Update python/ql/test/library-tests/ApiGraphs/py3/test_captured_flask.py
Co-authored-by: Rasmus Wriedt Larsen <rasmuswriedtlarsen@gmail.com>
2023-04-11 14:34:40 +02:00
Rasmus Lerchedahl Petersen
2318752c14 python: add reads of captured variables to
type tracking and the API graph.

- In `TypeTrackerSpecific.qll` we add a jump step
  - to every scope entry definition
  - from the value of any defining `DefinitionNode`
    (In our example, the definition is the class name, `Users`,
     while the assigned value is the class definition, and it is
     the latter which receives flow in this case.)
- In `LocalSources.qll` we allow scope entry definitions as local sources.
  - This feels natural enough, as they are a local source for the value, they represent.
    It is perhaps a bit funne to see an Ssa variable here,
    rather than a control flow node.
 - This is necessary in order for type tracking to see the local flow
    from the scope entry definition.
- In `ApiGraphs.qll` we no longer restrict the result of `trackUseNode`
  to be an `ExprNode`. To keep the positive formulation, we do not
  prohibit module variable nodes. Instead we restrict to the new
  `LocalSourceNodeNotModule` which avoids those cases.
2023-03-16 12:55:58 +01:00
Rasmus Lerchedahl Petersen
7e003f63b9 python: add test for flask example
This is a condensed versio of the user reported example
found [here](eb377d5918/app.py (L278))
The `MISSING` annotation indicates where our API graph falls short.
2023-03-16 12:53:40 +01:00
Tom Hvitved
404ead8a18 Python: Update expected test output 2023-03-16 08:40:53 +01:00
Rasmus Lerchedahl Petersen
32d95834d1 python: add test documenting effect of scopes 2023-03-09 10:19:44 +01:00
Rasmus Wriedt Larsen
61151d4aa7 Merge branch 'main' into call-graph-code 2023-01-16 13:39:15 +01:00
Tony Torralba
d87c8c75d6 Python: Remove omittable exists variables 2023-01-10 13:37:35 +01:00
Tom Hvitved
39fea378b8 Python: Update expected test output 2022-12-13 09:53:01 +01:00
Rasmus Wriedt Larsen
a826c4f48b Merge branch 'main' into call-graph-code 2022-12-08 11:39:30 +01:00
Tom Hvitved
8f701cf1cb Python: Update expected test output 2022-12-05 14:33:06 +01:00
Rasmus Wriedt Larsen
eb600f07b7 Python: Use config for dataflow-consistency.ql
And ignore post-update nodes for `**kwargs` arguments
2022-11-22 14:46:30 +01:00
Rasmus Wriedt Larsen
88f703af1f DataFlow: Accept changes to .expected 2022-11-10 22:13:34 +01:00
Rasmus Lerchedahl Petersen
0b8e908823 Python: fix def nodes for subscript
We were using `getMember` for dictionaries, these are now getIndex
Also add convenience predicate for string keys
2022-10-12 20:13:48 +02:00
Tom Hvitved
f4b82cb2e8 Python: Update expected test output 2022-09-22 15:01:40 +02:00
Asger F
296aa52ef0 Python: Add API::EntryPoint
Python: add EntryPoint test
2022-09-03 13:24:46 +02:00
Rasmus Wriedt Larsen
f89b32183f Merge branch 'main' into typetracker-decorators 2022-08-08 11:52:09 +02:00
Asger F
b096f9ec72 Python: Rename getAUse -> getAValueReachableFromSource 2022-06-21 12:44:06 +02:00
Rasmus Wriedt Larsen
d6e68258a4 Python: API-graphs: allow class decorators in .getASubclass() 2022-06-15 17:30:34 +02:00
Rasmus Wriedt Larsen
5f32f898d5 Python: API-graphs: test class decorators and subclass
A class decorator could change the class definition in any way.

In this specific case, it would be better if we allowed the subclass to
be found with API graphs still.

inspired by
c2250cfb80/tests/auth_tests/test_views.py (L40-L46)
2022-06-15 16:16:34 +02:00
Rasmus Wriedt Larsen
f8253f5fef Python: Fully disallow API::moduleImport of module with dots
Inspired by discussion about this for MaD in
https://github.com/github/codeql/pull/8883#discussion_r865858084
2022-05-12 13:30:26 +02:00
Rasmus Wriedt Larsen
597a8414d9 Python: Add test of API::moduleImport with dots
This is currently semi-works -- the import is allowed, but doesn't
always work when used :|
2022-05-12 13:29:16 +02:00
Erik Krogh Kristensen
5e23da813f rename named-parameters to keyword-parameters 2022-02-03 23:10:39 +01:00
Erik Krogh Kristensen
3801a158a8 remove module exporst nodes from API graphs 2022-02-03 23:10:39 +01:00
Erik Krogh Kristensen
3be3da2eb6 add recursive API-graph test 2022-02-03 23:10:38 +01:00
Erik Krogh Kristensen
ef5818e243 support import * in ApiGraphs 2022-02-03 23:10:38 +01:00
Erik Krogh Kristensen
16774ba285 add support for named parameters in API graphs 2022-02-03 23:10:38 +01:00
Erik Krogh Kristensen
095c73f1fe redo the ApiGraph testing framework 2022-02-03 23:10:38 +01:00