Commit Graph

4040 Commits

Author SHA1 Message Date
Sid Shankar
fb660b8f05 Py: Report any extracted file as successfully extracted 2024-01-08 22:20:51 +00:00
Rasmus Wriedt Larsen
95c24275f2 Merge pull request #15044 from RasmusWL/automated-subclass-models
Python: Automated subclass models
2024-01-05 10:43:48 +01:00
Rasmus Lerchedahl Petersen
da4aef80e9 Revert "Python: make it a real consistency check"
This reverts commit 45411f4a93.
2023-12-20 16:15:17 +01:00
Rasmus Lerchedahl Petersen
d039ceb689 Python: add test for fields 2023-12-20 15:23:45 +01:00
Rasmus Lerchedahl Petersen
706e9dc896 Python: fix compilation 2023-12-20 15:23:33 +01:00
Rasmus Lerchedahl Petersen
45411f4a93 Python: make it a real consistency check 2023-12-20 14:53:37 +01:00
Rasmus Lerchedahl Petersen
f8417b0dd8 Merge branch 'main' of https://github.com/github/codeql into python/captured-variables-basic 2023-12-20 13:16:42 +01:00
Rasmus Lerchedahl Petersen
169d7a3c98 Python: Add scope entry definition nodes
otherwise we confuse captured variables
in the single scope entry cfg node. Now
we have one for each defined variable.
2023-12-20 12:09:00 +01:00
Rasmus Lerchedahl Petersen
3b7e29bed6 Python: add test for crosstalk 2023-12-20 12:08:05 +01:00
Rasmus Wriedt Larsen
72687e0368 Merge branch 'main' into automated-subclass-models 2023-12-19 17:08:25 +01:00
Rasmus Wriedt Larsen
3e878f5a0b Python: Model django response subclass relationship 2023-12-19 17:07:02 +01:00
Rasmus Wriedt Larsen
abe6f1639a Python: Add example of models subclassing problem
In reality, we only want to model this as a `rest_framework.response.Response`, since our .qll modeling is more precise for rest-framework responses than if we also modeled it as a basic django http response. (specifically, that default mime-type handling is way different).
2023-12-19 17:07:02 +01:00
Rasmus Wriedt Larsen
933938d926 Python: Make rest_framework tests runnable again 2023-12-19 17:07:01 +01:00
Rasmus Wriedt Larsen
0fe29b6a86 Python: Recover subclass finder .expected after cherry picking commits from https://github.com/github/codeql/pull/15030 2023-12-19 17:07:01 +01:00
Rasmus Lerchedahl Petersen
75f9eeb4e9 Python: adjust test expectations
mostly removing of nodes from the graph.
One result lost:
```
check("submodule.submodule_attr", submodule.submodule_attr, "submodule_attr", globals()) #$ MISSING:prints=submodule_attr
```
2023-12-19 17:07:01 +01:00
Rasmus Wriedt Larsen
e050f2e998 Python: Adjust subclass finder to no ESSA nodes
But the new test results looks very strange indeed!
2023-12-19 17:07:01 +01:00
Rasmus Wriedt Larsen
2305d55967 Merge pull request #15101 from yoff/python/update-InlineTaintTestPaths-to-new-api
Python: update to new API update is in a comment, so compilation never failed in CI.
2023-12-19 11:10:55 +01:00
yoff
a60c52b8b7 Merge branch 'main' into python/captured-variables-basic 2023-12-18 23:44:46 +01:00
Rasmus Lerchedahl Petersen
78c484faab Python: remove support for capturing callbacks
This will be added in a follow-up PR instead.
2023-12-18 23:24:57 +01:00
Rasmus Lerchedahl Petersen
64655a0ea8 Python: Use enw class name 2023-12-16 01:36:46 +01:00
Rasmus Lerchedahl Petersen
1ee11ae7af Merge branch 'main' of https://github.com/github/codeql into python/captured-variables-basic 2023-12-15 14:31:57 +01:00
Rasmus Lerchedahl Petersen
abd544d96c Python: consistency failure gone 2023-12-15 00:38:58 +01:00
Tom Hvitved
3b1146bf98 Python: Adopt shared type tracking library 2023-12-14 13:22:44 +01:00
Tom Hvitved
c8b4a215bc Merge pull request #14573 from hvitved/flow-summary-impl-param
Move `FlowSummaryImpl.qll` to `dataflow` pack
2023-12-14 12:24:15 +01:00
Rasmus Lerchedahl Petersen
0b6d47b8bc Python: update to new API
update is in a comment, so compilation
never failed in CI.
2023-12-14 11:56:05 +01:00
Rasmus Lerchedahl Petersen
2a5736e73d Python: add consistency exception
this must have been lost in my
clean-up rebase.
2023-12-14 11:50:09 +01:00
Rasmus Lerchedahl Petersen
479d81fb75 Python: fix nonlocal captured variables
This depends on the extractor fix
2023-12-14 10:37:27 +01:00
Rasmus Lerchedahl Petersen
17a0029585 Python: support callbacks to library calls
TODO:
The member predicate `LibraryLambdaMethod::getACall` is
currently too permissive.
Ideally, we would have `libraryCallHasLambdaArg`
as in Ruby. But even a more precise
`libraryCall` predicate might be fine.
2023-12-14 10:27:15 +01:00
Rasmus Lerchedahl Petersen
7565873e83 Python: test callbacks to library calls 2023-12-14 10:27:15 +01:00
Rasmus Lerchedahl Petersen
797deebcdd Python: exclude CaptureNodes 2023-12-14 10:27:15 +01:00
Rasmus Lerchedahl Petersen
c054ba6a97 python: instantiate module for variable capture
This provides variable capture in standard situations:
- nested functions
- lambdas
There are some deficiencies:
- we do not yet handle objects capturing variables.
- we do not handle variables captured via the `nonlocal` keyword.
  This should be solved at the AST level, though, and then it
  should "just work".

There are still inconsistencies in the case where
a `SynthesizedCaptureNode` has a comprehensions
as its enclosing callable. In this case,
`TFunction(cn.getEnclosingCallable())` is not
defined and so getEnclosingCallable does not exist
for the `CaptureNode`.
2023-12-14 10:25:39 +01:00
Rasmus Lerchedahl Petersen
6db55cd12f Python: add missing annotation 2023-12-14 10:20:49 +01:00
Rasmus Wriedt Larsen
419130be21 Merge pull request #15030 from yoff/python/remove-module-entry-definitions
Python: Remove control flow nodes for module entry definitions from the dataflow graph.
2023-12-11 11:40:17 +01:00
Tom Hvitved
faaa558ed9 Python: Use FlowSummaryImpl from dataflow pack 2023-12-10 11:25:44 +01:00
Anders Schack-Mulligen
64eb4ff753 Merge pull request #14983 from aschackmull/dataflow/deprecate-old-api
Data Flow: Deprecate old data flow api.
2023-12-08 14:27:25 +01:00
Rasmus Wriedt Larsen
6ef9a2b11e Python: Fix problem if import is used
I fixed it in both predicates... I think we might still be able to remove
`newDirectAlias` -- but with it being better, it will allow us to better test if `newImportAlias` actually cover everything we need!
2023-12-08 11:27:52 +01:00
Rasmus Wriedt Larsen
fcdc8102e2 Python: Add test highlight problem is import is used :O 2023-12-08 11:27:52 +01:00
Rasmus Wriedt Larsen
f1fd9b4c7a Python: Fix underlying problem of not using Alias 2023-12-08 11:27:52 +01:00
Rasmus Wriedt Larsen
a956e1f613 Python: Use django View instead of MethodView
Due to the 'only model most specific spec' logic highlighted in previous
commit, I'm changing away from MethodView/View, and use Django view instead.

In practice this shouldn't matter at all, but for writing tests it would
have been a nice fix to only have the "same name but more specific"
logic apply when it's the same _definition_ location. We used to have
this information available, but right now we don't... so instead of
spending a lot of time rewriting the core library, I simply used a
different class :D :O :(
2023-12-08 11:27:52 +01:00
Rasmus Wriedt Larsen
03aa2e27df Python: Explain the funky logic in Find.ql 2023-12-08 11:27:52 +01:00
Rasmus Wriedt Larsen
1f8f6dd0ec Python: Ensure no deps visible in FindSubclass tests 2023-12-08 11:27:52 +01:00
Rasmus Wriedt Larsen
af2d783b38 Python: More examples of things to handle in find-subclass 2023-12-08 11:27:52 +01:00
Rasmus Wriedt Larsen
f19b672656 Python: Also capture alias with new name 2023-12-08 11:27:50 +01:00
Rasmus Wriedt Larsen
e7d55736b0 Python: Add test of find-subclass code 2023-12-08 11:27:50 +01:00
amammad
5795c72a99 added inline tests 2023-12-07 14:04:33 +01:00
amammad
6ebdae3bab Merge branch 'main' into amammad-python-bombs 2023-12-07 13:50:20 +01:00
Rasmus Lerchedahl Petersen
263c0aade7 Python: adjust test expectations
mostly removing of nodes from the graph.
One result lost:
```
check("submodule.submodule_attr", submodule.submodule_attr, "submodule_attr", globals()) #$ MISSING:prints=submodule_attr
```
2023-12-06 23:00:51 +01:00
Rasmus Lerchedahl Petersen
9e1c818db6 Python: address review comments 2023-12-04 17:49:26 +01:00
yoff
f5c176bd12 Apply suggestions from code review
Co-authored-by: Taus <tausbn@github.com>
2023-12-04 17:41:00 +01:00
Rasmus Wriedt Larsen
c952f6a648 Python: Update rest of tests to new dataflow lib
I had missed these originally, since I had just fixed the ones that were
highlighted in the actions logs, thinking they had covered everything :(
2023-12-04 14:49:40 +01:00