Commit Graph

9615 Commits

Author SHA1 Message Date
Anders Schack-Mulligen
4153a83a4f Python: Add workaround. 2024-10-16 16:14:51 +02:00
Anders Schack-Mulligen
5950c336e2 Python: Refactor references to NormalCall. 2024-10-16 16:04:31 +02:00
Rasmus Lerchedahl Petersen
22d621c625 shared: add locations to typetracking nodes 2024-10-16 15:16:18 +02:00
Anders Schack-Mulligen
c20f12fa6c Add qldoc. 2024-10-16 14:35:23 +02:00
Anders Schack-Mulligen
7497d9530d Python: Add tentative support for speculative taint flow. 2024-10-16 14:35:20 +02:00
Anders Schack-Mulligen
c80627a3d3 Dataflow: add plumbing for adding provenance to state-steps. 2024-10-16 14:35:18 +02:00
Taus
65dbc1de91 Python: Add copy.replace test to list of runnable tests 2024-10-15 18:17:00 +02:00
Taus
28f8874243 Merge pull request #17688 from github/tausbn/python-3.13-default-type-parser-support
Python: Add support for type parameter defaults
2024-10-15 18:01:51 +02:00
Taus
d4e0cb2ffa Merge pull request #17767 from github/tausbn/python-3.13-model-flow-in-replace
Python: Model `copy.replace`
2024-10-15 18:01:28 +02:00
yoff
9ed8fe5dd0 Update python/ql/test/library-tests/dataflow/coverage/functional.py
Co-authored-by: Taus <tausbn@github.com>
2024-10-15 17:35:36 +02:00
github-actions[bot]
079ab77a38 Post-release preparation for codeql-cli-2.19.2 2024-10-15 12:16:59 +00:00
Taus
3b60d8302b Python: Add change note 2024-10-15 12:14:20 +00:00
Taus
778b96aa39 Python: Update test expectations 2024-10-15 12:14:19 +00:00
Taus
eaef783f4b Python: Add partial model for copy.replace
Extends our modelling to partially cover the behaviour of
`copy.replace`. In particular, we model this in two ways:

Firstly, we extend the existing Models-as-Data row for `copy` and
`deepcopy` to also cover `replace`. This means that we treat the result
of `replace` as containing all of the fields of the original object.
This is somewhat _more_ than we want, as strictly speaking the fields
that are overwritten should _not_ propagate flow through the `replace`
call, but currently we don't have a good way of modelling this blocking
of flow.

Secondly, we add a flow summary that adds flow from named arguments of
the `replace` call to the corresponding fields on the base object. This
ensures that we at least have the new flow arising from the `replace`
call.

Note that the flow summary adds this flow for _all_ named arguments of
_all_ `replace` calls throughout the codebase. However, since any
particular `replace` call will only populate a subset of these (the
subset consisting of exactly those named arguments that are in that
particular call), this does not cause any unwanted crosstalk between
different `replace` calls.§
2024-10-15 12:14:19 +00:00
Taus
6f2cfa0ba8 Python: Update CopySummary to use getMaDRepresentation
Demonstrates the somewhat more ergonomic way to use
`getMaDRepresentation` when specifying summaries.

Note that this slightly extends the previous definition, in that
`DictionaryContentAny` is now _also_ propagated by a call to the
`.copy()` method, but I think this is correct.
2024-10-15 11:52:37 +00:00
Taus
ce914019c5 Python: Add getMaDRepresentation()
This adds a convenient way of getting the Models-as-Data representation
of a particular type of content. This avoids repeating the same
construction over and over in our various summaries. Currently this is
defined for all types of content except the captured variable content,
which to my knowledge doesn't have any representation in Models-as-Data.
2024-10-15 11:50:38 +00:00
Taus
e16405c675 Python: Add test for copy.replace
This test demonstrates the current state of affairs: that `copy.replace`
essentially blocks all flow of taint through it, because it has not been
modelled yet.
2024-10-15 11:48:43 +00:00
Taus
417e60a466 Python: Update extractor version 2024-10-15 11:22:54 +00:00
Taus
d905010aa8 Python: Add change note 2024-10-15 11:22:34 +00:00
Taus
8630f57710 Python: Add up-/downgrade scripts
Adds up- and downgrade scripts for the support for type parameter
defaults.

In the upgrade direction we do nothing, matching the behaviour of
`getDefault` not having a result for old databases.

In the downgrade direction, we explicitly remove the relevant child (via
the `py_exprs` database relation) for `TypeVar`, `TypeVarTuple`, and
`ParamSpec` parameters.
2024-10-15 11:22:33 +00:00
Taus
819b3d77ab Python: Update test expectations
Note that this still includes the somewhat puzzling parsing of
`Spam[**P2]` as an exponentiation with an empty left hand side. When we
fix that bug, we should also update this test to contain actually valid
syntax.
2024-10-15 11:22:33 +00:00
Taus
182a192cc0 Python: Regenerate dbscheme/AST 2024-10-15 11:22:33 +00:00
Taus
36d89745f9 Python: Fix dbscheme/AST autogeneration
There was an errant `ql` in the relevant paths, a leftover from the move
from the internal repo. Also, we can no longer rely on an intree version
of the CodeQL CLI, so from now on we'll just assume it's present in the
path. (On Codespaces, `gh codeql` is a decent replacement, especially if
using the `install-stub` functionality.
2024-10-15 11:22:32 +00:00
Taus
2af0d78435 Python: Add default field to the relevant AST nodes 2024-10-15 11:22:32 +00:00
Taus
55ee3eb36b Python: Add TSG support for type defaults 2024-10-15 11:22:31 +00:00
Taus
6545bfffa7 Python: Regenerate parser files
Two new files -- alloc.h and array.h -- suddenly appeared. Presumably
they are used by the somewhat newer version of tree-sitter. To be safe,
I included them in this commit.
2024-10-15 11:22:31 +00:00
Taus
882249ef82 Python: Add grammar support for type defaults
Also fixes an oversight in the grammar: starred expressions should be
allowed inside the subscript of an `Index` expression.
2024-10-15 11:22:30 +00:00
Taus
1ced5b44d7 Python: Add test for type parameter defaults 2024-10-15 11:22:30 +00:00
Arthur Baars
9ba8045837 Fix typos in changelog entries 2024-10-15 11:06:20 +00:00
Rasmus Lerchedahl Petersen
195b70aca6 python: Add test for functional-like programming
This can also serve for a place to add tests for
constructs like threading.Thread, mulitprocess.Process, concurrent.futures.ThreadPoolExecutor, and concurrent.futures.ProcessPoolExecutor.
2024-10-15 12:54:30 +02:00
github-actions[bot]
255f55cf1a Release preparation for version 2.19.2 2024-10-15 10:29:25 +00:00
yoff
da5e9ac18c python: more adjustments... 2024-10-14 14:54:33 +00:00
yoff
9d8d7ab237 python: update extractor expectations 2024-10-14 14:14:40 +00:00
Rasmus Lerchedahl Petersen
3402a729d0 Python: adjust test expectations for extractor test 2024-10-14 12:36:56 +02:00
yoff
8f681154ce Merge pull request #17683 from yoff/python/flip-default-for-stdlib-extraction
Python: Flip default for stdlib extraction
2024-10-11 18:34:44 +02:00
Rasmus Lerchedahl Petersen
5650694313 Python: update tests 2024-10-11 15:36:44 +02:00
Rasmus Lerchedahl Petersen
e2eb08b543 Python: improve messaging 2024-10-11 15:36:44 +02:00
Rasmus Lerchedahl Petersen
22588c9f85 Python: update ectractor version 2024-10-11 15:36:44 +02:00
Rasmus Lerchedahl Petersen
c013d4a59c Python: update test expectations 2024-10-11 15:36:44 +02:00
Rasmus Lerchedahl Petersen
4a291147e0 Python: only look for the py2 stdlib if we extract std lib 2024-10-11 15:36:44 +02:00
Rasmus Lerchedahl Petersen
bb78c2a67e Python: update test expectations 2024-10-11 15:36:44 +02:00
Rasmus Lerchedahl Petersen
e91efaa92e python: do not extract stdlib by default 2024-10-11 15:36:44 +02:00
Rasmus Lerchedahl Petersen
6bd46148e7 Python: add change note 2024-10-09 16:27:52 +02:00
Rasmus Lerchedahl Petersen
0ac4a10345 Python: model that finditer returns iterable of re.Match objects 2024-10-09 12:42:38 +02:00
Rasmus Lerchedahl Petersen
494b8bd7e1 python: model string property of resultof finditer 2024-10-09 12:40:47 +02:00
Rasmus Lerchedahl Petersen
073189ed6b python: add test for re.Match objects returned from finditer 2024-10-09 12:32:51 +02:00
yoff
6ffdf576d0 Merge pull request #17708 from yoff/python/ignore-extractor-test-output
Python: ignore some extractor test output
2024-10-09 12:31:42 +02:00
yoff
0b0e8a4bf5 Update python/extractor/tests/parser/.gitignore
As suggested by @tausbn
2024-10-09 12:22:17 +02:00
Rasmus Lerchedahl Petersen
ad630bc6ff Python: ignore some extractor test output
If you test the extractor locally, you want to ignore these files.
2024-10-09 11:34:58 +02:00
yoff
1f1b1b7aab Merge pull request #17653 from yoff/python/typetracking-through-comprehensions 2024-10-08 19:39:21 +02:00