Commit Graph

9380 Commits

Author SHA1 Message Date
yoff
930f1b50b9 Merge pull request #15397 from github/tausbn/python-fix-deepcopy-mutable-default-fp
Python: Fix `deepcopy` mutable default FP
2024-01-25 10:32:58 +01:00
Taus
96b1b8e402 Python: Remove empty lines from test file 2024-01-24 12:31:23 +00:00
github-actions[bot]
d0b74c00fe Post-release preparation for codeql-cli-2.16.1 2024-01-23 23:02:29 +00:00
github-actions[bot]
7ef611e6dc Release preparation for version 2.16.1 2024-01-23 19:45:16 +00:00
Erik Krogh Kristensen
f1d6f56621 Merge pull request #15393 from erik-krogh/deps-jan-2024
All: delete outdated deprecations
2024-01-23 13:52:38 +01:00
erik-krogh
865df920f9 add change-notes 2024-01-22 19:30:57 +01:00
Rasmus Wriedt Larsen
00dc55d825 Python: Add change-note 2024-01-22 17:32:33 +01:00
Rasmus Wriedt Larsen
cbed6e861d Python: Add html.escape as HTML sanitizer 2024-01-22 17:32:28 +01:00
Taus
d6d59377d3 Python: Fix flow through deepcopy
Or, more generally, any copy step, as these presumably do not preserve
object identity.

(Arguably, `copy` could still be susceptible to interior mutability, but
I think that's outside the scope of this query anyway.)
2024-01-22 15:40:30 +00:00
Taus
14c958ac4d Python: Remove mutable default sources from inside stdlib 2024-01-22 15:23:52 +00:00
Taus
411c107660 Python: Add tests for deepcopy FPs
There are two issues with `deepcopy` here. Firstly, the `deepcopy` function itself
has a mutable default value in its parameter `_nil` (set to the empty list by default).

Now, this value is never actually returned from `deepcopy`, as it is only used as a
sentinel, but our analysis is not clever enough to see this. Thus, it thinks that this
mutable default is returned, and hence the result of any call to `deepcopy` is a
potential source.

To remedy this, I opted to simply exclude all sources that originate from within the
standard library. It is very unlikely for any of the sources in the standard library
to be legit.

Secondly, `deepcopy` -- by virtue of being a function that we model as preserving
values -- admits data-flow through its calls, but this is not correct for the mutable
default query, as it is here the _identity_ of the default value in question that is
important. Thus, we get spurious flow through `deepcopy` for this specific query.
2024-01-22 15:21:57 +00:00
Taus
4742481070 Python: Consolidate "mutable default" tests
Moves the existing tests into the `ModificationOfParameterWithDefault` subdirectory
which already contained a bunch more tests. In the process, I also removed some
duplicated test cases.
2024-01-22 13:50:33 +00:00
Max Schaefer
a4639c7ff9 Update qhelp to mention solution using urlparse. 2024-01-22 13:36:12 +00:00
Max Schaefer
17e3a45ad7 Apply suggestions from code review
Co-authored-by: Taus <tausbn@github.com>
2024-01-22 13:36:12 +00:00
Max Schaefer
98178458d0 Python: Add support for more URL redirect sanitisers.
Since some sanitisers don't handle backslashes correctly, I updated the data-flow configuration to incorporate a flow state tracking whether or not backslashes have been eliminated or converted to forward slashes.
2024-01-22 13:24:18 +00:00
erik-krogh
f60c01e3a8 Py: delete import that no longer exists 2024-01-22 09:22:50 +01:00
erik-krogh
8be7eadace delete outdated deprecations 2024-01-22 09:11:35 +01:00
Sid Shankar
2c683c910f Merge branch 'change/adjust-extracted-files-diagnostics' of https://github.com/sidshank/codeql into change/adjust-extracted-files-diagnostics 2024-01-17 14:32:36 +00:00
Sid Shankar
0824ab77e9 Adds change notes 2024-01-17 14:31:40 +00:00
Sid Shankar
59098be8c4 Merge branch 'main' into change/adjust-extracted-files-diagnostics 2024-01-16 21:51:41 -05:00
Alexander Eyers-Taylor
934474681d Merge pull request #15254 from github/post-release-prep/codeql-cli-2.16.0
Post-release preparation for codeql-cli-2.16.0
2024-01-16 14:50:40 +00:00
github-actions[bot]
57df8b92df Post-release preparation for codeql-cli-2.16.0 2024-01-15 15:00:50 +00:00
Max Schaefer
a8336328fd Merge pull request #15176 from github/max-schaefer/py-url-redirection-qhelp
Python: Mention more sanitisation options in py/url-redirection qhelp.
2024-01-12 10:50:33 +00:00
Tom Hvitved
f90201eb56 Data flow: Remove column from mayBenefitFromCallContext 2024-01-09 11:34:43 +01:00
Sid Shankar
fb660b8f05 Py: Report any extracted file as successfully extracted 2024-01-08 22:20:51 +00:00
github-actions[bot]
a6c8cc9551 Release preparation for version 2.16.0 2024-01-08 13:11:26 +00:00
Chuan-kai Lin
2924be554c Python: Fix typo in upgrade script 2024-01-05 07:15:21 -08:00
Rasmus Wriedt Larsen
95c24275f2 Merge pull request #15044 from RasmusWL/automated-subclass-models
Python: Automated subclass models
2024-01-05 10:43:48 +01:00
Aditya Sharad
b1803d0ac2 Merge rc/3.12 into main 2023-12-21 16:40:51 -08:00
Rasmus Lerchedahl Petersen
da4aef80e9 Revert "Python: make it a real consistency check"
This reverts commit 45411f4a93.
2023-12-20 16:15:17 +01:00
Rasmus Lerchedahl Petersen
0f89f69555 Python: fix VariableWrite and remove unneded step 2023-12-20 15:45:18 +01:00
Rasmus Lerchedahl Petersen
d039ceb689 Python: add test for fields 2023-12-20 15:23:45 +01:00
Rasmus Lerchedahl Petersen
706e9dc896 Python: fix compilation 2023-12-20 15:23:33 +01:00
Rasmus Lerchedahl Petersen
45411f4a93 Python: make it a real consistency check 2023-12-20 14:53:37 +01:00
Rasmus Lerchedahl Petersen
215b146f06 Python: remove unused member predicate 2023-12-20 14:45:00 +01:00
Rasmus Lerchedahl Petersen
491ca3f1e6 Python: hide synthetic variable node 2023-12-20 14:42:45 +01:00
Rasmus Lerchedahl Petersen
afb3d1da6f Python: move capture node to DataFlowPrivate 2023-12-20 14:41:17 +01:00
Rasmus Lerchedahl Petersen
3cea46fe7b Python: fix typos 2023-12-20 14:35:10 +01:00
Rasmus Lerchedahl Petersen
f8417b0dd8 Merge branch 'main' of https://github.com/github/codeql into python/captured-variables-basic 2023-12-20 13:16:42 +01:00
Rasmus Lerchedahl Petersen
7749b8e60e Python: add change-note 2023-12-20 12:53:37 +01:00
Max Schaefer
66fe32ab82 Python: Mention more sanitisation options in py/url-redirection qhelp. 2023-12-20 11:31:07 +00:00
Rasmus Lerchedahl Petersen
07c88dc0be Python: remove unnecessary post-processing
also, it is slightly incorrect...
2023-12-20 12:09:15 +01:00
Rasmus Lerchedahl Petersen
169d7a3c98 Python: Add scope entry definition nodes
otherwise we confuse captured variables
in the single scope entry cfg node. Now
we have one for each defined variable.
2023-12-20 12:09:00 +01:00
Rasmus Lerchedahl Petersen
3b7e29bed6 Python: add test for crosstalk 2023-12-20 12:08:05 +01:00
Rasmus Wriedt Larsen
72687e0368 Merge branch 'main' into automated-subclass-models 2023-12-19 17:08:25 +01:00
Rasmus Wriedt Larsen
56d86f9980 Revert "NEVER MERGE: Ensure we don't use site-packages stuff"
This reverts commit 0ed363bd79f9d3f9e9a905c1192adfe88f1faffb.
2023-12-19 17:07:40 +01:00
Rasmus Wriedt Larsen
9863309631 Python: auto subclass capture
(locally done with split + 5 x modeling runs + join, but squashed into one commit)
2023-12-19 17:07:40 +01:00
Rasmus Wriedt Larsen
ca7b69ec1f NEVER MERGE: Ensure we don't use site-packages stuff 2023-12-19 17:07:40 +01:00
Rasmus Wriedt Larsen
de2a563a8e Python: Delete old auto subclass capture files
In the final git history this only deletes one file, but when working
locally I deleted ALL files.
2023-12-19 17:07:21 +01:00
Rasmus Wriedt Larsen
bf271d7f0f Python: refactor how subclasses are specified
A little more explicit, so less prone to be overlooked when adding a new spec
2023-12-19 17:07:02 +01:00