Commit Graph

3027 Commits

Author SHA1 Message Date
Taus
c75329d7b7 Python: Move metrics-related API to LegacyPointsTo module
Gets rid of the `getMetrics` methods on the `Function`, `Class`, and
`Module` classes. To access the metrics, one must first import the
`LegacyPointsTo` module, and then either change the type to
`{Function,Class,Module}Metrics` or cast to the appropriate type.
2025-11-26 17:06:55 +00:00
Taus
b9a5b3b628 Python: Remove points-to from SSA.ql
Happily, this was not as deeply entwined as it looked at first glance.
2025-11-26 17:06:55 +00:00
Taus
5b47fcbfa4 Python: Remove dependence on Builtins from attribute module
The `Builtins` module is deeply entwined with points-to, so it would be
nice to not have this dependence. Happily, the only thing we used
`Builtin` for was to get the names of known builtins, and for this we
already maintain such a set of names in
`dataflow.new.internal.Builtins`.
2025-11-26 12:30:31 +00:00
Taus
9dc774aaa3 Python: Remove points-to dependency from parts of SSA
For whatever reason, the CFG node for exceptions and exception groups
was placed with the points-to code. (Probably because a lot of the
predicates depended on points-to.)

However, as it turned out, two of the SSA modules only depended on
non-points-to properties of these nodes, and so it was fairly
straightforward to remove the imports of `LegacyPointsTo` for those
modules.

In the process, I moved the aforementioned CFG node types into
`Flow.qll`, and changed the classes in the `Exceptions` module to the
`...WithPointsTo` form that we introduced elsewhere.
2025-11-26 12:30:31 +00:00
Taus
e09840426c Python: Get rid of points-to from Definitions.qll
Turns out the `ImportTime` module (despite living in
`semmle.python.types` does not actually depend on points-to, so some of
the `LegacyPointsTo` imports could be replaced or removed.
2025-11-26 12:30:31 +00:00
Taus
7328f26311 Python: Fix reachability-related test failures 2025-11-26 12:30:31 +00:00
Taus
21e74a3f01 Python: Fully remove points-to from Flow.qll
Gets rid of a bunch of predicates relating to reachability (which
depended on the modelling of exceptions, which uses points-to), moving
them to `LegacyPointsTo`. In the process, we gained a new class
`BasicBlockWithPointsTo`.
2025-11-26 12:30:31 +00:00
Taus
0a4ec2ca10 Python: Move some non-points-to methods out of points-to
These methods were in `pointsto.Base` but did not actually interact with
the points-to machinery directly, so they were easy to move out.
2025-11-26 12:30:30 +00:00
Taus
f0465f441f Python: Get rid of some get...Object methods
This frees `Class.qll`, `Exprs.qll`, and `Function.qll` from the
clutches of points-to. For the somewhat complicated setup with
`getLiteralObject` (an abstract method), I opted for a slightly ugly but
workable solution of just defining a predicate on `ImmutableLiteral`
that inlines each predicate body, special-cased to the specific instance
to which it applies.
2025-11-26 12:30:30 +00:00
Taus
78c33ab55a Python: Remove points-to references from python.qll
For now, these have just been made into `private` imports. After doing
this, I went through all of the (now not compiling) files and added in
private imports to the modules that they actually depended on.

I also added an explicit import of `LegacyPointsTo` (even though it may
be unnecessary) in cases where the points-to dependency was somewhat
surprising (and one we want to get rid of). This was primarily inside
the various SSA layers.

For modules inside `semmle.python.{types, objects, pointsto}` I did not
bother, as these are fairly clearly related to points-to.
2025-11-26 12:30:30 +00:00
Asger F
613895e0c0 Merge pull request #20424 from asgerf/js/overlay-manual-v4
JS: Add overlay annotations
2025-11-20 11:10:46 +01:00
github-actions[bot]
5ee45af3aa Post-release preparation for codeql-cli-2.23.6 2025-11-18 09:53:12 +00:00
github-actions[bot]
18fa6799ce Release preparation for version 2.23.6 2025-11-17 16:38:07 +00:00
Asger F
ecfa94600f Sync ApiGraphModels.qll 2025-11-13 09:46:23 +01:00
Asger F
16e7dc1b8a Sync ApiGraphModelsExtensions.qll 2025-11-13 09:46:21 +01:00
Michael B. Gale
046db0419f Merge pull request #20758 from github/post-release-prep/codeql-cli-2.23.4
Post-release preparation for codeql-cli-2.23.4
2025-11-05 10:45:51 +00:00
github-actions[bot]
4014df9a6e Post-release preparation for codeql-cli-2.23.4 2025-11-04 17:57:52 +00:00
github-actions[bot]
64fcdd1f2f Release preparation for version 2.23.4 2025-11-03 14:52:23 +00:00
Taus
e702d3bfc8 Python: Add change note
I wasn't entirely sure if this should be classified as `deprecated` or
`breaking`, but seeing as these changes technically _could_ break
existing queries (requiring a small rewrite), I opted for the latter.
2025-10-30 15:16:51 +00:00
Taus
820d8e76c4 Python: Remove points-to from Module 2025-10-30 13:59:30 +00:00
Taus
b93ce98612 Python: Remove points-to from Expr 2025-10-30 13:58:59 +00:00
Taus
b434ce460e Python: Get rid of getLiteralValue
This had only two uses in our libraries, so I simply inlined the
predicate body in both places.
2025-10-30 13:30:04 +00:00
Taus
fef08afff9 Python: Remove points-to to from ControlFlowNode
Moves the existing points-to predicates to the newly added class
`ControlFlowNodeWithPointsTo` which resides in the `LegacyPointsTo`
module.

(Existing code that uses these predicates should import this module, and
references to `ControlFlowNode` should be changed to
`ControlFlowNodeWithPointsTo`.)

Also updates all existing points-to based code to do just this.
2025-10-30 13:30:04 +00:00
Nora Dimitrijević
37fff48dcd Python/ServerSideRequestForgeryQuery
python/ql/src/Security/CWE-918/PartialServerSideRequestForgery.ql
2025-10-28 09:40:24 +01:00
Nora Dimitrijević
baccdcc07f Python/PolynomialReDoSQuery
python/ql/src/Security/CWE-730/PolynomialReDoS.ql
2025-10-28 09:40:21 +01:00
Joe Farebrother
8c277bd1d9 Merge pull request #20494 from joefarebrother/python-insecure-cookie-split
Python: Split Insecure Cookie query into multiple queries
2025-10-24 11:10:20 +01:00
Nora Dimitrijević
e120e5c3ba Merge pull request #20337 from d10c/d10c/python-overlay-compilation-plus-extractor
Python: enable overlay compilation + extractor overlay support
2025-10-16 14:49:01 +02:00
github-actions[bot]
6dd07790ac Post-release preparation for codeql-cli-2.23.3 2025-10-14 11:16:33 +00:00
github-actions[bot]
33542f7d40 Release preparation for version 2.23.3 2025-10-14 09:30:24 +00:00
Taus
c4b27d5f28 Python: Fix ImportError in imp.py under Python 3.14
It seems `_ERR_MSG` was silently removed in Python 3.14, leading to an
`ImportError` when running the extractor.

To fix this, we explicitly set `_ERR_MSG` when the existing import fails
(using `_ERR_MSG_PREFIX` which is available in Python 3.14+, along with
the bits that make up the difference between this and `_ERR_MSG`).
2025-10-13 13:50:43 +00:00
Nora Dimitrijević
ece121070b Add change note. 2025-10-06 12:31:21 +02:00
Nora Dimitrijević
20d4e429ca Add consistency query (exactly one path for every entity) 2025-10-06 11:47:56 +02:00
Nora Dimitrijević
7174d4c8ba Overlay.qll: discard predicates
for dbscheme elements with direct or indirect location links in dbscheme.

- Unify discardable entities under one Discardable superclass.
- Two discard predicates depending on TRAP ID type.
- Future-proof the XML and Yaml discard predicates for when their
  extractors become incremental.
2025-10-06 11:47:51 +02:00
Nora Dimitrijević
1a9683f986 Add @top database type 2025-10-06 11:47:14 +02:00
Nora Dimitrijević
1574b5fd91 Add synthetic data to dbscheme.stats for databaseMetadata/overlayChangedFiles 2025-10-06 11:37:00 +02:00
Nora Dimitrijević
1c3a7f2b1e Add database upgrade/downgrade scripts 2025-10-06 11:36:58 +02:00
Nora Dimitrijević
a88d3397cd Add overlay builtins to python dbscheme 2025-10-06 11:36:56 +02:00
Nora Dimitrijević
dac50fa0c1 Enable overlay compilation in lib/qlpack.yml 2025-10-06 11:36:51 +02:00
github-actions[bot]
a7a4e43991 Post-release preparation for codeql-cli-2.23.2 2025-09-29 15:10:19 +00:00
github-actions[bot]
d2130a589b Release preparation for version 2.23.2 2025-09-29 10:28:45 +00:00
Taus
e592fd60ff Merge pull request #20495 from github/tausbn/python-fix-unmatchable-dollar-in-lookahead
Python: Fix false positive for unmatchable dollar/caret
2025-09-25 15:27:32 +02:00
Joe Farebrother
2cffb21604 Update and fix tests 2025-09-23 15:41:09 +01:00
Joe Farebrother
d28e8004fd Add sensitive data heuristic 2025-09-23 10:08:08 +01:00
Simon Friis Vindum
7d6e2060e5 Adapt all languages to changes in shared library 2025-09-22 14:18:58 +02:00
Joe Farebrother
463f79bed2 Merge pull request #20263 from joefarebrother/python-qual-exceptions
Python: Modernize the Unreachable Except Block query
2025-09-22 09:42:09 +01:00
Taus
95a84ad655 Python: Fix false positive for unmatchable dollar/caret
Our previous modelling did not account for the fact that a lookahead can
potentially extend all the way to the end of the input (and similarly,
that a lookbehind can extend all the way to the beginning).

To fix this, I extended `firstPart` and `lastPart` to handle lookbehinds
and lookaheads correctly, and added some test cases (all of which yield
no new results).

Fixes #20429.
2025-09-19 15:06:46 +00:00
Ian Lynagh
c653d939d9 Merge pull request #20451 from github/post-release-prep/codeql-cli-2.23.1
Post-release preparation for codeql-cli-2.23.1
2025-09-17 13:00:14 +01:00
github-actions[bot]
4e8343664f Post-release preparation for codeql-cli-2.23.1 2025-09-17 10:13:40 +00:00
Taus
8fd62252fd Python: Fix bad join in globalVariableNestedFieldJumpStep 2025-09-16 18:12:29 +02:00
Napalys Klicius
431fc8880e Python: Add change note 2025-09-16 18:08:53 +02:00