Commit Graph

948 Commits

Author SHA1 Message Date
Michael B. Gale
f4d8358454 Merge pull request #22110 from github/post-release-prep/codeql-cli-2.26.0
Post-release preparation for codeql-cli-2.26.0
2026-07-02 15:32:22 +01:00
Mathias Vorreiter Pedersen
4f4cdf434b Merge pull request #22061 from MathiasVP/mad-write-through-model
Shared: Support flow summaries from `ReturnValue`s
2026-07-02 12:38:44 +01:00
Tom Hvitved
32181cd7e8 Python: Improve some flow summaries 2026-07-02 11:54:28 +02:00
Tom Hvitved
a5444b573a Python: Improve some flow summaries 2026-07-01 12:05:53 +02:00
Taus
f251a572e1 Python: hotfix - disable instanceFieldStep to avoid type-tracker blowup
The `instanceFieldStep` disjunct of `TypeTrackingInput::levelStepCall`
that was added in 7.2.0 uses `classInstanceTracker(cls)` -- which is
itself a type-tracker -- inside `levelStepCall`. That creates a
structural mutual recursion between the main type-tracker fixpoint and
`classInstanceTracker`, causing the type-tracker delta to blow up to
~100M tuples per iteration on some OOP-heavy Python codebases.
Verified on the python/mypy database: SSRF query wall time goes from
~12s before the offending commit to >40 minutes after it.

This hotfix temporarily drops the `instanceFieldStep` disjunct and
keeps only `inheritedFieldStep`, which does not pull on the call
graph and is well-behaved (verified at ~12s on mypy). The
`instanceFieldStep` helper predicate itself is kept in place, and
the `levelStepCall` body has a commented-out call to it so the
change is trivial to re-enable once the recursion issue is properly
addressed.
2026-06-30 14:41:12 +00:00
Mathias Vorreiter Pedersen
f9e1305da3 Python: Fixes after changes to the flow summary API. 2026-06-23 20:33:22 +01:00
yoff
1a9bb2416a Python: deprecate Function.getAReturnValueFlowNode() and rewrite internal callers
Follow-up to the getAFlowNode deprecation in the same PR: same AST→legacy-CFG
bridge pattern. The 11 internal call sites (across objects/, types/,
frameworks/, and TypeTrackingImpl) are rewritten to bind a `Return ret`
explicitly, then constrain via `ret.getScope() = f and n.getNode() = ret.getValue()`.

The predicate itself is preserved with a deprecation note so external
users do not experience churn.

Semantic noop.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-06-22 14:55:19 +02:00
Copilot
717ff62d70 Python: deprecate AstNode.getAFlowNode() and rewrite internal callers
Preparatory refactor for the shared-CFG dataflow migration.

Deprecates the AstNode.getAFlowNode() cached predicate on the public
Python QL API and rewrites all ~140 internal callers across lib/, src/,
test/, and tools/ from `expr.getAFlowNode() = cfgNode` to
`cfgNode.getNode() = expr`, using ControlFlowNode.getNode() which
already exists in Flow.qll.

The predicate itself is preserved (with a deprecation note pointing at
the new pattern) so external users do not experience churn — they can
migrate at their own pace and the AST/CFG hierarchies still get the
intended untangling once the deprecation eventually elapses.

Semantic noop verified by:
- All 361 lib/ + src/ queries compile clean.
- All 122 ControlFlow + PointsTo library-tests pass.
- All 64 dataflow library-tests pass.
- All 113 Variables/Exceptions/Expressions/Statements/Functions/Imports/
  Security/CWE-798/ModificationOfParameterWithDefault query-tests pass.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-06-22 14:55:19 +02:00
yoff
8179bffe64 Merge pull request #21930 from github/yoff/python-dataflow-noop-simplifications
Python: inline init_module_submodule_defn into ImportResolution
2026-06-22 14:50:39 +02:00
Owen Mansel-Chan
1f9899d7db Extend added type tracking step to related types 2026-06-17 15:04:53 +01:00
Owen Mansel-Chan
ea7510bf72 Refactor ReExposedInstance logic into one place 2026-06-17 13:10:47 +01:00
Owen Mansel-Chan
9c65082189 Fix MISSING alert 2026-06-15 00:14:52 +01:00
copilot-swe-agent[bot]
73bc2d70ae Model instance-attribute type flow
Use a field level step like JS and Ruby.
2026-06-11 14:48:55 +02:00
Tom Hvitved
f5919875b7 Merge pull request #21941 from hvitved/python/content-approx
Python: Implement `ContentApprox`
2026-06-09 15:46:04 +02:00
yoff
0cea01c22f Merge pull request #21926 from github/yoff/python-simplify-decorator-predicates
Python: simplify decorator-detection predicates to pure AST match
2026-06-08 22:04:33 +02:00
Tom Hvitved
cc1ea25856 Python: Implement ContentApprox 2026-06-08 08:41:28 +02:00
Owen Mansel-Chan
da999ee440 Address review comments 2026-06-03 21:24:16 +01:00
Owen Mansel-Chan
ad97b6dd64 Use access path for str.join model 2026-06-02 16:14:56 +01:00
yoff
ac5fa629ef Python: inline init_module_submodule_defn into ImportResolution
The new-dataflow ImportResolution module only used
semmle.python.essa.SsaDefinitions for the 5-line helper predicate
SsaSource::init_module_submodule_defn. Inline it locally and drop the
dependency on legacy SsaDefinitions. This is the only remaining direct
import of semmle.python.essa.* in the new dataflow stack, so dropping
it makes the layering cleaner.

Semantic noop on the current SSA: SsaSourceVariable.getName() and
GlobalVariable.getId() both project the same DB column
(variable(_,_,result)), and the old call's 'init.getEntryNode() = f'
join was just constraining init = package via Scope.getEntryNode()'s
functional uniqueness. RA dump of accesses.ql confirms only the
expected predicate-rename shuffle; all 70 dataflow + ApiGraphs library
tests pass.

This factors out commit 8cab5a20f2 from the larger shared-CFG
migration #21925.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-06-02 08:24:17 +00:00
yoff
5fb75ac987 Python: simplify decorator-detection predicates to pure AST match
The internal predicates that identify `@staticmethod`, `@classmethod` and
`@property` decorators previously required the decorator's `NameNode` to
satisfy `isGlobal()` (i.e. no SSA def reaches the decorator's name use).
That filter was correct but unnecessarily indirect: these three names
are builtins, and even when a class body redefines one, the class body
has not started executing at the decorator position, so Python uses the
builtin.

Match the decorator's AST `Name` directly instead, dropping the CFG/SSA
detour. The slight semantic change — `isGlobal()` would have rejected
module-level shadowing of these builtins — is negligible in practice
and explicitly documented in the change note.

`hasContextmanagerDecorator` and `hasOverloadDecorator` keep the
`NameNode.isGlobal()` check because their target names (`contextmanager`,
`overload`) are imported, not builtin, and local shadowing is a real
concern.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-06-01 14:04:43 +00:00
Owen Mansel-Chan
b38440490a Address review comment 2026-05-31 21:47:44 +01:00
Owen Mansel-Chan
aee33a0cc9 Add missing code for TAnyTupleOrDictionaryElement 2026-05-29 10:26:24 +01:00
Owen Mansel-Chan
df15a719cb Add a ContentSet for any tuple or dictionary element 2026-05-28 16:48:23 +01:00
Owen Mansel-Chan
80c6f082d1 Fix TODO in containerStep 2026-05-28 11:34:02 +01:00
Owen Mansel-Chan
ec13e1bcd3 Add wildcard ContentSets to avoid performance problems 2026-05-27 15:28:07 +01:00
Rasmus Lerchedahl Petersen
9a180036a5 Python: conversion step for format_map
and adjust collection test
2026-05-21 16:59:08 +01:00
Rasmus Lerchedahl Petersen
facb3b681d Python: recover taint for % format strings 2026-05-21 16:57:50 +01:00
Rasmus Lerchedahl Petersen
b67694b2ab Python: Remove imprecise container steps
- remove `tupleStoreStep` and `dictStoreStep` from `containerStep`
   These are imprecise compared to the content being precise.
- add implicit reads to recover taint at sinks
- add implicit read steps for decoders
  to supplement the `AdditionalTaintStep`
  that now only covers when the full container is tainted.
2026-05-21 16:57:44 +01:00
Taus
ac48eca916 Python: Use cls.getMethod instead of getName 2026-03-23 15:26:00 +00:00
Taus
93e35661e6 Python: Make isNewType more precise
For module-level metaclass declarations, we now also check that the
right hand side in a `__metaclass__ = type` assignment is in fact the
built-in `type`.
2026-03-23 15:22:24 +00:00
Taus
a276f721f7 Python: Add ternary overridesMethod
This one also allows easy access to the method being overridden and the
class on which it resides. This let's us simplify DocStrings.ql
accordingly.
2026-03-23 15:21:27 +00:00
Taus
56c83e250e Python: Make comment more precise
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-03-23 15:09:27 +01:00
Taus
50b3b7ee1f Python: Add DuckTyping::hasUnreliableMro
Primarily used to filter out false positives in cases where our MRO
approximation may be wrong.
2026-03-20 13:30:29 +00:00
Taus
c04b615a07 Python: Extend DuckTyping module
Adds `overridesMethod` and `isPropertyAccessor`.
2026-03-20 13:28:45 +00:00
Taus
b57e92164c Python: Add declares/getAttribute API
These could arguably be moved to `Class` itself, but for now I'm
choosing to limit the changes to the `DuckTyping` module (until we
decide on a proper API).
2026-03-20 13:28:45 +00:00
Taus
cd92162920 Python: Add DuckTyping::isNewStyle
Approximates the behaviour of `Types::isNewStyle` but without depending
on points-to
2026-03-20 13:28:45 +00:00
Taus
33ed6034f6 Python: Introduce DuckTyping module
This module (which for convenience currently resides inside
`DataFlowDispatch`, but this may change later) contains convenience
predicates for bridging the gap between the data-flow layer and the old
points-to analysis.
2026-03-20 13:28:44 +00:00
Taus
f4841e1f39 Python: Use API graphs instead of points-to for simple built-ins
Also extends the list of known built-ins slightly, to add some that were
missing.
2026-03-20 13:28:44 +00:00
Taus
f2bad1e6e1 Python: Improve docstring and make predicate private 2026-03-09 13:41:38 +00:00
Taus
fa61f6f3df Python: Model @typing.overload in method resolution
Adds `hasOverloadDecorator` as a predicate on functions. It looks for
decorators called `overload` or `something.overload` (usually
`typing.overload` or `t.overload`). These are then filtered out in the
predicates that (approximate) resolving methods according to the MRO.

As the test introduced in the previous commit shows, this removes the
spurious resolutions we had before.
2026-03-05 22:20:03 +00:00
yoff
89e5a9bd72 Update python/ql/lib/semmle/python/dataflow/new/internal/DataFlowPublic.qll
Co-authored-by: Taus <tausbn@github.com>
2026-02-26 13:14:26 +01:00
yoff
9b9c9304c7 Python: simplify logic, suggested in review 2026-02-25 18:16:38 +01:00
yoff
7351e82c92 python: handle guards compared to boolean literals 2026-02-24 10:00:22 +01:00
Taus
cd62cdadff Python: Fix bad join in returnStep 2026-02-16 16:48:08 +00:00
Taus
306d7d1b5d Python: DataFlowDispatch.qll annotations 2026-02-16 13:48:32 +00:00
Taus
7ea96c43ec Python: DataFlowPrivate.qll annotations 2026-02-16 13:48:32 +00:00
Taus
bd71db87be Python: DataFlowPublic.qll annotations 2026-02-16 13:48:32 +00:00
Taus
c46c662b72 Python: LocalSources.qll annotations 2026-02-16 13:48:32 +00:00
Taus
df0f2f8ce4 Python: Simple dataflow annotations
None of these required any changes to the dataflow libraries, so it
seemed easiest to put them in their own commit.
2026-02-16 13:48:32 +00:00
Taus
62fb38d834 Python: Rename otherArgs to implicitArgumentNode
Co-authored-by: yoff <yoff@github.com>
2026-02-03 15:32:46 +00:00