Commit Graph

920 Commits

Author SHA1 Message Date
yoff
1a9bb2416a Python: deprecate Function.getAReturnValueFlowNode() and rewrite internal callers
Follow-up to the getAFlowNode deprecation in the same PR: same AST→legacy-CFG
bridge pattern. The 11 internal call sites (across objects/, types/,
frameworks/, and TypeTrackingImpl) are rewritten to bind a `Return ret`
explicitly, then constrain via `ret.getScope() = f and n.getNode() = ret.getValue()`.

The predicate itself is preserved with a deprecation note so external
users do not experience churn.

Semantic noop.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-06-22 14:55:19 +02:00
Copilot
717ff62d70 Python: deprecate AstNode.getAFlowNode() and rewrite internal callers
Preparatory refactor for the shared-CFG dataflow migration.

Deprecates the AstNode.getAFlowNode() cached predicate on the public
Python QL API and rewrites all ~140 internal callers across lib/, src/,
test/, and tools/ from `expr.getAFlowNode() = cfgNode` to
`cfgNode.getNode() = expr`, using ControlFlowNode.getNode() which
already exists in Flow.qll.

The predicate itself is preserved (with a deprecation note pointing at
the new pattern) so external users do not experience churn — they can
migrate at their own pace and the AST/CFG hierarchies still get the
intended untangling once the deprecation eventually elapses.

Semantic noop verified by:
- All 361 lib/ + src/ queries compile clean.
- All 122 ControlFlow + PointsTo library-tests pass.
- All 64 dataflow library-tests pass.
- All 113 Variables/Exceptions/Expressions/Statements/Functions/Imports/
  Security/CWE-798/ModificationOfParameterWithDefault query-tests pass.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-06-22 14:55:19 +02:00
yoff
8179bffe64 Merge pull request #21930 from github/yoff/python-dataflow-noop-simplifications
Python: inline init_module_submodule_defn into ImportResolution
2026-06-22 14:50:39 +02:00
Owen Mansel-Chan
1f9899d7db Extend added type tracking step to related types 2026-06-17 15:04:53 +01:00
Owen Mansel-Chan
ea7510bf72 Refactor ReExposedInstance logic into one place 2026-06-17 13:10:47 +01:00
Owen Mansel-Chan
9c65082189 Fix MISSING alert 2026-06-15 00:14:52 +01:00
copilot-swe-agent[bot]
73bc2d70ae Model instance-attribute type flow
Use a field level step like JS and Ruby.
2026-06-11 14:48:55 +02:00
Tom Hvitved
f5919875b7 Merge pull request #21941 from hvitved/python/content-approx
Python: Implement `ContentApprox`
2026-06-09 15:46:04 +02:00
yoff
0cea01c22f Merge pull request #21926 from github/yoff/python-simplify-decorator-predicates
Python: simplify decorator-detection predicates to pure AST match
2026-06-08 22:04:33 +02:00
Tom Hvitved
cc1ea25856 Python: Implement ContentApprox 2026-06-08 08:41:28 +02:00
Owen Mansel-Chan
da999ee440 Address review comments 2026-06-03 21:24:16 +01:00
Owen Mansel-Chan
ad97b6dd64 Use access path for str.join model 2026-06-02 16:14:56 +01:00
yoff
ac5fa629ef Python: inline init_module_submodule_defn into ImportResolution
The new-dataflow ImportResolution module only used
semmle.python.essa.SsaDefinitions for the 5-line helper predicate
SsaSource::init_module_submodule_defn. Inline it locally and drop the
dependency on legacy SsaDefinitions. This is the only remaining direct
import of semmle.python.essa.* in the new dataflow stack, so dropping
it makes the layering cleaner.

Semantic noop on the current SSA: SsaSourceVariable.getName() and
GlobalVariable.getId() both project the same DB column
(variable(_,_,result)), and the old call's 'init.getEntryNode() = f'
join was just constraining init = package via Scope.getEntryNode()'s
functional uniqueness. RA dump of accesses.ql confirms only the
expected predicate-rename shuffle; all 70 dataflow + ApiGraphs library
tests pass.

This factors out commit 8cab5a20f2 from the larger shared-CFG
migration #21925.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-06-02 08:24:17 +00:00
yoff
5fb75ac987 Python: simplify decorator-detection predicates to pure AST match
The internal predicates that identify `@staticmethod`, `@classmethod` and
`@property` decorators previously required the decorator's `NameNode` to
satisfy `isGlobal()` (i.e. no SSA def reaches the decorator's name use).
That filter was correct but unnecessarily indirect: these three names
are builtins, and even when a class body redefines one, the class body
has not started executing at the decorator position, so Python uses the
builtin.

Match the decorator's AST `Name` directly instead, dropping the CFG/SSA
detour. The slight semantic change — `isGlobal()` would have rejected
module-level shadowing of these builtins — is negligible in practice
and explicitly documented in the change note.

`hasContextmanagerDecorator` and `hasOverloadDecorator` keep the
`NameNode.isGlobal()` check because their target names (`contextmanager`,
`overload`) are imported, not builtin, and local shadowing is a real
concern.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-06-01 14:04:43 +00:00
Owen Mansel-Chan
b38440490a Address review comment 2026-05-31 21:47:44 +01:00
Owen Mansel-Chan
aee33a0cc9 Add missing code for TAnyTupleOrDictionaryElement 2026-05-29 10:26:24 +01:00
Owen Mansel-Chan
df15a719cb Add a ContentSet for any tuple or dictionary element 2026-05-28 16:48:23 +01:00
Owen Mansel-Chan
80c6f082d1 Fix TODO in containerStep 2026-05-28 11:34:02 +01:00
Owen Mansel-Chan
ec13e1bcd3 Add wildcard ContentSets to avoid performance problems 2026-05-27 15:28:07 +01:00
Rasmus Lerchedahl Petersen
9a180036a5 Python: conversion step for format_map
and adjust collection test
2026-05-21 16:59:08 +01:00
Rasmus Lerchedahl Petersen
facb3b681d Python: recover taint for % format strings 2026-05-21 16:57:50 +01:00
Rasmus Lerchedahl Petersen
b67694b2ab Python: Remove imprecise container steps
- remove `tupleStoreStep` and `dictStoreStep` from `containerStep`
   These are imprecise compared to the content being precise.
- add implicit reads to recover taint at sinks
- add implicit read steps for decoders
  to supplement the `AdditionalTaintStep`
  that now only covers when the full container is tainted.
2026-05-21 16:57:44 +01:00
Taus
ac48eca916 Python: Use cls.getMethod instead of getName 2026-03-23 15:26:00 +00:00
Taus
93e35661e6 Python: Make isNewType more precise
For module-level metaclass declarations, we now also check that the
right hand side in a `__metaclass__ = type` assignment is in fact the
built-in `type`.
2026-03-23 15:22:24 +00:00
Taus
a276f721f7 Python: Add ternary overridesMethod
This one also allows easy access to the method being overridden and the
class on which it resides. This let's us simplify DocStrings.ql
accordingly.
2026-03-23 15:21:27 +00:00
Taus
56c83e250e Python: Make comment more precise
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-03-23 15:09:27 +01:00
Taus
50b3b7ee1f Python: Add DuckTyping::hasUnreliableMro
Primarily used to filter out false positives in cases where our MRO
approximation may be wrong.
2026-03-20 13:30:29 +00:00
Taus
c04b615a07 Python: Extend DuckTyping module
Adds `overridesMethod` and `isPropertyAccessor`.
2026-03-20 13:28:45 +00:00
Taus
b57e92164c Python: Add declares/getAttribute API
These could arguably be moved to `Class` itself, but for now I'm
choosing to limit the changes to the `DuckTyping` module (until we
decide on a proper API).
2026-03-20 13:28:45 +00:00
Taus
cd92162920 Python: Add DuckTyping::isNewStyle
Approximates the behaviour of `Types::isNewStyle` but without depending
on points-to
2026-03-20 13:28:45 +00:00
Taus
33ed6034f6 Python: Introduce DuckTyping module
This module (which for convenience currently resides inside
`DataFlowDispatch`, but this may change later) contains convenience
predicates for bridging the gap between the data-flow layer and the old
points-to analysis.
2026-03-20 13:28:44 +00:00
Taus
f4841e1f39 Python: Use API graphs instead of points-to for simple built-ins
Also extends the list of known built-ins slightly, to add some that were
missing.
2026-03-20 13:28:44 +00:00
Taus
f2bad1e6e1 Python: Improve docstring and make predicate private 2026-03-09 13:41:38 +00:00
Taus
fa61f6f3df Python: Model @typing.overload in method resolution
Adds `hasOverloadDecorator` as a predicate on functions. It looks for
decorators called `overload` or `something.overload` (usually
`typing.overload` or `t.overload`). These are then filtered out in the
predicates that (approximate) resolving methods according to the MRO.

As the test introduced in the previous commit shows, this removes the
spurious resolutions we had before.
2026-03-05 22:20:03 +00:00
yoff
89e5a9bd72 Update python/ql/lib/semmle/python/dataflow/new/internal/DataFlowPublic.qll
Co-authored-by: Taus <tausbn@github.com>
2026-02-26 13:14:26 +01:00
yoff
9b9c9304c7 Python: simplify logic, suggested in review 2026-02-25 18:16:38 +01:00
yoff
7351e82c92 python: handle guards compared to boolean literals 2026-02-24 10:00:22 +01:00
Taus
cd62cdadff Python: Fix bad join in returnStep 2026-02-16 16:48:08 +00:00
Taus
306d7d1b5d Python: DataFlowDispatch.qll annotations 2026-02-16 13:48:32 +00:00
Taus
7ea96c43ec Python: DataFlowPrivate.qll annotations 2026-02-16 13:48:32 +00:00
Taus
bd71db87be Python: DataFlowPublic.qll annotations 2026-02-16 13:48:32 +00:00
Taus
c46c662b72 Python: LocalSources.qll annotations 2026-02-16 13:48:32 +00:00
Taus
df0f2f8ce4 Python: Simple dataflow annotations
None of these required any changes to the dataflow libraries, so it
seemed easiest to put them in their own commit.
2026-02-16 13:48:32 +00:00
Taus
62fb38d834 Python: Rename otherArgs to implicitArgumentNode
Co-authored-by: yoff <yoff@github.com>
2026-02-03 15:32:46 +00:00
Taus
3f718123a6 Python: Make capturing closure arguments synthetic and non-global
Uses the same trick as for `ExtractedArgumentNode`, wherein we postpone
the global restriction on the charpred to instead be in the `argumentOf`
predicate (which is global anyway).

In addition to this, we also converted `CapturedVariablesArgumentNode`
into a proper synthetic node, and added an explicit post-update node for
it. These nodes just act as wrappers for the function part of call
nodes. Thus, to make them work with the variable capture machinery, we
simply map them to the closure node for the corresponding control-flow
or post-update node.
2026-01-30 12:50:25 +00:00
Taus
7fccc23dbe Python: Make ExtractedArgumentNode local
Explicitly adds a bunch of nodes that were previously (using a global
analysis) identified as `ExtractedArgumentNode`s. These are then
subsequently filtered out in `argumentOf` (which is global) by putting
the call to `getCallArg` there instead of in the charpred.
2026-01-30 12:50:25 +00:00
Taus
30ce4069c7 Python: Remove global restriction on ModuleVariableNode
This may result in more nodes, but it should still be bounded by the
number of global variables in the source code.
2026-01-30 12:50:24 +00:00
Taus
4543c66d26 Python: Prepare LocalSourceNode for locality
Removes the dependence on the (global) `ModuleVariableNode.getARead()`,
by adding a local version (that doesn't include `import *` reads)
instead.
2026-01-30 12:50:24 +00:00
Tom Hvitved
b974a84bef Merge pull request #21051 from hvitved/shared/flow-summary-provenance-filtering
Shared: Provenance-based filtering of flow summaries
2026-01-26 17:24:34 +01:00
Tom Hvitved
0adece7cde Python: Adapt to changes in FlowSummaryImpl 2026-01-26 12:40:19 +01:00