The new (shared-CFG-based) Python control flow graph in
`semmle.python.controlflow.internal.Cfg` previously did not emit CFG
nodes for parameter type annotations (`def f(x: T): ...`) or for the
return type annotation (`-> T`). The legacy CFG emitted both, and a
small number of framework models rely on this: `LocalSources.qll`'s
`annotatedInstance` walks the parameter annotation expression by way
of its CFG node to track that a parameter receives an instance of the
annotated class.
After the dataflow flip to the new CFG/SSA this regression manifested
as lost flows in any test exercising annotation-based parameter
tracking: FastAPI `Depends()` receivers, Pydantic request bodies,
Starlette `WebSocket`, the call-graph type-annotation test, and so on.
Extend `FunctionDefExpr` to visit each annotation as a child of the
function-def expression, in CPython evaluation order: positional
parameter annotations, `*args` annotation, keyword-only parameter
annotations, `**kwargs` annotation, then the return annotation. (Lambda
expressions have no annotations in Python syntax, so `LambdaExpr` is
unchanged.) PEP 695 type parameters remain out of scope; they belong
to the inner annotation scope, not the enclosing CFG.
Restored test results across `framework/aiohttp`, `framework/fastapi`,
`framework/lxml`, the `CallGraph-type-annotations` test, and
`CWE-022-PathInjection`. Two FastAPI list-comprehension MISSING markers
become positive (`taint_test.py:41,55`). CPython CFG consistency
remains clean.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Flips the Python dataflow trunk from the legacy CFG (semmle/python/Flow.qll)
and legacy ESSA SSA (semmle/python/essa/*) to the new shared CFG facade
(semmle.python.controlflow.internal.Cfg) and the new SSA adapter
(semmle.python.dataflow.new.internal.SsaImpl), both introduced
additively in the preceding PRs in this stack.
This is the trunk-flip equivalent of the original draft PR #21894 (kept
around as documentation), rebased on top of the four preparatory PRs:
P1: Remove AstNode.getAFlowNode() and rewrite callers (#21919).
P2: Qualify Flow.qll's AST references with Py:: prefix (#21920).
P3: Add new shared-CFG-backed control flow graph (#21921).
P4: Add new shared-SSA-backed SSA adapter (#21923).
The Python dataflow library (semmle/python/dataflow/new/) now imports
the new CFG facade and SSA adapter. All CFG-typed predicates
(ControlFlowNode, CallNode, BasicBlock, NameNode, AttrNode, ...) are
qualified with the Cfg:: prefix; SSA references switch from
EssaVariable/EssaDefinition to SsaImpl::Definition/SourceVariable.
GuardNode is redesigned to use the new CFG's outcome-node model
(isAfterTrue / isAfterFalse) instead of the legacy ConditionBlock +
flipped indirection. Only BarrierGuard<...> is preserved as public
API.
Framework files (Bottle, FastApi, Django, Tornado, Pyramid, Stdlib,
...) are updated to take CFG nodes from the new facade.
A handful of dataflow consistency tweaks for the new CFG:
- Augmented-assignment targets are treated as both load and store.
- 'from X import *' produces uncertain SSA writes for unknown names.
- CFG nodes are canonicalised so dataflow does not see equivalent
pre/post-order pairs as distinct nodes.
Two AST tweaks for the new CFG:
- AstNodeImpl: omit PEP 695 type-parameter names from
FunctionDefExpr / ClassDefExpr children.
- ImportResolution: drop the legacy essa import.
Test churn (~175 files): reblessed library- and query-test .expected
files reflect slightly different CFG granularity, different toString
output, and a handful of true alert deltas in security queries.
Verification: all 367 lib + src + consistency-queries compile clean.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- remove `tupleStoreStep` and `dictStoreStep` from `containerStep`
These are imprecise compared to the content being precise.
- add implicit reads to recover taint at sinks
- add implicit read steps for decoders
to supplement the `AdditionalTaintStep`
that now only covers when the full container is tainted.
With `ModuleVariableNode`s now appearing for _all_ global variables (not
just the ones that actually seem to be used), some of the tests changed
a bit. Mostly this was in the form of new flow (because of new nodes
that popped into existence). For some inline expectation tests, I opted
to instead exclude these results, as there was no suitable location to
annotate. For the normal tests, I just accepted the output (after having
vetted it carefully, of course).
