Commit Graph

23 Commits

Author SHA1 Message Date
Copilot
4ed5722e3e Python: switch dataflow library to new (shared) CFG + SSA
Flips the Python dataflow trunk from the legacy CFG (semmle/python/Flow.qll)
and legacy ESSA SSA (semmle/python/essa/*) to the new shared CFG facade
(semmle.python.controlflow.internal.Cfg) and the new SSA adapter
(semmle.python.dataflow.new.internal.SsaImpl), both introduced
additively in the preceding PRs in this stack.

This is the trunk-flip equivalent of the original draft PR #21894 (kept
around as documentation), rebased on top of the four preparatory PRs:

  P1: Remove AstNode.getAFlowNode() and rewrite callers (#21919).
  P2: Qualify Flow.qll's AST references with Py:: prefix (#21920).
  P3: Add new shared-CFG-backed control flow graph (#21921).
  P4: Add new shared-SSA-backed SSA adapter (#21923).

The Python dataflow library (semmle/python/dataflow/new/) now imports
the new CFG facade and SSA adapter. All CFG-typed predicates
(ControlFlowNode, CallNode, BasicBlock, NameNode, AttrNode, ...) are
qualified with the Cfg:: prefix; SSA references switch from
EssaVariable/EssaDefinition to SsaImpl::Definition/SourceVariable.

GuardNode is redesigned to use the new CFG's outcome-node model
(isAfterTrue / isAfterFalse) instead of the legacy ConditionBlock +
flipped indirection. Only BarrierGuard<...> is preserved as public
API.

Framework files (Bottle, FastApi, Django, Tornado, Pyramid, Stdlib,
...) are updated to take CFG nodes from the new facade.

A handful of dataflow consistency tweaks for the new CFG:
- Augmented-assignment targets are treated as both load and store.
- 'from X import *' produces uncertain SSA writes for unknown names.
- CFG nodes are canonicalised so dataflow does not see equivalent
  pre/post-order pairs as distinct nodes.

Two AST tweaks for the new CFG:
- AstNodeImpl: omit PEP 695 type-parameter names from
  FunctionDefExpr / ClassDefExpr children.
- ImportResolution: drop the legacy essa import.

Test churn (~175 files): reblessed library- and query-test .expected
files reflect slightly different CFG granularity, different toString
output, and a handful of true alert deltas in security queries.

Verification: all 367 lib + src + consistency-queries compile clean.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-06-02 14:09:45 +00:00
Owen Mansel-Chan
aa28c94562 Remove double space after $ in inline expectations tests 2026-03-04 14:12:42 +00:00
Owen Mansel-Chan
91b6801db1 py: Inline expectation should have space before $ 2026-03-04 13:11:38 +00:00
yoff
158430af82 Merge pull request #17765 from yoff/python/test-functional-behaviour
Python: Add tests for functional-like programming
2025-02-11 16:28:37 +01:00
Joe Farebrother
2aea356756 Add change note + fix tests 2025-01-15 10:24:18 +00:00
Joe Farebrother
6a6585e415 Add tests for zip and enumerate 2025-01-15 09:57:15 +00:00
Joe Farebrother
460de3f7d5 Reduce generality of map and zip for performance 2025-01-14 09:39:57 +00:00
Joe Farebrother
4e36008ed9 Add tests 2025-01-14 09:39:56 +00:00
Michael Nebel
2321ca59f6 Python: Update all test util paths to point to the new location. 2024-12-12 13:54:30 +01:00
Jeroen Ketema
c3ea883b11 Python: Update expected test results 2024-12-03 19:18:57 +01:00
yoff
9ed8fe5dd0 Update python/ql/test/library-tests/dataflow/coverage/functional.py
Co-authored-by: Taus <tausbn@github.com>
2024-10-15 17:35:36 +02:00
Rasmus Lerchedahl Petersen
195b70aca6 python: Add test for functional-like programming
This can also serve for a place to add tests for
constructs like threading.Thread, mulitprocess.Process, concurrent.futures.ThreadPoolExecutor, and concurrent.futures.ProcessPoolExecutor.
2024-10-15 12:54:30 +02:00
Rasmus Lerchedahl Petersen
6f5b949ec8 Python: adjust test expectations
note that we do retain precision in
`test_dict_from_keyword()`
2024-10-04 15:30:02 +02:00
Rasmus Lerchedahl Petersen
a4c1a622b7 Merge branch 'main' of https://github.com/github/codeql into python/add-comprehension-capture-flow 2024-10-04 14:53:03 +02:00
Rasmus Lerchedahl Petersen
38b1eb7c71 Python: just use ListElementContent for iterables 2024-10-01 16:24:15 +02:00
yoff
62509a10c2 Update python/ql/test/library-tests/dataflow/coverage/test_builtins.py
Co-authored-by: Rasmus Wriedt Larsen <rasmuswriedtlarsen@gmail.com>
2024-10-01 11:39:12 +02:00
Rasmus Lerchedahl Petersen
bd68986fa4 Python: add test showing dict can take multiple arguments 2024-10-01 10:01:22 +02:00
Rasmus Lerchedahl Petersen
fb07a56de6 Python: adjust test expectations 2024-09-30 13:26:59 +02:00
Rasmus Lerchedahl Petersen
d4ea62edec Python: flow through yield
- add yield as a dataflow return
- replace comprehension store step
   with a store step to the yield
2024-09-30 09:01:29 +02:00
Rasmus Lerchedahl Petersen
fc2dc28f87 python: capture flow through comprehensions
- add comprehension functions as `DataFlowCallable`s
- add comprehension call as `DataFlowCall`
- create capture argument node for comprehension calls
2024-09-25 10:02:31 +02:00
Rasmus Wriedt Larsen
bb00d6919a Python: Move dataflow TestUtil to importable location 2024-04-23 09:40:59 +02:00
Rasmus Wriedt Larsen
e0e405bb31 Python: replace dataflow-test location in files 2024-04-23 09:40:59 +02:00
Rasmus Wriedt Larsen
ce711f7d2f Python: Move dataflow tests out of experimental 2024-04-23 09:40:44 +02:00