Commit Graph

26 Commits

Author SHA1 Message Date
yoff
3c9b0f770b Python: switch dataflow library to new (shared) CFG + SSA
Flips the Python dataflow trunk from the legacy CFG (semmle/python/Flow.qll)
and legacy ESSA SSA (semmle/python/essa/*) to the new shared CFG facade
(semmle.python.controlflow.internal.Cfg) and the new SSA adapter
(semmle.python.dataflow.new.internal.SsaImpl), both introduced
additively in the preceding PRs in this stack.

This is the trunk-flip equivalent of the original draft PR #21894 (kept
around as documentation), rebased on top of the four preparatory PRs:

  P1: Remove AstNode.getAFlowNode() and rewrite callers (#21919).
  P2: Qualify Flow.qll's AST references with Py:: prefix (#21920).
  P3: Add new shared-CFG-backed control flow graph (#21921).
  P4: Add new shared-SSA-backed SSA adapter (#21923).

The Python dataflow library (semmle/python/dataflow/new/) now imports
the new CFG facade and SSA adapter. All CFG-typed predicates
(ControlFlowNode, CallNode, BasicBlock, NameNode, AttrNode, ...) are
qualified with the Cfg:: prefix; SSA references switch from
EssaVariable/EssaDefinition to SsaImpl::Definition/SourceVariable.

GuardNode is redesigned to use the new CFG's outcome-node model
(isAfterTrue / isAfterFalse) instead of the legacy ConditionBlock +
flipped indirection. Only BarrierGuard<...> is preserved as public
API.

Framework files (Bottle, FastApi, Django, Tornado, Pyramid, Stdlib,
...) are updated to take CFG nodes from the new facade.

A handful of dataflow consistency tweaks for the new CFG:
- Augmented-assignment targets are treated as both load and store.
- 'from X import *' produces uncertain SSA writes for unknown names.
- CFG nodes are canonicalised so dataflow does not see equivalent
  pre/post-order pairs as distinct nodes.

Two AST tweaks for the new CFG:
- AstNodeImpl: omit PEP 695 type-parameter names from
  FunctionDefExpr / ClassDefExpr children.
- ImportResolution: drop the legacy essa import.

Test churn (~175 files): reblessed library- and query-test .expected
files reflect slightly different CFG granularity, different toString
output, and a handful of true alert deltas in security queries.

Verification: all 367 lib + src + consistency-queries compile clean.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-06-18 15:17:35 +00:00
Owen Mansel-Chan
5a97348e78 python: Inline expectation should have space after $
This was a regex-find-replace from `# \$(?! )` (using a negative lookahead) to `# $ `.
2026-03-04 12:45:05 +00:00
Michael Nebel
2321ca59f6 Python: Update all test util paths to point to the new location. 2024-12-12 13:54:30 +01:00
Jeroen Ketema
c3ea883b11 Python: Update expected test results 2024-12-03 19:18:57 +01:00
Rasmus Wriedt Larsen
bb00d6919a Python: Move dataflow TestUtil to importable location 2024-04-23 09:40:59 +02:00
Anders Schack-Mulligen
088a0a54ba Python: Add empty provenance column to expected files. 2024-02-09 11:32:08 +01:00
Rasmus Wriedt Larsen
55f5b26ba6 Python: Accept new ordering of query predicates in .expected 2023-11-15 10:09:54 +01:00
Rasmus Wriedt Larsen
49f5d38956 Merge pull request #14068 from RasmusWL/dataflow-config-refactor
Python: Use new dataflow API
2023-09-04 21:04:10 +02:00
yoff
da64ea40b9 Merge pull request #13782 from jorgectf/jorgectf/shlex-quote
Python: Add `shlex.quote` as `py/shell-command-constructed-from-input` sanitizer
2023-08-31 21:08:58 +02:00
Rasmus Wriedt Larsen
5ba8e102eb Python: Adopt tests to new DataflowQueryTest
Since we want to know the _sinks_ and not just the flow, we need to
expose the config as well :|
2023-08-28 15:31:08 +02:00
Rasmus Wriedt Larsen
889cb7a95b Python: Adopt tests to new DataflowQueryTest
Co-authored-by: Rasmus Lerchedahl Petersen <yoff@github.com>
2023-08-28 11:44:01 +02:00
Rasmus Wriedt Larsen
ca93f4d223 Python: Accept .expected changes 2023-08-11 10:36:05 +02:00
jorgectf
8f8c064632 Modify test 2023-07-24 17:50:22 +02:00
jorgectf
55648ac4de Add shlex.quote as sanitizer 2023-07-20 15:34:54 +02:00
Jeroen Ketema
dba4460526 Python: Update more inline expectation tests to use the paramterized module 2023-06-20 10:16:15 +02:00
erik-krogh
759854991a fix various nits based on feedback 2023-02-15 11:10:43 +01:00
erik-krogh
cf094c2f4f adjust which folders are seen as exported to remove an FP 2023-02-03 14:47:55 +01:00
erik-krogh
ef44cb86c2 remove FPs related to parameters that are meant to be commands 2023-02-03 14:47:55 +01:00
erik-krogh
e9ebba3350 assume shell=False for subprocess calls, fixes FPs in e.g. youtube-dl 2023-02-03 14:47:55 +01:00
erik-krogh
d228cf0e7b use more API-nodes to model subprocess.run (and friends) 2023-02-03 14:47:55 +01:00
erik-krogh
bce83bfc4e add failing test for indirectly setting the shell=true flag for subprocess.run 2023-02-03 14:47:55 +01:00
erik-krogh
0a2c7d062c add Fabric test, and add tracking of the shell flag in Fabric 2023-02-03 14:47:55 +01:00
erik-krogh
6bbc4f4a48 add more tests 2023-02-03 14:47:55 +01:00
erik-krogh
33c506d7fe add minimal test for Array join as a sink, and learn that the order is flipped compared to JS. Thanks Copilot! 2023-02-03 14:47:55 +01:00
erik-krogh
5bddfc0d79 add test for f-strings as sink 2023-02-03 14:47:55 +01:00
erik-krogh
47a06d2824 add library inputs as a source, and get minimal test to work 2023-02-03 14:47:55 +01:00