Commit Graph

573 Commits

Author SHA1 Message Date
Copilot
e68f2fd717 Python: add new shared-CFG-backed control flow graph
Preparatory refactor for the shared-CFG dataflow migration. Adds the
new Python CFG library additively, without changing any production
behaviour.

Library additions:

- semmle.python.controlflow.internal.AstNodeImpl — mediates between
  the Python AST and the shared codeql.controlflow.ControlFlowGraph
  signature. Wraps Python's Stmt/Expr/Scope/Pattern and adds two
  synthetic kinds of node (BlockStmt for body slots, intermediate
  nodes for multi-operand boolean expressions).

- semmle.python.controlflow.internal.Cfg — public facade
  re-exposing the same API surface as semmle/python/Flow.qll
  (ControlFlowNode, CallNode, BasicBlock, NameNode, DefinitionNode,
  CompareNode, ...), backed by the shared CFG.

- lib/printCfgNew.ql — debug/visualisation query for the new CFG.

- consistency-queries/CfgConsistency.ql — consistency query running
  the shared CFG's standard checks against Python.

Shared library:

- shared.controlflow.ControlFlowGraph — adds two defaulted
  getWhileElse / getForeachElse predicates to AstSig so Python can
  model while-else / for-else (no behavioural change for other
  languages).

Test additions:

- ControlFlow/bindings/* — annotation-driven SSA-binding tests for
  the new CFG (annassign, compound, comprehension, decorated,
  except_handler, imports, match_pattern, parameters, simple,
  type_params, walrus_starred, with_stmt, dead_under_no_raise).

- ControlFlow/store-load/* — basic store/load coverage.

- ControlFlow/evaluation-order/NewCfg*.ql — mirrors of the existing
  OldCfg evaluation-order self-validation suite, run against the
  new CFG via NewCfgImpl.qll.

- Minor extensions to existing test_if.py / test_boolean.py +
  cosmetic .expected churn on a handful of OldCfg tests.

No dataflow, SSA, or production query is migrated yet — that lands in
follow-up PRs. The new CFG library has zero callers in lib/ and src/.

Verified by:
- All lib + src + consistency-queries compile clean (367 queries).
- All 56 ControlFlow library-tests pass.
- All 474 dataflow + PointsTo library-tests + consistency tests pass.
- syntax_error/CONSISTENCY/CfgConsistency passes.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-06-02 14:09:28 +00:00
yoff
0b473e3763 Python: deprecate Function.getAReturnValueFlowNode() and rewrite internal callers
Follow-up to the getAFlowNode deprecation in the same PR: same AST→legacy-CFG
bridge pattern. The 11 internal call sites (across objects/, types/,
frameworks/, and TypeTrackingImpl) are rewritten to bind a `Return ret`
explicitly, then constrain via `ret.getScope() = f and n.getNode() = ret.getValue()`.

The predicate itself is preserved with a deprecation note so external
users do not experience churn.

Semantic noop.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-06-02 08:41:44 +00:00
Copilot
a13dfaa44f Python: deprecate AstNode.getAFlowNode() and rewrite internal callers
Preparatory refactor for the shared-CFG dataflow migration.

Deprecates the AstNode.getAFlowNode() cached predicate on the public
Python QL API and rewrites all ~140 internal callers across lib/, src/,
test/, and tools/ from `expr.getAFlowNode() = cfgNode` to
`cfgNode.getNode() = expr`, using ControlFlowNode.getNode() which
already exists in Flow.qll.

The predicate itself is preserved (with a deprecation note pointing at
the new pattern) so external users do not experience churn — they can
migrate at their own pace and the AST/CFG hierarchies still get the
intended untangling once the deprecation eventually elapses.

Semantic noop verified by:
- All 361 lib/ + src/ queries compile clean.
- All 122 ControlFlow + PointsTo library-tests pass.
- All 64 dataflow library-tests pass.
- All 113 Variables/Exceptions/Expressions/Statements/Functions/Imports/
  Security/CWE-798/ModificationOfParameterWithDefault query-tests pass.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-06-02 08:37:30 +00:00
yoff
5fb75ac987 Python: simplify decorator-detection predicates to pure AST match
The internal predicates that identify `@staticmethod`, `@classmethod` and
`@property` decorators previously required the decorator's `NameNode` to
satisfy `isGlobal()` (i.e. no SSA def reaches the decorator's name use).
That filter was correct but unnecessarily indirect: these three names
are builtins, and even when a class body redefines one, the class body
has not started executing at the decorator position, so Python uses the
builtin.

Match the decorator's AST `Name` directly instead, dropping the CFG/SSA
detour. The slight semantic change — `isGlobal()` would have rejected
module-level shadowing of these builtins — is negligible in practice
and explicitly documented in the change note.

`hasContextmanagerDecorator` and `hasOverloadDecorator` keep the
`NameNode.isGlobal()` check because their target names (`contextmanager`,
`overload`) are imported, not builtin, and local shadowing is a real
concern.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-06-01 14:04:43 +00:00
github-actions[bot]
8b6f969cdb Release preparation for version 2.25.6 2026-05-29 11:27:54 +00:00
Henry Mercer
9bc0c1b1ab Revert "Release preparation for version 2.25.6" 2026-05-29 12:13:50 +01:00
github-actions[bot]
44a914e40f Release preparation for version 2.25.6 2026-05-25 10:23:26 +00:00
Óscar San José
996e79131e Merge branch 'main' into post-release-prep/codeql-cli-2.25.5 2026-05-22 16:32:30 +02:00
github-actions[bot]
e38616a2ef Release preparation for version 2.25.5 2026-05-18 12:05:32 +00:00
Geoffrey White
a4b2c0f6fd Update change notes (Copilot's suggestions). 2026-05-15 09:24:29 +01:00
Geoffrey White
59dbd68a5e Add change notes. 2026-05-14 14:46:05 +01:00
github-actions[bot]
88e1d86c27 Release preparation for version 2.25.4 2026-05-05 09:34:30 +00:00
Owen Mansel-Chan
6efb21314a Merge pull request #21523 from owen-mc/docs/mad/barriers
Document models-as-data barriers and barrier guards and add change notes
2026-04-21 13:49:19 +01:00
Michael B. Gale
58e9bad0a0 Merge pull request #21737 from github/post-release-prep/codeql-cli-2.25.3
Post-release preparation for codeql-cli-2.25.3
2026-04-21 11:48:30 +02:00
Taus
b108e173a5 Merge pull request #21695 from github/tausbn/python-add-support-for-pep-798
Python: Add support for PEP-798
2026-04-20 15:01:01 +02:00
github-actions[bot]
c861d99802 Release preparation for version 2.25.3 2026-04-20 09:27:23 +00:00
Owen Mansel-Chan
8f17b73796 Fix link formatting in change notes 2026-04-14 15:27:37 +01:00
Owen Mansel-Chan
c86ba38a4e Add change notes 2026-04-14 15:27:31 +01:00
Henry Mercer
43c9b95e6f Merge branch 'main' into post-release-prep/codeql-cli-2.25.2 2026-04-14 13:56:52 +01:00
Taus
15790aa00c Python: Add change note 2026-04-14 13:27:31 +02:00
Taus
86020d9eed Python: Add change note 2026-04-10 14:43:30 +00:00
github-actions[bot]
4fe2f6d2b4 Release preparation for version 2.25.2 2026-04-06 10:30:38 +00:00
Óscar San José
59eec7ffa2 Merge branch 'main' of https://github.com/github/codeql into post-release-prep/codeql-cli-2.25.1 2026-03-30 10:51:12 +02:00
github-actions[bot]
fb011842c9 Release preparation for version 2.25.1 2026-03-25 23:43:06 +00:00
github-actions[bot]
8cf0954796 Release preparation for version 2.25.1 2026-03-25 08:28:30 +00:00
Óscar San José
2139b97628 Merge branch 'main' into post-release-prep/codeql-cli-2.25.0 2026-03-19 13:07:00 +01:00
Taus
a99b3f2c3b Merge pull request #21459 from github/tausbn/python-fix-missing-relative-imports
Python: Fix resolution of relative imports from namespace packages
2026-03-16 14:59:44 +01:00
github-actions[bot]
d6055754b6 Release preparation for version 2.25.0 2026-03-16 12:15:34 +00:00
Taus
3ee369b710 Python: Add change note 2026-03-12 13:29:24 +00:00
Taus
5d74ad5bc6 Merge pull request #21419 from github/tausbn/python-improve-overloaded-method-resolution
Python: Improve modelling of overloaded methods
2026-03-09 16:25:05 +01:00
Óscar San José
3b9eba2afc Merge branch 'main' of https://github.com/github/codeql into oscarsj/merge-back-rc-3.21 2026-03-06 16:20:36 +01:00
Taus
66ca10c338 Python: Add change note 2026-03-05 22:20:03 +00:00
Owen Mansel-Chan
c82f75604a Add change notes 2026-03-05 10:34:30 +00:00
github-actions[bot]
7795badd18 Release preparation for version 2.24.3 2026-03-02 13:23:40 +00:00
yoff
600f585a31 Merge pull request #21296 from yoff/python/bool-comparison-guards
Python: Handle guards being compared to boolean literals
2026-02-26 21:13:51 +01:00
Taus
6bfb1e1fae Merge pull request #21344 from github/tausbn/python-remove-points-to-from-metrics-libraries
Python: Remove points-to from metrics library
2026-02-24 15:55:16 +01:00
Taus
f107235db2 Update change note 2026-02-24 15:08:36 +01:00
yoff
7df44f9418 python: add change note 2026-02-24 10:00:22 +01:00
Jon Janego
e14b4f1c5c Merge branch 'main' into codeql-spark-run-22317536589 2026-02-23 11:52:17 -06:00
Jon Janego
79ac95d8a8 Fix syntax error with '=' in format specifier 2026-02-23 11:50:03 -06:00
Taus
480ae619e6 Merge pull request #21116 from github/tausbn/python-add-dataflow-overlay-annotations
Add `overlay[local]` annotations
2026-02-21 13:44:09 +01:00
Taus
07099f17d6 Python: Add change note 2026-02-19 12:32:27 +00:00
Taus
6b6d8862b0 Merge pull request #21288 from microsoft/azure_python_sanitizer_upstream2
Azure python sanitizer upstream2
2026-02-18 14:59:59 +01:00
Taus
3d4785f29f Python: Add change note 2026-02-18 12:51:35 +00:00
Ben Rodes
a1eaf42cbf Update python/ql/lib/change-notes/2026-02-09-ssrf_test_case_cleanup_and_new_ssrf_barriers.md
Co-authored-by: Owen Mansel-Chan <62447351+owen-mc@users.noreply.github.com>
2026-02-17 13:05:51 -05:00
github-actions[bot]
ef04f927fb Release preparation for version 2.24.2 2026-02-16 13:29:25 +00:00
REDMOND\brodes
23bab81855 Added change log 2026-02-09 13:22:35 -05:00
REDMOND\brodes
df54459552 Restore prior PR change log (accidentally removed) 2026-02-09 13:19:02 -05:00
Ben Rodes
85ae4045c5 Merge branch 'main' into azure_python_sanitizer_upstream2 2026-02-09 13:12:38 -05:00
yoff
5ad42f8bcc Merge pull request #20563 from microsoft/azure_python_sdk_url_summary_upstream
Azure python sdk url summary upstream
2026-02-09 18:34:36 +01:00