Commit Graph

3254 Commits

Author SHA1 Message Date
Copilot
250b63c216 Python: unify Py::BoolExpr handling via TBoolExprPair
Previously a Py::BoolExpr appeared in two newtype branches: as TExpr(be)
(the outermost pair) and TBoolExprPair(be, i) for inner pairs of 3+
operand expressions. This forced BinaryExpr/LogicalAndExpr/LogicalOrExpr
to disjoin two cases, and the synthetic-pair handling spanned multiple
layers.

Restrict TExpr to non-BoolExpr Py::Expr, and extend TBoolExprPair to
cover every operand pair (index 0..n-2). Now every Py::BoolExpr is
represented uniformly as TBoolExprPair(_, 0) for the whole expression
and TBoolExprPair(_, i) for inner pairs.

Extend AstNode.asExpr() to also recover the underlying Py::BoolExpr
from TBoolExprPair(_, 0). This makes asExpr() the inverse of
construction: every 'result = TExpr(e)' turns into 'result.asExpr() = e',
which works uniformly for BoolExprs and non-BoolExprs alike.

Consequences:

- BinaryExpr now extends TBoolExprPair directly with a single uniform
  rule for left/right operands.
- LogicalAndExpr/LogicalOrExpr are one-line char preds via
  getBoolExpr().
- The private BoolExprPair wrapper class folds into BinaryExpr.
- 60+ leaf wrappers now read 'result.asExpr() = py_expr' instead of
  'result = TExpr(py_expr)'.

No behaviour change: all 24 NewCfg evaluation-order tests pass; all
11 shared-CFG consistency queries report 0 violations on CPython.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-07 12:04:54 +00:00
Copilot
77e36d3cfa Python: merge T*AstNode wrappers into matching public classes
Five of the six per-newtype-branch wrapper classes had a natural
public class corresponding to that branch:

  TStmtAstNode      -> Stmt        (TStmt subset; BlockStmt overrides for TBlockStmt)
  TExprAstNode      -> Expr        (TExpr subset; BoolExprPair overrides for TBoolExprPair)
  TScopeAstNode     -> Callable    (= TScope exactly)
  TPatternAstNode   -> Pattern     (= TPattern exactly)
  TBlockStmtAstNode -> BlockStmt   (= TBlockStmt exactly)

Move toString/getLocation/getEnclosingCallable onto these classes and
delete the wrappers.

The sixth wrapper (TBoolExprPair) has no exact public counterpart -
BinaryExpr is broader, including TExpr-branch BoolExprs - so it
remains as a small private class, renamed BoolExprPair.

No behaviour change: all 24 NewCfg evaluation-order tests pass; all
11 shared-CFG consistency queries report 0 violations on CPython.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-07 11:31:07 +00:00
Copilot
f2151fe232 Python: dispatch toString/getLocation/getEnclosingCallable per branch
Replace the three big disjunctive predicates on AstNode with empty
defaults plus per-newtype-branch override classes:

  AstNode.toString()              { none() }
  AstNode.getLocation()           { none() }
  AstNode.getEnclosingCallable()  { none() }

Six private subclasses (one per newtype branch — TStmt, TExpr,
TScope, TPattern, TBoolExprPair, TBlockStmt) override these with
the branch-specific implementation. This mirrors the per-class
dispatch already used for getChild.

No behaviour change: all 24 NewCfg evaluation-order tests pass and
all 11 shared-CFG consistency queries still report 0 violations on
CPython.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-07 11:06:40 +00:00
Copilot
9781ee8d66 Python: adapt to new shared CFG signature
Main added two new requirements to AstSig:
- A 'Parameter' class with a 'getDefaultValue()' method, plus a
  'callableGetParameter(Callable, int)' predicate.
- A 'CallableContext' class in InputSig1, replacing the previous
  'CallableBodyPartContext'.

Add stub implementations: 'Parameter' is empty (none()) and
'callableGetParameter' returns nothing, mirroring Java's TODO. Rename
'CallableBodyPartContext = Void' to 'CallableContext = Void' in the
Python Input module.

NewCfg evaluation-order tests still pass at the 22/24 baseline; all
11 shared-CFG consistency queries still report 0 violations on
CPython.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-05 15:26:20 +00:00
Copilot
03b8e8fdde Python: refactor getChild into per-class OO dispatch
Replace the single ~240-line top-level getChild predicate with one
override per AST class. AstNode declares a default

  AstNode getChild(int index) { none() }

and each subclass with children overrides it (41 classes total).
The top-level predicate becomes a one-line dispatch:

  AstNode getChild(AstNode n, int index) { result = n.getChild(index) }

No behavioral change: NewCfg evaluation-order tests still pass at the
same 22/24 baseline, and all 11 shared-CFG consistency queries still
report 0 violations on CPython.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-05 15:21:43 +00:00
Copilot
93112b2b75 Python: include try-else in getChild for completion propagation
The shared CFG library propagates abrupt completions from child to
parent via getChild(parent, _) = child. Python's try.getElse() was
wired into normal step rules but not listed in getChild(TryStmt, ...),
so return/break/continue/raise statements occurring inside a try-else
block had no parent path and ended up as dead-end CFG nodes.

Add the else block at index -2 (alongside finally at -1). This affects
only completion propagation; the normal-flow CFG is unchanged because
TryStmt has explicit step rules.

Verified on a CPython database: all 11 shared-CFG consistency queries
now pass with 0 violations (deadEnd: 244 -> 0).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-05 15:21:43 +00:00
Copilot
76724c5391 Shared CFG: support for-else and while-else loops
Add two default predicates to AstSig:

  default AstNode getWhileElse(WhileStmt loop) { none() }
  default AstNode getForeachElse(ForeachStmt loop) { none() }

When defined, the explicit-step rules for While/Do and Foreach
route the loop's normal-completion exits through the else block
before reaching the after-loop node:

  - WhileStmt: after-false condition -> before-else -> after-while
    (instead of directly after-while).
  - ForeachStmt: after-collection [empty] and the LoopHeader exit
    are both routed through before-else -> after-foreach.

Python's Ast module overrides the predicates to return the
synthetic BlockStmt for the orelse slot, replacing the previous
customisations in Input::step. This eliminates parallel direct
successors emitted by the previous Python-side step additions
(verified: multipleSuccessors on a CPython database goes from
1340 to 0).

Java and C# CFG tests are unaffected.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-05 15:21:43 +00:00
Copilot
edfe91832b Python: compact-renumber FunctionExpr/Lambda defaults
`Args.getDefault(int)` and `Args.getKwDefault(int)` are indexed by
argument position (with gaps for args without defaults), not by
default position. The CFG `getChild` predicate for FunctionDefExpr
and LambdaExpr therefore had gaps at low indices and collisions
where defaults and kwdefaults overlapped, producing parallel
edges before the FunctionExpr.

Use `rank` to compact-renumber `getDefault(n)` and `getKwDefault(n)`
in source order. Verified on a CPython database: removes ~536
`multipleSuccessors` consistency results (1340 -> 804); the rest are
`for/else` and `while/else`.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-05 15:21:43 +00:00
Copilot
58cda914db Python: collapse two-layer AstNodeImpl into a single Ast module
Merge the previous `Ast` and `AstSigImpl` modules into a single
`module Ast implements AstSig<Py::Location>`. Classes now use the
signature names (IfStmt, WhileStmt, ForeachStmt, etc.) and signature
predicates (getCondition, getThen, getElse, etc.) directly, with no
intermediate renaming layer.

Drop the TStmtListNode newtype branch entirely. Replace it with a
synthetic TBlockStmt(parent, slot) keyed by a parent AST node and a
slot label string ('body', 'orelse', 'finally'). Py::StmtList no
longer appears in the newtype; the BlockStmt class provides indexed
access to the underlying body items via getStmt(n).

All 22 of 24 evaluation-order tests still pass; the same 2
comprehension-related failures predate this refactor.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-05 15:21:43 +00:00
yoff
3b0abad701 Python: add pattern nodes
Co-authored-by: Copilot <copilot@github.com>
2026-05-05 15:21:42 +00:00
Taus
68b3d57563 Cleanup, printCFG
Co-authored-by: yoff <yoff@github.com>
2026-05-05 15:21:42 +00:00
Taus
e3155ea544 Python: Handle dict unpacking in calls
Co-authored-by: yoff <yoff@github.com>
2026-05-05 15:21:42 +00:00
Taus
04b8c4bc7e Python: Fix exception issue
Co-authored-by: yoff <yoff@github.com>
2026-05-05 15:21:42 +00:00
Taus
f85b532bb3 Python: Fix match
Co-authored-by: yoff <yoff@github.com>
2026-05-05 15:21:42 +00:00
Taus
0e1f1d9f09 Python: Support match
Co-authored-by: yoff <yoff@github.com>
2026-05-05 15:21:42 +00:00
Taus
53da31bd15 Python: More nodes
Not entirely sure about the `else:` blocks.

Co-authored-by: yoff <yoff@github.com>
2026-05-05 15:21:42 +00:00
Taus
1f82dbc583 Python: Comprehensions
Co-authored-by: yoff <yoff@github.com>
2026-05-05 15:21:41 +00:00
Taus
b229066891 Python: Add with
Co-authored-by: yoff <yoff@github.com>
2026-05-05 15:21:41 +00:00
Taus
0acbb12fb9 Python: More simple statements
Co-authored-by: yoff <yoff@github.com>
2026-05-05 15:21:41 +00:00
Taus
542efce4a6 Python: assignments
Co-authored-by: yoff <yoff@github.com>
2026-05-05 15:21:41 +00:00
Taus
2db400aebd Python: Attributes
Co-authored-by: yoff <yoff@github.com>
2026-05-05 15:21:41 +00:00
Taus
66bbb60614 Python: Function calls
Co-authored-by: yoff <yoff@github.com>
2026-05-05 15:21:41 +00:00
Taus
971beb2d89 Python: Assert statements
Co-authored-by: yoff <yoff@github.com>
2026-05-05 15:21:41 +00:00
Taus
ea204ac75f Python: Support various literals
Co-authored-by: yoff <yoff@github.com>
2026-05-05 15:21:41 +00:00
Taus
dc0344e2fc Python: More AstNodeImpl improvements
Co-authored-by: yoff <yoff@github.com>
2026-05-05 15:21:41 +00:00
Taus
9974584102 Python: Instantiate CFG module fully
Co-authored-by: yoff <yoff@github.com>
2026-05-05 15:21:40 +00:00
Taus
6086b999f6 Python: Use fields everywhere in new AST classes
Co-authored-by: yoff <yoff@github.com>
2026-05-05 15:21:40 +00:00
Taus
d62e116fc2 Python: First stab at shared control-flow 2026-05-05 15:21:40 +00:00
Josef Svenningsson
68be006a29 Merge pull request #21641 from github/josefs/promptInjectionImprovements
Improve prompt inject for Python
2026-04-29 11:23:52 +01:00
Josef Svenningsson
bb18bb084c Improve prompt inject for Python 2026-04-28 18:24:16 +01:00
Owen Mansel-Chan
6efb21314a Merge pull request #21523 from owen-mc/docs/mad/barriers
Document models-as-data barriers and barrier guards and add change notes
2026-04-21 13:49:19 +01:00
Michael B. Gale
58e9bad0a0 Merge pull request #21737 from github/post-release-prep/codeql-cli-2.25.3
Post-release preparation for codeql-cli-2.25.3
2026-04-21 11:48:30 +02:00
Taus
b108e173a5 Merge pull request #21695 from github/tausbn/python-add-support-for-pep-798
Python: Add support for PEP-798
2026-04-20 15:01:01 +02:00
github-actions[bot]
a0bab539bb Post-release preparation for codeql-cli-2.25.3 2026-04-20 12:40:34 +00:00
github-actions[bot]
c861d99802 Release preparation for version 2.25.3 2026-04-20 09:27:23 +00:00
Owen Mansel-Chan
8f17b73796 Fix link formatting in change notes 2026-04-14 15:27:37 +01:00
Owen Mansel-Chan
c86ba38a4e Add change notes 2026-04-14 15:27:31 +01:00
Henry Mercer
43c9b95e6f Merge branch 'main' into post-release-prep/codeql-cli-2.25.2 2026-04-14 13:56:52 +01:00
Taus
15790aa00c Python: Add change note 2026-04-14 13:27:31 +02:00
Taus
c748fdf8ee Merge pull request #21694 from github/tausbn/python-add-support-for-pep-810
Python: Add support for PEP 810
2026-04-14 13:27:08 +02:00
Owen Mansel-Chan
7458674470 Merge pull request #21584 from owen-mc/shared/update-mad-comments
Shared: update code comments explaining models-as-data format to include barriers and barrier guards
2026-04-14 09:30:28 +01:00
Taus
86020d9eed Python: Add change note 2026-04-10 14:43:30 +00:00
Taus
1ddfed6b6b Python: Add QL support for lazy imports
Adds a new `isLazy` predicate to the relevant classes, and adds the
relevant dbscheme (and up/downgrade) changes. On upgrades we do nothing,
and on downgrades we remove the `is_lazy` bits.
2026-04-10 14:25:08 +00:00
Taus
16683aee0e Merge pull request #21590 from github/tausbn/python-improve-bind-all-interfaces-query
Python: Improve "bind all interfaces" query
2026-04-07 17:59:48 +02:00
github-actions[bot]
242090e0ac Post-release preparation for codeql-cli-2.25.2 2026-04-06 13:49:20 +00:00
github-actions[bot]
4fe2f6d2b4 Release preparation for version 2.25.2 2026-04-06 10:30:38 +00:00
Óscar San José
59eec7ffa2 Merge branch 'main' of https://github.com/github/codeql into post-release-prep/codeql-cli-2.25.1 2026-03-30 10:51:12 +02:00
github-actions[bot]
ce6e6d5db3 Post-release preparation for codeql-cli-2.25.1 2026-03-30 08:43:48 +00:00
Owen Mansel-Chan
37aac05964 Replace branch with acceptingValue 2026-03-27 22:39:10 +00:00
Owen Mansel-Chan
10fddc7b96 Add barriers and barrier guards to MaD format explanations 2026-03-27 09:47:24 +00:00