Five of the six per-newtype-branch wrapper classes had a natural
public class corresponding to that branch:
TStmtAstNode -> Stmt (TStmt subset; BlockStmt overrides for TBlockStmt)
TExprAstNode -> Expr (TExpr subset; BoolExprPair overrides for TBoolExprPair)
TScopeAstNode -> Callable (= TScope exactly)
TPatternAstNode -> Pattern (= TPattern exactly)
TBlockStmtAstNode -> BlockStmt (= TBlockStmt exactly)
Move toString/getLocation/getEnclosingCallable onto these classes and
delete the wrappers.
The sixth wrapper (TBoolExprPair) has no exact public counterpart -
BinaryExpr is broader, including TExpr-branch BoolExprs - so it
remains as a small private class, renamed BoolExprPair.
No behaviour change: all 24 NewCfg evaluation-order tests pass; all
11 shared-CFG consistency queries report 0 violations on CPython.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Replace the three big disjunctive predicates on AstNode with empty
defaults plus per-newtype-branch override classes:
AstNode.toString() { none() }
AstNode.getLocation() { none() }
AstNode.getEnclosingCallable() { none() }
Six private subclasses (one per newtype branch — TStmt, TExpr,
TScope, TPattern, TBoolExprPair, TBlockStmt) override these with
the branch-specific implementation. This mirrors the per-class
dispatch already used for getChild.
No behaviour change: all 24 NewCfg evaluation-order tests pass and
all 11 shared-CFG consistency queries still report 0 violations on
CPython.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Main added two new requirements to AstSig:
- A 'Parameter' class with a 'getDefaultValue()' method, plus a
'callableGetParameter(Callable, int)' predicate.
- A 'CallableContext' class in InputSig1, replacing the previous
'CallableBodyPartContext'.
Add stub implementations: 'Parameter' is empty (none()) and
'callableGetParameter' returns nothing, mirroring Java's TODO. Rename
'CallableBodyPartContext = Void' to 'CallableContext = Void' in the
Python Input module.
NewCfg evaluation-order tests still pass at the 22/24 baseline; all
11 shared-CFG consistency queries still report 0 violations on
CPython.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Replace the single ~240-line top-level getChild predicate with one
override per AST class. AstNode declares a default
AstNode getChild(int index) { none() }
and each subclass with children overrides it (41 classes total).
The top-level predicate becomes a one-line dispatch:
AstNode getChild(AstNode n, int index) { result = n.getChild(index) }
No behavioral change: NewCfg evaluation-order tests still pass at the
same 22/24 baseline, and all 11 shared-CFG consistency queries still
report 0 violations on CPython.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The shared CFG library propagates abrupt completions from child to
parent via getChild(parent, _) = child. Python's try.getElse() was
wired into normal step rules but not listed in getChild(TryStmt, ...),
so return/break/continue/raise statements occurring inside a try-else
block had no parent path and ended up as dead-end CFG nodes.
Add the else block at index -2 (alongside finally at -1). This affects
only completion propagation; the normal-flow CFG is unchanged because
TryStmt has explicit step rules.
Verified on a CPython database: all 11 shared-CFG consistency queries
now pass with 0 violations (deadEnd: 244 -> 0).
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add two default predicates to AstSig:
default AstNode getWhileElse(WhileStmt loop) { none() }
default AstNode getForeachElse(ForeachStmt loop) { none() }
When defined, the explicit-step rules for While/Do and Foreach
route the loop's normal-completion exits through the else block
before reaching the after-loop node:
- WhileStmt: after-false condition -> before-else -> after-while
(instead of directly after-while).
- ForeachStmt: after-collection [empty] and the LoopHeader exit
are both routed through before-else -> after-foreach.
Python's Ast module overrides the predicates to return the
synthetic BlockStmt for the orelse slot, replacing the previous
customisations in Input::step. This eliminates parallel direct
successors emitted by the previous Python-side step additions
(verified: multipleSuccessors on a CPython database goes from
1340 to 0).
Java and C# CFG tests are unaffected.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
`Args.getDefault(int)` and `Args.getKwDefault(int)` are indexed by
argument position (with gaps for args without defaults), not by
default position. The CFG `getChild` predicate for FunctionDefExpr
and LambdaExpr therefore had gaps at low indices and collisions
where defaults and kwdefaults overlapped, producing parallel
edges before the FunctionExpr.
Use `rank` to compact-renumber `getDefault(n)` and `getKwDefault(n)`
in source order. Verified on a CPython database: removes ~536
`multipleSuccessors` consistency results (1340 -> 804); the rest are
`for/else` and `while/else`.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Merge the previous `Ast` and `AstSigImpl` modules into a single
`module Ast implements AstSig<Py::Location>`. Classes now use the
signature names (IfStmt, WhileStmt, ForeachStmt, etc.) and signature
predicates (getCondition, getThen, getElse, etc.) directly, with no
intermediate renaming layer.
Drop the TStmtListNode newtype branch entirely. Replace it with a
synthetic TBlockStmt(parent, slot) keyed by a parent AST node and a
slot label string ('body', 'orelse', 'finally'). Py::StmtList no
longer appears in the newtype; the BlockStmt class provides indexed
access to the underlying body items via getStmt(n).
All 22 of 24 evaluation-order tests still pass; the same 2
comprehension-related failures predate this refactor.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Adds a new `isLazy` predicate to the relevant classes, and adds the
relevant dbscheme (and up/downgrade) changes. On upgrades we do nothing,
and on downgrades we remove the `is_lazy` bits.
This takes care of most of the false negatives from the preceding
commit.
Additionally, we add models for some known wrappers of `socket.socket`
from the `gevent` and `eventlet` packages.
For module-level metaclass declarations, we now also check that the
right hand side in a `__metaclass__ = type` assignment is in fact the
built-in `type`.
These could arguably be moved to `Class` itself, but for now I'm
choosing to limit the changes to the `DuckTyping` module (until we
decide on a proper API).
This module (which for convenience currently resides inside
`DataFlowDispatch`, but this may change later) contains convenience
predicates for bridging the gap between the data-flow layer and the old
points-to analysis.
The fix may look a bit obscure, so here's what's going on.
When we see `from . import helper`, we create an `ImportExpr` with level
equal to 1 (corresponding to the number of dots). To resolve such
imports, we compute the name of the enclosing package, as part of
`ImportExpr.qualifiedTopName()`. For this form of import expression, it
is equivalent to `this.getEnclosingModule().getPackageName()`. But
`qualifiedTopName` requires that `valid_module_name` holds for its
result, and this was _not_ the case for namespace packages.
To fix this, we extend `valid_module_name` to include the module names
of _any_ folder, not just regular package (which are the ones where
there's a `__init__.py` in the folder). Note that this doesn't simply
include all folders -- only the ones that result in valid module names
in Python.
Adds `hasOverloadDecorator` as a predicate on functions. It looks for
decorators called `overload` or `something.overload` (usually
`typing.overload` or `t.overload`). These are then filtered out in the
predicates that (approximate) resolving methods according to the MRO.
As the test introduced in the previous commit shows, this removes the
spurious resolutions we had before.
