Commit Graph

2552 Commits

Author SHA1 Message Date
Copilot
76724c5391 Shared CFG: support for-else and while-else loops
Add two default predicates to AstSig:

  default AstNode getWhileElse(WhileStmt loop) { none() }
  default AstNode getForeachElse(ForeachStmt loop) { none() }

When defined, the explicit-step rules for While/Do and Foreach
route the loop's normal-completion exits through the else block
before reaching the after-loop node:

  - WhileStmt: after-false condition -> before-else -> after-while
    (instead of directly after-while).
  - ForeachStmt: after-collection [empty] and the LoopHeader exit
    are both routed through before-else -> after-foreach.

Python's Ast module overrides the predicates to return the
synthetic BlockStmt for the orelse slot, replacing the previous
customisations in Input::step. This eliminates parallel direct
successors emitted by the previous Python-side step additions
(verified: multipleSuccessors on a CPython database goes from
1340 to 0).

Java and C# CFG tests are unaffected.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-05 15:21:43 +00:00
Copilot
edfe91832b Python: compact-renumber FunctionExpr/Lambda defaults
`Args.getDefault(int)` and `Args.getKwDefault(int)` are indexed by
argument position (with gaps for args without defaults), not by
default position. The CFG `getChild` predicate for FunctionDefExpr
and LambdaExpr therefore had gaps at low indices and collisions
where defaults and kwdefaults overlapped, producing parallel
edges before the FunctionExpr.

Use `rank` to compact-renumber `getDefault(n)` and `getKwDefault(n)`
in source order. Verified on a CPython database: removes ~536
`multipleSuccessors` consistency results (1340 -> 804); the rest are
`for/else` and `while/else`.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-05 15:21:43 +00:00
Copilot
58cda914db Python: collapse two-layer AstNodeImpl into a single Ast module
Merge the previous `Ast` and `AstSigImpl` modules into a single
`module Ast implements AstSig<Py::Location>`. Classes now use the
signature names (IfStmt, WhileStmt, ForeachStmt, etc.) and signature
predicates (getCondition, getThen, getElse, etc.) directly, with no
intermediate renaming layer.

Drop the TStmtListNode newtype branch entirely. Replace it with a
synthetic TBlockStmt(parent, slot) keyed by a parent AST node and a
slot label string ('body', 'orelse', 'finally'). Py::StmtList no
longer appears in the newtype; the BlockStmt class provides indexed
access to the underlying body items via getStmt(n).

All 22 of 24 evaluation-order tests still pass; the same 2
comprehension-related failures predate this refactor.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-05 15:21:43 +00:00
yoff
3b0abad701 Python: add pattern nodes
Co-authored-by: Copilot <copilot@github.com>
2026-05-05 15:21:42 +00:00
Taus
68b3d57563 Cleanup, printCFG
Co-authored-by: yoff <yoff@github.com>
2026-05-05 15:21:42 +00:00
Taus
e3155ea544 Python: Handle dict unpacking in calls
Co-authored-by: yoff <yoff@github.com>
2026-05-05 15:21:42 +00:00
Taus
04b8c4bc7e Python: Fix exception issue
Co-authored-by: yoff <yoff@github.com>
2026-05-05 15:21:42 +00:00
Taus
f85b532bb3 Python: Fix match
Co-authored-by: yoff <yoff@github.com>
2026-05-05 15:21:42 +00:00
Taus
0e1f1d9f09 Python: Support match
Co-authored-by: yoff <yoff@github.com>
2026-05-05 15:21:42 +00:00
Taus
53da31bd15 Python: More nodes
Not entirely sure about the `else:` blocks.

Co-authored-by: yoff <yoff@github.com>
2026-05-05 15:21:42 +00:00
Taus
1f82dbc583 Python: Comprehensions
Co-authored-by: yoff <yoff@github.com>
2026-05-05 15:21:41 +00:00
Taus
b229066891 Python: Add with
Co-authored-by: yoff <yoff@github.com>
2026-05-05 15:21:41 +00:00
Taus
0acbb12fb9 Python: More simple statements
Co-authored-by: yoff <yoff@github.com>
2026-05-05 15:21:41 +00:00
Taus
542efce4a6 Python: assignments
Co-authored-by: yoff <yoff@github.com>
2026-05-05 15:21:41 +00:00
Taus
2db400aebd Python: Attributes
Co-authored-by: yoff <yoff@github.com>
2026-05-05 15:21:41 +00:00
Taus
66bbb60614 Python: Function calls
Co-authored-by: yoff <yoff@github.com>
2026-05-05 15:21:41 +00:00
Taus
971beb2d89 Python: Assert statements
Co-authored-by: yoff <yoff@github.com>
2026-05-05 15:21:41 +00:00
Taus
ea204ac75f Python: Support various literals
Co-authored-by: yoff <yoff@github.com>
2026-05-05 15:21:41 +00:00
Taus
dc0344e2fc Python: More AstNodeImpl improvements
Co-authored-by: yoff <yoff@github.com>
2026-05-05 15:21:41 +00:00
Taus
9974584102 Python: Instantiate CFG module fully
Co-authored-by: yoff <yoff@github.com>
2026-05-05 15:21:40 +00:00
Taus
6086b999f6 Python: Use fields everywhere in new AST classes
Co-authored-by: yoff <yoff@github.com>
2026-05-05 15:21:40 +00:00
Taus
d62e116fc2 Python: First stab at shared control-flow 2026-05-05 15:21:40 +00:00
Josef Svenningsson
68be006a29 Merge pull request #21641 from github/josefs/promptInjectionImprovements
Improve prompt inject for Python
2026-04-29 11:23:52 +01:00
Josef Svenningsson
bb18bb084c Improve prompt inject for Python 2026-04-28 18:24:16 +01:00
Taus
c748fdf8ee Merge pull request #21694 from github/tausbn/python-add-support-for-pep-810
Python: Add support for PEP 810
2026-04-14 13:27:08 +02:00
Owen Mansel-Chan
7458674470 Merge pull request #21584 from owen-mc/shared/update-mad-comments
Shared: update code comments explaining models-as-data format to include barriers and barrier guards
2026-04-14 09:30:28 +01:00
Taus
1ddfed6b6b Python: Add QL support for lazy imports
Adds a new `isLazy` predicate to the relevant classes, and adds the
relevant dbscheme (and up/downgrade) changes. On upgrades we do nothing,
and on downgrades we remove the `is_lazy` bits.
2026-04-10 14:25:08 +00:00
Taus
16683aee0e Merge pull request #21590 from github/tausbn/python-improve-bind-all-interfaces-query
Python: Improve "bind all interfaces" query
2026-04-07 17:59:48 +02:00
Owen Mansel-Chan
37aac05964 Replace branch with acceptingValue 2026-03-27 22:39:10 +00:00
Owen Mansel-Chan
10fddc7b96 Add barriers and barrier guards to MaD format explanations 2026-03-27 09:47:24 +00:00
Taus
c439fc5d45 Python: Replace type tracking with global data-flow
This takes care of most of the false negatives from the preceding
commit.

Additionally, we add models for some known wrappers of `socket.socket`
from the `gevent` and `eventlet` packages.
2026-03-26 15:35:33 +00:00
Taus
ac48eca916 Python: Use cls.getMethod instead of getName 2026-03-23 15:26:00 +00:00
Taus
93e35661e6 Python: Make isNewType more precise
For module-level metaclass declarations, we now also check that the
right hand side in a `__metaclass__ = type` assignment is in fact the
built-in `type`.
2026-03-23 15:22:24 +00:00
Taus
a276f721f7 Python: Add ternary overridesMethod
This one also allows easy access to the method being overridden and the
class on which it resides. This let's us simplify DocStrings.ql
accordingly.
2026-03-23 15:21:27 +00:00
Taus
56c83e250e Python: Make comment more precise
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-03-23 15:09:27 +01:00
Taus
50b3b7ee1f Python: Add DuckTyping::hasUnreliableMro
Primarily used to filter out false positives in cases where our MRO
approximation may be wrong.
2026-03-20 13:30:29 +00:00
Taus
c04b615a07 Python: Extend DuckTyping module
Adds `overridesMethod` and `isPropertyAccessor`.
2026-03-20 13:28:45 +00:00
Taus
b57e92164c Python: Add declares/getAttribute API
These could arguably be moved to `Class` itself, but for now I'm
choosing to limit the changes to the `DuckTyping` module (until we
decide on a proper API).
2026-03-20 13:28:45 +00:00
Taus
cd92162920 Python: Add DuckTyping::isNewStyle
Approximates the behaviour of `Types::isNewStyle` but without depending
on points-to
2026-03-20 13:28:45 +00:00
Taus
33ed6034f6 Python: Introduce DuckTyping module
This module (which for convenience currently resides inside
`DataFlowDispatch`, but this may change later) contains convenience
predicates for bridging the gap between the data-flow layer and the old
points-to analysis.
2026-03-20 13:28:44 +00:00
Taus
f4841e1f39 Python: Use API graphs instead of points-to for simple built-ins
Also extends the list of known built-ins slightly, to add some that were
missing.
2026-03-20 13:28:44 +00:00
Taus
a99b3f2c3b Merge pull request #21459 from github/tausbn/python-fix-missing-relative-imports
Python: Fix resolution of relative imports from namespace packages
2026-03-16 14:59:44 +01:00
Taus
e16bb226c0 Python: Fix resolution of relative imports from namespace packages
The fix may look a bit obscure, so here's what's going on.

When we see `from . import helper`, we create an `ImportExpr` with level
equal to 1 (corresponding to the number of dots). To resolve such
imports, we compute the name of the enclosing package, as part of
`ImportExpr.qualifiedTopName()`. For this form of import expression, it
is equivalent to `this.getEnclosingModule().getPackageName()`. But
`qualifiedTopName` requires that `valid_module_name` holds for its
result, and this was _not_ the case for namespace packages.

To fix this, we extend `valid_module_name` to include the module names
of _any_ folder, not just regular package (which are the ones where
there's a `__init__.py` in the folder). Note that this doesn't simply
include all folders -- only the ones that result in valid module names
in Python.
2026-03-12 13:29:23 +00:00
Taus
f2bad1e6e1 Python: Improve docstring and make predicate private 2026-03-09 13:41:38 +00:00
Taus
fa61f6f3df Python: Model @typing.overload in method resolution
Adds `hasOverloadDecorator` as a predicate on functions. It looks for
decorators called `overload` or `something.overload` (usually
`typing.overload` or `t.overload`). These are then filtered out in the
predicates that (approximate) resolving methods according to the MRO.

As the test introduced in the previous commit shows, this removes the
spurious resolutions we had before.
2026-03-05 22:20:03 +00:00
yoff
600f585a31 Merge pull request #21296 from yoff/python/bool-comparison-guards
Python: Handle guards being compared to boolean literals
2026-02-26 21:13:51 +01:00
yoff
89e5a9bd72 Update python/ql/lib/semmle/python/dataflow/new/internal/DataFlowPublic.qll
Co-authored-by: Taus <tausbn@github.com>
2026-02-26 13:14:26 +01:00
yoff
cfbae50845 Python: convert barrier guard to MaD 2026-02-26 13:12:34 +01:00
yoff
9b9c9304c7 Python: simplify logic, suggested in review 2026-02-25 18:16:38 +01:00
yoff
c4f8748a42 Python: simplify barrier guard 2026-02-25 18:03:40 +01:00