Compare commits

..

9 Commits

Author SHA1 Message Date
Copilot
b2ff09f70a Python: add new shared-SSA-backed SSA adapter
Preparatory refactor for the shared-CFG dataflow migration. Adds the
new Python SSA adapter additively, without changing any production
behaviour.

Library additions:

- semmle.python.dataflow.new.internal.SsaImpl — Python SSA
  implementation built on the new (shared) CFG. Mirrors the Java SSA
  adapter (java/ql/lib/semmle/code/java/dataflow/internal/SsaImpl.qll):
  an InputSig is defined in terms of positional (BasicBlock, int)
  variable references, and the shared
  codeql.ssa.Ssa::Make<Location, Cfg, Input> module is then
  instantiated.

  SourceVariable is the AST-level Py::Variable. Variable references
  are looked up via the new CFG facade's NameNode.defines/uses/deletes
  predicates (added in the preceding PR), which themselves are
  one-line bridges to AST-level Name.defines/uses/deletes.

  Implicit-entry definitions are inserted for non-local/global/builtin
  reads, captured variables, and (when needed) parameters.

Test additions:

- library-tests/dataflow-new-ssa/ — exercises the new SSA over a
  representative test corpus and checks expected def/use chains.

- library-tests/dataflow-new-ssa-vs-legacy/ — runs both new SSA and
  legacy ESSA over the same corpus and diffs the results, so any
  semantic divergence shows up as a test failure.

Production impact:

None. The new SSA adapter has zero callers in lib/ and src/ — the
legacy ESSA SSA (semmle/python/essa/*) remains the default. The
dataflow library is not migrated yet; that lands in a follow-up PR.

Verified by:
- All 367 lib + src + consistency-queries compile clean.
- All 641 ControlFlow + PointsTo + dataflow + essa + consistency
  library-tests pass.
- Both new dataflow-new-ssa[/vs-legacy] test packs pass.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-06-05 08:03:48 +00:00
Copilot
4aee0b3c87 Python: add new shared-CFG-backed control flow graph
Preparatory refactor for the shared-CFG dataflow migration. Adds the
new Python CFG library additively, without changing any production
behaviour.

Library additions:

- semmle.python.controlflow.internal.AstNodeImpl — mediates between
  the Python AST and the shared codeql.controlflow.ControlFlowGraph
  signature. Wraps Python's Stmt/Expr/Scope/Pattern and adds two
  synthetic kinds of node (BlockStmt for body slots, intermediate
  nodes for multi-operand boolean expressions).

- semmle.python.controlflow.internal.Cfg — public facade
  re-exposing the same API surface as semmle/python/Flow.qll
  (ControlFlowNode, CallNode, BasicBlock, NameNode, DefinitionNode,
  CompareNode, ...), backed by the shared CFG.

- lib/printCfgNew.ql — debug/visualisation query for the new CFG.

- consistency-queries/CfgConsistency.ql — consistency query running
  the shared CFG's standard checks against Python.

Shared library:

- shared.controlflow.ControlFlowGraph — adds two defaulted
  getWhileElse / getForeachElse predicates to AstSig so Python can
  model while-else / for-else (no behavioural change for other
  languages).

Test additions:

- ControlFlow/bindings/* — annotation-driven SSA-binding tests for
  the new CFG (annassign, compound, comprehension, decorated,
  except_handler, imports, match_pattern, parameters, simple,
  type_params, walrus_starred, with_stmt, dead_under_no_raise).

- ControlFlow/store-load/* — basic store/load coverage.

- ControlFlow/evaluation-order/NewCfg*.ql — mirrors of the existing
  OldCfg evaluation-order self-validation suite, run against the
  new CFG via NewCfgImpl.qll.

- Minor extensions to existing test_if.py / test_boolean.py +
  cosmetic .expected churn on a handful of OldCfg tests.

No dataflow, SSA, or production query is migrated yet — that lands in
follow-up PRs. The new CFG library has zero callers in lib/ and src/.

Verified by:
- All lib + src + consistency-queries compile clean (367 queries).
- All 56 ControlFlow library-tests pass.
- All 474 dataflow + PointsTo library-tests + consistency tests pass.
- syntax_error/CONSISTENCY/CfgConsistency passes.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-06-05 08:03:29 +00:00
yoff
a2295e7216 Apply suggestions from code review
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
2026-06-05 07:57:43 +00:00
Copilot
0623acc7f5 Python: qualify Flow.qll's AST references with Py:: prefix
Preparatory refactor for the shared-CFG dataflow migration. Switches
'import python' to 'import python as Py' inside Flow.qll, and qualifies
every AST-class reference (Expr, Bytes, Dict, AssignExpr, Compare,
Module, Scope, Call, Attribute, SsaVariable, AugAssign, etc.) with the
Py:: prefix.

Flow.qll's own CFG types (ControlFlowNode, BasicBlock, CallNode,
NameNode, DefinitionNode, CompareNode, ...) keep their unqualified
names — they remain the public CFG API exported from this file.

This is a semantic noop: the qualification was applied mechanically by
script and no name resolution changes. Verified by:
- All 361 lib/ + src/ queries compile clean.
- All 186 ControlFlow + PointsTo + dataflow library-tests pass.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-06-05 07:57:42 +00:00
yoff
dc5aa8e0f5 Python: deprecate Function.getAReturnValueFlowNode() and rewrite internal callers
Follow-up to the getAFlowNode deprecation in the same PR: same AST→legacy-CFG
bridge pattern. The 11 internal call sites (across objects/, types/,
frameworks/, and TypeTrackingImpl) are rewritten to bind a `Return ret`
explicitly, then constrain via `ret.getScope() = f and n.getNode() = ret.getValue()`.

The predicate itself is preserved with a deprecation note so external
users do not experience churn.

Semantic noop.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-06-05 07:57:42 +00:00
Copilot
db1e5035b4 Python: deprecate AstNode.getAFlowNode() and rewrite internal callers
Preparatory refactor for the shared-CFG dataflow migration.

Deprecates the AstNode.getAFlowNode() cached predicate on the public
Python QL API and rewrites all ~140 internal callers across lib/, src/,
test/, and tools/ from `expr.getAFlowNode() = cfgNode` to
`cfgNode.getNode() = expr`, using ControlFlowNode.getNode() which
already exists in Flow.qll.

The predicate itself is preserved (with a deprecation note pointing at
the new pattern) so external users do not experience churn — they can
migrate at their own pace and the AST/CFG hierarchies still get the
intended untangling once the deprecation eventually elapses.

Semantic noop verified by:
- All 361 lib/ + src/ queries compile clean.
- All 122 ControlFlow + PointsTo library-tests pass.
- All 64 dataflow library-tests pass.
- All 113 Variables/Exceptions/Expressions/Statements/Functions/Imports/
  Security/CWE-798/ModificationOfParameterWithDefault query-tests pass.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-06-05 07:57:42 +00:00
yoff
7a3f546587 Python: inline init_module_submodule_defn into ImportResolution
The new-dataflow ImportResolution module only used
semmle.python.essa.SsaDefinitions for the 5-line helper predicate
SsaSource::init_module_submodule_defn. Inline it locally and drop the
dependency on legacy SsaDefinitions. This is the only remaining direct
import of semmle.python.essa.* in the new dataflow stack, so dropping
it makes the layering cleaner.

Semantic noop on the current SSA: SsaSourceVariable.getName() and
GlobalVariable.getId() both project the same DB column
(variable(_,_,result)), and the old call's 'init.getEntryNode() = f'
join was just constraining init = package via Scope.getEntryNode()'s
functional uniqueness. RA dump of accesses.ql confirms only the
expected predicate-rename shuffle; all 70 dataflow + ApiGraphs library
tests pass.

This factors out commit 8cab5a20f2 from the larger shared-CFG
migration #21925.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-06-05 07:57:42 +00:00
yoff
821325b7e5 Python: simplify decorator-detection predicates to pure AST match
The internal predicates that identify `@staticmethod`, `@classmethod` and
`@property` decorators previously required the decorator's `NameNode` to
satisfy `isGlobal()` (i.e. no SSA def reaches the decorator's name use).
That filter was correct but unnecessarily indirect: these three names
are builtins, and even when a class body redefines one, the class body
has not started executing at the decorator position, so Python uses the
builtin.

Match the decorator's AST `Name` directly instead, dropping the CFG/SSA
detour. The slight semantic change — `isGlobal()` would have rejected
module-level shadowing of these builtins — is negligible in practice
and explicitly documented in the change note.

`hasContextmanagerDecorator` and `hasOverloadDecorator` keep the
`NameNode.isGlobal()` check because their target names (`contextmanager`,
`overload`) are imported, not builtin, and local shadowing is a real
concern.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-06-05 07:57:42 +00:00
yoff
4d2296d4f0 Shared CFG: add defaulted getWhileElse/getForeachElse/getCatchType to AstSig
Adds three new defaulted signature predicates to the shared CFG library:

- getWhileElse / getForeachElse: `else` block of a while/for loop, if
  any (used by Python's `while-else` / `for-else` constructs).
- getCatchType: type expression of a catch clause, if any (used by
  Python's `except SomeExpr:` where the catch type is a runtime
  expression that needs CFG evaluation).

Each predicate defaults to `none()`, so behaviour is unchanged for any
language that doesn't override it (verified by re-running
java/ql/test/library-tests/controlflow/).

The Make0 succession rules are extended:
- WhileStmt/ForeachStmt: route the loop-exit edge through the else
  block before reaching the after-position.
- CatchClause: route the matching-evaluation through the type
  expression (if present) before reaching the after-value position.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-06-05 07:57:37 +00:00
296 changed files with 5943 additions and 16835 deletions

View File

@@ -237,6 +237,9 @@ use_repo(
kotlin_extractor_deps,
"codeql_kotlin_defaults",
"codeql_kotlin_embeddable",
"kotlin-compiler-1.8.0",
"kotlin-compiler-1.9.0-Beta",
"kotlin-compiler-1.9.20-Beta",
"kotlin-compiler-2.0.0-RC1",
"kotlin-compiler-2.0.20-Beta2",
"kotlin-compiler-2.1.0-Beta1",
@@ -245,7 +248,9 @@ use_repo(
"kotlin-compiler-2.2.20-Beta2",
"kotlin-compiler-2.3.0",
"kotlin-compiler-2.3.20",
"kotlin-compiler-2.4.0",
"kotlin-compiler-embeddable-1.8.0",
"kotlin-compiler-embeddable-1.9.0-Beta",
"kotlin-compiler-embeddable-1.9.20-Beta",
"kotlin-compiler-embeddable-2.0.0-RC1",
"kotlin-compiler-embeddable-2.0.20-Beta2",
"kotlin-compiler-embeddable-2.1.0-Beta1",
@@ -254,7 +259,9 @@ use_repo(
"kotlin-compiler-embeddable-2.2.20-Beta2",
"kotlin-compiler-embeddable-2.3.0",
"kotlin-compiler-embeddable-2.3.20",
"kotlin-compiler-embeddable-2.4.0",
"kotlin-stdlib-1.8.0",
"kotlin-stdlib-1.9.0-Beta",
"kotlin-stdlib-1.9.20-Beta",
"kotlin-stdlib-2.0.0-RC1",
"kotlin-stdlib-2.0.20-Beta2",
"kotlin-stdlib-2.1.0-Beta1",
@@ -263,7 +270,6 @@ use_repo(
"kotlin-stdlib-2.2.20-Beta2",
"kotlin-stdlib-2.3.0",
"kotlin-stdlib-2.3.20",
"kotlin-stdlib-2.4.0",
)
go_sdk = use_extension("@rules_go//go:extensions.bzl", "go_sdk")

View File

@@ -21,7 +21,7 @@
Java,"Java 7 to 26 [6]_","javac (OpenJDK and Oracle JDK),
Eclipse compiler for Java (ECJ) [7]_",``.java``
Kotlin,"Kotlin 2.0.0 to 2.4.\ *x*","kotlinc",``.kt``
Kotlin,"Kotlin 1.8.0 to 2.3.2\ *x*","kotlinc",``.kt``
JavaScript,ECMAScript 2022 or lower,Not applicable,"``.js``, ``.jsx``, ``.mjs``, ``.es``, ``.es6``, ``.htm``, ``.html``, ``.xhtm``, ``.xhtml``, ``.vue``, ``.hbs``, ``.ejs``, ``.njk``, ``.json``, ``.yaml``, ``.yml``, ``.raml``, ``.xml`` [8]_"
Python [9]_,"2.7, 3.5, 3.6, 3.7, 3.8, 3.9, 3.10, 3.11, 3.12, 3.13",Not applicable,``.py``
Ruby [10]_,"up to 3.3",Not applicable,"``.rb``, ``.erb``, ``.gemspec``, ``Gemfile``"

View File

@@ -9,9 +9,9 @@ import semmle.go.dataflow.internal.FlowSummaryImpl as FlowSummaryImpl
import utils.test.InlineFlowTest
module Config implements DataFlow::ConfigSig {
predicate isSource(DataFlow::Node source) { sourceNode(source, "qltest") }
predicate isSource(DataFlow::Node src) { sourceNode(src, "qltest") }
predicate isSink(DataFlow::Node sink) { sinkNode(sink, "qltest") }
predicate isSink(DataFlow::Node src) { sinkNode(src, "qltest") }
}
import ValueFlowTest<Config>

View File

@@ -1,2 +0,0 @@
reverseRead
| main.go:23:3:23:5 | out | Origin of readStep is missing a PostUpdateNode. |

View File

@@ -4,7 +4,7 @@ func source() string {
return "untrusted data"
}
func sink(any) {
func sink(string) {
}
type A struct {
@@ -19,10 +19,6 @@ func functionWithVarArgsParameter(s ...string) string {
return s[1]
}
func functionWithVarArgsOutParameter(in string, out ...*string) {
*out[0] = in
}
func functionWithSliceOfStructsParameter(s []A) string {
return s[1].f
}
@@ -42,12 +38,6 @@ func main() {
sink(functionWithVarArgsParameter(sSlice...)) // $ hasValueFlow="call to functionWithVarArgsParameter"
sink(functionWithVarArgsParameter(s0, s1)) // $ hasValueFlow="call to functionWithVarArgsParameter"
var out1 *string
var out2 *string
functionWithVarArgsOutParameter(source(), out1, out2)
sink(out1) // $ MISSING: hasValueFlow="out1"
sink(out2) // $ MISSING: hasValueFlow="out2"
sliceOfStructs := []A{{f: source()}}
sink(sliceOfStructs[0].f) // $ hasValueFlow="selection of f"

View File

@@ -1,2 +0,0 @@
invalidModelRow
testFailures

View File

@@ -1,21 +0,0 @@
extensions:
- addsTo:
pack: codeql/go-all
extensible: summaryModel
data:
- ["github.com/nonexistent/test", "", False, "FunctionWithParameter", "", "", "Argument[0]", "ReturnValue", "value", "manual"]
- ["github.com/nonexistent/test", "", False, "FunctionWithSliceParameter", "", "", "Argument[0].ArrayElement", "ReturnValue", "value", "manual"]
- ["github.com/nonexistent/test", "", False, "FunctionWithVarArgsParameter", "", "", "Argument[0].ArrayElement", "ReturnValue", "value", "manual"]
- ["github.com/nonexistent/test", "", False, "FunctionWithVarArgsOutParameter", "", "", "Argument[0]", "Argument[1].ArrayElement", "value", "manual"]
- ["github.com/nonexistent/test", "", False, "FunctionWithSliceOfStructsParameter", "", "", "Argument[0].ArrayElement.Field[github.com/nonexistent/test.A.Field]", "ReturnValue", "value", "manual"]
- ["github.com/nonexistent/test", "", False, "FunctionWithVarArgsOfStructsParameter", "", "", "Argument[0].ArrayElement.Field[github.com/nonexistent/test.A.Field]", "ReturnValue", "value", "manual"]
- addsTo:
pack: codeql/go-all
extensible: sourceModel
data:
- ["github.com/nonexistent/test", "", False, "VariadicSource", "", "", "Argument[0]", "qltest", "manual"]
- addsTo:
pack: codeql/go-all
extensible: sinkModel
data:
- ["github.com/nonexistent/test", "", False, "VariadicSink", "", "", "Argument[0]", "qltest", "manual"]

View File

@@ -1,22 +0,0 @@
import go
import semmle.go.dataflow.ExternalFlow
import ModelValidation
import utils.test.InlineFlowTest
module Config implements DataFlow::ConfigSig {
predicate isSource(DataFlow::Node source) {
sourceNode(source, "qltest")
or
exists(Function fn | fn.hasQualifiedName(_, ["source", "taint"]) |
source = fn.getACall().getResult()
)
}
predicate isSink(DataFlow::Node sink) {
sinkNode(sink, "qltest")
or
exists(Function fn | fn.hasQualifiedName(_, "sink") | sink = fn.getACall().getAnArgument())
}
}
import FlowTest<Config, Config>

View File

@@ -1,5 +0,0 @@
module semmle.go.Packages
go 1.25
require github.com/nonexistent/test v0.0.0-20200203000000-0000000000000

View File

@@ -1,56 +0,0 @@
package main
import (
"github.com/nonexistent/test"
)
func source() string {
return "untrusted data"
}
func sink(any) {
}
func main() {
s := source()
sink(test.FunctionWithParameter(s)) // $ hasValueFlow="call to FunctionWithParameter"
stringSlice := []string{source()}
sink(stringSlice[0]) // $ hasValueFlow="index expression"
s0 := ""
s1 := source()
sSlice := []string{s0, s1}
sink(test.FunctionWithParameter(sSlice[1])) // $ hasValueFlow="call to FunctionWithParameter"
sink(test.FunctionWithSliceParameter(sSlice)) // $ hasValueFlow="call to FunctionWithSliceParameter"
sink(test.FunctionWithVarArgsParameter(sSlice...)) // $ hasValueFlow="call to FunctionWithVarArgsParameter"
sink(test.FunctionWithVarArgsParameter(s0, s1)) // $ hasValueFlow="call to FunctionWithVarArgsParameter"
var out1 *string
var out2 *string
test.FunctionWithVarArgsOutParameter(source(), out1, out2)
sink(out1) // $ MISSING: hasValueFlow="out1"
sink(out2) // $ MISSING: hasValueFlow="out2"
sliceOfStructs := []test.A{{Field: source()}}
sink(sliceOfStructs[0].Field) // $ hasValueFlow="selection of Field"
a0 := test.A{Field: ""}
a1 := test.A{Field: source()}
aSlice := []test.A{a0, a1}
sink(test.FunctionWithSliceOfStructsParameter(aSlice)) // $ hasValueFlow="call to FunctionWithSliceOfStructsParameter"
sink(test.FunctionWithVarArgsOfStructsParameter(aSlice...)) // $ hasValueFlow="call to FunctionWithVarArgsOfStructsParameter"
sink(test.FunctionWithVarArgsOfStructsParameter(a0, a1)) // $ hasValueFlow="call to FunctionWithVarArgsOfStructsParameter"
var variadicSource string
test.VariadicSource(&variadicSource)
sink(variadicSource) // $ MISSING: hasTaintFlow="variadicSource"
sink(&variadicSource) // $ MISSING: hasTaintFlow="&..."
var variadicSourcePtr *string
test.VariadicSource(variadicSourcePtr)
sink(variadicSourcePtr) // $ MISSING: hasTaintFlow="variadicSourcePtr"
sink(*variadicSourcePtr) // $ MISSING: hasTaintFlow="star expression"
test.VariadicSink(source()) // $ hasTaintFlow="[]type{args}"
}

View File

@@ -1,32 +0,0 @@
package test
type A struct {
Field string
}
func FunctionWithParameter(s string) string {
return ""
}
func FunctionWithSliceParameter(s []string) string {
return ""
}
func FunctionWithVarArgsParameter(s ...string) string {
return ""
}
func FunctionWithVarArgsOutParameter(in string, out ...*string) {
}
func FunctionWithSliceOfStructsParameter(s []A) string {
return ""
}
func FunctionWithVarArgsOfStructsParameter(s ...A) string {
return ""
}
func VariadicSource(s ...*string) {}
func VariadicSink(s ...string) {}

View File

@@ -1,3 +0,0 @@
# github.com/nonexistent/test v0.0.0-20200203000000-0000000000000
## explicit
github.com/nonexistent/test

View File

@@ -20,9 +20,6 @@ class SummaryModelTest extends DataFlow::FunctionModel {
this.hasQualifiedName("github.com/nonexistent/test", "FunctionWithVarArgsParameter") and
(inp.isParameter(_) and outp.isResult())
or
this.hasQualifiedName("github.com/nonexistent/test", "FunctionWithVarArgsOutParameter") and
(inp.isParameter(0) and outp.isParameter(any(int i | i >= 1)))
or
this.hasQualifiedName("github.com/nonexistent/test", "FunctionWithSliceOfStructsParameter") and
(inp.isParameter(0) and outp.isResult())
or

View File

@@ -1,5 +1,5 @@
module semmle.go.Packages
go 1.25
go 1.17
require github.com/nonexistent/test v0.0.0-20200203000000-0000000000000

View File

@@ -8,7 +8,7 @@ func source() string {
return "untrusted data"
}
func sink(any) {
func sink(string) {
}
func main() {
@@ -21,17 +21,10 @@ func main() {
s0 := ""
s1 := source()
sSlice := []string{s0, s1}
sink(test.FunctionWithParameter(sSlice[1])) // $ hasValueFlow="call to FunctionWithParameter"
sink(test.FunctionWithSliceParameter(sSlice)) // $ hasTaintFlow="call to FunctionWithSliceParameter" MISSING: hasValueFlow="call to FunctionWithSliceParameter"
sink(test.FunctionWithVarArgsParameter(sSlice...)) // $ hasTaintFlow="call to FunctionWithVarArgsParameter" MISSING: hasValueFlow="call to FunctionWithVarArgsParameter"
randomFunctionWithMoreThanOneParameter(1, 2, 3, 4, 5) // This is needed to make the next line pass, because we need to have seen a call to a function with at least 2 parameters for ParameterInput to exist with index 1.
sink(test.FunctionWithVarArgsParameter(s0, s1)) // $ hasValueFlow="call to FunctionWithVarArgsParameter"
var out1 *string
var out2 *string
test.FunctionWithVarArgsOutParameter(source(), out1, out2)
sink(out1) // $ hasValueFlow="out1"
sink(out2) // $ hasValueFlow="out2"
sink(test.FunctionWithParameter(sSlice[1])) // $ hasValueFlow="call to FunctionWithParameter"
sink(test.FunctionWithSliceParameter(sSlice)) // $ hasTaintFlow="call to FunctionWithSliceParameter" MISSING: hasValueFlow="call to FunctionWithSliceParameter"
sink(test.FunctionWithVarArgsParameter(sSlice...)) // $ hasTaintFlow="call to FunctionWithVarArgsParameter" MISSING: hasValueFlow="call to FunctionWithVarArgsParameter"
sink(test.FunctionWithVarArgsParameter(s0, s1)) // $ MISSING: hasValueFlow="call to FunctionWithVarArgsParameter"
sliceOfStructs := []test.A{{Field: source()}}
sink(sliceOfStructs[0].Field) // $ hasValueFlow="selection of Field"
@@ -44,6 +37,3 @@ func main() {
sink(test.FunctionWithVarArgsOfStructsParameter(aSlice...)) // $ MISSING: hasValueFlow="call to FunctionWithVarArgsOfStructsParameter"
sink(test.FunctionWithVarArgsOfStructsParameter(a0, a1)) // $ MISSING: hasValueFlow="call to FunctionWithVarArgsOfStructsParameter"
}
func randomFunctionWithMoreThanOneParameter(i1, i2, i3, i4, i5 int) {
}

View File

@@ -16,9 +16,6 @@ func FunctionWithVarArgsParameter(s ...string) string {
return ""
}
func FunctionWithVarArgsOutParameter(in string, out ...*string) {
}
func FunctionWithSliceOfStructsParameter(s []A) string {
return ""
}

View File

@@ -64,14 +64,8 @@ _resources = [
r[len("src/main/resources/"):],
)
for r in glob(["src/main/resources/**"])
if r != "src/main/resources/META-INF/services/org.jetbrains.kotlin.compiler.plugin.CompilerPluginRegistrar"
]
_compiler_plugin_registrar_service = (
"src/main/resources/META-INF/services/org.jetbrains.kotlin.compiler.plugin.CompilerPluginRegistrar",
"META-INF/services/org.jetbrains.kotlin.compiler.plugin.CompilerPluginRegistrar",
)
kt_javac_options(
name = "javac-options",
release = "8",
@@ -97,32 +91,19 @@ kt_javac_options(
# * `resource_strip_prefix` is unique per jar, so we must also put other resources under the same version prefix
genrule(
name = "resources-%s" % v,
srcs = [src for src, _ in _resources] + (
[_compiler_plugin_registrar_service[0]] if not version_less(v, "2.4.0") else []
),
srcs = [src for src, _ in _resources],
outs = [
"%s/com/github/codeql/extractor.name" % v,
] + [
"%s/%s" % (v, target)
for _, target in _resources
] + (
["%s/%s" % (
v,
_compiler_plugin_registrar_service[1],
)] if not version_less(v, "2.4.0") else []
),
],
cmd = "\n".join([
"echo %s-%s > $(RULEDIR)/%s/com/github/codeql/extractor.name" % (_extractor_name_prefix, v, v),
] + [
"cp $(execpath %s) $(RULEDIR)/%s/%s" % (source, v, target)
for source, target in _resources
] + (
["cp $(execpath %s) $(RULEDIR)/%s/%s" % (
_compiler_plugin_registrar_service[0],
v,
_compiler_plugin_registrar_service[1],
)] if not version_less(v, "2.4.0") else []
)),
]),
),
kt_jvm_library(
name = "%s-%s" % (_extractor_name_prefix, v),

Binary file not shown.

Binary file not shown.

Binary file not shown.

View File

@@ -27,7 +27,7 @@ import shutil
import io
import os
DEFAULT_VERSION = "2.4.0"
DEFAULT_VERSION = "2.3.20"
def options():

View File

@@ -3,21 +3,32 @@
package com.github.codeql
import com.intellij.mock.MockProject
import com.intellij.openapi.extensions.LoadingOrder
import org.jetbrains.kotlin.backend.common.extensions.IrGenerationExtension
import org.jetbrains.kotlin.config.CompilerConfiguration
class KotlinExtractorComponentRegistrar : Kotlin2ComponentRegistrar() {
override fun doRegisterExtensions(configuration: CompilerConfiguration) {
override fun registerProjectComponents(
project: MockProject,
configuration: CompilerConfiguration
) {
val invocationTrapFile = configuration[KEY_INVOCATION_TRAP_FILE]
if (invocationTrapFile == null) {
throw Exception("Required argument for TRAP invocation file not given")
}
registerExtractorExtension(
// Register with LoadingOrder.LAST to ensure the extractor runs after other
// IR generation plugins (like kotlinx.serialization) have generated their code.
val extensionPoint = project.extensionArea.getExtensionPoint(IrGenerationExtension.extensionPointName)
extensionPoint.registerExtension(
KotlinExtractorExtension(
invocationTrapFile,
configuration[KEY_CHECK_TRAP_IDENTICAL] ?: false,
configuration[KEY_COMPILATION_STARTTIME],
configuration[KEY_EXIT_AFTER_EXTRACTION] ?: false
)
),
LoadingOrder.LAST,
project
)
}
}

View File

@@ -173,9 +173,9 @@ open class KotlinFileExtractor(
when (d) {
is IrFunction ->
when (d.name.asString()) {
"toString" -> d.codeQlValueParameters.isEmpty()
"hashCode" -> d.codeQlValueParameters.isEmpty()
"equals" -> d.codeQlValueParameters.singleOrNull()?.type?.isNullableAny() ?: false
"toString" -> d.valueParameters.isEmpty()
"hashCode" -> d.valueParameters.isEmpty()
"equals" -> d.valueParameters.singleOrNull()?.type?.isNullableAny() ?: false
else -> false
} && isJavaBinaryDeclaration(d)
else -> false
@@ -721,7 +721,7 @@ open class KotlinFileExtractor(
(it.type as? IrSimpleType)?.classFqName?.asString() != "kotlin.Deprecated"
} +
// Note we lose any arguments to @java.lang.Deprecated that were written in source.
codeQlAnnotationFromSymbolOwner(
IrConstructorCallImpl.fromSymbolOwner(
UNDEFINED_OFFSET,
UNDEFINED_OFFSET,
jldConstructor.returnType,
@@ -781,13 +781,13 @@ open class KotlinFileExtractor(
val locId = tw.getLocation(constructorCall)
tw.writeHasLocation(id, locId)
for (i in 0 until constructorCall.codeQlValueArgumentsCount) {
val param = constructorCall.symbol.owner.codeQlValueParameters[i]
for (i in 0 until constructorCall.valueArgumentsCount) {
val param = constructorCall.symbol.owner.valueParameters[i]
val prop =
constructorCall.symbol.owner.parentAsClass.declarations
.filterIsInstance<IrProperty>()
.first { it.name == param.name }
val v = constructorCall.codeQlGetValueArgument(i) ?: param.defaultValue?.expression
val v = constructorCall.getValueArgument(i) ?: param.defaultValue?.expression
val getter = prop.getter
if (getter == null) {
logger.warnElement("Expected annotation property to define a getter", prop)
@@ -1115,9 +1115,9 @@ open class KotlinFileExtractor(
returnId,
0,
returnId,
f.codeQlValueParameters.size,
f.valueParameters.size,
{ argParent, idxOffset ->
f.codeQlValueParameters.forEachIndexed { idx, param ->
f.valueParameters.forEachIndexed { idx, param ->
val syntheticParamId = useValueParameter(param, proxyFunctionId)
extractVariableAccess(
syntheticParamId,
@@ -1695,9 +1695,9 @@ open class KotlinFileExtractor(
returnId,
0,
returnId,
f.codeQlValueParameters.size,
f.valueParameters.size,
{ argParentId, idxOffset ->
f.codeQlValueParameters.mapIndexed { idx, param ->
f.valueParameters.mapIndexed { idx, param ->
val syntheticParamId = useValueParameter(param, functionId)
extractVariableAccess(
syntheticParamId,
@@ -1792,7 +1792,7 @@ open class KotlinFileExtractor(
extractBody: Boolean,
extractMethodAndParameterTypeAccesses: Boolean
) {
if (f.codeQlValueParameters.none { it.defaultValue != null }) return
if (f.valueParameters.none { it.defaultValue != null }) return
val id = getDefaultsMethodLabel(f)
if (id == null) {
@@ -1800,7 +1800,7 @@ open class KotlinFileExtractor(
return
}
val locId = getLocation(f, null)
val extReceiver = f.codeQlExtensionReceiverParameter
val extReceiver = f.extensionReceiverParameter
val dispatchReceiver = if (f.shouldExtractAsStatic) null else f.dispatchReceiverParameter
val parameterTypes = getDefaultsMethodArgTypes(f)
val allParamTypeResults =
@@ -1869,7 +1869,7 @@ open class KotlinFileExtractor(
tw.writeCompiler_generated(id, CompilerGeneratedKinds.DEFAULT_ARGUMENTS_METHOD.kind)
if (extractBody) {
val nonSyntheticParams = listOfNotNull(dispatchReceiver) + f.codeQlValueParameters
val nonSyntheticParams = listOfNotNull(dispatchReceiver) + f.valueParameters
// This stack entry represents as if we're extracting the 'real' function `f`, giving
// the indices of its non-synthetic parameters
// such that when we extract the default expressions below, any reference to f's nth
@@ -1895,12 +1895,12 @@ open class KotlinFileExtractor(
val realParamsVarId = getValueParameterLabel(id, parameterTypes.size - 2)
val intType = pluginContext.irBuiltIns.intType
val paramIdxOffset =
listOf(dispatchReceiver, f.codeQlExtensionReceiverParameter).count { it != null }
listOf(dispatchReceiver, f.extensionReceiverParameter).count { it != null }
extractBlockBody(id, locId).also { blockId ->
var nextStmt = 0
// For each parameter with a default, sub in the default value if the caller
// hasn't supplied a value:
f.codeQlValueParameters.forEachIndexed { paramIdx, param ->
f.valueParameters.forEachIndexed { paramIdx, param ->
val defaultVal = param.defaultValue
if (defaultVal != null) {
extractIfStmt(locId, blockId, nextStmt++, id).also { ifId ->
@@ -1975,7 +1975,7 @@ open class KotlinFileExtractor(
id
)
tw.writeHasLocation(thisCallId, locId)
f.codeQlValueParameters.forEachIndexed { idx, param ->
f.valueParameters.forEachIndexed { idx, param ->
extractVariableAccess(
tw.getLabelFor<DbParam>(getValueParameterLabel(id, idx)),
param.type,
@@ -2003,9 +2003,9 @@ open class KotlinFileExtractor(
)
.also { thisCallId ->
val realFnIdxOffset =
if (f.codeQlExtensionReceiverParameter != null) 1 else 0
if (f.extensionReceiverParameter != null) 1 else 0
val paramMappings =
f.codeQlValueParameters.mapIndexed { idx, param ->
f.valueParameters.mapIndexed { idx, param ->
Triple(
param.type,
idx + paramIdxOffset,
@@ -2156,7 +2156,7 @@ open class KotlinFileExtractor(
val dispatchReceiver =
f.dispatchReceiverParameter?.let { IrGetValueImpl(-1, -1, it.symbol) }
val extensionReceiver =
f.codeQlExtensionReceiverParameter?.let { IrGetValueImpl(-1, -1, it.symbol) }
f.extensionReceiverParameter?.let { IrGetValueImpl(-1, -1, it.symbol) }
extractExpressionBody(overloadId, realFunctionLocId).also { returnId ->
extractsDefaultsCall(
@@ -2180,28 +2180,28 @@ open class KotlinFileExtractor(
if (!f.hasAnnotation(jvmOverloadsFqName)) {
if (
f is IrConstructor &&
f.codeQlValueParameters.isNotEmpty() &&
f.codeQlValueParameters.all { it.defaultValue != null } &&
f.valueParameters.isNotEmpty() &&
f.valueParameters.all { it.defaultValue != null } &&
f.parentClassOrNull?.let {
// Don't create a default constructor for an annotation class, or a class
// that explicitly declares a no-arg constructor.
!it.isAnnotationClass &&
it.declarations.none { d ->
d is IrConstructor && d.codeQlValueParameters.isEmpty()
d is IrConstructor && d.valueParameters.isEmpty()
}
} == true
) {
// Per https://kotlinlang.org/docs/classes.html#creating-instances-of-classes, a
// single default overload gets created specifically
// when we have all default parameters, regardless of `@JvmOverloads`.
extractGeneratedOverload(f.codeQlValueParameters.map { _ -> null })
extractGeneratedOverload(f.valueParameters.map { _ -> null })
}
return
}
val paramList: MutableList<IrValueParameter?> = f.codeQlValueParameters.toMutableList()
for (n in (f.codeQlValueParameters.size - 1) downTo 0) {
if (f.codeQlValueParameters[n].defaultValue != null) {
val paramList: MutableList<IrValueParameter?> = f.valueParameters.toMutableList()
for (n in (f.valueParameters.size - 1) downTo 0) {
if (f.valueParameters[n].defaultValue != null) {
paramList[n] = null // Remove this parameter, to be replaced by a default value
extractGeneratedOverload(paramList)
}
@@ -2327,7 +2327,7 @@ open class KotlinFileExtractor(
getClassByFqName(pluginContext, it)?.let { annotationClass ->
annotationClass.owner.declarations.firstIsInstanceOrNull<IrConstructor>()?.let {
annotationConstructor ->
codeQlAnnotationFromSymbolOwner(
IrConstructorCallImpl.fromSymbolOwner(
UNDEFINED_OFFSET,
UNDEFINED_OFFSET,
annotationConstructor.returnType,
@@ -2388,13 +2388,13 @@ open class KotlinFileExtractor(
id
}
val extReceiver = f.codeQlExtensionReceiverParameter
val extReceiver = f.extensionReceiverParameter
// The following parameter order is correct, because member $default methods (where
// the order would be [dispatchParam], [extensionParam], normalParams) are not
// extracted here
val fParameters =
listOfNotNull(extReceiver) +
(overriddenAttributes?.valueParameters ?: f.codeQlValueParameters)
(overriddenAttributes?.valueParameters ?: f.valueParameters)
val paramTypes =
fParameters.mapIndexed { i, vp ->
extractValueParameter(
@@ -3069,14 +3069,14 @@ open class KotlinFileExtractor(
logger.errorElement("Unexpected dispatch receiver found", c)
}
if (c.codeQlValueArgumentsCount < 1) {
if (c.valueArgumentsCount < 1) {
logger.errorElement("No arguments found", c)
return
}
extractArgument(id, c, callable, enclosingStmt, 0, "Operand null")
if (c.codeQlValueArgumentsCount > 1) {
if (c.valueArgumentsCount > 1) {
logger.errorElement("Extra arguments found", c)
}
}
@@ -3095,21 +3095,21 @@ open class KotlinFileExtractor(
logger.errorElement("Unexpected dispatch receiver found", c)
}
if (c.codeQlValueArgumentsCount < 1) {
if (c.valueArgumentsCount < 1) {
logger.errorElement("No arguments found", c)
return
}
extractArgument(id, c, callable, enclosingStmt, 0, "LHS null")
if (c.codeQlValueArgumentsCount < 2) {
if (c.valueArgumentsCount < 2) {
logger.errorElement("No RHS found", c)
return
}
extractArgument(id, c, callable, enclosingStmt, 1, "RHS null")
if (c.codeQlValueArgumentsCount > 2) {
if (c.valueArgumentsCount > 2) {
logger.errorElement("Extra arguments found", c)
}
}
@@ -3122,7 +3122,7 @@ open class KotlinFileExtractor(
idx: Int,
msg: String
) {
val op = c.codeQlGetValueArgument(idx)
val op = c.getValueArgument(idx)
if (op == null) {
logger.errorElement(msg, c)
} else {
@@ -3267,8 +3267,8 @@ open class KotlinFileExtractor(
// and which should be replaced by defaults. The final Object parameter is apparently always
// null.
(listOfNotNull(if (f.shouldExtractAsStatic) null else f.dispatchReceiverParameter?.type) +
listOfNotNull(f.codeQlExtensionReceiverParameter?.type) +
f.codeQlValueParameters.map { it.type } +
listOfNotNull(f.extensionReceiverParameter?.type) +
f.valueParameters.map { it.type } +
listOf(pluginContext.irBuiltIns.intType, getDefaultsMethodLastArgType(f)))
.map { erase(it) }
@@ -3345,7 +3345,7 @@ open class KotlinFileExtractor(
val overriddenCallTarget =
(callTarget as? IrSimpleFunction)?.allOverridden(includeSelf = true)?.firstOrNull {
it.overriddenSymbols.isEmpty() &&
it.codeQlValueParameters.any { p -> p.defaultValue != null }
it.valueParameters.any { p -> p.defaultValue != null }
} ?: callTarget
if (isExternalDeclaration(overriddenCallTarget)) {
// Likewise, ensure the overridden target gets extracted.
@@ -3419,7 +3419,7 @@ open class KotlinFileExtractor(
}
val valueArgsWithDummies =
valueArguments.zip(callTarget.codeQlValueParameters).map { (expr, param) ->
valueArguments.zip(callTarget.valueParameters).map { (expr, param) ->
expr ?: IrConstImpl.defaultValueForType(0, 0, param.type)
}
@@ -3529,7 +3529,7 @@ open class KotlinFileExtractor(
callTarget: IrFunction,
valueArguments: List<IrExpression?>
): Boolean {
val varargParam = callTarget.codeQlValueParameters.withIndex().find { it.value.isVararg }
val varargParam = callTarget.valueParameters.withIndex().find { it.value.isVararg }
// If the vararg param is the only one not specified, and it has no default value, then we
// don't need to call a $default method,
// as omitting it already implies passing an empty vararg array.
@@ -3805,7 +3805,7 @@ open class KotlinFileExtractor(
) =
extractCallValueArguments(
callId,
(0 until call.codeQlValueArgumentsCount).map { call.codeQlGetValueArgument(it) },
(0 until call.valueArgumentsCount).map { call.getValueArgument(it) },
enclosingStmt,
enclosingCallable,
idxOffset
@@ -3874,7 +3874,7 @@ open class KotlinFileExtractor(
(owner.parentClassOrNull?.fqNameWhenAvailable?.asString() == type ||
(owner.parent is IrExternalPackageFragment &&
getFileClassFqName(owner)?.asString() == type)) &&
owner.codeQlValueParameters
owner.valueParameters
.map { it.type.classFqName?.asString() }
.toTypedArray() contentEquals parameterTypes
}
@@ -3926,8 +3926,8 @@ open class KotlinFileExtractor(
val result =
javaLangString?.declarations?.findSubType<IrFunction> {
it.name.asString() == "valueOf" &&
it.codeQlValueParameters.size == 1 &&
it.codeQlValueParameters[0].type == pluginContext.irBuiltIns.anyNType
it.valueParameters.size == 1 &&
it.valueParameters[0].type == pluginContext.irBuiltIns.anyNType
}
if (result == null) {
logger.error("Couldn't find declaration java.lang.String.valueOf(Object)")
@@ -3951,7 +3951,7 @@ open class KotlinFileExtractor(
val kotlinNoWhenBranchMatchedConstructor by lazy {
val result =
kotlinNoWhenBranchMatchedExn?.declarations?.findSubType<IrConstructor> {
it.codeQlValueParameters.isEmpty()
it.valueParameters.isEmpty()
}
if (result == null) {
logger.error("Couldn't find no-arg constructor for kotlin.NoWhenBranchMatchedException")
@@ -3990,7 +3990,7 @@ open class KotlinFileExtractor(
verboseln("No match as function name is ${target.name.asString()} not $fName")
return false
}
val extensionReceiverParameter = target.codeQlExtensionReceiverParameter
val extensionReceiverParameter = target.extensionReceiverParameter
val targetClass =
if (extensionReceiverParameter == null) {
if (isNullable == true) {
@@ -4098,8 +4098,8 @@ open class KotlinFileExtractor(
) {
val typeArgs =
if (extractMethodTypeArguments)
(0 until c.codeQlTypeArgumentsCount)
.map { c.codeQlGetTypeArgument(it) }
(0 until c.typeArgumentsCount)
.map { c.getTypeArgument(it) }
.requireNoNullsOrNull()
else listOf()
@@ -4116,9 +4116,9 @@ open class KotlinFileExtractor(
parent,
idx,
enclosingStmt,
(0 until c.codeQlValueArgumentsCount).map { c.codeQlGetValueArgument(it) },
(0 until c.valueArgumentsCount).map { c.getValueArgument(it) },
c.dispatchReceiver,
c.codeQlExtensionReceiver,
c.extensionReceiver,
typeArgs,
extractClassTypeArguments,
c.superQualifierSymbol
@@ -4126,12 +4126,12 @@ open class KotlinFileExtractor(
}
fun extractSpecialEnumFunction(fnName: String) {
if (c.codeQlTypeArgumentsCount != 1) {
if (c.typeArgumentsCount != 1) {
logger.errorElement("Expected to find exactly one type argument", c)
return
}
val enumType = (c.codeQlGetTypeArgument(0) as? IrSimpleType)?.classifier?.owner
val enumType = (c.getTypeArgument(0) as? IrSimpleType)?.classifier?.owner
if (enumType == null) {
logger.errorElement("Couldn't find type of enum type", c)
return
@@ -4178,13 +4178,13 @@ open class KotlinFileExtractor(
} else {
extractExpressionExpr(receiver, callable, id, 0, enclosingStmt)
}
if (c.codeQlValueArgumentsCount < 1) {
if (c.valueArgumentsCount < 1) {
logger.errorElement("No RHS found", c)
} else {
if (c.codeQlValueArgumentsCount > 1) {
if (c.valueArgumentsCount > 1) {
logger.errorElement("Extra arguments found", c)
}
val arg = c.codeQlGetValueArgument(0)
val arg = c.getValueArgument(0)
if (arg == null) {
logger.errorElement("RHS null", c)
} else {
@@ -4205,7 +4205,7 @@ open class KotlinFileExtractor(
} else {
extractExpressionExpr(receiver, callable, id, 0, enclosingStmt)
}
if (c.codeQlValueArgumentsCount > 0) {
if (c.valueArgumentsCount > 0) {
logger.errorElement("Extra arguments found", c)
}
}
@@ -4219,7 +4219,7 @@ open class KotlinFileExtractor(
}
fun binopExt(id: Label<out DbExpr>) {
binopReceiver(id, c.codeQlExtensionReceiver, "Extension receiver")
binopReceiver(id, c.extensionReceiver, "Extension receiver")
}
fun unaryopDisp(id: Label<out DbExpr>) {
@@ -4227,7 +4227,7 @@ open class KotlinFileExtractor(
}
fun unaryopExt(id: Label<out DbExpr>) {
unaryopReceiver(id, c.codeQlExtensionReceiver, "Extension receiver")
unaryopReceiver(id, c.extensionReceiver, "Extension receiver")
}
val dr = c.dispatchReceiver
@@ -4249,7 +4249,7 @@ open class KotlinFileExtractor(
parent,
idx,
enclosingStmt,
listOf(c.codeQlExtensionReceiver, c.codeQlGetValueArgument(0)),
listOf(c.extensionReceiver, c.getValueArgument(0)),
null,
null
)
@@ -4350,7 +4350,7 @@ open class KotlinFileExtractor(
// != gets desugared into not and ==. Here we resugar it.
c.origin == IrStatementOrigin.EXCLEQ &&
isFunction(target, "kotlin", "Boolean", "not") &&
c.codeQlValueArgumentsCount == 0 &&
c.valueArgumentsCount == 0 &&
dr != null &&
dr is IrCall &&
isBuiltinCallInternal(dr, "EQEQ") -> {
@@ -4362,7 +4362,7 @@ open class KotlinFileExtractor(
}
c.origin == IrStatementOrigin.EXCLEQEQ &&
isFunction(target, "kotlin", "Boolean", "not") &&
c.codeQlValueArgumentsCount == 0 &&
c.valueArgumentsCount == 0 &&
dr != null &&
dr is IrCall &&
isBuiltinCallInternal(dr, "EQEQEQ") -> {
@@ -4374,7 +4374,7 @@ open class KotlinFileExtractor(
}
c.origin == IrStatementOrigin.EXCLEQ &&
isFunction(target, "kotlin", "Boolean", "not") &&
c.codeQlValueArgumentsCount == 0 &&
c.valueArgumentsCount == 0 &&
dr != null &&
dr is IrCall &&
isBuiltinCallInternal(dr, "ieee754equals") -> {
@@ -4576,7 +4576,7 @@ open class KotlinFileExtractor(
parent,
idx,
enclosingStmt,
listOf(c.codeQlExtensionReceiver),
listOf(c.extensionReceiver),
null,
null
)
@@ -4596,8 +4596,8 @@ open class KotlinFileExtractor(
val locId = tw.getLocation(c)
extractExprContext(id, locId, callable, enclosingStmt)
if (c.codeQlTypeArgumentsCount == 1) {
val typeArgument = c.codeQlGetTypeArgument(0)
if (c.typeArgumentsCount == 1) {
val typeArgument = c.getTypeArgument(0)
if (typeArgument == null) {
logger.errorElement("Type argument missing in an arrayOfNulls call", c)
} else {
@@ -4618,8 +4618,8 @@ open class KotlinFileExtractor(
)
}
if (c.codeQlValueArgumentsCount == 1) {
val dim = c.codeQlGetValueArgument(0)
if (c.valueArgumentsCount == 1) {
val dim = c.getValueArgument(0)
if (dim != null) {
extractExpressionExpr(dim, callable, id, 0, enclosingStmt)
} else {
@@ -4651,8 +4651,8 @@ open class KotlinFileExtractor(
c.type.getArrayElementTypeCodeQL(pluginContext.irBuiltIns)
} else {
// TODO: is there any reason not to always use getArrayElementTypeCodeQL?
if (c.codeQlTypeArgumentsCount == 1) {
c.codeQlGetTypeArgument(0).also {
if (c.typeArgumentsCount == 1) {
c.getTypeArgument(0).also {
if (it == null) {
logger.errorElement(
"Type argument missing in an arrayOf call",
@@ -4670,7 +4670,7 @@ open class KotlinFileExtractor(
}
val arg =
if (c.codeQlValueArgumentsCount == 1) c.codeQlGetValueArgument(0)
if (c.valueArgumentsCount == 1) c.getValueArgument(0)
else {
logger.errorElement(
"Expected to find only one (vararg) argument in ${c.symbol.owner.name.asString()} call",
@@ -4719,7 +4719,7 @@ open class KotlinFileExtractor(
return
}
val ext = c.codeQlExtensionReceiver
val ext = c.extensionReceiver
if (ext == null) {
logger.errorElement(
"No extension receiver found for `KClass::java` call",
@@ -4826,8 +4826,8 @@ open class KotlinFileExtractor(
c.origin == IrStatementOrigin.EQ &&
c.dispatchReceiver != null -> {
val array = c.dispatchReceiver
val arrayIdx = c.codeQlGetValueArgument(0)
val assignedValue = c.codeQlGetValueArgument(1)
val arrayIdx = c.getValueArgument(0)
val assignedValue = c.getValueArgument(1)
if (array != null && arrayIdx != null && assignedValue != null) {
@@ -4882,22 +4882,22 @@ open class KotlinFileExtractor(
}
isBuiltinCall(c, "<unsafe-coerce>", "kotlin.jvm.internal") -> {
if (c.codeQlValueArgumentsCount != 1) {
if (c.valueArgumentsCount != 1) {
logger.errorElement(
"Expected to find one argument for a kotlin.jvm.internal.<unsafe-coerce>() call, but found ${c.codeQlValueArgumentsCount}",
"Expected to find one argument for a kotlin.jvm.internal.<unsafe-coerce>() call, but found ${c.valueArgumentsCount}",
c
)
return
}
if (c.codeQlTypeArgumentsCount != 2) {
if (c.typeArgumentsCount != 2) {
logger.errorElement(
"Expected to find two type arguments for a kotlin.jvm.internal.<unsafe-coerce>() call, but found ${c.codeQlTypeArgumentsCount}",
"Expected to find two type arguments for a kotlin.jvm.internal.<unsafe-coerce>() call, but found ${c.typeArgumentsCount}",
c
)
return
}
val valueArg = c.codeQlGetValueArgument(0)
val valueArg = c.getValueArgument(0)
if (valueArg == null) {
logger.errorElement(
"Cannot find value argument for a kotlin.jvm.internal.<unsafe-coerce>() call",
@@ -4905,7 +4905,7 @@ open class KotlinFileExtractor(
)
return
}
val typeArg = c.codeQlGetTypeArgument(1)
val typeArg = c.getTypeArgument(1)
if (typeArg == null) {
logger.errorElement(
"Cannot find type argument for a kotlin.jvm.internal.<unsafe-coerce>() call",
@@ -4924,7 +4924,7 @@ open class KotlinFileExtractor(
extractExpressionExpr(valueArg, callable, id, 1, enclosingStmt)
}
isBuiltinCallInternal(c, "dataClassArrayMemberToString") -> {
val arrayArg = c.codeQlGetValueArgument(0)
val arrayArg = c.getValueArgument(0)
val realArrayClass = arrayArg?.type?.classOrNull
if (realArrayClass == null) {
logger.errorElement(
@@ -4936,8 +4936,8 @@ open class KotlinFileExtractor(
val realCallee =
javaUtilArrays?.declarations?.findSubType<IrFunction> { decl ->
decl.name.asString() == "toString" &&
decl.codeQlValueParameters.size == 1 &&
decl.codeQlValueParameters[0].type.classOrNull?.let {
decl.valueParameters.size == 1 &&
decl.valueParameters[0].type.classOrNull?.let {
it == realArrayClass
} == true
}
@@ -4962,7 +4962,7 @@ open class KotlinFileExtractor(
}
}
isBuiltinCallInternal(c, "dataClassArrayMemberHashCode") -> {
val arrayArg = c.codeQlGetValueArgument(0)
val arrayArg = c.getValueArgument(0)
val realArrayClass = arrayArg?.type?.classOrNull
if (realArrayClass == null) {
logger.errorElement(
@@ -4974,8 +4974,8 @@ open class KotlinFileExtractor(
val realCallee =
javaUtilArrays?.declarations?.findSubType<IrFunction> { decl ->
decl.name.asString() == "hashCode" &&
decl.codeQlValueParameters.size == 1 &&
decl.codeQlValueParameters[0].type.classOrNull?.let {
decl.valueParameters.size == 1 &&
decl.valueParameters[0].type.classOrNull?.let {
it == realArrayClass
} == true
}
@@ -5155,7 +5155,7 @@ open class KotlinFileExtractor(
val type = useType(eType)
val isAnonymous = eType.isAnonymous
val locId = tw.getLocation(e)
val valueArgs = (0 until e.codeQlValueArgumentsCount).map { e.codeQlGetValueArgument(it) }
val valueArgs = (0 until e.valueArgumentsCount).map { e.getValueArgument(it) }
val id =
if (
@@ -5211,10 +5211,10 @@ open class KotlinFileExtractor(
realCallTarget is IrConstructor &&
realCallTarget.parentClassOrNull?.fqNameWhenAvailable?.asString() ==
"kotlin.Enum" &&
realCallTarget.codeQlValueParameters.size == 2 &&
realCallTarget.codeQlValueParameters[0].type ==
realCallTarget.valueParameters.size == 2 &&
realCallTarget.valueParameters[0].type ==
pluginContext.irBuiltIns.stringType &&
realCallTarget.codeQlValueParameters[1].type == pluginContext.irBuiltIns.intType
realCallTarget.valueParameters[1].type == pluginContext.irBuiltIns.intType
) {
val id0 =
@@ -5287,7 +5287,7 @@ open class KotlinFileExtractor(
}
val args =
(0 until e.codeQlTypeArgumentsCount).map { e.codeQlGetTypeArgument(it) }.requireNoNullsOrNull()
(0 until e.typeArgumentsCount).map { e.getTypeArgument(it) }.requireNoNullsOrNull()
if (args == null) {
logger.warnElement("Found null type argument in enum constructor call", e)
return
@@ -5365,7 +5365,7 @@ open class KotlinFileExtractor(
// Check for an expression like x = get(x).op(e):
val opReceiver = updateRhs.dispatchReceiver
if (isExpectedLhs(opReceiver)) {
updateRhs.codeQlGetValueArgument(0)
updateRhs.getValueArgument(0)
} else null
} else null
}
@@ -5560,7 +5560,7 @@ open class KotlinFileExtractor(
"set"
)
) {
val updateRhs0 = arraySetCall.codeQlGetValueArgument(1)
val updateRhs0 = arraySetCall.getValueArgument(1)
if (updateRhs0 == null) {
logger.errorElement("Update RHS not found", e)
return false
@@ -6403,12 +6403,12 @@ open class KotlinFileExtractor(
val ids = getLocallyVisibleFunctionLabels(e.function)
val locId = tw.getLocation(e)
val ext = e.function.codeQlExtensionReceiverParameter
val ext = e.function.extensionReceiverParameter
val parameters =
if (ext != null) {
listOf(ext) + e.function.codeQlValueParameters
listOf(ext) + e.function.valueParameters
} else {
e.function.codeQlValueParameters
e.function.valueParameters
}
var types = parameters.map { it.type }
@@ -6670,7 +6670,7 @@ open class KotlinFileExtractor(
is IrFunction -> {
if (
ownerParent.dispatchReceiverParameter == owner &&
ownerParent.codeQlExtensionReceiverParameter != null
ownerParent.extensionReceiverParameter != null
) {
val ownerParent2 = ownerParent.parent
@@ -7089,7 +7089,7 @@ open class KotlinFileExtractor(
makeReceiverInfo(callableReferenceExpr.dispatchReceiver, 0)
private val extensionReceiverInfo =
makeReceiverInfo(
callableReferenceExpr.codeQlExtensionReceiver,
callableReferenceExpr.extensionReceiver,
if (dispatchReceiverInfo == null) 0 else 1
)
@@ -7627,8 +7627,8 @@ open class KotlinFileExtractor(
}
val expressionTypeArguments =
(0 until propertyReferenceExpr.codeQlTypeArgumentsCount).mapNotNull {
propertyReferenceExpr.codeQlGetTypeArgument(it)
(0 until propertyReferenceExpr.typeArgumentsCount).mapNotNull {
propertyReferenceExpr.getTypeArgument(it)
}
val idPropertyRef = tw.getFreshIdLabel<DbPropertyref>()
@@ -7829,7 +7829,7 @@ open class KotlinFileExtractor(
if (
functionReferenceExpr.dispatchReceiver != null &&
functionReferenceExpr.codeQlExtensionReceiver != null
functionReferenceExpr.extensionReceiver != null
) {
logger.errorElement(
"Unexpected: dispatchReceiver and extensionReceiver are both non-null",
@@ -7840,7 +7840,7 @@ open class KotlinFileExtractor(
if (
target.owner.dispatchReceiverParameter != null &&
target.owner.codeQlExtensionReceiverParameter != null
target.owner.extensionReceiverParameter != null
) {
logger.errorElement(
"Unexpected: dispatch and extension parameters are both non-null",
@@ -7899,8 +7899,8 @@ open class KotlinFileExtractor(
null
}
expressionTypeArguments =
(0 until functionReferenceExpr.codeQlTypeArgumentsCount).mapNotNull {
functionReferenceExpr.codeQlGetTypeArgument(it)
(0 until functionReferenceExpr.typeArgumentsCount).mapNotNull {
functionReferenceExpr.getTypeArgument(it)
}
dispatchReceiverIdx = -1
}
@@ -7965,7 +7965,7 @@ open class KotlinFileExtractor(
functionReferenceExpr,
declarationParent,
null,
{ it.codeQlValueParameters.size == 1 }
{ it.valueParameters.size == 1 }
) {
// The argument to FunctionReference's constructor is the function arity.
extractConstantInteger(
@@ -8572,7 +8572,7 @@ open class KotlinFileExtractor(
reverse: Boolean = false
) {
val typeArguments =
(0 until c.codeQlTypeArgumentsCount).map { c.codeQlGetTypeArgument(it) }.requireNoNullsOrNull()
(0 until c.typeArgumentsCount).map { c.getTypeArgument(it) }.requireNoNullsOrNull()
if (typeArguments == null) {
logger.errorElement("Found a null type argument for a member access expression", c)
} else {
@@ -8923,11 +8923,11 @@ open class KotlinFileExtractor(
tw.writeVariableBinding(lhsId, fieldId)
val parameters = mutableListOf<IrValueParameter>()
val extParam = samMember.codeQlExtensionReceiverParameter
val extParam = samMember.extensionReceiverParameter
if (extParam != null) {
parameters.add(extParam)
}
parameters.addAll(samMember.codeQlValueParameters)
parameters.addAll(samMember.valueParameters)
fun extractArgument(
p: IrValueParameter,
@@ -9032,7 +9032,7 @@ open class KotlinFileExtractor(
elementToReportOn: IrElement,
declarationParent: IrDeclarationParent,
compilerGeneratedKindOverride: CompilerGeneratedKinds? = null,
superConstructorSelector: (IrFunction) -> Boolean = { it.codeQlValueParameters.isEmpty() },
superConstructorSelector: (IrFunction) -> Boolean = { it.valueParameters.isEmpty() },
extractSuperconstructorArgs: (Label<DbSuperconstructorinvocationstmt>) -> Unit = {},
): Label<out DbClassorinterface> {
// Write class

View File

@@ -12,7 +12,7 @@ import org.jetbrains.kotlin.ir.ObsoleteDescriptorBasedAPI
import org.jetbrains.kotlin.ir.declarations.*
import org.jetbrains.kotlin.ir.expressions.*
import org.jetbrains.kotlin.ir.symbols.*
import com.github.codeql.utils.versions.codeQlAddAnnotations
import org.jetbrains.kotlin.ir.types.addAnnotations
import org.jetbrains.kotlin.ir.types.classFqName
import org.jetbrains.kotlin.ir.types.classifierOrNull
import org.jetbrains.kotlin.ir.types.classOrNull
@@ -355,7 +355,7 @@ open class KotlinUsesExtractor(
}
private fun propertySignature(p: IrProperty) =
((p.getter ?: p.setter)?.codeQlExtensionReceiverParameter?.let {
((p.getter ?: p.setter)?.extensionReceiverParameter?.let {
useType(erase(it.type)).javaResult.signature
} ?: "")
@@ -368,7 +368,7 @@ open class KotlinUsesExtractor(
// useDeclarationParent -> useFunction
// -> extractFunctionLaterIfExternalFileMember, which would result for `fun <T> f(t:
// T) { ... }` for example.
(listOfNotNull(d.codeQlExtensionReceiverParameter) + d.codeQlValueParameters)
(listOfNotNull(d.extensionReceiverParameter) + d.valueParameters)
.map { useType(erase(it.type)).javaResult.signature }
.joinToString(separator = ",", prefix = "(", postfix = ")")
is IrProperty -> propertySignature(d) + externalClassExtractor.propertySignature
@@ -488,8 +488,8 @@ open class KotlinUsesExtractor(
val result =
replacementClass.declarations.findSubType<IrSimpleFunction> { replacementDecl ->
replacementDecl.name == f.name &&
replacementDecl.codeQlValueParameters.size == f.codeQlValueParameters.size &&
replacementDecl.codeQlValueParameters.zip(f.codeQlValueParameters).all {
replacementDecl.valueParameters.size == f.valueParameters.size &&
replacementDecl.valueParameters.zip(f.valueParameters).all {
erase(it.first.type) == erase(it.second.type)
}
}
@@ -1265,7 +1265,7 @@ open class KotlinUsesExtractor(
private fun getWildcardSuppressionDirective(t: IrAnnotationContainer): Boolean? =
t.getAnnotation(jvmWildcardSuppressionAnnotation)?.let {
@Suppress("USELESS_CAST") // `as? Boolean` is not needed for Kotlin < 2.1
(it.codeQlGetValueArgument(0) as? CodeQLIrConst<Boolean>)?.value as? Boolean ?: true
(it.getValueArgument(0) as? CodeQLIrConst<Boolean>)?.value as? Boolean ?: true
}
private fun addJavaLoweringArgumentWildcards(
@@ -1376,9 +1376,9 @@ open class KotlinUsesExtractor(
f.parent,
parentId,
getFunctionShortName(f).nameInDB,
(maybeParameterList ?: f.codeQlValueParameters).map { it.type },
(maybeParameterList ?: f.valueParameters).map { it.type },
getAdjustedReturnType(f),
f.codeQlExtensionReceiverParameter?.type,
f.extensionReceiverParameter?.type,
getFunctionTypeParameters(f),
classTypeArgsIncludingOuterClasses,
overridesCollectionsMethodWithAlteredParameterTypes(f),
@@ -1401,12 +1401,12 @@ open class KotlinUsesExtractor(
// The name of the function; normally f.name.asString().
name: String,
// The types of the value parameters that the functions takes; normally
// f.codeQlValueParameters.map { it.type }.
// f.valueParameters.map { it.type }.
parameterTypes: List<IrType>,
// The return type of the function; normally f.returnType.
returnType: IrType,
// The extension receiver of the function, if any; normally
// f.codeQlExtensionReceiverParameter?.type.
// f.extensionReceiverParameter?.type.
extensionParamType: IrType?,
// The type parameters of the function. This does not include type parameters of enclosing
// classes.
@@ -1579,7 +1579,7 @@ open class KotlinUsesExtractor(
parentClass.fqNameWhenAvailable?.asString() !=
"java.util.concurrent.ConcurrentHashMap" ||
getFunctionShortName(f).nameInDB != "keySet" ||
f.codeQlValueParameters.isNotEmpty() ||
f.valueParameters.isNotEmpty() ||
f.returnType.classFqName?.asString() != "kotlin.collections.MutableSet"
) {
return f.returnType
@@ -1587,7 +1587,7 @@ open class KotlinUsesExtractor(
val otherKeySet =
parentClass.declarations.findSubType<IrFunction> {
it.name.asString() == "keySet" && it.codeQlValueParameters.size == 1
it.name.asString() == "keySet" && it.valueParameters.size == 1
} ?: return f.returnType
return otherKeySet.returnType.codeQlWithHasQuestionMark(false)
@@ -1695,8 +1695,8 @@ open class KotlinUsesExtractor(
javaClass.declarations.findSubType<IrFunction> { decl ->
!decl.isFakeOverride &&
decl.name.asString() == jvmName &&
decl.codeQlValueParameters.size == f.codeQlValueParameters.size &&
decl.codeQlValueParameters.zip(f.codeQlValueParameters).all { p ->
decl.valueParameters.size == f.valueParameters.size &&
decl.valueParameters.zip(f.valueParameters).all { p ->
erase(p.first.type).classifierOrNull ==
erase(p.second.type).classifierOrNull
}
@@ -2125,7 +2125,7 @@ open class KotlinUsesExtractor(
}
return if (t.arguments.isNotEmpty())
t.codeQlAddAnnotations(listOf(RawTypeAnnotation.annotationConstructor))
t.addAnnotations(listOf(RawTypeAnnotation.annotationConstructor))
else t
}
}
@@ -2153,7 +2153,7 @@ open class KotlinUsesExtractor(
val idxOffset =
if (
declarationParent is IrFunction &&
declarationParent.codeQlExtensionReceiverParameter != null
declarationParent.extensionReceiverParameter != null
)
// For extension functions increase the index to match what the java extractor sees:
1
@@ -2187,7 +2187,7 @@ open class KotlinUsesExtractor(
// Gets a field's corresponding property's extension receiver type, if any
fun getExtensionReceiverType(f: IrField) =
f.correspondingPropertySymbol?.owner?.let {
(it.getter ?: it.setter)?.codeQlExtensionReceiverParameter?.type
(it.getter ?: it.setter)?.extensionReceiverParameter?.type
}
fun getFieldLabel(f: IrField): String {
@@ -2222,14 +2222,14 @@ open class KotlinUsesExtractor(
val setter = p.setter
val func = getter ?: setter
val ext = func?.codeQlExtensionReceiverParameter
val ext = func?.extensionReceiverParameter
return if (ext == null) {
"@\"property;{$parentId};${p.name.asString()}\""
} else {
val returnType =
getter?.returnType
?: setter?.codeQlValueParameters?.singleOrNull()?.type
?: setter?.valueParameters?.singleOrNull()?.type
?: pluginContext.irBuiltIns.unitType
val typeParams = getFunctionTypeParameters(func)

View File

@@ -1,10 +1,5 @@
package com.github.codeql
import com.github.codeql.utils.versions.codeQlAnnotationFromSymbolOwner
import com.github.codeql.utils.versions.codeQlGetValueArgument
import com.github.codeql.utils.versions.codeQlPutValueArgument
import com.github.codeql.utils.versions.codeQlSetAnnotations
import com.github.codeql.utils.versions.codeQlSetDispatchReceiverParameter
import com.github.codeql.utils.versions.createImplicitParameterDeclarationWithWrappedDescriptor
import java.lang.annotation.ElementType
import java.util.HashSet
@@ -100,7 +95,7 @@ class MetaAnnotationSupport(
JvmAnnotationNames.REPEATABLE_ANNOTATION
}
return if (jvmRepeatable != null) {
((jvmRepeatable.codeQlGetValueArgument(0) as? IrClassReference)?.symbol as? IrClassSymbol)
((jvmRepeatable.getValueArgument(0) as? IrClassReference)?.symbol as? IrClassSymbol)
?.owner
} else {
getOrCreateSyntheticRepeatableAnnotationContainer(annotationClass)
@@ -122,12 +117,12 @@ class MetaAnnotationSupport(
)
return null
} else {
return codeQlAnnotationFromSymbolOwner(
return IrConstructorCallImpl.fromSymbolOwner(
containerClass.defaultType,
containerConstructor.symbol
)
.apply {
codeQlPutValueArgument(
putValueArgument(
0,
IrVarargImpl(
UNDEFINED_OFFSET,
@@ -149,7 +144,7 @@ class MetaAnnotationSupport(
// Taken from AdditionalClassAnnotationLowering.kt
private fun loadAnnotationTargets(targetEntry: IrConstructorCall): Set<KotlinTarget>? {
val valueArgument = targetEntry.codeQlGetValueArgument(0) as? IrVararg ?: return null
val valueArgument = targetEntry.getValueArgument(0) as? IrVararg ?: return null
return valueArgument.elements
.filterIsInstance<IrGetEnumValue>()
.mapNotNull { KotlinTarget.valueOrNull(it.symbol.owner.name.asString()) }
@@ -235,14 +230,14 @@ class MetaAnnotationSupport(
)
}
return codeQlAnnotationFromSymbolOwner(
return IrConstructorCallImpl.fromSymbolOwner(
UNDEFINED_OFFSET,
UNDEFINED_OFFSET,
targetConstructor.returnType,
targetConstructor.symbol,
0
)
.apply { codeQlPutValueArgument(0, vararg) }
.apply { putValueArgument(0, vararg) }
}
private val javaAnnotationRetention by lazy {
@@ -268,7 +263,7 @@ class MetaAnnotationSupport(
// Taken from AnnotationCodegen.kt (not available in Kotlin < 1.6.20)
private fun IrClass.getAnnotationRetention(): KotlinRetention? {
val retentionArgument =
getAnnotation(StandardNames.FqNames.retention)?.codeQlGetValueArgument(0) as? IrGetEnumValue
getAnnotation(StandardNames.FqNames.retention)?.getValueArgument(0) as? IrGetEnumValue
?: return null
val retentionArgumentValue = retentionArgument.symbol.owner
return KotlinRetention.valueOf(retentionArgumentValue.name.asString())
@@ -288,7 +283,7 @@ class MetaAnnotationSupport(
val targetConstructor =
retentionType.declarations.firstIsInstanceOrNull<IrConstructor>() ?: return null
return codeQlAnnotationFromSymbolOwner(
return IrConstructorCallImpl.fromSymbolOwner(
UNDEFINED_OFFSET,
UNDEFINED_OFFSET,
targetConstructor.returnType,
@@ -296,7 +291,7 @@ class MetaAnnotationSupport(
0
)
.apply {
codeQlPutValueArgument(
putValueArgument(
0,
IrGetEnumValueImpl(
UNDEFINED_OFFSET,
@@ -338,7 +333,7 @@ class MetaAnnotationSupport(
return
}
val newParam = thisReceiever.copyTo(this)
codeQlSetDispatchReceiverParameter(newParam)
dispatchReceiverParameter = newParam
body =
factory
.createBlockBody(UNDEFINED_OFFSET, UNDEFINED_OFFSET)
@@ -411,7 +406,7 @@ class MetaAnnotationSupport(
val repeatableContainerAnnotation =
kotlinAnnotationRepeatableContainer?.constructors?.single()
codeQlSetAnnotations(containerClass,
containerClass.annotations =
annotationClass.annotations
.filter {
it.isAnnotationWithEqualFqName(StandardNames.FqNames.retention) ||
@@ -420,7 +415,7 @@ class MetaAnnotationSupport(
.map { it.deepCopyWithSymbols(containerClass) } +
listOfNotNull(
repeatableContainerAnnotation?.let {
codeQlAnnotationFromSymbolOwner(
IrConstructorCallImpl.fromSymbolOwner(
UNDEFINED_OFFSET,
UNDEFINED_OFFSET,
it.returnType,
@@ -429,7 +424,6 @@ class MetaAnnotationSupport(
)
}
)
)
containerClass
}
@@ -468,14 +462,14 @@ class MetaAnnotationSupport(
containerClass.symbol,
containerClass.defaultType
)
return codeQlAnnotationFromSymbolOwner(
return IrConstructorCallImpl.fromSymbolOwner(
UNDEFINED_OFFSET,
UNDEFINED_OFFSET,
repeatableConstructor.returnType,
repeatableConstructor.symbol,
0
)
.apply { codeQlPutValueArgument(0, containerReference) }
.apply { putValueArgument(0, containerReference) }
}
private val javaAnnotationDocumented by lazy {
@@ -494,7 +488,7 @@ class MetaAnnotationSupport(
javaAnnotationDocumented?.declarations?.firstIsInstanceOrNull<IrConstructor>()
?: return null
return codeQlAnnotationFromSymbolOwner(
return IrConstructorCallImpl.fromSymbolOwner(
UNDEFINED_OFFSET,
UNDEFINED_OFFSET,
documentedConstructor.returnType,

View File

@@ -1,7 +1,6 @@
package com.github.codeql
import com.github.codeql.KotlinUsesExtractor.LocallyVisibleFunctionLabels
import com.github.codeql.utils.versions.codeQlExtensionReceiver
import com.semmle.extractor.java.PopulateFile
import com.semmle.util.unicode.UTF8Util
import java.io.BufferedWriter
@@ -332,7 +331,7 @@ open class FileTrapWriter(
is IrCall -> {
// Calls have incorrect startOffset, so we adjust them:
val dr = e.dispatchReceiver?.let { getStartOffset(it) }
val er = e.codeQlExtensionReceiver?.let { getStartOffset(it) }
val er = e.extensionReceiver?.let { getStartOffset(it) }
offsetMinOf(e.startOffset, dr, er)
}
else -> e.startOffset

View File

@@ -2,7 +2,6 @@ package com.github.codeql.comments
import com.github.codeql.*
import com.github.codeql.utils.isLocalFunction
import com.github.codeql.utils.versions.codeQlExtensionReceiverParameter
import com.github.codeql.utils.versions.isDispatchReceiver
import org.jetbrains.kotlin.ir.IrElement
import org.jetbrains.kotlin.ir.declarations.*
@@ -12,7 +11,7 @@ import org.jetbrains.kotlin.ir.util.parentClassOrNull
private fun IrValueParameter.isExtensionReceiver(): Boolean {
val parentFun = parent as? IrFunction ?: return false
return parentFun.codeQlExtensionReceiverParameter == this
return parentFun.extensionReceiverParameter == this
}
open class CommentExtractor(

View File

@@ -1,8 +1,6 @@
package com.github.codeql.utils
import com.github.codeql.utils.versions.CodeQLIrConst
import com.github.codeql.utils.versions.codeQlGetValueArgument
import com.github.codeql.utils.versions.codeQlValueArgumentsCount
import org.jetbrains.kotlin.builtins.StandardNames
import org.jetbrains.kotlin.ir.declarations.IrAnnotationContainer
import org.jetbrains.kotlin.ir.declarations.IrClass
@@ -78,9 +76,9 @@ private fun getSpecialJvmName(f: IrFunction): String? {
fun getJvmName(container: IrAnnotationContainer): String? {
for (a: IrConstructorCall in container.annotations) {
val t = a.type
if (t is IrSimpleType && a.codeQlValueArgumentsCount == 1) {
if (t is IrSimpleType && a.valueArgumentsCount == 1) {
val owner = t.classifier.owner
val v = a.codeQlGetValueArgument(0)
val v = a.getValueArgument(0)
if (owner is IrClass) {
val aPkg = owner.packageFqName?.asString()
val name = owner.name.asString()

View File

@@ -18,7 +18,7 @@ import org.jetbrains.kotlin.ir.expressions.IrConstructorCall
import org.jetbrains.kotlin.ir.expressions.impl.*
import org.jetbrains.kotlin.ir.symbols.IrTypeParameterSymbol
import org.jetbrains.kotlin.ir.symbols.impl.DescriptorlessExternalPackageFragmentSymbol
import com.github.codeql.utils.versions.codeQlAddAnnotations
import org.jetbrains.kotlin.ir.types.addAnnotations
import org.jetbrains.kotlin.ir.types.classifierOrNull
import org.jetbrains.kotlin.ir.types.makeNotNull
import org.jetbrains.kotlin.ir.types.makeNullable
@@ -192,7 +192,7 @@ object RawTypeAnnotation {
addConstructor { isPrimary = true }
}
val constructor = annoClass.constructors.single()
codeQlAnnotationFromSymbolOwner(constructor.constructedClassType, constructor.symbol)
IrConstructorCallImpl.fromSymbolOwner(constructor.constructedClassType, constructor.symbol)
}
}
@@ -202,7 +202,7 @@ fun IrType.toRawType(): IrType =
when (val owner = this.classifier.owner) {
is IrClass -> {
if (this.arguments.isNotEmpty())
this.codeQlAddAnnotations(listOf(RawTypeAnnotation.annotationConstructor))
this.addAnnotations(listOf(RawTypeAnnotation.annotationConstructor))
else this
}
is IrTypeParameter -> owner.superTypes[0].toRawType()
@@ -215,7 +215,7 @@ fun IrType.toRawType(): IrType =
fun IrClass.toRawType(): IrType {
val result = this.typeWith(listOf())
return if (this.typeParameters.isNotEmpty())
result.codeQlAddAnnotations(listOf(RawTypeAnnotation.annotationConstructor))
result.addAnnotations(listOf(RawTypeAnnotation.annotationConstructor))
else result
}

View File

@@ -1,70 +0,0 @@
package com.github.codeql.utils.versions
import org.jetbrains.kotlin.ir.declarations.IrFunction
import org.jetbrains.kotlin.ir.declarations.IrValueParameter
import org.jetbrains.kotlin.ir.expressions.IrConstructorCall
import org.jetbrains.kotlin.ir.expressions.IrExpression
import org.jetbrains.kotlin.ir.expressions.IrMemberAccessExpression
import org.jetbrains.kotlin.ir.expressions.impl.*
import org.jetbrains.kotlin.ir.symbols.IrConstructorSymbol
import org.jetbrains.kotlin.ir.types.IrType
import org.jetbrains.kotlin.ir.types.addAnnotations
/**
* Compatibility accessors for pre-2.4.0 API patterns.
* In pre-2.4.0 versions, these delegate directly to the existing APIs.
*/
// IrFunction: valueParameters
val IrFunction.codeQlValueParameters: List<IrValueParameter>
get() = valueParameters
// IrFunction: extensionReceiverParameter
val IrFunction.codeQlExtensionReceiverParameter: IrValueParameter?
get() = extensionReceiverParameter
// IrMemberAccessExpression: valueArgumentsCount
val IrMemberAccessExpression<*>.codeQlValueArgumentsCount: Int
get() = valueArgumentsCount
// IrMemberAccessExpression: getValueArgument
fun IrMemberAccessExpression<*>.codeQlGetValueArgument(index: Int): IrExpression? = getValueArgument(index)
// IrMemberAccessExpression: putValueArgument
fun IrMemberAccessExpression<*>.codeQlPutValueArgument(index: Int, value: IrExpression?) {
putValueArgument(index, value)
}
// IrMemberAccessExpression: extensionReceiver
val IrMemberAccessExpression<*>.codeQlExtensionReceiver: IrExpression?
get() = extensionReceiver
// IrMemberAccessExpression: typeArgumentsCount
val IrMemberAccessExpression<*>.codeQlTypeArgumentsCount: Int
get() = typeArgumentsCount
// IrMemberAccessExpression: getTypeArgument
fun IrMemberAccessExpression<*>.codeQlGetTypeArgument(index: Int): IrType? = getTypeArgument(index)
// addAnnotations compat: in pre-2.4.0, addAnnotations expects List<IrConstructorCall>
fun IrType.codeQlAddAnnotations(annotations: List<IrConstructorCall>): IrType =
addAnnotations(annotations)
// IrMutableAnnotationContainer.annotations setter: in pre-2.4.0, annotations is var with List<IrConstructorCall>
fun codeQlSetAnnotations(container: org.jetbrains.kotlin.ir.declarations.IrMutableAnnotationContainer, annotations: List<IrConstructorCall>) {
container.annotations = annotations
}
// IrFunction: set dispatch receiver parameter (pre-2.4.0 it's a var)
fun IrFunction.codeQlSetDispatchReceiverParameter(param: IrValueParameter?) {
dispatchReceiverParameter = param
}
// In pre-2.4.0, annotations are List<IrConstructorCall> so IrConstructorCallImpl works directly.
fun codeQlAnnotationFromSymbolOwner(
startOffset: Int, endOffset: Int, type: IrType, symbol: IrConstructorSymbol, typeArgumentsCount: Int
): IrConstructorCall =
IrConstructorCallImpl.fromSymbolOwner(startOffset, endOffset, type, symbol, typeArgumentsCount)
fun codeQlAnnotationFromSymbolOwner(type: IrType, symbol: IrConstructorSymbol): IrConstructorCall =
IrConstructorCallImpl.fromSymbolOwner(type, symbol)

View File

@@ -3,32 +3,10 @@
package com.github.codeql
import com.intellij.mock.MockProject
import com.intellij.openapi.extensions.LoadingOrder
import org.jetbrains.kotlin.backend.common.extensions.IrGenerationExtension
import org.jetbrains.kotlin.compiler.plugin.ComponentRegistrar
import org.jetbrains.kotlin.compiler.plugin.ExperimentalCompilerApi
import org.jetbrains.kotlin.config.CompilerConfiguration
@OptIn(ExperimentalCompilerApi::class)
abstract class Kotlin2ComponentRegistrar : ComponentRegistrar {
/* Nothing to do; supportsK2 doesn't exist yet. */
private var project: MockProject? = null
override fun registerProjectComponents(
project: MockProject,
configuration: CompilerConfiguration
) {
this.project = project
doRegisterExtensions(configuration)
}
abstract fun doRegisterExtensions(configuration: CompilerConfiguration)
fun registerExtractorExtension(extension: IrGenerationExtension) {
val p = project ?: throw IllegalStateException("registerExtractorExtension called before registerProjectComponents")
val extensionPoint = p.extensionArea.getExtensionPoint(IrGenerationExtension.extensionPointName)
extensionPoint.registerExtension(extension, LoadingOrder.LAST, p)
}
}

View File

@@ -3,35 +3,11 @@
package com.github.codeql
import com.intellij.mock.MockProject
import com.intellij.openapi.extensions.LoadingOrder
import org.jetbrains.kotlin.backend.common.extensions.IrGenerationExtension
import org.jetbrains.kotlin.compiler.plugin.ComponentRegistrar
import org.jetbrains.kotlin.compiler.plugin.ExperimentalCompilerApi
import org.jetbrains.kotlin.config.CompilerConfiguration
@OptIn(ExperimentalCompilerApi::class)
abstract class Kotlin2ComponentRegistrar : ComponentRegistrar {
override val supportsK2: Boolean
get() = true
private var project: MockProject? = null
override fun registerProjectComponents(
project: MockProject,
configuration: CompilerConfiguration
) {
this.project = project
doRegisterExtensions(configuration)
}
abstract fun doRegisterExtensions(configuration: CompilerConfiguration)
fun registerExtractorExtension(extension: IrGenerationExtension) {
val p = project ?: throw IllegalStateException("registerExtractorExtension called before registerProjectComponents")
// Register with LoadingOrder.LAST to ensure the extractor runs after other
// IR generation plugins (like kotlinx.serialization) have generated their code.
val extensionPoint = p.extensionArea.getExtensionPoint(IrGenerationExtension.extensionPointName)
extensionPoint.registerExtension(extension, LoadingOrder.LAST, p)
}
}

View File

@@ -1,121 +0,0 @@
@file:Suppress("DEPRECATION")
package com.github.codeql.utils.versions
import org.jetbrains.kotlin.ir.declarations.IrFunction
import org.jetbrains.kotlin.ir.declarations.IrValueParameter
import org.jetbrains.kotlin.ir.expressions.IrAnnotation
import org.jetbrains.kotlin.ir.expressions.IrConstructorCall
import org.jetbrains.kotlin.ir.expressions.IrExpression
import org.jetbrains.kotlin.ir.expressions.IrMemberAccessExpression
import org.jetbrains.kotlin.ir.expressions.impl.IrAnnotationImpl
import org.jetbrains.kotlin.ir.expressions.impl.fromSymbolOwner
import org.jetbrains.kotlin.ir.symbols.IrConstructorSymbol
import org.jetbrains.kotlin.ir.types.IrType
import org.jetbrains.kotlin.ir.types.addAnnotations
/**
* Compatibility accessors for pre-2.4.0 API patterns.
* In 2.4.0, valueParameters/extensionReceiverParameter/extensionReceiver/
* getValueArgument/putValueArgument/valueArgumentsCount/typeArgumentsCount/getTypeArgument
* have been removed. This file provides the 2.4.0 implementations.
*/
// IrFunction: valueParameters -> parameters filtered to Regular kind
val IrFunction.codeQlValueParameters: List<IrValueParameter>
get() = parameters.filter { it.kind == org.jetbrains.kotlin.ir.declarations.IrParameterKind.Regular }
// IrFunction: extensionReceiverParameter
val IrFunction.codeQlExtensionReceiverParameter: IrValueParameter?
get() = parameters.firstOrNull { it.kind == org.jetbrains.kotlin.ir.declarations.IrParameterKind.ExtensionReceiver }
// Helper: get the offset of value arguments in the arguments list
// In 2.4.0, arguments[] includes dispatch/extension receivers before regular params
private fun IrMemberAccessExpression<*>.valueArgumentOffset(): Int {
val owner = symbol.owner as? IrFunction ?: return 0
return owner.parameters.count { it.kind != org.jetbrains.kotlin.ir.declarations.IrParameterKind.Regular }
}
// IrMemberAccessExpression: valueArgumentsCount
val IrMemberAccessExpression<*>.codeQlValueArgumentsCount: Int
get() = arguments.size - valueArgumentOffset()
// IrMemberAccessExpression: getValueArgument
fun IrMemberAccessExpression<*>.codeQlGetValueArgument(index: Int): IrExpression? = arguments[index + valueArgumentOffset()]
// IrMemberAccessExpression: putValueArgument
fun IrMemberAccessExpression<*>.codeQlPutValueArgument(index: Int, value: IrExpression?) {
arguments[index + valueArgumentOffset()] = value
}
// IrMemberAccessExpression: extensionReceiver
// For IrCall/IrFunctionReference, look at symbol.owner (IrFunction) directly.
// For IrPropertyReference, symbol.owner is IrProperty; use the getter's parameters instead.
val IrMemberAccessExpression<*>.codeQlExtensionReceiver: IrExpression?
get() {
val erp = extensionReceiverParameterIndex() ?: return null
return arguments[erp]
}
private fun IrMemberAccessExpression<*>.extensionReceiverParameterIndex(): Int? {
// Direct function owner (IrCall, IrFunctionReference, etc.)
(symbol.owner as? IrFunction)?.codeQlExtensionReceiverParameter?.let {
return it.indexInParameters
}
// Property reference: look at getter or setter function
(this as? org.jetbrains.kotlin.ir.expressions.IrPropertyReference)?.let { propRef ->
propRef.getter?.owner?.codeQlExtensionReceiverParameter?.let {
return it.indexInParameters
}
propRef.setter?.owner?.codeQlExtensionReceiverParameter?.let {
return it.indexInParameters
}
}
return null
}
// IrMemberAccessExpression: typeArgumentsCount
val IrMemberAccessExpression<*>.codeQlTypeArgumentsCount: Int
get() = typeArguments.size
// IrMemberAccessExpression: getTypeArgument
fun IrMemberAccessExpression<*>.codeQlGetTypeArgument(index: Int): IrType? = typeArguments[index]
// addAnnotations compat: in 2.4.0, addAnnotations expects List<IrAnnotation>
// IrConstructorCall implements IrAnnotation in 2.4.0, so filterIsInstance is identity
fun IrType.codeQlAddAnnotations(annotations: List<IrConstructorCall>): IrType =
addAnnotations(annotations.filterIsInstance<IrAnnotation>())
// IrMutableAnnotationContainer.annotations setter: in 2.4.0, expects List<IrAnnotation>
fun codeQlSetAnnotations(container: org.jetbrains.kotlin.ir.declarations.IrMutableAnnotationContainer, annotations: List<IrConstructorCall>) {
container.annotations = annotations.filterIsInstance<IrAnnotation>()
}
// IrFunction: set dispatch receiver parameter
// In 2.4.0, dispatchReceiverParameter is val; modify the parameters list directly.
fun IrFunction.codeQlSetDispatchReceiverParameter(param: IrValueParameter?) {
val existing = parameters.indexOfFirst { it.kind == org.jetbrains.kotlin.ir.declarations.IrParameterKind.DispatchReceiver }
val mutableParams = parameters.toMutableList()
if (existing >= 0) {
if (param != null) {
mutableParams[existing] = param
} else {
mutableParams.removeAt(existing)
}
} else if (param != null) {
param.kind = org.jetbrains.kotlin.ir.declarations.IrParameterKind.DispatchReceiver
mutableParams.add(0, param)
}
parameters = mutableParams
}
// In 2.4.0, annotation lists require IrAnnotation instances.
// Use IrAnnotationImpl.fromSymbolOwner instead of IrConstructorCallImpl.fromSymbolOwner.
fun codeQlAnnotationFromSymbolOwner(
startOffset: Int, endOffset: Int, type: IrType, symbol: IrConstructorSymbol, typeArgumentsCount: Int
): IrConstructorCall =
IrAnnotationImpl.fromSymbolOwner(startOffset, endOffset, type, symbol, typeArgumentsCount)
fun codeQlAnnotationFromSymbolOwner(type: IrType, symbol: IrConstructorSymbol): IrConstructorCall =
IrAnnotationImpl.fromSymbolOwner(type, symbol)

View File

@@ -1,45 +0,0 @@
@file:Suppress("DEPRECATION", "DEPRECATION_ERROR")
@file:OptIn(ExperimentalCompilerApi::class)
package com.github.codeql
import com.intellij.mock.MockProject
import org.jetbrains.kotlin.backend.common.extensions.IrGenerationExtension
import org.jetbrains.kotlin.compiler.plugin.ComponentRegistrar
import org.jetbrains.kotlin.compiler.plugin.CompilerPluginRegistrar
import org.jetbrains.kotlin.compiler.plugin.ExperimentalCompilerApi
import org.jetbrains.kotlin.config.CompilerConfiguration
abstract class Kotlin2ComponentRegistrar : CompilerPluginRegistrar(), ComponentRegistrar {
override val supportsK2: Boolean
get() = true
override val pluginId: String
get() = "kotlin-extractor"
// ComponentRegistrar implementation (legacy path, still called by Kotlin compiler)
override fun registerProjectComponents(
project: MockProject,
configuration: CompilerConfiguration
) {
// In 2.4.0, we use CompilerPluginRegistrar path instead.
// This is only called if the compiler uses the ComponentRegistrar service file.
// We do nothing here since registerExtensions will be called separately.
}
private var extensionStorage: CompilerPluginRegistrar.ExtensionStorage? = null
override fun ExtensionStorage.registerExtensions(configuration: CompilerConfiguration) {
this@Kotlin2ComponentRegistrar.extensionStorage = this
doRegisterExtensions(configuration)
}
abstract fun doRegisterExtensions(configuration: CompilerConfiguration)
protected fun registerExtractorExtension(extension: IrGenerationExtension) {
val storage = extensionStorage ?: throw IllegalStateException("registerExtractorExtension called before registerExtensions")
with(storage) {
IrGenerationExtension.registerExtension(extension)
}
}
}

View File

@@ -1,13 +0,0 @@
package com.github.codeql.utils.versions
import org.jetbrains.kotlin.ir.declarations.IrFunction
import org.jetbrains.kotlin.ir.declarations.IrParameterKind
import org.jetbrains.kotlin.ir.declarations.IrValueParameter
fun parameterIndexExcludingReceivers(vp: IrValueParameter): Int {
val offset =
(vp.parent as? IrFunction)?.let { f ->
f.parameters.count { it.kind == IrParameterKind.DispatchReceiver || it.kind == IrParameterKind.ExtensionReceiver || it.kind == IrParameterKind.Context }
} ?: 0
return vp.indexInParameters - offset
}

View File

@@ -1 +0,0 @@
com.github.codeql.KotlinExtractorComponentRegistrar

View File

@@ -1,5 +1,8 @@
# when updating this list, `bazel mod tidy` should be run from `codeql` to update `MODULE.bazel`
VERSIONS = [
"1.8.0",
"1.9.0-Beta",
"1.9.20-Beta",
"2.0.0-RC1",
"2.0.20-Beta2",
"2.1.0-Beta1",
@@ -8,7 +11,6 @@ VERSIONS = [
"2.2.20-Beta2",
"2.3.0",
"2.3.20",
"2.4.0",
]
def _version_to_tuple(v):

View File

@@ -1,5 +1,5 @@
{
"markdownMessage": "The Kotlin version installed (`999.999.999`) is too recent for this version of CodeQL. Install a version lower than 2.4.10.",
"markdownMessage": "The Kotlin version installed (`999.999.999`) is too recent for this version of CodeQL. Install a version lower than 2.3.30.",
"severity": "error",
"source": {
"extractorName": "java",

View File

@@ -1,3 +0,0 @@
[VALUE_NOT_IN_TYPE] predicate callableBinding(@caller callerid, @callable callee): Value 44417 of field callee is not in type @callable. Appears in tuple (-16777158,44417)
Relevant element: callee=44417
Full ID for 44417: @"callable;(0).f((55))(55)". The ID may expand to @"callable;{@"class;Test"}.f({@"type;int"}){@"type;int"}"

View File

@@ -2,9 +2,9 @@ exprs
| Test.java:5:19:5:25 | Integer | Integer |
| Test.java:5:38:5:44 | Integer | Integer |
| Test.java:5:58:5:58 | p | Integer |
| user.kt:2:3:2:16 | x | int |
| user.kt:2:7:2:7 | x | int |
| user.kt:2:11:2:11 | t | Test |
| user.kt:2:11:2:16 | <Call to unknown method> | int |
| user.kt:2:11:2:16 | f(...) | Integer |
| user.kt:2:13:2:16 | <implicit not null> | int |
| user.kt:2:13:2:16 | int | int |
| user.kt:2:15:2:15 | 5 | int |

View File

@@ -6,6 +6,6 @@ def test(codeql, java_full):
codeql.database.create(
command=[
f"javac {java_srcs} -d build",
"kotlinc -language-version 2.0 user.kt -cp build",
"kotlinc -language-version 1.9 user.kt -cp build",
]
)

View File

@@ -1,2 +1,2 @@
| user.kt:3:14:3:22 | getF(...) | file:///!unknown-binary-location/lib/TestKt.class:0:0:0:0 | getF |
| user.kt:3:26:3:28 | getF(...) | file:///!unknown-binary-location/lib/TestKt.class:0:0:0:0 | getF |
| user.kt:3:14:3:22 | getF(...) | lib/lib/TestKt.class:0:0:0:0 | getF |
| user.kt:3:26:3:28 | getF(...) | lib/lib/TestKt.class:0:0:0:0 | getF |

View File

@@ -2,5 +2,5 @@ import commands
def test(codeql, java_full):
commands.run("kotlinc -language-version 2.0 test.kt -d lib")
codeql.database.create(command="kotlinc -language-version 2.0 user.kt -cp lib")
commands.run("kotlinc -language-version 1.9 test.kt -d lib")
codeql.database.create(command="kotlinc -language-version 1.9 user.kt -cp lib")

View File

@@ -9,4 +9,4 @@
| Percentage of calls with call target | 100 |
| Total number of lines | 3 |
| Total number of lines with extension kt | 3 |
| Uses Kotlin 2: true | 1 |
| Uses Kotlin 2: false | 1 |

View File

@@ -1,2 +1,2 @@
def test(codeql, java_full):
codeql.database.create(command="kotlinc -J-Xmx2G -language-version 2.0 SomeClass.kt")
codeql.database.create(command="kotlinc -J-Xmx2G -language-version 1.9 SomeClass.kt")

View File

@@ -1,2 +1,3 @@
| AKt.class:0:0:0:0 | AKt | true |
| B.kt:0:0:0:0 | BKt | true |
| C.kt:1:1:3:1 | C | false |

View File

@@ -2,5 +2,5 @@ import commands
def test(codeql, java_full):
commands.run("kotlinc -language-version 2.0 A.kt")
codeql.database.create(command="kotlinc -cp . -language-version 2.0 B.kt C.kt")
commands.run("kotlinc -language-version 1.9 A.kt")
codeql.database.create(command="kotlinc -cp . -language-version 1.9 B.kt C.kt")

View File

@@ -0,0 +1,4 @@
| equals | Test |
| hashCode | Test |
| toString | Test |
| toString | java.lang.CharSequence |

View File

@@ -3,4 +3,4 @@ import commands
def test(codeql, java_full):
commands.run(["javac", "Test.java", "-d", "bin"])
codeql.database.create(command="kotlinc -language-version 2.0 user.kt -cp bin")
codeql.database.create(command="kotlinc -language-version 1.9 user.kt -cp bin")

View File

@@ -1,9 +0,0 @@
[VALUE_NOT_IN_TYPE] predicate callableBinding(@caller callerid, @callable callee): Value 43828 of field callee is not in type @callable. Appears in tuple (-16776213,43828)
Relevant element: callee=43828
Full ID for 43828: @"callable;(0).takesComparable((35),(35))(36)". The ID may expand to @"callable;{@"class;JavaDefns"}.takesComparable({@"class;java.lang.Comparable;{@"wildcard;super{@"class;java.lang.CharSequence"}"}"},{@"class;java.lang.Comparable;{@"wildcard;super{@"class;java.lang.CharSequence"}"}"}){@"type;void"}"
[VALUE_NOT_IN_TYPE] predicate callableBinding(@caller callerid, @callable callee): Value 43832 of field callee is not in type @callable. Appears in tuple (-16776208,43832)
Relevant element: callee=43832
Full ID for 43832: @"callable;(0).takesArrayOfComparable((54),(54))(36)". The ID may expand to @"callable;{@"class;JavaDefns"}.takesArrayOfComparable({@"array;1;{@"class;java.lang.Comparable;{@"wildcard;super(19)"}"}"},{@"array;1;{@"class;java.lang.Comparable;{@"wildcard;super(19)"}"}"}){@"type;void"}"
[VALUE_NOT_IN_TYPE] predicate callableBinding(@caller callerid, @callable callee): Value 43837 of field callee is not in type @callable. Appears in tuple (-16776201,43837)
Relevant element: callee=43837
Full ID for 43837: @"callable;(0).<init>((35),(35))(36)". The ID may expand to @"callable;{@"class;JavaDefns"}.<init>({@"class;java.lang.Comparable;{@"wildcard;super{@"class;java.lang.CharSequence"}"}"},{@"class;java.lang.Comparable;{@"wildcard;super{@"class;java.lang.CharSequence"}"}"}){@"type;void"}"

View File

@@ -8,16 +8,16 @@
| JavaDefns | takesComparable | invar | Comparable<CharSequence> |
| JavaDefns | takesNestedComparable | innerContravar | Comparable<Comparable<? super CharSequence>> |
| JavaDefns | takesNestedComparable | outerContravar | Comparable<? super Comparable<CharSequence>> |
| JavaDefns2 | JavaDefns2 | p0 | Comparable<? super CharSequence> |
| JavaDefns2 | JavaDefns2 | p0 | Comparable<CharSequence> |
| JavaDefns2 | JavaDefns2 | p1 | Comparable<? super CharSequence> |
| JavaDefns2 | returnsInvariant | return | Comparable<CharSequence> |
| JavaDefns2 | returnsWildcard | return | Comparable<? super CharSequence> |
| JavaDefns2 | takesArrayOfComparable | p0 | Comparable<? super CharSequence>[] |
| JavaDefns2 | takesArrayOfComparable | p0 | Comparable<CharSequence>[] |
| JavaDefns2 | takesArrayOfComparable | p1 | Comparable<? super CharSequence>[] |
| JavaDefns2 | takesComparable | p0 | Comparable<? super CharSequence> |
| JavaDefns2 | takesComparable | p0 | Comparable<CharSequence> |
| JavaDefns2 | takesComparable | p1 | Comparable<? super CharSequence> |
| JavaDefns2 | takesNestedComparable | p0 | Comparable<? super Comparable<? super CharSequence>> |
| JavaDefns2 | takesNestedComparable | p1 | Comparable<? super Comparable<? super CharSequence>> |
| JavaDefns2 | takesNestedComparable | p0 | Comparable<Comparable<? super CharSequence>> |
| JavaDefns2 | takesNestedComparable | p1 | Comparable<? super Comparable<CharSequence>> |
| KotlinDefns | returnsContravar | return | Comparable<CharSequence> |
| KotlinDefns | returnsContravarForced | return | Comparable<? super CharSequence> |
| KotlinDefns | returnsCovar | return | List<CharSequence> |

View File

@@ -8,6 +8,6 @@ def test(codeql, java_full):
command=[
"kotlinc kotlindefns.kt",
"javac JavaUser.java JavaDefns.java -cp .",
"kotlinc -language-version 2.0 -cp . kotlinuser.kt",
"kotlinc -language-version 1.9 -cp . kotlinuser.kt",
]
)

View File

@@ -1,4 +0,0 @@
---
category: minorAnalysis
---
* Improved modeling of Apache HttpClient `execute` method sinks for `java/ssrf` and `java/non-https-url`.

View File

@@ -1,4 +0,0 @@
---
category: feature
---
* Kotlin 2.4.0 can now be analysed.

View File

@@ -1,4 +0,0 @@
---
category: deprecated
---
* Kotlin versions below 2.0.0 are no longer supported for analysis. The minimum supported version is now Kotlin 2.0.0.

View File

@@ -11,7 +11,7 @@ extensions:
- ["org.apache.http.client.methods", "HttpPost", False, "HttpPost", "", "", "Argument[0]", "request-forgery", "manual"]
- ["org.apache.http.client.methods", "HttpPut", False, "HttpPut", "", "", "Argument[0]", "request-forgery", "manual"]
- ["org.apache.http.client.methods", "HttpRequestBase", True, "setURI", "", "", "Argument[0]", "request-forgery", "manual"]
- ["org.apache.http.client.methods", "HttpRequestWrapper", True, "setURI", "(URI)", "", "Argument[0]", "request-forgery", "ai-manual"]
- ["org.apache.http.client.methods", "HttpRequestWrapper", True, "setURI", "(URI)", "", "Argument[0]", "request-forgery", "hq-manual"]
- ["org.apache.http.client.methods", "HttpTrace", False, "HttpTrace", "", "", "Argument[0]", "request-forgery", "manual"]
- ["org.apache.http.client.methods", "RequestBuilder", False, "delete", "", "", "Argument[0]", "request-forgery", "manual"]
- ["org.apache.http.client.methods", "RequestBuilder", False, "get", "", "", "Argument[0]", "request-forgery", "manual"]
@@ -22,29 +22,3 @@ extensions:
- ["org.apache.http.client.methods", "RequestBuilder", False, "put", "", "", "Argument[0]", "request-forgery", "manual"]
- ["org.apache.http.client.methods", "RequestBuilder", False, "setUri", "", "", "Argument[0]", "request-forgery", "manual"]
- ["org.apache.http.client.methods", "RequestBuilder", False, "trace", "", "", "Argument[0]", "request-forgery", "manual"]
- addsTo:
pack: codeql/java-all
extensible: summaryModel
data:
- ["org.apache.http.client.methods", "RequestBuilder", True, "build", "()", "", "Argument[this]", "ReturnValue", "taint", "ai-manual"]
- ["org.apache.http.client.methods", "RequestBuilder", True, "delete", "(String)", "", "Argument[0]", "ReturnValue", "taint", "ai-manual"]
- ["org.apache.http.client.methods", "RequestBuilder", True, "delete", "(URI)", "", "Argument[0]", "ReturnValue", "taint", "ai-manual"]
- ["org.apache.http.client.methods", "RequestBuilder", True, "get", "(String)", "", "Argument[0]", "ReturnValue", "taint", "ai-manual"]
- ["org.apache.http.client.methods", "RequestBuilder", True, "get", "(URI)", "", "Argument[0]", "ReturnValue", "taint", "ai-manual"]
- ["org.apache.http.client.methods", "RequestBuilder", True, "getUri", "()", "", "Argument[this]", "ReturnValue", "taint", "ai-manual"]
- ["org.apache.http.client.methods", "RequestBuilder", True, "head", "(String)", "", "Argument[0]", "ReturnValue", "taint", "ai-manual"]
- ["org.apache.http.client.methods", "RequestBuilder", True, "head", "(URI)", "", "Argument[0]", "ReturnValue", "taint", "ai-manual"]
- ["org.apache.http.client.methods", "RequestBuilder", True, "options", "(String)", "", "Argument[0]", "ReturnValue", "taint", "ai-manual"]
- ["org.apache.http.client.methods", "RequestBuilder", True, "options", "(URI)", "", "Argument[0]", "ReturnValue", "taint", "ai-manual"]
- ["org.apache.http.client.methods", "RequestBuilder", True, "patch", "(String)", "", "Argument[0]", "ReturnValue", "taint", "ai-manual"]
- ["org.apache.http.client.methods", "RequestBuilder", True, "patch", "(URI)", "", "Argument[0]", "ReturnValue", "taint", "ai-manual"]
- ["org.apache.http.client.methods", "RequestBuilder", True, "post", "(String)", "", "Argument[0]", "ReturnValue", "taint", "ai-manual"]
- ["org.apache.http.client.methods", "RequestBuilder", True, "post", "(URI)", "", "Argument[0]", "ReturnValue", "taint", "ai-manual"]
- ["org.apache.http.client.methods", "RequestBuilder", True, "put", "(String)", "", "Argument[0]", "ReturnValue", "taint", "ai-manual"]
- ["org.apache.http.client.methods", "RequestBuilder", True, "put", "(URI)", "", "Argument[0]", "ReturnValue", "taint", "ai-manual"]
- ["org.apache.http.client.methods", "RequestBuilder", True, "setUri", "(String)", "", "Argument[0]", "Argument[this]", "taint", "ai-manual"]
- ["org.apache.http.client.methods", "RequestBuilder", True, "setUri", "(String)", "", "Argument[0]", "ReturnValue", "taint", "ai-manual"]
- ["org.apache.http.client.methods", "RequestBuilder", True, "setUri", "(URI)", "", "Argument[0]", "Argument[this]", "taint", "ai-manual"]
- ["org.apache.http.client.methods", "RequestBuilder", True, "setUri", "(URI)", "", "Argument[0]", "ReturnValue", "taint", "ai-manual"]
- ["org.apache.http.client.methods", "RequestBuilder", True, "trace", "(String)", "", "Argument[0]", "ReturnValue", "taint", "ai-manual"]
- ["org.apache.http.client.methods", "RequestBuilder", True, "trace", "(URI)", "", "Argument[0]", "ReturnValue", "taint", "ai-manual"]

View File

@@ -3,11 +3,6 @@ extensions:
pack: codeql/java-all
extensible: sinkModel
data:
- ["org.apache.http.client", "HttpClient", True, "execute", "(HttpHost,HttpRequest)", "", "Argument[0]", "request-forgery", "ai-manual"]
- ["org.apache.http.client", "HttpClient", True, "execute", "(HttpHost,HttpRequest,HttpContext)", "", "Argument[0]", "request-forgery", "ai-manual"]
- ["org.apache.http.client", "HttpClient", True, "execute", "(HttpHost,HttpRequest,ResponseHandler)", "", "Argument[0]", "request-forgery", "ai-manual"]
- ["org.apache.http.client", "HttpClient", True, "execute", "(HttpHost,HttpRequest,ResponseHandler,HttpContext)", "", "Argument[0]", "request-forgery", "ai-manual"]
- ["org.apache.http.client", "HttpClient", True, "execute", "(HttpUriRequest)", "", "Argument[0]", "request-forgery", "ai-manual"]
- ["org.apache.http.client", "HttpClient", True, "execute", "(HttpUriRequest,HttpContext)", "", "Argument[0]", "request-forgery", "ai-manual"]
- ["org.apache.http.client", "HttpClient", True, "execute", "(HttpUriRequest,ResponseHandler)", "", "Argument[0]", "request-forgery", "ai-manual"]
- ["org.apache.http.client", "HttpClient", True, "execute", "(HttpUriRequest,ResponseHandler,HttpContext)", "", "Argument[0]", "request-forgery", "ai-manual"]
- ["org.apache.http.client", "HttpClient", True, "execute", "(HttpUriRequest)", "", "Argument[0]", "request-forgery", "ai-manual"]

View File

@@ -1,16 +0,0 @@
[VALUE_NOT_IN_TYPE] predicate callableBinding(@caller callerid, @callable callee): Value 79083 of field callee is not in type @callable. Appears in tuple (-16776495,79083)
Relevant element: callee=79083
Full ID for 79083: @"callable;(21913).toString()(64)". The ID may expand to @"callable;{@"class;java.nio.file.Path"}.toString(){@"class;java.lang.String"}"
[VALUE_NOT_IN_TYPE] predicate callableBinding(@caller callerid, @callable callee): Value 79083 of field callee is not in type @callable. Appears in tuple (-16776429,79083)
Relevant element: callee=79083
Full ID for 79083: @"callable;(21913).toString()(64)". The ID may expand to @"callable;{@"class;java.nio.file.Path"}.toString(){@"class;java.lang.String"}"
[VALUE_NOT_IN_TYPE] predicate callableBinding(@caller callerid, @callable callee): Value 79083 of field callee is not in type @callable. Appears in tuple (-16776357,79083)
Relevant element: callee=79083
Full ID for 79083: @"callable;(21913).toString()(64)". The ID may expand to @"callable;{@"class;java.nio.file.Path"}.toString(){@"class;java.lang.String"}"
[VALUE_NOT_IN_TYPE] predicate callableBinding(@caller callerid, @callable callee): Value 79083 of field callee is not in type @callable. Appears in tuple (-16776266,79083)
Relevant element: callee=79083
Full ID for 79083: @"callable;(21913).toString()(64)". The ID may expand to @"callable;{@"class;java.nio.file.Path"}.toString(){@"class;java.lang.String"}"
[VALUE_NOT_IN_TYPE] predicate callableBinding(@caller callerid, @callable callee): Value 79083 of field callee is not in type @callable. Appears in tuple (-16776200,79083)
Relevant element: callee=79083
Full ID for 79083: @"callable;(21913).toString()(64)". The ID may expand to @"callable;{@"class;java.nio.file.Path"}.toString(){@"class;java.lang.String"}"
[VALUE_NOT_IN_TYPE] predicate callableBinding(@caller callerid, @callable callee): More errors, not displayed. There are 16 values of field callee that are not in type @callable for a relation of size 1821

View File

@@ -1,31 +0,0 @@
| Test.java:137:22:137:27 | source | Unexpected result: hasTaintFlow |
| Test.java:141:35:141:51 | // $ hasTaintFlow | Missing result: hasTaintFlow |
| Test.java:148:22:148:27 | source | Unexpected result: hasTaintFlow |
| Test.java:152:35:152:51 | // $ hasTaintFlow | Missing result: hasTaintFlow |
| Test.java:159:22:159:27 | source | Unexpected result: hasTaintFlow |
| Test.java:163:35:163:51 | // $ hasTaintFlow | Missing result: hasTaintFlow |
| Test.java:178:35:178:51 | // $ hasTaintFlow | Missing result: hasTaintFlow |
| Test.java:181:35:181:51 | // $ hasTaintFlow | Missing result: hasTaintFlow |
| Test.java:189:35:189:51 | // $ hasTaintFlow | Missing result: hasTaintFlow |
| Test.java:192:35:192:51 | // $ hasTaintFlow | Missing result: hasTaintFlow |
| Test.java:200:35:200:51 | // $ hasTaintFlow | Missing result: hasTaintFlow |
| Test.java:203:35:203:51 | // $ hasTaintFlow | Missing result: hasTaintFlow |
| Test.java:362:22:362:27 | source | Unexpected result: hasTaintFlow |
| Test.java:366:35:366:51 | // $ hasTaintFlow | Missing result: hasTaintFlow |
| Test.java:373:22:373:27 | source | Unexpected result: hasTaintFlow |
| Test.java:377:35:377:51 | // $ hasTaintFlow | Missing result: hasTaintFlow |
| Test.java:384:22:384:27 | source | Unexpected result: hasTaintFlow |
| Test.java:388:35:388:51 | // $ hasTaintFlow | Missing result: hasTaintFlow |
| Test.java:402:22:402:27 | source | Unexpected result: hasTaintFlow |
| Test.java:406:35:406:51 | // $ hasTaintFlow | Missing result: hasTaintFlow |
| Test.java:413:22:413:27 | source | Unexpected result: hasTaintFlow |
| Test.java:417:35:417:51 | // $ hasTaintFlow | Missing result: hasTaintFlow |
| Test.java:424:22:424:27 | source | Unexpected result: hasTaintFlow |
| Test.java:428:35:428:51 | // $ hasTaintFlow | Missing result: hasTaintFlow |
| Test.java:436:22:436:27 | source | Unexpected result: hasTaintFlow |
| Test.java:440:35:440:51 | // $ hasTaintFlow | Missing result: hasTaintFlow |
| Test.java:447:22:447:27 | source | Unexpected result: hasTaintFlow |
| Test.java:451:35:451:51 | // $ hasTaintFlow | Missing result: hasTaintFlow |
| Test.java:458:22:458:27 | source | Unexpected result: hasTaintFlow |
| Test.java:462:35:462:51 | // $ hasTaintFlow | Missing result: hasTaintFlow |
| Test.java:604:31:604:47 | // $ hasTaintFlow | Missing result: hasTaintFlow |

View File

@@ -1 +0,0 @@
| CleartextStorageSharedPrefsTest.java:110:84:110:118 | // $ hasCleartextStorageSharedPrefs | Missing result: hasCleartextStorageSharedPrefs |

View File

@@ -1,6 +0,0 @@
[VALUE_NOT_IN_TYPE] predicate callableBinding(@caller callerid, @callable callee): Value 80453 of field callee is not in type @callable. Appears in tuple (-16773210,80453)
Relevant element: callee=80453
Full ID for 80453: @"callable;(846).toString()(21)". The ID may expand to @"callable;{@"class;java.lang.CharSequence"}.toString(){@"class;java.lang.String"}"
[VALUE_NOT_IN_TYPE] predicate callableBinding(@caller callerid, @callable callee): Value 80453 of field callee is not in type @callable. Appears in tuple (-16773194,80453)
Relevant element: callee=80453
Full ID for 80453: @"callable;(846).toString()(21)". The ID may expand to @"callable;{@"class;java.lang.CharSequence"}.toString(){@"class;java.lang.String"}"

View File

@@ -1,45 +0,0 @@
import java.io.IOException;
import org.apache.http.HttpHost;
import org.apache.http.HttpRequest;
import org.apache.http.client.HttpClient;
import org.apache.http.client.ResponseHandler;
import org.apache.http.client.methods.HttpUriRequest;
import org.apache.http.client.methods.RequestBuilder;
import org.apache.http.impl.client.HttpClients;
import org.apache.http.message.BasicHttpRequest;
import org.apache.http.protocol.HttpContext;
import javax.servlet.ServletException;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
public class ApacheHttpClientExecuteSSRF extends HttpServlet {
protected void doGet(HttpServletRequest request, HttpServletResponse response)
throws ServletException, IOException {
try {
String source = request.getParameter("host"); // $ Source
HttpHost host = new HttpHost(source);
HttpRequest req = new BasicHttpRequest("GET", "/");
HttpUriRequest uriReq = RequestBuilder.get(source).build(); // $ Alert
HttpContext context = null;
HttpClient client = HttpClients.createDefault();
ResponseHandler<Object> handler = null;
client.execute(host, req); // $ Alert
client.execute(host, req, context); // $ Alert
client.execute(host, req, handler); // $ Alert
client.execute(host, req, handler, context); // $ Alert
client.execute(uriReq); // $ Alert
client.execute(uriReq, context); // $ Alert
client.execute(uriReq, handler); // $ Alert
client.execute(uriReq, handler, context); // $ Alert
} catch (Exception e) {
// TODO: handle exception
}
}
}

View File

@@ -1 +1 @@
//semmle-extractor-options: --javac-args -source 11 -target 11 -cp ${testdir}/../../../stubs/javax-validation-constraints:${testdir}/../../../stubs/springframework-5.8.x:${testdir}/../../../stubs/javax-ws-rs-api-2.1.1:${testdir}/../../../stubs/javax-ws-rs-api-3.0.0:${testdir}/../../../stubs/apache-http-4.4.13/:${testdir}/../../../stubs/apache-http-client-4.4.13:${testdir}/../../../stubs/projectreactor-3.4.3/:${testdir}/../../../stubs/postgresql-42.3.3/:${testdir}/../../../stubs/HikariCP-3.4.5/:${testdir}/../../../stubs/spring-jdbc-5.3.8/:${testdir}/../../../stubs/jdbi3-core-3.27.2/:${testdir}/../../../stubs/cargo:${testdir}/../../../stubs/javafx-web:${testdir}/../../../stubs/apache-commons-jelly-1.0.1:${testdir}/../../../stubs/dom4j-2.1.1:${testdir}/../../../stubs/jaxen-1.2.0:${testdir}/../../../stubs/stapler-1.263:${testdir}/../../../stubs/javax-servlet-2.5:${testdir}/../../../stubs/apache-commons-fileupload-1.4:${testdir}/../../../stubs/saxon-xqj-9.x:${testdir}/../../../stubs/apache-commons-beanutils:${testdir}/../../../stubs/apache-commons-lang:${testdir}/../../../stubs/apache-http-5:${testdir}/../../../stubs/playframework-2.6.x:${testdir}/../../../stubs/jaxws-api-2.0:${testdir}/../../../stubs/apache-cxf
//semmle-extractor-options: --javac-args -source 11 -target 11 -cp ${testdir}/../../../stubs/javax-validation-constraints:${testdir}/../../../stubs/springframework-5.8.x:${testdir}/../../../stubs/javax-ws-rs-api-2.1.1:${testdir}/../../../stubs/javax-ws-rs-api-3.0.0:${testdir}/../../../stubs/apache-http-4.4.13/:${testdir}/../../../stubs/projectreactor-3.4.3/:${testdir}/../../../stubs/postgresql-42.3.3/:${testdir}/../../../stubs/HikariCP-3.4.5/:${testdir}/../../../stubs/spring-jdbc-5.3.8/:${testdir}/../../../stubs/jdbi3-core-3.27.2/:${testdir}/../../../stubs/cargo:${testdir}/../../../stubs/javafx-web:${testdir}/../../../stubs/apache-commons-jelly-1.0.1:${testdir}/../../../stubs/dom4j-2.1.1:${testdir}/../../../stubs/jaxen-1.2.0:${testdir}/../../../stubs/stapler-1.263:${testdir}/../../../stubs/javax-servlet-2.5:${testdir}/../../../stubs/apache-commons-fileupload-1.4:${testdir}/../../../stubs/saxon-xqj-9.x:${testdir}/../../../stubs/apache-commons-beanutils:${testdir}/../../../stubs/apache-commons-lang:${testdir}/../../../stubs/apache-http-5:${testdir}/../../../stubs/playframework-2.6.x:${testdir}/../../../stubs/jaxws-api-2.0:${testdir}/../../../stubs/apache-cxf

View File

@@ -1,23 +0,0 @@
// Generated automatically from org.apache.http.client.HttpClient for testing purposes
package org.apache.http.client;
import java.io.IOException;
import org.apache.http.HttpHost;
import org.apache.http.HttpRequest;
import org.apache.http.HttpResponse;
import org.apache.http.client.methods.HttpUriRequest;
import org.apache.http.protocol.HttpContext;
public interface HttpClient {
HttpResponse execute(HttpHost target, HttpRequest request) throws IOException;
HttpResponse execute(HttpHost target, HttpRequest request, HttpContext context) throws IOException;
<T> T execute(HttpHost target, HttpRequest request, ResponseHandler<? extends T> responseHandler) throws IOException;
<T> T execute(HttpHost target, HttpRequest request, ResponseHandler<? extends T> responseHandler, HttpContext context)
throws IOException;
HttpResponse execute(HttpUriRequest request) throws IOException;
HttpResponse execute(HttpUriRequest request, HttpContext context) throws IOException;
<T> T execute(HttpUriRequest request, ResponseHandler<? extends T> responseHandler) throws IOException;
<T> T execute(HttpUriRequest request, ResponseHandler<? extends T> responseHandler, HttpContext context)
throws IOException;
}

View File

@@ -1,9 +0,0 @@
// Generated automatically from org.apache.http.client.ResponseHandler for testing purposes
package org.apache.http.client;
import org.apache.http.HttpResponse;
public interface ResponseHandler<T> {
T handleResponse(HttpResponse response);
}

View File

@@ -1,7 +0,0 @@
package org.apache.http.impl.client;
import org.apache.http.client.HttpClient;
public abstract class CloseableHttpClient implements HttpClient {
}

View File

@@ -1,10 +0,0 @@
// Generated automatically from org.apache.http.client.HttpClient for testing purposes
package org.apache.http.impl.client;
import java.io.IOException;
import org.apache.http.impl.client.CloseableHttpClient;
public final class HttpClients {
public static CloseableHttpClient createDefault() { return null; }
}

View File

@@ -21,19 +21,13 @@ file_coverage_languages:
scc_languages:
- TypeScript
- TypeScript Typings
- name: vue
display_name: Vue.js component
scc_languages:
- Vue
github_api_languages:
- JavaScript
- TypeScript
- Vue
scc_languages:
- JavaScript
- TypeScript
- TypeScript Typings
- Vue
file_types:
- name: javascript
display_name: JavaScript

View File

@@ -0,0 +1,2 @@
import semmle.python.controlflow.internal.AstNodeImpl
import ControlFlow::Consistency

View File

@@ -213,9 +213,11 @@ class ExprWithPointsTo extends Expr {
* Gets what this expression might "refer-to" in the given `context`.
*/
predicate refersTo(Context context, Object obj, ClassObject cls, AstNode origin) {
this.getAFlowNode()
.(ControlFlowNodeWithPointsTo)
.refersTo(context, obj, cls, origin.getAFlowNode())
exists(ControlFlowNode this_, ControlFlowNode origin_ |
this_.getNode() = this and origin_.getNode() = origin
|
this_.(ControlFlowNodeWithPointsTo).refersTo(context, obj, cls, origin_)
)
}
/**
@@ -226,7 +228,11 @@ class ExprWithPointsTo extends Expr {
*/
pragma[nomagic]
predicate refersTo(Object obj, AstNode origin) {
this.getAFlowNode().(ControlFlowNodeWithPointsTo).refersTo(obj, origin.getAFlowNode())
exists(ControlFlowNode this_, ControlFlowNode origin_ |
this_.getNode() = this and origin_.getNode() = origin
|
this_.(ControlFlowNodeWithPointsTo).refersTo(obj, origin_)
)
}
/**
@@ -240,16 +246,22 @@ class ExprWithPointsTo extends Expr {
* in the given `context`.
*/
predicate pointsTo(Context context, Value value, AstNode origin) {
this.getAFlowNode()
.(ControlFlowNodeWithPointsTo)
.pointsTo(context, value, origin.getAFlowNode())
exists(ControlFlowNode this_, ControlFlowNode origin_ |
this_.getNode() = this and origin_.getNode() = origin
|
this_.(ControlFlowNodeWithPointsTo).pointsTo(context, value, origin_)
)
}
/**
* Holds if this expression might "point-to" to `value` which is from `origin`.
*/
predicate pointsTo(Value value, AstNode origin) {
this.getAFlowNode().(ControlFlowNodeWithPointsTo).pointsTo(value, origin.getAFlowNode())
exists(ControlFlowNode this_, ControlFlowNode origin_ |
this_.getNode() = this and origin_.getNode() = origin
|
this_.(ControlFlowNodeWithPointsTo).pointsTo(value, origin_)
)
}
/**
@@ -475,7 +487,10 @@ class FunctionMetricsWithPointsTo extends FunctionMetrics {
not non_coupling_method(result) and
exists(Call call | call.getScope() = this |
exists(FunctionObject callee | callee.getFunction() = result |
call.getAFlowNode().getFunction().(ControlFlowNodeWithPointsTo).refersTo(callee)
exists(CallNode call_ |
call_.getNode() = call and
call_.getFunction().(ControlFlowNodeWithPointsTo).refersTo(callee)
)
)
or
exists(Attribute a | call.getFunc() = a |

View File

@@ -64,7 +64,7 @@ private predicate jump_to_defn(ControlFlowNode use, Definition defn) {
private predicate preferred_jump_to_defn(Expr use, Definition def) {
not use instanceof ClassExpr and
not use instanceof FunctionExpr and
jump_to_defn(use.getAFlowNode(), def)
exists(ControlFlowNode useNode | useNode.getNode() = use | jump_to_defn(useNode, def))
}
private predicate unique_jump_to_defn(Expr use, Definition def) {
@@ -452,7 +452,7 @@ private predicate self_parameter_jump_to_defn_attribute(
* This exists primarily for testing use `getPreferredDefinition()` instead.
*/
Definition getADefinition(Expr use) {
jump_to_defn(use.getAFlowNode(), result) and
exists(ControlFlowNode useNode | useNode.getNode() = use | jump_to_defn(useNode, result)) and
not use instanceof Call and
not use.isArtificial() and
// Not the use itself

View File

@@ -0,0 +1,4 @@
---
category: minorAnalysis
---
* A new Python control flow graph implementation has been added under `semmle.python.controlflow.internal.Cfg` (backed by `AstNodeImpl.qll`), built on the shared `codeql.controlflow.ControlFlowGraph` library. It is not yet used by the dataflow library or any production query; the legacy CFG in `semmle/python/Flow.qll` remains the default. The new library is exposed for tests and for upcoming migrations.

View File

@@ -0,0 +1,4 @@
---
category: minorAnalysis
---
* A new SSA adapter has been added under `semmle.python.dataflow.new.internal.SsaImpl`, built on the shared `codeql.ssa.Ssa` library and the new shared CFG (`semmle.python.controlflow.internal.Cfg`). It is not yet used by the dataflow library or any production query; the legacy ESSA SSA in `semmle/python/essa/*` remains the default. The new SSA adapter is exposed for tests and for the upcoming dataflow migration.

View File

@@ -0,0 +1,5 @@
---
category: deprecated
---
* The `AstNode.getAFlowNode()` predicate has been deprecated. Use `ControlFlowNode.getNode()` from the other direction instead: replace `e.getAFlowNode() = n` with `n.getNode() = e`. This is a preparatory step towards migrating the dataflow library off the legacy CFG; it has no semantic effect.

View File

@@ -0,0 +1,4 @@
---
category: minorAnalysis
---
* Simplified the internal predicates that detect `@staticmethod`, `@classmethod` and `@property` decorators to match the decorator's AST `Name` directly, rather than going through the CFG and requiring the name to resolve globally. Code that shadows these three builtin decorators at the module-scope will now be classified by the decorator name alone; in practice, shadowing these names is extremely rare and the call-graph results are unchanged.

View File

@@ -0,0 +1,4 @@
---
category: deprecated
---
* The `Function.getAReturnValueFlowNode()` predicate has been deprecated. Bind a `Return` node explicitly instead — `exists(Return ret | ret.getScope() = f and n.getNode() = ret.getValue())`. This is a preparatory step towards migrating the dataflow library off the legacy CFG; it has no semantic effect.

View File

@@ -0,0 +1,45 @@
/**
* @name Print CFG (New)
* @description Produces a representation of a file's Control Flow Graph
* using the new shared control flow library.
* This query is used by the VS Code extension.
* @id python/print-cfg
* @kind graph
* @tags ide-contextual-queries/print-cfg
*/
private import python as Py
import semmle.python.controlflow.internal.AstNodeImpl
external string selectedSourceFile();
private predicate selectedSourceFileAlias = selectedSourceFile/0;
external int selectedSourceLine();
private predicate selectedSourceLineAlias = selectedSourceLine/0;
external int selectedSourceColumn();
private predicate selectedSourceColumnAlias = selectedSourceColumn/0;
module ViewCfgQueryInput implements ControlFlow::ViewCfgQueryInputSig<Py::File> {
predicate selectedSourceFile = selectedSourceFileAlias/0;
predicate selectedSourceLine = selectedSourceLineAlias/0;
predicate selectedSourceColumn = selectedSourceColumnAlias/0;
predicate cfgScopeSpan(
Ast::Callable callable, Py::File file, int startLine, int startColumn, int endLine,
int endColumn
) {
exists(Py::Scope scope |
scope = callable.asScope() and
file = scope.getLocation().getFile() and
scope.getLocation().hasLocationInfo(_, startLine, startColumn, endLine, endColumn)
)
}
}
import ControlFlow::ViewCfgQuery<Py::File, ViewCfgQueryInput>

View File

@@ -16,21 +16,26 @@ abstract class AstNode extends AstNode_ {
/** Gets the scope that this node occurs in */
abstract Scope getScope();
/**
* Gets a flow node corresponding directly to this node.
* NOTE: For some statements and other purely syntactic elements,
* there may not be a `ControlFlowNode`
*/
cached
ControlFlowNode getAFlowNode() {
Stages::AST::ref() and
py_flow_bb_node(result, this, _, _)
}
/** Gets the location for this AST node */
cached
Location getLocation() { none() }
/**
* DEPRECATED: use `ControlFlowNode.getNode()` from the other direction instead;
* that is, replace `e.getAFlowNode() = n` with `n.getNode() = e`. This API is
* being removed to untangle the AST and CFG hierarchies in preparation for
* migrating the dataflow library off the legacy CFG.
*
* Gets a flow node corresponding directly to this node.
* NOTE: For some statements and other purely syntactic elements,
* there may not be a `ControlFlowNode`.
*/
cached
deprecated ControlFlowNode getAFlowNode() {
Stages::AST::ref() and
py_flow_bb_node(result, this, _, _)
}
/**
* Whether this syntactic element is artificial, that is it is generated
* by the compiler and is not present in the source

View File

@@ -28,7 +28,9 @@ class Expr extends Expr_, AstNode {
/** Whether this expression may have a side effect (as determined purely from its syntax) */
predicate hasSideEffects() {
/* If an exception raised by this expression handled, count that as a side effect */
this.getAFlowNode().getASuccessor().getNode() instanceof ExceptStmt
exists(ControlFlowNode n | n.getNode() = this |
n.getASuccessor().getNode() instanceof ExceptStmt
)
or
this.getASubExpression().hasSideEffects()
}
@@ -68,8 +70,6 @@ class Attribute extends Attribute_ {
/* syntax: Expr.name */
override Expr getASubExpression() { result = this.getObject() }
override AttrNode getAFlowNode() { result = super.getAFlowNode() }
/** Gets the name of this attribute. That is the `name` in `obj.name` */
string getName() { result = Attribute_.super.getAttr() }
@@ -96,8 +96,6 @@ class Subscript extends Subscript_ {
}
Expr getObject() { result = Subscript_.super.getValue() }
override SubscriptNode getAFlowNode() { result = super.getAFlowNode() }
}
/** A call expression, such as `func(...)` */
@@ -113,8 +111,6 @@ class Call extends Call_ {
override string toString() { result = this.getFunc().toString() + "()" }
override CallNode getAFlowNode() { result = super.getAFlowNode() }
/** Gets a tuple (*) argument of this call. */
Expr getStarargs() { result = this.getAPositionalArg().(Starred).getValue() }
@@ -200,8 +196,6 @@ class IfExp extends IfExp_ {
override Expr getASubExpression() {
result = this.getTest() or result = this.getBody() or result = this.getOrelse()
}
override IfExprNode getAFlowNode() { result = super.getAFlowNode() }
}
/** A starred expression, such as the `*rest` in the assignment `first, *rest = seq` */
@@ -410,8 +404,6 @@ class PlaceHolder extends PlaceHolder_ {
override Expr getASubExpression() { none() }
override string toString() { result = "$" + this.getId() }
override NameNode getAFlowNode() { result = super.getAFlowNode() }
}
/** A tuple expression such as `( 1, 3, 5, 7, 9 )` */
@@ -478,8 +470,6 @@ class Name extends Name_ {
override string toString() { result = this.getId() }
override NameNode getAFlowNode() { result = super.getAFlowNode() }
override predicate isArtificial() {
/* Artificial variable names in comprehensions all start with "." */
this.getId().charAt(0) = "."
@@ -585,8 +575,6 @@ abstract class NameConstant extends Name, ImmutableLiteral {
override predicate isConstant() { any() }
override NameConstantNode getAFlowNode() { result = Name.super.getAFlowNode() }
override predicate isArtificial() { none() }
}

View File

@@ -1,7 +1,7 @@
overlay[local]
module;
import python
import python as Py
private import semmle.python.internal.CachedStages
private import codeql.controlflow.BasicBlock as BB
@@ -17,7 +17,7 @@ private import codeql.controlflow.BasicBlock as BB
*/
private predicate augstore(ControlFlowNode load, ControlFlowNode store) {
exists(Expr load_store | exists(AugAssign aa | aa.getTarget() = load_store) |
exists(Py::Expr load_store | exists(Py::AugAssign aa | aa.getTarget() = load_store) |
toAst(load) = load_store and
toAst(store) = load_store and
load.strictlyDominates(store)
@@ -25,7 +25,7 @@ private predicate augstore(ControlFlowNode load, ControlFlowNode store) {
}
/** A non-dispatched getNode() to avoid negative recursion issues */
private AstNode toAst(ControlFlowNode n) { py_flow_bb_node(n, result, _, _) }
private Py::AstNode toAst(ControlFlowNode n) { py_flow_bb_node(n, result, _, _) }
/**
* A control flow node. Control flow nodes have a many-to-one relation with syntactic nodes,
@@ -35,19 +35,19 @@ private AstNode toAst(ControlFlowNode n) { py_flow_bb_node(n, result, _, _) }
class ControlFlowNode extends @py_flow_node {
/** Whether this control flow node is a load (including those in augmented assignments) */
predicate isLoad() {
exists(Expr e | e = toAst(this) | py_expr_contexts(_, 3, e) and not augstore(_, this))
exists(Py::Expr e | e = toAst(this) | py_expr_contexts(_, 3, e) and not augstore(_, this))
}
/** Whether this control flow node is a store (including those in augmented assignments) */
predicate isStore() {
exists(Expr e | e = toAst(this) | py_expr_contexts(_, 5, e) or augstore(_, this))
exists(Py::Expr e | e = toAst(this) | py_expr_contexts(_, 5, e) or augstore(_, this))
}
/** Whether this control flow node is a delete */
predicate isDelete() { exists(Expr e | e = toAst(this) | py_expr_contexts(_, 2, e)) }
predicate isDelete() { exists(Py::Expr e | e = toAst(this) | py_expr_contexts(_, 2, e)) }
/** Whether this control flow node is a parameter */
predicate isParameter() { exists(Expr e | e = toAst(this) | py_expr_contexts(_, 4, e)) }
predicate isParameter() { exists(Py::Expr e | e = toAst(this) | py_expr_contexts(_, 4, e)) }
/** Whether this control flow node is a store in an augmented assignment */
predicate isAugStore() { augstore(_, this) }
@@ -57,61 +57,61 @@ class ControlFlowNode extends @py_flow_node {
/** Whether this flow node corresponds to a literal */
predicate isLiteral() {
toAst(this) instanceof Bytes
toAst(this) instanceof Py::Bytes
or
toAst(this) instanceof Dict
toAst(this) instanceof Py::Dict
or
toAst(this) instanceof DictComp
toAst(this) instanceof Py::DictComp
or
toAst(this) instanceof Set
toAst(this) instanceof Py::Set
or
toAst(this) instanceof SetComp
toAst(this) instanceof Py::SetComp
or
toAst(this) instanceof Ellipsis
toAst(this) instanceof Py::Ellipsis
or
toAst(this) instanceof GeneratorExp
toAst(this) instanceof Py::GeneratorExp
or
toAst(this) instanceof Lambda
toAst(this) instanceof Py::Lambda
or
toAst(this) instanceof ListComp
toAst(this) instanceof Py::ListComp
or
toAst(this) instanceof List
toAst(this) instanceof Py::List
or
toAst(this) instanceof Num
toAst(this) instanceof Py::Num
or
toAst(this) instanceof Tuple
toAst(this) instanceof Py::Tuple
or
toAst(this) instanceof Unicode
toAst(this) instanceof Py::Unicode
or
toAst(this) instanceof NameConstant
toAst(this) instanceof Py::NameConstant
}
/** Whether this flow node corresponds to an attribute expression */
predicate isAttribute() { toAst(this) instanceof Attribute }
predicate isAttribute() { toAst(this) instanceof Py::Attribute }
/** Whether this flow node corresponds to an subscript expression */
predicate isSubscript() { toAst(this) instanceof Subscript }
predicate isSubscript() { toAst(this) instanceof Py::Subscript }
/** Whether this flow node corresponds to an import member */
predicate isImportMember() { toAst(this) instanceof ImportMember }
predicate isImportMember() { toAst(this) instanceof Py::ImportMember }
/** Whether this flow node corresponds to a call */
predicate isCall() { toAst(this) instanceof Call }
predicate isCall() { toAst(this) instanceof Py::Call }
/** Whether this flow node is the first in a module */
predicate isModuleEntry() { this.isEntryNode() and toAst(this) instanceof Module }
predicate isModuleEntry() { this.isEntryNode() and toAst(this) instanceof Py::Module }
/** Whether this flow node corresponds to an import */
predicate isImport() { toAst(this) instanceof ImportExpr }
predicate isImport() { toAst(this) instanceof Py::ImportExpr }
/** Whether this flow node corresponds to a conditional expression */
predicate isIfExp() { toAst(this) instanceof IfExp }
predicate isIfExp() { toAst(this) instanceof Py::IfExp }
/** Whether this flow node corresponds to a function definition expression */
predicate isFunction() { toAst(this) instanceof FunctionExpr }
predicate isFunction() { toAst(this) instanceof Py::FunctionExpr }
/** Whether this flow node corresponds to a class definition expression */
predicate isClass() { toAst(this) instanceof ClassExpr }
predicate isClass() { toAst(this) instanceof Py::ClassExpr }
/** Gets a predecessor of this flow node */
ControlFlowNode getAPredecessor() { this = result.getASuccessor() }
@@ -123,25 +123,25 @@ class ControlFlowNode extends @py_flow_node {
ControlFlowNode getImmediateDominator() { py_idoms(this, result) }
/** Gets the syntactic element corresponding to this flow node */
AstNode getNode() { py_flow_bb_node(this, result, _, _) }
Py::AstNode getNode() { py_flow_bb_node(this, result, _, _) }
/** Gets a textual representation of this element. */
cached
string toString() {
Stages::AST::ref() and
// Since modules can have ambigous names, entry nodes can too, if we do not collate them.
exists(Scope s | s.getEntryNode() = this |
exists(Py::Scope s | s.getEntryNode() = this |
result = "Entry node for " + concat( | | s.toString(), ",")
)
or
exists(Scope s | s.getANormalExit() = this | result = "Exit node for " + s.toString())
exists(Py::Scope s | s.getANormalExit() = this | result = "Exit node for " + s.toString())
or
not exists(Scope s | s.getEntryNode() = this or s.getANormalExit() = this) and
not exists(Py::Scope s | s.getEntryNode() = this or s.getANormalExit() = this) and
result = "ControlFlowNode for " + this.getNode().toString()
}
/** Gets the location of this ControlFlowNode */
Location getLocation() { result = this.getNode().getLocation() }
Py::Location getLocation() { result = this.getNode().getLocation() }
/** Whether this flow node is the first in its scope */
predicate isEntryNode() { py_scope_flow(this, _, -1) }
@@ -151,9 +151,9 @@ class ControlFlowNode extends @py_flow_node {
/** Gets the scope containing this flow node */
cached
Scope getScope() {
Py::Scope getScope() {
Stages::AST::ref() and
if this.getNode() instanceof Scope
if this.getNode() instanceof Py::Scope
then
/* Entry or exit node */
result = this.getNode()
@@ -161,7 +161,7 @@ class ControlFlowNode extends @py_flow_node {
}
/** Gets the enclosing module */
Module getEnclosingModule() { result = this.getScope().getEnclosingModule() }
Py::Module getEnclosingModule() { result = this.getScope().getEnclosingModule() }
/** Gets a successor for this node if the relevant condition is True. */
ControlFlowNode getATrueSuccessor() {
@@ -188,7 +188,7 @@ class ControlFlowNode extends @py_flow_node {
}
/** Whether the scope may be exited as a result of this node raising an exception */
predicate isExceptionalExit(Scope s) { py_scope_flow(this, s, 1) }
predicate isExceptionalExit(Py::Scope s) { py_scope_flow(this, s, 1) }
/** Whether this node is a normal (non-exceptional) exit */
predicate isNormalExit() { py_scope_flow(this, _, 0) or py_scope_flow(this, _, 2) }
@@ -236,7 +236,7 @@ class ControlFlowNode extends @py_flow_node {
/* join-ordering helper for `getAChild() */
pragma[noinline]
private ControlFlowNode getExprChild(BasicBlock dom) {
this.getNode().(Expr).getAChildNode() = result.getNode() and
this.getNode().(Py::Expr).getAChildNode() = result.getNode() and
result.getBasicBlock().dominates(dom) and
not this instanceof UnaryExprNode
}
@@ -249,16 +249,16 @@ class ControlFlowNode extends @py_flow_node {
*/
private class AnyNode extends ControlFlowNode {
override AstNode getNode() { result = super.getNode() }
override Py::AstNode getNode() { result = super.getNode() }
}
/** A control flow node corresponding to a call expression, such as `func(...)` */
class CallNode extends ControlFlowNode {
CallNode() { toAst(this) instanceof Call }
CallNode() { toAst(this) instanceof Py::Call }
/** Gets the flow node corresponding to the function expression for the call corresponding to this flow node */
ControlFlowNode getFunction() {
exists(Call c |
exists(Py::Call c |
this.getNode() = c and
c.getFunc() = result.getNode() and
result.getBasicBlock().dominates(this.getBasicBlock())
@@ -267,7 +267,7 @@ class CallNode extends ControlFlowNode {
/** Gets the flow node corresponding to the n'th positional argument of the call corresponding to this flow node */
ControlFlowNode getArg(int n) {
exists(Call c |
exists(Py::Call c |
this.getNode() = c and
c.getArg(n) = result.getNode() and
result.getBasicBlock().dominates(this.getBasicBlock())
@@ -276,7 +276,7 @@ class CallNode extends ControlFlowNode {
/** Gets the flow node corresponding to the named argument of the call corresponding to this flow node */
ControlFlowNode getArgByName(string name) {
exists(Call c, Keyword k |
exists(Py::Call c, Py::Keyword k |
this.getNode() = c and
k = c.getANamedArg() and
k.getValue() = result.getNode() and
@@ -292,7 +292,7 @@ class CallNode extends ControlFlowNode {
result = this.getArgByName(_)
}
override Call getNode() { result = super.getNode() }
override Py::Call getNode() { result = super.getNode() }
predicate isDecoratorCall() {
this.isClassDecoratorCall()
@@ -301,11 +301,11 @@ class CallNode extends ControlFlowNode {
}
predicate isClassDecoratorCall() {
exists(ClassExpr cls | this.getNode() = cls.getADecoratorCall())
exists(Py::ClassExpr cls | this.getNode() = cls.getADecoratorCall())
}
predicate isFunctionDecoratorCall() {
exists(FunctionExpr func | this.getNode() = func.getADecoratorCall())
exists(Py::FunctionExpr func | this.getNode() = func.getADecoratorCall())
}
/** Gets the first tuple (*) argument of this call, if any. */
@@ -323,11 +323,11 @@ class CallNode extends ControlFlowNode {
/** A control flow corresponding to an attribute expression, such as `value.attr` */
class AttrNode extends ControlFlowNode {
AttrNode() { toAst(this) instanceof Attribute }
AttrNode() { toAst(this) instanceof Py::Attribute }
/** Gets the flow node corresponding to the object of the attribute expression corresponding to this flow node */
ControlFlowNode getObject() {
exists(Attribute a |
exists(Py::Attribute a |
this.getNode() = a and
a.getObject() = result.getNode() and
result.getBasicBlock().dominates(this.getBasicBlock())
@@ -339,7 +339,7 @@ class AttrNode extends ControlFlowNode {
* with the matching name
*/
ControlFlowNode getObject(string name) {
exists(Attribute a |
exists(Py::Attribute a |
this.getNode() = a and
a.getObject() = result.getNode() and
a.getName() = name and
@@ -348,57 +348,57 @@ class AttrNode extends ControlFlowNode {
}
/** Gets the attribute name of the attribute expression corresponding to this flow node */
string getName() { exists(Attribute a | this.getNode() = a and a.getName() = result) }
string getName() { exists(Py::Attribute a | this.getNode() = a and a.getName() = result) }
override Attribute getNode() { result = super.getNode() }
override Py::Attribute getNode() { result = super.getNode() }
}
/** A control flow node corresponding to a `from ... import ...` expression */
class ImportMemberNode extends ControlFlowNode {
ImportMemberNode() { toAst(this) instanceof ImportMember }
ImportMemberNode() { toAst(this) instanceof Py::ImportMember }
/**
* Gets the flow node corresponding to the module in the import-member expression corresponding to this flow node,
* with the matching name
*/
ControlFlowNode getModule(string name) {
exists(ImportMember i | this.getNode() = i and i.getModule() = result.getNode() |
exists(Py::ImportMember i | this.getNode() = i and i.getModule() = result.getNode() |
i.getName() = name and
result.getBasicBlock().dominates(this.getBasicBlock())
)
}
override ImportMember getNode() { result = super.getNode() }
override Py::ImportMember getNode() { result = super.getNode() }
}
/** A control flow node corresponding to an artificial expression representing an import */
class ImportExprNode extends ControlFlowNode {
ImportExprNode() { toAst(this) instanceof ImportExpr }
ImportExprNode() { toAst(this) instanceof Py::ImportExpr }
override ImportExpr getNode() { result = super.getNode() }
override Py::ImportExpr getNode() { result = super.getNode() }
}
/** A control flow node corresponding to a `from ... import *` statement */
class ImportStarNode extends ControlFlowNode {
ImportStarNode() { toAst(this) instanceof ImportStar }
ImportStarNode() { toAst(this) instanceof Py::ImportStar }
/** Gets the flow node corresponding to the module in the import-star corresponding to this flow node */
ControlFlowNode getModule() {
exists(ImportStar i | this.getNode() = i and i.getModuleExpr() = result.getNode() |
exists(Py::ImportStar i | this.getNode() = i and i.getModuleExpr() = result.getNode() |
result.getBasicBlock().dominates(this.getBasicBlock())
)
}
override ImportStar getNode() { result = super.getNode() }
override Py::ImportStar getNode() { result = super.getNode() }
}
/** A control flow node corresponding to a subscript expression, such as `value[slice]` */
class SubscriptNode extends ControlFlowNode {
SubscriptNode() { toAst(this) instanceof Subscript }
SubscriptNode() { toAst(this) instanceof Py::Subscript }
/** flow node corresponding to the value of the sequence in a subscript operation */
ControlFlowNode getObject() {
exists(Subscript s |
exists(Py::Subscript s |
this.getNode() = s and
s.getObject() = result.getNode() and
result.getBasicBlock().dominates(this.getBasicBlock())
@@ -407,23 +407,23 @@ class SubscriptNode extends ControlFlowNode {
/** flow node corresponding to the index in a subscript operation */
ControlFlowNode getIndex() {
exists(Subscript s |
exists(Py::Subscript s |
this.getNode() = s and
s.getIndex() = result.getNode() and
result.getBasicBlock().dominates(this.getBasicBlock())
)
}
override Subscript getNode() { result = super.getNode() }
override Py::Subscript getNode() { result = super.getNode() }
}
/** A control flow node corresponding to a comparison operation, such as `x<y` */
class CompareNode extends ControlFlowNode {
CompareNode() { toAst(this) instanceof Compare }
CompareNode() { toAst(this) instanceof Py::Compare }
/** Whether left and right are a pair of operands for this comparison */
predicate operands(ControlFlowNode left, Cmpop op, ControlFlowNode right) {
exists(Compare c, Expr eleft, Expr eright |
predicate operands(ControlFlowNode left, Py::Cmpop op, ControlFlowNode right) {
exists(Py::Compare c, Py::Expr eleft, Py::Expr eright |
this.getNode() = c and left.getNode() = eleft and right.getNode() = eright
|
eleft = c.getLeft() and eright = c.getComparator(0) and op = c.getOp(0)
@@ -436,26 +436,26 @@ class CompareNode extends ControlFlowNode {
right.getBasicBlock().dominates(this.getBasicBlock())
}
override Compare getNode() { result = super.getNode() }
override Py::Compare getNode() { result = super.getNode() }
}
/** A control flow node corresponding to a conditional expression such as, `body if test else orelse` */
class IfExprNode extends ControlFlowNode {
IfExprNode() { toAst(this) instanceof IfExp }
IfExprNode() { toAst(this) instanceof Py::IfExp }
/** flow node corresponding to one of the operands of an if-expression */
ControlFlowNode getAnOperand() { result = this.getAPredecessor() }
override IfExp getNode() { result = super.getNode() }
override Py::IfExp getNode() { result = super.getNode() }
}
/** A control flow node corresponding to an assignment expression such as `lhs := rhs`. */
class AssignmentExprNode extends ControlFlowNode {
AssignmentExprNode() { toAst(this) instanceof AssignExpr }
AssignmentExprNode() { toAst(this) instanceof Py::AssignExpr }
/** Gets the flow node corresponding to the left-hand side of the assignment expression */
ControlFlowNode getTarget() {
exists(AssignExpr a |
exists(Py::AssignExpr a |
this.getNode() = a and
a.getTarget() = result.getNode() and
result.getBasicBlock().dominates(this.getBasicBlock())
@@ -464,27 +464,27 @@ class AssignmentExprNode extends ControlFlowNode {
/** Gets the flow node corresponding to the right-hand side of the assignment expression */
ControlFlowNode getValue() {
exists(AssignExpr a |
exists(Py::AssignExpr a |
this.getNode() = a and
a.getValue() = result.getNode() and
result.getBasicBlock().dominates(this.getBasicBlock())
)
}
override AssignExpr getNode() { result = super.getNode() }
override Py::AssignExpr getNode() { result = super.getNode() }
}
/** A control flow node corresponding to a binary expression, such as `x + y` */
class BinaryExprNode extends ControlFlowNode {
BinaryExprNode() { toAst(this) instanceof BinaryExpr }
BinaryExprNode() { toAst(this) instanceof Py::BinaryExpr }
/** flow node corresponding to one of the operands of a binary expression */
ControlFlowNode getAnOperand() { result = this.getLeft() or result = this.getRight() }
override BinaryExpr getNode() { result = super.getNode() }
override Py::BinaryExpr getNode() { result = super.getNode() }
ControlFlowNode getLeft() {
exists(BinaryExpr b |
exists(Py::BinaryExpr b |
this.getNode() = b and
result.getNode() = b.getLeft() and
result.getBasicBlock().dominates(this.getBasicBlock())
@@ -492,7 +492,7 @@ class BinaryExprNode extends ControlFlowNode {
}
ControlFlowNode getRight() {
exists(BinaryExpr b |
exists(Py::BinaryExpr b |
this.getNode() = b and
result.getNode() = b.getRight() and
result.getBasicBlock().dominates(this.getBasicBlock())
@@ -500,11 +500,11 @@ class BinaryExprNode extends ControlFlowNode {
}
/** Gets the operator of this binary expression node. */
Operator getOp() { result = this.getNode().getOp() }
Py::Operator getOp() { result = this.getNode().getOp() }
/** Whether left and right are a pair of operands for this binary expression */
predicate operands(ControlFlowNode left, Operator op, ControlFlowNode right) {
exists(BinaryExpr b, Expr eleft, Expr eright |
predicate operands(ControlFlowNode left, Py::Operator op, ControlFlowNode right) {
exists(Py::BinaryExpr b, Py::Expr eleft, Py::Expr eright |
this.getNode() = b and left.getNode() = eleft and right.getNode() = eright
|
eleft = b.getLeft() and eright = b.getRight() and op = b.getOp()
@@ -516,20 +516,20 @@ class BinaryExprNode extends ControlFlowNode {
/** A control flow node corresponding to a boolean shortcut (and/or) operation */
class BoolExprNode extends ControlFlowNode {
BoolExprNode() { toAst(this) instanceof BoolExpr }
BoolExprNode() { toAst(this) instanceof Py::BoolExpr }
/** flow node corresponding to one of the operands of a boolean expression */
ControlFlowNode getAnOperand() {
exists(BoolExpr b | this.getNode() = b and result.getNode() = b.getAValue()) and
exists(Py::BoolExpr b | this.getNode() = b and result.getNode() = b.getAValue()) and
this.getBasicBlock().dominates(result.getBasicBlock())
}
override BoolExpr getNode() { result = super.getNode() }
override Py::BoolExpr getNode() { result = super.getNode() }
}
/** A control flow node corresponding to a unary expression: (`+x`), (`-x`) or (`~x`) */
class UnaryExprNode extends ControlFlowNode {
UnaryExprNode() { toAst(this) instanceof UnaryExpr }
UnaryExprNode() { toAst(this) instanceof Py::UnaryExpr }
/**
* Gets flow node corresponding to the operand of a unary expression.
@@ -540,7 +540,7 @@ class UnaryExprNode extends ControlFlowNode {
*/
ControlFlowNode getOperand() { result = this.getAPredecessor() }
override UnaryExpr getNode() { result = super.getNode() }
override Py::UnaryExpr getNode() { result = super.getNode() }
override ControlFlowNode getAChild() { result = this.getAPredecessor() }
}
@@ -555,27 +555,27 @@ class DefinitionNode extends ControlFlowNode {
cached
DefinitionNode() {
Stages::AST::ref() and
exists(Assign a | a.getATarget().getAFlowNode() = this)
exists(Py::Assign a | this.getNode() = a.getATarget())
or
exists(AssignExpr a | a.getTarget().getAFlowNode() = this)
exists(Py::AssignExpr a | this.getNode() = a.getTarget())
or
exists(AnnAssign a | a.getTarget().getAFlowNode() = this and exists(a.getValue()))
exists(Py::AnnAssign a | this.getNode() = a.getTarget() and exists(a.getValue()))
or
exists(Alias a | a.getAsname().getAFlowNode() = this)
exists(Py::Alias a | this.getNode() = a.getAsname())
or
augstore(_, this)
or
// `x, y = 1, 2` where LHS is a combination of list or tuples
exists(Assign a | list_or_tuple_nested_element(a.getATarget()).getAFlowNode() = this)
exists(Py::Assign a | this.getNode() = list_or_tuple_nested_element(a.getATarget()))
or
exists(For for | for.getTarget().getAFlowNode() = this)
exists(Py::For for | this.getNode() = for.getTarget())
or
exists(Parameter param | this = param.asName().getAFlowNode() and exists(param.getDefault()))
exists(Py::Parameter param | this.getNode() = param.asName() and exists(param.getDefault()))
}
/** flow node corresponding to the value assigned for the definition corresponding to this flow node */
ControlFlowNode getValue() {
result = assigned_value(this.getNode()).getAFlowNode() and
result.getNode() = assigned_value(this.getNode()) and
(
result.getBasicBlock().dominates(this.getBasicBlock())
or
@@ -584,16 +584,16 @@ class DefinitionNode extends ControlFlowNode {
// since the default value for a parameter is evaluated in the same basic block as
// the function definition, but the parameter belongs to the basic block of the function,
// there is no dominance relationship between the two.
exists(Parameter param | this = param.asName().getAFlowNode())
exists(Py::Parameter param | this.getNode() = param.asName())
)
}
}
private Expr list_or_tuple_nested_element(Expr list_or_tuple) {
exists(Expr elt |
elt = list_or_tuple.(Tuple).getAnElt()
private Py::Expr list_or_tuple_nested_element(Py::Expr list_or_tuple) {
exists(Py::Expr elt |
elt = list_or_tuple.(Py::Tuple).getAnElt()
or
elt = list_or_tuple.(List).getAnElt()
elt = list_or_tuple.(Py::List).getAnElt()
|
result = elt
or
@@ -603,12 +603,12 @@ private Expr list_or_tuple_nested_element(Expr list_or_tuple) {
/**
* A control flow node corresponding to a deletion statement, such as `del x`.
* There can be multiple `DeletionNode`s for each `Delete` such that each
* There can be multiple `DeletionNode`s for each `Py::Delete` such that each
* target has own `DeletionNode`. The CFG for `del a, x.y` looks like:
* `NameNode('a') -> DeletionNode -> NameNode('b') -> AttrNode('y') -> DeletionNode`.
*/
class DeletionNode extends ControlFlowNode {
DeletionNode() { toAst(this) instanceof Delete }
DeletionNode() { toAst(this) instanceof Py::Delete }
/** Gets the unique target of this deletion node. */
ControlFlowNode getTarget() { result.getASuccessor() = this }
@@ -617,9 +617,9 @@ class DeletionNode extends ControlFlowNode {
/** A control flow node corresponding to a sequence (tuple or list) literal */
abstract class SequenceNode extends ControlFlowNode {
SequenceNode() {
toAst(this) instanceof Tuple
toAst(this) instanceof Py::Tuple
or
toAst(this) instanceof List
toAst(this) instanceof Py::List
}
/** Gets the control flow node for an element of this sequence */
@@ -632,11 +632,11 @@ abstract class SequenceNode extends ControlFlowNode {
/** A control flow node corresponding to a tuple expression such as `( 1, 3, 5, 7, 9 )` */
class TupleNode extends SequenceNode {
TupleNode() { toAst(this) instanceof Tuple }
TupleNode() { toAst(this) instanceof Py::Tuple }
override ControlFlowNode getElement(int n) {
Stages::AST::ref() and
exists(Tuple t | this.getNode() = t and result.getNode() = t.getElt(n)) and
exists(Py::Tuple t | this.getNode() = t and result.getNode() = t.getElt(n)) and
(
result.getBasicBlock().dominates(this.getBasicBlock())
or
@@ -647,10 +647,10 @@ class TupleNode extends SequenceNode {
/** A control flow node corresponding to a list expression, such as `[ 1, 3, 5, 7, 9 ]` */
class ListNode extends SequenceNode {
ListNode() { toAst(this) instanceof List }
ListNode() { toAst(this) instanceof Py::List }
override ControlFlowNode getElement(int n) {
exists(List l | this.getNode() = l and result.getNode() = l.getElt(n)) and
exists(Py::List l | this.getNode() = l and result.getNode() = l.getElt(n)) and
(
result.getBasicBlock().dominates(this.getBasicBlock())
or
@@ -661,10 +661,10 @@ class ListNode extends SequenceNode {
/** A control flow node corresponding to a set expression, such as `{ 1, 3, 5, 7, 9 }` */
class SetNode extends ControlFlowNode {
SetNode() { toAst(this) instanceof Set }
SetNode() { toAst(this) instanceof Py::Set }
ControlFlowNode getAnElement() {
exists(Set s | this.getNode() = s and result.getNode() = s.getElt(_)) and
exists(Py::Set s | this.getNode() = s and result.getNode() = s.getElt(_)) and
(
result.getBasicBlock().dominates(this.getBasicBlock())
or
@@ -675,20 +675,20 @@ class SetNode extends ControlFlowNode {
/** A control flow node corresponding to a dictionary literal, such as `{ 'a': 1, 'b': 2 }` */
class DictNode extends ControlFlowNode {
DictNode() { toAst(this) instanceof Dict }
DictNode() { toAst(this) instanceof Py::Dict }
/**
* Gets a key of this dictionary literal node, for those items that have keys
* E.g, in {'a':1, **b} this returns only 'a'
*/
ControlFlowNode getAKey() {
exists(Dict d | this.getNode() = d and result.getNode() = d.getAKey()) and
exists(Py::Dict d | this.getNode() = d and result.getNode() = d.getAKey()) and
result.getBasicBlock().dominates(this.getBasicBlock())
}
/** Gets a value of this dictionary literal node */
ControlFlowNode getAValue() {
exists(Dict d | this.getNode() = d and result.getNode() = d.getAValue()) and
exists(Py::Dict d | this.getNode() = d and result.getNode() = d.getAValue()) and
result.getBasicBlock().dominates(this.getBasicBlock())
}
}
@@ -712,21 +712,23 @@ class IterableNode extends ControlFlowNode {
}
}
private AstNode assigned_value(Expr lhs) {
private Py::AstNode assigned_value(Py::Expr lhs) {
/* lhs = result */
exists(Assign a | a.getATarget() = lhs and result = a.getValue())
exists(Py::Assign a | a.getATarget() = lhs and result = a.getValue())
or
/* lhs := result */
exists(AssignExpr a | a.getTarget() = lhs and result = a.getValue())
exists(Py::AssignExpr a | a.getTarget() = lhs and result = a.getValue())
or
/* lhs : annotation = result */
exists(AnnAssign a | a.getTarget() = lhs and result = a.getValue())
exists(Py::AnnAssign a | a.getTarget() = lhs and result = a.getValue())
or
/* import result as lhs */
exists(Alias a | a.getAsname() = lhs and result = a.getValue())
exists(Py::Alias a | a.getAsname() = lhs and result = a.getValue())
or
/* lhs += x => result = (lhs + x) */
exists(AugAssign a, BinaryExpr b | b = a.getOperation() and result = b and lhs = b.getLeft())
exists(Py::AugAssign a, Py::BinaryExpr b |
b = a.getOperation() and result = b and lhs = b.getLeft()
)
or
/*
* ..., lhs, ... = ..., result, ...
@@ -734,31 +736,31 @@ private AstNode assigned_value(Expr lhs) {
* ..., (..., lhs, ...), ... = ..., (..., result, ...), ...
*/
exists(Assign a | nested_sequence_assign(a.getATarget(), a.getValue(), lhs, result))
exists(Py::Assign a | nested_sequence_assign(a.getATarget(), a.getValue(), lhs, result))
or
/* for lhs in seq: => `result` is the `for` node, representing the `iter(next(seq))` operation. */
result.(For).getTarget() = lhs
result.(Py::For).getTarget() = lhs
or
exists(Parameter param | lhs = param.asName() and result = param.getDefault())
exists(Py::Parameter param | lhs = param.asName() and result = param.getDefault())
}
predicate nested_sequence_assign(
Expr left_parent, Expr right_parent, Expr left_result, Expr right_result
Py::Expr left_parent, Py::Expr right_parent, Py::Expr left_result, Py::Expr right_result
) {
exists(Assign a |
exists(Py::Assign a |
a.getATarget().getASubExpression*() = left_parent and
a.getValue().getASubExpression*() = right_parent
) and
exists(int i, Expr left_elem, Expr right_elem |
exists(int i, Py::Expr left_elem, Py::Expr right_elem |
(
left_elem = left_parent.(Tuple).getElt(i)
left_elem = left_parent.(Py::Tuple).getElt(i)
or
left_elem = left_parent.(List).getElt(i)
left_elem = left_parent.(Py::List).getElt(i)
) and
(
right_elem = right_parent.(Tuple).getElt(i)
right_elem = right_parent.(Py::Tuple).getElt(i)
or
right_elem = right_parent.(List).getElt(i)
right_elem = right_parent.(Py::List).getElt(i)
)
|
left_result = left_elem and right_result = right_elem
@@ -769,9 +771,9 @@ predicate nested_sequence_assign(
/** A flow node for a `for` statement. */
class ForNode extends ControlFlowNode {
ForNode() { toAst(this) instanceof For }
ForNode() { toAst(this) instanceof Py::For }
override For getNode() { result = super.getNode() }
override Py::For getNode() { result = super.getNode() }
/** Holds if this `for` statement causes iteration over `sequence` storing each step of the iteration in `target` */
predicate iterates(ControlFlowNode target, ControlFlowNode sequence) {
@@ -782,7 +784,7 @@ class ForNode extends ControlFlowNode {
/** Gets the sequence node for this `for` statement. */
ControlFlowNode getSequence() {
exists(For for |
exists(Py::For for |
toAst(this) = for and
for.getIter() = result.getNode()
|
@@ -792,7 +794,7 @@ class ForNode extends ControlFlowNode {
/** A possible `target` for this `for` statement, not accounting for loop unrolling */
private ControlFlowNode possibleTarget() {
exists(For for |
exists(Py::For for |
toAst(this) = for and
for.getTarget() = result.getNode() and
this.getBasicBlock().dominates(result.getBasicBlock())
@@ -809,11 +811,11 @@ class ForNode extends ControlFlowNode {
/** A flow node for a `raise` statement */
class RaiseStmtNode extends ControlFlowNode {
RaiseStmtNode() { toAst(this) instanceof Raise }
RaiseStmtNode() { toAst(this) instanceof Py::Raise }
/** Gets the control flow node for the exception raised by this raise statement */
ControlFlowNode getException() {
exists(Raise r |
exists(Py::Raise r |
r = toAst(this) and
r.getException() = toAst(result) and
result.getBasicBlock().dominates(this.getBasicBlock())
@@ -827,36 +829,36 @@ class RaiseStmtNode extends ControlFlowNode {
*/
class NameNode extends ControlFlowNode {
NameNode() {
exists(Name n | py_flow_bb_node(this, n, _, _))
exists(Py::Name n | py_flow_bb_node(this, n, _, _))
or
exists(PlaceHolder p | py_flow_bb_node(this, p, _, _))
exists(Py::PlaceHolder p | py_flow_bb_node(this, p, _, _))
}
/** Whether this flow node defines the variable `v`. */
predicate defines(Variable v) {
exists(Name d | this.getNode() = d and d.defines(v)) and
predicate defines(Py::Variable v) {
exists(Py::Name d | this.getNode() = d and d.defines(v)) and
not this.isLoad()
}
/** Whether this flow node deletes the variable `v`. */
predicate deletes(Variable v) { exists(Name d | this.getNode() = d and d.deletes(v)) }
predicate deletes(Py::Variable v) { exists(Py::Name d | this.getNode() = d and d.deletes(v)) }
/** Whether this flow node uses the variable `v`. */
predicate uses(Variable v) {
predicate uses(Py::Variable v) {
this.isLoad() and
exists(Name u | this.getNode() = u and u.uses(v))
exists(Py::Name u | this.getNode() = u and u.uses(v))
or
exists(PlaceHolder u |
this.getNode() = u and u.getVariable() = v and u.getCtx() instanceof Load
exists(Py::PlaceHolder u |
this.getNode() = u and u.getVariable() = v and u.getCtx() instanceof Py::Load
)
or
Scopes::use_of_global_variable(this, v.getScope(), v.getId())
}
string getId() {
result = this.getNode().(Name).getId()
result = this.getNode().(Py::Name).getId()
or
result = this.getNode().(PlaceHolder).getId()
result = this.getNode().(Py::PlaceHolder).getId()
}
/** Whether this is a use of a local variable. */
@@ -868,82 +870,84 @@ class NameNode extends ControlFlowNode {
/** Whether this is a use of a global (including builtin) variable. */
predicate isGlobal() { Scopes::use_of_global_variable(this, _, _) }
predicate isSelf() { exists(SsaVariable selfvar | selfvar.isSelf() and selfvar.getAUse() = this) }
predicate isSelf() {
exists(Py::SsaVariable selfvar | selfvar.isSelf() and selfvar.getAUse() = this)
}
}
/** A control flow node corresponding to a named constant, one of `None`, `True` or `False`. */
class NameConstantNode extends NameNode {
NameConstantNode() { exists(NameConstant n | py_flow_bb_node(this, n, _, _)) }
NameConstantNode() { exists(Py::NameConstant n | py_flow_bb_node(this, n, _, _)) }
/*
* We ought to override uses as well, but that has
* a serious performance impact.
* deprecated predicate uses(Variable v) { none() }
* deprecated predicate uses(Py::Variable v) { none() }
*/
}
/** A control flow node corresponding to a starred expression, `*a`. */
class StarredNode extends ControlFlowNode {
StarredNode() { toAst(this) instanceof Starred }
StarredNode() { toAst(this) instanceof Py::Starred }
ControlFlowNode getValue() { toAst(result) = toAst(this).(Starred).getValue() }
ControlFlowNode getValue() { toAst(result) = toAst(this).(Py::Starred).getValue() }
}
/** The ControlFlowNode for an 'except' statement. */
class ExceptFlowNode extends ControlFlowNode {
ExceptFlowNode() { this.getNode() instanceof ExceptStmt }
ExceptFlowNode() { this.getNode() instanceof Py::ExceptStmt }
/**
* Gets the type handled by this exception handler.
* `ExceptionType` in `except ExceptionType as e:`
* `Py::ExceptionType` in `except Py::ExceptionType as e:`
*/
ControlFlowNode getType() {
exists(ExceptStmt ex |
exists(Py::ExceptStmt ex |
this.getBasicBlock().dominates(result.getBasicBlock()) and
ex = this.getNode() and
result = ex.getType().getAFlowNode()
result.getNode() = ex.getType()
)
}
/**
* Gets the name assigned to the handled exception, if any.
* `e` in `except ExceptionType as e:`
* `e` in `except Py::ExceptionType as e:`
*/
ControlFlowNode getName() {
exists(ExceptStmt ex |
exists(Py::ExceptStmt ex |
this.getBasicBlock().dominates(result.getBasicBlock()) and
ex = this.getNode() and
result = ex.getName().getAFlowNode()
result.getNode() = ex.getName()
)
}
}
/** The ControlFlowNode for an 'except*' statement. */
class ExceptGroupFlowNode extends ControlFlowNode {
ExceptGroupFlowNode() { this.getNode() instanceof ExceptGroupStmt }
ExceptGroupFlowNode() { this.getNode() instanceof Py::ExceptGroupStmt }
/**
* Gets the type handled by this exception handler.
* `ExceptionType` in `except* ExceptionType as e:`
* `Py::ExceptionType` in `except* Py::ExceptionType as e:`
*/
ControlFlowNode getType() {
this.getBasicBlock().dominates(result.getBasicBlock()) and
result = this.getNode().(ExceptGroupStmt).getType().getAFlowNode()
result.getNode() = this.getNode().(Py::ExceptGroupStmt).getType()
}
/**
* Gets the name assigned to the handled exception, if any.
* `e` in `except* ExceptionType as e:`
* `e` in `except* Py::ExceptionType as e:`
*/
ControlFlowNode getName() {
this.getBasicBlock().dominates(result.getBasicBlock()) and
result = this.getNode().(ExceptGroupStmt).getName().getAFlowNode()
result.getNode() = this.getNode().(Py::ExceptGroupStmt).getName()
}
}
private module Scopes {
private predicate fast_local(NameNode n) {
exists(FastLocalVariable v |
exists(Py::FastLocalVariable v |
n.uses(v) and
v.getScope() = n.getScope()
)
@@ -952,15 +956,15 @@ private module Scopes {
predicate local(NameNode n) {
fast_local(n)
or
exists(SsaVariable var |
exists(Py::SsaVariable var |
var.getAUse() = n and
n.getScope() instanceof Class and
n.getScope() instanceof Py::Class and
exists(var.getDefinition())
)
}
predicate non_local(NameNode n) {
exists(FastLocalVariable flv |
exists(Py::FastLocalVariable flv |
flv.getALoad() = n.getNode() and
not flv.getScope() = n.getScope()
)
@@ -968,20 +972,20 @@ private module Scopes {
// magic is fine, but we get questionable join-ordering of it
pragma[nomagic]
predicate use_of_global_variable(NameNode n, Module scope, string name) {
predicate use_of_global_variable(NameNode n, Py::Module scope, string name) {
n.isLoad() and
not non_local(n) and
not exists(SsaVariable var | var.getAUse() = n |
var.getVariable() instanceof FastLocalVariable
not exists(Py::SsaVariable var | var.getAUse() = n |
var.getVariable() instanceof Py::FastLocalVariable
or
n.getScope() instanceof Class and
n.getScope() instanceof Py::Class and
not maybe_undefined(var)
) and
name = n.getId() and
scope = n.getEnclosingModule()
}
private predicate maybe_undefined(SsaVariable var) {
private predicate maybe_undefined(Py::SsaVariable var) {
not exists(var.getDefinition()) and not py_ssa_phi(var, _)
or
var.getDefinition().isDelete()
@@ -1058,13 +1062,13 @@ class BasicBlock extends @py_flow_node {
private predicate oneNodeBlock() { this.firstNode() = this.getLastNode() }
private predicate startLocationInfo(string file, int line, int col) {
if this.firstNode().getNode() instanceof Scope
if this.firstNode().getNode() instanceof Py::Scope
then this.firstNode().getASuccessor().getLocation().hasLocationInfo(file, line, col, _, _)
else this.firstNode().getLocation().hasLocationInfo(file, line, col, _, _)
}
private predicate endLocationInfo(int endl, int endc) {
if this.getLastNode().getNode() instanceof Scope and not this.oneNodeBlock()
if this.getLastNode().getNode() instanceof Py::Scope and not this.oneNodeBlock()
then this.getLastNode().getAPredecessor().getLocation().hasLocationInfo(_, _, _, endl, endc)
else this.getLastNode().getLocation().hasLocationInfo(_, _, _, endl, endc)
}
@@ -1081,7 +1085,7 @@ class BasicBlock extends @py_flow_node {
/** Whether flow from this basic block reaches a normal exit from its scope */
predicate reachesExit() {
exists(Scope s | s.getANormalExit().getBasicBlock() = this)
exists(Py::Scope s | s.getANormalExit().getBasicBlock() = this)
or
this.getASuccessor().reachesExit()
}
@@ -1122,7 +1126,7 @@ class BasicBlock extends @py_flow_node {
/** Gets the scope of this block */
pragma[nomagic]
Scope getScope() {
Py::Scope getScope() {
exists(ControlFlowNode n | n.getBasicBlock() = this |
/* Take care not to use an entry or exit node as that node's scope will be the outer scope */
not py_scope_flow(n, _, -1) and
@@ -1145,17 +1149,17 @@ class BasicBlock extends @py_flow_node {
predicate reaches(BasicBlock other) { this = other or this.strictlyReaches(other) }
/**
* Gets the `ConditionBlock`, if any, that controls this block and
* does not control any other `ConditionBlock`s that control this block.
* That is the `ConditionBlock` that is closest dominator.
* Gets the `Py::ConditionBlock`, if any, that controls this block and
* does not control any other `Py::ConditionBlock`s that control this block.
* That is the `Py::ConditionBlock` that is closest dominator.
*/
ConditionBlock getImmediatelyControllingBlock() {
Py::ConditionBlock getImmediatelyControllingBlock() {
result = this.nonControllingImmediateDominator*().getImmediateDominator()
}
private BasicBlock nonControllingImmediateDominator() {
result = this.getImmediateDominator() and
not result.(ConditionBlock).controls(this, _)
not result.(Py::ConditionBlock).controls(this, _)
}
/**
@@ -1175,7 +1179,7 @@ private class ControlFlowNodeAlias = ControlFlowNode;
final private class FinalBasicBlock = BasicBlock;
module Cfg implements BB::CfgSig<Location> {
module Cfg implements BB::CfgSig<Py::Location> {
private import codeql.controlflow.SuccessorType
class ControlFlowNode = ControlFlowNodeAlias;
@@ -1186,7 +1190,7 @@ module Cfg implements BB::CfgSig<Location> {
// Using the location of the first node is simple
// and we just need a way to identify the basic block
// during debugging, so this will be serviceable.
Location getLocation() { result = super.getNode(0).getLocation() }
Py::Location getLocation() { result = super.getNode(0).getLocation() }
int length() { result = count(int i | exists(this.getNode(i))) }

View File

@@ -153,8 +153,16 @@ class Function extends Function_, Scope, AstNode {
override predicate contains(AstNode inner) { Scope.super.contains(inner) }
/** Gets a control flow node for a return value of this function */
ControlFlowNode getAReturnValueFlowNode() {
/**
* DEPRECATED: bind a `Return` node explicitly instead, e.g.
* `exists(Return ret | ret.getScope() = this and n.getNode() = ret.getValue())`.
* This API is being phased out together with `AstNode.getAFlowNode()` to
* untangle the AST and CFG hierarchies in preparation for migrating the
* dataflow library off the legacy CFG.
*
* Gets a control flow node for a return value of this function.
*/
deprecated ControlFlowNode getAReturnValueFlowNode() {
exists(Return ret |
ret.getScope() = this and
ret.getValue() = result.getNode()

View File

@@ -162,8 +162,6 @@ class ImportMember extends ImportMember_ {
string getImportedModuleName() {
result = this.getModule().(ImportExpr).getImportedModuleName() + "." + this.getName()
}
override ImportMemberNode getAFlowNode() { result = super.getAFlowNode() }
}
/** An import statement */

View File

@@ -46,20 +46,23 @@ class SelfAttributeRead extends SelfAttribute {
}
predicate guardedByHasattr() {
exists(Variable var, ControlFlowNode n |
var.getAUse() = this.getObject().getAFlowNode() and
exists(Variable var, ControlFlowNode n, ControlFlowNode this_, ControlFlowNode obj_ |
this_.getNode() = this and obj_.getNode() = this.getObject()
|
var.getAUse() = obj_ and
hasattr(n, var.getAUse(), this.getName()) and
n.strictlyDominates(this.getAFlowNode())
n.strictlyDominates(this_)
)
}
pragma[noinline]
predicate locallyDefined() {
exists(SelfAttributeStore store |
this.getName() = store.getName() and
this.getScope() = store.getScope()
exists(SelfAttributeStore store, ControlFlowNode store_, ControlFlowNode this_ |
store_.getNode() = store and this_.getNode() = this
|
store.getAFlowNode().strictlyDominates(this.getAFlowNode())
this.getName() = store.getName() and
this.getScope() = store.getScope() and
store_.strictlyDominates(this_)
)
}
}

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

View File

@@ -5,24 +5,30 @@ private import semmle.python.dataflow.new.DataFlow
private predicate constCompare(DataFlow::GuardNode g, ControlFlowNode node, boolean branch) {
exists(CompareNode cn | cn = g |
exists(ImmutableLiteral const, Cmpop op |
op = any(Eq eq) and branch = true
or
op = any(NotEq ne) and branch = false
exists(ImmutableLiteral const, Cmpop op, ControlFlowNode c |
c.getNode() = const and
(
op = any(Eq eq) and branch = true
or
op = any(NotEq ne) and branch = false
)
|
cn.operands(const.getAFlowNode(), op, node)
cn.operands(c, op, node)
or
cn.operands(node, op, const.getAFlowNode())
cn.operands(node, op, c)
)
or
exists(NameConstant const, Cmpop op |
op = any(Is is_) and branch = true
or
op = any(IsNot isn) and branch = false
exists(NameConstant const, Cmpop op, ControlFlowNode c |
c.getNode() = const and
(
op = any(Is is_) and branch = true
or
op = any(IsNot isn) and branch = false
)
|
cn.operands(const.getAFlowNode(), op, node)
cn.operands(c, op, node)
or
cn.operands(node, op, const.getAFlowNode())
cn.operands(node, op, c)
)
or
exists(IterableNode const_iterable, Cmpop op |

View File

@@ -228,7 +228,7 @@ private class ClassDefinitionAsAttrWrite extends AttrWrite, CfgNode {
override Node getValue() { result.asCfgNode() = node.getValue() }
override Node getObject() { result.asCfgNode() = cls.getAFlowNode() }
override Node getObject() { result.asCfgNode().getNode() = cls }
override ExprNode getAttributeNameExpr() { none() }

View File

@@ -256,9 +256,12 @@ predicate parameterMatch(ParameterPosition ppos, ArgumentPosition apos) {
*/
overlay[local]
predicate isStaticmethod(Function func) {
exists(NameNode id | id.getId() = "staticmethod" and id.isGlobal() |
func.getADecorator() = id.getNode()
)
// The decorator is *syntactically* a `Name` "staticmethod" — we don't
// care which variable it resolves to. `staticmethod` is a builtin and
// is almost never shadowed in a module-level scope; even if a class
// redefines `staticmethod` in its body, the class body has not started
// executing yet at the decorator position, so Python uses the builtin.
func.getADecorator().(Name).getId() = "staticmethod"
}
/**
@@ -268,9 +271,9 @@ predicate isStaticmethod(Function func) {
*/
overlay[local]
predicate isClassmethod(Function func) {
exists(NameNode id | id.getId() = "classmethod" and id.isGlobal() |
func.getADecorator() = id.getNode()
)
// See `isStaticmethod` for the rationale for matching on the AST `Name`
// rather than going via the CFG and `isGlobal()`.
func.getADecorator().(Name).getId() = "classmethod"
or
exists(Class cls |
cls.getAMethod() = func and
@@ -285,9 +288,8 @@ predicate isClassmethod(Function func) {
/** Holds if the function `func` has a `property` decorator. */
overlay[local]
predicate hasPropertyDecorator(Function func) {
exists(NameNode id | id.getId() = "property" and id.isGlobal() |
func.getADecorator() = id.getNode()
)
// See `isStaticmethod` for the rationale for matching on the AST `Name`.
func.getADecorator().(Name).getId() = "property"
}
/**
@@ -1911,8 +1913,8 @@ abstract class ReturnNode extends Node {
class ExtractedReturnNode extends ReturnNode, CfgNode {
// See `TaintTrackingImplementation::returnFlowStep`
ExtractedReturnNode() {
node = any(Return ret).getValue().getAFlowNode() or
node = any(Yield yield).getAFlowNode()
node.getNode() = any(Return ret).getValue() or
node.getNode() = any(Yield yield)
}
override ReturnKind getKind() { any() }
@@ -1930,7 +1932,7 @@ class ExtractedReturnNode extends ReturnNode, CfgNode {
class YieldNodeInContextManagerFunction extends ReturnNode, CfgNode {
YieldNodeInContextManagerFunction() {
hasContextmanagerDecorator(node.getScope()) and
node = any(Yield yield).getValue().getAFlowNode()
node.getNode() = any(Yield yield).getValue()
}
override ReturnKind getKind() { any() }

View File

@@ -185,8 +185,8 @@ private predicate synthDictSplatArgumentNodeStoreStep(
*/
predicate yieldStoreStep(Node nodeFrom, Content c, Node nodeTo) {
exists(Yield yield |
nodeTo.asCfgNode() = yield.getAFlowNode() and
nodeFrom.asCfgNode() = yield.getValue().getAFlowNode() and
nodeTo.asCfgNode().getNode() = yield and
nodeFrom.asCfgNode().getNode() = yield.getValue() and
// TODO: Consider if this will also need to transfer dictionary content
// once dictionary comprehensions are supported.
c instanceof ListElementContent

View File

@@ -485,7 +485,7 @@ class ModuleVariableNode extends Node, TModuleVariableNode {
/** Gets a node that reads this variable, excluding reads that happen through `from ... import *`. */
Node getALocalRead() {
result.asCfgNode() = var.getALoad().getAFlowNode() and
result.asCfgNode().getNode() = var.getALoad() and
not result.getScope() = mod
}

View File

@@ -9,7 +9,19 @@ private import semmle.python.dataflow.new.DataFlow
private import semmle.python.dataflow.new.internal.ImportStar
private import semmle.python.dataflow.new.TypeTracking
private import semmle.python.dataflow.new.internal.DataFlowPrivate
private import semmle.python.essa.SsaDefinitions
/**
* Holds if `init` is a package's `__init__.py` and `var` is a global variable in
* `init` whose name matches a submodule of the package.
*
* Inlined from `SsaSource::init_module_submodule_defn` to avoid pulling
* `semmle.python.essa.SsaDefinitions` into the new dataflow stack.
*/
private predicate initModuleSubmoduleDefn(GlobalVariable var, Module init) {
init.isPackageInit() and
exists(init.getPackage().getSubModule(var.getId())) and
var.getScope() = init
}
/**
* Python modules and the way imports are resolved are... complicated. Here's a crash course in how
@@ -326,7 +338,7 @@ module ImportResolution {
// imported yet.
exists(string submodule, Module package, EssaVariable var |
submodule = var.getName() and
SsaSource::init_module_submodule_defn(var.getSourceVariable(), package.getEntryNode()) and
initModuleSubmoduleDefn(var.getSourceVariable(), package) and
m = getModuleFromName(package.getPackageName() + "." + submodule) and
result.asCfgNode() = var.getDefinition().(EssaNodeDefinition).getDefiningNode()
)

View File

@@ -0,0 +1,547 @@
/**
* Provides the Python SSA implementation built on the new (shared) CFG.
*
* Mirrors the Java SSA adapter at
* `java/ql/lib/semmle/code/java/dataflow/internal/SsaImpl.qll`:
* an `InputSig` is defined in terms of positional `(BasicBlock, int)`
* variable references, and the shared
* `codeql.ssa.Ssa::Make<Location, Cfg, Input>` module is then
* instantiated.
*
* `SourceVariable` is the AST-level `Py::Variable`. Variable references
* are looked up via the CFG facade's `NameNode.defines`/`uses`/`deletes`
* predicates, which themselves are one-line bridges to AST-level
* `Name.defines`/`uses`/`deletes`.
*
* Implicit-entry definitions are inserted for:
* - non-local / global / builtin variables that are read in the scope
* but never assigned (no enclosing CFG node defines them),
* - captured variables (variables defined in an enclosing scope that
* are read inside the scope), and
* - parameters, but only if the corresponding parameter name is *not*
* itself a CFG node. With the C#-style parameter wiring already
* installed in `AstNodeImpl.qll`, parameter names *are* CFG nodes,
* so the regular `variableWrite` path handles them — no `i = -1`
* entry is needed for ordinary parameters.
*/
overlay[local?]
module;
private import python as Py
private import semmle.python.controlflow.internal.AstNodeImpl as CfgImpl
private import semmle.python.controlflow.internal.Cfg as Cfg
private import codeql.ssa.Ssa as SsaImplCommon
private import codeql.controlflow.BasicBlock as BB
/**
* Adapts the Python `Cfg` facade to the shared SSA library's `CfgSig`.
* All members are inherited from `Cfg::ControlFlowNode` and
* `Cfg::BasicBlock`.
*/
private module CfgForSsa implements BB::CfgSig<Py::Location> {
class ControlFlowNode = CfgImpl::ControlFlowNode;
class BasicBlock = CfgImpl::BasicBlock;
class EntryBasicBlock = CfgImpl::Cfg::EntryBasicBlock;
predicate dominatingEdge = CfgImpl::Cfg::dominatingEdge/2;
}
/**
* A source variable for SSA, wrapping a Python AST `Variable`.
*
* We only track variables that are read at least once in their scope —
* tracking write-only variables would be unnecessary work — *except*
* for module-scope globals, where the "read" can be external (e.g.
* `import mymodule; mymodule.x`). Such globals are tracked
* unconditionally so that import-resolution can find their defining
* write.
*/
private newtype TSsaSourceVariable =
TPyVar(Py::Variable v) {
// Has a use somewhere — read-relevant for SSA.
exists(Cfg::NameNode n | n.uses(v))
or
// Or has a deletion (treated as a write that destroys the value).
exists(Cfg::NameNode n | n.deletes(v))
or
// Or is a module-scope global written in this module — must be
// tracked even if never read locally, because importers may read
// it as an attribute on the module object.
v.getScope() instanceof Py::Module and
exists(Cfg::NameNode n | n.defines(v))
or
// Or is a parameter — parameters must always have a
// `ParameterDefinition` for dataflow argument-routing to work,
// even if the parameter is never read in its scope. Mirrors
// legacy ESSA's `ParameterDefinition` (which fired for every
// parameter binding regardless of liveness).
exists(Py::Parameter p | p.asName() = v.getAStore())
}
/**
* A source variable for SSA, wrapping a Python AST `Variable`.
*/
class SsaSourceVariable extends TSsaSourceVariable {
/** Gets the underlying Python AST variable. */
Py::Variable getVariable() { this = TPyVar(result) }
/** Gets the (textual) name of this variable. */
string getName() { result = this.getVariable().getId() }
/** Gets a textual representation of this source variable. */
string toString() { result = this.getVariable().toString() }
/** Gets the location of this source variable. */
Py::Location getLocation() { result = this.getVariable().getScope().getLocation() }
/** Gets the scope in which this variable lives. */
Py::Scope getScope() { result = this.getVariable().getScope() }
/**
* Gets a use of this variable as it appears in the source — a `NameNode`
* that loads or deletes the variable. Mirrors legacy
* `SsaSourceVariable.getASourceUse()`.
*/
Cfg::ControlFlowNode getASourceUse() {
exists(Cfg::NameNode n | result = n |
n.uses(this.getVariable()) or n.deletes(this.getVariable())
)
}
/**
* Gets an implicit use of this variable. The new SSA does not have
* implicit-use refinements, but we keep this for API parity — every
* normal-exit of the variable's scope counts as a sink, ensuring
* variables stay live to scope exit for taint-tracking.
*/
Cfg::ControlFlowNode getAnImplicitUse() {
result.isNormalExit() and result.getScope() = this.getScope()
}
/**
* Gets a use of this variable — either an explicit source use or an
* implicit use at scope exit. Mirrors legacy `SsaSourceVariable.getAUse()`.
*/
Cfg::ControlFlowNode getAUse() {
result = this.getASourceUse() or result = this.getAnImplicitUse()
}
}
/**
* Holds if `v` is a non-local read in scope `s`, in the sense that `s`
* uses `v` but does not write it within `s`. This includes globals,
* builtins, and variables captured from an enclosing function scope.
*
* The `Py::Variable` `v` lives in some defining scope (the module for
* globals, an outer function for closures, etc.); the reading scope
* `s` is the scope where the use of `v` occurs.
*/
private predicate nonLocalReadIn(Py::Variable v, Py::Scope s) {
exists(Cfg::NameNode n |
n.uses(v) and
n.getScope() = s and
not exists(Cfg::NameNode def | def.defines(v) and def.getScope() = s)
) and
// Match legacy ESSA: only create entry defs for variables that have
// at least one defining store somewhere — otherwise the entry def
// represents "nothing reaches here", which is the default anyway and
// introduces no useful flow. (Legacy's `ModuleVariable` required a
// store; this is the closure-aware generalisation.)
exists(Cfg::NameNode store | store.defines(v))
}
/**
* Holds if `bb` is the entry basic block of a scope where `v` should
* have an implicit entry definition. This covers:
* - non-local / global / builtin variables read in `s`, and
* - captured variables (defined in an enclosing scope but read in `s`).
*
* Each reading scope gets its own entry def, so a closure variable can
* have multiple entry defs across all functions/methods that read it.
*
* Parameters are *not* included: their bound `Name` is itself a CFG
* node (per the C#-style parameter wiring), so `variableWrite` fires at
* the parameter's natural CFG index.
*/
private predicate hasEntryDefIn(SsaSourceVariable v, CfgImpl::BasicBlock bb) {
exists(Py::Scope s |
nonLocalReadIn(v.getVariable(), s) and
bb = entryBlock(s)
)
}
/**
* Gets the entry basic block of scope `s`, where implicit entry
* definitions are placed (at synthetic index `-1`).
*/
private CfgImpl::BasicBlock entryBlock(Py::Scope s) {
exists(CfgImpl::ControlFlowNode entry |
entry instanceof CfgImpl::ControlFlow::EntryNode and
entry.getEnclosingCallable().asScope() = s and
result = entry.getBasicBlock()
)
}
/**
* The SSA `InputSig` for Python. References are positional
* `(BasicBlock, int)` pairs into the new CFG.
*/
private module SsaImplInput implements SsaImplCommon::InputSig<Py::Location, CfgImpl::BasicBlock> {
class SourceVariable = SsaSourceVariable;
predicate variableWrite(CfgImpl::BasicBlock bb, int i, SourceVariable v, boolean certain) {
// Explicit binding at a CFG node — includes assignments,
// parameter Names (wired in via the C# pattern), exception-handler
// `as`-bindings, import aliases, and match-pattern captures.
exists(Cfg::NameNode n |
bb.getNode(i) = n and
n.defines(v.getVariable()) and
certain = true
)
or
// `del x` — removes the binding. Modelled as a certain write that
// makes any subsequent read invalid.
exists(Cfg::NameNode n |
bb.getNode(i) = n and
n.deletes(v.getVariable()) and
certain = true
)
or
// Implicit entry definition for non-local / captured / global /
// builtin variables read in some scope. Each reading scope's entry
// block gets one such write, allowing closures: e.g. when `x` is a
// parameter of an outer function and read inside a nested
// function, both scopes get entry defs for `x`.
hasEntryDefIn(v, bb) and
i = -1 and
certain = true
or
// `from X import *` — possibly rebinds every name in the importing
// scope. Modelled as an uncertain write at the import-star's CFG
// position for every variable that lives in (or is referenced
// from) the same scope as the import-star. Mirrors legacy ESSA's
// `ImportStarRefinement` (see `essa/SsaDefinitions.qll`'s
// `import_star_refinement` predicate). The write is uncertain so
// that prior definitions of the variable remain available — the
// shared-SSA `SsaUncertainWrite` merges the new value with the
// immediately preceding definition.
exists(Cfg::ImportStarNode imp |
bb.getNode(i) = imp and
certain = false and
(
v.getVariable().getScope() = imp.getScope()
or
// Variable is defined in some other scope but referenced in
// the same scope as the import-star (matches legacy clause 2:
// `other.uses(v) and def.getScope() = other.getScope()`).
exists(Cfg::NameNode other |
other.uses(v.getVariable()) and
imp.getScope() = other.getScope()
)
)
)
}
predicate variableRead(CfgImpl::BasicBlock bb, int i, SourceVariable v, boolean certain) {
// Explicit source use — a `Name` load or a `del x` of the variable.
exists(Cfg::NameNode n |
bb.getNode(i) = n and
n.uses(v.getVariable()) and
certain = true
)
or
// Synthetic use at the normal exit of the variable's defining scope.
// This keeps every variable live to scope exit so that callers (e.g.
// `module_export` in ImportResolution.qll, or taint-tracking pass-through
// through unread locals) can ask "which definition reaches end of
// scope?". Mirrors legacy ESSA's `SsaSourceVariable.getAUse()` which
// included `getScope().getANormalExit()`.
exists(Cfg::ControlFlowNode exit |
exit.isNormalExit() and
exit.getScope() = v.getVariable().getScope() and
bb.getNode(i) = exit and
certain = true
)
}
}
/**
* The shared SSA instantiation for Python.
*
* Members:
* - `Definition` — the union of explicit, uncertain, and phi definitions
* - `WriteDefinition`, `UncertainWriteDefinition`, `PhiNode`
* - the standard SSA predicates (`getAUse`, `getAnUltimateDefinition`, ...).
*/
module Ssa = SsaImplCommon::Make<Py::Location, CfgForSsa, SsaImplInput>;
final class Definition = Ssa::Definition;
final class WriteDefinition = Ssa::WriteDefinition;
final class UncertainWriteDefinition = Ssa::UncertainWriteDefinition;
final class PhiNode = Ssa::PhiNode;
// ===========================================================================
// ESSA-shaped adapter layer
//
// The dataflow library (`python/ql/lib/semmle/python/dataflow/new/`) and
// related modules (`ApiGraphs.qll`, etc.) consume the legacy ESSA API
// (`EssaVariable`, `EssaDefinition`, `AssignmentDefinition`,
// `ScopeEntryDefinition`, `ParameterDefinition`, `WithDefinition`,
// `PhiFunction`, plus the `AdjacentUses` module). To migrate them off
// the legacy CFG, we expose the same API surface on top of the
// shared SSA built above.
//
// This adapter is intentionally narrow: it covers only the predicates
// that new dataflow consumes. The richer legacy ESSA — refinement
// nodes, attribute refinements, edge refinements — stays available
// via `semmle.python.essa.Essa` for points-to / legacy code.
// ===========================================================================
/**
* Gets the CFG node at which a write definition's binding takes place.
*
* For ordinary writes (assignment, deletion, parameter) this is the
* canonical CFG node of the bound Name. For implicit entry definitions
* (synthesised at position `-1` of a scope's entry BB) this is the
* scope's entry node.
*/
private Cfg::ControlFlowNode writeDefNode(Ssa::WriteDefinition def) {
exists(CfgImpl::BasicBlock bb, int i | def.definesAt(_, bb, i) |
i >= 0 and result = bb.getNode(i)
or
i = -1 and result = bb.getNode(0)
)
}
/**
* A write definition whose binding has a corresponding CFG node — i.e.
* everything that's not a phi node. Mirrors legacy ESSA's
* `EssaNodeDefinition`.
*/
class EssaNodeDefinition extends Ssa::WriteDefinition {
/** Gets the CFG node where this definition's binding takes place. */
Cfg::ControlFlowNode getDefiningNode() { result = writeDefNode(this) }
/** Gets the variable defined here (legacy name). */
SsaSourceVariable getVariable() { result = this.getSourceVariable() }
/** Gets the enclosing scope. */
Py::Scope getScope() {
exists(Cfg::ControlFlowNode n | n = this.getDefiningNode() | result = n.getScope())
}
/**
* Holds if this definition defines source variable `v` at CFG node
* `defNode`. Flatter form of `getSourceVariable()` +
* `getDefiningNode()`, matching legacy ESSA's `definedBy`.
*/
predicate definedBy(SsaSourceVariable v, Cfg::ControlFlowNode defNode) {
v = this.getSourceVariable() and defNode = this.getDefiningNode()
}
}
/**
* An assignment definition: any binding where the value being assigned
* is statically known via `Cfg::DefinitionNode.getValue()`. Includes
* plain assignments, walrus, annotated assignments, augmented
* assignments, import aliases (`import x` / `from m import x [as y]`),
* `with ... as x`, and for-target bindings (where `getValue()` returns
* the iter expression's CFG node). Excludes parameter bindings —
* those are modelled by `ParameterDefinition`.
*/
class AssignmentDefinition extends EssaNodeDefinition {
AssignmentDefinition() {
exists(Cfg::NameNode n | n = this.getDefiningNode() |
exists(n.(Cfg::DefinitionNode).getValue()) and
not n.(Cfg::ControlFlowNode).isParameter()
)
}
/** Gets the CFG node for the value being assigned, if statically known. */
Cfg::ControlFlowNode getValue() {
result = this.getDefiningNode().(Cfg::DefinitionNode).getValue()
}
}
/**
* A parameter definition — the binding of a parameter name in a
* function's scope.
*/
class ParameterDefinition extends EssaNodeDefinition {
ParameterDefinition() { this.getDefiningNode().isParameter() }
/** Gets the AST `Parameter` (a `Py::Name` in param context). */
Py::Name getParameter() { result = this.getDefiningNode().getNode() }
}
/**
* A definition introduced by a `with ... as x:` clause.
*/
class WithDefinition extends EssaNodeDefinition {
WithDefinition() {
exists(Cfg::NameNode n, Py::With w |
n = this.getDefiningNode() and
w.getOptionalVars() = n.getNode()
)
}
}
/**
* An assignment where the LHS is a tuple/list and the RHS is unpacked:
* `a, b = (1, 2)` or `a, *rest = xs`. The SSA def lives at the inner
* `Name` CFG node, but for IterableUnpacking integration we expose
* the enclosing `StarredNode` as the `getDefiningNode()` for `*rest`
* patterns — mirroring legacy ESSA's `multi_assignment_definition`,
* which placed the def at the StarredNode CFG node.
*/
class MultiAssignmentDefinition extends EssaNodeDefinition {
MultiAssignmentDefinition() {
exists(Cfg::NameNode n | n = super.getDefiningNode() |
exists(Py::Assign a, Py::Expr lhs |
a.getATarget() = lhs and
(lhs instanceof Py::Tuple or lhs instanceof Py::List) and
lhs.getASubExpression+() = n.getNode()
)
or
// For-loop with tuple/list target: `for a, b in xs:` —
// tuple-unpacking semantics applies to the for-target.
exists(Py::For f, Py::Expr lhs |
f.getTarget() = lhs and
(lhs instanceof Py::Tuple or lhs instanceof Py::List) and
lhs.getASubExpression+() = n.getNode()
)
)
}
override Cfg::ControlFlowNode getDefiningNode() {
// Default: the underlying `Name` CFG node (where the SSA def lives).
not exists(Cfg::StarredNode s |
s.getNode().(Py::Starred).getValue() = super.getDefiningNode().getNode()
) and
result = super.getDefiningNode()
or
// Exception: for `*rest`, expose the enclosing `Starred` CFG node
// so that `IterableUnpacking::iterableUnpackingStarredElementStoreStep`
// can attach the rest-list to it.
exists(Cfg::StarredNode s |
s.getNode().(Py::Starred).getValue() = super.getDefiningNode().getNode()
|
result = s
)
}
}
/**
* An implicit entry definition for a non-local / captured / global /
* builtin variable read in a scope but not defined there.
*
* Inherits from `EssaNodeDefinition` and exposes the scope's entry node
* as its defining node (matching legacy ESSA semantics).
*/
class ScopeEntryDefinition extends EssaNodeDefinition {
ScopeEntryDefinition() {
exists(CfgImpl::BasicBlock bb |
this.definesAt(_, bb, -1) and
bb instanceof CfgImpl::Cfg::EntryBasicBlock
)
}
/** Gets the enclosing scope (the scope whose entry block this def is in). */
override Py::Scope getScope() {
exists(CfgImpl::BasicBlock bb |
this.definesAt(_, bb, -1) and
result = bb.getNode(0).(Cfg::ControlFlowNode).getScope()
)
}
}
/** A phi node (alias matching legacy naming). */
class PhiFunction extends PhiNode {
/**
* Gets an input to this phi function (a definition that flows into
* the phi from one of its predecessor blocks). Mirrors legacy
* ESSA's `PhiFunction.getAnInput()`.
*/
Ssa::Definition getAnInput() { Ssa::phiHasInputFromBlock(this, result, _) }
}
/** Base class for all ESSA definitions (legacy-shaped). */
class EssaDefinition = Ssa::Definition;
/**
* An adapter representing a single SSA-defined "variable" — wrapping
* one `Ssa::Definition`. Mirrors legacy `EssaVariable` API.
*/
class EssaVariable extends Ssa::Definition {
/** Gets the underlying SSA definition (legacy name). */
Ssa::Definition getDefinition() { result = this }
/**
* Gets a CFG node where this definition is used. Includes regular
* `Name` reads as well as the synthetic scope-exit "use" registered
* via `SsaImplInput::variableRead` — mirrors legacy ESSA's
* `EssaVariable.getAUse()` which inherited the synthetic exit-use
* from `SsaSourceVariable`.
*/
Cfg::ControlFlowNode getAUse() {
exists(CfgImpl::BasicBlock bb, int i |
Ssa::ssaDefReachesRead(this.getSourceVariable(), this, bb, i) and
bb.getNode(i) = result
)
}
/** Gets the (textual) name of the underlying variable. */
string getName() { result = this.getSourceVariable().getVariable().getId() }
/** Gets the scope in which this variable lives. */
Py::Scope getScope() { result = this.getSourceVariable().getVariable().getScope() }
/** Gets an ultimate non-phi ancestor of this definition. */
EssaVariable getAnUltimateDefinition() {
if this instanceof PhiNode
then
exists(Ssa::Definition input |
Ssa::phiHasInputFromBlock(this, input, _) and
result = input.(EssaVariable).getAnUltimateDefinition()
)
else result = this
}
}
/**
* Adjacent use-use and def-use relations exposed by the shared SSA
* library. Provides the same interface as legacy
* `semmle.python.essa.SsaCompute::AdjacentUses`.
*/
module AdjacentUses {
/** Holds if `nodeFrom` and `nodeTo` are adjacent uses of the same SSA variable. */
predicate adjacentUseUse(Cfg::NameNode nodeFrom, Cfg::NameNode nodeTo) {
exists(SsaSourceVariable v, CfgImpl::BasicBlock bb1, int i1, CfgImpl::BasicBlock bb2, int i2 |
Ssa::adjacentUseUse(bb1, i1, bb2, i2, v, _) and
nodeFrom = bb1.getNode(i1) and
nodeTo = bb2.getNode(i2)
)
}
/** Holds if `use` is a first use of definition `def`. */
predicate firstUse(Ssa::Definition def, Cfg::NameNode use) {
exists(CfgImpl::BasicBlock bb, int i |
Ssa::firstUse(def, bb, i, _) and
use = bb.getNode(i)
)
}
/**
* Holds if `use` is any reachable use of definition `def`. Combines
* `firstUse` with transitive use-use adjacency.
*/
predicate useOfDef(Ssa::Definition def, Cfg::NameNode use) {
firstUse(def, use)
or
exists(Cfg::NameNode mid | useOfDef(def, mid) and adjacentUseUse(mid, use))
}
}

View File

@@ -94,8 +94,10 @@ private module SummaryTypeTrackerInput implements SummaryTypeTracker::Input {
Node returnOf(Node callable, SummaryComponent return) {
return = FlowSummaryImpl::Private::SummaryComponent::return() and
// `result` should be the return value of a callable expression (lambda or function) referenced by `callable`
result.asCfgNode() =
callable.getALocalSource().asExpr().(CallableExpr).getInnerScope().getAReturnValueFlowNode()
exists(Return ret |
ret.getScope() = callable.getALocalSource().asExpr().(CallableExpr).getInnerScope() and
result.asCfgNode().getNode() = ret.getValue()
)
}
// Relating callables to nodes

View File

@@ -61,7 +61,7 @@ private module CaptureInput implements Shared::InputSig<Location, Cfg::BasicBloc
class VariableWrite extends ControlFlowNode {
CapturedVariable v;
VariableWrite() { this = v.getAStore().getAFlowNode().(DefinitionNode).getValue() }
VariableWrite() { exists(DefinitionNode d | d.getNode() = v.getAStore() | this = d.getValue()) }
CapturedVariable getVariable() { result = v }
@@ -71,7 +71,7 @@ private module CaptureInput implements Shared::InputSig<Location, Cfg::BasicBloc
class VariableRead extends Expr {
CapturedVariable v;
VariableRead() { this = v.getALoad().getAFlowNode() }
VariableRead() { this.getNode() = v.getALoad() }
CapturedVariable getVariable() { result = v }
}

View File

@@ -448,8 +448,7 @@ class TaintTrackingImplementation extends string instanceof TaintTracking::Confi
context = TNoParam() and
src = TTaintTrackingNode_(retval, TNoParam(), path, kind, this) and
node.asCfgNode() = call and
retval.asCfgNode() =
any(Return ret | ret.getScope() = pyfunc.getScope()).getValue().getAFlowNode()
retval.asCfgNode().getNode() = any(Return ret | ret.getScope() = pyfunc.getScope()).getValue()
) and
edgeLabel = "return"
}
@@ -471,8 +470,7 @@ class TaintTrackingImplementation extends string instanceof TaintTracking::Confi
this.callContexts(call, src, pyfunc, context, callee) and
retnode = TTaintTrackingNode_(retval, callee, path, kind, this) and
node.asCfgNode() = call and
retval.asCfgNode() =
any(Return ret | ret.getScope() = pyfunc.getScope()).getValue().getAFlowNode()
retval.asCfgNode().getNode() = any(Return ret | ret.getScope() = pyfunc.getScope()).getValue()
) and
edgeLabel = "call"
}
@@ -716,8 +714,10 @@ private class EssaTaintTracking extends string instanceof TaintTracking::Configu
src = TTaintTrackingNode_(srcnode, context, path, srckind, this) and
path.noAttribute()
|
assign.getValue().getAFlowNode() = srcnode.asCfgNode() and
depth = iterable_unpacking_descent(assign.getATarget().getAFlowNode(), defn.getDefiningNode()) and
srcnode.asCfgNode().getNode() = assign.getValue() and
exists(SequenceNode left_parent | left_parent.getNode() = assign.getATarget() |
depth = iterable_unpacking_descent(left_parent, defn.getDefiningNode())
) and
kind = taint_at_depth(srckind, depth)
)
}
@@ -964,7 +964,7 @@ private TaintKind taint_at_depth(SequenceKind parent_kind, int depth) {
* - with `left_defn` = `*y`, `left_parent` = `((x, *y), ...)`, result = 1
*/
int iterable_unpacking_descent(SequenceNode left_parent, ControlFlowNode left_defn) {
exists(Assign a | a.getATarget().getASubExpression*().getAFlowNode() = left_parent) and
exists(Assign a | left_parent.getNode() = a.getATarget().getASubExpression*()) and
left_parent.getAnElement() = left_defn and
// Handle `a, *b = some_iterable`
if left_defn instanceof StarredNode then result = 0 else result = 1

View File

@@ -56,7 +56,7 @@ module SsaSource {
predicate with_definition(Variable v, ControlFlowNode defn) {
exists(With with, Name var |
with.getOptionalVars() = var and
var.getAFlowNode() = defn
defn.getNode() = var
|
var = v.getAStore()
)
@@ -67,7 +67,7 @@ module SsaSource {
predicate pattern_capture_definition(Variable v, ControlFlowNode defn) {
exists(MatchCapturePattern capture, Name var |
capture.getVariable() = var and
var.getAFlowNode() = defn
defn.getNode() = var
|
var = v.getAStore()
)
@@ -78,7 +78,7 @@ module SsaSource {
predicate pattern_alias_definition(Variable v, ControlFlowNode defn) {
exists(MatchAsPattern pattern, Name var |
pattern.getAlias() = var and
var.getAFlowNode() = defn
defn.getNode() = var
|
var = v.getAStore()
)

View File

@@ -59,7 +59,7 @@ module Bottle {
override Parameter getARoutedParameter() { none() }
override Function getARequestHandler() { result.getADecorator().getAFlowNode() = node }
override Function getARequestHandler() { node.getNode() = result.getADecorator() }
}
}
@@ -73,7 +73,10 @@ module Bottle {
/** A response returned by a view callable. */
class BottleReturnResponse extends Http::Server::HttpResponse::Range {
BottleReturnResponse() {
this.asCfgNode() = any(View::ViewCallable vc).getAReturnValueFlowNode()
exists(Return ret |
ret.getScope() = any(View::ViewCallable vc) and
this.asCfgNode().getNode() = ret.getValue()
)
}
override DataFlow::Node getBody() { result = this }

View File

@@ -2872,7 +2872,10 @@ module PrivateDjango {
DataFlow::CfgNode
{
DjangoRedirectViewGetRedirectUrlReturn() {
node = any(GetRedirectUrlFunction f).getAReturnValueFlowNode()
exists(Return ret |
ret.getScope() = any(GetRedirectUrlFunction f) and
node.getNode() = ret.getValue()
)
}
override DataFlow::Node getRedirectLocation() { result = this }

Some files were not shown because too many files have changed in this diff Show More