hohn/codeql
1
0
Fork 0
mirror of https://github.com/github/codeql.git synced 2025-12-22 11:46:32 +01:00
Code Issues Packages Projects Releases Wiki Activity

Merge branch 'main' into python/rewrite-InsecureContextConfiguration

Browse Source
This commit is contained in:
yoff
2023-03-27 10:20:53 +02:00
committed by GitHub
parent 3c407eaa23 6b265104cf
commit 2121ed784f
2921 changed files with 113469 additions and 53264 deletions
Download Patch File Download Diff File Expand all files Collapse all files

30
python/ql/lib/CHANGELOG.md
View File

@@ -1,3 +1,33 @@
## 0.8.2
### New Features
* Added support for merging two `PathGraph`s via disjoint union to allow results from multiple data flow computations in a single `path-problem` query.
### Major Analysis Improvements
* The main data flow and taint tracking APIs have been changed. The old APIs
remain in place for now and translate to the new through a
backwards-compatible wrapper. If multiple configurations are in scope
simultaneously, then this may affect results slightly. The new API is quite
similar to the old, but makes use of a configuration module instead of a
configuration class.
### Minor Analysis Improvements
* Deleted the deprecated `getPath` and `getFolder` predicates from the `XmlFile` class.
## 0.8.1
### Major Analysis Improvements
* We use a new analysis for the call-graph (determining which function is called). This can lead to changed results. In most cases this is much more accurate than the old call-graph that was based on points-to, but we do lose a few valid edges in the call-graph, especially around methods that are not defined inside its class.
### Minor Analysis Improvements
* Fixed module resolution so we properly recognize definitions made within if-then-else statements.
* Added modeling of cryptographic operations in the `hmac` library.
## 0.8.0
### Breaking Changes

4
python/ql/lib/change-notes/2023-02-13-hmac-modeling.md
View File

@@ -1,4 +0,0 @@
---
category: minorAnalysis
---
* Added modeling of cryptographic operations in the `hmac` library.

4
python/ql/lib/change-notes/2023-02-16-import-if-then-else.md
View File

@@ -1,4 +0,0 @@
---
category: minorAnalysis
---
* Fixed module resolution so we properly recognize definitions made within if-then-else statements.

4
python/ql/lib/change-notes/2023-02-17-import-refined-variable.md Normal file
View File

@@ -0,0 +1,4 @@
---
category: minorAnalysis
---
* Fixed module resolution so we allow imports of definitions that have had an attribute assigned to it, such as `class Foo; Foo.bar = 42`.

6
python/ql/lib/change-notes/2023-03-23-dataflow-renaming.md Normal file
View File

@@ -0,0 +1,6 @@
---
category: deprecated
---
* The recently introduced new data flow and taint tracking APIs have had a
number of module and predicate renamings. The old APIs remain in place for
now.

14
python/ql/lib/change-notes/2023-01-16-new-call-graph.md → python/ql/lib/change-notes/released/0.8.1.md
View File

@@ -1,4 +1,10 @@
---
category: majorAnalysis
---
* We use a new analysis for the call-graph (determining which function is called). This can lead to changed results. In most cases this is much more accurate than the old call-graph that was based on points-to, but we do lose a few valid edges in the call-graph, especially around methods that are not defined inside its' class.
## 0.8.1
### Major Analysis Improvements
* We use a new analysis for the call-graph (determining which function is called). This can lead to changed results. In most cases this is much more accurate than the old call-graph that was based on points-to, but we do lose a few valid edges in the call-graph, especially around methods that are not defined inside its class.
### Minor Analysis Improvements
* Fixed module resolution so we properly recognize definitions made within if-then-else statements.
* Added modeling of cryptographic operations in the `hmac` library.

15
python/ql/lib/change-notes/2023-03-02-dataflow-conf-module.md → python/ql/lib/change-notes/released/0.8.2.md
View File

@@ -1,9 +1,18 @@
---
category: majorAnalysis
---
## 0.8.2
### New Features
* Added support for merging two `PathGraph`s via disjoint union to allow results from multiple data flow computations in a single `path-problem` query.
### Major Analysis Improvements
* The main data flow and taint tracking APIs have been changed. The old APIs
remain in place for now and translate to the new through a
backwards-compatible wrapper. If multiple configurations are in scope
simultaneously, then this may affect results slightly. The new API is quite
similar to the old, but makes use of a configuration module instead of a
configuration class.
### Minor Analysis Improvements
* Deleted the deprecated `getPath` and `getFolder` predicates from the `XmlFile` class.

2
python/ql/lib/codeql-pack.release.yml
View File

@@ -1,2 +1,2 @@
---
lastReleaseVersion: 0.8.0
lastReleaseVersion: 0.8.2

3
python/ql/lib/qlpack.yml
View File

@@ -1,5 +1,5 @@
name: codeql/python-all
version: 0.8.1-dev
version: 0.8.3-dev
groups: python
dbscheme: semmlecode.python.dbscheme
extractor: python
@@ -8,5 +8,6 @@ upgrades: upgrades
dependencies:
codeql/regex: ${workspace}
codeql/tutorial: ${workspace}
codeql/util: ${workspace}
dataExtensions:
- semmle/python/frameworks/**/model.yml

9
python/ql/lib/semmle/python/Concepts.qll
View File

@@ -19,6 +19,9 @@ private import semmle.python.security.internal.EncryptionKeySizes
* extend `SystemCommandExecution::Range` instead.
*/
class SystemCommandExecution extends DataFlow::Node instanceof SystemCommandExecution::Range {
/** Holds if a shell interprets `arg`. */
predicate isShellInterpreted(DataFlow::Node arg) { super.isShellInterpreted(arg) }
/** Gets the argument that specifies the command to be executed. */
DataFlow::Node getCommand() { result = super.getCommand() }
}
@@ -35,6 +38,9 @@ module SystemCommandExecution {
abstract class Range extends DataFlow::Node {
/** Gets the argument that specifies the command to be executed. */
abstract DataFlow::Node getCommand();
/** Holds if a shell interprets `arg`. */
predicate isShellInterpreted(DataFlow::Node arg) { none() }
}
}
@@ -1019,7 +1025,8 @@ module Http {
* Extend this class to refine existing API models. If you want to model new APIs,
* extend `CsrfLocalProtectionSetting::Range` instead.
*/
class CsrfLocalProtectionSetting extends DataFlow::Node instanceof CsrfLocalProtectionSetting::Range {
class CsrfLocalProtectionSetting extends DataFlow::Node instanceof CsrfLocalProtectionSetting::Range
{
/**
* Gets a request handler whose CSRF protection is changed.
*/

1
python/ql/lib/semmle/python/Frameworks.qll
View File

@@ -51,6 +51,7 @@ private import semmle.python.frameworks.Simplejson
private import semmle.python.frameworks.SqlAlchemy
private import semmle.python.frameworks.Starlette
private import semmle.python.frameworks.Stdlib
private import semmle.python.frameworks.Setuptools
private import semmle.python.frameworks.Toml
private import semmle.python.frameworks.Tornado
private import semmle.python.frameworks.Twisted

38
python/ql/lib/semmle/python/RegexTreeView.qll
View File

@@ -468,6 +468,8 @@ module Impl implements RegexTreeViewSig {
*/
class RegExpCharEscape = RegExpEscape;
private import codeql.util.Numbers as Numbers
/**
* An escaped regular expression term, that is, a regular expression
* term starting with a backslash, which is not a backreference.
@@ -528,42 +530,8 @@ module Impl implements RegexTreeViewSig {
* E.g. for `\u0061` this returns "a".
*/
private string getUnicode() {
exists(int codepoint | codepoint = sum(this.getHexValueFromUnicode(_)) |
result = codepoint.toUnicode()
)
result = Numbers::parseHexInt(this.getText().suffix(2)).toUnicode()
}
/**
* Gets int value for the `index`th char in the hex number of the unicode escape.
* E.g. for `\u0061` and `index = 2` this returns 96 (the number `6` interpreted as hex).
*/
private int getHexValueFromUnicode(int index) {
this.isUnicode() and
exists(string hex, string char | hex = this.getText().suffix(2) |
char = hex.charAt(index) and
result = 16.pow(hex.length() - index - 1) * toHex(char)
)
}
}
/**
* Gets the hex number for the `hex` char.
*/
private int toHex(string hex) {
hex = [0 .. 9].toString() and
result = hex.toInt()
or
result = 10 and hex = ["a", "A"]
or
result = 11 and hex = ["b", "B"]
or
result = 12 and hex = ["c", "C"]
or
result = 13 and hex = ["d", "D"]
or
result = 14 and hex = ["e", "E"]
or
result = 15 and hex = ["f", "F"]
}
/**

9
python/ql/lib/semmle/python/Unit.qll
View File

@@ -1,10 +1,3 @@
/** Provides the `Unit` class. */
/** The unit type. */
private newtype TUnit = TMkUnit()
/** The trivial type with a single element. */
class Unit extends TUnit {
/** Gets a textual representation of this element. */
string toString() { result = "unit" }
}
import codeql.util.Unit

150
python/ql/lib/semmle/python/dataflow/new/internal/DataFlow.qll
View File

@@ -2,7 +2,7 @@
* Provides an implementation of global (interprocedural) data flow. This file
* re-exports the local (intraprocedural) data flow analysis from
* `DataFlowImplSpecific::Public` and adds a global analysis, mainly exposed
* through the `Make` and `MakeWithState` modules.
* through the `Global` and `GlobalWithState` modules.
*/
private import DataFlowImplCommon
@@ -73,10 +73,10 @@ signature module ConfigSig {
*/
default FlowFeature getAFeature() { none() }
/** Holds if sources should be grouped in the result of `hasFlowPath`. */
/** Holds if sources should be grouped in the result of `flowPath`. */
default predicate sourceGrouping(Node source, string sourceGroup) { none() }
/** Holds if sinks should be grouped in the result of `hasFlowPath`. */
/** Holds if sinks should be grouped in the result of `flowPath`. */
default predicate sinkGrouping(Node sink, string sinkGroup) { none() }
/**
@@ -166,10 +166,10 @@ signature module StateConfigSig {
*/
default FlowFeature getAFeature() { none() }
/** Holds if sources should be grouped in the result of `hasFlowPath`. */
/** Holds if sources should be grouped in the result of `flowPath`. */
default predicate sourceGrouping(Node source, string sourceGroup) { none() }
/** Holds if sinks should be grouped in the result of `hasFlowPath`. */
/** Holds if sinks should be grouped in the result of `flowPath`. */
default predicate sinkGrouping(Node sink, string sinkGroup) { none() }
/**
@@ -182,15 +182,15 @@ signature module StateConfigSig {
}
/**
* Gets the exploration limit for `hasPartialFlow` and `hasPartialFlowRev`
* Gets the exploration limit for `partialFlow` and `partialFlowRev`
* measured in approximate number of interprocedural steps.
*/
signature int explorationLimitSig();
/**
* The output of a data flow computation.
* The output of a global data flow computation.
*/
signature module DataFlowSig {
signature module GlobalFlowSig {
/**
* A `Node` augmented with a call context (except for sinks) and an access path.
* Only those `PathNode`s that are reachable from a source, and which can reach a sink, are generated.
@@ -203,28 +203,28 @@ signature module DataFlowSig {
* The corresponding paths are generated from the end-points and the graph
* included in the module `PathGraph`.
*/
predicate hasFlowPath(PathNode source, PathNode sink);
predicate flowPath(PathNode source, PathNode sink);
/**
* Holds if data can flow from `source` to `sink`.
*/
predicate hasFlow(Node source, Node sink);
predicate flow(Node source, Node sink);
/**
* Holds if data can flow from some source to `sink`.
*/
predicate hasFlowTo(Node sink);
predicate flowTo(Node sink);
/**
* Holds if data can flow from some source to `sink`.
*/
predicate hasFlowToExpr(DataFlowExpr sink);
predicate flowToExpr(DataFlowExpr sink);
}
/**
* Constructs a standard data flow computation.
* Constructs a global data flow computation.
*/
module Make<ConfigSig Config> implements DataFlowSig {
module Global<ConfigSig Config> implements GlobalFlowSig {
private module C implements FullStateConfigSig {
import DefaultState<Config>
import Config
@@ -233,13 +233,131 @@ module Make<ConfigSig Config> implements DataFlowSig {
import Impl<C>
}
/** DEPRECATED: Use `Global` instead. */
deprecated module Make<ConfigSig Config> implements GlobalFlowSig {
import Global<Config>
}
/**
* Constructs a data flow computation using flow state.
* Constructs a global data flow computation using flow state.
*/
module MakeWithState<StateConfigSig Config> implements DataFlowSig {
module GlobalWithState<StateConfigSig Config> implements GlobalFlowSig {
private module C implements FullStateConfigSig {
import Config
}
import Impl<C>
}
/** DEPRECATED: Use `GlobalWithState` instead. */
deprecated module MakeWithState<StateConfigSig Config> implements GlobalFlowSig {
import GlobalWithState<Config>
}
signature class PathNodeSig {
/** Gets a textual representation of this element. */
string toString();
/**
* Holds if this element is at the specified location.
* The location spans column `startcolumn` of line `startline` to
* column `endcolumn` of line `endline` in file `filepath`.
* For more information, see
* [Locations](https://codeql.github.com/docs/writing-codeql-queries/providing-locations-in-codeql-queries/).
*/
predicate hasLocationInfo(
string filepath, int startline, int startcolumn, int endline, int endcolumn
);
/** Gets the underlying `Node`. */
Node getNode();
}
signature module PathGraphSig<PathNodeSig PathNode> {
/** Holds if `(a,b)` is an edge in the graph of data flow path explanations. */
predicate edges(PathNode a, PathNode b);
/** Holds if `n` is a node in the graph of data flow path explanations. */
predicate nodes(PathNode n, string key, string val);
/**
* Holds if `(arg, par, ret, out)` forms a subpath-tuple, that is, flow through
* a subpath between `par` and `ret` with the connecting edges `arg -> par` and
* `ret -> out` is summarized as the edge `arg -> out`.
*/
predicate subpaths(PathNode arg, PathNode par, PathNode ret, PathNode out);
}
/**
* Constructs a `PathGraph` from two `PathGraph`s by disjoint union.
*/
module MergePathGraph<
PathNodeSig PathNode1, PathNodeSig PathNode2, PathGraphSig<PathNode1> Graph1,
PathGraphSig<PathNode2> Graph2>
{
private newtype TPathNode =
TPathNode1(PathNode1 p) or
TPathNode2(PathNode2 p)
/** A node in a graph of path explanations that is formed by disjoint union of the two given graphs. */
class PathNode extends TPathNode {
/** Gets this as a projection on the first given `PathGraph`. */
PathNode1 asPathNode1() { this = TPathNode1(result) }
/** Gets this as a projection on the second given `PathGraph`. */
PathNode2 asPathNode2() { this = TPathNode2(result) }
/** Gets a textual representation of this element. */
string toString() {
result = this.asPathNode1().toString() or
result = this.asPathNode2().toString()
}
/**
* Holds if this element is at the specified location.
* The location spans column `startcolumn` of line `startline` to
* column `endcolumn` of line `endline` in file `filepath`.
* For more information, see
* [Locations](https://codeql.github.com/docs/writing-codeql-queries/providing-locations-in-codeql-queries/).
*/
predicate hasLocationInfo(
string filepath, int startline, int startcolumn, int endline, int endcolumn
) {
this.asPathNode1().hasLocationInfo(filepath, startline, startcolumn, endline, endcolumn) or
this.asPathNode2().hasLocationInfo(filepath, startline, startcolumn, endline, endcolumn)
}
/** Gets the underlying `Node`. */
Node getNode() {
result = this.asPathNode1().getNode() or
result = this.asPathNode2().getNode()
}
}
/**
* Provides the query predicates needed to include a graph in a path-problem query.
*/
module PathGraph implements PathGraphSig<PathNode> {
/** Holds if `(a,b)` is an edge in the graph of data flow path explanations. */
query predicate edges(PathNode a, PathNode b) {
Graph1::edges(a.asPathNode1(), b.asPathNode1()) or
Graph2::edges(a.asPathNode2(), b.asPathNode2())
}
/** Holds if `n` is a node in the graph of data flow path explanations. */
query predicate nodes(PathNode n, string key, string val) {
Graph1::nodes(n.asPathNode1(), key, val) or
Graph2::nodes(n.asPathNode2(), key, val)
}
/**
* Holds if `(arg, par, ret, out)` forms a subpath-tuple, that is, flow through
* a subpath between `par` and `ret` with the connecting edges `arg -> par` and
* `ret -> out` is summarized as the edge `arg -> out`.
*/
query predicate subpaths(PathNode arg, PathNode par, PathNode ret, PathNode out) {
Graph1::subpaths(arg.asPathNode1(), par.asPathNode1(), ret.asPathNode1(), out.asPathNode1()) or
Graph2::subpaths(arg.asPathNode2(), par.asPathNode2(), ret.asPathNode2(), out.asPathNode2())
}
}
}

43
python/ql/lib/semmle/python/dataflow/new/internal/DataFlowDispatch.qll
View File

@@ -64,7 +64,14 @@ newtype TParameterPosition =
index = any(Parameter p).getPosition() + 1
} or
TSynthStarArgsElementParameterPosition(int index) { exists(TStarArgsParameterPosition(index)) } or
TDictSplatParameterPosition()
TDictSplatParameterPosition() or
// To get flow from a **kwargs argument to a keyword parameter, we add a read-step
// from a synthetic **kwargs parameter. We need this separate synthetic ParameterNode,
// since we clear content of the normal **kwargs parameter for the names that
// correspond to normal keyword parameters. Since we cannot re-use the same parameter
// position for multiple parameter nodes in the same callable, we introduce this
// synthetic parameter position.
TSynthDictSplatParameterPosition()
/** A parameter position. */
class ParameterPosition extends TParameterPosition {
@@ -92,6 +99,12 @@ class ParameterPosition extends TParameterPosition {
/** Holds if this position represents a `**kwargs` parameter. */
predicate isDictSplat() { this = TDictSplatParameterPosition() }
/**
* Holds if this position represents a **synthetic** `**kwargs` parameter
* (see comment for `TSynthDictSplatParameterPosition`).
*/
predicate isSynthDictSplat() { this = TSynthDictSplatParameterPosition() }
/** Gets a textual representation of this element. */
string toString() {
this.isSelf() and result = "self"
@@ -108,6 +121,8 @@ class ParameterPosition extends TParameterPosition {
)
or
this.isDictSplat() and result = "**"
or
this.isSynthDictSplat() and result = "synthetic **"
}
}
@@ -179,6 +194,8 @@ predicate parameterMatch(ParameterPosition ppos, ArgumentPosition apos) {
)
or
ppos.isDictSplat() and apos.isDictSplat()
or
ppos.isSynthDictSplat() and apos.isDictSplat()
}
// =============================================================================
@@ -324,16 +341,9 @@ abstract class DataFlowFunction extends DataFlowCallable, TFunction {
)
or
// `**kwargs`
// since the dataflow library has the restriction that we can only have ONE result per
// parameter position, if there is both a synthetic **kwargs and a real **kwargs
// parameter, we only give the result for the synthetic, and add local flow from the
// synthetic to the real. It might seem more natural to do it in the other
// direction, but since we have a clearStep on the real **kwargs parameter, we would have that
// content-clearing would also affect the synthetic parameter, which we don't want.
ppos.isDictSplat() and
if exists(func.getArgByName(_))
then result = TSynthDictSplatParameterNode(this)
else result.getParameter() = func.getKwarg()
ppos.isDictSplat() and result.getParameter() = func.getKwarg()
or
ppos.isSynthDictSplat() and result = TSynthDictSplatParameterNode(this)
}
}
@@ -1460,16 +1470,7 @@ class SummaryParameterNode extends ParameterNodeImpl, TSummaryParameterNode {
override Parameter getParameter() { none() }
override predicate isParameterOf(DataFlowCallable c, ParameterPosition ppos) {
sc = c.asLibraryCallable() and
ppos = pos and
// avoid overlap with `SynthDictSplatParameterNode`
not (
pos.isDictSplat() and
exists(ParameterPosition keywordPos |
FlowSummaryImpl::Private::summaryParameterNodeRange(sc, keywordPos) and
keywordPos.isKeyword(_)
)
)
sc = c.asLibraryCallable() and ppos = pos
}
override DataFlowCallable getEnclosingCallable() { result.asLibraryCallable() = sc }

93
python/ql/lib/semmle/python/dataflow/new/internal/DataFlowImpl.qll
View File

@@ -91,10 +91,10 @@ signature module FullStateConfigSig {
*/
FlowFeature getAFeature();
/** Holds if sources should be grouped in the result of `hasFlowPath`. */
/** Holds if sources should be grouped in the result of `flowPath`. */
predicate sourceGrouping(Node source, string sourceGroup);
/** Holds if sinks should be grouped in the result of `hasFlowPath`. */
/** Holds if sinks should be grouped in the result of `flowPath`. */
predicate sinkGrouping(Node sink, string sinkGroup);
/**
@@ -418,6 +418,10 @@ module Impl<FullStateConfigSig Config> {
)
}
private predicate sourceCallCtx(CallContext cc) {
if hasSourceCallCtx() then cc instanceof CallContextSomeCall else cc instanceof CallContextAny
}
private predicate hasSinkCallCtx() {
exists(FlowFeature feature | feature = Config::getAFeature() |
feature instanceof FeatureHasSinkCallContext or
@@ -441,11 +445,7 @@ module Impl<FullStateConfigSig Config> {
}
private module Stage1 implements StageSig {
class Ap extends int {
// workaround for bad functionality-induced joins (happens when using `Unit`)
pragma[nomagic]
Ap() { this in [0 .. 1] and this < 1 }
}
class Ap = Unit;
private class Cc = boolean;
@@ -456,6 +456,7 @@ module Impl<FullStateConfigSig Config> {
* The Boolean `cc` records whether the node is reached through an
* argument in a call.
*/
pragma[assume_small_delta]
private predicate fwdFlow(NodeEx node, Cc cc) {
sourceNode(node, _) and
if hasSourceCallCtx() then cc = true else cc = false
@@ -1140,19 +1141,13 @@ module Impl<FullStateConfigSig Config> {
import Param
/* Begin: Stage logic. */
// use an alias as a workaround for bad functionality-induced joins
pragma[nomagic]
private predicate revFlowApAlias(NodeEx node, ApApprox apa) {
PrevStage::revFlowAp(node, apa)
}
pragma[nomagic]
private predicate flowIntoCallApa(
DataFlowCall call, ArgNodeEx arg, ParamNodeEx p, boolean allowsFieldFlow, ApApprox apa
) {
flowIntoCall(call, arg, p, allowsFieldFlow) and
PrevStage::revFlowAp(p, pragma[only_bind_into](apa)) and
revFlowApAlias(arg, pragma[only_bind_into](apa))
PrevStage::revFlowAp(arg, pragma[only_bind_into](apa))
}
pragma[nomagic]
@@ -1162,7 +1157,7 @@ module Impl<FullStateConfigSig Config> {
) {
flowOutOfCall(call, ret, kind, out, allowsFieldFlow) and
PrevStage::revFlowAp(out, pragma[only_bind_into](apa)) and
revFlowApAlias(ret, pragma[only_bind_into](apa))
PrevStage::revFlowAp(ret, pragma[only_bind_into](apa))
}
pragma[nomagic]
@@ -1690,16 +1685,6 @@ module Impl<FullStateConfigSig Config> {
pragma[nomagic]
predicate revFlowAp(NodeEx node, Ap ap) { revFlow(node, _, _, _, ap) }
// use an alias as a workaround for bad functionality-induced joins
pragma[nomagic]
additional predicate revFlowAlias(NodeEx node) { revFlow(node, _, _, _, _) }
// use an alias as a workaround for bad functionality-induced joins
pragma[nomagic]
additional predicate revFlowAlias(NodeEx node, FlowState state, Ap ap) {
revFlow(node, state, ap)
}
private predicate fwdConsCand(TypedContent tc, Ap ap) { storeStepFwd(_, ap, tc, _, _) }
private predicate revConsCand(TypedContent tc, Ap ap) { storeStepCand(_, ap, tc, _, _) }
@@ -1973,7 +1958,7 @@ module Impl<FullStateConfigSig Config> {
) {
flowOutOfCallNodeCand1(call, node1, kind, node2, allowsFieldFlow) and
Stage2::revFlow(node2) and
Stage2::revFlowAlias(node1)
Stage2::revFlow(node1)
}
pragma[nomagic]
@@ -1982,7 +1967,7 @@ module Impl<FullStateConfigSig Config> {
) {
flowIntoCallNodeCand1(call, node1, node2, allowsFieldFlow) and
Stage2::revFlow(node2) and
Stage2::revFlowAlias(node1)
Stage2::revFlow(node1)
}
private module LocalFlowBigStep {
@@ -2064,11 +2049,11 @@ module Impl<FullStateConfigSig Config> {
additionalLocalFlowStepNodeCand1(node1, node2) and
state1 = state2 and
Stage2::revFlow(node1, pragma[only_bind_into](state1), false) and
Stage2::revFlowAlias(node2, pragma[only_bind_into](state2), false)
Stage2::revFlow(node2, pragma[only_bind_into](state2), false)
or
additionalLocalStateStep(node1, state1, node2, state2) and
Stage2::revFlow(node1, state1, false) and
Stage2::revFlowAlias(node2, state2, false)
Stage2::revFlow(node2, state2, false)
}
/**
@@ -2261,7 +2246,7 @@ module Impl<FullStateConfigSig Config> {
) {
localFlowBigStep(node1, state1, node2, state2, preservesValue, ap.getType(), _) and
PrevStage::revFlow(node1, pragma[only_bind_into](state1), _) and
PrevStage::revFlowAlias(node2, pragma[only_bind_into](state2), _) and
PrevStage::revFlow(node2, pragma[only_bind_into](state2), _) and
exists(lcc)
}
@@ -2272,7 +2257,7 @@ module Impl<FullStateConfigSig Config> {
exists(FlowState state |
flowOutOfCallNodeCand2(call, node1, kind, node2, allowsFieldFlow) and
PrevStage::revFlow(node2, pragma[only_bind_into](state), _) and
PrevStage::revFlowAlias(node1, pragma[only_bind_into](state), _)
PrevStage::revFlow(node1, pragma[only_bind_into](state), _)
)
}
@@ -2283,7 +2268,7 @@ module Impl<FullStateConfigSig Config> {
exists(FlowState state |
flowIntoCallNodeCand2(call, node1, node2, allowsFieldFlow) and
PrevStage::revFlow(node2, pragma[only_bind_into](state), _) and
PrevStage::revFlowAlias(node1, pragma[only_bind_into](state), _)
PrevStage::revFlow(node1, pragma[only_bind_into](state), _)
)
}
@@ -2585,7 +2570,7 @@ module Impl<FullStateConfigSig Config> {
) {
localFlowBigStep(node1, state1, node2, state2, preservesValue, ap.getType(), lcc) and
PrevStage::revFlow(node1, pragma[only_bind_into](state1), _) and
PrevStage::revFlowAlias(node2, pragma[only_bind_into](state2), _)
PrevStage::revFlow(node2, pragma[only_bind_into](state2), _)
}
pragma[nomagic]
@@ -2595,7 +2580,7 @@ module Impl<FullStateConfigSig Config> {
exists(FlowState state |
flowOutOfCallNodeCand2(call, node1, kind, node2, allowsFieldFlow) and
PrevStage::revFlow(node2, pragma[only_bind_into](state), _) and
PrevStage::revFlowAlias(node1, pragma[only_bind_into](state), _)
PrevStage::revFlow(node1, pragma[only_bind_into](state), _)
)
}
@@ -2606,7 +2591,7 @@ module Impl<FullStateConfigSig Config> {
exists(FlowState state |
flowIntoCallNodeCand2(call, node1, node2, allowsFieldFlow) and
PrevStage::revFlow(node2, pragma[only_bind_into](state), _) and
PrevStage::revFlowAlias(node1, pragma[only_bind_into](state), _)
PrevStage::revFlow(node1, pragma[only_bind_into](state), _)
)
}
@@ -2803,11 +2788,7 @@ module Impl<FullStateConfigSig Config> {
// A PathNode is introduced by a source ...
Stage5::revFlow(node, state) and
sourceNode(node, state) and
(
if hasSourceCallCtx()
then cc instanceof CallContextSomeCall
else cc instanceof CallContextAny
) and
sourceCallCtx(cc) and
sc instanceof SummaryCtxNone and
ap = TAccessPathNil(node.getDataFlowType())
or
@@ -3156,7 +3137,7 @@ module Impl<FullStateConfigSig Config> {
/**
* Provides the query predicates needed to include a graph in a path-problem query.
*/
module PathGraph {
module PathGraph implements PathGraphSig<PathNode> {
/** Holds if `(a,b)` is an edge in the graph of data flow path explanations. */
query predicate edges(PathNode a, PathNode b) { a.getASuccessor() = b }
@@ -3213,11 +3194,7 @@ module Impl<FullStateConfigSig Config> {
override predicate isSource() {
sourceNode(node, state) and
(
if hasSourceCallCtx()
then cc instanceof CallContextSomeCall
else cc instanceof CallContextAny
) and
sourceCallCtx(cc) and
sc instanceof SummaryCtxNone and
ap = TAccessPathNil(node.getDataFlowType())
}
@@ -3652,7 +3629,7 @@ module Impl<FullStateConfigSig Config> {
* The corresponding paths are generated from the end-points and the graph
* included in the module `PathGraph`.
*/
predicate hasFlowPath(PathNode source, PathNode sink) {
predicate flowPath(PathNode source, PathNode sink) {
exists(PathNodeImpl flowsource, PathNodeImpl flowsink |
source = flowsource and sink = flowsink
|
@@ -3662,6 +3639,9 @@ module Impl<FullStateConfigSig Config> {
)
}
/** DEPRECATED: Use `flowPath` instead. */
deprecated predicate hasFlowPath = flowPath/2;
private predicate flowsTo(PathNodeImpl flowsource, PathNodeSink flowsink, Node source, Node sink) {
flowsource.isSource() and
flowsource.getNodeEx().asNode() = source and
@@ -3672,17 +3652,26 @@ module Impl<FullStateConfigSig Config> {
/**
* Holds if data can flow from `source` to `sink`.
*/
predicate hasFlow(Node source, Node sink) { flowsTo(_, _, source, sink) }
predicate flow(Node source, Node sink) { flowsTo(_, _, source, sink) }
/** DEPRECATED: Use `flow` instead. */
deprecated predicate hasFlow = flow/2;
/**
* Holds if data can flow from some source to `sink`.
*/
predicate hasFlowTo(Node sink) { sink = any(PathNodeSink n).getNodeEx().asNode() }
predicate flowTo(Node sink) { sink = any(PathNodeSink n).getNodeEx().asNode() }
/** DEPRECATED: Use `flowTo` instead. */
deprecated predicate hasFlowTo = flowTo/1;
/**
* Holds if data can flow from some source to `sink`.
*/
predicate hasFlowToExpr(DataFlowExpr sink) { hasFlowTo(exprNode(sink)) }
predicate flowToExpr(DataFlowExpr sink) { flowTo(exprNode(sink)) }
/** DEPRECATED: Use `flowToExpr` instead. */
deprecated predicate hasFlowToExpr = flowToExpr/1;
private predicate finalStats(
boolean fwd, int nodes, int fields, int conscand, int states, int tuples
@@ -4593,7 +4582,7 @@ module Impl<FullStateConfigSig Config> {
*
* To use this in a `path-problem` query, import the module `PartialPathGraph`.
*/
predicate hasPartialFlow(PartialPathNode source, PartialPathNode node, int dist) {
predicate partialFlow(PartialPathNode source, PartialPathNode node, int dist) {
partialFlow(source, node) and
dist = node.getSourceDistance()
}
@@ -4613,7 +4602,7 @@ module Impl<FullStateConfigSig Config> {
* Note that reverse flow has slightly lower precision than the corresponding
* forward flow, as reverse flow disregards type pruning among other features.
*/
predicate hasPartialFlowRev(PartialPathNode node, PartialPathNode sink, int dist) {
predicate partialFlowRev(PartialPathNode node, PartialPathNode sink, int dist) {
revPartialFlow(node, sink) and
dist = node.getSinkDistance()
}

4
python/ql/lib/semmle/python/dataflow/new/internal/DataFlowImpl1.qll
View File

@@ -1,5 +1,5 @@
/**
* DEPRECATED: Use `Make` and `MakeWithState` instead.
* DEPRECATED: Use `Global` and `GlobalWithState` instead.
*
* Provides a `Configuration` class backwards-compatible interface to the data
* flow library.
@@ -388,7 +388,7 @@ private predicate hasFlow(Node source, Node sink, Configuration config) {
}
private predicate hasFlowPath(PathNode source, PathNode sink, Configuration config) {
hasFlowPath(source, sink) and source.getConfiguration() = config
flowPath(source, sink) and source.getConfiguration() = config
}
private predicate hasFlowTo(Node sink, Configuration config) { hasFlow(_, sink, config) }

4
python/ql/lib/semmle/python/dataflow/new/internal/DataFlowImpl2.qll
View File

@@ -1,5 +1,5 @@
/**
* DEPRECATED: Use `Make` and `MakeWithState` instead.
* DEPRECATED: Use `Global` and `GlobalWithState` instead.
*
* Provides a `Configuration` class backwards-compatible interface to the data
* flow library.
@@ -388,7 +388,7 @@ private predicate hasFlow(Node source, Node sink, Configuration config) {
}
private predicate hasFlowPath(PathNode source, PathNode sink, Configuration config) {
hasFlowPath(source, sink) and source.getConfiguration() = config
flowPath(source, sink) and source.getConfiguration() = config
}
private predicate hasFlowTo(Node sink, Configuration config) { hasFlow(_, sink, config) }

4
python/ql/lib/semmle/python/dataflow/new/internal/DataFlowImpl3.qll
View File

@@ -1,5 +1,5 @@
/**
* DEPRECATED: Use `Make` and `MakeWithState` instead.
* DEPRECATED: Use `Global` and `GlobalWithState` instead.
*
* Provides a `Configuration` class backwards-compatible interface to the data
* flow library.
@@ -388,7 +388,7 @@ private predicate hasFlow(Node source, Node sink, Configuration config) {
}
private predicate hasFlowPath(PathNode source, PathNode sink, Configuration config) {
hasFlowPath(source, sink) and source.getConfiguration() = config
flowPath(source, sink) and source.getConfiguration() = config
}
private predicate hasFlowTo(Node sink, Configuration config) { hasFlow(_, sink, config) }

4
python/ql/lib/semmle/python/dataflow/new/internal/DataFlowImpl4.qll
View File

@@ -1,5 +1,5 @@
/**
* DEPRECATED: Use `Make` and `MakeWithState` instead.
* DEPRECATED: Use `Global` and `GlobalWithState` instead.
*
* Provides a `Configuration` class backwards-compatible interface to the data
* flow library.
@@ -388,7 +388,7 @@ private predicate hasFlow(Node source, Node sink, Configuration config) {
}
private predicate hasFlowPath(PathNode source, PathNode sink, Configuration config) {
hasFlowPath(source, sink) and source.getConfiguration() = config
flowPath(source, sink) and source.getConfiguration() = config
}
private predicate hasFlowTo(Node sink, Configuration config) { hasFlow(_, sink, config) }

16
python/ql/lib/semmle/python/dataflow/new/internal/DataFlowImplCommon.qll
View File

@@ -140,10 +140,8 @@ private module LambdaFlow {
}
pragma[nomagic]
private TReturnPositionSimple viableReturnPosLambda(
DataFlowCall call, DataFlowCallOption lastCall, ReturnKind kind
) {
result = TReturnPositionSimple0(viableCallableLambda(call, lastCall), kind)
private TReturnPositionSimple viableReturnPosLambda(DataFlowCall call, ReturnKind kind) {
result = TReturnPositionSimple0(viableCallableLambda(call, _), kind)
}
private predicate viableReturnPosOutNonLambda(
@@ -155,11 +153,12 @@ private module LambdaFlow {
)
}
pragma[nomagic]
private predicate viableReturnPosOutLambda(
DataFlowCall call, DataFlowCallOption lastCall, TReturnPositionSimple pos, OutNode out
DataFlowCall call, TReturnPositionSimple pos, OutNode out
) {
exists(ReturnKind kind |
pos = viableReturnPosLambda(call, lastCall, kind) and
pos = viableReturnPosLambda(call, kind) and
out = getAnOutNode(call, kind)
)
}
@@ -182,11 +181,13 @@ private module LambdaFlow {
boolean toJump, DataFlowCallOption lastCall
) {
revLambdaFlow0(lambdaCall, kind, node, t, toReturn, toJump, lastCall) and
not expectsContent(node, _) and
if castNode(node) or node instanceof ArgNode or node instanceof ReturnNode
then compatibleTypes(t, getNodeDataFlowType(node))
else any()
}
pragma[assume_small_delta]
pragma[nomagic]
predicate revLambdaFlow0(
DataFlowCall lambdaCall, LambdaCallKind kind, Node node, DataFlowType t, boolean toReturn,
@@ -273,6 +274,7 @@ private module LambdaFlow {
)
}
pragma[assume_small_delta]
pragma[nomagic]
predicate revLambdaFlowOut(
DataFlowCall lambdaCall, LambdaCallKind kind, TReturnPositionSimple pos, DataFlowType t,
@@ -284,7 +286,7 @@ private module LambdaFlow {
or
// non-linear recursion
revLambdaFlowOutLambdaCall(lambdaCall, kind, out, t, toJump, call, lastCall) and
viableReturnPosOutLambda(call, _, pos, out)
viableReturnPosOutLambda(call, pos, out)
)
}

12
python/ql/lib/semmle/python/dataflow/new/internal/DataFlowImplConsistency.qll
View File

@@ -18,6 +18,9 @@ module Consistency {
/** Holds if `n` should be excluded from the consistency test `uniqueEnclosingCallable`. */
predicate uniqueEnclosingCallableExclude(Node n) { none() }
/** Holds if `call` should be excluded from the consistency test `uniqueCallEnclosingCallable`. */
predicate uniqueCallEnclosingCallableExclude(DataFlowCall call) { none() }
/** Holds if `n` should be excluded from the consistency test `uniqueNodeLocation`. */
predicate uniqueNodeLocationExclude(Node n) { none() }
@@ -86,6 +89,15 @@ module Consistency {
)
}
query predicate uniqueCallEnclosingCallable(DataFlowCall call, string msg) {
exists(int c |
c = count(call.getEnclosingCallable()) and
c != 1 and
not any(ConsistencyConfiguration conf).uniqueCallEnclosingCallableExclude(call) and
msg = "Call should have one enclosing callable but has " + c + "."
)
}
query predicate uniqueType(Node n, string msg) {
exists(int c |
n instanceof RelevantNode and

4
python/ql/lib/semmle/python/dataflow/new/internal/DataFlowImplForRegExp.qll
View File

@@ -1,5 +1,5 @@
/**
* DEPRECATED: Use `Make` and `MakeWithState` instead.
* DEPRECATED: Use `Global` and `GlobalWithState` instead.
*
* Provides a `Configuration` class backwards-compatible interface to the data
* flow library.
@@ -388,7 +388,7 @@ private predicate hasFlow(Node source, Node sink, Configuration config) {
}
private predicate hasFlowPath(PathNode source, PathNode sink, Configuration config) {
hasFlowPath(source, sink) and source.getConfiguration() = config
flowPath(source, sink) and source.getConfiguration() = config
}
private predicate hasFlowTo(Node sink, Configuration config) { hasFlow(_, sink, config) }

33
python/ql/lib/semmle/python/dataflow/new/internal/DataFlowPrivate.qll
View File

@@ -110,7 +110,8 @@ class SyntheticPreUpdateNode extends Node, TSyntheticPreUpdateNode {
* func(1, 2, 3)
*/
class SynthStarArgsElementParameterNode extends ParameterNodeImpl,
TSynthStarArgsElementParameterNode {
TSynthStarArgsElementParameterNode
{
DataFlowCallable callable;
SynthStarArgsElementParameterNode() { this = TSynthStarArgsElementParameterNode(callable) }
@@ -181,11 +182,7 @@ private predicate synthDictSplatArgumentNodeStoreStep(
private predicate dictSplatParameterNodeClearStep(ParameterNode n, DictionaryElementContent c) {
exists(DataFlowCallable callable, ParameterPosition dictSplatPos, ParameterPosition keywordPos |
dictSplatPos.isDictSplat() and
(
n.getParameter() = callable.(DataFlowFunction).getScope().getKwarg()
or
n = TSummaryParameterNode(callable.asLibraryCallable(), dictSplatPos)
) and
n = callable.getParameter(dictSplatPos) and
exists(callable.getParameter(keywordPos)) and
keywordPos.isKeyword(c.getKey())
)
@@ -236,28 +233,6 @@ class SynthDictSplatParameterNode extends ParameterNodeImpl, TSynthDictSplatPara
override Parameter getParameter() { none() }
}
/**
* Flow step from the synthetic `**kwargs` parameter to the real `**kwargs` parameter.
* Due to restriction in dataflow library, we can only give one of them as result for
* `DataFlowCallable.getParameter`, so this is a workaround to ensure there is flow to
* _both_ of them.
*/
private predicate dictSplatParameterNodeFlowStep(
ParameterNodeImpl nodeFrom, ParameterNodeImpl nodeTo
) {
exists(DataFlowCallable callable |
nodeFrom = TSynthDictSplatParameterNode(callable) and
(
nodeTo.getParameter() = callable.(DataFlowFunction).getScope().getKwarg()
or
exists(ParameterPosition pos |
nodeTo = TSummaryParameterNode(callable.asLibraryCallable(), pos) and
pos.isDictSplat()
)
)
)
}
/**
* Reads from the synthetic **kwargs parameter to each keyword parameter.
*/
@@ -403,8 +378,6 @@ predicate simpleLocalFlowStep(Node nodeFrom, Node nodeTo) {
simpleLocalFlowStepForTypetracking(nodeFrom, nodeTo)
or
summaryFlowSteps(nodeFrom, nodeTo)
or
dictSplatParameterNodeFlowStep(nodeFrom, nodeTo)
}
/**

24
python/ql/lib/semmle/python/dataflow/new/internal/FlowSummaryImpl.qll
View File

@@ -109,6 +109,7 @@ module Public {
}
/** Gets the stack obtained by dropping the first `i` elements, if any. */
pragma[assume_small_delta]
SummaryComponentStack drop(int i) {
i = 0 and result = this
or
@@ -301,8 +302,8 @@ module Private {
TWithoutContentSummaryComponent(ContentSet c) or
TWithContentSummaryComponent(ContentSet c)
private TParameterSummaryComponent thisParam() {
result = TParameterSummaryComponent(instanceParameterPosition())
private TParameterSummaryComponent callbackSelfParam() {
result = TParameterSummaryComponent(callbackSelfParameterPosition())
}
newtype TSummaryComponentStack =
@@ -311,7 +312,7 @@ module Private {
any(RequiredSummaryComponentStack x).required(head, tail)
or
any(RequiredSummaryComponentStack x).required(TParameterSummaryComponent(_), tail) and
head = thisParam()
head = callbackSelfParam()
or
derivedFluentFlowPush(_, _, _, head, tail, _)
}
@@ -336,7 +337,7 @@ module Private {
callbackRef = s.drop(_) and
(isCallbackParameter(callbackRef) or callbackRef.head() = TReturnSummaryComponent(_)) and
input = callbackRef.tail() and
output = TConsSummaryComponentStack(thisParam(), input) and
output = TConsSummaryComponentStack(callbackSelfParam(), input) and
preservesValue = true
)
or
@@ -439,6 +440,9 @@ module Private {
out.head() = TParameterSummaryComponent(_) and
s = out.tail()
)
or
// Add the post-update node corresponding to the requested argument node
outputState(c, s) and isCallbackParameter(s)
}
private newtype TSummaryNodeState =
@@ -1012,7 +1016,7 @@ module Private {
private predicate relevantSummaryElementGenerated(
AccessPath inSpec, AccessPath outSpec, string kind
) {
summaryElement(this, inSpec, outSpec, kind, "generated") and
summaryElement(this, inSpec, outSpec, kind, ["generated", "ai-generated"]) and
not summaryElement(this, _, _, _, "manual")
}
@@ -1047,6 +1051,16 @@ module Private {
not exists(interpretComponent(c))
}
/**
* Holds if token `part` of specification `spec` has an invalid index.
* E.g., `Argument[-1]`.
*/
predicate invalidIndexComponent(AccessPath spec, AccessPathToken part) {
part = spec.getToken(_) and
part.getName() = ["Parameter", "Argument"] and
AccessPath::parseInt(part.getArgumentList()) < 0
}
private predicate inputNeedsReference(AccessPathToken c) {
c.getName() = "Argument" or
inputNeedsReferenceSpecific(c)

2
python/ql/lib/semmle/python/dataflow/new/internal/FlowSummaryImplSpecific.qll
View File

@@ -45,7 +45,7 @@ class SummarizedCallableBase = string;
DataFlowCallable inject(SummarizedCallable c) { result.asLibraryCallable() = c }
/** Gets the parameter position of the instance parameter. */
ArgumentPosition instanceParameterPosition() { none() } // disables implicit summary flow to `this` for callbacks
ArgumentPosition callbackSelfParameterPosition() { none() } // disables implicit summary flow to `this` for callbacks
/** Gets the synthesized summary data-flow node for the given values. */
Node summaryNode(SummarizedCallable c, SummaryNodeState state) { result = TSummaryNode(c, state) }

94
python/ql/lib/semmle/python/dataflow/new/internal/ImportResolution.qll
View File

@@ -65,31 +65,75 @@ private import semmle.python.dataflow.new.internal.DataFlowPrivate
*/
module ImportResolution {
/**
* Holds if the module `m` defines a name `name` by assigning `defn` to it. This is an
* overapproximation, as `name` may not in fact be exported (e.g. by defining an `__all__` that does
* not include `name`).
* Holds if there is an ESSA step from `defFrom` to `defTo`, which should be allowed
* for import resolution.
*/
private predicate allowedEssaImportStep(EssaDefinition defFrom, EssaDefinition defTo) {
// to handle definitions guarded by if-then-else
defFrom = defTo.(PhiFunction).getAnInput()
or
// refined variable
// example: https://github.com/nvbn/thefuck/blob/ceeaeab94b5df5a4fe9d94d61e4f6b0bbea96378/thefuck/utils.py#L25-L45
defFrom = defTo.(EssaNodeRefinement).getInput().getDefinition()
}
/**
* Holds if the module `m` defines a name `name` with the value `val`. The value
* represents the value `name` will have at the end of the module (the last place we
* have def-use flow to).
*
* Note: The handling of re-exporting imports is a bit simplistic. We assume that if
* an import is made, it will be re-exported (which will not be the case if a new
* value is assigned to the name, or it is deleted).
*/
pragma[nomagic]
predicate module_export(Module m, string name, DataFlow::CfgNode defn) {
exists(EssaVariable v, EssaDefinition essaDef |
v.getName() = name and
v.getAUse() = ImportStar::getStarImported*(m).getANormalExit() and
(
essaDef = v.getDefinition()
or
// to handle definitions guarded by if-then-else
essaDef = v.getDefinition().(PhiFunction).getAnInput()
)
predicate module_export(Module m, string name, DataFlow::Node val) {
// Definitions made inside `m` itself
//
// for code such as `foo = ...; foo.bar = ...` there will be TWO
// EssaDefinition/EssaVariable. One for `foo = ...` (AssignmentDefinition) and one
// for `foo.bar = ...`. The one for `foo.bar = ...` (EssaNodeRefinement). The
// EssaNodeRefinement is the one that will reach the end of the module (normal
// exit).
//
// However, we cannot just use the EssaNodeRefinement as the `val`, because the
// normal data-flow depends on use-use flow, and use-use flow targets CFG nodes not
// EssaNodes. So we need to go back from the EssaDefinition/EssaVariable that
// reaches the end of the module, to the first definition of the variable, and then
// track forwards using use-use flow to find a suitable CFG node that has flow into
// it from use-use flow.
exists(EssaVariable lastUseVar, EssaVariable firstDef |
lastUseVar.getName() = name and
// we ignore special variable $ introduced by our analysis (not used for anything)
// we ignore special variable * introduced by `from <pkg> import *` -- TODO: understand why we even have this?
not name in ["$", "*"] and
lastUseVar.getAUse() = m.getANormalExit() and
allowedEssaImportStep*(firstDef, lastUseVar) and
not allowedEssaImportStep(_, firstDef)
|
defn.getNode() = essaDef.(AssignmentDefinition).getValue()
not EssaFlow::defToFirstUse(firstDef, _) and
val.asVar() = firstDef
or
defn.getNode() = essaDef.(ArgumentRefinement).getArgument()
exists(ControlFlowNode mid, ControlFlowNode end |
EssaFlow::defToFirstUse(firstDef, mid) and
EssaFlow::useToNextUse*(mid, end) and
not EssaFlow::useToNextUse(end, _) and
val.asCfgNode() = end
)
)
or
// re-exports from `from <pkg> import *`
exists(Module importedFrom |
importedFrom = ImportStar::getStarImported(m) and
module_export(importedFrom, name, val) and
potential_module_export(importedFrom, name)
)
or
// re-exports from `import <pkg>` or `from <pkg> import <stuff>`
exists(Alias a |
defn.asExpr() = [a.getValue(), a.getValue().(ImportMember).getModule()] and
val.asExpr() = a.getValue() and
a.getAsname().(Name).getId() = name and
defn.getScope() = m
val.getScope() = m
)
}
@@ -263,9 +307,21 @@ module ImportResolution {
module_reexport(reexporter, attr_name, m)
)
or
// Submodules that are implicitly defined with relative imports of the form `from .foo import ...`.
// In practice, we create a definition for each module in a package, even if it is not imported.
// submodules of packages will be available as `<pkg>.<submodule>` after doing
// `import <pkg>.<submodule>` at least once in the program, or can be directly
// imported with `from <pkg> import <submodule>` (even with an empty
// `<pkg>.__init__` file).
//
// Until an import of `<pkg>.<submodule>` is executed, it is technically possible
// that `<pkg>.<submodule>` (or `from <pkg> import <submodule>`) can refer to an
// attribute set in `<pkg>.__init__`.
//
// Therefore, if there is an attribute defined in `<pkg>.__init__` with the same
// name as a submodule, we always consider that this attribute _could_ be a
// reference to the submodule, even if we don't know that the submodule has been
// imported yet.
exists(string submodule, Module package |
submodule = result.asVar().getName() and
SsaSource::init_module_submodule_defn(result.asVar().getSourceVariable(),
package.getEntryNode()) and
m = getModuleFromName(package.getPackageName() + "." + submodule)

21
python/ql/lib/semmle/python/dataflow/new/internal/tainttracking1/TaintTracking.qll
View File

@@ -7,7 +7,8 @@ import TaintTrackingParameter::Public
private import TaintTrackingParameter::Private
private module AddTaintDefaults<DataFlowInternal::FullStateConfigSig Config> implements
DataFlowInternal::FullStateConfigSig {
DataFlowInternal::FullStateConfigSig
{
import Config
predicate isBarrier(DataFlow::Node node) {
@@ -32,9 +33,9 @@ DataFlowInternal::FullStateConfigSig {
}
/**
* Constructs a standard taint tracking computation.
* Constructs a global taint tracking computation.
*/
module Make<DataFlow::ConfigSig Config> implements DataFlow::DataFlowSig {
module Global<DataFlow::ConfigSig Config> implements DataFlow::GlobalFlowSig {
private module Config0 implements DataFlowInternal::FullStateConfigSig {
import DataFlowInternal::DefaultState<Config>
import Config
@@ -47,10 +48,15 @@ module Make<DataFlow::ConfigSig Config> implements DataFlow::DataFlowSig {
import DataFlowInternal::Impl<C>
}
/** DEPRECATED: Use `Global` instead. */
deprecated module Make<DataFlow::ConfigSig Config> implements DataFlow::GlobalFlowSig {
import Global<Config>
}
/**
* Constructs a taint tracking computation using flow state.
* Constructs a global taint tracking computation using flow state.
*/
module MakeWithState<DataFlow::StateConfigSig Config> implements DataFlow::DataFlowSig {
module GlobalWithState<DataFlow::StateConfigSig Config> implements DataFlow::GlobalFlowSig {
private module Config0 implements DataFlowInternal::FullStateConfigSig {
import Config
}
@@ -61,3 +67,8 @@ module MakeWithState<DataFlow::StateConfigSig Config> implements DataFlow::DataF
import DataFlowInternal::Impl<C>
}
/** DEPRECATED: Use `GlobalWithState` instead. */
deprecated module MakeWithState<DataFlow::StateConfigSig Config> implements DataFlow::GlobalFlowSig {
import GlobalWithState<Config>
}

24
python/ql/lib/semmle/python/frameworks/Aiohttp.qll
View File

@@ -59,7 +59,8 @@ module AiohttpWebModel {
* Extend this class to refine existing API models. If you want to model new APIs,
* extend `AiohttpRouteSetup::Range` instead.
*/
class AiohttpRouteSetup extends Http::Server::RouteSetup::Range instanceof AiohttpRouteSetup::Range {
class AiohttpRouteSetup extends Http::Server::RouteSetup::Range instanceof AiohttpRouteSetup::Range
{
override Parameter getARoutedParameter() { none() }
override string getFramework() { result = "aiohttp.web" }
@@ -252,7 +253,8 @@ module AiohttpWebModel {
}
/** A request handler defined in an `aiohttp.web` view class, that has no known route. */
private class AiohttpViewClassRequestHandlerWithoutKnownRoute extends Http::Server::RequestHandler::Range {
private class AiohttpViewClassRequestHandlerWithoutKnownRoute extends Http::Server::RequestHandler::Range
{
AiohttpViewClassRequestHandlerWithoutKnownRoute() {
exists(AiohttpViewClass vc | vc.getARequestHandler() = this) and
not exists(AiohttpRouteSetup setup | setup.getARequestHandler() = this)
@@ -440,7 +442,8 @@ module AiohttpWebModel {
* handler is invoked.
*/
class AiohttpRequestHandlerRequestParam extends Request::InstanceSource, RemoteFlowSource::Range,
DataFlow::ParameterNode {
DataFlow::ParameterNode
{
AiohttpRequestHandlerRequestParam() {
exists(Function requestHandler |
requestHandler = any(AiohttpCoroutineRouteSetup setup).getARequestHandler() and
@@ -470,7 +473,8 @@ module AiohttpWebModel {
* which is the request being processed currently.
*/
class AiohttpViewClassRequestAttributeRead extends Request::InstanceSource,
RemoteFlowSource::Range, DataFlow::Node {
RemoteFlowSource::Range, DataFlow::Node
{
AiohttpViewClassRequestAttributeRead() {
this.(DataFlow::AttrRead).getObject() = any(AiohttpViewClass vc).getASelfRef() and
this.(DataFlow::AttrRead).getAttributeName() = "request"
@@ -494,7 +498,8 @@ module AiohttpWebModel {
* - https://docs.aiohttp.org/en/stable/web_quickstart.html#aiohttp-web-exceptions
*/
class AiohttpWebResponseInstantiation extends Http::Server::HttpResponse::Range,
Response::InstanceSource, DataFlow::CallCfgNode {
Response::InstanceSource, DataFlow::CallCfgNode
{
API::Node apiNode;
AiohttpWebResponseInstantiation() {
@@ -562,7 +567,8 @@ module AiohttpWebModel {
* See the part about redirects at https://docs.aiohttp.org/en/stable/web_quickstart.html#aiohttp-web-exceptions
*/
class AiohttpRedirectExceptionInstantiation extends AiohttpWebResponseInstantiation,
Http::Server::HttpRedirectResponse::Range {
Http::Server::HttpRedirectResponse::Range
{
AiohttpRedirectExceptionInstantiation() {
exists(string httpRedirectExceptionClassName |
httpRedirectExceptionClassName in [
@@ -585,7 +591,8 @@ module AiohttpWebModel {
/**
* A call to `set_cookie` on a HTTP Response.
*/
class AiohttpResponseSetCookieCall extends Http::Server::CookieWrite::Range, DataFlow::CallCfgNode {
class AiohttpResponseSetCookieCall extends Http::Server::CookieWrite::Range, DataFlow::CallCfgNode
{
AiohttpResponseSetCookieCall() {
this = aiohttpResponseInstance().getMember("set_cookie").getACall()
}
@@ -600,7 +607,8 @@ module AiohttpWebModel {
/**
* A call to `del_cookie` on a HTTP Response.
*/
class AiohttpResponseDelCookieCall extends Http::Server::CookieWrite::Range, DataFlow::CallCfgNode {
class AiohttpResponseDelCookieCall extends Http::Server::CookieWrite::Range, DataFlow::CallCfgNode
{
AiohttpResponseDelCookieCall() {
this = aiohttpResponseInstance().getMember("del_cookie").getACall()
}

18
python/ql/lib/semmle/python/frameworks/Cryptodome.qll
View File

@@ -23,7 +23,8 @@ private module CryptodomeModel {
* See https://pycryptodome.readthedocs.io/en/latest/src/public_key/rsa.html#Crypto.PublicKey.RSA.generate
*/
class CryptodomePublicKeyRsaGenerateCall extends Cryptography::PublicKey::KeyGeneration::RsaRange,
DataFlow::CallCfgNode {
DataFlow::CallCfgNode
{
CryptodomePublicKeyRsaGenerateCall() {
this =
API::moduleImport(["Crypto", "Cryptodome"])
@@ -44,7 +45,8 @@ private module CryptodomeModel {
* See https://pycryptodome.readthedocs.io/en/latest/src/public_key/dsa.html#Crypto.PublicKey.DSA.generate
*/
class CryptodomePublicKeyDsaGenerateCall extends Cryptography::PublicKey::KeyGeneration::DsaRange,
DataFlow::CallCfgNode {
DataFlow::CallCfgNode
{
CryptodomePublicKeyDsaGenerateCall() {
this =
API::moduleImport(["Crypto", "Cryptodome"])
@@ -65,7 +67,8 @@ private module CryptodomeModel {
* See https://pycryptodome.readthedocs.io/en/latest/src/public_key/ecc.html#Crypto.PublicKey.ECC.generate
*/
class CryptodomePublicKeyEccGenerateCall extends Cryptography::PublicKey::KeyGeneration::EccRange,
DataFlow::CallCfgNode {
DataFlow::CallCfgNode
{
CryptodomePublicKeyEccGenerateCall() {
this =
API::moduleImport(["Crypto", "Cryptodome"])
@@ -105,7 +108,8 @@ private module CryptodomeModel {
* A cryptographic operation on an instance from the `Cipher` subpackage of `Cryptodome`/`Crypto`.
*/
class CryptodomeGenericCipherOperation extends Cryptography::CryptographicOperation::Range,
DataFlow::CallCfgNode {
DataFlow::CallCfgNode
{
string methodName;
string cipherName;
API::CallNode newCall;
@@ -175,7 +179,8 @@ private module CryptodomeModel {
* A cryptographic operation on an instance from the `Signature` subpackage of `Cryptodome`/`Crypto`.
*/
class CryptodomeGenericSignatureOperation extends Cryptography::CryptographicOperation::Range,
DataFlow::CallCfgNode {
DataFlow::CallCfgNode
{
string methodName;
string signatureName;
@@ -214,7 +219,8 @@ private module CryptodomeModel {
* A cryptographic operation on an instance from the `Hash` subpackage of `Cryptodome`/`Crypto`.
*/
class CryptodomeGenericHashOperation extends Cryptography::CryptographicOperation::Range,
DataFlow::CallCfgNode {
DataFlow::CallCfgNode
{
string hashName;
CryptodomeGenericHashOperation() {

15
python/ql/lib/semmle/python/frameworks/Cryptography.qll
View File

@@ -82,7 +82,8 @@ private module CryptographyModel {
* See https://cryptography.io/en/latest/hazmat/primitives/asymmetric/rsa.html#cryptography.hazmat.primitives.asymmetric.rsa.generate_private_key
*/
class CryptographyRsaGeneratePrivateKeyCall extends Cryptography::PublicKey::KeyGeneration::RsaRange,
DataFlow::CallCfgNode {
DataFlow::CallCfgNode
{
CryptographyRsaGeneratePrivateKeyCall() {
this =
API::moduleImport("cryptography")
@@ -105,7 +106,8 @@ private module CryptographyModel {
* See https://cryptography.io/en/latest/hazmat/primitives/asymmetric/dsa.html#cryptography.hazmat.primitives.asymmetric.dsa.generate_private_key
*/
class CryptographyDsaGeneratePrivateKeyCall extends Cryptography::PublicKey::KeyGeneration::DsaRange,
DataFlow::CallCfgNode {
DataFlow::CallCfgNode
{
CryptographyDsaGeneratePrivateKeyCall() {
this =
API::moduleImport("cryptography")
@@ -128,7 +130,8 @@ private module CryptographyModel {
* See https://cryptography.io/en/latest/hazmat/primitives/asymmetric/ec.html#cryptography.hazmat.primitives.asymmetric.ec.generate_private_key
*/
class CryptographyEcGeneratePrivateKeyCall extends Cryptography::PublicKey::KeyGeneration::EccRange,
DataFlow::CallCfgNode {
DataFlow::CallCfgNode
{
CryptographyEcGeneratePrivateKeyCall() {
this =
API::moduleImport("cryptography")
@@ -204,7 +207,8 @@ private module CryptographyModel {
* An encrypt or decrypt operation from `cryptography.hazmat.primitives.ciphers`.
*/
class CryptographyGenericCipherOperation extends Cryptography::CryptographicOperation::Range,
DataFlow::MethodCallNode {
DataFlow::MethodCallNode
{
string algorithmName;
string modeName;
@@ -262,7 +266,8 @@ private module CryptographyModel {
* An hashing operation from `cryptography.hazmat.primitives.hashes`.
*/
class CryptographyGenericHashOperation extends Cryptography::CryptographicOperation::Range,
DataFlow::MethodCallNode {
DataFlow::MethodCallNode
{
string algorithmName;
CryptographyGenericHashOperation() {

45
python/ql/lib/semmle/python/frameworks/Django.qll
View File

@@ -1271,7 +1271,8 @@ module PrivateDjango {
}
/** An attribute read on an django request that is a `MultiValueDict` instance. */
private class DjangoHttpRequestMultiValueDictInstances extends Django::MultiValueDict::InstanceSource {
private class DjangoHttpRequestMultiValueDictInstances extends Django::MultiValueDict::InstanceSource
{
DjangoHttpRequestMultiValueDictInstances() {
this.(DataFlow::AttrRead).getObject() = instance() and
this.(DataFlow::AttrRead).getAttributeName() in ["GET", "POST", "FILES"]
@@ -1279,7 +1280,8 @@ module PrivateDjango {
}
/** An attribute read on an django request that is a `ResolverMatch` instance. */
private class DjangoHttpRequestResolverMatchInstances extends Django::ResolverMatch::InstanceSource {
private class DjangoHttpRequestResolverMatchInstances extends Django::ResolverMatch::InstanceSource
{
DjangoHttpRequestResolverMatchInstances() {
this.(DataFlow::AttrRead).getObject() = instance() and
this.(DataFlow::AttrRead).getAttributeName() = "resolver_match"
@@ -1287,7 +1289,8 @@ module PrivateDjango {
}
/** An `UploadedFile` instance that originates from a django request. */
private class DjangoHttpRequestUploadedFileInstances extends Django::UploadedFile::InstanceSource {
private class DjangoHttpRequestUploadedFileInstances extends Django::UploadedFile::InstanceSource
{
DjangoHttpRequestUploadedFileInstances() {
// TODO: this currently only works in local-scope, since writing type-trackers for
// this is a little too much effort. Once API-graphs are available for more
@@ -1421,7 +1424,8 @@ module PrivateDjango {
* Use the predicate `HttpResponseRedirect::instance()` to get references to instances of `django.http.response.HttpResponseRedirect`.
*/
abstract class InstanceSource extends HttpResponse::InstanceSource,
Http::Server::HttpRedirectResponse::Range, DataFlow::Node { }
Http::Server::HttpRedirectResponse::Range, DataFlow::Node
{ }
/** A direct instantiation of `django.http.response.HttpResponseRedirect`. */
private class ClassInstantiation extends InstanceSource, DataFlow::CallCfgNode {
@@ -1483,7 +1487,8 @@ module PrivateDjango {
* Use the predicate `HttpResponsePermanentRedirect::instance()` to get references to instances of `django.http.response.HttpResponsePermanentRedirect`.
*/
abstract class InstanceSource extends HttpResponse::InstanceSource,
Http::Server::HttpRedirectResponse::Range, DataFlow::Node { }
Http::Server::HttpRedirectResponse::Range, DataFlow::Node
{ }
/** A direct instantiation of `django.http.response.HttpResponsePermanentRedirect`. */
private class ClassInstantiation extends InstanceSource, DataFlow::CallCfgNode {
@@ -2086,7 +2091,8 @@ module PrivateDjango {
*
* See https://docs.djangoproject.com/en/3.1/ref/request-response/#django.http.HttpResponse.write
*/
class HttpResponseWriteCall extends Http::Server::HttpResponse::Range, DataFlow::CallCfgNode {
class HttpResponseWriteCall extends Http::Server::HttpResponse::Range, DataFlow::CallCfgNode
{
DjangoImpl::DjangoHttp::Response::HttpResponse::InstanceSource instance;
HttpResponseWriteCall() { this.getFunction() = write(instance) }
@@ -2106,7 +2112,8 @@ module PrivateDjango {
* A call to `set_cookie` on a HTTP Response.
*/
class DjangoResponseSetCookieCall extends Http::Server::CookieWrite::Range,
DataFlow::MethodCallNode {
DataFlow::MethodCallNode
{
DjangoResponseSetCookieCall() {
this.calls(DjangoImpl::DjangoHttp::Response::HttpResponse::instance(), "set_cookie")
}
@@ -2126,7 +2133,8 @@ module PrivateDjango {
* A call to `delete_cookie` on a HTTP Response.
*/
class DjangoResponseDeleteCookieCall extends Http::Server::CookieWrite::Range,
DataFlow::MethodCallNode {
DataFlow::MethodCallNode
{
DjangoResponseDeleteCookieCall() {
this.calls(DjangoImpl::DjangoHttp::Response::HttpResponse::instance(), "delete_cookie")
}
@@ -2429,7 +2437,8 @@ module PrivateDjango {
/** A request handler defined in a django view class, that has no known route. */
private class DjangoViewClassHandlerWithoutKnownRoute extends Http::Server::RequestHandler::Range,
DjangoRouteHandler {
DjangoRouteHandler
{
DjangoViewClassHandlerWithoutKnownRoute() {
exists(DjangoViewClass vc | vc.getARequestHandler() = this) and
not exists(DjangoRouteSetup setup | setup.getARequestHandler() = this)
@@ -2587,7 +2596,8 @@ module PrivateDjango {
// ---------------------------------------------------------------------------
/** A parameter that will receive the django `HttpRequest` instance when a request handler is invoked. */
private class DjangoRequestHandlerRequestParam extends DjangoImpl::DjangoHttp::Request::HttpRequest::InstanceSource,
RemoteFlowSource::Range, DataFlow::ParameterNode {
RemoteFlowSource::Range, DataFlow::ParameterNode
{
DjangoRequestHandlerRequestParam() {
this.getParameter() = any(DjangoRouteSetup setup).getARequestHandler().getRequestParam()
or
@@ -2604,7 +2614,8 @@ module PrivateDjango {
* See https://docs.djangoproject.com/en/3.1/topics/class-based-views/generic-display/#dynamic-filtering
*/
private class DjangoViewClassRequestAttributeRead extends DjangoImpl::DjangoHttp::Request::HttpRequest::InstanceSource,
RemoteFlowSource::Range, DataFlow::Node {
RemoteFlowSource::Range, DataFlow::Node
{
DjangoViewClassRequestAttributeRead() {
exists(DataFlow::AttrRead read | this = read |
read.getObject() = any(DjangoViewClass vc).getASelfRef() and
@@ -2624,7 +2635,8 @@ module PrivateDjango {
* See https://docs.djangoproject.com/en/3.1/topics/class-based-views/generic-display/#dynamic-filtering
*/
private class DjangoViewClassRoutedParamsAttributeRead extends RemoteFlowSource::Range,
DataFlow::Node {
DataFlow::Node
{
DjangoViewClassRoutedParamsAttributeRead() {
exists(DataFlow::AttrRead read | this = read |
read.getObject() = any(DjangoViewClass vc).getASelfRef() and
@@ -2652,7 +2664,8 @@ module PrivateDjango {
* - https://docs.djangoproject.com/en/3.1/topics/http/file-uploads/#handling-uploaded-files-with-a-model
*/
private class DjangoFileFieldUploadToFunctionFilenameParam extends RemoteFlowSource::Range,
DataFlow::ParameterNode {
DataFlow::ParameterNode
{
DjangoFileFieldUploadToFunctionFilenameParam() {
exists(DataFlow::CallCfgNode call, DataFlow::Node uploadToArg, Function func |
this.getParameter() = func.getArg(1) and
@@ -2679,7 +2692,8 @@ module PrivateDjango {
* See https://docs.djangoproject.com/en/3.1/topics/http/shortcuts/#redirect
*/
private class DjangoShortcutsRedirectCall extends Http::Server::HttpRedirectResponse::Range,
DataFlow::CallCfgNode {
DataFlow::CallCfgNode
{
DjangoShortcutsRedirectCall() { this = DjangoImpl::Shortcuts::redirect().getACall() }
/**
@@ -2713,7 +2727,8 @@ module PrivateDjango {
* See https://docs.djangoproject.com/en/3.1/ref/class-based-views/base/#redirectview
*/
private class DjangoRedirectViewGetRedirectUrlReturn extends Http::Server::HttpRedirectResponse::Range,
DataFlow::CfgNode {
DataFlow::CfgNode
{
DjangoRedirectViewGetRedirectUrlReturn() {
node = any(GetRedirectUrlFunction f).getAReturnValueFlowNode()
}

23
python/ql/lib/semmle/python/frameworks/Fabric.qll
View File

@@ -43,13 +43,22 @@ private module FabricV1 {
* - https://docs.fabfile.org/en/1.14/api/core/operations.html#fabric.operations.run
* - https://docs.fabfile.org/en/1.14/api/core/operations.html#fabric.operations.sudo
*/
private class FabricApiLocalRunSudoCall extends SystemCommandExecution::Range,
DataFlow::CallCfgNode {
private class FabricApiLocalRunSudoCall extends SystemCommandExecution::Range, API::CallNode {
FabricApiLocalRunSudoCall() { this = api().getMember(["local", "run", "sudo"]).getACall() }
override DataFlow::Node getCommand() {
result = [this.getArg(0), this.getArgByName("command")]
}
override predicate isShellInterpreted(DataFlow::Node arg) {
arg = this.getCommand() and
// defaults to running in a shell
not this.getParameter(1, "shell")
.getAValueReachingSink()
.asExpr()
.(ImmutableLiteral)
.booleanValue() = false
}
}
}
}
@@ -153,7 +162,8 @@ private module FabricV2 {
* - https://docs.fabfile.org/en/2.5/api/connection.html#fabric.connection.Connection.local
*/
private class FabricConnectionRunSudoLocalCall extends SystemCommandExecution::Range,
DataFlow::CallCfgNode {
DataFlow::CallCfgNode
{
FabricConnectionRunSudoLocalCall() {
this.getFunction() = Fabric::Connection::ConnectionClass::instanceRunMethods()
}
@@ -161,6 +171,8 @@ private module FabricV2 {
override DataFlow::Node getCommand() {
result = [this.getArg(0), this.getArgByName("command")]
}
override predicate isShellInterpreted(DataFlow::Node arg) { arg = this.getCommand() }
}
// -------------------------------------------------------------------------
@@ -176,7 +188,8 @@ private module FabricV2 {
}
class FabricTaskFirstParamConnectionInstance extends Fabric::Connection::ConnectionClass::InstanceSource,
DataFlow::ParameterNode {
DataFlow::ParameterNode
{
FabricTaskFirstParamConnectionInstance() {
exists(Function func |
func.getADecorator() = Fabric::Tasks::task().getAValueReachableFromSource().asExpr() and
@@ -243,6 +256,8 @@ private module FabricV2 {
override DataFlow::Node getCommand() {
result = [this.getArg(0), this.getArgByName("command")]
}
override predicate isShellInterpreted(DataFlow::Node arg) { arg = this.getCommand() }
}
/**

24
python/ql/lib/semmle/python/frameworks/FastApi.qll
View File

@@ -88,7 +88,8 @@ private module FastApi {
* Pydantic model.
*/
private class PydanticModelRequestHandlerParam extends Pydantic::BaseModel::InstanceSource,
DataFlow::ParameterNode {
DataFlow::ParameterNode
{
PydanticModelRequestHandlerParam() {
this.getParameter().getAnnotation() =
Pydantic::BaseModel::subclassRef().getAValueReachableFromSource().asExpr() and
@@ -103,7 +104,8 @@ private module FastApi {
* A parameter to a request handler that has a WebSocket type-annotation.
*/
private class WebSocketRequestHandlerParam extends Starlette::WebSocket::InstanceSource,
DataFlow::ParameterNode {
DataFlow::ParameterNode
{
WebSocketRequestHandlerParam() {
this.getParameter().getAnnotation() =
Starlette::WebSocket::classRef().getAValueReachableFromSource().asExpr() and
@@ -196,7 +198,8 @@ private module FastApi {
/** A direct instantiation of a response class. */
private class ResponseInstantiation extends InstanceSource, Http::Server::HttpResponse::Range,
DataFlow::CallCfgNode {
DataFlow::CallCfgNode
{
API::Node baseApiNode;
API::Node responseClass;
@@ -223,7 +226,8 @@ private module FastApi {
* A direct instantiation of a redirect response.
*/
private class RedirectResponseInstantiation extends ResponseInstantiation,
Http::Server::HttpRedirectResponse::Range {
Http::Server::HttpRedirectResponse::Range
{
RedirectResponseInstantiation() { baseApiNode = getModeledResponseClass("RedirectResponse") }
override DataFlow::Node getRedirectLocation() {
@@ -246,7 +250,8 @@ private module FastApi {
* An implicit response from a return of FastAPI request handler.
*/
private class FastApiRequestHandlerReturn extends Http::Server::HttpResponse::Range,
DataFlow::CfgNode {
DataFlow::CfgNode
{
FastApiRouteSetup routeSetup;
FastApiRequestHandlerReturn() {
@@ -273,7 +278,8 @@ private module FastApi {
* `response_class` set to a `FileResponse`.
*/
private class FastApiRequestHandlerFileResponseReturn extends FastApiRequestHandlerReturn,
FileSystemAccess::Range {
FileSystemAccess::Range
{
FastApiRequestHandlerFileResponseReturn() {
exists(API::Node responseClass |
responseClass.getAValueReachableFromSource() = routeSetup.getResponseClassArg() and
@@ -291,7 +297,8 @@ private module FastApi {
* `response_class` set to a `RedirectResponse`.
*/
private class FastApiRequestHandlerRedirectReturn extends FastApiRequestHandlerReturn,
Http::Server::HttpRedirectResponse::Range {
Http::Server::HttpRedirectResponse::Range
{
FastApiRequestHandlerRedirectReturn() {
exists(API::Node responseClass |
responseClass.getAValueReachableFromSource() = routeSetup.getResponseClassArg() and
@@ -349,7 +356,8 @@ private module FastApi {
* header-key.
*/
private class HeadersAppendCookie extends Http::Server::CookieWrite::Range,
DataFlow::MethodCallNode {
DataFlow::MethodCallNode
{
HeadersAppendCookie() {
exists(DataFlow::AttrRead headers, DataFlow::Node keyArg |
headers.accesses(instance(), "headers") and

12
python/ql/lib/semmle/python/frameworks/Flask.qll
View File

@@ -447,7 +447,8 @@ module Flask {
// ---------------------------------------------------------------------------
// Implicit response from returns of flask request handlers
// ---------------------------------------------------------------------------
private class FlaskRouteHandlerReturn extends Http::Server::HttpResponse::Range, DataFlow::CfgNode {
private class FlaskRouteHandlerReturn extends Http::Server::HttpResponse::Range, DataFlow::CfgNode
{
FlaskRouteHandlerReturn() {
exists(Function routeHandler |
routeHandler = any(FlaskRouteSetup rs).getARequestHandler() and
@@ -471,7 +472,8 @@ module Flask {
* See https://flask.palletsprojects.com/en/1.1.x/api/#flask.redirect
*/
private class FlaskRedirectCall extends Http::Server::HttpRedirectResponse::Range,
DataFlow::CallCfgNode {
DataFlow::CallCfgNode
{
FlaskRedirectCall() { this = API::moduleImport("flask").getMember("redirect").getACall() }
override DataFlow::Node getRedirectLocation() {
@@ -499,7 +501,8 @@ module Flask {
* See https://flask.palletsprojects.com/en/2.0.x/api/#flask.Response.set_cookie
*/
class FlaskResponseSetCookieCall extends Http::Server::CookieWrite::Range,
DataFlow::MethodCallNode {
DataFlow::MethodCallNode
{
FlaskResponseSetCookieCall() { this.calls(Flask::Response::instance(), "set_cookie") }
override DataFlow::Node getHeaderArg() { none() }
@@ -515,7 +518,8 @@ module Flask {
* See https://flask.palletsprojects.com/en/2.0.x/api/#flask.Response.delete_cookie
*/
class FlaskResponseDeleteCookieCall extends Http::Server::CookieWrite::Range,
DataFlow::MethodCallNode {
DataFlow::MethodCallNode
{
FlaskResponseDeleteCookieCall() { this.calls(Flask::Response::instance(), "delete_cookie") }
override DataFlow::Node getHeaderArg() { none() }

2
python/ql/lib/semmle/python/frameworks/Invoke.qll
View File

@@ -81,5 +81,7 @@ private module Invoke {
override DataFlow::Node getCommand() {
result in [this.getArg(0), this.getArgByName("command")]
}
override predicate isShellInterpreted(DataFlow::Node arg) { arg = this.getCommand() }
}
}

3
python/ql/lib/semmle/python/frameworks/Lxml.qll
View File

@@ -307,7 +307,8 @@ private module Lxml {
* - https://lxml.de/apidoc/lxml.etree.html?highlight=parseids#lxml.etree.iterparse
*/
private class LxmlIterparseCall extends API::CallNode, XML::XmlParsing::Range,
FileSystemAccess::Range {
FileSystemAccess::Range
{
LxmlIterparseCall() {
this = API::moduleImport("lxml").getMember("etree").getMember("iterparse").getACall()
}

6
python/ql/lib/semmle/python/frameworks/MarkupSafe.qll
View File

@@ -101,7 +101,8 @@ private module MarkupSafeModel {
/** A call to any of the escaping functions in `markupsafe` */
private class MarkupSafeEscapeCall extends Markup::InstanceSource, MarkupSafeEscape,
DataFlow::CallCfgNode {
DataFlow::CallCfgNode
{
MarkupSafeEscapeCall() {
this = API::moduleImport("markupsafe").getMember(["escape", "escape_silent"]).getACall()
or
@@ -141,7 +142,8 @@ private module MarkupSafeModel {
/** A escape from %-style string format with `markupsafe.Markup` as the format string. */
private class MarkupEscapeFromPercentStringFormat extends MarkupSafeEscape,
Markup::PercentStringFormat {
Markup::PercentStringFormat
{
override DataFlow::Node getAnInput() {
result.asCfgNode() = node.getRight() and
not result = Markup::instance()

3
python/ql/lib/semmle/python/frameworks/Peewee.qll
View File

@@ -164,7 +164,8 @@ private module Peewee {
* https://docs.peewee-orm.com/en/latest/peewee/api.html#Database.connection.
*/
class PeeweeDatabaseConnectionCall extends PEP249::Connection::InstanceSource,
DataFlow::CallCfgNode {
DataFlow::CallCfgNode
{
PeeweeDatabaseConnectionCall() {
this = Database::instance().getMember("connection").getACall()
}

16
python/ql/lib/semmle/python/frameworks/RestFramework.qll
View File

@@ -159,7 +159,8 @@ private module RestFramework {
* known route setup.
*/
class RestFrameworkFunctionBasedViewWithoutKnownRoute extends Http::Server::RequestHandler::Range,
PrivateDjango::DjangoRouteHandler instanceof RestFrameworkFunctionBasedView {
PrivateDjango::DjangoRouteHandler instanceof RestFrameworkFunctionBasedView
{
RestFrameworkFunctionBasedViewWithoutKnownRoute() {
not exists(PrivateDjango::DjangoRouteSetup setup | setup.getARequestHandler() = this)
}
@@ -183,7 +184,8 @@ private module RestFramework {
* request handler is invoked.
*/
private class RestFrameworkRequestHandlerRequestParam extends Request::InstanceSource,
RemoteFlowSource::Range, DataFlow::ParameterNode {
RemoteFlowSource::Range, DataFlow::ParameterNode
{
RestFrameworkRequestHandlerRequestParam() {
// rest_framework.views.APIView subclass
exists(RestFrameworkApiViewClass vc |
@@ -220,8 +222,8 @@ private module RestFramework {
*
* Use the predicate `Request::instance()` to get references to instances of `rest_framework.request.Request`.
*/
abstract class InstanceSource extends PrivateDjango::DjangoImpl::DjangoHttp::Request::HttpRequest::InstanceSource {
}
abstract class InstanceSource extends PrivateDjango::DjangoImpl::DjangoHttp::Request::HttpRequest::InstanceSource
{ }
/** A direct instantiation of `rest_framework.request.Request`. */
private class ClassInstantiation extends InstanceSource, DataFlow::CallCfgNode {
@@ -297,7 +299,8 @@ private module RestFramework {
/** A direct instantiation of `rest_framework.response.Response`. */
private class ClassInstantiation extends PrivateDjango::DjangoImpl::DjangoHttp::Response::HttpResponse::InstanceSource,
DataFlow::CallCfgNode {
DataFlow::CallCfgNode
{
ClassInstantiation() { this = classRef().getACall() }
override DataFlow::Node getBody() { result in [this.getArg(0), this.getArgByName("data")] }
@@ -321,7 +324,8 @@ private module RestFramework {
module ApiException {
/** A direct instantiation of `rest_framework.exceptions.ApiException` or subclass. */
private class ClassInstantiation extends Http::Server::HttpResponse::Range,
DataFlow::CallCfgNode {
DataFlow::CallCfgNode
{
string className;
ClassInstantiation() {

6
python/ql/lib/semmle/python/frameworks/Rsa.qll
View File

@@ -20,7 +20,8 @@ private module Rsa {
* See https://stuvel.eu/python-rsa-doc/reference.html#rsa.newkeys
*/
class RsaNewkeysCall extends Cryptography::PublicKey::KeyGeneration::RsaRange,
DataFlow::CallCfgNode {
DataFlow::CallCfgNode
{
RsaNewkeysCall() { this = API::moduleImport("rsa").getMember("newkeys").getACall() }
override DataFlow::Node getKeySizeArg() {
@@ -116,7 +117,8 @@ private module Rsa {
* See https://stuvel.eu/python-rsa-doc/reference.html#rsa.compute_hash
*/
class RsaComputeHashCall extends Cryptography::CryptographicOperation::Range,
DataFlow::CallCfgNode {
DataFlow::CallCfgNode
{
RsaComputeHashCall() { this = API::moduleImport("rsa").getMember("compute_hash").getACall() }
override Cryptography::CryptographicAlgorithm getAlgorithm() {

74
python/ql/lib/semmle/python/frameworks/Setuptools.qll Normal file
View File

@@ -0,0 +1,74 @@
/**
* Provides classes modeling package setup as defined by `setuptools`.
*/
private import python
private import semmle.python.dataflow.new.DataFlow
/** Provides models for the use of `setuptools` in setup scripts, and the APIs exported by the library defined using `setuptools`. */
module Setuptools {
/**
* Gets a file that sets up a package using `setuptools` (or the deprecated `distutils`).
*/
private File setupFile() {
// all of these might not be extracted, but the support is ready for when they are
result.getBaseName() = ["setup.py", "setup.cfg", "pyproject.toml"]
}
/**
* Gets a file or folder that is exported by a library.
*/
private Container getALibraryExportedContainer() {
// a child folder of the root that has a setup.py file
result = setupFile().getParent().(Folder).getAFolder() and
// where the folder has __init__.py file
exists(result.(Folder).getFile("__init__.py")) and
// and is not a test folder
not result.(Folder).getBaseName() = ["test", "tests", "testing"]
or
// child of a library exported container
result = getALibraryExportedContainer().getAChildContainer() and
(
// either any file
not result instanceof Folder
or
// or a folder with an __init__.py file
exists(result.(Folder).getFile("__init__.py"))
)
}
/**
* Gets an AST node that is exported by a library.
*/
private AstNode getAnExportedLibraryFeature() {
result.(Module).getFile() = getALibraryExportedContainer()
or
result = getAnExportedLibraryFeature().(Module).getAStmt()
or
result = getAnExportedLibraryFeature().(ClassDef).getDefinedClass().getAMethod()
or
result = getAnExportedLibraryFeature().(ClassDef).getDefinedClass().getInitMethod()
or
result = getAnExportedLibraryFeature().(FunctionDef).getDefinedFunction()
}
/**
* Gets a public function (or __init__) that is exported by a library.
*/
private Function getAnExportedFunction() {
result = getAnExportedLibraryFeature() and
(
result.isPublic()
or
result.isInitMethod()
)
}
/**
* Gets a parameter to a public function that is exported by a library.
*/
DataFlow::ParameterNode getALibraryInput() {
result.getParameter() = getAnExportedFunction().getAnArg() and
not result.getParameter().isSelf()
}
}

3
python/ql/lib/semmle/python/frameworks/Starlette.qll
View File

@@ -152,7 +152,8 @@ module Starlette {
}
/** An attribute read on a `starlette.requests.URL` instance that is a `urllib.parse.SplitResult` instance. */
private class UrlSplitInstances extends Stdlib::SplitResult::InstanceSource instanceof DataFlow::AttrRead {
private class UrlSplitInstances extends Stdlib::SplitResult::InstanceSource instanceof DataFlow::AttrRead
{
UrlSplitInstances() {
super.getObject() = instance() and
super.getAttributeName() = "components"

129
python/ql/lib/semmle/python/frameworks/Stdlib.qll
View File

@@ -1060,7 +1060,11 @@ private module StdlibPrivate {
private class OsSystemCall extends SystemCommandExecution::Range, DataFlow::CallCfgNode {
OsSystemCall() { this = os().getMember("system").getACall() }
override DataFlow::Node getCommand() { result = this.getArg(0) }
override DataFlow::Node getCommand() {
result in [this.getArg(0), this.getArgByName("command")]
}
override predicate isShellInterpreted(DataFlow::Node arg) { arg = this.getCommand() }
}
/**
@@ -1071,7 +1075,7 @@ private module StdlibPrivate {
* Although deprecated since version 2.6, they still work in 2.7.
* See https://docs.python.org/2.7/library/os.html#os.popen2
*/
private class OsPopenCall extends SystemCommandExecution::Range, DataFlow::CallCfgNode {
private class OsPopenCall extends SystemCommandExecution::Range, API::CallNode {
string name;
OsPopenCall() {
@@ -1085,6 +1089,8 @@ private module StdlibPrivate {
not name = "popen" and
result = this.getArgByName("cmd")
}
override predicate isShellInterpreted(DataFlow::Node arg) { arg = this.getCommand() }
}
/**
@@ -1092,7 +1098,8 @@ private module StdlibPrivate {
* See https://docs.python.org/3.8/library/os.html#os.execl
*/
private class OsExecCall extends SystemCommandExecution::Range, FileSystemAccess::Range,
DataFlow::CallCfgNode {
DataFlow::CallCfgNode
{
OsExecCall() {
exists(string name |
name in ["execl", "execle", "execlp", "execlpe", "execv", "execve", "execvp", "execvpe"] and
@@ -1103,6 +1110,10 @@ private module StdlibPrivate {
override DataFlow::Node getCommand() { result = this.getArg(0) }
override DataFlow::Node getAPathArgument() { result = this.getCommand() }
override predicate isShellInterpreted(DataFlow::Node arg) {
none() // this is a safe API.
}
}
/**
@@ -1110,7 +1121,8 @@ private module StdlibPrivate {
* See https://docs.python.org/3.8/library/os.html#os.spawnl
*/
private class OsSpawnCall extends SystemCommandExecution::Range, FileSystemAccess::Range,
DataFlow::CallCfgNode {
DataFlow::CallCfgNode
{
OsSpawnCall() {
exists(string name |
name in [
@@ -1129,6 +1141,10 @@ private module StdlibPrivate {
}
override DataFlow::Node getAPathArgument() { result = this.getCommand() }
override predicate isShellInterpreted(DataFlow::Node arg) {
none() // this is a safe API.
}
}
/**
@@ -1136,12 +1152,17 @@ private module StdlibPrivate {
* See https://docs.python.org/3.8/library/os.html#os.posix_spawn
*/
private class OsPosixSpawnCall extends SystemCommandExecution::Range, FileSystemAccess::Range,
DataFlow::CallCfgNode {
DataFlow::CallCfgNode
{
OsPosixSpawnCall() { this = os().getMember(["posix_spawn", "posix_spawnp"]).getACall() }
override DataFlow::Node getCommand() { result in [this.getArg(0), this.getArgByName("path")] }
override DataFlow::Node getAPathArgument() { result = this.getCommand() }
override predicate isShellInterpreted(DataFlow::Node arg) {
none() // this is a safe API.
}
}
/** An additional taint step for calls to `os.path.join` */
@@ -1167,7 +1188,7 @@ private module StdlibPrivate {
* See https://docs.python.org/3.8/library/subprocess.html#subprocess.Popen
* ref: https://docs.python.org/3/library/subprocess.html#legacy-shell-invocation-functions
*/
private class SubprocessPopenCall extends SystemCommandExecution::Range, DataFlow::CallCfgNode {
private class SubprocessPopenCall extends SystemCommandExecution::Range, API::CallNode {
SubprocessPopenCall() {
exists(string name |
name in [
@@ -1177,43 +1198,33 @@ private module StdlibPrivate {
)
}
/** Gets the ControlFlowNode for the `args` argument, if any. */
private DataFlow::Node get_args_arg() { result in [this.getArg(0), this.getArgByName("args")] }
/** Gets the API-node for the `args` argument, if any. */
private API::Node get_args_arg() { result = this.getParameter(0, "args") }
/** Gets the ControlFlowNode for the `shell` argument, if any. */
private DataFlow::Node get_shell_arg() {
result in [this.getArg(8), this.getArgByName("shell")]
}
/** Gets the API-node for the `shell` argument, if any. */
private API::Node get_shell_arg() { result = this.getParameter(8, "shell") }
private boolean get_shell_arg_value() {
not exists(this.get_shell_arg()) and
result = false
or
exists(DataFlow::Node shell_arg | shell_arg = this.get_shell_arg() |
result = shell_arg.asCfgNode().getNode().(ImmutableLiteral).booleanValue()
or
// TODO: Track the "shell" argument to determine possible values
not shell_arg.asCfgNode().getNode() instanceof ImmutableLiteral and
(
result = true
or
result = false
)
)
result =
this.get_shell_arg().getAValueReachingSink().asExpr().(ImmutableLiteral).booleanValue()
or
not this.get_shell_arg().getAValueReachingSink().asExpr() instanceof ImmutableLiteral and
result = false // defaults to `False`
}
/** Gets the ControlFlowNode for the `executable` argument, if any. */
private DataFlow::Node get_executable_arg() {
result in [this.getArg(2), this.getArgByName("executable")]
}
/** Gets the API-node for the `executable` argument, if any. */
private API::Node get_executable_arg() { result = this.getParameter(2, "executable") }
override DataFlow::Node getCommand() {
// TODO: Track arguments ("args" and "shell")
// TODO: Handle using `args=["sh", "-c", <user-input>]`
result = this.get_executable_arg()
result = this.get_executable_arg().asSink()
or
exists(DataFlow::Node arg_args, boolean shell |
arg_args = this.get_args_arg() and
arg_args = this.get_args_arg().asSink() and
shell = this.get_shell_arg_value()
|
// When "executable" argument is set, and "shell" argument is `False`, the
@@ -1239,6 +1250,11 @@ private module StdlibPrivate {
)
)
}
override predicate isShellInterpreted(DataFlow::Node arg) {
arg = [this.get_executable_arg(), this.get_args_arg()].asSink() and
this.get_shell_arg_value() = true
}
}
// ---------------------------------------------------------------------------
@@ -1348,7 +1364,8 @@ private module StdlibPrivate {
* argument as being deserialized...
*/
private class ShelveOpenCall extends Decoding::Range, FileSystemAccess::Range,
DataFlow::CallCfgNode {
DataFlow::CallCfgNode
{
ShelveOpenCall() { this = API::moduleImport("shelve").getMember("open").getACall() }
override predicate mayExecuteInput() { any() }
@@ -1385,6 +1402,8 @@ private module StdlibPrivate {
}
override DataFlow::Node getCommand() { result in [this.getArg(0), this.getArgByName("cmd")] }
override predicate isShellInterpreted(DataFlow::Node arg) { arg = this.getCommand() }
}
// ---------------------------------------------------------------------------
@@ -1401,6 +1420,8 @@ private module StdlibPrivate {
PlatformPopenCall() { this = platform().getMember("popen").getACall() }
override DataFlow::Node getCommand() { result in [this.getArg(0), this.getArgByName("cmd")] }
override predicate isShellInterpreted(DataFlow::Node arg) { arg = this.getCommand() }
}
// ---------------------------------------------------------------------------
@@ -1452,7 +1473,8 @@ private module StdlibPrivate {
* See https://docs.python.org/3/library/functions.html#open
*/
private class OpenCall extends FileSystemAccess::Range, Stdlib::FileLikeObject::InstanceSource,
DataFlow::CallCfgNode {
DataFlow::CallCfgNode
{
OpenCall() { this = getOpenFunctionRef().getACall() }
override DataFlow::Node getAPathArgument() {
@@ -1712,7 +1734,8 @@ private module StdlibPrivate {
* if it turns out to be a problem, we'll have to refine.
*/
private class ClassInstantiation extends InstanceSource, RemoteFlowSource::Range,
DataFlow::CallCfgNode {
DataFlow::CallCfgNode
{
ClassInstantiation() { this = classRef().getACall() }
override string getSourceType() { result = "cgi.FieldStorage" }
@@ -1970,7 +1993,8 @@ private module StdlibPrivate {
abstract class InstanceSource extends DataFlow::Node { }
/** The `self` parameter in a method on the `BaseHttpRequestHandler` class or any subclass. */
private class SelfParam extends InstanceSource, RemoteFlowSource::Range, DataFlow::ParameterNode {
private class SelfParam extends InstanceSource, RemoteFlowSource::Range, DataFlow::ParameterNode
{
SelfParam() {
exists(HttpRequestHandlerClassDef cls | cls.getAMethod().getArg(0) = this.getParameter())
}
@@ -2008,14 +2032,16 @@ private module StdlibPrivate {
}
/** An `HttpMessage` instance that originates from a `BaseHttpRequestHandler` instance. */
private class BaseHttpRequestHandlerHeadersInstances extends Stdlib::HttpMessage::InstanceSource {
private class BaseHttpRequestHandlerHeadersInstances extends Stdlib::HttpMessage::InstanceSource
{
BaseHttpRequestHandlerHeadersInstances() {
this.(DataFlow::AttrRead).accesses(instance(), "headers")
}
}
/** A file-like object that originates from a `BaseHttpRequestHandler` instance. */
private class BaseHttpRequestHandlerFileLikeObjectInstances extends Stdlib::FileLikeObject::InstanceSource {
private class BaseHttpRequestHandlerFileLikeObjectInstances extends Stdlib::FileLikeObject::InstanceSource
{
BaseHttpRequestHandlerFileLikeObjectInstances() {
this.(DataFlow::AttrRead).accesses(instance(), "rfile")
}
@@ -2167,7 +2193,8 @@ private module StdlibPrivate {
* See https://github.com/python/cpython/blob/b567b9d74bd9e476a3027335873bb0508d6e450f/Lib/wsgiref/handlers.py#L276
*/
class WsgirefSimpleServerApplicationWriteCall extends Http::Server::HttpResponse::Range,
DataFlow::CallCfgNode {
DataFlow::CallCfgNode
{
WsgirefSimpleServerApplicationWriteCall() { this.getFunction() = writeFunction() }
override DataFlow::Node getBody() { result in [this.getArg(0), this.getArgByName("data")] }
@@ -2181,7 +2208,8 @@ private module StdlibPrivate {
* A return from a `WsgirefSimpleServerApplication`, which is included in the response body.
*/
class WsgirefSimpleServerApplicationReturn extends Http::Server::HttpResponse::Range,
DataFlow::CfgNode {
DataFlow::CfgNode
{
WsgirefSimpleServerApplicationReturn() {
exists(WsgirefSimpleServerApplication requestHandler |
node = requestHandler.getAReturnValueFlowNode()
@@ -2292,7 +2320,8 @@ private module StdlibPrivate {
/** A call to the `getresponse` method. */
private class HttpConnectionGetResponseCall extends DataFlow::MethodCallNode,
HttpResponse::InstanceSource {
HttpResponse::InstanceSource
{
HttpConnectionGetResponseCall() { this.calls(instance(_), "getresponse") }
}
@@ -2351,7 +2380,8 @@ private module StdlibPrivate {
* Use the predicate `HTTPResponse::instance()` to get references to instances of `http.client.HTTPResponse`.
*/
abstract class InstanceSource extends Stdlib::FileLikeObject::InstanceSource,
DataFlow::LocalSourceNode { }
DataFlow::LocalSourceNode
{ }
/** A direct instantiation of `http.client.HttpResponse`. */
private class ClassInstantiation extends InstanceSource, DataFlow::CallCfgNode {
@@ -2722,7 +2752,8 @@ private module StdlibPrivate {
* `HashlibNewCall` and `HashlibNewUpdateCall`.
*/
abstract class HashlibGenericHashOperation extends Cryptography::CryptographicOperation::Range,
DataFlow::CallCfgNode {
DataFlow::CallCfgNode
{
string hashName;
API::Node hashClass;
@@ -2768,7 +2799,8 @@ private module StdlibPrivate {
// hmac
// ---------------------------------------------------------------------------
abstract class HmacCryptographicOperation extends Cryptography::CryptographicOperation::Range,
API::CallNode {
API::CallNode
{
abstract API::Node getDigestArg();
override Cryptography::CryptographicAlgorithm getAlgorithm() {
@@ -2996,7 +3028,8 @@ private module StdlibPrivate {
}
/** Extra taint-step such that the result of `urllib.parse.urlsplit(tainted_string)` is tainted. */
private class UrllibParseUrlsplitCallAdditionalTaintStep extends TaintTracking::AdditionalTaintStep {
private class UrllibParseUrlsplitCallAdditionalTaintStep extends TaintTracking::AdditionalTaintStep
{
override predicate step(DataFlow::Node nodeFrom, DataFlow::Node nodeTo) {
nodeTo.(UrllibParseUrlsplitCall).getUrl() = nodeFrom
}
@@ -3027,7 +3060,8 @@ private module StdlibPrivate {
* See https://docs.python.org/3/library/tempfile.html#tempfile.NamedTemporaryFile
*/
private class TempfileNamedTemporaryFileCall extends FileSystemAccess::Range,
DataFlow::CallCfgNode {
DataFlow::CallCfgNode
{
TempfileNamedTemporaryFileCall() {
this = API::moduleImport("tempfile").getMember("NamedTemporaryFile").getACall()
}
@@ -3064,7 +3098,8 @@ private module StdlibPrivate {
* See https://docs.python.org/3/library/tempfile.html#tempfile.SpooledTemporaryFile
*/
private class TempfileSpooledTemporaryFileCall extends FileSystemAccess::Range,
DataFlow::CallCfgNode {
DataFlow::CallCfgNode
{
TempfileSpooledTemporaryFileCall() {
this = API::moduleImport("tempfile").getMember("SpooledTemporaryFile").getACall()
}
@@ -3099,7 +3134,8 @@ private module StdlibPrivate {
* See https://docs.python.org/3/library/tempfile.html#tempfile.TemporaryDirectory
*/
private class TempfileTemporaryDirectoryCall extends FileSystemAccess::Range,
DataFlow::CallCfgNode {
DataFlow::CallCfgNode
{
TempfileTemporaryDirectoryCall() {
this = API::moduleImport("tempfile").getMember("TemporaryDirectory").getACall()
}
@@ -3556,7 +3592,8 @@ private module StdlibPrivate {
* See https://docs.python.org/3/library/xml.sax.reader.html#xml.sax.xmlreader.XMLReader.parse
*/
private class XmlSaxInstanceParsing extends DataFlow::MethodCallNode, XML::XmlParsing::Range,
FileSystemAccess::Range {
FileSystemAccess::Range
{
XmlSaxInstanceParsing() {
this =
API::moduleImport("xml")

12
python/ql/lib/semmle/python/frameworks/Tornado.qll
View File

@@ -200,7 +200,8 @@ module Tornado {
override string getAsyncMethodName() { none() }
}
private class RequestAttrAccess extends TornadoModule::HttpUtil::HttpServerRequest::InstanceSource {
private class RequestAttrAccess extends TornadoModule::HttpUtil::HttpServerRequest::InstanceSource
{
RequestAttrAccess() {
this.(DataFlow::AttrRead).getObject() = instance() and
this.(DataFlow::AttrRead).getAttributeName() = "request"
@@ -463,7 +464,8 @@ module Tornado {
* See https://www.tornadoweb.org/en/stable/web.html#tornado.web.RequestHandler.redirect
*/
private class TornadoRequestHandlerRedirectCall extends Http::Server::HttpRedirectResponse::Range,
DataFlow::CallCfgNode {
DataFlow::CallCfgNode
{
TornadoRequestHandlerRedirectCall() {
this.getFunction() = TornadoModule::Web::RequestHandler::redirectMethod()
}
@@ -485,7 +487,8 @@ module Tornado {
* See https://www.tornadoweb.org/en/stable/web.html#tornado.web.RequestHandler.write
*/
private class TornadoRequestHandlerWriteCall extends Http::Server::HttpResponse::Range,
DataFlow::CallCfgNode {
DataFlow::CallCfgNode
{
TornadoRequestHandlerWriteCall() {
this.getFunction() = TornadoModule::Web::RequestHandler::writeMethod()
}
@@ -503,7 +506,8 @@ module Tornado {
* See https://www.tornadoweb.org/en/stable/web.html#tornado.web.RequestHandler.set_cookie
*/
class TornadoRequestHandlerSetCookieCall extends Http::Server::CookieWrite::Range,
DataFlow::MethodCallNode {
DataFlow::MethodCallNode
{
TornadoRequestHandlerSetCookieCall() {
this.calls(TornadoModule::Web::RequestHandler::instance(), "set_cookie")
}

18
python/ql/lib/semmle/python/frameworks/Twisted.qll
View File

@@ -143,7 +143,8 @@ private module Twisted {
* when a twisted request handler is called.
*/
class TwistedResourceRequestHandlerRequestParam extends RemoteFlowSource::Range,
Request::InstanceSource, DataFlow::ParameterNode {
Request::InstanceSource, DataFlow::ParameterNode
{
TwistedResourceRequestHandlerRequestParam() {
this.getParameter() = any(TwistedResourceRequestHandler handler).getRequestParameter()
}
@@ -156,7 +157,8 @@ private module Twisted {
* that is also given remote user input. (a bit like RoutedParameter).
*/
class TwistedResourceRequestHandlerExtraSources extends RemoteFlowSource::Range,
DataFlow::ParameterNode {
DataFlow::ParameterNode
{
TwistedResourceRequestHandlerExtraSources() {
exists(TwistedResourceRequestHandler func, int i |
func.getName() in ["getChild", "getChildWithDefault"] and i = 1
@@ -177,7 +179,8 @@ private module Twisted {
* Implicit response from returns of render methods.
*/
private class TwistedResourceRenderMethodReturn extends Http::Server::HttpResponse::Range,
DataFlow::CfgNode {
DataFlow::CfgNode
{
TwistedResourceRenderMethodReturn() {
this.asCfgNode() = any(TwistedResourceRenderMethod meth).getAReturnValueFlowNode()
}
@@ -212,7 +215,8 @@ private module Twisted {
* See https://twistedmatrix.com/documents/21.2.0/api/twisted.web.http.Request.html#redirect
*/
class TwistedRequestRedirectCall extends Http::Server::HttpRedirectResponse::Range,
DataFlow::MethodCallNode {
DataFlow::MethodCallNode
{
TwistedRequestRedirectCall() { this.calls(Request::instance(), "redirect") }
override DataFlow::Node getBody() { none() }
@@ -232,7 +236,8 @@ private module Twisted {
* See https://twistedmatrix.com/documents/21.2.0/api/twisted.web.http.Request.html#addCookie
*/
class TwistedRequestAddCookieCall extends Http::Server::CookieWrite::Range,
DataFlow::MethodCallNode {
DataFlow::MethodCallNode
{
TwistedRequestAddCookieCall() { this.calls(Twisted::Request::instance(), "addCookie") }
override DataFlow::Node getHeaderArg() { none() }
@@ -248,7 +253,8 @@ private module Twisted {
* See https://twistedmatrix.com/documents/21.2.0/api/twisted.web.http.Request.html#cookies
*/
class TwistedRequestCookiesAppendCall extends Http::Server::CookieWrite::Range,
DataFlow::MethodCallNode {
DataFlow::MethodCallNode
{
TwistedRequestCookiesAppendCall() {
exists(DataFlow::AttrRead cookiesLookup |
cookiesLookup.getObject() = Twisted::Request::instance() and

3
python/ql/lib/semmle/python/frameworks/Werkzeug.qll
View File

@@ -83,7 +83,8 @@ module Werkzeug {
// possible to do storage.read() instead of the long form storage.stream.read(). So
// that's why InstanceSource also extends `Stdlib::FileLikeObject::InstanceSource`
abstract class InstanceSource extends Stdlib::FileLikeObject::InstanceSource,
DataFlow::LocalSourceNode { }
DataFlow::LocalSourceNode
{ }
/** Gets a reference to an instance of `werkzeug.datastructures.FileStorage`. */
private DataFlow::TypeTrackingNode instance(DataFlow::TypeTracker t) {

6
python/ql/lib/semmle/python/regex.qll
View File

@@ -140,7 +140,11 @@ string mode_from_node(DataFlow::Node node) { node = re_flag_tracker(result) }
/** A StrConst used as a regular expression */
abstract class RegexString extends Expr {
RegexString() { (this instanceof Bytes or this instanceof Unicode) }
RegexString() {
(this instanceof Bytes or this instanceof Unicode) and
// is part of the user code
exists(this.getLocation().getFile().getRelativePath())
}
/**
* Helper predicate for `char_set_start(int start, int end)`.

159
python/ql/lib/semmle/python/security/dataflow/UnsafeShellCommandConstructionCustomizations.qll Normal file
View File

@@ -0,0 +1,159 @@
/**
* Provides default sources, sinks and sanitizers for reasoning about
* shell command constructed from library input vulnerabilities, as
* well as extension points for adding your own.
*/
private import python
private import semmle.python.dataflow.new.DataFlow
private import semmle.python.dataflow.new.TaintTracking
private import CommandInjectionCustomizations::CommandInjection as CommandInjection
private import semmle.python.Concepts as Concepts
/**
* Module containing sources, sinks, and sanitizers for shell command constructed from library input.
*/
module UnsafeShellCommandConstruction {
/** A source for shell command constructed from library input vulnerabilities. */
abstract class Source extends DataFlow::Node { }
private import semmle.python.frameworks.Setuptools
/** An input parameter to a gem seen as a source. */
private class LibraryInputAsSource extends Source instanceof DataFlow::ParameterNode {
LibraryInputAsSource() {
this = Setuptools::getALibraryInput() and
not this.getParameter().getName().matches(["cmd%", "command%", "%_command", "%_cmd"])
}
}
/** A sink for shell command constructed from library input vulnerabilities. */
abstract class Sink extends DataFlow::Node {
Sink() { not this.asExpr() instanceof StrConst } // filter out string constants, makes testing easier
/** Gets a description of how the string in this sink was constructed. */
abstract string describe();
/** Gets the dataflow node where the string is constructed. */
DataFlow::Node getStringConstruction() { result = this }
/** Gets the dataflow node that executed the string as a shell command. */
abstract DataFlow::Node getCommandExecution();
}
/** Holds if the string constructed at `source` is executed at `shellExec` */
predicate isUsedAsShellCommand(DataFlow::Node source, Concepts::SystemCommandExecution shellExec) {
source = backtrackShellExec(TypeTracker::TypeBackTracker::end(), shellExec)
}
import semmle.python.dataflow.new.TypeTracker as TypeTracker
private DataFlow::LocalSourceNode backtrackShellExec(
TypeTracker::TypeBackTracker t, Concepts::SystemCommandExecution shellExec
) {
t.start() and
result = any(DataFlow::Node n | shellExec.isShellInterpreted(n)).getALocalSource()
or
exists(TypeTracker::TypeBackTracker t2 |
result = backtrackShellExec(t2, shellExec).backtrack(t2, t)
)
}
/**
* A string constructed from a string-literal (e.g. `f'foo {sink}'`),
* where the resulting string ends up being executed as a shell command.
*/
class StringInterpolationAsSink extends Sink {
Concepts::SystemCommandExecution s;
Fstring fstring;
StringInterpolationAsSink() {
isUsedAsShellCommand(DataFlow::exprNode(fstring), s) and
this.asExpr() = fstring.getASubExpression()
}
override string describe() { result = "f-string" }
override DataFlow::Node getCommandExecution() { result = s }
override DataFlow::Node getStringConstruction() { result.asExpr() = fstring }
}
/**
* A component of a string-concatenation (e.g. `"foo " + sink`),
* where the resulting string ends up being executed as a shell command.
*/
class StringConcatAsSink extends Sink {
Concepts::SystemCommandExecution s;
BinaryExpr add;
StringConcatAsSink() {
add.getOp() instanceof Add and
isUsedAsShellCommand(any(DataFlow::Node n | n.asExpr() = add), s) and
this.asExpr() = add.getASubExpression()
}
override DataFlow::Node getCommandExecution() { result = s }
override string describe() { result = "string concatenation" }
override DataFlow::Node getStringConstruction() { result.asExpr() = add }
}
/**
* A string constructed using a `" ".join(...)` call, where the resulting string ends up being executed as a shell command.
*/
class ArrayJoin extends Sink {
Concepts::SystemCommandExecution s;
DataFlow::MethodCallNode call;
ArrayJoin() {
call.getMethodName() = "join" and
unique( | | call.getArg(_)).asExpr().(Str).getText() = " " and
isUsedAsShellCommand(call, s) and
(
this = call.getArg(0) and
not call.getArg(0).asExpr() instanceof List
or
this.asExpr() = call.getArg(0).asExpr().(List).getASubExpression()
)
}
override string describe() { result = "array" }
override DataFlow::Node getCommandExecution() { result = s }
override DataFlow::Node getStringConstruction() { result = call }
}
/**
* A string constructed from a format call,
* where the resulting string ends up being executed as a shell command.
* Either a call to `.format(..)` or a string-interpolation with a `%` operator.
*/
class TaintedFormatStringAsSink extends Sink {
Concepts::SystemCommandExecution s;
DataFlow::Node formatCall;
TaintedFormatStringAsSink() {
(
formatCall.asExpr().(BinaryExpr).getOp() instanceof Mod and
this.asExpr() = formatCall.asExpr().(BinaryExpr).getASubExpression()
or
formatCall.(DataFlow::MethodCallNode).getMethodName() = "format" and
this =
[
formatCall.(DataFlow::MethodCallNode).getArg(_),
formatCall.(DataFlow::MethodCallNode).getObject()
]
) and
isUsedAsShellCommand(formatCall, s)
}
override string describe() { result = "formatted string" }
override DataFlow::Node getCommandExecution() { result = s }
override DataFlow::Node getStringConstruction() { result = formatCall }
}
}

34
python/ql/lib/semmle/python/security/dataflow/UnsafeShellCommandConstructionQuery.qll Normal file
View File

@@ -0,0 +1,34 @@
/**
* Provides a taint tracking configuration for reasoning about shell command
* constructed from library input vulnerabilities
*
* Note, for performance reasons: only import this file if `Configuration` is needed,
* otherwise `UnsafeShellCommandConstructionCustomizations` should be imported instead.
*/
import python
import semmle.python.dataflow.new.DataFlow
import UnsafeShellCommandConstructionCustomizations::UnsafeShellCommandConstruction
private import semmle.python.dataflow.new.TaintTracking
private import CommandInjectionCustomizations::CommandInjection as CommandInjection
private import semmle.python.dataflow.new.BarrierGuards
/**
* A taint-tracking configuration for detecting shell command constructed from library input vulnerabilities.
*/
class Configuration extends TaintTracking::Configuration {
Configuration() { this = "UnsafeShellCommandConstruction" }
override predicate isSource(DataFlow::Node source) { source instanceof Source }
override predicate isSink(DataFlow::Node sink) { sink instanceof Sink }
override predicate isSanitizer(DataFlow::Node node) {
node instanceof CommandInjection::Sanitizer // using all sanitizers from `rb/command-injection`
}
// override to require the path doesn't have unmatched return steps
override DataFlow::FlowFeature getAFeature() {
result instanceof DataFlow::FeatureHasSourceCallContext
}
}

14
python/ql/lib/semmle/python/xml/XML.qll
View File

@@ -108,20 +108,6 @@ class XmlFile extends XmlParent, File {
/** Gets the name of this XML file. */
override string getName() { result = File.super.getAbsolutePath() }
/**
* DEPRECATED: Use `getAbsolutePath()` instead.
*
* Gets the path of this XML file.
*/
deprecated string getPath() { result = this.getAbsolutePath() }
/**
* DEPRECATED: Use `getParentContainer().getAbsolutePath()` instead.
*
* Gets the path of the folder that contains this XML file.
*/
deprecated string getFolder() { result = this.getParentContainer().getAbsolutePath() }
/** Gets the encoding of this XML file. */
string getEncoding() { xmlEncoding(this, result) }

10
python/ql/src/CHANGELOG.md
View File

@@ -1,3 +1,13 @@
## 0.6.5
### New Queries
* Added a new query, `py/shell-command-constructed-from-input`, to detect libraries that unsafely construct shell commands from their inputs.
## 0.6.4
No user-facing changes.
## 0.6.3
No user-facing changes.

73
python/ql/src/Security/CWE-078/UnsafeShellCommandConstruction.qhelp Normal file
View File

@@ -0,0 +1,73 @@
<!DOCTYPE qhelp PUBLIC
"-//Semmle//qhelp//EN"
"qhelp.dtd">
<qhelp>
<overview>
<p>
Dynamically constructing a shell command with inputs from library
functions may inadvertently change the meaning of the shell command.
Clients using the exported function may use inputs containing
characters that the shell interprets in a special way, for instance
quotes and spaces.
This can result in the shell command misbehaving, or even
allowing a malicious user to execute arbitrary commands on the system.
</p>
</overview>
<recommendation>
<p>
If possible, provide the dynamic arguments to the shell as an array
to APIs such as <code>subprocess.run</code> to avoid interpretation by the shell.
</p>
<p>
Alternatively, if the shell command must be constructed
dynamically, then add code to ensure that special characters
do not alter the shell command unexpectedly.
</p>
</recommendation>
<example>
<p>
The following example shows a dynamically constructed shell
command that downloads a file from a remote URL.
</p>
<sample src="examples/unsafe-shell-command-construction.py" />
<p>
The shell command will, however, fail to work as intended if the
input contains spaces or other special characters interpreted in a
special way by the shell.
</p>
<p>
Even worse, a client might pass in user-controlled
data, not knowing that the input is interpreted as a shell command.
This could allow a malicious user to provide the input <code>http://example.org; cat /etc/passwd</code>
in order to execute the command <code>cat /etc/passwd</code>.
</p>
<p>
To avoid such potentially catastrophic behaviors, provide the
input from library functions as an argument that does not
get interpreted by a shell:
</p>
<sample src="examples/unsafe-shell-command-construction_fixed.py" />
</example>
<references>
<li>
OWASP:
<a href="https://www.owasp.org/index.php/Command_Injection">Command Injection</a>.
</li>
</references>
</qhelp>

27
python/ql/src/Security/CWE-078/UnsafeShellCommandConstruction.ql Normal file
View File

@@ -0,0 +1,27 @@
/**
* @name Unsafe shell command constructed from library input
* @description Using externally controlled strings in a command line may allow a malicious
* user to change the meaning of the command.
* @kind path-problem
* @problem.severity error
* @security-severity 6.3
* @precision medium
* @id py/shell-command-constructed-from-input
* @tags correctness
* security
* external/cwe/cwe-078
* external/cwe/cwe-088
* external/cwe/cwe-073
*/
import python
import semmle.python.security.dataflow.UnsafeShellCommandConstructionQuery
import DataFlow::PathGraph
from Configuration config, DataFlow::PathNode source, DataFlow::PathNode sink, Sink sinkNode
where
config.hasFlowPath(source, sink) and
sinkNode = sink.getNode()
select sinkNode.getStringConstruction(), source, sink,
"This " + sinkNode.describe() + " which depends on $@ is later used in a $@.", source.getNode(),
"library input", sinkNode.getCommandExecution(), "shell command"

4
python/ql/src/Security/CWE-078/examples/unsafe-shell-command-construction.py Normal file
View File

@@ -0,0 +1,4 @@
import os
def download(path):
os.system("wget " + path) # NOT OK

4
python/ql/src/Security/CWE-078/examples/unsafe-shell-command-construction_fixed.py Normal file
View File

@@ -0,0 +1,4 @@
import subprocess
def download(path):
subprocess.run(["wget", path]) # OK

4
python/ql/src/Security/CWE-327/FluentApiModel.qll
View File

@@ -112,7 +112,7 @@ module InsecureContextConfiguration implements DataFlow::StateConfigSig {
}
}
private module InsecureContextFlow = DataFlow::MakeWithState<InsecureContextConfiguration>;
private module InsecureContextFlow = DataFlow::GlobalWithState<InsecureContextConfiguration>;
/**
* Holds if `conectionCreation` marks the creation of a connection based on the contex
@@ -127,7 +127,7 @@ predicate unsafe_connection_creation_with_context(
) {
// Connection created from a context allowing `insecure_version`.
exists(InsecureContextFlow::PathNode src, InsecureContextFlow::PathNode sink |
InsecureContextFlow::hasFlowPath(src, sink) and
InsecureContextFlow::flowPath(src, sink) and
src.getNode() = contextOrigin and
sink.getNode() = connectionCreation and
sink.getState().allowsInsecureVersion(insecure_version) and

4
python/ql/src/Variables/UninitializedLocal.ql
View File

@@ -15,7 +15,9 @@ import Undefined
import semmle.python.pointsto.PointsTo
predicate uninitialized_local(NameNode use) {
exists(FastLocalVariable local | use.uses(local) or use.deletes(local) | not local.escapes()) and
exists(FastLocalVariable local | use.uses(local) or use.deletes(local) |
not local.escapes() and not local = any(Nonlocal nl).getAVariable()
) and
(
any(Uninitialized uninit).taints(use) and
PointsToInternal::reachableBlock(use.getBasicBlock(), _)

4
python/ql/src/change-notes/2023-03-20-uninitialized-local.md Normal file
View File

@@ -0,0 +1,4 @@
---
category: fix
---
* Nonlocal variables are excluded from alerts.

3
python/ql/src/change-notes/released/0.6.4.md Normal file
View File

@@ -0,0 +1,3 @@
## 0.6.4
No user-facing changes.

5
python/ql/src/change-notes/released/0.6.5.md Normal file
View File

@@ -0,0 +1,5 @@
## 0.6.5
### New Queries
* Added a new query, `py/shell-command-constructed-from-input`, to detect libraries that unsafely construct shell commands from their inputs.

2
python/ql/src/codeql-pack.release.yml
View File

@@ -1,2 +1,2 @@
---
lastReleaseVersion: 0.6.3
lastReleaseVersion: 0.6.5

9
python/ql/src/experimental/Security/CWE-348/ClientSuppliedIpUsedInSecurityCheckLib.qll
View File

@@ -16,7 +16,8 @@ private import semmle.python.frameworks.Tornado
abstract class ClientSuppliedIpUsedInSecurityCheck extends DataFlow::Node { }
private class FlaskClientSuppliedIpUsedInSecurityCheck extends ClientSuppliedIpUsedInSecurityCheck,
DataFlow::MethodCallNode {
DataFlow::MethodCallNode
{
FlaskClientSuppliedIpUsedInSecurityCheck() {
this = Flask::request().getMember("headers").getMember(["get", "get_all", "getlist"]).getACall() and
this.getArg(0).asExpr().(StrConst).getText().toLowerCase() = clientIpParameterName()
@@ -24,7 +25,8 @@ private class FlaskClientSuppliedIpUsedInSecurityCheck extends ClientSuppliedIpU
}
private class DjangoClientSuppliedIpUsedInSecurityCheck extends ClientSuppliedIpUsedInSecurityCheck,
DataFlow::MethodCallNode {
DataFlow::MethodCallNode
{
DjangoClientSuppliedIpUsedInSecurityCheck() {
exists(DataFlow::Node req, DataFlow::AttrRead headers |
// a call to request.headers.get or request.META.get
@@ -38,7 +40,8 @@ private class DjangoClientSuppliedIpUsedInSecurityCheck extends ClientSuppliedIp
}
private class TornadoClientSuppliedIpUsedInSecurityCheck extends ClientSuppliedIpUsedInSecurityCheck,
DataFlow::MethodCallNode {
DataFlow::MethodCallNode
{
TornadoClientSuppliedIpUsedInSecurityCheck() {
// a call to self.request.headers.get or self.request.headers.get_list inside a tornado requesthandler
exists(

3
python/ql/src/qlpack.yml
View File

@@ -1,12 +1,11 @@
name: codeql/python-queries
version: 0.6.4-dev
version: 0.6.6-dev
groups:
- python
- queries
dependencies:
codeql/python-all: ${workspace}
codeql/suite-helpers: ${workspace}
codeql/util: ${workspace}
suites: codeql-suites
extractor: python
defaultSuiteFile: codeql-suites/python-code-scanning.qls

13
python/ql/test/experimental/dataflow/TestUtil/DataFlowConsistency.qll
View File

@@ -20,15 +20,6 @@ private class MyConsistencyConfiguration extends ConsistencyConfiguration {
n instanceof SynthDictSplatParameterNode
}
override predicate uniqueParameterNodeAtPositionExclude(
DataFlowCallable c, ParameterPosition pos, Node p
) {
// TODO: This can be removed once we solve the overlap of dictionary splat parameters
c.getParameter(pos) = p and
pos.isDictSplat() and
not exists(p.getLocation().getFile().getRelativePath())
}
override predicate uniqueParameterNodePositionExclude(
DataFlowCallable c, ParameterPosition pos, Node p
) {
@@ -44,4 +35,8 @@ private class MyConsistencyConfiguration extends ConsistencyConfiguration {
param = func.getArgByName(_)
)
}
override predicate uniqueCallEnclosingCallableExclude(DataFlowCall call) {
not exists(call.getLocation().getFile().getRelativePath())
}
}

1
python/ql/test/experimental/dataflow/basic/dataflow-consistency.expected
View File

@@ -1,4 +1,5 @@
uniqueEnclosingCallable
uniqueCallEnclosingCallable
uniqueType
uniqueNodeLocation
missingLocation

1
python/ql/test/experimental/dataflow/callgraph_crosstalk/dataflow-consistency.expected
View File

@@ -1,4 +1,5 @@
uniqueEnclosingCallable
uniqueCallEnclosingCallable
uniqueType
uniqueNodeLocation
missingLocation

5
python/ql/test/experimental/dataflow/calls/dataflow-consistency.expected
View File

@@ -1,4 +1,9 @@
uniqueEnclosingCallable
uniqueCallEnclosingCallable
| new_cls_param.py:14:6:14:16 | classmethod() | Call should have one enclosing callable but has 0. |
| test.py:21:6:21:17 | staticmethod() | Call should have one enclosing callable but has 0. |
| test.py:25:6:25:16 | classmethod() | Call should have one enclosing callable but has 0. |
| test.py:29:6:29:16 | classmethod() | Call should have one enclosing callable but has 0. |
uniqueType
uniqueNodeLocation
missingLocation

1
python/ql/test/experimental/dataflow/consistency/dataflow-consistency.expected
View File

@@ -1,4 +1,5 @@
uniqueEnclosingCallable
uniqueCallEnclosingCallable
uniqueType
uniqueNodeLocation
missingLocation

0
python/ql/test/experimental/dataflow/coverage-py2/argumentRoutingTest.expected Normal file
View File

1
python/ql/test/experimental/dataflow/coverage-py2/argumentRoutingTest.qlref Normal file
View File

@@ -0,0 +1 @@
../coverage/argumentRoutingTest.ql

54
python/ql/test/experimental/dataflow/coverage-py2/classes.py Normal file
View File

@@ -0,0 +1,54 @@
# Python 2 specific tests, like the one in coverage/classes.py
#
# User-defined methods, both instance methods and class methods, can be called in many non-standard ways
# i.e. differently from simply `c.f()` or `C.f()`. For example, a user-defined `__await__` method on a
# class `C` will be called by the syntactic construct `await c` when `c` is an instance of `C`.
#
# These tests should cover all the class calls that we hope to support.
# It is based on https://docs.python.org/3/reference/datamodel.html, and headings refer there.
#
# All functions starting with "test_" should run and execute `print("OK")` exactly once.
# This can be checked by running validTest.py.
import sys
import os
sys.path.append(os.path.dirname(os.path.dirname((__file__))))
from testlib import expects
def SINK1(x):
pass
def SINK2(x):
pass
def SINK3(x):
pass
def SINK4(x):
pass
def OK():
print("OK")
# 3.3.8. Emulating numeric types
# object.__index__(self)
class With_index:
def __index__(self):
SINK1(self)
OK() # Call not found
return 0
def test_index():
import operator
with_index = With_index() #$ MISSING: arg1="SSA variable with_index" func=With_index.__index__
operator.index(with_index)

1
python/ql/test/experimental/dataflow/coverage-py2/options Normal file
View File

@@ -0,0 +1 @@
semmle-extractor-options: --max-import-depth=1 --lang=2

0
python/ql/test/experimental/dataflow/coverage-py3/argumentRoutingTest.expected Normal file
View File

1
python/ql/test/experimental/dataflow/coverage-py3/argumentRoutingTest.qlref Normal file
View File

@@ -0,0 +1 @@
../coverage/argumentRoutingTest.ql

72
python/ql/test/experimental/dataflow/coverage-py3/classes.py Normal file
View File

@@ -0,0 +1,72 @@
# Python 3 specific tests, like the one in coverage/classes.py
#
# User-defined methods, both instance methods and class methods, can be called in many non-standard ways
# i.e. differently from simply `c.f()` or `C.f()`. For example, a user-defined `__await__` method on a
# class `C` will be called by the syntactic construct `await c` when `c` is an instance of `C`.
#
# These tests should cover all the class calls that we hope to support.
# It is based on https://docs.python.org/3/reference/datamodel.html, and headings refer there.
#
# All functions starting with "test_" should run and execute `print("OK")` exactly once.
# This can be checked by running validTest.py.
import sys
import os
sys.path.append(os.path.dirname(os.path.dirname((__file__))))
from testlib import expects
def SINK1(x):
pass
def SINK2(x):
pass
def SINK3(x):
pass
def SINK4(x):
pass
def OK():
print("OK")
# 3.3.7. Emulating container types
# object.__length_hint__(self)
class With_length_hint:
def __length_hint__(self):
SINK1(self)
OK()
return 0
def test_length_hint():
import operator
with_length_hint = With_length_hint() #$ arg1="SSA variable with_length_hint" func=With_length_hint.__length_hint__
operator.length_hint(with_length_hint)
# 3.3.8. Emulating numeric types
# object.__index__(self)
class With_index:
def __index__(self):
SINK1(self)
OK() # Call not found
return 0
def test_index():
import operator
with_index = With_index() #$ arg1="SSA variable with_index" func=With_index.__index__
operator.index(with_index)

1
python/ql/test/experimental/dataflow/coverage-py3/options Normal file
View File

@@ -0,0 +1 @@
semmle-extractor-options: --max-import-depth=1 --lang=3

12
python/ql/test/experimental/dataflow/coverage/argumentPassing.py
View File

@@ -204,6 +204,18 @@ def test_mixed():
mixed(**args)
def kwargs_same_name_as_positional_only(a, /, **kwargs):
SINK1(a)
SINK2(kwargs["a"])
@expects(2*2)
def test_kwargs_same_name_as_positional_only():
kwargs_same_name_as_positional_only(arg1, a=arg2) # $ arg1 SPURIOUS: bad1="arg2" MISSING: arg2
kwargs = {"a": arg2} # $ func=kwargs_same_name_as_positional_only SPURIOUS: bad1="arg2" MISSING: arg2
kwargs_same_name_as_positional_only(arg1, **kwargs) # $ arg1
def starargs_only(*args):
SINK1(args[0])
SINK2(args[1])

22
python/ql/test/experimental/dataflow/coverage/classes.py
View File

@@ -535,21 +535,6 @@ def test_len_if():
pass
# object.__length_hint__(self)
class With_length_hint:
def __length_hint__(self):
SINK1(self)
OK() # Call not found
return 0
def test_length_hint():
import operator
with_length_hint = With_length_hint() #$ MISSING: arg1="SSA variable with_length_hint" func=With_length_hint.__length_hint__
operator.length_hint(with_length_hint)
# object.__getitem__(self, key)
class With_getitem:
def __getitem__(self, key):
@@ -1378,13 +1363,6 @@ class With_index:
return 0
def test_index():
import operator
with_index = With_index() #$ MISSING: arg1="SSA variable with_index" func=With_index.__index__
operator.index(with_index)
def test_index_slicing():
with_index = With_index() #$ MISSING: arg1="SSA variable with_index" func=With_index.__index__
[0][with_index:1]

3
python/ql/test/experimental/dataflow/coverage/dataflow-consistency.expected
View File

@@ -1,4 +1,7 @@
uniqueEnclosingCallable
uniqueCallEnclosingCallable
| datamodel.py:71:6:71:16 | classmethod() | Call should have one enclosing callable but has 0. |
| datamodel.py:76:6:76:17 | staticmethod() | Call should have one enclosing callable but has 0. |
uniqueType
uniqueNodeLocation
missingLocation

1
python/ql/test/experimental/dataflow/exceptions/dataflow-consistency.expected
View File

@@ -1,4 +1,5 @@
uniqueEnclosingCallable
uniqueCallEnclosingCallable
uniqueType
uniqueNodeLocation
missingLocation

1
python/ql/test/experimental/dataflow/fieldflow/dataflow-consistency.expected
View File

@@ -1,4 +1,5 @@
uniqueEnclosingCallable
uniqueCallEnclosingCallable
uniqueType
uniqueNodeLocation
missingLocation

1
python/ql/test/experimental/dataflow/global-flow/dataflow-consistency.expected
View File

@@ -1,4 +1,5 @@
uniqueEnclosingCallable
uniqueCallEnclosingCallable
uniqueType
uniqueNodeLocation
missingLocation

1
python/ql/test/experimental/dataflow/match/dataflow-consistency.expected
View File

@@ -1,4 +1,5 @@
uniqueEnclosingCallable
uniqueCallEnclosingCallable
uniqueType
uniqueNodeLocation
missingLocation

1
python/ql/test/experimental/dataflow/pep_328/dataflow-consistency.expected
View File

@@ -1,4 +1,5 @@
uniqueEnclosingCallable
uniqueCallEnclosingCallable
uniqueType
uniqueNodeLocation
missingLocation

1
python/ql/test/experimental/dataflow/regression/dataflow-consistency.expected
View File

@@ -1,4 +1,5 @@
uniqueEnclosingCallable
uniqueCallEnclosingCallable
uniqueType
uniqueNodeLocation
missingLocation

1
python/ql/test/experimental/dataflow/strange-essaflow/dataflow-consistency.expected
View File

@@ -1,4 +1,5 @@
uniqueEnclosingCallable
uniqueCallEnclosingCallable
uniqueType
uniqueNodeLocation
missingLocation

1
python/ql/test/experimental/dataflow/tainttracking/basic/dataflow-consistency.expected
View File

@@ -1,4 +1,5 @@
uniqueEnclosingCallable
uniqueCallEnclosingCallable
uniqueType
uniqueNodeLocation
missingLocation

1
python/ql/test/experimental/dataflow/tainttracking/commonSanitizer/dataflow-consistency.expected
View File

@@ -1,4 +1,5 @@
uniqueEnclosingCallable
uniqueCallEnclosingCallable
uniqueType
uniqueNodeLocation
missingLocation

1
python/ql/test/experimental/dataflow/tainttracking/customSanitizer/dataflow-consistency.expected
View File

@@ -1,4 +1,5 @@
uniqueEnclosingCallable
uniqueCallEnclosingCallable
uniqueType
uniqueNodeLocation
missingLocation

1
python/ql/test/experimental/dataflow/tainttracking/defaultAdditionalTaintStep-py3/dataflow-consistency.expected
View File

@@ -1,4 +1,5 @@
uniqueEnclosingCallable
uniqueCallEnclosingCallable
uniqueType
uniqueNodeLocation
missingLocation

1
python/ql/test/experimental/dataflow/tainttracking/defaultAdditionalTaintStep/dataflow-consistency.expected
View File

@@ -1,4 +1,5 @@
uniqueEnclosingCallable
uniqueCallEnclosingCallable
uniqueType
uniqueNodeLocation
missingLocation

1
python/ql/test/experimental/dataflow/tainttracking/generator-flow/dataflow-consistency.expected
View File

@@ -1,4 +1,5 @@
uniqueEnclosingCallable
uniqueCallEnclosingCallable
uniqueType
uniqueNodeLocation
missingLocation

1
python/ql/test/experimental/dataflow/tainttracking/unwanted-global-flow/dataflow-consistency.expected
View File

@@ -1,4 +1,5 @@
uniqueEnclosingCallable
uniqueCallEnclosingCallable
uniqueType
uniqueNodeLocation
missingLocation

1
python/ql/test/experimental/dataflow/typetracking/dataflow-consistency.expected
View File

@@ -1,4 +1,5 @@
uniqueEnclosingCallable
uniqueCallEnclosingCallable
uniqueType
uniqueNodeLocation
missingLocation

3
python/ql/test/experimental/dataflow/validTest.py
View File

@@ -64,9 +64,12 @@ if __name__ == "__main__":
check_tests_valid("coverage.test")
check_tests_valid("coverage.argumentPassing")
check_tests_valid("coverage.datamodel")
check_tests_valid("coverage-py2.classes")
check_tests_valid("coverage-py3.classes")
check_tests_valid("variable-capture.in")
check_tests_valid("variable-capture.nonlocal")
check_tests_valid("variable-capture.dict")
check_tests_valid("variable-capture.collections")
check_tests_valid("module-initialization.multiphase")
check_tests_valid("fieldflow.test")
check_tests_valid_after_version("match.test", (3, 10))

64
python/ql/test/experimental/dataflow/variable-capture/collections.py Normal file
View File

@@ -0,0 +1,64 @@
# Here we test the case where a captured variable is being read.
# All functions starting with "test_" should run and execute `print("OK")` exactly once.
# This can be checked by running validTest.py.
import sys
import os
sys.path.append(os.path.dirname(os.path.dirname((__file__))))
from testlib import expects
# These are defined so that we can evaluate the test code.
NONSOURCE = "not a source"
SOURCE = "source"
def is_source(x):
return x == "source" or x == b"source" or x == 42 or x == 42.0 or x == 42j
def SINK(x):
if is_source(x):
print("OK")
else:
print("Unexpected flow", x)
def SINK_F(x):
if is_source(x):
print("Unexpected flow", x)
else:
print("OK")
l = [NONSOURCE]
SINK_F(l[0])
l_mod = [SOURCE for x in l]
SINK(l_mod[0]) #$ captured
l_mod_lambda = [(lambda a : SOURCE)(x) for x in l]
SINK(l_mod_lambda[0]) #$ captured
def mod(x):
return SOURCE
l_mod_function = [mod(x) for x in l]
SINK(l_mod_function[0]) #$ captured
def mod_list(l):
def mod_local(x):
return SOURCE
return [mod_local(x) for x in l]
l_modded = mod_list(l)
SINK(l_modded[0]) #$ MISSING: captured
def mod_list_first(l):
def mod_local(x):
return SOURCE
return [mod_local(l[0])]
l_modded_first = mod_list_first(l)
SINK(l_modded_first[0]) #$ captured

6
python/ql/test/experimental/dataflow/variable-capture/dataflow-consistency.expected
View File

@@ -1,4 +1,10 @@
uniqueEnclosingCallable
uniqueCallEnclosingCallable
| collections.py:39:17:39:38 | Lambda() | Call should have one enclosing callable but has 0. |
| collections.py:39:17:39:38 | Lambda() | Call should have one enclosing callable but has 0. |
| collections.py:45:19:45:24 | mod() | Call should have one enclosing callable but has 0. |
| collections.py:45:19:45:24 | mod() | Call should have one enclosing callable but has 0. |
| collections.py:52:13:52:24 | mod_local() | Call should have one enclosing callable but has 0. |
uniqueType
uniqueNodeLocation
missingLocation

160
python/ql/test/experimental/import-resolution/ModuleExport.expected Normal file
View File

@@ -0,0 +1,160 @@
| attr_clash.__init__ | __file__ | attr_clash/__init__.py:6:6:6:13 | ControlFlowNode for __file__ |
| attr_clash.__init__ | __name__ | attr_clash/__init__.py:0:0:0:0 | GSSA Variable __name__ |
| attr_clash.__init__ | __package__ | attr_clash/__init__.py:0:0:0:0 | GSSA Variable __package__ |
| attr_clash.__init__ | clashing_attr | attr_clash/__init__.py:4:1:4:13 | GSSA Variable clashing_attr |
| attr_clash.__init__ | enter | attr_clash/__init__.py:2:1:2:5 | ControlFlowNode for enter |
| attr_clash.__init__ | exit | attr_clash/__init__.py:6:1:6:4 | ControlFlowNode for exit |
| attr_clash.clashing_attr | __file__ | attr_clash/clashing_attr.py:4:6:4:13 | ControlFlowNode for __file__ |
| attr_clash.clashing_attr | __name__ | attr_clash/clashing_attr.py:0:0:0:0 | GSSA Variable __name__ |
| attr_clash.clashing_attr | __package__ | attr_clash/clashing_attr.py:0:0:0:0 | GSSA Variable __package__ |
| attr_clash.clashing_attr | enter | attr_clash/clashing_attr.py:2:1:2:5 | ControlFlowNode for enter |
| attr_clash.clashing_attr | exit | attr_clash/clashing_attr.py:4:1:4:4 | ControlFlowNode for exit |
| attr_clash.non_clashing_submodule | __file__ | attr_clash/non_clashing_submodule.py:4:6:4:13 | ControlFlowNode for __file__ |
| attr_clash.non_clashing_submodule | __name__ | attr_clash/non_clashing_submodule.py:0:0:0:0 | GSSA Variable __name__ |
| attr_clash.non_clashing_submodule | __package__ | attr_clash/non_clashing_submodule.py:0:0:0:0 | GSSA Variable __package__ |
| attr_clash.non_clashing_submodule | enter | attr_clash/non_clashing_submodule.py:2:1:2:5 | ControlFlowNode for enter |
| attr_clash.non_clashing_submodule | exit | attr_clash/non_clashing_submodule.py:4:1:4:4 | ControlFlowNode for exit |
| bar | __file__ | bar.py:6:6:6:13 | ControlFlowNode for __file__ |
| bar | __name__ | bar.py:0:0:0:0 | GSSA Variable __name__ |
| bar | __package__ | bar.py:0:0:0:0 | GSSA Variable __package__ |
| bar | bar_attr | bar.py:4:1:4:8 | GSSA Variable bar_attr |
| bar | enter | bar.py:2:1:2:5 | ControlFlowNode for enter |
| bar | exit | bar.py:6:1:6:4 | ControlFlowNode for exit |
| baz | __file__ | baz.py:6:6:6:13 | ControlFlowNode for __file__ |
| baz | __name__ | baz.py:0:0:0:0 | GSSA Variable __name__ |
| baz | __package__ | baz.py:0:0:0:0 | GSSA Variable __package__ |
| baz | baz_attr | baz.py:4:1:4:8 | GSSA Variable baz_attr |
| baz | enter | baz.py:2:1:2:5 | ControlFlowNode for enter |
| baz | exit | baz.py:6:1:6:4 | ControlFlowNode for exit |
| block_flow_check | SOURCE | block_flow_check.py:12:25:12:30 | ControlFlowNode for SOURCE |
| block_flow_check | __file__ | block_flow_check.py:14:6:14:13 | ControlFlowNode for __file__ |
| block_flow_check | __name__ | block_flow_check.py:0:0:0:0 | GSSA Variable __name__ |
| block_flow_check | __package__ | block_flow_check.py:0:0:0:0 | GSSA Variable __package__ |
| block_flow_check | check | block_flow_check.py:12:1:12:5 | ControlFlowNode for check |
| block_flow_check | enter | block_flow_check.py:2:1:2:5 | ControlFlowNode for enter |
| block_flow_check | exit | block_flow_check.py:14:1:14:4 | ControlFlowNode for exit |
| block_flow_check | globals | block_flow_check.py:12:33:12:39 | ControlFlowNode for globals |
| block_flow_check | object | block_flow_check.py:4:14:4:19 | ControlFlowNode for object |
| block_flow_check | staticmethod | block_flow_check.py:0:0:0:0 | GSSA Variable staticmethod |
| foo | __file__ | foo.py:14:6:14:13 | ControlFlowNode for __file__ |
| foo | __name__ | foo.py:0:0:0:0 | GSSA Variable __name__ |
| foo | __package__ | foo.py:0:0:0:0 | GSSA Variable __package__ |
| foo | __private_foo_attr | foo.py:8:1:8:18 | GSSA Variable __private_foo_attr |
| foo | bar_reexported | foo.py:11:8:11:10 | ControlFlowNode for ImportExpr |
| foo | bar_reexported | foo.py:12:34:12:47 | ControlFlowNode for bar_reexported |
| foo | check | foo.py:12:1:12:5 | ControlFlowNode for check |
| foo | enter | foo.py:2:1:2:5 | ControlFlowNode for enter |
| foo | exit | foo.py:14:1:14:4 | ControlFlowNode for exit |
| foo | foo_attr | foo.py:5:1:5:8 | GSSA Variable foo_attr |
| foo | globals | foo.py:12:71:12:77 | ControlFlowNode for globals |
| generous_export | Exception | generous_export.py:16:11:16:19 | ControlFlowNode for Exception |
| generous_export | SOURCE | generous_export.py:15:11:15:16 | ControlFlowNode for SOURCE |
| generous_export | SOURCE | generous_export.py:20:25:20:30 | ControlFlowNode for SOURCE |
| generous_export | __file__ | generous_export.py:22:6:22:13 | ControlFlowNode for __file__ |
| generous_export | __name__ | generous_export.py:0:0:0:0 | GSSA Variable __name__ |
| generous_export | __package__ | generous_export.py:0:0:0:0 | GSSA Variable __package__ |
| generous_export | check | generous_export.py:20:1:20:5 | ControlFlowNode for check |
| generous_export | enter | generous_export.py:2:1:2:5 | ControlFlowNode for enter |
| generous_export | eval | generous_export.py:10:4:10:7 | ControlFlowNode for eval |
| generous_export | exit | generous_export.py:22:1:22:4 | ControlFlowNode for exit |
| generous_export | globals | generous_export.py:20:33:20:39 | ControlFlowNode for globals |
| generous_export | object | generous_export.py:4:14:4:19 | ControlFlowNode for object |
| generous_export | print | generous_export.py:15:5:15:9 | ControlFlowNode for print |
| generous_export | staticmethod | generous_export.py:0:0:0:0 | GSSA Variable staticmethod |
| has_defined_all | __all__ | has_defined_all.py:7:1:7:7 | GSSA Variable __all__ |
| has_defined_all | __file__ | has_defined_all.py:9:6:9:13 | ControlFlowNode for __file__ |
| has_defined_all | __name__ | has_defined_all.py:0:0:0:0 | GSSA Variable __name__ |
| has_defined_all | __package__ | has_defined_all.py:0:0:0:0 | GSSA Variable __package__ |
| has_defined_all | all_defined_bar | has_defined_all.py:5:1:5:15 | GSSA Variable all_defined_bar |
| has_defined_all | all_defined_foo | has_defined_all.py:4:1:4:15 | GSSA Variable all_defined_foo |
| has_defined_all | enter | has_defined_all.py:2:1:2:5 | ControlFlowNode for enter |
| has_defined_all | exit | has_defined_all.py:9:1:9:4 | ControlFlowNode for exit |
| has_defined_all_copy | __all__ | has_defined_all_copy.py:9:1:9:7 | GSSA Variable __all__ |
| has_defined_all_copy | __file__ | has_defined_all_copy.py:11:6:11:13 | ControlFlowNode for __file__ |
| has_defined_all_copy | __name__ | has_defined_all_copy.py:0:0:0:0 | GSSA Variable __name__ |
| has_defined_all_copy | __package__ | has_defined_all_copy.py:0:0:0:0 | GSSA Variable __package__ |
| has_defined_all_copy | all_defined_bar_copy | has_defined_all_copy.py:7:1:7:20 | GSSA Variable all_defined_bar_copy |
| has_defined_all_copy | all_defined_foo_copy | has_defined_all_copy.py:6:1:6:20 | GSSA Variable all_defined_foo_copy |
| has_defined_all_copy | enter | has_defined_all_copy.py:4:1:4:5 | ControlFlowNode for enter |
| has_defined_all_copy | exit | has_defined_all_copy.py:11:1:11:4 | ControlFlowNode for exit |
| has_defined_all_indirection | __file__ | has_defined_all_indirection.py:6:6:6:13 | ControlFlowNode for __file__ |
| has_defined_all_indirection | __name__ | has_defined_all_indirection.py:0:0:0:0 | GSSA Variable __name__ |
| has_defined_all_indirection | __package__ | has_defined_all_indirection.py:0:0:0:0 | GSSA Variable __package__ |
| has_defined_all_indirection | all_defined_foo_copy | has_defined_all_copy.py:6:1:6:20 | GSSA Variable all_defined_foo_copy |
| has_defined_all_indirection | enter | has_defined_all_indirection.py:2:1:2:5 | ControlFlowNode for enter |
| has_defined_all_indirection | exit | has_defined_all_indirection.py:6:1:6:4 | ControlFlowNode for exit |
| if_then_else | __file__ | if_then_else.py:16:6:16:13 | ControlFlowNode for __file__ |
| if_then_else | __name__ | if_then_else.py:0:0:0:0 | GSSA Variable __name__ |
| if_then_else | __package__ | if_then_else.py:0:0:0:0 | GSSA Variable __package__ |
| if_then_else | enter | if_then_else.py:2:1:2:5 | ControlFlowNode for enter |
| if_then_else | eval | if_then_else.py:11:8:11:11 | ControlFlowNode for eval |
| if_then_else | exit | if_then_else.py:16:1:16:4 | ControlFlowNode for exit |
| if_then_else | if_then_else_defined | if_then_else.py:7:5:7:24 | GSSA Variable if_then_else_defined |
| if_then_else | if_then_else_defined | if_then_else.py:12:9:12:28 | GSSA Variable if_then_else_defined |
| if_then_else | if_then_else_defined | if_then_else.py:14:9:14:28 | GSSA Variable if_then_else_defined |
| if_then_else_refined | SOURCE | if_then_else_refined.py:11:11:11:16 | ControlFlowNode for SOURCE |
| if_then_else_refined | SOURCE | if_then_else_refined.py:13:11:13:16 | ControlFlowNode for SOURCE |
| if_then_else_refined | __file__ | if_then_else_refined.py:19:6:19:13 | ControlFlowNode for __file__ |
| if_then_else_refined | __name__ | if_then_else_refined.py:0:0:0:0 | GSSA Variable __name__ |
| if_then_else_refined | __package__ | if_then_else_refined.py:0:0:0:0 | GSSA Variable __package__ |
| if_then_else_refined | check | if_then_else_refined.py:17:1:17:5 | ControlFlowNode for check |
| if_then_else_refined | enter | if_then_else_refined.py:4:1:4:5 | ControlFlowNode for enter |
| if_then_else_refined | eval | if_then_else_refined.py:10:4:10:7 | ControlFlowNode for eval |
| if_then_else_refined | exit | if_then_else_refined.py:19:1:19:4 | ControlFlowNode for exit |
| if_then_else_refined | globals | if_then_else_refined.py:17:24:17:30 | ControlFlowNode for globals |
| if_then_else_refined | src | if_then_else_refined.py:17:19:17:21 | ControlFlowNode for src |
| package.__init__ | __file__ | package/__init__.py:7:6:7:13 | ControlFlowNode for __file__ |
| package.__init__ | __name__ | package/__init__.py:0:0:0:0 | GSSA Variable __name__ |
| package.__init__ | __package__ | package/__init__.py:0:0:0:0 | GSSA Variable __package__ |
| package.__init__ | attr_used_in_subpackage | package/__init__.py:4:1:4:23 | GSSA Variable attr_used_in_subpackage |
| package.__init__ | enter | package/__init__.py:2:1:2:5 | ControlFlowNode for enter |
| package.__init__ | exit | package/__init__.py:7:1:7:4 | ControlFlowNode for exit |
| package.__init__ | package_attr | package/__init__.py:5:1:5:12 | GSSA Variable package_attr |
| package.subpackage2.__init__ | __file__ | package/subpackage2/__init__.py:6:6:6:13 | ControlFlowNode for __file__ |
| package.subpackage2.__init__ | __name__ | package/subpackage2/__init__.py:0:0:0:0 | GSSA Variable __name__ |
| package.subpackage2.__init__ | __package__ | package/subpackage2/__init__.py:0:0:0:0 | GSSA Variable __package__ |
| package.subpackage2.__init__ | enter | package/subpackage2/__init__.py:2:1:2:5 | ControlFlowNode for enter |
| package.subpackage2.__init__ | exit | package/subpackage2/__init__.py:6:1:6:4 | ControlFlowNode for exit |
| package.subpackage2.__init__ | subpackage2_attr | package/subpackage2/__init__.py:4:1:4:16 | GSSA Variable subpackage2_attr |
| package.subpackage.__init__ | __file__ | package/subpackage/__init__.py:14:6:14:13 | ControlFlowNode for __file__ |
| package.subpackage.__init__ | __name__ | package/subpackage/__init__.py:0:0:0:0 | GSSA Variable __name__ |
| package.subpackage.__init__ | __package__ | package/subpackage/__init__.py:0:0:0:0 | GSSA Variable __package__ |
| package.subpackage.__init__ | check | package/subpackage/__init__.py:12:1:12:5 | ControlFlowNode for check |
| package.subpackage.__init__ | enter | package/subpackage/__init__.py:2:1:2:5 | ControlFlowNode for enter |
| package.subpackage.__init__ | exit | package/subpackage/__init__.py:14:1:14:4 | ControlFlowNode for exit |
| package.subpackage.__init__ | globals | package/subpackage/__init__.py:12:79:12:85 | ControlFlowNode for globals |
| package.subpackage.__init__ | imported_attr | package/subpackage/__init__.py:7:16:7:55 | ControlFlowNode for ImportMember |
| package.subpackage.__init__ | imported_attr | package/subpackage/__init__.py:8:24:8:36 | ControlFlowNode for imported_attr |
| package.subpackage.__init__ | irrelevant_attr | package/subpackage/__init__.py:11:24:11:38 | ControlFlowNode for ImportMember |
| package.subpackage.__init__ | irrelevant_attr | package/subpackage/__init__.py:11:24:11:38 | GSSA Variable irrelevant_attr |
| package.subpackage.__init__ | submodule | package/subpackage/__init__.py:12:35:12:43 | ControlFlowNode for submodule |
| package.subpackage.__init__ | subpackage_attr | package/subpackage/__init__.py:4:1:4:15 | GSSA Variable subpackage_attr |
| package.subpackage.submodule | __file__ | package/subpackage/submodule.py:7:6:7:13 | ControlFlowNode for __file__ |
| package.subpackage.submodule | __name__ | package/subpackage/submodule.py:0:0:0:0 | GSSA Variable __name__ |
| package.subpackage.submodule | __package__ | package/subpackage/submodule.py:0:0:0:0 | GSSA Variable __package__ |
| package.subpackage.submodule | enter | package/subpackage/submodule.py:2:1:2:5 | ControlFlowNode for enter |
| package.subpackage.submodule | exit | package/subpackage/submodule.py:7:1:7:4 | ControlFlowNode for exit |
| package.subpackage.submodule | irrelevant_attr | package/subpackage/submodule.py:5:1:5:15 | GSSA Variable irrelevant_attr |
| package.subpackage.submodule | submodule_attr | package/subpackage/submodule.py:4:1:4:14 | GSSA Variable submodule_attr |
| refined | SOURCE | refined.py:12:25:12:30 | ControlFlowNode for SOURCE |
| refined | __file__ | refined.py:14:6:14:13 | ControlFlowNode for __file__ |
| refined | __name__ | refined.py:0:0:0:0 | GSSA Variable __name__ |
| refined | __package__ | refined.py:0:0:0:0 | GSSA Variable __package__ |
| refined | check | refined.py:12:1:12:5 | ControlFlowNode for check |
| refined | enter | refined.py:2:1:2:5 | ControlFlowNode for enter |
| refined | exit | refined.py:14:1:14:4 | ControlFlowNode for exit |
| refined | globals | refined.py:12:33:12:39 | ControlFlowNode for globals |
| refined | object | refined.py:4:14:4:19 | ControlFlowNode for object |
| simplistic_reexport | __file__ | simplistic_reexport.py:19:6:19:13 | ControlFlowNode for __file__ |
| simplistic_reexport | __name__ | simplistic_reexport.py:0:0:0:0 | GSSA Variable __name__ |
| simplistic_reexport | __package__ | simplistic_reexport.py:0:0:0:0 | GSSA Variable __package__ |
| simplistic_reexport | bar_attr | simplistic_reexport.py:6:17:6:24 | ControlFlowNode for ImportMember |
| simplistic_reexport | bar_attr | simplistic_reexport.py:10:19:10:26 | ControlFlowNode for bar_attr |
| simplistic_reexport | baz_attr | baz.py:4:1:4:8 | GSSA Variable baz_attr |
| simplistic_reexport | baz_attr | simplistic_reexport.py:17:19:17:26 | ControlFlowNode for baz_attr |
| simplistic_reexport | check | simplistic_reexport.py:17:1:17:5 | ControlFlowNode for check |
| simplistic_reexport | enter | baz.py:2:1:2:5 | ControlFlowNode for enter |
| simplistic_reexport | enter | simplistic_reexport.py:4:1:4:5 | ControlFlowNode for enter |
| simplistic_reexport | exit | baz.py:6:1:6:4 | ControlFlowNode for exit |
| simplistic_reexport | exit | simplistic_reexport.py:19:1:19:4 | ControlFlowNode for exit |
| simplistic_reexport | globals | simplistic_reexport.py:17:44:17:50 | ControlFlowNode for globals |

17
python/ql/test/experimental/import-resolution/ModuleExport.ql Normal file
View File

@@ -0,0 +1,17 @@
import python
import semmle.python.dataflow.new.DataFlow
import semmle.python.dataflow.new.internal.ImportResolution
from Module m, string name, DataFlow::Node defn
where
ImportResolution::module_export(m, name, defn) and
exists(m.getLocation().getFile().getRelativePath()) and
not defn.getScope() = any(Module trace | trace.getName() = "trace") and
not m.getName() = "main" and
// Since we test on both Python 2 and Python 3, but `namespace_package` is not allowed
// on Python 2 because of the missing `__init__.py` files, we remove those results
// from Python 3 tests as well. One alternative is to only run these tests under
// Python 3, but that does not seems like a good solution -- we could easily miss a
// Python 2 only regression then :O
not m.getName() = "namespace_package.namespace_module"
select m.getName(), name, defn

Some files were not shown because too many files have changed in this diff Show More