hohn/codeql
1
0
Fork 0
mirror of https://github.com/github/codeql.git synced 2026-04-25 16:55:19 +02:00
Code Issues Packages Projects Releases Wiki Activity

Merge branch 'main' into amammad-python-FileSystemAccess

Browse Source
This commit is contained in:
Rasmus Wriedt Larsen
2023-11-06 11:30:09 +01:00
parent 3d8a7e0ee3 c6bc1a3f3a
commit 92b13c4259
2230 changed files with 173570 additions and 80538 deletions
Download Patch File Download Diff File Expand all files Collapse all files

31
python/ql/lib/CHANGELOG.md
View File

@@ -1,3 +1,34 @@
## 0.11.2
### Minor Analysis Improvements
* Added support for functions decorated with `contextlib.contextmanager`.
* Namespace packages in the form of regular packages with missing `__init__.py`-files are now allowed. This enables the analysis to resolve modules and functions inside such packages.
## 0.11.1
### Minor Analysis Improvements
* Added better support for API graphs when encountering `from ... import *`. For example in the code `from foo import *; Bar()`, we will now find a result for `API::moduleImport("foo").getMember("Bar").getACall()`
* Deleted the deprecated `isBarrierGuard` predicate from the dataflow library and its uses, use `isBarrier` and the `BarrierGuard` module instead.
* Deleted the deprecated `getAUse`, `getAnImmediateUse`, `getARhs`, and `getAValueReachingRhs` predicates from the `API::Node` class.
* Deleted the deprecated `fullyQualifiedToAPIGraphPath` class from `SubclassFinder.qll`, use `fullyQualifiedToApiGraphPath` instead.
* Deleted the deprecated `Paths.qll` file.
* Deleted the deprecated `semmle.python.security.performance` folder, use `semmle.python.security.regexp` instead.
* Deleted the deprecated `semmle.python.security.strings` and `semmle.python.web` folders.
* Improved modeling of decoding through pickle related functions (which can lead to code execution), resulting in additional sinks for the _Deserializing untrusted input_ query (`py/unsafe-deserialization`). Added support for `pandas.read_pickle`, `numpy.load` and `joblib.load`.
## 0.11.0
### Minor Analysis Improvements
* Django Rest Framework better handles custom `ModelViewSet` classes functions
* Regular expression fragments residing inside implicitly concatenated strings now have better location information.
### Bug Fixes
* Subterms of regular expressions encoded as single-line string literals now have better source-location information.
## 0.10.5
No user-facing changes.

4
python/ql/lib/change-notes/2023-09-22-regex-prefix.md
View File

@@ -1,4 +0,0 @@
---
category: fix
---
* Subterms of regular expressions encoded as single-line string literals now have better source-location information.

4
python/ql/lib/change-notes/2023-09-26-regex-locations-in-string-parts.md
View File

@@ -1,4 +0,0 @@
---
category: minorAnalysis
---
* Regular expression fragments residing inside implicitly concatenated strings now have better location information.

4
python/ql/lib/change-notes/2023-09-29-django-restframework-improvements.md
View File

@@ -1,4 +0,0 @@
---
category: minorAnalysis
---
* Django Rest Framework better handles custom `ModelViewSet` classes functions

10
python/ql/lib/change-notes/released/0.11.0.md Normal file
View File

@@ -0,0 +1,10 @@
## 0.11.0
### Minor Analysis Improvements
* Django Rest Framework better handles custom `ModelViewSet` classes functions
* Regular expression fragments residing inside implicitly concatenated strings now have better location information.
### Bug Fixes
* Subterms of regular expressions encoded as single-line string literals now have better source-location information.

12
python/ql/lib/change-notes/released/0.11.1.md Normal file
View File

@@ -0,0 +1,12 @@
## 0.11.1
### Minor Analysis Improvements
* Added better support for API graphs when encountering `from ... import *`. For example in the code `from foo import *; Bar()`, we will now find a result for `API::moduleImport("foo").getMember("Bar").getACall()`
* Deleted the deprecated `isBarrierGuard` predicate from the dataflow library and its uses, use `isBarrier` and the `BarrierGuard` module instead.
* Deleted the deprecated `getAUse`, `getAnImmediateUse`, `getARhs`, and `getAValueReachingRhs` predicates from the `API::Node` class.
* Deleted the deprecated `fullyQualifiedToAPIGraphPath` class from `SubclassFinder.qll`, use `fullyQualifiedToApiGraphPath` instead.
* Deleted the deprecated `Paths.qll` file.
* Deleted the deprecated `semmle.python.security.performance` folder, use `semmle.python.security.regexp` instead.
* Deleted the deprecated `semmle.python.security.strings` and `semmle.python.web` folders.
* Improved modeling of decoding through pickle related functions (which can lead to code execution), resulting in additional sinks for the _Deserializing untrusted input_ query (`py/unsafe-deserialization`). Added support for `pandas.read_pickle`, `numpy.load` and `joblib.load`.

6
python/ql/lib/change-notes/released/0.11.2.md Normal file
View File

@@ -0,0 +1,6 @@
## 0.11.2
### Minor Analysis Improvements
* Added support for functions decorated with `contextlib.contextmanager`.
* Namespace packages in the form of regular packages with missing `__init__.py`-files are now allowed. This enables the analysis to resolve modules and functions inside such packages.

2
python/ql/lib/codeql-pack.release.yml
View File

@@ -1,2 +1,2 @@
---
lastReleaseVersion: 0.10.5
lastReleaseVersion: 0.11.2

2
python/ql/lib/qlpack.yml
View File

@@ -1,5 +1,5 @@
name: codeql/python-all
version: 0.11.0-dev
version: 0.11.3-dev
groups: python
dbscheme: semmlecode.python.dbscheme
extractor: python

12
python/ql/lib/semmle/python/ApiGraphs.qll
View File

@@ -155,18 +155,6 @@ module API {
*/
DataFlow::LocalSourceNode asSource() { Impl::use(this, result) }
/** DEPRECATED. This predicate has been renamed to `getAValueReachableFromSource()`. */
deprecated DataFlow::Node getAUse() { result = this.getAValueReachableFromSource() }
/** DEPRECATED. This predicate has been renamed to `asSource()`. */
deprecated DataFlow::LocalSourceNode getAnImmediateUse() { result = this.asSource() }
/** DEPRECATED. This predicate has been renamed to `asSink()`. */
deprecated DataFlow::Node getARhs() { result = this.asSink() }
/** DEPRECATED. This predicate has been renamed to `getAValueReachingSink()`. */
deprecated DataFlow::Node getAValueReachingRhs() { result = this.getAValueReachingSink() }
/**
* Gets a call to the function represented by this API component.
*/

30
python/ql/lib/semmle/python/Flow.qll
View File

@@ -547,6 +547,31 @@ class IfExprNode extends ControlFlowNode {
override IfExp getNode() { result = super.getNode() }
}
/** A control flow node corresponding to an assignment expression such as `lhs := rhs`. */
class AssignmentExprNode extends ControlFlowNode {
AssignmentExprNode() { toAst(this) instanceof AssignExpr }
/** Gets the flow node corresponding to the left-hand side of the assignment expression */
ControlFlowNode getTarget() {
exists(AssignExpr a |
this.getNode() = a and
a.getTarget() = result.getNode() and
result.getBasicBlock().dominates(this.getBasicBlock())
)
}
/** Gets the flow node corresponding to the right-hand side of the assignment expression */
ControlFlowNode getValue() {
exists(AssignExpr a |
this.getNode() = a and
a.getValue() = result.getNode() and
result.getBasicBlock().dominates(this.getBasicBlock())
)
}
override AssignExpr getNode() { result = super.getNode() }
}
/** A control flow node corresponding to a binary expression, such as `x + y` */
class BinaryExprNode extends ControlFlowNode {
BinaryExprNode() { toAst(this) instanceof BinaryExpr }
@@ -630,6 +655,8 @@ class DefinitionNode extends ControlFlowNode {
Stages::AST::ref() and
exists(Assign a | a.getATarget().getAFlowNode() = this)
or
exists(AssignExpr a | a.getTarget().getAFlowNode() = this)
or
exists(AnnAssign a | a.getTarget().getAFlowNode() = this and exists(a.getValue()))
or
exists(Alias a | a.getAsname().getAFlowNode() = this)
@@ -787,6 +814,9 @@ private AstNode assigned_value(Expr lhs) {
/* lhs = result */
exists(Assign a | a.getATarget() = lhs and result = a.getValue())
or
/* lhs := result */
exists(AssignExpr a | a.getTarget() = lhs and result = a.getValue())
or
/* lhs : annotation = result */
exists(AnnAssign a | a.getTarget() = lhs and result = a.getValue())
or

7
python/ql/lib/semmle/python/Frameworks.qll
View File

@@ -7,8 +7,8 @@
private import semmle.python.frameworks.Aioch
private import semmle.python.frameworks.Aiohttp
private import semmle.python.frameworks.Aiomysql
private import semmle.python.frameworks.Aiosqlite
private import semmle.python.frameworks.Aiopg
private import semmle.python.frameworks.Aiosqlite
private import semmle.python.frameworks.Asyncpg
private import semmle.python.frameworks.BSon
private import semmle.python.frameworks.CassandraDriver
@@ -30,6 +30,7 @@ private import semmle.python.frameworks.Httpx
private import semmle.python.frameworks.Idna
private import semmle.python.frameworks.Invoke
private import semmle.python.frameworks.Jmespath
private import semmle.python.frameworks.Joblib
private import semmle.python.frameworks.Ldap
private import semmle.python.frameworks.Ldap3
private import semmle.python.frameworks.Libtaxii
@@ -39,7 +40,9 @@ private import semmle.python.frameworks.MarkupSafe
private import semmle.python.frameworks.Multidict
private import semmle.python.frameworks.Mysql
private import semmle.python.frameworks.MySQLdb
private import semmle.python.frameworks.Numpy
private import semmle.python.frameworks.Oracledb
private import semmle.python.frameworks.Pandas
private import semmle.python.frameworks.Peewee
private import semmle.python.frameworks.Phoenixdb
private import semmle.python.frameworks.Psycopg2
@@ -55,11 +58,11 @@ private import semmle.python.frameworks.Rsa
private import semmle.python.frameworks.RuamelYaml
private import semmle.python.frameworks.Sanic
private import semmle.python.frameworks.ServerLess
private import semmle.python.frameworks.Setuptools
private import semmle.python.frameworks.Simplejson
private import semmle.python.frameworks.SqlAlchemy
private import semmle.python.frameworks.Starlette
private import semmle.python.frameworks.Stdlib
private import semmle.python.frameworks.Setuptools
private import semmle.python.frameworks.Toml
private import semmle.python.frameworks.Tornado
private import semmle.python.frameworks.Twisted

124
python/ql/lib/semmle/python/Module.qll
View File

@@ -179,21 +179,6 @@ private predicate legalDottedName(string name) {
bindingset[name]
private predicate legalShortName(string name) { name.regexpMatch("(\\p{L}|_)(\\p{L}|\\d|_)*") }
/**
* Holds if `f` is potentially a source package.
* Does it have an __init__.py file (or --respect-init=False for Python 2) and is it within the source archive?
*/
private predicate isPotentialSourcePackage(Folder f) {
f.getRelativePath() != "" and
isPotentialPackage(f)
}
private predicate isPotentialPackage(Folder f) {
exists(f.getFile("__init__.py"))
or
py_flags_versioned("options.respect_init", "False", _) and major_version() = 2 and exists(f)
}
private string moduleNameFromBase(Container file) {
// We used to also require `isPotentialPackage(f)` to hold in this case,
// but we saw modules not getting resolved because their folder did not
@@ -236,31 +221,114 @@ private predicate transitively_imported_from_entry_point(File file) {
)
}
/**
* Holds if the folder `f` is a regular Python package,
* containing an `__init__.py` file.
*/
private predicate isRegularPackage(Folder f, string name) {
legalShortName(name) and
name = f.getStem() and
exists(f.getFile("__init__.py"))
}
/** Gets the name of a module imported in package `c`. */
private string moduleImportedInPackage(Container c) {
legalShortName(result) and
// it has to be imported in this folder
result =
any(ImportExpr i | i.getLocation().getFile().getParent() = c)
.getName()
// strip everything after the first `.`
.regexpReplaceAll("\\..*", "") and
result != ""
}
/** Holds if the file `f` could be resolved to a module named `name`. */
private predicate isPotentialModuleFile(File file, string name) {
legalShortName(name) and
name = file.getStem() and
file.getExtension() = ["py", "pyc", "so", "pyd"] and
// it has to be imported in this folder
name = moduleImportedInPackage(file.getParent())
}
/**
* Holds if the folder `f` is a namespace package named `name`.
*
* See https://peps.python.org/pep-0420/#specification
* for details on namespace packages.
*/
private predicate isNameSpacePackage(Folder f, string name) {
legalShortName(name) and
name = f.getStem() and
not isRegularPackage(f, name) and
// it has to be imported in a file
// either in this folder or next to this folder
name = moduleImportedInPackage([f, f.getParent()]) and
// no sibling regular package
// and no sibling module
not exists(Folder sibling | sibling.getParent() = f.getParent() |
isRegularPackage(sibling.getFolder(name), name)
or
isPotentialModuleFile(sibling.getAFile(), name)
)
}
/**
* Holds if the folder `f` is a package (either a regular package
* or a namespace package) named `name`.
*/
private predicate isPackage(Folder f, string name) {
isRegularPackage(f, name)
or
isNameSpacePackage(f, name)
}
/**
* Holds if the file `f` is a module named `name`.
*/
private predicate isModuleFile(File file, string name) {
isPotentialModuleFile(file, name) and
not isPackage(file.getParent(), _)
}
/**
* Holds if the folder `f` is a package named `name`
* and does reside inside another package.
*/
private predicate isOutermostPackage(Folder f, string name) {
isPackage(f, name) and
not isPackage(f.getParent(), _)
}
/** Gets the name of the module that `c` resolves to, if any. */
cached
string moduleNameFromFile(Container file) {
string moduleNameFromFile(Container c) {
// package
isOutermostPackage(c, result)
or
// module
isModuleFile(c, result)
or
Stages::AST::ref() and
exists(string basename |
basename = moduleNameFromBase(file) and
basename = moduleNameFromBase(c) and
legalShortName(basename)
|
result = moduleNameFromFile(file.getParent()) + "." + basename
// recursive case
result = moduleNameFromFile(c.getParent()) + "." + basename
or
// If `file` is a transitive import of a file that's executed directly, we allow references
// to it by its `basename`.
transitively_imported_from_entry_point(file) and
transitively_imported_from_entry_point(c) and
result = basename
)
or
isPotentialSourcePackage(file) and
result = file.getStem() and
(
not isPotentialSourcePackage(file.getParent()) or
not legalShortName(file.getParent().getBaseName())
)
//
// standard library
result = c.getStem() and c.getParent() = c.getImportRoot()
or
result = file.getStem() and file.getParent() = file.getImportRoot()
or
result = file.getStem() and isStubRoot(file.getParent())
result = c.getStem() and isStubRoot(c.getParent())
}
private predicate isStubRoot(Folder f) {

37
python/ql/lib/semmle/python/dataflow/new/BarrierGuards.qll
View File

@@ -34,40 +34,3 @@ class StringConstCompareBarrier extends DataFlow::Node {
this = DataFlow::BarrierGuard<stringConstCompare/3>::getABarrierNode()
}
}
/**
* DEPRECATED: Use `StringConstCompareBarrier` instead.
*
* A validation of unknown node by comparing with a constant string value.
*/
deprecated class StringConstCompare extends DataFlow::BarrierGuard, CompareNode {
ControlFlowNode checked_node;
boolean safe_branch;
StringConstCompare() {
exists(StrConst str_const, Cmpop op |
op = any(Eq eq) and safe_branch = true
or
op = any(NotEq ne) and safe_branch = false
|
this.operands(str_const.getAFlowNode(), op, checked_node)
or
this.operands(checked_node, op, str_const.getAFlowNode())
)
or
exists(IterableNode str_const_iterable, Cmpop op |
op = any(In in_) and safe_branch = true
or
op = any(NotIn ni) and safe_branch = false
|
forall(ControlFlowNode elem | elem = str_const_iterable.getAnElement() |
elem.getNode() instanceof StrConst
) and
this.operands(checked_node, op, str_const_iterable)
)
}
override predicate checks(ControlFlowNode node, boolean branch) {
node = checked_node and branch = safe_branch
}
}

31
python/ql/lib/semmle/python/dataflow/new/internal/DataFlowDispatch.qll
View File

@@ -240,6 +240,19 @@ predicate hasPropertyDecorator(Function func) {
)
}
/**
* Holds if the function `func` has a `contextlib.contextmanager`.
*/
predicate hasContextmanagerDecorator(Function func) {
exists(ControlFlowNode contextmanager |
contextmanager.(NameNode).getId() = "contextmanager" and contextmanager.(NameNode).isGlobal()
or
contextmanager.(AttrNode).getObject("contextmanager").(NameNode).getId() = "contextlib"
|
func.getADecorator() = contextmanager.getNode()
)
}
// =============================================================================
// Callables
// =============================================================================
@@ -1604,6 +1617,24 @@ class ExtractedReturnNode extends ReturnNode, CfgNode {
override ReturnKind getKind() { any() }
}
/**
* A data flow node that represents the value yielded by a callable with a
* `contextlib.contextmanager` decorator. We treat this as a normal return, which makes
* things just work when used in a `with` statement -- technically calling the function
* directly will give you a `contextlib._GeneratorContextManager` instance, so it's a
* slight workaround solution.
*
* See https://docs.python.org/3/library/contextlib.html#contextlib.contextmanager
*/
class YieldNodeInContextManagerFunction extends ReturnNode, CfgNode {
YieldNodeInContextManagerFunction() {
hasContextmanagerDecorator(node.getScope()) and
node = any(Yield yield).getValue().getAFlowNode()
}
override ReturnKind getKind() { any() }
}
/** A data-flow node that represents the output of a call. */
abstract class OutNode extends Node {
/** Gets the underlying call, where this node is a corresponding output of kind `kind`. */

42
python/ql/lib/semmle/python/dataflow/new/internal/DataFlowImpl1.qll
View File

@@ -91,21 +91,6 @@ abstract class Configuration extends string {
/** Holds if data flow out of `node` is prohibited. */
predicate isBarrierOut(Node node) { none() }
/**
* DEPRECATED: Use `isBarrier` and `BarrierGuard` module instead.
*
* Holds if data flow through nodes guarded by `guard` is prohibited.
*/
deprecated predicate isBarrierGuard(BarrierGuard guard) { none() }
/**
* DEPRECATED: Use `isBarrier` and `BarrierGuard` module instead.
*
* Holds if data flow through nodes guarded by `guard` is prohibited when
* the flow state is `state`
*/
deprecated predicate isBarrierGuard(BarrierGuard guard, FlowState state) { none() }
/**
* Holds if data may flow from `node1` to `node2` in addition to the normal data-flow steps.
*/
@@ -225,29 +210,6 @@ abstract private class ConfigurationRecursionPrevention extends Configuration {
}
}
/** A bridge class to access the deprecated `isBarrierGuard`. */
private class BarrierGuardGuardedNodeBridge extends Unit {
abstract predicate guardedNode(Node n, Configuration config);
abstract predicate guardedNode(Node n, FlowState state, Configuration config);
}
private class BarrierGuardGuardedNode extends BarrierGuardGuardedNodeBridge {
deprecated override predicate guardedNode(Node n, Configuration config) {
exists(BarrierGuard g |
config.isBarrierGuard(g) and
n = g.getAGuardedNode()
)
}
deprecated override predicate guardedNode(Node n, FlowState state, Configuration config) {
exists(BarrierGuard g |
config.isBarrierGuard(g, state) and
n = g.getAGuardedNode()
)
}
}
private FlowState relevantState(Configuration config) {
config.isSource(_, result) or
config.isSink(_, result) or
@@ -288,9 +250,7 @@ private module Config implements FullStateConfigSig {
predicate isBarrier(Node node, FlowState state) {
getConfig(state).isBarrier(node, getState(state)) or
getConfig(state).isBarrier(node) or
any(BarrierGuardGuardedNodeBridge b).guardedNode(node, getState(state), getConfig(state)) or
any(BarrierGuardGuardedNodeBridge b).guardedNode(node, getConfig(state))
getConfig(state).isBarrier(node)
}
predicate isBarrierIn(Node node) { any(Configuration config).isBarrierIn(node) }

42
python/ql/lib/semmle/python/dataflow/new/internal/DataFlowImpl2.qll
View File

@@ -91,21 +91,6 @@ abstract class Configuration extends string {
/** Holds if data flow out of `node` is prohibited. */
predicate isBarrierOut(Node node) { none() }
/**
* DEPRECATED: Use `isBarrier` and `BarrierGuard` module instead.
*
* Holds if data flow through nodes guarded by `guard` is prohibited.
*/
deprecated predicate isBarrierGuard(BarrierGuard guard) { none() }
/**
* DEPRECATED: Use `isBarrier` and `BarrierGuard` module instead.
*
* Holds if data flow through nodes guarded by `guard` is prohibited when
* the flow state is `state`
*/
deprecated predicate isBarrierGuard(BarrierGuard guard, FlowState state) { none() }
/**
* Holds if data may flow from `node1` to `node2` in addition to the normal data-flow steps.
*/
@@ -225,29 +210,6 @@ abstract private class ConfigurationRecursionPrevention extends Configuration {
}
}
/** A bridge class to access the deprecated `isBarrierGuard`. */
private class BarrierGuardGuardedNodeBridge extends Unit {
abstract predicate guardedNode(Node n, Configuration config);
abstract predicate guardedNode(Node n, FlowState state, Configuration config);
}
private class BarrierGuardGuardedNode extends BarrierGuardGuardedNodeBridge {
deprecated override predicate guardedNode(Node n, Configuration config) {
exists(BarrierGuard g |
config.isBarrierGuard(g) and
n = g.getAGuardedNode()
)
}
deprecated override predicate guardedNode(Node n, FlowState state, Configuration config) {
exists(BarrierGuard g |
config.isBarrierGuard(g, state) and
n = g.getAGuardedNode()
)
}
}
private FlowState relevantState(Configuration config) {
config.isSource(_, result) or
config.isSink(_, result) or
@@ -288,9 +250,7 @@ private module Config implements FullStateConfigSig {
predicate isBarrier(Node node, FlowState state) {
getConfig(state).isBarrier(node, getState(state)) or
getConfig(state).isBarrier(node) or
any(BarrierGuardGuardedNodeBridge b).guardedNode(node, getState(state), getConfig(state)) or
any(BarrierGuardGuardedNodeBridge b).guardedNode(node, getConfig(state))
getConfig(state).isBarrier(node)
}
predicate isBarrierIn(Node node) { any(Configuration config).isBarrierIn(node) }

42
python/ql/lib/semmle/python/dataflow/new/internal/DataFlowImpl3.qll
View File

@@ -91,21 +91,6 @@ abstract class Configuration extends string {
/** Holds if data flow out of `node` is prohibited. */
predicate isBarrierOut(Node node) { none() }
/**
* DEPRECATED: Use `isBarrier` and `BarrierGuard` module instead.
*
* Holds if data flow through nodes guarded by `guard` is prohibited.
*/
deprecated predicate isBarrierGuard(BarrierGuard guard) { none() }
/**
* DEPRECATED: Use `isBarrier` and `BarrierGuard` module instead.
*
* Holds if data flow through nodes guarded by `guard` is prohibited when
* the flow state is `state`
*/
deprecated predicate isBarrierGuard(BarrierGuard guard, FlowState state) { none() }
/**
* Holds if data may flow from `node1` to `node2` in addition to the normal data-flow steps.
*/
@@ -225,29 +210,6 @@ abstract private class ConfigurationRecursionPrevention extends Configuration {
}
}
/** A bridge class to access the deprecated `isBarrierGuard`. */
private class BarrierGuardGuardedNodeBridge extends Unit {
abstract predicate guardedNode(Node n, Configuration config);
abstract predicate guardedNode(Node n, FlowState state, Configuration config);
}
private class BarrierGuardGuardedNode extends BarrierGuardGuardedNodeBridge {
deprecated override predicate guardedNode(Node n, Configuration config) {
exists(BarrierGuard g |
config.isBarrierGuard(g) and
n = g.getAGuardedNode()
)
}
deprecated override predicate guardedNode(Node n, FlowState state, Configuration config) {
exists(BarrierGuard g |
config.isBarrierGuard(g, state) and
n = g.getAGuardedNode()
)
}
}
private FlowState relevantState(Configuration config) {
config.isSource(_, result) or
config.isSink(_, result) or
@@ -288,9 +250,7 @@ private module Config implements FullStateConfigSig {
predicate isBarrier(Node node, FlowState state) {
getConfig(state).isBarrier(node, getState(state)) or
getConfig(state).isBarrier(node) or
any(BarrierGuardGuardedNodeBridge b).guardedNode(node, getState(state), getConfig(state)) or
any(BarrierGuardGuardedNodeBridge b).guardedNode(node, getConfig(state))
getConfig(state).isBarrier(node)
}
predicate isBarrierIn(Node node) { any(Configuration config).isBarrierIn(node) }

42
python/ql/lib/semmle/python/dataflow/new/internal/DataFlowImpl4.qll
View File

@@ -91,21 +91,6 @@ abstract class Configuration extends string {
/** Holds if data flow out of `node` is prohibited. */
predicate isBarrierOut(Node node) { none() }
/**
* DEPRECATED: Use `isBarrier` and `BarrierGuard` module instead.
*
* Holds if data flow through nodes guarded by `guard` is prohibited.
*/
deprecated predicate isBarrierGuard(BarrierGuard guard) { none() }
/**
* DEPRECATED: Use `isBarrier` and `BarrierGuard` module instead.
*
* Holds if data flow through nodes guarded by `guard` is prohibited when
* the flow state is `state`
*/
deprecated predicate isBarrierGuard(BarrierGuard guard, FlowState state) { none() }
/**
* Holds if data may flow from `node1` to `node2` in addition to the normal data-flow steps.
*/
@@ -225,29 +210,6 @@ abstract private class ConfigurationRecursionPrevention extends Configuration {
}
}
/** A bridge class to access the deprecated `isBarrierGuard`. */
private class BarrierGuardGuardedNodeBridge extends Unit {
abstract predicate guardedNode(Node n, Configuration config);
abstract predicate guardedNode(Node n, FlowState state, Configuration config);
}
private class BarrierGuardGuardedNode extends BarrierGuardGuardedNodeBridge {
deprecated override predicate guardedNode(Node n, Configuration config) {
exists(BarrierGuard g |
config.isBarrierGuard(g) and
n = g.getAGuardedNode()
)
}
deprecated override predicate guardedNode(Node n, FlowState state, Configuration config) {
exists(BarrierGuard g |
config.isBarrierGuard(g, state) and
n = g.getAGuardedNode()
)
}
}
private FlowState relevantState(Configuration config) {
config.isSource(_, result) or
config.isSink(_, result) or
@@ -288,9 +250,7 @@ private module Config implements FullStateConfigSig {
predicate isBarrier(Node node, FlowState state) {
getConfig(state).isBarrier(node, getState(state)) or
getConfig(state).isBarrier(node) or
any(BarrierGuardGuardedNodeBridge b).guardedNode(node, getState(state), getConfig(state)) or
any(BarrierGuardGuardedNodeBridge b).guardedNode(node, getConfig(state))
getConfig(state).isBarrier(node)
}
predicate isBarrierIn(Node node) { any(Configuration config).isBarrierIn(node) }

228
python/ql/lib/semmle/python/dataflow/new/internal/DataFlowPrivate.qll
View File

@@ -279,14 +279,16 @@ class NonSyntheticPostUpdateNode extends PostUpdateNodeImpl, CfgNode {
class DataFlowExpr = Expr;
/**
* Flow between ESSA variables.
* This includes both local and global variables.
* Flow comes from definitions, uses and refinements.
* A module to compute local flow.
*
* Flow will generally go from control flow nodes into essa variables at definitions,
* and from there via use-use flow to other control flow nodes.
*
* Some syntaxtic constructs are handled separately.
*/
// TODO: Consider constraining `nodeFrom` and `nodeTo` to be in the same scope.
// If they have different enclosing callables, we get consistency errors.
module EssaFlow {
predicate essaFlowStep(Node nodeFrom, Node nodeTo) {
module LocalFlow {
/** Holds if `nodeFrom` is the control flow node defining the essa variable `nodeTo`. */
predicate definitionFlowStep(Node nodeFrom, Node nodeTo) {
// Definition
// `x = f(42)`
// nodeFrom is `f(42)`, cfg node
@@ -336,10 +338,37 @@ module EssaFlow {
// nodeFrom is `x`, cfgNode
// nodeTo is `x`, essa var
exists(ParameterDefinition pd |
nodeFrom.asCfgNode() = pd.getDefiningNode() and
nodeTo.asVar() = pd.getVariable()
nodeFrom.(CfgNode).getNode() = pd.getDefiningNode() and
nodeTo.(EssaNode).getVar() = pd.getVariable()
)
}
predicate expressionFlowStep(Node nodeFrom, Node nodeTo) {
// If expressions
nodeFrom.asCfgNode() = nodeTo.asCfgNode().(IfExprNode).getAnOperand()
or
// Assignment expressions
nodeFrom.asCfgNode() = nodeTo.asCfgNode().(AssignmentExprNode).getValue()
or
// boolean inline expressions such as `x or y` or `x and y`
nodeFrom.asCfgNode() = nodeTo.asCfgNode().(BoolExprNode).getAnOperand()
or
// Flow inside an unpacking assignment
iterableUnpackingFlowStep(nodeFrom, nodeTo)
or
// Flow inside a match statement
matchFlowStep(nodeFrom, nodeTo)
}
predicate useToNextUse(NameNode nodeFrom, NameNode nodeTo) {
AdjacentUses::adjacentUseUse(nodeFrom, nodeTo)
}
predicate defToFirstUse(EssaVariable var, NameNode nodeTo) {
AdjacentUses::firstUse(var.getDefinition(), nodeTo)
}
predicate useUseFlowStep(Node nodeFrom, Node nodeTo) {
// First use after definition
// `y = 42`
// `x = f(y)`
@@ -353,31 +382,105 @@ module EssaFlow {
// nodeFrom is 'y' on first line, cfg node
// nodeTo is `y` on second line, cfg node
useToNextUse(nodeFrom.asCfgNode(), nodeTo.asCfgNode())
or
// If expressions
nodeFrom.asCfgNode() = nodeTo.asCfgNode().(IfExprNode).getAnOperand()
or
// boolean inline expressions such as `x or y` or `x and y`
nodeFrom.asCfgNode() = nodeTo.asCfgNode().(BoolExprNode).getAnOperand()
or
// Flow inside an unpacking assignment
iterableUnpackingFlowStep(nodeFrom, nodeTo)
or
matchFlowStep(nodeFrom, nodeTo)
}
predicate useToNextUse(NameNode nodeFrom, NameNode nodeTo) {
AdjacentUses::adjacentUseUse(nodeFrom, nodeTo)
}
predicate defToFirstUse(EssaVariable var, NameNode nodeTo) {
AdjacentUses::firstUse(var.getDefinition(), nodeTo)
predicate localFlowStep(Node nodeFrom, Node nodeTo) {
IncludePostUpdateFlow<PhaseDependentFlow<definitionFlowStep/2>::step/2>::step(nodeFrom, nodeTo)
or
IncludePostUpdateFlow<PhaseDependentFlow<expressionFlowStep/2>::step/2>::step(nodeFrom, nodeTo)
or
// Blindly applying use-use flow can result in a node that steps to itself, for
// example in while-loops. To uphold dataflow consistency checks, we don't want
// that. However, we do want to allow `[post] n` to `n` (to handle while loops), so
// we should only do the filtering after `IncludePostUpdateFlow` has ben applied.
IncludePostUpdateFlow<PhaseDependentFlow<useUseFlowStep/2>::step/2>::step(nodeFrom, nodeTo) and
nodeFrom != nodeTo
}
}
//--------
// Local flow
//--------
/** A module for transforming step relations. */
module StepRelationTransformations {
/**
* Holds if there is a step from `nodeFrom` to `nodeTo` in
* the step relation to be transformed.
*
* This is the input relation to the transformations.
*/
signature predicate stepSig(Node nodeFrom, Node nodeTo);
/**
* A module to separate import-time from run-time.
*
* We really have two local flow relations, one for module initialisation time (or _import time_) and one for runtime.
* Consider a read from a global variable `x = foo`. At import time there should be a local flow step from `foo` to `x`,
* while at runtime there should be a jump step from the module variable corresponding to `foo` to `x`.
*
* Similarly, for a write `foo = y`, at import time, there is a local flow step from `y` to `foo` while at runtime there
* is a jump step from `y` to the module variable corresponding to `foo`.
*
* We need a way of distinguishing if we are looking at import time or runtime. We have the following helpful facts:
* - All top-level executable statements are import time (and import time only)
* - All non-top-level code may be executed at runtime (but could also be executed at import time)
*
* We could write an analysis to determine which functions are called at import time, but until we have that, we will go
* with the heuristic that global variables act according to import time rules at top-level program points and according
* to runtime rules everywhere else. This will forego some import time local flow but otherwise be consistent.
*/
module PhaseDependentFlow<stepSig/2 rawStep> {
/**
* Holds if `node` is found at the top level of a module.
*/
pragma[inline]
private predicate isTopLevel(Node node) { node.getScope() instanceof Module }
/** Holds if a step can be taken from `nodeFrom` to `nodeTo` at import time. */
predicate importTimeStep(Node nodeFrom, Node nodeTo) {
// As a proxy for whether statements can be executed at import time,
// we check if they appear at the top level.
// This will miss statements inside functions called from the top level.
isTopLevel(nodeFrom) and
isTopLevel(nodeTo) and
rawStep(nodeFrom, nodeTo)
}
/** Holds if a step can be taken from `nodeFrom` to `nodeTo` at runtime. */
predicate runtimeStep(Node nodeFrom, Node nodeTo) {
// Anything not at the top level can be executed at runtime.
not isTopLevel(nodeFrom) and
not isTopLevel(nodeTo) and
rawStep(nodeFrom, nodeTo)
}
/**
* Holds if a step can be taken from `nodeFrom` to `nodeTo`.
*/
predicate step(Node nodeFrom, Node nodeTo) {
importTimeStep(nodeFrom, nodeTo) or
runtimeStep(nodeFrom, nodeTo)
}
}
/**
* A module to add steps from post-update nodes.
* Whenever there is a step from `x` to `y`,
* we add a step from `[post] x` to `y`.
*/
module IncludePostUpdateFlow<stepSig/2 rawStep> {
predicate step(Node nodeFrom, Node nodeTo) {
// We either have a raw step from `nodeFrom`...
rawStep(nodeFrom, nodeTo)
or
// ...or we have a raw step from a pre-update node of `nodeFrom`
rawStep(nodeFrom.(PostUpdateNode).getPreUpdateNode(), nodeTo)
}
}
}
import StepRelationTransformations
/**
* This is the local flow predicate that is used as a building block in global
* data flow.
@@ -398,69 +501,17 @@ predicate simpleLocalFlowStep(Node nodeFrom, Node nodeTo) {
* or at runtime when callables in the module are called.
*/
predicate simpleLocalFlowStepForTypetracking(Node nodeFrom, Node nodeTo) {
// If there is local flow out of a node `node`, we want flow
// both out of `node` and any post-update node of `node`.
exists(Node node |
nodeFrom = update(node) and
(
importTimeLocalFlowStep(node, nodeTo) or
runtimeLocalFlowStep(node, nodeTo)
)
)
IncludePostUpdateFlow<PhaseDependentFlow<LocalFlow::localFlowStep/2>::step/2>::step(nodeFrom,
nodeTo)
}
/**
* Holds if `node` is found at the top level of a module.
*/
pragma[inline]
predicate isTopLevel(Node node) { node.getScope() instanceof Module }
/** Holds if there is local flow from `nodeFrom` to `nodeTo` at import time. */
predicate importTimeLocalFlowStep(Node nodeFrom, Node nodeTo) {
// As a proxy for whether statements can be executed at import time,
// we check if they appear at the top level.
// This will miss statements inside functions called from the top level.
isTopLevel(nodeFrom) and
isTopLevel(nodeTo) and
EssaFlow::essaFlowStep(nodeFrom, nodeTo)
}
/** Holds if there is local flow from `nodeFrom` to `nodeTo` at runtime. */
predicate runtimeLocalFlowStep(Node nodeFrom, Node nodeTo) {
// Anything not at the top level can be executed at runtime.
not isTopLevel(nodeFrom) and
not isTopLevel(nodeTo) and
EssaFlow::essaFlowStep(nodeFrom, nodeTo)
private predicate summaryLocalStep(Node nodeFrom, Node nodeTo) {
FlowSummaryImpl::Private::Steps::summaryLocalStep(nodeFrom.(FlowSummaryNode).getSummaryNode(),
nodeTo.(FlowSummaryNode).getSummaryNode(), true)
}
predicate summaryFlowSteps(Node nodeFrom, Node nodeTo) {
// If there is local flow out of a node `node`, we want flow
// both out of `node` and any post-update node of `node`.
exists(Node node |
nodeFrom = update(node) and
(
importTimeSummaryFlowStep(node, nodeTo) or
runtimeSummaryFlowStep(node, nodeTo)
)
)
}
predicate importTimeSummaryFlowStep(Node nodeFrom, Node nodeTo) {
// As a proxy for whether statements can be executed at import time,
// we check if they appear at the top level.
// This will miss statements inside functions called from the top level.
isTopLevel(nodeFrom) and
isTopLevel(nodeTo) and
FlowSummaryImpl::Private::Steps::summaryLocalStep(nodeFrom.(FlowSummaryNode).getSummaryNode(),
nodeTo.(FlowSummaryNode).getSummaryNode(), true)
}
predicate runtimeSummaryFlowStep(Node nodeFrom, Node nodeTo) {
// Anything not at the top level can be executed at runtime.
not isTopLevel(nodeFrom) and
not isTopLevel(nodeTo) and
FlowSummaryImpl::Private::Steps::summaryLocalStep(nodeFrom.(FlowSummaryNode).getSummaryNode(),
nodeTo.(FlowSummaryNode).getSummaryNode(), true)
IncludePostUpdateFlow<PhaseDependentFlow<summaryLocalStep/2>::step/2>::step(nodeFrom, nodeTo)
}
/** `ModuleVariable`s are accessed via jump steps at runtime. */
@@ -484,15 +535,6 @@ predicate runtimeJumpStep(Node nodeFrom, Node nodeTo) {
)
}
/**
* Holds if `result` is either `node`, or the post-update node for `node`.
*/
private Node update(Node node) {
result = node
or
result.(PostUpdateNode).getPreUpdateNode() = node
}
//--------
// Type pruning
//--------

28
python/ql/lib/semmle/python/dataflow/new/internal/DataFlowPublic.qll
View File

@@ -400,7 +400,7 @@ class ModuleVariableNode extends Node, TModuleVariableNode {
override Scope getScope() { result = mod }
override string toString() {
result = "ModuleVariableNode in " + mod.toString() + " for " + var.getId()
result = "ModuleVariableNode in " + concat( | | mod.toString(), ",") + " for " + var.getId()
}
/** Gets the module in which this variable appears. */
@@ -580,32 +580,6 @@ module BarrierGuard<guardChecksSig/3 guardChecks> {
}
}
/**
* DEPRECATED: Use `BarrierGuard` module instead.
*
* A guard that validates some expression.
*
* To use this in a configuration, extend the class and provide a
* characteristic predicate precisely specifying the guard, and override
* `checks` to specify what is being validated and in which branch.
*
* It is important that all extending classes in scope are disjoint.
*/
deprecated class BarrierGuard extends GuardNode {
/** Holds if this guard validates `node` upon evaluating to `branch`. */
abstract predicate checks(ControlFlowNode node, boolean branch);
/** Gets a node guarded by this guard. */
final ExprNode getAGuardedNode() {
exists(EssaDefinition def, ControlFlowNode node, boolean branch |
AdjacentUses::useOfDef(def, node) and
this.checks(node, branch) and
AdjacentUses::useOfDef(def, result.asCfgNode()) and
this.controlsBlock(result.asCfgNode().getBasicBlock(), branch)
)
}
}
/**
* Algebraic datatype for tracking data content associated with values.
* Content can be collection elements or object attributes.

8
python/ql/lib/semmle/python/dataflow/new/internal/ImportResolution.qll
View File

@@ -111,13 +111,13 @@ module ImportResolution {
allowedEssaImportStep*(firstDef, lastUseVar) and
not allowedEssaImportStep(_, firstDef)
|
not EssaFlow::defToFirstUse(firstDef, _) and
not LocalFlow::defToFirstUse(firstDef, _) and
val.asVar() = firstDef
or
exists(ControlFlowNode mid, ControlFlowNode end |
EssaFlow::defToFirstUse(firstDef, mid) and
EssaFlow::useToNextUse*(mid, end) and
not EssaFlow::useToNextUse(end, _) and
LocalFlow::defToFirstUse(firstDef, mid) and
LocalFlow::useToNextUse*(mid, end) and
not LocalFlow::useToNextUse(end, _) and
val.asCfgNode() = end
)
)

17
python/ql/lib/semmle/python/dataflow/new/internal/LocalSources.qll
View File

@@ -11,6 +11,7 @@ import DataFlowPublic
private import DataFlowPrivate
private import semmle.python.internal.CachedStages
private import semmle.python.internal.Awaited
private import semmle.python.dataflow.new.internal.ImportStar
/**
* A data flow node that is a source of local flow. This includes things like
@@ -39,6 +40,22 @@ class LocalSourceNode extends Node {
this instanceof ExprNode and
not simpleLocalFlowStepForTypetracking(_, this)
or
// For `from foo import *; foo_function()`, we want to let the variables we think
// could originate in `foo` (such as `foo_function`) to be available in the API
// graph. This requires them to be local sources. They would not be from the code
// just above, since the CFG node has flow going into it from its corresponding
// `GlobalSsaVariable`. (a different work-around is to change API graphs to not rely
// as heavily on LocalSourceNode; I initially tried this, but it relied on a lot of
// copy-pasted code, and it requires some non-trivial deprecation for downgrading
// the result type of `.asSource()` to DataFlow::Node, so we've opted for this
// approach instead).
//
// Note: This is only needed at the module level -- uses inside functions appear as
// LocalSourceNodes as we expect.
//
// TODO: When rewriting SSA, we should be able to remove this workaround
ImportStar::namePossiblyDefinedInImportStar(this.(ExprNode).getNode(), _, any(Module m))
or
// We include all module variable nodes, as these act as stepping stones between writes and
// reads of global variables. Without them, type tracking based on `LocalSourceNode`s would be
// unable to track across global variables.

71
python/ql/lib/semmle/python/dataflow/new/internal/SummaryTypeTracker.qll
View File

@@ -73,8 +73,12 @@ signature module Input {
}
// Relating nodes to summaries
/** Gets a dataflow node respresenting the argument of `call` indicated by `arg`. */
Node argumentOf(Node call, SummaryComponent arg);
/**
* Gets a dataflow node respresenting the argument of `call` indicated by `arg`.
*
* Returns the post-update node of the argument when `isPostUpdate` is true.
*/
Node argumentOf(Node call, SummaryComponent arg, boolean isPostUpdate);
/** Gets a dataflow node respresenting the parameter of `callable` indicated by `param`. */
Node parameterOf(Node callable, SummaryComponent param);
@@ -221,14 +225,18 @@ module SummaryFlow<Input I> implements Output<I> {
/**
* Gets a data flow `I::Node` corresponding an argument or return value of `call`,
* as specified by `component`.
* as specified by `component`. `isOutput` indicates whether the node represents
* an output node or an input node.
*/
bindingset[call, component]
private I::Node evaluateSummaryComponentLocal(I::Node call, I::SummaryComponent component) {
result = I::argumentOf(call, component)
private I::Node evaluateSummaryComponentLocal(
I::Node call, I::SummaryComponent component, boolean isOutput
) {
result = I::argumentOf(call, component, isOutput)
or
component = I::return() and
result = call
result = call and
isOutput = true
}
/**
@@ -280,27 +288,40 @@ module SummaryFlow<Input I> implements Output<I> {
*/
pragma[nomagic]
private I::Node evaluateSummaryComponentStackLocal(
I::SummarizedCallable callable, I::Node call, I::SummaryComponentStack stack
I::SummarizedCallable callable, I::Node call, I::SummaryComponentStack stack, boolean isOutput
) {
exists(I::SummaryComponent component |
dependsOnSummaryComponentStackLeaf(callable, component) and
stack = I::singleton(component) and
call = I::callTo(callable) and
result = evaluateSummaryComponentLocal(call, component)
result = evaluateSummaryComponentLocal(call, component, isOutput)
)
or
exists(I::Node prev, I::SummaryComponent head, I::SummaryComponentStack tail |
prev = evaluateSummaryComponentStackLocal(callable, call, tail) and
exists(
I::Node prev, I::SummaryComponent head, I::SummaryComponentStack tail, boolean isOutput0
|
prev = evaluateSummaryComponentStackLocal(callable, call, tail, isOutput0) and
dependsOnSummaryComponentStackConsLocal(callable, pragma[only_bind_into](head),
pragma[only_bind_out](tail)) and
stack = I::push(pragma[only_bind_out](head), pragma[only_bind_out](tail))
|
result = I::parameterOf(prev, head)
// `Parameter[X]` is only allowed in the output of flow summaries (hence `isOutput = true`),
// however the target of the parameter (e.g. `Argument[Y].Parameter[X]`) should be fetched
// not from a post-update argument node (hence `isOutput0 = false`)
result = I::parameterOf(prev, head) and
isOutput0 = false and
isOutput = true
or
result = I::returnOf(prev, head)
// `ReturnValue` is only allowed in the input of flow summaries (hence `isOutput = false`),
// and the target of the return value (e.g. `Argument[X].ReturnValue`) should be fetched not
// from a post-update argument node (hence `isOutput0 = false`)
result = I::returnOf(prev, head) and
isOutput0 = false and
isOutput = false
or
componentLevelStep(head) and
result = prev
result = prev and
isOutput = isOutput0
)
}
@@ -312,8 +333,8 @@ module SummaryFlow<Input I> implements Output<I> {
|
callable.propagatesFlow(input, output, true) and
call = I::callTo(callable) and
nodeFrom = evaluateSummaryComponentStackLocal(callable, call, input) and
nodeTo = evaluateSummaryComponentStackLocal(callable, call, output)
nodeFrom = evaluateSummaryComponentStackLocal(callable, call, input, false) and
nodeTo = evaluateSummaryComponentStackLocal(callable, call, output, true)
)
}
@@ -325,8 +346,8 @@ module SummaryFlow<Input I> implements Output<I> {
hasLoadSummary(callable, content, pragma[only_bind_into](input),
pragma[only_bind_into](output)) and
call = I::callTo(callable) and
nodeFrom = evaluateSummaryComponentStackLocal(callable, call, input) and
nodeTo = evaluateSummaryComponentStackLocal(callable, call, output)
nodeFrom = evaluateSummaryComponentStackLocal(callable, call, input, false) and
nodeTo = evaluateSummaryComponentStackLocal(callable, call, output, true)
)
}
@@ -338,8 +359,8 @@ module SummaryFlow<Input I> implements Output<I> {
hasStoreSummary(callable, content, pragma[only_bind_into](input),
pragma[only_bind_into](output)) and
call = I::callTo(callable) and
nodeFrom = evaluateSummaryComponentStackLocal(callable, call, input) and
nodeTo = evaluateSummaryComponentStackLocal(callable, call, output)
nodeFrom = evaluateSummaryComponentStackLocal(callable, call, input, false) and
nodeTo = evaluateSummaryComponentStackLocal(callable, call, output, true)
)
}
@@ -354,8 +375,8 @@ module SummaryFlow<Input I> implements Output<I> {
hasLoadStoreSummary(callable, loadContent, storeContent, pragma[only_bind_into](input),
pragma[only_bind_into](output)) and
call = I::callTo(callable) and
nodeFrom = evaluateSummaryComponentStackLocal(callable, call, input) and
nodeTo = evaluateSummaryComponentStackLocal(callable, call, output)
nodeFrom = evaluateSummaryComponentStackLocal(callable, call, input, false) and
nodeTo = evaluateSummaryComponentStackLocal(callable, call, output, true)
)
}
@@ -369,8 +390,8 @@ module SummaryFlow<Input I> implements Output<I> {
hasWithoutContentSummary(callable, filter, pragma[only_bind_into](input),
pragma[only_bind_into](output)) and
call = I::callTo(callable) and
nodeFrom = evaluateSummaryComponentStackLocal(callable, call, input) and
nodeTo = evaluateSummaryComponentStackLocal(callable, call, output)
nodeFrom = evaluateSummaryComponentStackLocal(callable, call, input, false) and
nodeTo = evaluateSummaryComponentStackLocal(callable, call, output, true)
)
}
@@ -384,8 +405,8 @@ module SummaryFlow<Input I> implements Output<I> {
hasWithContentSummary(callable, filter, pragma[only_bind_into](input),
pragma[only_bind_into](output)) and
call = I::callTo(callable) and
nodeFrom = evaluateSummaryComponentStackLocal(callable, call, input) and
nodeTo = evaluateSummaryComponentStackLocal(callable, call, output)
nodeFrom = evaluateSummaryComponentStackLocal(callable, call, input, false) and
nodeTo = evaluateSummaryComponentStackLocal(callable, call, output, true)
)
}
}

5
python/ql/lib/semmle/python/dataflow/new/internal/TypeTrackerSpecific.qll
View File

@@ -214,10 +214,11 @@ private module SummaryTypeTrackerInput implements SummaryTypeTracker::Input {
predicate return = FlowSummary::SummaryComponent::return/0;
// Relating nodes to summaries
Node argumentOf(Node call, SummaryComponent arg) {
Node argumentOf(Node call, SummaryComponent arg, boolean isPostUpdate) {
exists(DataFlowDispatch::ParameterPosition pos |
arg = FlowSummary::SummaryComponent::argument(pos) and
argumentPositionMatch(call, result, pos)
argumentPositionMatch(call, result, pos) and
isPostUpdate = [false, true] // todo: implement when/if Python uses post-update nodes in type tracking
)
}

27
python/ql/lib/semmle/python/dataflow/new/internal/tainttracking1/TaintTrackingImpl.qll
View File

@@ -116,33 +116,6 @@ abstract class Configuration extends DataFlow::Configuration {
final override predicate isBarrierOut(DataFlow::Node node) { this.isSanitizerOut(node) }
/**
* DEPRECATED: Use `isSanitizer` and `BarrierGuard` module instead.
*
* Holds if taint propagation through nodes guarded by `guard` is prohibited.
*/
deprecated predicate isSanitizerGuard(DataFlow::BarrierGuard guard) { none() }
deprecated final override predicate isBarrierGuard(DataFlow::BarrierGuard guard) {
this.isSanitizerGuard(guard)
}
/**
* DEPRECATED: Use `isSanitizer` and `BarrierGuard` module instead.
*
* Holds if taint propagation through nodes guarded by `guard` is prohibited
* when the flow state is `state`.
*/
deprecated predicate isSanitizerGuard(DataFlow::BarrierGuard guard, DataFlow::FlowState state) {
none()
}
deprecated final override predicate isBarrierGuard(
DataFlow::BarrierGuard guard, DataFlow::FlowState state
) {
this.isSanitizerGuard(guard, state)
}
/**
* Holds if taint may propagate from `node1` to `node2` in addition to the normal data-flow and taint steps.
*/

27
python/ql/lib/semmle/python/dataflow/new/internal/tainttracking2/TaintTrackingImpl.qll
View File

@@ -116,33 +116,6 @@ abstract class Configuration extends DataFlow::Configuration {
final override predicate isBarrierOut(DataFlow::Node node) { this.isSanitizerOut(node) }
/**
* DEPRECATED: Use `isSanitizer` and `BarrierGuard` module instead.
*
* Holds if taint propagation through nodes guarded by `guard` is prohibited.
*/
deprecated predicate isSanitizerGuard(DataFlow::BarrierGuard guard) { none() }
deprecated final override predicate isBarrierGuard(DataFlow::BarrierGuard guard) {
this.isSanitizerGuard(guard)
}
/**
* DEPRECATED: Use `isSanitizer` and `BarrierGuard` module instead.
*
* Holds if taint propagation through nodes guarded by `guard` is prohibited
* when the flow state is `state`.
*/
deprecated predicate isSanitizerGuard(DataFlow::BarrierGuard guard, DataFlow::FlowState state) {
none()
}
deprecated final override predicate isBarrierGuard(
DataFlow::BarrierGuard guard, DataFlow::FlowState state
) {
this.isSanitizerGuard(guard, state)
}
/**
* Holds if taint may propagate from `node1` to `node2` in addition to the normal data-flow and taint steps.
*/

27
python/ql/lib/semmle/python/dataflow/new/internal/tainttracking3/TaintTrackingImpl.qll
View File

@@ -116,33 +116,6 @@ abstract class Configuration extends DataFlow::Configuration {
final override predicate isBarrierOut(DataFlow::Node node) { this.isSanitizerOut(node) }
/**
* DEPRECATED: Use `isSanitizer` and `BarrierGuard` module instead.
*
* Holds if taint propagation through nodes guarded by `guard` is prohibited.
*/
deprecated predicate isSanitizerGuard(DataFlow::BarrierGuard guard) { none() }
deprecated final override predicate isBarrierGuard(DataFlow::BarrierGuard guard) {
this.isSanitizerGuard(guard)
}
/**
* DEPRECATED: Use `isSanitizer` and `BarrierGuard` module instead.
*
* Holds if taint propagation through nodes guarded by `guard` is prohibited
* when the flow state is `state`.
*/
deprecated predicate isSanitizerGuard(DataFlow::BarrierGuard guard, DataFlow::FlowState state) {
none()
}
deprecated final override predicate isBarrierGuard(
DataFlow::BarrierGuard guard, DataFlow::FlowState state
) {
this.isSanitizerGuard(guard, state)
}
/**
* Holds if taint may propagate from `node1` to `node2` in addition to the normal data-flow and taint steps.
*/

27
python/ql/lib/semmle/python/dataflow/new/internal/tainttracking4/TaintTrackingImpl.qll
View File

@@ -116,33 +116,6 @@ abstract class Configuration extends DataFlow::Configuration {
final override predicate isBarrierOut(DataFlow::Node node) { this.isSanitizerOut(node) }
/**
* DEPRECATED: Use `isSanitizer` and `BarrierGuard` module instead.
*
* Holds if taint propagation through nodes guarded by `guard` is prohibited.
*/
deprecated predicate isSanitizerGuard(DataFlow::BarrierGuard guard) { none() }
deprecated final override predicate isBarrierGuard(DataFlow::BarrierGuard guard) {
this.isSanitizerGuard(guard)
}
/**
* DEPRECATED: Use `isSanitizer` and `BarrierGuard` module instead.
*
* Holds if taint propagation through nodes guarded by `guard` is prohibited
* when the flow state is `state`.
*/
deprecated predicate isSanitizerGuard(DataFlow::BarrierGuard guard, DataFlow::FlowState state) {
none()
}
deprecated final override predicate isBarrierGuard(
DataFlow::BarrierGuard guard, DataFlow::FlowState state
) {
this.isSanitizerGuard(guard, state)
}
/**
* Holds if taint may propagate from `node1` to `node2` in addition to the normal data-flow and taint steps.
*/

33
python/ql/lib/semmle/python/frameworks/Joblib.qll Normal file
View File

@@ -0,0 +1,33 @@
/**
* Provides classes modeling security-relevant aspects of the `joblib` PyPI package.
* See https://pypi.org/project/joblib/.
*/
private import python
private import semmle.python.dataflow.new.DataFlow
private import semmle.python.Concepts
private import semmle.python.ApiGraphs
/**
* Provides models for the `joblib` PyPI package.
* See https://pypi.org/project/joblib/.
*/
private module Joblib {
/**
* A call to `joblib.load`
* See https://pypi.org/project/joblib/
*/
private class JoblibLoadCall extends Decoding::Range, DataFlow::CallCfgNode {
JoblibLoadCall() { this = API::moduleImport("joblib").getMember("load").getACall() }
override predicate mayExecuteInput() { any() }
override DataFlow::Node getAnInput() {
result in [this.getArg(0), this.getArgByName("filename")]
}
override DataFlow::Node getOutput() { result = this }
override string getFormat() { result = "joblib" }
}
}

42
python/ql/lib/semmle/python/frameworks/Numpy.qll Normal file
View File

@@ -0,0 +1,42 @@
/**
* Provides classes modeling security-relevant aspects of the `numpy` PyPI package.
* See https://pypi.org/project/numpy/.
*/
private import python
private import semmle.python.dataflow.new.DataFlow
private import semmle.python.dataflow.new.RemoteFlowSources
private import semmle.python.Concepts
private import semmle.python.ApiGraphs
/**
* Provides models for the `numpy` PyPI package.
* See https://pypi.org/project/numpy/.
*/
private module Numpy {
/**
* A call to `numpy.load`
* See https://numpy.org/doc/stable/reference/generated/numpy.load.html
*/
private class NumpyLoadCall extends Decoding::Range, API::CallNode {
NumpyLoadCall() { this = API::moduleImport("numpy").getMember("load").getACall() }
override predicate mayExecuteInput() {
this.getParameter(2, "allow_pickle")
.getAValueReachingSink()
.asExpr()
.(ImmutableLiteral)
.booleanValue() = true
}
override DataFlow::Node getAnInput() { result = this.getParameter(0, "filename").asSink() }
override DataFlow::Node getOutput() { result = this }
override string getFormat() {
result = "numpy"
or
this.mayExecuteInput() and result = "pickle"
}
}
}

37
python/ql/lib/semmle/python/frameworks/Pandas.qll Normal file
View File

@@ -0,0 +1,37 @@
/**
* Provides classes modeling security-relevant aspects of the `pandas` PyPI package.
* See https://pypi.org/project/pandas/.
*/
private import python
private import semmle.python.dataflow.new.DataFlow
private import semmle.python.dataflow.new.RemoteFlowSources
private import semmle.python.Concepts
private import semmle.python.ApiGraphs
/**
* Provides models for the `pandas` PyPI package.
* See https://pypi.org/project/pandas/.
*/
private module Pandas {
/**
* A call to `pandas.read_pickle`
* See https://pypi.org/project/pandas/
* See https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.read_pickle.html
*/
private class PandasReadPickleCall extends Decoding::Range, DataFlow::CallCfgNode {
PandasReadPickleCall() {
this = API::moduleImport("pandas").getMember("read_pickle").getACall()
}
override predicate mayExecuteInput() { any() }
override DataFlow::Node getAnInput() {
result in [this.getArg(0), this.getArgByName("filepath_or_buffer")]
}
override DataFlow::Node getOutput() { result = this }
override string getFormat() { result = "pickle" }
}
}

9
python/ql/lib/semmle/python/frameworks/data/internal/ApiGraphModelsExtensions.qll
View File

@@ -17,7 +17,7 @@ extensible predicate sourceModel(string type, string path, string kind);
extensible predicate sinkModel(string type, string path, string kind);
/**
* Holds if calls to `(type, path)`, the value referred to by `input`
* Holds if in calls to `(type, path)`, the value referred to by `input`
* can flow to the value referred to by `output`.
*
* `kind` should be either `value` or `taint`, for value-preserving or taint-preserving steps,
@@ -25,6 +25,13 @@ extensible predicate sinkModel(string type, string path, string kind);
*/
extensible predicate summaryModel(string type, string path, string input, string output, string kind);
/**
* Holds if calls to `(type, path)` should be considered neutral. The meaning of this depends on the `kind`.
* If `kind` is `summary`, the call does not propagate data flow. If `kind` is `source`, the call is not a source.
* If `kind` is `sink`, the call is not a sink.
*/
extensible predicate neutralModel(string type, string path, string kind);
/**
* Holds if `(type2, path)` should be seen as an instance of `type1`.
*/

5
python/ql/lib/semmle/python/frameworks/data/internal/empty.model.yml
View File

@@ -15,6 +15,11 @@ extensions:
extensible: summaryModel
data: []
- addsTo:
pack: codeql/python-all
extensible: neutralModel
data: []
- addsTo:
pack: codeql/python-all
extensible: typeModel

3
python/ql/lib/semmle/python/frameworks/internal/SubclassFinder.qll
View File

@@ -73,9 +73,6 @@ private module NotExposed {
result = "moduleImport(\"" + fullyQualified.replaceAll(".", "\").getMember(\"") + "\")"
}
/** DEPRECATED: Alias for fullyQualifiedToApiGraphPath */
deprecated predicate fullyQualifiedToAPIGraphPath = fullyQualifiedToApiGraphPath/1;
bindingset[this]
abstract class FindSubclassesSpec extends string {
abstract API::Node getAlreadyModeledClass();

16
python/ql/lib/semmle/python/security/Paths.qll
View File

@@ -1,16 +0,0 @@
import semmle.python.dataflow.Implementation
deprecated module TaintTrackingPaths {
predicate edge(TaintTrackingNode src, TaintTrackingNode dest, string label) {
exists(TaintTrackingNode source, TaintTrackingNode sink |
source.getConfiguration().hasFlowPath(source, sink) and
source.getASuccessor*() = src and
src.getASuccessor(label) = dest and
dest.getASuccessor*() = sink
)
}
}
deprecated query predicate edges(TaintTrackingNode fromnode, TaintTrackingNode tonode) {
TaintTrackingPaths::edge(fromnode, tonode, _)
}

7
python/ql/lib/semmle/python/security/dataflow/CodeInjectionCustomizations.qll
View File

@@ -31,13 +31,6 @@ module CodeInjection {
*/
abstract class Sanitizer extends DataFlow::Node { }
/**
* DEPRECATED: Use `Sanitizer` instead.
*
* A sanitizer guard for "code injection" vulnerabilities.
*/
abstract deprecated class SanitizerGuard extends DataFlow::BarrierGuard { }
/**
* A source of remote user input, considered as a flow source.
*/

4
python/ql/lib/semmle/python/security/dataflow/CodeInjectionQuery.qll
View File

@@ -24,10 +24,6 @@ deprecated class Configuration extends TaintTracking::Configuration {
override predicate isSink(DataFlow::Node sink) { sink instanceof Sink }
override predicate isSanitizer(DataFlow::Node node) { node instanceof Sanitizer }
deprecated override predicate isSanitizerGuard(DataFlow::BarrierGuard guard) {
guard instanceof SanitizerGuard
}
}
private module CodeInjectionConfig implements DataFlow::ConfigSig {

7
python/ql/lib/semmle/python/security/dataflow/CommandInjectionCustomizations.qll
View File

@@ -31,13 +31,6 @@ module CommandInjection {
*/
abstract class Sanitizer extends DataFlow::Node { }
/**
* DEPRECATED: Use `Sanitizer` instead.
*
* A sanitizer guard for "command injection" vulnerabilities.
*/
abstract deprecated class SanitizerGuard extends DataFlow::BarrierGuard { }
/**
* A source of remote user input, considered as a flow source.
*/

4
python/ql/lib/semmle/python/security/dataflow/CommandInjectionQuery.qll
View File

@@ -24,10 +24,6 @@ deprecated class Configuration extends TaintTracking::Configuration {
override predicate isSink(DataFlow::Node sink) { sink instanceof Sink }
override predicate isSanitizer(DataFlow::Node node) { node instanceof Sanitizer }
deprecated override predicate isSanitizerGuard(DataFlow::BarrierGuard guard) {
guard instanceof SanitizerGuard
}
}
/**

14
python/ql/lib/semmle/python/security/dataflow/LdapInjectionCustomizations.qll
View File

@@ -41,20 +41,6 @@ module LdapInjection {
*/
abstract class FilterSanitizer extends DataFlow::Node { }
/**
* DEPRECATED: Use `DnSanitizer` instead.
*
* A sanitizer guard for "ldap injection" vulnerabilities.
*/
abstract deprecated class DnSanitizerGuard extends DataFlow::BarrierGuard { }
/**
* DEPRECATED: Use `FilterSanitizer` instead.
*
* A sanitizer guard for "ldap injection" vulnerabilities.
*/
abstract deprecated class FilterSanitizerGuard extends DataFlow::BarrierGuard { }
/**
* A source of remote user input, considered as a flow source.
*/

8
python/ql/lib/semmle/python/security/dataflow/LdapInjectionQuery.qll
View File

@@ -27,10 +27,6 @@ deprecated class DnConfiguration extends TaintTracking::Configuration {
override predicate isSink(DataFlow::Node sink) { sink instanceof DnSink }
override predicate isSanitizer(DataFlow::Node node) { node instanceof DnSanitizer }
deprecated override predicate isSanitizerGuard(DataFlow::BarrierGuard guard) {
guard instanceof DnSanitizerGuard
}
}
private module LdapInjectionDnConfig implements DataFlow::ConfigSig {
@@ -58,10 +54,6 @@ deprecated class FilterConfiguration extends TaintTracking::Configuration {
override predicate isSink(DataFlow::Node sink) { sink instanceof FilterSink }
override predicate isSanitizer(DataFlow::Node node) { node instanceof FilterSanitizer }
deprecated override predicate isSanitizerGuard(DataFlow::BarrierGuard guard) {
guard instanceof FilterSanitizerGuard
}
}
private module LdapInjectionFilterConfig implements DataFlow::ConfigSig {

7
python/ql/lib/semmle/python/security/dataflow/LogInjectionCustomizations.qll
View File

@@ -31,13 +31,6 @@ module LogInjection {
*/
abstract class Sanitizer extends DataFlow::Node { }
/**
* DEPRECATED: Use `Sanitizer` instead.
*
* A sanitizer guard for "log injection" vulnerabilities.
*/
abstract deprecated class SanitizerGuard extends DataFlow::BarrierGuard { }
/**
* A source of remote user input, considered as a flow source.
*/

4
python/ql/lib/semmle/python/security/dataflow/LogInjectionQuery.qll
View File

@@ -24,10 +24,6 @@ deprecated class Configuration extends TaintTracking::Configuration {
override predicate isSink(DataFlow::Node sink) { sink instanceof Sink }
override predicate isSanitizer(DataFlow::Node node) { node instanceof Sanitizer }
deprecated override predicate isSanitizerGuard(DataFlow::BarrierGuard guard) {
guard instanceof SanitizerGuard
}
}
private module LogInjectionConfig implements DataFlow::ConfigSig {

7
python/ql/lib/semmle/python/security/dataflow/PathInjectionCustomizations.qll
View File

@@ -42,13 +42,6 @@ module PathInjection {
*/
abstract class Sanitizer extends DataFlow::Node { }
/**
* DEPRECATED: Use `Sanitizer` instead.
*
* A sanitizer guard for "path injection" vulnerabilities.
*/
abstract deprecated class SanitizerGuard extends DataFlow::BarrierGuard { }
/**
* A source of remote user input, considered as a flow source.
*/

4
python/ql/lib/semmle/python/security/dataflow/PathInjectionQuery.qll
View File

@@ -53,10 +53,6 @@ deprecated class Configuration extends TaintTracking::Configuration {
state instanceof NormalizedUnchecked
}
deprecated override predicate isSanitizerGuard(DataFlow::BarrierGuard guard) {
guard instanceof SanitizerGuard
}
override predicate isAdditionalTaintStep(
DataFlow::Node nodeFrom, DataFlow::FlowState stateFrom, DataFlow::Node nodeTo,
DataFlow::FlowState stateTo

7
python/ql/lib/semmle/python/security/dataflow/PolynomialReDoSCustomizations.qll
View File

@@ -46,13 +46,6 @@ module PolynomialReDoS {
*/
abstract class Sanitizer extends DataFlow::Node { }
/**
* DEPRECATED: Use `Sanitizer` instead.
*
* A sanitizer guard for "polynomial regular expression denial of service (ReDoS)" vulnerabilities.
*/
abstract deprecated class SanitizerGuard extends DataFlow::BarrierGuard { }
/**
* A source of remote user input, considered as a flow source.
*/

4
python/ql/lib/semmle/python/security/dataflow/PolynomialReDoSQuery.qll
View File

@@ -24,10 +24,6 @@ deprecated class Configuration extends TaintTracking::Configuration {
override predicate isSink(DataFlow::Node sink) { sink instanceof Sink }
override predicate isSanitizer(DataFlow::Node node) { node instanceof Sanitizer }
deprecated override predicate isSanitizerGuard(DataFlow::BarrierGuard guard) {
guard instanceof SanitizerGuard
}
}
private module PolynomialReDoSConfig implements DataFlow::ConfigSig {

7
python/ql/lib/semmle/python/security/dataflow/ReflectedXSSCustomizations.qll
View File

@@ -32,13 +32,6 @@ module ReflectedXss {
*/
abstract class Sanitizer extends DataFlow::Node { }
/**
* DEPRECATED: Use `Sanitizer` instead.
*
* A sanitizer guard for "reflected server-side cross-site scripting" vulnerabilities.
*/
abstract deprecated class SanitizerGuard extends DataFlow::BarrierGuard { }
/**
* A source of remote user input, considered as a flow source.
*/

4
python/ql/lib/semmle/python/security/dataflow/ReflectedXssQuery.qll
View File

@@ -24,10 +24,6 @@ deprecated class Configuration extends TaintTracking::Configuration {
override predicate isSink(DataFlow::Node sink) { sink instanceof Sink }
override predicate isSanitizer(DataFlow::Node node) { node instanceof Sanitizer }
deprecated override predicate isSanitizerGuard(DataFlow::BarrierGuard guard) {
guard instanceof SanitizerGuard
}
}
private module ReflectedXssConfig implements DataFlow::ConfigSig {

7
python/ql/lib/semmle/python/security/dataflow/RegexInjectionCustomizations.qll
View File

@@ -39,13 +39,6 @@ module RegexInjection {
*/
abstract class Sanitizer extends DataFlow::Node { }
/**
* DEPRECATED: Use `Sanitizer` instead.
*
* A sanitizer guard for "regular expression injection" vulnerabilities.
*/
abstract deprecated class SanitizerGuard extends DataFlow::BarrierGuard { }
/**
* A source of remote user input, considered as a flow source.
*/

4
python/ql/lib/semmle/python/security/dataflow/RegexInjectionQuery.qll
View File

@@ -25,10 +25,6 @@ deprecated class Configuration extends TaintTracking::Configuration {
override predicate isSink(DataFlow::Node sink) { sink instanceof Sink }
override predicate isSanitizer(DataFlow::Node node) { node instanceof Sanitizer }
deprecated override predicate isSanitizerGuard(DataFlow::BarrierGuard guard) {
guard instanceof SanitizerGuard
}
}
private module RegexInjectionConfig implements DataFlow::ConfigSig {

7
python/ql/lib/semmle/python/security/dataflow/ServerSideRequestForgeryCustomizations.qll
View File

@@ -43,13 +43,6 @@ module ServerSideRequestForgery {
*/
abstract class FullUrlControlSanitizer extends DataFlow::Node { }
/**
* DEPRECATED: Use `Sanitizer` instead.
*
* A sanitizer guard for "Server-side request forgery" vulnerabilities.
*/
abstract deprecated class SanitizerGuard extends DataFlow::BarrierGuard { }
/**
* A source of remote user input, considered as a flow source.
*/

8
python/ql/lib/semmle/python/security/dataflow/ServerSideRequestForgeryQuery.qll
View File

@@ -35,10 +35,6 @@ deprecated class FullServerSideRequestForgeryConfiguration extends TaintTracking
or
node instanceof FullUrlControlSanitizer
}
deprecated override predicate isSanitizerGuard(DataFlow::BarrierGuard guard) {
guard instanceof SanitizerGuard
}
}
/**
@@ -93,10 +89,6 @@ deprecated class PartialServerSideRequestForgeryConfiguration extends TaintTrack
override predicate isSink(DataFlow::Node sink) { sink instanceof Sink }
override predicate isSanitizer(DataFlow::Node node) { node instanceof Sanitizer }
deprecated override predicate isSanitizerGuard(DataFlow::BarrierGuard guard) {
guard instanceof SanitizerGuard
}
}
/**

7
python/ql/lib/semmle/python/security/dataflow/SqlInjectionCustomizations.qll
View File

@@ -31,13 +31,6 @@ module SqlInjection {
*/
abstract class Sanitizer extends DataFlow::Node { }
/**
* DEPRECATED: Use `Sanitizer` instead.
*
* A sanitizer guard for "SQL injection" vulnerabilities.
*/
abstract deprecated class SanitizerGuard extends DataFlow::BarrierGuard { }
/**
* A source of remote user input, considered as a flow source.
*/

4
python/ql/lib/semmle/python/security/dataflow/SqlInjectionQuery.qll
View File

@@ -24,10 +24,6 @@ deprecated class Configuration extends TaintTracking::Configuration {
override predicate isSink(DataFlow::Node sink) { sink instanceof Sink }
override predicate isSanitizer(DataFlow::Node node) { node instanceof Sanitizer }
deprecated override predicate isSanitizerGuard(DataFlow::BarrierGuard guard) {
guard instanceof SanitizerGuard
}
}
private module SqlInjectionConfig implements DataFlow::ConfigSig {

7
python/ql/lib/semmle/python/security/dataflow/StackTraceExposureCustomizations.qll
View File

@@ -31,13 +31,6 @@ module StackTraceExposure {
*/
abstract class Sanitizer extends DataFlow::Node { }
/**
* DEPRECATED: Use `Sanitizer` instead.
*
* A sanitizer guard for "stack trace exposure" vulnerabilities.
*/
abstract deprecated class SanitizerGuard extends DataFlow::BarrierGuard { }
/**
* A source of exception info, considered as a flow source.
*/

4
python/ql/lib/semmle/python/security/dataflow/StackTraceExposureQuery.qll
View File

@@ -25,10 +25,6 @@ deprecated class Configuration extends TaintTracking::Configuration {
override predicate isSanitizer(DataFlow::Node node) { node instanceof Sanitizer }
deprecated override predicate isSanitizerGuard(DataFlow::BarrierGuard guard) {
guard instanceof SanitizerGuard
}
// A stack trace is accessible as the `__traceback__` attribute of a caught exception.
// see https://docs.python.org/3/reference/datamodel.html#traceback-objects
override predicate isAdditionalTaintStep(DataFlow::Node nodeFrom, DataFlow::Node nodeTo) {

7
python/ql/lib/semmle/python/security/dataflow/UnsafeDeserializationCustomizations.qll
View File

@@ -31,13 +31,6 @@ module UnsafeDeserialization {
*/
abstract class Sanitizer extends DataFlow::Node { }
/**
* DEPRECATED: Use `Sanitizer` instead.
*
* A sanitizer guard for "code execution from deserialization" vulnerabilities.
*/
abstract deprecated class SanitizerGuard extends DataFlow::BarrierGuard { }
/**
* A source of remote user input, considered as a flow source.
*/

4
python/ql/lib/semmle/python/security/dataflow/UnsafeDeserializationQuery.qll
View File

@@ -24,10 +24,6 @@ deprecated class Configuration extends TaintTracking::Configuration {
override predicate isSink(DataFlow::Node sink) { sink instanceof Sink }
override predicate isSanitizer(DataFlow::Node node) { node instanceof Sanitizer }
deprecated override predicate isSanitizerGuard(DataFlow::BarrierGuard guard) {
guard instanceof SanitizerGuard
}
}
private module UnsafeDeserializationConfig implements DataFlow::ConfigSig {

7
python/ql/lib/semmle/python/security/dataflow/UrlRedirectCustomizations.qll
View File

@@ -31,13 +31,6 @@ module UrlRedirect {
*/
abstract class Sanitizer extends DataFlow::Node { }
/**
* DEPRECATED: Use `Sanitizer` instead.
*
* A sanitizer guard for "URL redirection" vulnerabilities.
*/
abstract deprecated class SanitizerGuard extends DataFlow::BarrierGuard { }
/**
* A source of remote user input, considered as a flow source.
*/

4
python/ql/lib/semmle/python/security/dataflow/UrlRedirectQuery.qll
View File

@@ -24,10 +24,6 @@ deprecated class Configuration extends TaintTracking::Configuration {
override predicate isSink(DataFlow::Node sink) { sink instanceof Sink }
override predicate isSanitizer(DataFlow::Node node) { node instanceof Sanitizer }
deprecated override predicate isSanitizerGuard(DataFlow::BarrierGuard guard) {
guard instanceof SanitizerGuard
}
}
private module UrlRedirectConfig implements DataFlow::ConfigSig {

7
python/ql/lib/semmle/python/security/dataflow/XpathInjectionCustomizations.qll
View File

@@ -29,13 +29,6 @@ module XpathInjection {
*/
abstract class Sanitizer extends DataFlow::Node { }
/**
* DEPRECATED: Use `Sanitizer` instead.
*
* A sanitizer guard for "XPath injection" vulnerabilities.
*/
abstract deprecated class SanitizerGuard extends DataFlow::BarrierGuard { }
/**
* A source of remote user input, considered as a flow source.
*/

4
python/ql/lib/semmle/python/security/dataflow/XpathInjectionQuery.qll
View File

@@ -24,10 +24,6 @@ deprecated class Configuration extends TaintTracking::Configuration {
override predicate isSink(DataFlow::Node sink) { sink instanceof Sink }
override predicate isSanitizer(DataFlow::Node node) { node instanceof Sanitizer }
deprecated override predicate isSanitizerGuard(DataFlow::BarrierGuard guard) {
guard instanceof SanitizerGuard
}
}
private module XpathInjectionConfig implements DataFlow::ConfigSig {

4
python/ql/lib/semmle/python/security/performance/ExponentialBackTracking.qll
View File

@@ -1,4 +0,0 @@
/** DEPRECATED. Import `semmle.python.security.regexp.ExponentialBackTracking` instead. */
deprecated import semmle.python.security.regexp.ExponentialBackTracking as Dep
import Dep

4
python/ql/lib/semmle/python/security/performance/ReDoSUtil.qll
View File

@@ -1,4 +0,0 @@
/** DEPRECATED. Import `semmle.python.security.regexp.NfaUtils` instead. */
deprecated import semmle.python.security.regexp.NfaUtils as Dep
import Dep

4
python/ql/lib/semmle/python/security/performance/SuperlinearBackTracking.qll
View File

@@ -1,4 +0,0 @@
/** DEPRECATED. Import `semmle.python.security.regexp.SuperlinearBackTracking` instead. */
deprecated import semmle.python.security.regexp.SuperlinearBackTracking as Dep
import Dep

124
python/ql/lib/semmle/python/security/strings/Basic.qll
View File

@@ -1,124 +0,0 @@
import python
private import Common
import semmle.python.dataflow.TaintTracking
/** An extensible kind of taint representing any kind of string. */
abstract deprecated class StringKind extends TaintKind {
bindingset[this]
StringKind() { this = this }
override TaintKind getTaintOfMethodResult(string name) {
name in [
"capitalize", "casefold", "center", "expandtabs", "format", "format_map", "ljust", "lstrip",
"lower", "replace", "rjust", "rstrip", "strip", "swapcase", "title", "upper", "zfill",
/* encode/decode is technically not correct, but close enough */
"encode", "decode"
] and
result = this
or
name in ["partition", "rpartition", "rsplit", "split", "splitlines"] and
result.(SequenceKind).getItem() = this
}
override TaintKind getTaintForFlowStep(ControlFlowNode fromnode, ControlFlowNode tonode) {
result = this and
(
slice(fromnode, tonode) or
tonode.(BinaryExprNode).getAnOperand() = fromnode or
os_path_join(fromnode, tonode) or
str_format(fromnode, tonode) or
encode_decode(fromnode, tonode) or
to_str(fromnode, tonode) or
f_string(fromnode, tonode)
)
or
result = this and copy_call(fromnode, tonode)
}
override ClassValue getType() {
result = Value::named("bytes") or
result = Value::named("str") or
result = Value::named("unicode")
}
}
deprecated private class StringEqualitySanitizer extends Sanitizer {
StringEqualitySanitizer() { this = "string equality sanitizer" }
/* The test `if untrusted == "KNOWN_VALUE":` sanitizes `untrusted` on its `true` edge. */
override predicate sanitizingEdge(TaintKind taint, PyEdgeRefinement test) {
taint instanceof StringKind and
exists(ControlFlowNode const, Cmpop op | const.getNode() instanceof StrConst |
(
test.getTest().(CompareNode).operands(const, op, _)
or
test.getTest().(CompareNode).operands(_, op, const)
) and
(
op instanceof Eq and test.getSense() = true
or
op instanceof NotEq and test.getSense() = false
)
)
}
}
/** tonode = ....format(fromnode) */
deprecated private predicate str_format(ControlFlowNode fromnode, CallNode tonode) {
tonode.getFunction().(AttrNode).getName() = "format" and
tonode.getAnArg() = fromnode
}
/** tonode = codec.[en|de]code(fromnode) */
deprecated private predicate encode_decode(ControlFlowNode fromnode, CallNode tonode) {
exists(FunctionObject func, string name |
not func.getFunction().isMethod() and
func.getACall() = tonode and
tonode.getAnArg() = fromnode and
func.getName() = name
|
name = "encode" or
name = "decode" or
name = "decodestring"
)
}
/** tonode = str(fromnode) */
deprecated private predicate to_str(ControlFlowNode fromnode, CallNode tonode) {
tonode.getAnArg() = fromnode and
(
tonode = ClassValue::bytes().getACall()
or
tonode = ClassValue::unicode().getACall()
)
}
/** tonode = fromnode[:] */
deprecated private predicate slice(ControlFlowNode fromnode, SubscriptNode tonode) {
exists(Slice all |
all = tonode.getIndex().getNode() and
not exists(all.getStart()) and
not exists(all.getStop()) and
tonode.getObject() = fromnode
)
}
/** tonode = os.path.join(..., fromnode, ...) */
deprecated private predicate os_path_join(ControlFlowNode fromnode, CallNode tonode) {
tonode = Value::named("os.path.join").getACall() and
tonode.getAnArg() = fromnode
}
/** tonode = f"... {fromnode} ..." */
deprecated private predicate f_string(ControlFlowNode fromnode, ControlFlowNode tonode) {
tonode.getNode().(Fstring).getAValue() = fromnode.getNode()
}
/**
* A kind of "taint", representing a dictionary mapping str->"taint"
*
* DEPRECATED: Use `ExternalStringDictKind` instead.
*/
deprecated class StringDictKind extends DictKind {
StringDictKind() { this.getValue() instanceof StringKind }
}

14
python/ql/lib/semmle/python/security/strings/Common.qll
View File

@@ -1,14 +0,0 @@
import python
/** A call that returns a copy (or similar) of the argument */
deprecated predicate copy_call(ControlFlowNode fromnode, CallNode tonode) {
tonode.getFunction().(AttrNode).getObject("copy") = fromnode
or
exists(ModuleValue copy, string name | name = "copy" or name = "deepcopy" |
copy.attr(name).(FunctionValue).getACall() = tonode and
tonode.getArg(0) = fromnode
)
or
tonode.getFunction().pointsTo(Value::named("reversed")) and
tonode.getArg(0) = fromnode
}

318
python/ql/lib/semmle/python/security/strings/External.qll
View File

@@ -1,318 +0,0 @@
import python
import Basic
private import Common
/**
* An extensible kind of taint representing an externally controlled string.
*/
abstract deprecated class ExternalStringKind extends StringKind {
bindingset[this]
ExternalStringKind() { this = this }
override TaintKind getTaintForFlowStep(ControlFlowNode fromnode, ControlFlowNode tonode) {
result = StringKind.super.getTaintForFlowStep(fromnode, tonode)
or
tonode.(SequenceNode).getElement(_) = fromnode and
result.(ExternalStringSequenceKind).getItem() = this
or
json_load(fromnode, tonode) and result.(ExternalJsonKind).getValue() = this
or
tonode.(DictNode).getAValue() = fromnode and result.(ExternalStringDictKind).getValue() = this
or
urlsplit(fromnode, tonode) and result.(ExternalUrlSplitResult).getItem() = this
or
urlparse(fromnode, tonode) and result.(ExternalUrlParseResult).getItem() = this
or
parse_qs(fromnode, tonode) and result.(ExternalStringDictKind).getValue() = this
or
parse_qsl(fromnode, tonode) and result.(SequenceKind).getItem().(SequenceKind).getItem() = this
}
}
/** A kind of "taint", representing a sequence, with a "taint" member */
deprecated class ExternalStringSequenceKind extends SequenceKind {
ExternalStringSequenceKind() { this.getItem() instanceof ExternalStringKind }
}
/**
* An hierarchical dictionary or list where the entire structure is externally controlled
* This is typically a parsed JSON object.
*/
deprecated class ExternalJsonKind extends TaintKind {
ExternalJsonKind() { this = "json[" + any(ExternalStringKind key) + "]" }
/** Gets the taint kind for item in this sequence */
TaintKind getValue() {
this = "json[" + result + "]"
or
result = this
}
override TaintKind getTaintForFlowStep(ControlFlowNode fromnode, ControlFlowNode tonode) {
this.taints(fromnode) and
json_subscript_taint(tonode, fromnode, this, result)
or
result = this and copy_call(fromnode, tonode)
}
override TaintKind getTaintOfMethodResult(string name) {
name = "get" and result = this.getValue()
}
}
/** A kind of "taint", representing a dictionary mapping keys to tainted strings. */
deprecated class ExternalStringDictKind extends DictKind {
ExternalStringDictKind() { this.getValue() instanceof ExternalStringKind }
}
/**
* A kind of "taint", representing a dictionary mapping keys to sequences of
* tainted strings.
*/
deprecated class ExternalStringSequenceDictKind extends DictKind {
ExternalStringSequenceDictKind() { this.getValue() instanceof ExternalStringSequenceKind }
}
/** TaintKind for the result of `urlsplit(tainted_string)` */
deprecated class ExternalUrlSplitResult extends ExternalStringSequenceKind {
// https://docs.python.org/3/library/urllib.parse.html#urllib.parse.urlsplit
override TaintKind getTaintOfAttribute(string name) {
result = super.getTaintOfAttribute(name)
or
name in [
// namedtuple field names
"scheme", "netloc", "path", "query", "fragment",
// class methods
"password", "username", "hostname",
] and
result instanceof ExternalStringKind
}
override TaintKind getTaintOfMethodResult(string name) {
result = super.getTaintOfMethodResult(name)
or
name = "geturl" and
result instanceof ExternalStringKind
}
}
/** TaintKind for the result of `urlparse(tainted_string)` */
deprecated class ExternalUrlParseResult extends ExternalStringSequenceKind {
// https://docs.python.org/3/library/urllib.parse.html#urllib.parse.urlparse
override TaintKind getTaintOfAttribute(string name) {
result = super.getTaintOfAttribute(name)
or
name in [
// namedtuple field names
"scheme", "netloc", "path", "params", "query", "fragment",
// class methods
"username", "password", "hostname",
] and
result instanceof ExternalStringKind
}
override TaintKind getTaintOfMethodResult(string name) {
result = super.getTaintOfMethodResult(name)
or
name = "geturl" and
result instanceof ExternalStringKind
}
}
/* Helper for getTaintForStep() */
pragma[noinline]
deprecated private predicate json_subscript_taint(
SubscriptNode sub, ControlFlowNode obj, ExternalJsonKind seq, TaintKind key
) {
sub.isLoad() and
sub.getObject() = obj and
key = seq.getValue()
}
deprecated private predicate json_load(ControlFlowNode fromnode, CallNode tonode) {
tonode = Value::named("json.loads").getACall() and
tonode.getArg(0) = fromnode
}
deprecated private predicate urlsplit(ControlFlowNode fromnode, CallNode tonode) {
// This could be implemented as `exists(FunctionValue` without the explicit six part,
// but then our tests will need to import +100 modules, so for now this slightly
// altered version gets to live on.
exists(Value urlsplit |
(
urlsplit = Value::named("six.moves.urllib.parse.urlsplit")
or
// Python 2
urlsplit = Value::named("urlparse.urlsplit")
or
// Python 3
urlsplit = Value::named("urllib.parse.urlsplit")
) and
tonode = urlsplit.getACall() and
tonode.getArg(0) = fromnode
)
}
deprecated private predicate urlparse(ControlFlowNode fromnode, CallNode tonode) {
// This could be implemented as `exists(FunctionValue` without the explicit six part,
// but then our tests will need to import +100 modules, so for now this slightly
// altered version gets to live on.
exists(Value urlparse |
(
urlparse = Value::named("six.moves.urllib.parse.urlparse")
or
// Python 2
urlparse = Value::named("urlparse.urlparse")
or
// Python 3
urlparse = Value::named("urllib.parse.urlparse")
) and
tonode = urlparse.getACall() and
tonode.getArg(0) = fromnode
)
}
deprecated private predicate parse_qs(ControlFlowNode fromnode, CallNode tonode) {
// This could be implemented as `exists(FunctionValue` without the explicit six part,
// but then our tests will need to import +100 modules, so for now this slightly
// altered version gets to live on.
exists(Value parse_qs |
(
parse_qs = Value::named("six.moves.urllib.parse.parse_qs")
or
// Python 2
parse_qs = Value::named("urlparse.parse_qs")
or
// Python 2 deprecated version of `urlparse.parse_qs`
parse_qs = Value::named("cgi.parse_qs")
or
// Python 3
parse_qs = Value::named("urllib.parse.parse_qs")
) and
tonode = parse_qs.getACall() and
(
tonode.getArg(0) = fromnode
or
tonode.getArgByName("qs") = fromnode
)
)
}
deprecated private predicate parse_qsl(ControlFlowNode fromnode, CallNode tonode) {
// This could be implemented as `exists(FunctionValue` without the explicit six part,
// but then our tests will need to import +100 modules, so for now this slightly
// altered version gets to live on.
exists(Value parse_qsl |
(
parse_qsl = Value::named("six.moves.urllib.parse.parse_qsl")
or
// Python 2
parse_qsl = Value::named("urlparse.parse_qsl")
or
// Python 2 deprecated version of `urlparse.parse_qsl`
parse_qsl = Value::named("cgi.parse_qsl")
or
// Python 3
parse_qsl = Value::named("urllib.parse.parse_qsl")
) and
tonode = parse_qsl.getACall() and
(
tonode.getArg(0) = fromnode
or
tonode.getArgByName("qs") = fromnode
)
)
}
/** A kind of "taint", representing an open file-like object from an external source. */
deprecated class ExternalFileObject extends TaintKind {
ExternalStringKind valueKind;
ExternalFileObject() { this = "file[" + valueKind + "]" }
/** Gets the taint kind for the contents of this file */
TaintKind getValue() { result = valueKind }
override TaintKind getTaintOfMethodResult(string name) {
name in ["read", "readline"] and result = this.getValue()
or
name = "readlines" and result.(SequenceKind).getItem() = this.getValue()
}
override TaintKind getTaintForIteration() { result = this.getValue() }
}
/**
* Temporary sanitizer for the tainted result from `urlsplit` and `urlparse`. Can be used to reduce FPs until
* we have better support for namedtuples.
*
* Will clear **all** taint on a test of the kind. That is, on the true edge of any matching test,
* all fields/indexes will be cleared of taint.
*
* Handles:
* - `if splitres.netloc == "KNOWN_VALUE"`
* - `if splitres[0] == "KNOWN_VALUE"`
*/
deprecated class UrlsplitUrlparseTempSanitizer extends Sanitizer {
// TODO: remove this once we have better support for named tuples
UrlsplitUrlparseTempSanitizer() { this = "UrlsplitUrlparseTempSanitizer" }
override predicate sanitizingEdge(TaintKind taint, PyEdgeRefinement test) {
(
taint instanceof ExternalUrlSplitResult
or
taint instanceof ExternalUrlParseResult
) and
exists(ControlFlowNode full_use |
full_use.(SubscriptNode).getObject() = test.getInput().getAUse()
or
full_use.(AttrNode).getObject() = test.getInput().getAUse()
|
this.clears_taint(full_use, test.getTest(), test.getSense())
)
}
private predicate clears_taint(ControlFlowNode tainted, ControlFlowNode test, boolean sense) {
this.test_equality_with_const(test, tainted, sense)
or
this.test_in_const_seq(test, tainted, sense)
or
test.(UnaryExprNode).getNode().getOp() instanceof Not and
exists(ControlFlowNode nested_test |
nested_test = test.(UnaryExprNode).getOperand() and
this.clears_taint(tainted, nested_test, sense.booleanNot())
)
}
/** holds for `== "KNOWN_VALUE"` on `true` edge, and `!= "KNOWN_VALUE"` on `false` edge */
private predicate test_equality_with_const(CompareNode cmp, ControlFlowNode tainted, boolean sense) {
exists(ControlFlowNode const, Cmpop op | const.getNode() instanceof StrConst |
(
cmp.operands(const, op, tainted)
or
cmp.operands(tainted, op, const)
) and
(
op instanceof Eq and sense = true
or
op instanceof NotEq and sense = false
)
)
}
/** holds for `in ["KNOWN_VALUE", ...]` on `true` edge, and `not in ["KNOWN_VALUE", ...]` on `false` edge */
private predicate test_in_const_seq(CompareNode cmp, ControlFlowNode tainted, boolean sense) {
exists(SequenceNode const_seq, Cmpop op |
forall(ControlFlowNode elem | elem = const_seq.getAnElement() |
elem.getNode() instanceof StrConst
)
|
cmp.operands(tainted, op, const_seq) and
(
op instanceof In and sense = true
or
op instanceof NotIn and sense = false
)
)
}
}

10
python/ql/lib/semmle/python/security/strings/Untrusted.qll
View File

@@ -1,10 +0,0 @@
import python
import External
/**
* A kind of taint representing an externally controlled string.
* This class is a simple sub-class of `ExternalStringKind`.
*/
deprecated class UntrustedStringKind extends ExternalStringKind {
UntrustedStringKind() { this = "externally controlled string" }
}

119
python/ql/lib/semmle/python/web/Http.qll
View File

@@ -1,119 +0,0 @@
import python
import semmle.python.dataflow.Implementation
import semmle.python.security.strings.External
import HttpConstants
/** Generic taint source from a http request */
abstract deprecated class HttpRequestTaintSource extends TaintSource { }
/**
* Taint kind representing the WSGI environment.
* As specified in PEP 3333. https://www.python.org/dev/peps/pep-3333/#environ-variables
*/
deprecated class WsgiEnvironment extends TaintKind {
WsgiEnvironment() { this = "wsgi.environment" }
override TaintKind getTaintForFlowStep(ControlFlowNode fromnode, ControlFlowNode tonode) {
result = this and Implementation::copyCall(fromnode, tonode)
or
result = this and
tonode.(CallNode).getFunction().pointsTo(ClassValue::dict()) and
tonode.(CallNode).getArg(0) = fromnode
or
exists(Value key, string text |
tonode.(CallNode).getFunction().(AttrNode).getObject("get") = fromnode and
tonode.(CallNode).getArg(0).pointsTo(key)
or
tonode.(SubscriptNode).getObject() = fromnode and
tonode.isLoad() and
tonode.(SubscriptNode).getIndex().pointsTo(key)
|
key = Value::forString(text) and
result instanceof ExternalStringKind and
(
text = "QUERY_STRING" or
text = "PATH_INFO" or
text.matches("HTTP\\_%")
)
)
}
}
/**
* A standard morsel object from a HTTP request, a value in a cookie,
* typically an instance of `http.cookies.Morsel`
*/
deprecated class UntrustedMorsel extends TaintKind {
UntrustedMorsel() { this = "http.Morsel" }
override TaintKind getTaintOfAttribute(string name) {
result instanceof ExternalStringKind and
name = "value"
}
}
/** A standard cookie object from a HTTP request, typically an instance of `http.cookies.SimpleCookie` */
deprecated class UntrustedCookie extends TaintKind {
UntrustedCookie() { this = "http.Cookie" }
override TaintKind getTaintForFlowStep(ControlFlowNode fromnode, ControlFlowNode tonode) {
tonode.(SubscriptNode).getObject() = fromnode and
result instanceof UntrustedMorsel
}
}
abstract deprecated class CookieOperation extends @py_flow_node {
/** Gets a textual representation of this element. */
abstract string toString();
abstract ControlFlowNode getKey();
abstract ControlFlowNode getValue();
}
abstract deprecated class CookieGet extends CookieOperation { }
abstract deprecated class CookieSet extends CookieOperation { }
/** Generic taint sink in a http response */
abstract deprecated class HttpResponseTaintSink extends TaintSink {
override predicate sinks(TaintKind kind) { kind instanceof ExternalStringKind }
}
abstract deprecated class HttpRedirectTaintSink extends TaintSink {
override predicate sinks(TaintKind kind) { kind instanceof ExternalStringKind }
}
deprecated module Client {
// TODO: user-input in other than URL:
// - `data`, `json` for `requests.post`
// - `body` for `HTTPConnection.request`
// - headers?
// TODO: Add more library support
// - urllib3 https://github.com/urllib3/urllib3
// - httpx https://github.com/encode/httpx
/**
* An outgoing http request
*
* For example:
* conn = HTTPConnection('example.com')
* conn.request('GET', '/path')
*/
abstract class HttpRequest extends ControlFlowNode {
/**
* Get any ControlFlowNode that is used to construct the final URL.
*
* In the HTTPConnection example, there is a result for both `'example.com'` and for `'/path'`.
*/
abstract ControlFlowNode getAUrlPart();
abstract string getMethodUpper();
}
/** Taint sink for the URL-part of an outgoing http request */
class HttpRequestUrlTaintSink extends TaintSink {
HttpRequestUrlTaintSink() { this = any(HttpRequest r).getAUrlPart() }
override predicate sinks(TaintKind kind) { kind instanceof ExternalStringKind }
}
}

7
python/ql/lib/semmle/python/web/HttpConstants.qll
View File

@@ -1,7 +0,0 @@
/** Gets an HTTP verb, in upper case */
deprecated string httpVerb() {
result in ["GET", "POST", "PUT", "PATCH", "DELETE", "OPTIONS", "HEAD"]
}
/** Gets an HTTP verb, in lower case */
deprecated string httpVerbLower() { result = httpVerb().toLowerCase() }

10
python/ql/lib/semmle/python/web/HttpRequest.qll
View File

@@ -1,10 +0,0 @@
import semmle.python.web.django.Request
import semmle.python.web.flask.Request
import semmle.python.web.tornado.Request
import semmle.python.web.pyramid.Request
import semmle.python.web.twisted.Request
import semmle.python.web.bottle.Request
import semmle.python.web.turbogears.Request
import semmle.python.web.falcon.Request
import semmle.python.web.cherrypy.Request
import semmle.python.web.stdlib.Request

46
python/ql/lib/semmle/python/web/bottle/General.qll
View File

@@ -1,46 +0,0 @@
import python
import semmle.python.web.Http
import semmle.python.types.Extensions
/** Gets the bottle module */
deprecated ModuleValue theBottleModule() { result = Module::named("bottle") }
/** Gets the bottle.Bottle class */
deprecated ClassValue theBottleClass() { result = theBottleModule().attr("Bottle") }
/**
* Holds if `route` is routed to `func`
* by decorating `func` with `app.route(route)` or `route(route)`
*/
deprecated predicate bottle_route(CallNode route_call, ControlFlowNode route, Function func) {
exists(CallNode decorator_call, string name |
route_call.getFunction().(AttrNode).getObject(name).pointsTo().getClass() = theBottleClass() or
route_call.getFunction().pointsTo(theBottleModule().attr(name))
|
(name = "route" or name = httpVerbLower()) and
decorator_call.getFunction() = route_call and
route_call.getArg(0) = route and
decorator_call.getArg(0).getNode().(FunctionExpr).getInnerScope() = func
)
}
deprecated class BottleRoute extends ControlFlowNode {
BottleRoute() { bottle_route(this, _, _) }
string getUrl() {
exists(StrConst url |
bottle_route(this, url.getAFlowNode(), _) and
result = url.getText()
)
}
Function getFunction() { bottle_route(this, _, result) }
Parameter getANamedArgument() {
exists(string name, Function func |
func = this.getFunction() and
func.getArgByName(name) = result and
this.getUrl().matches("%<" + name + ">%")
)
}
}

80
python/ql/lib/semmle/python/web/bottle/Request.qll
View File

@@ -1,80 +0,0 @@
import python
import semmle.python.dataflow.TaintTracking
import semmle.python.security.strings.External
import semmle.python.web.Http
import semmle.python.web.bottle.General
deprecated private Value theBottleRequestObject() { result = theBottleModule().attr("request") }
deprecated class BottleRequestKind extends TaintKind {
BottleRequestKind() { this = "bottle.request" }
override TaintKind getTaintOfAttribute(string name) {
result instanceof BottleFormsDict and
(name = "cookies" or name = "query" or name = "form")
or
result instanceof ExternalStringKind and
(name = "query_string" or name = "url_args")
or
result.(DictKind).getValue() instanceof FileUpload and
name = "files"
}
}
deprecated private class RequestSource extends HttpRequestTaintSource {
RequestSource() { this.(ControlFlowNode).pointsTo(theBottleRequestObject()) }
override predicate isSourceOf(TaintKind kind) { kind instanceof BottleRequestKind }
}
deprecated class BottleFormsDict extends TaintKind {
BottleFormsDict() { this = "bottle.FormsDict" }
override TaintKind getTaintForFlowStep(ControlFlowNode fromnode, ControlFlowNode tonode) {
/* Cannot use `getTaintOfAttribute(name)` as it wouldn't bind `name` */
exists(string name |
fromnode = tonode.(AttrNode).getObject(name) and
result instanceof ExternalStringKind
|
name != "get" and name != "getunicode" and name != "getall"
)
}
override TaintKind getTaintOfMethodResult(string name) {
(name = "get" or name = "getunicode") and
result instanceof ExternalStringKind
or
name = "getall" and result.(SequenceKind).getItem() instanceof ExternalStringKind
}
}
deprecated class FileUpload extends TaintKind {
FileUpload() { this = "bottle.FileUpload" }
override TaintKind getTaintOfAttribute(string name) {
name = "filename" and result instanceof ExternalStringKind
or
name = "raw_filename" and result instanceof ExternalStringKind
or
name = "file" and result instanceof UntrustedFile
}
}
deprecated class UntrustedFile extends TaintKind {
UntrustedFile() { this = "Untrusted file" }
}
//
// TO DO.. File uploads -- Should check about file uploads for other frameworks as well.
// Move UntrustedFile to shared location
//
/** A parameter to a bottle request handler function */
deprecated class BottleRequestParameter extends HttpRequestTaintSource {
BottleRequestParameter() {
exists(BottleRoute route | route.getANamedArgument() = this.(ControlFlowNode).getNode())
}
override predicate isSourceOf(TaintKind kind) { kind instanceof ExternalStringKind }
override string toString() { result = "bottle handler function argument" }
}

44
python/ql/lib/semmle/python/web/cherrypy/General.qll
View File

@@ -1,44 +0,0 @@
import python
import semmle.python.web.Http
deprecated module CherryPy {
FunctionValue expose() { result = Value::named("cherrypy.expose") }
}
deprecated class CherryPyExposedFunction extends Function {
CherryPyExposedFunction() {
this.getADecorator().pointsTo(CherryPy::expose())
or
this.getADecorator().(Call).getFunc().pointsTo(CherryPy::expose())
}
}
deprecated class CherryPyRoute extends CallNode {
CherryPyRoute() {
/* cherrypy.quickstart(root, script_name, config) */
Value::named("cherrypy.quickstart").(FunctionValue).getACall() = this
or
/* cherrypy.tree.mount(root, script_name, config) */
this.getFunction().(AttrNode).getObject("mount").pointsTo(Value::named("cherrypy.tree"))
}
ClassValue getAppClass() {
this.getArg(0).pointsTo().getClass() = result
or
this.getArgByName("root").pointsTo().getClass() = result
}
string getPath() {
exists(Value path | path = Value::forString(result) |
this.getArg(1).pointsTo(path)
or
this.getArgByName("script_name").pointsTo(path)
)
}
ClassValue getConfig() {
this.getArg(2).pointsTo().getClass() = result
or
this.getArgByName("config").pointsTo().getClass() = result
}
}

41
python/ql/lib/semmle/python/web/cherrypy/Request.qll
View File

@@ -1,41 +0,0 @@
import python
import semmle.python.dataflow.TaintTracking
import semmle.python.security.strings.Basic
import semmle.python.web.Http
import semmle.python.web.cherrypy.General
/** The cherrypy.request local-proxy object */
deprecated class CherryPyRequest extends TaintKind {
CherryPyRequest() { this = "cherrypy.request" }
override TaintKind getTaintOfAttribute(string name) {
name = "params" and result instanceof ExternalStringDictKind
or
name = "cookie" and result instanceof UntrustedCookie
}
override TaintKind getTaintOfMethodResult(string name) {
name in ["getHeader", "getCookie", "getUser", "getPassword"] and
result instanceof ExternalStringKind
}
}
deprecated class CherryPyExposedFunctionParameter extends HttpRequestTaintSource {
CherryPyExposedFunctionParameter() {
exists(Parameter p |
p = any(CherryPyExposedFunction f).getAnArg() and
not p.isSelf() and
p.asName().getAFlowNode() = this
)
}
override string toString() { result = "CherryPy handler function parameter" }
override predicate isSourceOf(TaintKind kind) { kind instanceof ExternalStringKind }
}
deprecated class CherryPyRequestSource extends HttpRequestTaintSource {
CherryPyRequestSource() { this.(ControlFlowNode).pointsTo(Value::named("cherrypy.request")) }
override predicate isSourceOf(TaintKind kind) { kind instanceof CherryPyRequest }
}

136
python/ql/lib/semmle/python/web/django/General.qll
View File

@@ -1,136 +0,0 @@
import python
import semmle.python.regex
import semmle.python.web.Http
// TODO: Since django uses `path = partial(...)`, our analysis doesn't understand this is
// a FunctionValue, so we can't use `FunctionValue.getArgumentForCall`
// https://github.com/django/django/blob/master/django/urls/conf.py#L76
abstract deprecated class DjangoRoute extends CallNode {
DjangoViewHandler getViewHandler() {
result = view_handler_from_view_arg(this.getArg(1))
or
result = view_handler_from_view_arg(this.getArgByName("view"))
}
abstract string getANamedArgument();
/**
* Get the number of positional arguments that will be passed to the view.
* Will only return a result if there are no named arguments.
*/
abstract int getNumPositionalArguments();
}
/**
* For function based views -- also see `DjangoClassBasedViewHandler`
* https://docs.djangoproject.com/en/1.11/topics/http/views/
* https://docs.djangoproject.com/en/3.0/topics/http/views/
*/
deprecated class DjangoViewHandler extends PythonFunctionValue {
/** Gets the index of the 'request' argument */
int getRequestArgIndex() { result = 0 }
}
/**
* Class based views
* https://docs.djangoproject.com/en/1.11/topics/class-based-views/
* https://docs.djangoproject.com/en/3.0/topics/class-based-views/
*/
deprecated private class DjangoViewClass extends ClassValue {
DjangoViewClass() {
Value::named("django.views.generic.View") = this.getASuperType()
or
Value::named("django.views.View") = this.getASuperType()
}
}
deprecated class DjangoClassBasedViewHandler extends DjangoViewHandler {
DjangoClassBasedViewHandler() { exists(DjangoViewClass cls | cls.lookup(httpVerbLower()) = this) }
override int getRequestArgIndex() {
// due to `self` being the first parameter
result = 1
}
}
/**
* Gets the function that will handle requests when `view_arg` is used as the view argument to a
* django route. That is, this methods handles Class-based Views and its `as_view()` function.
*/
deprecated private DjangoViewHandler view_handler_from_view_arg(ControlFlowNode view_arg) {
// Function-based view
result = view_arg.pointsTo()
or
// Class-based view
exists(ClassValue cls |
cls = view_arg.(CallNode).getFunction().(AttrNode).getObject("as_view").pointsTo() and
result = cls.lookup(httpVerbLower())
)
}
// We need this "dummy" class, since otherwise the regex argument would not be considered
// a regex (RegexString is abstract)
deprecated class DjangoRouteRegex extends RegexString {
DjangoRouteRegex() { exists(DjangoRegexRoute route | route.getRouteArg() = this.getAFlowNode()) }
}
deprecated class DjangoRegexRoute extends DjangoRoute {
ControlFlowNode route;
DjangoRegexRoute() {
exists(FunctionValue route_maker |
// Django 1.x: https://docs.djangoproject.com/en/1.11/ref/urls/#django.conf.urls.url
Value::named("django.conf.urls.url") = route_maker and
route_maker.getArgumentForCall(this, 0) = route
)
or
// Django 2.x and 3.x: https://docs.djangoproject.com/en/3.0/ref/urls/#re-path
this = Value::named("django.urls.re_path").getACall() and
(
route = this.getArg(0)
or
route = this.getArgByName("route")
)
}
ControlFlowNode getRouteArg() { result = route }
override string getANamedArgument() {
exists(DjangoRouteRegex regex | regex.getAFlowNode() = route |
result = regex.getGroupName(_, _)
)
}
override int getNumPositionalArguments() {
not exists(this.getANamedArgument()) and
exists(DjangoRouteRegex regex | regex.getAFlowNode() = route |
result = count(regex.getGroupNumber(_, _))
)
}
}
deprecated class DjangoPathRoute extends DjangoRoute {
ControlFlowNode route;
DjangoPathRoute() {
// Django 2.x and 3.x: https://docs.djangoproject.com/en/3.0/ref/urls/#path
this = Value::named("django.urls.path").getACall() and
(
route = this.getArg(0)
or
route = this.getArgByName("route")
)
}
override string getANamedArgument() {
// regexp taken from django:
// https://github.com/django/django/blob/7d1bf29977bb368d7c28e7c6eb146db3b3009ae7/django/urls/resolvers.py#L199
exists(StrConst route_str, string match |
route_str = route.getNode() and
match = route_str.getText().regexpFind("<(?:(?<converter>[^>:]+):)?(?<parameter>\\w+)>", _, _) and
result = match.regexpCapture("<(?:(?<converter>[^>:]+):)?(?<parameter>\\w+)>", 2)
)
}
override int getNumPositionalArguments() { none() }
}

78
python/ql/lib/semmle/python/web/django/Request.qll
View File

@@ -1,78 +0,0 @@
import python
import semmle.python.dataflow.TaintTracking
import semmle.python.web.Http
import semmle.python.web.django.General
/** A django.request.HttpRequest object */
deprecated class DjangoRequest extends TaintKind {
DjangoRequest() { this = "django.request.HttpRequest" }
override TaintKind getTaintOfAttribute(string name) {
(name = "GET" or name = "POST") and
result instanceof DjangoQueryDict
}
override TaintKind getTaintOfMethodResult(string name) {
(name = "body" or name = "path") and
result instanceof ExternalStringKind
}
}
/* Helper for getTaintForStep() */
pragma[noinline]
deprecated private predicate subscript_taint(SubscriptNode sub, ControlFlowNode obj, TaintKind kind) {
sub.getObject() = obj and
kind instanceof ExternalStringKind
}
/** A django.request.QueryDict object */
deprecated class DjangoQueryDict extends TaintKind {
DjangoQueryDict() { this = "django.http.request.QueryDict" }
override TaintKind getTaintForFlowStep(ControlFlowNode fromnode, ControlFlowNode tonode) {
this.taints(fromnode) and
subscript_taint(tonode, fromnode, result)
}
override TaintKind getTaintOfMethodResult(string name) {
name = "get" and result instanceof ExternalStringKind
}
}
/** A Django request parameter */
deprecated class DjangoRequestSource extends HttpRequestTaintSource {
DjangoRequestSource() {
exists(DjangoRoute route, DjangoViewHandler view, int request_arg_index |
route.getViewHandler() = view and
request_arg_index = view.getRequestArgIndex() and
this = view.getScope().getArg(request_arg_index).asName().getAFlowNode()
)
}
override string toString() { result = "Django request source" }
override predicate isSourceOf(TaintKind kind) { kind instanceof DjangoRequest }
}
/** An argument specified in a url routing table */
deprecated class DjangoRequestParameter extends HttpRequestTaintSource {
DjangoRequestParameter() {
exists(DjangoRoute route, Function f, DjangoViewHandler view, int request_arg_index |
route.getViewHandler() = view and
request_arg_index = view.getRequestArgIndex() and
f = view.getScope()
|
this.(ControlFlowNode).getNode() = f.getArgByName(route.getANamedArgument())
or
exists(int i | i >= 0 |
i < route.getNumPositionalArguments() and
// +1 because first argument is always the request
this.(ControlFlowNode).getNode() = f.getArg(request_arg_index + 1 + i)
)
)
}
override predicate isSourceOf(TaintKind kind) { kind instanceof ExternalStringKind }
override string toString() { result = "django.http.request.parameter" }
}

46
python/ql/lib/semmle/python/web/falcon/General.qll
View File

@@ -1,46 +0,0 @@
import python
import semmle.python.web.Http
/** Gets the falcon API class */
deprecated ClassValue theFalconAPIClass() { result = Value::named("falcon.API") }
/** Holds if `route` is routed to `resource` */
deprecated private predicate api_route(
CallNode route_call, ControlFlowNode route, ClassValue resource
) {
route_call.getFunction().(AttrNode).getObject("add_route").pointsTo().getClass() =
theFalconAPIClass() and
route_call.getArg(0) = route and
route_call.getArg(1).pointsTo().getClass() = resource
}
deprecated private predicate route(FalconRoute route, Function target, string funcname) {
route.getResourceClass().lookup("on_" + funcname).(FunctionValue).getScope() = target
}
deprecated class FalconRoute extends ControlFlowNode {
FalconRoute() { api_route(this, _, _) }
string getUrl() {
exists(StrConst url |
api_route(this, url.getAFlowNode(), _) and
result = url.getText()
)
}
ClassValue getResourceClass() { api_route(this, _, result) }
FalconHandlerFunction getHandlerFunction(string method) { route(this, result, method) }
}
deprecated class FalconHandlerFunction extends Function {
FalconHandlerFunction() { route(_, this, _) }
private string methodName() { route(_, this, result) }
string getMethod() { result = this.methodName().toUpperCase() }
Parameter getRequest() { result = this.getArg(1) }
Parameter getResponse() { result = this.getArg(2) }
}

37
python/ql/lib/semmle/python/web/falcon/Request.qll
View File

@@ -1,37 +0,0 @@
import python
import semmle.python.dataflow.TaintTracking
import semmle.python.web.Http
import semmle.python.web.falcon.General
/** https://falcon.readthedocs.io/en/stable/api/request_and_response.html */
deprecated class FalconRequest extends TaintKind {
FalconRequest() { this = "falcon.request" }
override TaintKind getTaintOfAttribute(string name) {
name = "env" and result instanceof WsgiEnvironment
or
result instanceof ExternalStringKind and
name in ["uri", "url", "forwarded_uri", "relative_uri", "query_string"]
or
result instanceof ExternalStringDictKind and
(name = "cookies" or name = "params")
or
name = "stream" and result instanceof ExternalFileObject
}
override TaintKind getTaintOfMethodResult(string name) {
name = "get_param" and result instanceof ExternalStringKind
or
name = "get_param_as_json" and result instanceof ExternalJsonKind
or
name = "get_param_as_list" and result instanceof ExternalStringSequenceKind
}
}
deprecated class FalconRequestParameter extends HttpRequestTaintSource {
FalconRequestParameter() {
exists(FalconHandlerFunction f | f.getRequest() = this.(ControlFlowNode).getNode())
}
override predicate isSourceOf(TaintKind k) { k instanceof FalconRequest }
}

104
python/ql/lib/semmle/python/web/flask/General.qll
View File

@@ -1,104 +0,0 @@
import python
import semmle.python.web.Http
import semmle.python.web.flask.Response
/** Gets the flask app class */
deprecated ClassValue theFlaskClass() { result = Value::named("flask.Flask") }
/** Gets the flask MethodView class */
deprecated ClassValue theFlaskMethodViewClass() { result = Value::named("flask.views.MethodView") }
deprecated ClassValue theFlaskReponseClass() { result = Value::named("flask.Response") }
/**
* Holds if `route` is routed to `func`
* by decorating `func` with `app.route(route)`
*/
deprecated predicate app_route(ControlFlowNode route, Function func) {
exists(CallNode route_call, CallNode decorator_call |
route_call.getFunction().(AttrNode).getObject("route").pointsTo().getClass() = theFlaskClass() and
decorator_call.getFunction() = route_call and
route_call.getArg(0) = route and
decorator_call.getArg(0).getNode().(FunctionExpr).getInnerScope() = func
)
}
/* Helper for add_url_rule */
deprecated private predicate add_url_rule_call(ControlFlowNode regex, ControlFlowNode callable) {
exists(CallNode call |
call.getFunction().(AttrNode).getObject("add_url_rule").pointsTo().getClass() = theFlaskClass() and
regex = call.getArg(0)
|
callable = call.getArg(2) or
callable = call.getArgByName("view_func")
)
}
/** Holds if urls matching `regex` are routed to `func` */
deprecated predicate add_url_rule(ControlFlowNode regex, Function func) {
exists(ControlFlowNode callable | add_url_rule_call(regex, callable) |
exists(PythonFunctionValue f | f.getScope() = func and callable.pointsTo(f))
or
/* MethodView.as_view() */
exists(MethodViewClass view_cls | view_cls.asTaint().taints(callable) |
func = view_cls.lookup(httpVerbLower()).(FunctionValue).getScope()
)
/* TODO: -- Handle Views that aren't MethodViews */
)
}
/**
* Holds if urls matching `regex` are routed to `func` using
* any of flask's routing mechanisms.
*/
deprecated predicate flask_routing(ControlFlowNode regex, Function func) {
app_route(regex, func)
or
add_url_rule(regex, func)
}
/** A class that extends flask.views.MethodView */
deprecated private class MethodViewClass extends ClassValue {
MethodViewClass() { this.getASuperType() = theFlaskMethodViewClass() }
/* As we are restricted to strings for taint kinds, we need to map these classes to strings. */
string taintString() { result = "flask/" + this.getQualifiedName() + ".as.view" }
/* As we are restricted to strings for taint kinds, we need to map these classes to strings. */
TaintKind asTaint() { result = this.taintString() }
}
deprecated private class MethodViewTaint extends TaintKind {
MethodViewTaint() { any(MethodViewClass cls).taintString() = this }
}
/** A source of method view "taint"s. */
deprecated private class AsView extends TaintSource {
AsView() {
exists(ClassValue view_class |
view_class.getASuperType() = theFlaskMethodViewClass() and
this.(CallNode).getFunction().(AttrNode).getObject("as_view").pointsTo(view_class)
)
}
override string toString() { result = "flask.MethodView.as_view()" }
override predicate isSourceOf(TaintKind kind) {
exists(MethodViewClass view_class |
kind = view_class.asTaint() and
this.(CallNode).getFunction().(AttrNode).getObject("as_view").pointsTo(view_class)
)
}
}
deprecated class FlaskCookieSet extends CookieSet, CallNode {
FlaskCookieSet() {
any(FlaskResponseTaintKind t).taints(this.getFunction().(AttrNode).getObject("set_cookie"))
}
override string toString() { result = CallNode.super.toString() }
override ControlFlowNode getKey() { result = this.getArg(0) }
override ControlFlowNode getValue() { result = this.getArg(1) }
}

80
python/ql/lib/semmle/python/web/flask/Request.qll
View File

@@ -1,80 +0,0 @@
import python
import semmle.python.dataflow.TaintTracking
import semmle.python.web.Http
import semmle.python.web.flask.General
deprecated private Value theFlaskRequestObject() { result = Value::named("flask.request") }
/** Holds if `attr` is an access of attribute `name` of the flask request object */
deprecated private predicate flask_request_attr(AttrNode attr, string name) {
attr.isLoad() and
attr.getObject(name).pointsTo(theFlaskRequestObject())
}
/** Source of external data from a flask request */
deprecated class FlaskRequestData extends HttpRequestTaintSource {
FlaskRequestData() {
not this instanceof FlaskRequestArgs and
exists(string name | flask_request_attr(this, name) |
name in ["path", "full_path", "base_url", "url"]
)
}
override predicate isSourceOf(TaintKind kind) { kind instanceof ExternalStringKind }
override string toString() { result = "flask.request" }
}
/** Source of dictionary whose values are externally controlled */
deprecated class FlaskRequestArgs extends HttpRequestTaintSource {
FlaskRequestArgs() {
exists(string attr | flask_request_attr(this, attr) |
attr in ["args", "form", "values", "files", "headers", "json"]
)
}
override predicate isSourceOf(TaintKind kind) { kind instanceof ExternalStringDictKind }
override string toString() { result = "flask.request.args" }
}
/** Source of dictionary whose values are externally controlled */
deprecated class FlaskRequestJson extends HttpRequestTaintSource {
FlaskRequestJson() { flask_request_attr(this, "json") }
override predicate isSourceOf(TaintKind kind) { kind instanceof ExternalJsonKind }
override string toString() { result = "flask.request.json" }
}
/**
* A parameter to a flask request handler, that can capture a part of the URL (as specified in
* the url-pattern of a route).
*
* For example, the `name` parameter in:
* ```
* @app.route('/hello/<name>')
* def hello(name):
* ```
*/
deprecated class FlaskRoutedParameter extends HttpRequestTaintSource {
FlaskRoutedParameter() {
exists(string name, Function func, StrConst url_pattern |
this.(ControlFlowNode).getNode() = func.getArgByName(name) and
flask_routing(url_pattern.getAFlowNode(), func) and
exists(string match |
match = url_pattern.getS().regexpFind(werkzeug_rule_re(), _, _) and
name = match.regexpCapture(werkzeug_rule_re(), 4)
)
)
}
override predicate isSourceOf(TaintKind kind) { kind instanceof ExternalStringKind }
}
deprecated private string werkzeug_rule_re() {
// since flask uses werkzeug internally, we are using its routing rules from
// https://github.com/pallets/werkzeug/blob/4dc8d6ab840d4b78cbd5789cef91b01e3bde01d5/src/werkzeug/routing.py#L138-L151
result =
"(?<static>[^<]*)<(?:(?<converter>[a-zA-Z_][a-zA-Z0-9_]*)(?:\\((?<args>.*?)\\))?\\:)?(?<variable>[a-zA-Z_][a-zA-Z0-9_]*)>"
}

55
python/ql/lib/semmle/python/web/flask/Response.qll
View File

@@ -1,55 +0,0 @@
import python
import semmle.python.dataflow.TaintTracking
import semmle.python.security.strings.Basic
import semmle.python.web.flask.General
/**
* A flask response, which is vulnerable to any sort of
* http response malice.
*/
deprecated class FlaskRoutedResponse extends HttpResponseTaintSink {
FlaskRoutedResponse() {
exists(PythonFunctionValue response |
flask_routing(_, response.getScope()) and
this = response.getAReturnedNode()
)
}
override predicate sinks(TaintKind kind) { kind instanceof StringKind }
override string toString() { result = "flask.routed.response" }
}
deprecated class FlaskResponseArgument extends HttpResponseTaintSink {
FlaskResponseArgument() {
exists(CallNode call |
(
call.getFunction().pointsTo(theFlaskReponseClass())
or
call.getFunction().pointsTo(Value::named("flask.make_response"))
) and
call.getArg(0) = this
)
}
override predicate sinks(TaintKind kind) { kind instanceof StringKind }
override string toString() { result = "flask.response.argument" }
}
deprecated class FlaskResponseTaintKind extends TaintKind {
FlaskResponseTaintKind() { this = "flask.Response" }
}
deprecated class FlaskResponseConfiguration extends TaintTracking::Configuration {
FlaskResponseConfiguration() { this = "Flask response configuration" }
override predicate isSource(DataFlow::Node node, TaintKind kind) {
kind instanceof FlaskResponseTaintKind and
(
node.asCfgNode().(CallNode).getFunction().pointsTo(theFlaskReponseClass())
or
node.asCfgNode().(CallNode).getFunction().pointsTo(Value::named("flask.make_response"))
)
}
}

25
python/ql/lib/semmle/python/web/pyramid/Request.qll
View File

@@ -1,25 +0,0 @@
import python
import semmle.python.dataflow.TaintTracking
import semmle.python.web.Http
private import semmle.python.web.webob.Request
private import semmle.python.web.pyramid.View
deprecated class PyramidRequest extends BaseWebobRequest {
PyramidRequest() { this = "pyramid.request" }
override ClassValue getType() { result = Value::named("pyramid.request.Request") }
}
/** Source of pyramid request objects */
deprecated class PyramidViewArgument extends HttpRequestTaintSource {
PyramidViewArgument() {
exists(Function view_func |
is_pyramid_view_function(view_func) and
this.(ControlFlowNode).getNode() = view_func.getArg(0)
)
}
override predicate isSourceOf(TaintKind kind) { kind instanceof PyramidRequest }
override string toString() { result = "pyramid.view.argument" }
}

9
python/ql/lib/semmle/python/web/pyramid/View.qll
View File

@@ -1,9 +0,0 @@
import python
deprecated ModuleValue thePyramidViewModule() { result.getName() = "pyramid.view" }
deprecated Value thePyramidViewConfig() { result = thePyramidViewModule().attr("view_config") }
deprecated predicate is_pyramid_view_function(Function func) {
func.getADecorator().pointsTo().getClass() = thePyramidViewConfig()
}

126
python/ql/lib/semmle/python/web/stdlib/Request.qll
View File

@@ -1,126 +0,0 @@
/**
* Provides the sources and taint-flow for HTTP servers defined using the standard library (stdlib).
* Specifically, we model `HttpRequestTaintSource`s from instances of `BaseHTTPRequestHandler`
* (or subclasses) and form parsing using `cgi.FieldStorage`.
*/
import python
import semmle.python.dataflow.TaintTracking
import semmle.python.web.Http
/** Source of BaseHttpRequestHandler instances. */
deprecated class StdLibRequestSource extends HttpRequestTaintSource {
StdLibRequestSource() {
exists(ClassValue cls |
cls.getABaseType+() = Value::named("BaseHTTPServer.BaseHTTPRequestHandler")
or
cls.getABaseType+() = Value::named("http.server.BaseHTTPRequestHandler")
|
this.(ControlFlowNode).pointsTo().getClass() = cls
)
}
override predicate isSourceOf(TaintKind kind) { kind instanceof BaseHTTPRequestHandlerKind }
}
/** TaintKind for an instance of BaseHttpRequestHandler. */
deprecated class BaseHTTPRequestHandlerKind extends TaintKind {
BaseHTTPRequestHandlerKind() { this = "BaseHTTPRequestHandlerKind" }
override TaintKind getTaintOfAttribute(string name) {
name in ["requestline", "path"] and
result instanceof ExternalStringKind
or
name = "headers" and
result instanceof HTTPMessageKind
or
name = "rfile" and
result instanceof ExternalFileObject
}
}
/** TaintKind for headers (instance of HttpMessage). */
deprecated class HTTPMessageKind extends ExternalStringDictKind {
override TaintKind getTaintOfMethodResult(string name) {
result = super.getTaintOfMethodResult(name)
or
name = "get_all" and
result.(SequenceKind).getItem() = this.getValue()
or
name in ["as_bytes", "as_string"] and
result instanceof ExternalStringKind
}
override TaintKind getTaintForFlowStep(ControlFlowNode fromnode, ControlFlowNode tonode) {
result = super.getTaintForFlowStep(fromnode, tonode)
or
exists(ClassValue cls | cls = ClassValue::unicode() or cls = ClassValue::bytes() |
tonode = cls.getACall() and
tonode.(CallNode).getArg(0) = fromnode and
result instanceof ExternalStringKind
)
}
}
/** Source of parsed HTTP forms (by using the `cgi` module). */
deprecated class CgiFieldStorageSource extends HttpRequestTaintSource {
CgiFieldStorageSource() { this = Value::named("cgi.FieldStorage").getACall() }
override predicate isSourceOf(TaintKind kind) { kind instanceof CgiFieldStorageFormKind }
}
/** TaintKind for a parsed HTTP form. */
deprecated class CgiFieldStorageFormKind extends TaintKind {
/*
* There is a slight difference between how we model form/fields and how it is handled by the code.
* In the code
* ```
* form = cgi.FieldStorage()
* field = form['myfield']
* ```
* both `form` and `field` have the type `cgi.FieldStorage`. This allows the code to represent
* nested forms as `form['nested_form']['myfield']`. However, since HTML forms can't be nested
* we ignore that detail since it allows for a more clean modeling.
*/
CgiFieldStorageFormKind() { this = "CgiFieldStorageFormKind" }
override TaintKind getTaintOfAttribute(string name) {
name = "value" and result.(SequenceKind).getItem() instanceof CgiFieldStorageFieldKind
}
override TaintKind getTaintOfMethodResult(string name) {
name = "getvalue" and
(
result instanceof ExternalStringKind
or
result.(SequenceKind).getItem() instanceof ExternalStringKind
)
or
name = "getfirst" and
result instanceof ExternalStringKind
or
name = "getlist" and
result.(SequenceKind).getItem() instanceof ExternalStringKind
}
override TaintKind getTaintForFlowStep(ControlFlowNode fromnode, ControlFlowNode tonode) {
tonode.(SubscriptNode).getObject() = fromnode and
(
result instanceof CgiFieldStorageFieldKind
or
result.(SequenceKind).getItem() instanceof CgiFieldStorageFieldKind
)
}
}
/** TaintKind for the field of a parsed HTTP form. */
deprecated class CgiFieldStorageFieldKind extends TaintKind {
CgiFieldStorageFieldKind() { this = "CgiFieldStorageFieldKind" }
override TaintKind getTaintOfAttribute(string name) {
name in ["filename", "value"] and result instanceof ExternalStringKind
or
name = "file" and result instanceof ExternalFileObject
}
}

69
python/ql/lib/semmle/python/web/tornado/Request.qll
View File

@@ -1,69 +0,0 @@
import python
import semmle.python.dataflow.TaintTracking
import semmle.python.web.Http
import Tornado
/** A tornado.request.HttpRequest object */
deprecated class TornadoRequest extends TaintKind {
TornadoRequest() { this = "tornado.request.HttpRequest" }
override TaintKind getTaintOfAttribute(string name) {
result instanceof ExternalStringDictKind and
(
name = "headers" or
name = "cookies"
)
or
result instanceof ExternalStringKind and
(
name = "uri" or
name = "query" or
name = "body"
)
or
result instanceof ExternalStringSequenceDictKind and
(
name = "arguments" or
name = "query_arguments" or
name = "body_arguments"
)
}
}
deprecated class TornadoRequestSource extends HttpRequestTaintSource {
TornadoRequestSource() { isTornadoRequestHandlerInstance(this.(AttrNode).getObject("request")) }
override string toString() { result = "Tornado request source" }
override predicate isSourceOf(TaintKind kind) { kind instanceof TornadoRequest }
}
deprecated class TornadoExternalInputSource extends HttpRequestTaintSource {
TornadoExternalInputSource() {
exists(string name |
name in ["get_argument", "get_query_argument", "get_body_argument", "decode_argument"]
|
this = callToNamedTornadoRequestHandlerMethod(name)
)
}
override string toString() { result = "Tornado request method" }
override predicate isSourceOf(TaintKind kind) { kind instanceof ExternalStringKind }
}
deprecated class TornadoExternalInputListSource extends HttpRequestTaintSource {
TornadoExternalInputListSource() {
exists(string name |
name = "get_arguments" or
name = "get_query_arguments" or
name = "get_body_arguments"
|
this = callToNamedTornadoRequestHandlerMethod(name)
)
}
override string toString() { result = "Tornado request method" }
override predicate isSourceOf(TaintKind kind) { kind instanceof ExternalStringSequenceKind }
}

50
python/ql/lib/semmle/python/web/tornado/Tornado.qll
View File

@@ -1,50 +0,0 @@
import python
import semmle.python.dataflow.TaintTracking
import semmle.python.web.Http
deprecated private ClassValue theTornadoRequestHandlerClass() {
result = Value::named("tornado.web.RequestHandler")
}
deprecated ClassValue aTornadoRequestHandlerClass() {
result.getABaseType+() = theTornadoRequestHandlerClass()
}
/**
* Holds if `node` is likely to refer to an instance of a tornado
* `RequestHandler` class.
*/
deprecated predicate isTornadoRequestHandlerInstance(ControlFlowNode node) {
node.pointsTo().getClass() = aTornadoRequestHandlerClass()
or
/*
* In some cases, the points-to analysis won't capture all instances we care
* about. For these, we use the following syntactic check. First, that
* `node` appears inside a method of a subclass of
* `tornado.web.RequestHandler`:
*/
node.getScope().getEnclosingScope() = aTornadoRequestHandlerClass().getScope() and
/* Secondly, that `node` refers to the `self` argument: */
node.isLoad() and
node.(NameNode).isSelf()
}
deprecated CallNode callToNamedTornadoRequestHandlerMethod(string name) {
isTornadoRequestHandlerInstance(result.getFunction().(AttrNode).getObject(name))
}
deprecated class TornadoCookieSet extends CookieSet, CallNode {
TornadoCookieSet() {
exists(ControlFlowNode f |
f = this.getFunction().(AttrNode).getObject("set_cookie") and
isTornadoRequestHandlerInstance(f)
)
}
override string toString() { result = CallNode.super.toString() }
override ControlFlowNode getKey() { result = this.getArg(0) }
override ControlFlowNode getValue() { result = this.getArg(1) }
}

26
python/ql/lib/semmle/python/web/turbogears/Request.qll
View File

@@ -1,26 +0,0 @@
import python
import semmle.python.security.strings.External
import semmle.python.web.Http
import TurboGears
deprecated private class ValidatedMethodParameter extends Parameter {
ValidatedMethodParameter() {
exists(string name, TurboGearsControllerMethod method |
method.getArgByName(name) = this and
method.getValidationDict().getItem(_).(KeyValuePair).getKey().(StrConst).getText() = name
)
}
}
deprecated class UnvalidatedControllerMethodParameter extends HttpRequestTaintSource {
UnvalidatedControllerMethodParameter() {
exists(Parameter p |
any(TurboGearsControllerMethod m | not m.getName() = "onerror").getAnArg() = p and
not p instanceof ValidatedMethodParameter and
not p.isSelf() and
p.(Name).getAFlowNode() = this
)
}
override predicate isSourceOf(TaintKind kind) { kind instanceof ExternalStringKind }
}

37
python/ql/lib/semmle/python/web/turbogears/TurboGears.qll
View File

@@ -1,37 +0,0 @@
import python
import semmle.python.dataflow.TaintTracking
deprecated private ClassValue theTurboGearsControllerClass() {
result = Value::named("tg.TGController")
}
deprecated ClassValue aTurboGearsControllerClass() {
result.getABaseType+() = theTurboGearsControllerClass()
}
deprecated class TurboGearsControllerMethod extends Function {
ControlFlowNode decorator;
TurboGearsControllerMethod() {
aTurboGearsControllerClass().getScope() = this.getScope() and
decorator = this.getADecorator().getAFlowNode() and
/* Is decorated with @expose() or @expose(path) */
(
decorator.(CallNode).getFunction().(NameNode).getId() = "expose"
or
decorator.pointsTo().getClass() = Value::named("tg.expose")
)
}
private ControlFlowNode templateName() { result = decorator.(CallNode).getArg(0) }
predicate isTemplated() { exists(this.templateName()) }
Dict getValidationDict() {
exists(Call call |
call = this.getADecorator() and
call.getFunc().(Name).getId() = "validate" and
call.getArg(0).pointsTo(_, result)
)
}
}

30
python/ql/lib/semmle/python/web/twisted/Request.qll
View File

@@ -1,30 +0,0 @@
import python
import semmle.python.dataflow.TaintTracking
import semmle.python.web.Http
import Twisted
/** A twisted.web.http.Request object */
deprecated class TwistedRequest extends TaintKind {
TwistedRequest() { this = "twisted.request.http.Request" }
override TaintKind getTaintOfAttribute(string name) {
result instanceof ExternalStringSequenceDictKind and
name = "args"
or
result instanceof ExternalStringKind and
name = "uri"
}
override TaintKind getTaintOfMethodResult(string name) {
name in ["getHeader", "getCookie", "getUser", "getPassword"] and
result instanceof ExternalStringKind
}
}
deprecated class TwistedRequestSource extends HttpRequestTaintSource {
TwistedRequestSource() { isTwistedRequestInstance(this) }
override string toString() { result = "Twisted request source" }
override predicate isSourceOf(TaintKind kind) { kind instanceof TwistedRequest }
}

52
python/ql/lib/semmle/python/web/twisted/Twisted.qll
View File

@@ -1,52 +0,0 @@
import python
import semmle.python.dataflow.TaintTracking
deprecated private ClassValue theTwistedHttpRequestClass() {
result = Value::named("twisted.web.http.Request")
}
deprecated private ClassValue theTwistedHttpResourceClass() {
result = Value::named("twisted.web.resource.Resource")
}
deprecated ClassValue aTwistedRequestHandlerClass() {
result.getABaseType+() = theTwistedHttpResourceClass()
}
deprecated FunctionValue getTwistedRequestHandlerMethod(string name) {
result = aTwistedRequestHandlerClass().declaredAttribute(name)
}
bindingset[name]
deprecated predicate isKnownRequestHandlerMethodName(string name) {
name = "render" or
name.matches("render_%")
}
/**
* Holds if `node` is likely to refer to an instance of the twisted
* `Request` class.
*/
deprecated predicate isTwistedRequestInstance(NameNode node) {
node.pointsTo().getClass() = theTwistedHttpRequestClass()
or
/*
* In points-to analysis cannot infer that a given object is an instance of
* the `twisted.web.http.Request` class, we also include any parameter
* called `request` that appears inside a subclass of a request handler
* class, and the appropriate arguments of known request handler methods.
*/
exists(Function func |
func = node.getScope() and
func.getEnclosingScope() = aTwistedRequestHandlerClass().getScope()
|
/* Any parameter called `request` */
node.getId() = "request" and
node.isParameter()
or
/* Any request parameter of a known request handler method */
isKnownRequestHandlerMethodName(func.getName()) and
node.getNode() = func.getArg(1)
)
}

38
python/ql/lib/semmle/python/web/webob/Request.qll
View File

@@ -1,38 +0,0 @@
import python
import semmle.python.dataflow.TaintTracking
import semmle.python.web.Http
abstract deprecated class BaseWebobRequest extends TaintKind {
bindingset[this]
BaseWebobRequest() { any() }
override TaintKind getTaintOfAttribute(string name) {
result instanceof ExternalStringDictKind and
(
name = "GET" or
name = "POST" or
name = "headers"
)
or
result instanceof ExternalStringKind and
name = "body"
}
override TaintKind getTaintOfMethodResult(string name) {
result = this and
(
name = "copy" or
name = "copy_get" or
name = "copy_body"
)
or
result instanceof ExternalStringKind and
name = "as_bytes"
}
}
deprecated class WebobRequest extends BaseWebobRequest {
WebobRequest() { this = "webob.Request" }
override ClassValue getType() { result = Value::named("webob.request.Request") }
}

19
python/ql/src/CHANGELOG.md
View File

@@ -1,3 +1,22 @@
## 0.9.2
No user-facing changes.
## 0.9.1
No user-facing changes.
## 0.9.0
### New Queries
* The query `py/nosql-injection` for finding NoSQL injection vulnerabilities is now available in the default security suite.
### Minor Analysis Improvements
* Improved _URL redirection from remote source_ (`py/url-redirection`) query to not alert when URL has been checked with `django.utils.http. url_has_allowed_host_and_scheme`.
* Extended the `py/command-line-injection` query with sinks from Python's `asyncio` module.
## 0.8.5
No user-facing changes.

4
python/ql/src/change-notes/2023-09-05-asyncio-cmdi-sinks.md
View File

@@ -1,4 +0,0 @@
---
category: minorAnalysis
---
* Extended the `py/command-line-injection` query with sinks from Python's `asyncio` module.

4
python/ql/src/change-notes/2023-09-13-django-url-allowed-host.md
View File

@@ -1,4 +0,0 @@
---
category: minorAnalysis
---
* Improved _URL redirection from remote source_ (`py/url-redirection`) query to not alert when URL has been checked with `django.utils.http. url_has_allowed_host_and_scheme`.

4
python/ql/src/change-notes/2023-09-18-promoted-nosql-injection-query.md
View File

@@ -1,4 +0,0 @@
---
category: newQuery
---
* The query `py/nosql-injection` for finding NoSQL injection vulnerabilities is now available in the default security suite.

10
python/ql/src/change-notes/released/0.9.0.md Normal file
View File

@@ -0,0 +1,10 @@
## 0.9.0
### New Queries
* The query `py/nosql-injection` for finding NoSQL injection vulnerabilities is now available in the default security suite.
### Minor Analysis Improvements
* Improved _URL redirection from remote source_ (`py/url-redirection`) query to not alert when URL has been checked with `django.utils.http. url_has_allowed_host_and_scheme`.
* Extended the `py/command-line-injection` query with sinks from Python's `asyncio` module.

Some files were not shown because too many files have changed in this diff Show More