hohn/codeql
1
0
Fork 0
mirror of https://github.com/github/codeql.git synced 2026-04-30 19:26:02 +02:00
Code Issues Packages Projects Releases Wiki Activity

Merge branch 'main' into small-cleanups

Browse Source
This commit is contained in:
Rasmus Wriedt Larsen
2021-06-22 15:01:53 +02:00
parent cc311ac4cd 38a38fd2c1
commit 0b767bb853
784 changed files with 21649 additions and 3561 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
python/change-notes/2021-05-21-api-graph-await.md Normal file
View File

@@ -0,0 +1,2 @@
lgtm,codescanning
* API graph nodes now contain a `getAwaited()` member predicate, for getting the result of awaiting an item, such as `await foo`.

2
python/change-notes/2021-05-25-add-ClickHouse-sql-libs.md Normal file
View File

@@ -0,0 +1,2 @@
lgtm,codescanning
* Added model of SQL execution in `clickhouse-driver` and `aioch` PyPI packages, resulting in additional sinks for the SQL Injection query (`py/sql-injection`). This modeling was originally [submitted as a contribution by @japroc](https://github.com/github/codeql/pull/5889).

2
python/change-notes/2021-06-04-sensitive-data-modeling-expanded.md Normal file
View File

@@ -0,0 +1,2 @@
lgtm,codescanning
* Expanded modeling of sensitive data sources to include: subscripting with a key that indicates sensitive data (`obj["password"]`), parameters whose names indicate sensitive data (`def func(password):`), and assignments to variables whose names indicate sensitive data (`password = ...`).

2
python/change-notes/2021-06-08-twisted-add-modeling.md Normal file
View File

@@ -0,0 +1,2 @@
lgtm,codescanning
* Added modeling of sources/sinks when using `twisted` to create web servers.

5
python/change-notes/2021-06-15-add-method-call-conveniences.md Normal file
View File

@@ -0,0 +1,5 @@
lgtm,codescanning
* A new class `DataFlow::MethodCallNode` extends `DataFlow::CallCfgNode` with convenient methods for
accessing the receiver and method name of a method call.
* The `LocalSourceNode` class now has a `getAMethodCall` method, with which one can easily access
method calls with the given node as a receiver.

5
python/ql/src/Expressions/UseofInput.ql
View File

@@ -4,9 +4,10 @@
* @kind problem
* @tags security
* correctness
* security/cwe/cwe-78
* security/cwe/cwe-94
* security/cwe/cwe-95
* @problem.severity error
* @security-severity 5.9
* @security-severity 9.8
* @sub-severity high
* @precision high
* @id py/use-of-input

2
python/ql/src/Security/CVE-2018-1281/BindToAllInterfaces.ql
View File

@@ -6,7 +6,7 @@
* @tags security
* external/cwe/cwe-200
* @problem.severity error
* @security-severity 3.6
* @security-severity 6.5
* @sub-severity low
* @precision high
* @id py/bind-socket-all-network-interfaces

2
python/ql/src/Security/CWE-020-ExternalAPIs/UntrustedDataToExternalAPI.ql
View File

@@ -5,7 +5,7 @@
* @kind path-problem
* @precision low
* @problem.severity error
* @security-severity 5.9
* @security-severity 7.8
* @tags security external/cwe/cwe-20
*/

2
python/ql/src/Security/CWE-020/IncompleteHostnameRegExp.ql
View File

@@ -3,7 +3,7 @@
* @description Matching a URL or hostname against a regular expression that contains an unescaped dot as part of the hostname might match more hostnames than expected.
* @kind problem
* @problem.severity warning
* @security-severity 5.9
* @security-severity 7.8
* @precision high
* @id py/incomplete-hostname-regexp
* @tags correctness

2
python/ql/src/Security/CWE-020/IncompleteUrlSubstringSanitization.ql
View File

@@ -3,7 +3,7 @@
* @description Security checks on the substrings of an unparsed URL are often vulnerable to bypassing.
* @kind problem
* @problem.severity warning
* @security-severity 5.9
* @security-severity 7.8
* @precision high
* @id py/incomplete-url-substring-sanitization
* @tags correctness

2
python/ql/src/Security/CWE-022/PathInjection.ql
View File

@@ -3,7 +3,7 @@
* @description Accessing paths influenced by users can allow an attacker to access unexpected resources.
* @kind path-problem
* @problem.severity error
* @security-severity 6.4
* @security-severity 7.5
* @sub-severity high
* @precision high
* @id py/path-injection

2
python/ql/src/Security/CWE-022/TarSlip.ql
View File

@@ -6,7 +6,7 @@
* @kind path-problem
* @id py/tarslip
* @problem.severity error
* @security-severity 6.4
* @security-severity 7.5
* @precision medium
* @tags security
* external/cwe/cwe-022

2
python/ql/src/Security/CWE-078/CommandInjection.ql
View File

@@ -4,7 +4,7 @@
* user to change the meaning of the command.
* @kind path-problem
* @problem.severity error
* @security-severity 5.9
* @security-severity 9.8
* @sub-severity high
* @precision high
* @id py/command-line-injection

2
python/ql/src/Security/CWE-079/Jinja2WithoutEscaping.ql
View File

@@ -4,7 +4,7 @@
* cause a cross-site scripting vulnerability.
* @kind problem
* @problem.severity error
* @security-severity 2.9
* @security-severity 6.1
* @precision medium
* @id py/jinja2/autoescape-false
* @tags security

2
python/ql/src/Security/CWE-079/ReflectedXss.ql
View File

@@ -4,7 +4,7 @@
* allows for a cross-site scripting vulnerability.
* @kind path-problem
* @problem.severity error
* @security-severity 2.9
* @security-severity 6.1
* @sub-severity high
* @precision high
* @id py/reflective-xss

2
python/ql/src/Security/CWE-089/SqlInjection.ql
View File

@@ -4,7 +4,7 @@
* malicious SQL code by the user.
* @kind path-problem
* @problem.severity error
* @security-severity 6.4
* @security-severity 8.8
* @precision high
* @id py/sql-injection
* @tags security

2
python/ql/src/Security/CWE-094/CodeInjection.ql
View File

@@ -4,7 +4,7 @@
* code execution.
* @kind path-problem
* @problem.severity error
* @security-severity 10.0
* @security-severity 9.3
* @sub-severity high
* @precision high
* @id py/code-injection

2
python/ql/src/Security/CWE-209/StackTraceExposure.ql
View File

@@ -5,7 +5,7 @@
* developing a subsequent exploit.
* @kind path-problem
* @problem.severity error
* @security-severity 3.6
* @security-severity 5.4
* @precision high
* @id py/stack-trace-exposure
* @tags security

2
python/ql/src/Security/CWE-215/FlaskDebug.ql
View File

@@ -3,7 +3,7 @@
* @description Running a Flask app in debug mode may allow an attacker to run arbitrary code through the Werkzeug debugger.
* @kind problem
* @problem.severity error
* @security-severity 6.4
* @security-severity 7.5
* @precision high
* @id py/flask-debug
* @tags security

2
python/ql/src/Security/CWE-295/MissingHostKeyValidation.ql
View File

@@ -3,7 +3,7 @@
* @description Accepting unknown host keys can allow man-in-the-middle attacks.
* @kind problem
* @problem.severity error
* @security-severity 5.2
* @security-severity 7.5
* @precision high
* @id py/paramiko-missing-host-key-validation
* @tags security

2
python/ql/src/Security/CWE-295/RequestWithoutValidation.ql
View File

@@ -3,7 +3,7 @@
* @description Making a request without certificate validation can allow man-in-the-middle attacks.
* @kind problem
* @problem.severity error
* @security-severity 5.2
* @security-severity 7.5
* @precision medium
* @id py/request-without-cert-validation
* @tags security

2
python/ql/src/Security/CWE-312/CleartextLogging.ql
View File

@@ -4,7 +4,7 @@
* expose it to an attacker.
* @kind path-problem
* @problem.severity error
* @security-severity 5.9
* @security-severity 7.5
* @precision high
* @id py/clear-text-logging-sensitive-data
* @tags security

2
python/ql/src/Security/CWE-312/CleartextStorage.ql
View File

@@ -4,7 +4,7 @@
* attacker.
* @kind path-problem
* @problem.severity error
* @security-severity 5.9
* @security-severity 7.5
* @precision high
* @id py/clear-text-storage-sensitive-data
* @tags security

2
python/ql/src/Security/CWE-326/WeakCryptoKey.ql
View File

@@ -3,7 +3,7 @@
* @description Use of a cryptographic key that is too small may allow the encryption to be broken.
* @kind problem
* @problem.severity error
* @security-severity 5.2
* @security-severity 7.5
* @precision high
* @id py/weak-crypto-key
* @tags security

2
python/ql/src/Security/CWE-327/BrokenCryptoAlgorithm.ql
View File

@@ -3,7 +3,7 @@
* @description Using broken or weak cryptographic algorithms can compromise security.
* @kind problem
* @problem.severity warning
* @security-severity 5.2
* @security-severity 7.5
* @precision high
* @id py/weak-cryptographic-algorithm
* @tags security

2
python/ql/src/Security/CWE-327/InsecureDefaultProtocol.ql
View File

@@ -5,7 +5,7 @@
* @id py/insecure-default-protocol
* @kind problem
* @problem.severity warning
* @security-severity 5.2
* @security-severity 7.5
* @precision high
* @tags security
* external/cwe/cwe-327

2
python/ql/src/Security/CWE-327/InsecureProtocol.ql
View File

@@ -4,7 +4,7 @@
* @id py/insecure-protocol
* @kind problem
* @problem.severity warning
* @security-severity 5.2
* @security-severity 7.5
* @precision high
* @tags security
* external/cwe/cwe-327

6
python/ql/src/Security/CWE-327/Ssl.qll
View File

@@ -39,12 +39,10 @@ API::Node sslContextInstance() {
result = API::moduleImport("ssl").getMember(["SSLContext", "create_default_context"]).getReturn()
}
class WrapSocketCall extends ConnectionCreation, DataFlow::CallCfgNode {
class WrapSocketCall extends ConnectionCreation, DataFlow::MethodCallNode {
WrapSocketCall() { this = sslContextInstance().getMember("wrap_socket").getACall() }
override DataFlow::Node getContext() {
result = this.getFunction().(DataFlow::AttrRead).getObject()
}
override DataFlow::Node getContext() { result = this.getObject() }
}
class OptionsAugOr extends ProtocolRestriction, DataFlow::CfgNode {

2
python/ql/src/Security/CWE-327/WeakSensitiveDataHashing.ql
View File

@@ -3,7 +3,7 @@
* @description Using broken or weak cryptographic hashing algorithms can compromise security.
* @kind path-problem
* @problem.severity warning
* @security-severity 5.9
* @security-severity 7.5
* @precision high
* @id py/weak-sensitive-data-hashing
* @tags security

2
python/ql/src/Security/CWE-377/InsecureTemporaryFile.ql
View File

@@ -4,7 +4,7 @@
* @kind problem
* @id py/insecure-temporary-file
* @problem.severity error
* @security-severity 5.9
* @security-severity 7.0
* @sub-severity high
* @precision high
* @tags external/cwe/cwe-377

2
python/ql/src/Security/CWE-502/UnsafeDeserialization.ql
View File

@@ -4,7 +4,7 @@
* @kind path-problem
* @id py/unsafe-deserialization
* @problem.severity error
* @security-severity 5.9
* @security-severity 9.8
* @sub-severity high
* @precision high
* @tags external/cwe/cwe-502

2
python/ql/src/Security/CWE-601/UrlRedirect.ql
View File

@@ -4,7 +4,7 @@
* may cause redirection to malicious web sites.
* @kind path-problem
* @problem.severity error
* @security-severity 2.7
* @security-severity 6.1
* @sub-severity low
* @id py/url-redirection
* @tags security

2
python/ql/src/Security/CWE-732/WeakFilePermissions.ql
View File

@@ -4,7 +4,7 @@
* @kind problem
* @id py/overly-permissive-file
* @problem.severity warning
* @security-severity 5.9
* @security-severity 7.8
* @sub-severity high
* @precision medium
* @tags external/cwe/cwe-732

2
python/ql/src/Security/CWE-798/HardcodedCredentials.ql
View File

@@ -3,7 +3,7 @@
* @description Credentials are hard coded in the source code of the application.
* @kind path-problem
* @problem.severity error
* @security-severity 5.9
* @security-severity 9.8
* @precision medium
* @id py/hardcoded-credentials
* @tags security

28
python/ql/src/experimental/Security/CWE-089/ClickHouseSQLInjection.py
View File

@@ -1,28 +0,0 @@
from django.conf.urls import url
from clickhouse_driver import Client
from clickhouse_driver import connect
from aioch import Client as aiochClient
class MyClient(Client):
def dummy(self):
return None
def show_user(request, username):
# BAD -- Untrusted user input is directly injected into the sql query using async library 'aioch'
aclient = aiochClient("localhost")
progress = await aclient.execute_with_progress("SELECT * FROM users WHERE username = '%s'" % username)
# BAD -- Untrusted user input is directly injected into the sql query using native client of library 'clickhouse_driver'
Client('localhost').execute("SELECT * FROM users WHERE username = '%s'" % username)
# GOOD -- query uses prepared statements
query = "SELECT * FROM users WHERE username = %(username)s"
Client('localhost').execute(query, {"username": username})
# BAD -- PEP249 interface
conn = connect('clickhouse://localhost')
cursor = conn.cursor()
cursor.execute("SELECT * FROM users WHERE username = '%s'" % username)
urlpatterns = [url(r'^users/(?P<username>[^/]+)$', show_user)]

59
python/ql/src/experimental/Security/CWE-089/ClickHouseSQLInjection.qhelp
View File

@@ -1,59 +0,0 @@
<!DOCTYPE qhelp PUBLIC
"-//Semmle//qhelp//EN"
"qhelp.dtd">
<qhelp>
<overview>
<p>
If a database query (such as a SQL or NoSQL query) is built from
user-provided data without sufficient sanitization, a user
may be able to run malicious database queries.
</p>
</overview>
<recommendation>
<p>
Most database connector libraries offer a way of safely
embedding untrusted data into a query by means of query parameters
or prepared statements.
</p>
</recommendation>
<example>
<p>
In the following snippet, a user is fetched from a <code>ClickHouse</code> database
using five different queries. In the "BAD" cases the query is built directly from user-controlled data.
In the "GOOD" case the user-controlled data is safely embedded into the query by using query parameters.
</p>
<p>
In the first case, the query executed via aioch Client. aioch - is a module
for asynchronous queries to database.
</p>
<p>
In the second and third cases, the connection is established via `Client` class.
This class implement different method to execute a query.
</p>
<p>
In the forth case, good pattern is presented. Query parameters are send through
second dict-like argument.
</p>
<p>
In the fifth case, there is example of PEP249 interface usage.
</p>
<p>
In the sixth case, there is custom Class usge which is a subclass of default Client.
</p>
<sample src="ClickHouseSQLInjection.py" />
</example>
<references>
<li>Wikipedia: <a href="https://en.wikipedia.org/wiki/SQL_injection">SQL injection</a>.</li>
<li>OWASP: <a href="https://cheatsheetseries.owasp.org/cheatsheets/SQL_Injection_Prevention_Cheat_Sheet.html">SQL Injection Prevention Cheat Sheet</a>.</li>
</references>
</qhelp>

22
python/ql/src/experimental/Security/CWE-089/ClickHouseSQLInjection.ql
View File

@@ -1,22 +0,0 @@
/**
* @id py/yandex/clickhouse-sql-injection
* @name Clickhouse SQL query built from user-controlled sources
* @description Building a SQL query from user-controlled sources is vulnerable to insertion of
* malicious SQL code by the user.
* @kind path-problem
* @problem.severity error
* @precision high
* @tags security
* external/cwe/cwe-089
* external/owasp/owasp-a1
*/
import python
import experimental.semmle.python.frameworks.ClickHouseDriver
import semmle.python.security.dataflow.SqlInjection
import DataFlow::PathGraph
from SQLInjectionConfiguration config, DataFlow::PathNode source, DataFlow::PathNode sink
where config.hasFlowPath(source, sink)
select sink.getNode(), source, sink, "This SQL query depends on $@.", source.getNode(),
"a user-provided value"

85
python/ql/src/experimental/semmle/python/frameworks/ClickHouseDriver.qll
View File

@@ -1,85 +0,0 @@
/**
* Provides classes modeling security-relevant aspects of `clickhouse-driver` and `aioch` PyPI packages.
* See
* - https://pypi.org/project/clickhouse-driver/
* - https://pypi.org/project/aioch/
* - https://clickhouse-driver.readthedocs.io/en/latest/
*/
private import python
private import semmle.python.Concepts
private import semmle.python.ApiGraphs
private import semmle.python.frameworks.PEP249
/**
* Provides models for `clickhouse-driver` and `aioch` PyPI packages.
* See
* - https://pypi.org/project/clickhouse-driver/
* - https://pypi.org/project/aioch/
* - https://clickhouse-driver.readthedocs.io/en/latest/
*/
module ClickHouseDriver {
/** Gets a reference to the `clickhouse_driver` module. */
API::Node clickhouse_driver() { result = API::moduleImport("clickhouse_driver") }
/** Gets a reference to the `aioch` module. This module allows to make async db queries. */
API::Node aioch() { result = API::moduleImport("aioch") }
/**
* `clickhouse_driver` implements PEP249,
* providing ways to execute SQL statements against a database.
*/
class ClickHouseDriverPEP249 extends PEP249ModuleApiNode {
ClickHouseDriverPEP249() { this = clickhouse_driver() }
}
module Client {
/** Gets a reference to a Client call. */
private DataFlow::Node client_ref() {
result = clickhouse_driver().getMember("Client").getASubclass*().getAUse()
or
result = aioch().getMember("Client").getASubclass*().getAUse()
}
/** A direct instantiation of `clickhouse_driver.Client`. */
private class ClientInstantiation extends DataFlow::CallCfgNode {
ClientInstantiation() { this.getFunction() = client_ref() }
}
/** Gets a reference to an instance of `clickhouse_driver.Client`. */
private DataFlow::LocalSourceNode instance(DataFlow::TypeTracker t) {
t.start() and
result instanceof ClientInstantiation
or
exists(DataFlow::TypeTracker t2 | result = instance(t2).track(t2, t))
}
/** Gets a reference to an instance of `clickhouse_driver.Client`. */
DataFlow::Node instance() { instance(DataFlow::TypeTracker::end()).flowsTo(result) }
}
/** clickhouse_driver.Client execute methods */
private string execute_function() {
result in ["execute_with_progress", "execute", "execute_iter"]
}
/** Gets a reference to the `clickhouse_driver.Client.execute` method */
private DataFlow::LocalSourceNode clickhouse_execute(DataFlow::TypeTracker t) {
t.startInAttr(execute_function()) and
result = Client::instance()
or
exists(DataFlow::TypeTracker t2 | result = clickhouse_execute(t2).track(t2, t))
}
/** Gets a reference to the `clickhouse_driver.Client.execute` method */
DataFlow::Node clickhouse_execute() {
clickhouse_execute(DataFlow::TypeTracker::end()).flowsTo(result)
}
/** A call to the `clickhouse_driver.Client.execute` method */
private class ExecuteCall extends SqlExecution::Range, DataFlow::CallCfgNode {
ExecuteCall() { this.getFunction() = clickhouse_execute() }
override DataFlow::Node getSql() { result.asCfgNode() = node.getArg(0) }
}
}

9
python/ql/src/experimental/semmle/python/frameworks/Stdlib.qll
View File

@@ -64,15 +64,14 @@ private module Re {
*
* See https://docs.python.org/3/library/re.html#regular-expression-objects
*/
private class CompiledRegex extends DataFlow::CallCfgNode, RegexExecution::Range {
private class CompiledRegex extends DataFlow::MethodCallNode, RegexExecution::Range {
DataFlow::Node regexNode;
CompiledRegex() {
exists(DataFlow::CallCfgNode patternCall, DataFlow::AttrRead reMethod |
this.getFunction() = reMethod and
exists(DataFlow::MethodCallNode patternCall |
patternCall = API::moduleImport("re").getMember("compile").getACall() and
patternCall.flowsTo(reMethod.getObject()) and
reMethod.getAttributeName() instanceof RegexExecutionMethods and
patternCall.flowsTo(this.getObject()) and
this.getMethodName() instanceof RegexExecutionMethods and
regexNode = patternCall.getArg(0)
)
}

17
python/ql/src/semmle/python/ApiGraphs.qll
View File

@@ -97,6 +97,11 @@ module API {
*/
Node getASubclass() { result = getASuccessor(Label::subclass()) }
/**
* Gets a node representing the result from awaiting this node.
*/
Node getAwaited() { result = getASuccessor(Label::await()) }
/**
* Gets a string representation of the lexicographically least among all shortest access paths
* from the root to this node.
@@ -469,6 +474,14 @@ module API {
exists(DataFlow::Node superclass | pred.flowsTo(superclass) |
ref.asExpr().(ClassExpr).getABase() = superclass.asExpr()
)
or
// awaiting
exists(Await await, DataFlow::Node awaitedValue |
lbl = Label::await() and
ref.asExpr() = await and
await.getValue() = awaitedValue.asExpr() and
pred.flowsTo(awaitedValue)
)
)
or
// Built-ins, treated as members of the module `builtins`
@@ -585,5 +598,9 @@ private module Label {
/** Gets the `return` edge label. */
string return() { result = "getReturn()" }
/** Gets the `subclass` edge label. */
string subclass() { result = "getASubclass()" }
/** Gets the `await` edge label. */
string await() { result = "getAwaited()" }
}

10
python/ql/src/semmle/python/Files.qll
View File

@@ -89,7 +89,15 @@ class File extends Container {
i.getTest().(Compare).compares(name, op, main) and
name.getId() = "__name__" and
main.getText() = "__main__"
)
) and
// Exclude files named `__main__.py`. These are often _not_ meant to be run directly, but
// contain this construct anyway.
//
// Their presence in a package (say, `foo`) means one can execute the package directly using
// `python -m foo` (which will run the `foo/__main__.py` file). Since being an entry point for
// execution means treating imports as absolute, this causes trouble, since when run with
// `python -m`, the interpreter uses the usual package semantics.
not this.getShortName() = "__main__.py"
or
// The file contains a `#!` line referencing the python interpreter
exists(Comment c |

5
python/ql/src/semmle/python/Frameworks.qll
View File

@@ -4,7 +4,9 @@
// If you add modeling of a new framework/library, remember to add it it to the docs in
// `docs/codeql/support/reusables/frameworks.rst`
private import semmle.python.frameworks.Aioch
private import semmle.python.frameworks.Aiohttp
private import semmle.python.frameworks.ClickhouseDriver
private import semmle.python.frameworks.Cryptodome
private import semmle.python.frameworks.Cryptography
private import semmle.python.frameworks.Dill
@@ -14,13 +16,14 @@ private import semmle.python.frameworks.Flask
private import semmle.python.frameworks.Idna
private import semmle.python.frameworks.Invoke
private import semmle.python.frameworks.Multidict
private import semmle.python.frameworks.MysqlConnectorPython
private import semmle.python.frameworks.Mysql
private import semmle.python.frameworks.MySQLdb
private import semmle.python.frameworks.Psycopg2
private import semmle.python.frameworks.PyMySQL
private import semmle.python.frameworks.Simplejson
private import semmle.python.frameworks.Stdlib
private import semmle.python.frameworks.Tornado
private import semmle.python.frameworks.Twisted
private import semmle.python.frameworks.Ujson
private import semmle.python.frameworks.Yaml
private import semmle.python.frameworks.Yarl

241
python/ql/src/semmle/python/dataflow/new/SensitiveDataSources.qll
View File

@@ -5,10 +5,14 @@
private import python
private import semmle.python.dataflow.new.DataFlow
// Need to import since frameworks can extend `RemoteFlowSource::Range`
// Need to import `semmle.python.Frameworks` since frameworks can extend `SensitiveDataSource::Range`
private import semmle.python.Frameworks
private import semmle.python.Concepts
private import semmle.python.security.SensitiveData as OldSensitiveData
private import semmle.python.security.internal.SensitiveDataHeuristics as SensitiveDataHeuristics
// We export these explicitly, so we don't also export the `HeuristicNames` module.
class SensitiveDataClassification = SensitiveDataHeuristics::SensitiveDataClassification;
module SensitiveDataClassification = SensitiveDataHeuristics::SensitiveDataClassification;
/**
* A data flow source of sensitive data, such as secrets, certificates, or passwords.
@@ -22,13 +26,9 @@ class SensitiveDataSource extends DataFlow::Node {
SensitiveDataSource() { this = range }
/**
* INTERNAL: Do not use.
*
* This will be rewritten to have better types soon, and therefore should only be used internally until then.
*
* Gets the classification of the sensitive data.
*/
string getClassification() { result = range.getClassification() }
SensitiveDataClassification getClassification() { result = range.getClassification() }
}
/** Provides a class for modeling new sources of sensitive data, such as secrets, certificates, or passwords. */
@@ -41,26 +41,225 @@ module SensitiveDataSource {
*/
abstract class Range extends DataFlow::Node {
/**
* INTERNAL: Do not use.
*
* This will be rewritten to have better types soon, and therefore should only be used internally until then.
*
* Gets the classification of the sensitive data.
*/
abstract string getClassification();
abstract SensitiveDataClassification getClassification();
}
}
private class PortOfOldModeling extends SensitiveDataSource::Range {
OldSensitiveData::SensitiveData::Source oldSensitiveSource;
/** Actual sensitive data modeling */
private module SensitiveDataModeling {
private import SensitiveDataHeuristics::HeuristicNames
PortOfOldModeling() { this.asCfgNode() = oldSensitiveSource }
/**
* Gets a reference to a function that is considered to be a sensitive source of
* `classification`.
*/
private DataFlow::LocalSourceNode sensitiveFunction(
DataFlow::TypeTracker t, SensitiveDataClassification classification
) {
t.start() and
exists(Function f |
nameIndicatesSensitiveData(f.getName(), classification) and
result.asExpr() = f.getDefinition()
)
or
exists(DataFlow::TypeTracker t2 | result = sensitiveFunction(t2, classification).track(t2, t))
}
override string getClassification() {
exists(OldSensitiveData::SensitiveData classification |
oldSensitiveSource.isSourceOf(classification)
|
classification = "sensitive.data." + result
/**
* Gets a reference to a function that is considered to be a sensitive source of
* `classification`.
*/
DataFlow::Node sensitiveFunction(SensitiveDataClassification classification) {
sensitiveFunction(DataFlow::TypeTracker::end(), classification).flowsTo(result)
}
/**
* Gets a reference to a string constant that, if used as the key in a lookup,
* indicates the presence of sensitive data with `classification`.
*/
private DataFlow::LocalSourceNode sensitiveLookupStringConst(
DataFlow::TypeTracker t, SensitiveDataClassification classification
) {
t.start() and
nameIndicatesSensitiveData(result.asExpr().(StrConst).getText(), classification)
or
exists(DataFlow::TypeTracker t2 |
result = sensitiveLookupStringConst(t2, classification).track(t2, t)
)
}
/**
* Gets a reference to a string constant that, if used as the key in a lookup,
* indicates the presence of sensitive data with `classification`.
*
* Also see `extraStepForCalls`.
*/
DataFlow::Node sensitiveLookupStringConst(SensitiveDataClassification classification) {
sensitiveLookupStringConst(DataFlow::TypeTracker::end(), classification).flowsTo(result)
}
/** A function call that is considered a source of sensitive data. */
class SensitiveFunctionCall extends SensitiveDataSource::Range, DataFlow::CallCfgNode {
SensitiveDataClassification classification;
SensitiveFunctionCall() {
this.getFunction() = sensitiveFunction(classification)
or
// to cover functions that we don't have the definition for, and where the
// reference to the function has not already been marked as being sensitive
nameIndicatesSensitiveData(this.getFunction().asCfgNode().(NameNode).getId(), classification)
}
override SensitiveDataClassification getClassification() { result = classification }
}
/**
* Tracks any modeled source of sensitive data (with any classification),
* to limit the scope of `extraStepForCalls`. See it's QLDoc for more context.
*/
private DataFlow::LocalSourceNode possibleSensitiveCallable(DataFlow::TypeTracker t) {
t.start() and
result instanceof SensitiveDataSource
or
exists(DataFlow::TypeTracker t2 | result = possibleSensitiveCallable(t2).track(t2, t))
}
/**
* Tracks any modeled source of sensitive data (with any classification),
* to limit the scope of `extraStepForCalls`. See it's QLDoc for more context.
*/
private DataFlow::Node possibleSensitiveCallable() {
possibleSensitiveCallable(DataFlow::TypeTracker::end()).flowsTo(result)
}
/**
* Holds if the step from `nodeFrom` to `nodeTo` should be considered a
* taint-flow step for sensitive-data, to ensure calls are handled correctly.
*
* To handle calls properly, while preserving a good source for path explanations,
* you need to include this predicate as an additional taint step in your taint-tracking
* configurations.
*
* The core problem can be illustrated by the example below. If we consider the
* `print` a sink, what path and what source do we want to show? My initial approach
* would be to use type-tracking to propagate from the `not_found.get_passwd` attribute
* lookup, to the use of `non_sensitive_name`, and then create a new `SensitiveDataSource::Range`
* like `SensitiveFunctionCall`. Although that seems likely to work, it will also end up
* with a non-optimal path, which starts at _bad source_, and therefore doesn't show
* how we figured out that `non_sensitive_name`
* could be a function that returns a password (and in cases where there is many calls to
* `my_func` it will be annoying for someone to figure this out manually).
*
* By including this additional taint-step in the taint-tracking configuration, it's possible
* to get a path explanation going from _good source_ to the sink.
*
* ```python
* def my_func(non_sensitive_name):
* x = non_sensitive_name() # <-- bad source
* print(x) # <-- sink
*
* import not_found
* f = not_found.get_passwd # <-- good source
* my_func(f)
* ```
*/
predicate extraStepForCalls(DataFlow::Node nodeFrom, DataFlow::CallCfgNode nodeTo) {
// However, we do still use the type-tracking approach to limit the size of this
// predicate.
nodeTo.getFunction() = nodeFrom and
nodeFrom = possibleSensitiveCallable()
}
/**
* Any kind of variable assignment (also including with/for) where the name indicates
* it contains sensitive data.
*
* Note: We _could_ make any access to a variable with a sensitive name a source of
* sensitive data, but to make path explanations in data-flow/taint-tracking good,
* we don't want that, since it works against allowing users to understand the flow
* in the program (which is the whole point).
*
* Note: To make data-flow/taint-tracking work, the expression that is _assigned_ to
* the variable is marked as the source (as compared to marking the variable as the
* source).
*/
class SensitiveVariableAssignment extends SensitiveDataSource::Range {
SensitiveDataClassification classification;
SensitiveVariableAssignment() {
exists(DefinitionNode def |
nameIndicatesSensitiveData(def.(NameNode).getId(), classification) and
(
this.asCfgNode() = def.getValue()
or
this.asCfgNode() = def.getValue().(ForNode).getSequence()
) and
not this.asExpr() instanceof FunctionExpr and
not this.asExpr() instanceof ClassExpr
)
or
exists(With with |
nameIndicatesSensitiveData(with.getOptionalVars().(Name).getId(), classification) and
this.asExpr() = with.getContextExpr()
)
}
override SensitiveDataClassification getClassification() { result = classification }
}
/** An attribute access that is considered a source of sensitive data. */
class SensitiveAttributeAccess extends SensitiveDataSource::Range {
SensitiveDataClassification classification;
SensitiveAttributeAccess() {
// Things like `foo.<sensitive-name>` or `from <module> import <sensitive-name>`
// I considered excluding any `from ... import something_sensitive`, but then realized that
// we should flag up `form ... import password as ...` as a password
nameIndicatesSensitiveData(this.(DataFlow::AttrRead).getAttributeName(), classification)
or
// Things like `getattr(foo, <reference-to-string>)`
this.(DataFlow::AttrRead).getAttributeNameExpr() = sensitiveLookupStringConst(classification)
}
override SensitiveDataClassification getClassification() { result = classification }
}
/** A subscript, where the key indicates the result will be sensitive data. */
class SensitiveSubscript extends SensitiveDataSource::Range {
SensitiveDataClassification classification;
SensitiveSubscript() {
this.asCfgNode().(SubscriptNode).getIndex() =
sensitiveLookupStringConst(classification).asCfgNode()
}
override SensitiveDataClassification getClassification() { result = classification }
}
/** A call to `get` on an object, where the key indicates the result will be sensitive data. */
class SensitiveGetCall extends SensitiveDataSource::Range, DataFlow::CallCfgNode {
SensitiveDataClassification classification;
SensitiveGetCall() {
this.getFunction().asCfgNode().(AttrNode).getName() = "get" and
this.getArg(0) = sensitiveLookupStringConst(classification)
}
override SensitiveDataClassification getClassification() { result = classification }
}
/** A parameter where the name indicates it will receive sensitive data. */
class SensitiveParameter extends SensitiveDataSource::Range, DataFlow::ParameterNode {
SensitiveDataClassification classification;
SensitiveParameter() {
nameIndicatesSensitiveData(this.getParameter().getName(), classification)
}
override SensitiveDataClassification getClassification() { result = classification }
}
}
predicate sensitiveDataExtraStepForCalls = SensitiveDataModeling::extraStepForCalls/2;

432
python/ql/src/semmle/python/dataflow/new/TypeTracker.qll
View File

@@ -1,167 +1,16 @@
/** Step Summaries and Type Tracking */
/**
* This file acts as a wrapper for `internal.TypeTracker`, exposing some of the functionality with
* names that are more appropriate for Python.
*/
private import python
private import internal.DataFlowPublic
private import internal.DataFlowPrivate
private import internal.TypeTracker as Internal
/** Any string that may appear as the name of an attribute or access path. */
class AttributeName extends string {
AttributeName() { this = any(AttrRef a).getAttributeName() }
}
class AttributeName = Internal::ContentName;
/** Either an attribute name, or the empty string (representing no attribute). */
class OptionalAttributeName extends string {
OptionalAttributeName() { this instanceof AttributeName or this = "" }
}
/**
* A description of a step on an inter-procedural data flow path.
*/
private newtype TStepSummary =
LevelStep() or
CallStep() or
ReturnStep() or
StoreStep(AttributeName attr) or
LoadStep(AttributeName attr)
/**
* INTERNAL: Use `TypeTracker` or `TypeBackTracker` instead.
*
* A description of a step on an inter-procedural data flow path.
*/
class StepSummary extends TStepSummary {
/** Gets a textual representation of this step summary. */
string toString() {
this instanceof LevelStep and result = "level"
or
this instanceof CallStep and result = "call"
or
this instanceof ReturnStep and result = "return"
or
exists(string attr | this = StoreStep(attr) | result = "store " + attr)
or
exists(string attr | this = LoadStep(attr) | result = "load " + attr)
}
}
/** Provides predicates for updating step summaries (`StepSummary`s). */
module StepSummary {
/**
* Gets the summary that corresponds to having taken a forwards
* heap and/or inter-procedural step from `nodeFrom` to `nodeTo`.
*/
cached
predicate step(LocalSourceNode nodeFrom, LocalSourceNode nodeTo, StepSummary summary) {
exists(Node mid | nodeFrom.flowsTo(mid) and smallstep(mid, nodeTo, summary))
}
/**
* Gets the summary that corresponds to having taken a forwards
* local, heap and/or inter-procedural step from `nodeFrom` to `nodeTo`.
*
* Unlike `StepSummary::step`, this predicate does not compress
* type-preserving steps.
*/
predicate smallstep(Node nodeFrom, Node nodeTo, StepSummary summary) {
jumpStep(nodeFrom, nodeTo) and
summary = LevelStep()
or
callStep(nodeFrom, nodeTo) and summary = CallStep()
or
returnStep(nodeFrom, nodeTo) and
summary = ReturnStep()
or
exists(string attr |
basicStoreStep(nodeFrom, nodeTo, attr) and
summary = StoreStep(attr)
or
basicLoadStep(nodeFrom, nodeTo, attr) and summary = LoadStep(attr)
)
}
}
/**
* Gets a callable for the call where `nodeFrom` is used as the `i`'th argument.
*
* Helper predicate to avoid bad join order experienced in `callStep`.
* This happened when `isParameterOf` was joined _before_ `getCallable`.
*/
pragma[nomagic]
private DataFlowCallable getCallableForArgument(ArgumentNode nodeFrom, int i) {
exists(DataFlowCall call |
nodeFrom.argumentOf(call, i) and
result = call.getCallable()
)
}
/** Holds if `nodeFrom` steps to `nodeTo` by being passed as a parameter in a call. */
predicate callStep(ArgumentNode nodeFrom, ParameterNode nodeTo) {
// TODO: Support special methods?
exists(DataFlowCallable callable, int i |
callable = getCallableForArgument(nodeFrom, i) and
nodeTo.isParameterOf(callable, i)
)
}
/** Holds if `nodeFrom` steps to `nodeTo` by being returned from a call. */
predicate returnStep(ReturnNode nodeFrom, Node nodeTo) {
exists(DataFlowCall call |
nodeFrom.getEnclosingCallable() = call.getCallable() and nodeTo.asCfgNode() = call.getNode()
)
}
/**
* Holds if `nodeFrom` is being written to the `attr` attribute of the object in `nodeTo`.
*
* Note that the choice of `nodeTo` does not have to make sense "chronologically".
* All we care about is whether the `attr` attribute of `nodeTo` can have a specific type,
* and the assumption is that if a specific type appears here, then any access of that
* particular attribute can yield something of that particular type.
*
* Thus, in an example such as
*
* ```python
* def foo(y):
* x = Foo()
* bar(x)
* x.attr = y
* baz(x)
*
* def bar(x):
* z = x.attr
* ```
* for the attribute write `x.attr = y`, we will have `attr` being the literal string `"attr"`,
* `nodeFrom` will be `y`, and `nodeTo` will be the object `Foo()` created on the first line of the
* function. This means we will track the fact that `x.attr` can have the type of `y` into the
* assignment to `z` inside `bar`, even though this attribute write happens _after_ `bar` is called.
*/
predicate basicStoreStep(Node nodeFrom, LocalSourceNode nodeTo, string attr) {
exists(AttrWrite a |
a.mayHaveAttributeName(attr) and
nodeFrom = a.getValue() and
nodeTo.flowsTo(a.getObject())
)
}
/**
* Holds if `nodeTo` is the result of accessing the `attr` attribute of `nodeFrom`.
*/
predicate basicLoadStep(Node nodeFrom, Node nodeTo, string attr) {
exists(AttrRead a |
a.mayHaveAttributeName(attr) and
nodeFrom = a.getObject() and
nodeTo = a
)
}
/**
* A utility class that is equivalent to `boolean` but does not require type joining.
*/
private class Boolean extends boolean {
Boolean() { this = true or this = false }
}
private newtype TTypeTracker = MkTypeTracker(Boolean hasCall, OptionalAttributeName attr)
class OptionalAttributeName = Internal::OptionalContentName;
/**
* Summary of the steps needed to track a value to a given dataflow node.
@@ -173,8 +22,8 @@ private newtype TTypeTracker = MkTypeTracker(Boolean hasCall, OptionalAttributeN
*
* It is recommended that all uses of this type are written in the following form,
* for tracking some type `myType`:
* ```
* private DataFlow::LocalSourceNode myType(DataFlow::TypeTracker t) {
* ```ql
* DataFlow::LocalSourceNode myType(DataFlow::TypeTracker t) {
* t.start() and
* result = < source of myType >
* or
@@ -183,279 +32,34 @@ private newtype TTypeTracker = MkTypeTracker(Boolean hasCall, OptionalAttributeN
* )
* }
*
* DataFlow::Node myType() { myType(DataFlow::TypeTracker::end()).flowsTo(result) }
* DataFlow::LocalSourceNode myType() { myType(DataFlow::TypeTracker::end()) }
* ```
*
* Instead of `result = myType(t2).track(t2, t)`, you can also use the equivalent
* `t = t2.step(myType(t2), result)`. If you additionally want to track individual
* intra-procedural steps, use `t = t2.smallstep(myCallback(t2), result)`.
*/
class TypeTracker extends TTypeTracker {
Boolean hasCall;
OptionalAttributeName attr;
TypeTracker() { this = MkTypeTracker(hasCall, attr) }
/** Gets the summary resulting from appending `step` to this type-tracking summary. */
cached
TypeTracker append(StepSummary step) {
step = LevelStep() and result = this
or
step = CallStep() and result = MkTypeTracker(true, attr)
or
step = ReturnStep() and hasCall = false and result = this
or
step = LoadStep(attr) and result = MkTypeTracker(hasCall, "")
or
exists(string p | step = StoreStep(p) and attr = "" and result = MkTypeTracker(hasCall, p))
}
/** Gets a textual representation of this summary. */
string toString() {
exists(string withCall, string withAttr |
(if hasCall = true then withCall = "with" else withCall = "without") and
(if attr != "" then withAttr = " with attribute " + attr else withAttr = "") and
result = "type tracker " + withCall + " call steps" + withAttr
)
}
/**
* Holds if this is the starting point of type tracking.
*/
predicate start() { hasCall = false and attr = "" }
class TypeTracker extends Internal::TypeTracker {
/**
* Holds if this is the starting point of type tracking, and the value starts in the attribute named `attrName`.
* The type tracking only ends after the attribute has been loaded.
*/
predicate startInAttr(AttributeName attrName) { hasCall = false and attr = attrName }
/**
* Holds if this is the starting point of type tracking
* when tracking a parameter into a call, but not out of it.
*/
predicate call() { hasCall = true and attr = "" }
/**
* Holds if this is the end point of type tracking.
*/
predicate end() { attr = "" }
/**
* INTERNAL. DO NOT USE.
*
* Holds if this type has been tracked into a call.
*/
boolean hasCall() { result = hasCall }
predicate startInAttr(string attrName) { this.startInContent(attrName) }
/**
* INTERNAL. DO NOT USE.
*
* Gets the attribute associated with this type tracker.
*/
string getAttr() { result = attr }
/**
* Gets a type tracker that starts where this one has left off to allow continued
* tracking.
*
* This predicate is only defined if the type has not been tracked into an attribute.
*/
TypeTracker continue() { attr = "" and result = this }
/**
* Gets the summary that corresponds to having taken a forwards
* heap and/or inter-procedural step from `nodeFrom` to `nodeTo`.
*/
pragma[inline]
TypeTracker step(LocalSourceNode nodeFrom, LocalSourceNode nodeTo) {
exists(StepSummary summary |
StepSummary::step(nodeFrom, pragma[only_bind_out](nodeTo), pragma[only_bind_into](summary)) and
result = this.append(pragma[only_bind_into](summary))
)
}
/**
* Gets the summary that corresponds to having taken a forwards
* local, heap and/or inter-procedural step from `nodeFrom` to `nodeTo`.
*
* Unlike `TypeTracker::step`, this predicate exposes all edges
* in the flow graph, and not just the edges between `Node`s.
* It may therefore be less performant.
*
* Type tracking predicates using small steps typically take the following form:
* ```ql
* DataFlow::Node myType(DataFlow::TypeTracker t) {
* t.start() and
* result = < source of myType >
* or
* exists (DataFlow::TypeTracker t2 |
* t = t2.smallstep(myType(t2), result)
* )
* }
*
* DataFlow::Node myType() {
* result = myType(DataFlow::TypeTracker::end())
* }
* ```
*/
pragma[inline]
TypeTracker smallstep(Node nodeFrom, Node nodeTo) {
exists(StepSummary summary |
StepSummary::smallstep(nodeFrom, nodeTo, summary) and
result = this.append(summary)
)
or
simpleLocalFlowStep(nodeFrom, nodeTo) and
result = this
}
string getAttr() { result = this.getContent() }
}
/** Provides predicates for implementing custom `TypeTracker`s. */
module TypeTracker {
/**
* Gets a valid end point of type tracking.
*/
TypeTracker end() { result.end() }
}
module TypeTracker = Internal::TypeTracker;
private newtype TTypeBackTracker = MkTypeBackTracker(Boolean hasReturn, OptionalAttributeName attr)
class StepSummary = Internal::StepSummary;
/**
* Summary of the steps needed to back-track a use of a value to a given dataflow node.
*
* This can for example be used to track callbacks that are passed to a certain API,
* so we can model specific parameters of that callback as having a certain type.
*
* Note that type back-tracking does not provide a source/sink relation, that is,
* it may determine that a node will be used in an API call somewhere, but it won't
* determine exactly where that use was, or the path that led to the use.
*
* It is recommended that all uses of this type are written in the following form,
* for back-tracking some callback type `myCallback`:
*
* ```
* private DataFlow::LocalSourceNode myCallback(DataFlow::TypeBackTracker t) {
* t.start() and
* result = (< some API call >).getArgument(< n >).getALocalSource()
* or
* exists (DataFlow::TypeBackTracker t2 |
* result = myCallback(t2).backtrack(t2, t)
* )
* }
*
* DataFlow::LocalSourceNode myCallback() { result = myCallback(DataFlow::TypeBackTracker::end()) }
* ```
*
* Instead of `result = myCallback(t2).backtrack(t2, t)`, you can also use the equivalent
* `t2 = t.step(result, myCallback(t2))`. If you additionally want to track individual
* intra-procedural steps, use `t2 = t.smallstep(result, myCallback(t2))`.
*/
class TypeBackTracker extends TTypeBackTracker {
Boolean hasReturn;
string attr;
module StepSummary = Internal::StepSummary;
TypeBackTracker() { this = MkTypeBackTracker(hasReturn, attr) }
class TypeBackTracker = Internal::TypeBackTracker;
/** Gets the summary resulting from prepending `step` to this type-tracking summary. */
TypeBackTracker prepend(StepSummary step) {
step = LevelStep() and result = this
or
step = CallStep() and hasReturn = false and result = this
or
step = ReturnStep() and result = MkTypeBackTracker(true, attr)
or
exists(string p | step = LoadStep(p) and attr = "" and result = MkTypeBackTracker(hasReturn, p))
or
step = StoreStep(attr) and result = MkTypeBackTracker(hasReturn, "")
}
/** Gets a textual representation of this summary. */
string toString() {
exists(string withReturn, string withAttr |
(if hasReturn = true then withReturn = "with" else withReturn = "without") and
(if attr != "" then withAttr = " with attribute " + attr else withAttr = "") and
result = "type back-tracker " + withReturn + " return steps" + withAttr
)
}
/**
* Holds if this is the starting point of type tracking.
*/
predicate start() { hasReturn = false and attr = "" }
/**
* Holds if this is the end point of type tracking.
*/
predicate end() { attr = "" }
/**
* INTERNAL. DO NOT USE.
*
* Holds if this type has been back-tracked into a call through return edge.
*/
boolean hasReturn() { result = hasReturn }
/**
* Gets a type tracker that starts where this one has left off to allow continued
* tracking.
*
* This predicate is only defined if the type has not been tracked into an attribute.
*/
TypeBackTracker continue() { attr = "" and result = this }
/**
* Gets the summary that corresponds to having taken a backwards
* heap and/or inter-procedural step from `nodeTo` to `nodeFrom`.
*/
pragma[inline]
TypeBackTracker step(LocalSourceNode nodeFrom, LocalSourceNode nodeTo) {
exists(StepSummary summary |
StepSummary::step(pragma[only_bind_out](nodeFrom), nodeTo, pragma[only_bind_into](summary)) and
this = result.prepend(pragma[only_bind_into](summary))
)
}
/**
* Gets the summary that corresponds to having taken a backwards
* local, heap and/or inter-procedural step from `nodeTo` to `nodeFrom`.
*
* Unlike `TypeBackTracker::step`, this predicate exposes all edges
* in the flowgraph, and not just the edges between
* `LocalSourceNode`s. It may therefore be less performant.
*
* Type tracking predicates using small steps typically take the following form:
* ```ql
* DataFlow::Node myType(DataFlow::TypeBackTracker t) {
* t.start() and
* result = < some API call >.getArgument(< n >)
* or
* exists (DataFlow::TypeBackTracker t2 |
* t = t2.smallstep(result, myType(t2))
* )
* }
*
* DataFlow::Node myType() {
* result = myType(DataFlow::TypeBackTracker::end())
* }
* ```
*/
pragma[inline]
TypeBackTracker smallstep(Node nodeFrom, Node nodeTo) {
exists(StepSummary summary |
StepSummary::smallstep(nodeFrom, nodeTo, summary) and
this = result.prepend(summary)
)
or
simpleLocalFlowStep(nodeFrom, nodeTo) and
this = result
}
}
/** Provides predicates for implementing custom `TypeBackTracker`s. */
module TypeBackTracker {
/**
* Gets a valid end point of type back-tracking.
*/
TypeBackTracker end() { result.end() }
}
module TypeBackTracker = Internal::TypeBackTracker;

2
python/ql/src/semmle/python/dataflow/new/internal/Attributes.qll
View File

@@ -200,7 +200,7 @@ private class ClassDefinitionAsAttrWrite extends AttrWrite, CfgNode {
* - Dynamic attribute reads using `getattr`: `getattr(object, attr)`
* - Qualified imports: `from module import attr as name`
*/
abstract class AttrRead extends AttrRef, Node { }
abstract class AttrRead extends AttrRef, Node, LocalSourceNode { }
/** A simple attribute read, e.g. `object.attr` */
private class AttributeReadAsAttrRead extends AttrRead, CfgNode {

8
python/ql/src/semmle/python/dataflow/new/internal/DataFlowImpl.qll
View File

@@ -2220,7 +2220,9 @@ private module Stage4 {
bindingset[node, cc, config]
private LocalCc getLocalCc(Node node, Cc cc, Configuration config) {
localFlowEntry(node, config) and
result = getLocalCallContext(pragma[only_bind_out](cc), getNodeEnclosingCallable(node))
result =
getLocalCallContext(pragma[only_bind_into](pragma[only_bind_out](cc)),
getNodeEnclosingCallable(node))
}
private predicate localStep(
@@ -3255,7 +3257,9 @@ private predicate pathStep(PathNodeMid mid, Node node, CallContext cc, SummaryCt
conf = mid.getConfiguration() and
cc = mid.getCallContext() and
sc = mid.getSummaryCtx() and
localCC = getLocalCallContext(pragma[only_bind_out](cc), getNodeEnclosingCallable(midnode)) and
localCC =
getLocalCallContext(pragma[only_bind_into](pragma[only_bind_out](cc)),
getNodeEnclosingCallable(midnode)) and
ap0 = mid.getAp()
|
localFlowBigStep(midnode, node, true, _, conf, localCC) and

8
python/ql/src/semmle/python/dataflow/new/internal/DataFlowImpl2.qll
View File

@@ -2220,7 +2220,9 @@ private module Stage4 {
bindingset[node, cc, config]
private LocalCc getLocalCc(Node node, Cc cc, Configuration config) {
localFlowEntry(node, config) and
result = getLocalCallContext(pragma[only_bind_out](cc), getNodeEnclosingCallable(node))
result =
getLocalCallContext(pragma[only_bind_into](pragma[only_bind_out](cc)),
getNodeEnclosingCallable(node))
}
private predicate localStep(
@@ -3255,7 +3257,9 @@ private predicate pathStep(PathNodeMid mid, Node node, CallContext cc, SummaryCt
conf = mid.getConfiguration() and
cc = mid.getCallContext() and
sc = mid.getSummaryCtx() and
localCC = getLocalCallContext(pragma[only_bind_out](cc), getNodeEnclosingCallable(midnode)) and
localCC =
getLocalCallContext(pragma[only_bind_into](pragma[only_bind_out](cc)),
getNodeEnclosingCallable(midnode)) and
ap0 = mid.getAp()
|
localFlowBigStep(midnode, node, true, _, conf, localCC) and

8
python/ql/src/semmle/python/dataflow/new/internal/DataFlowImpl3.qll
View File

@@ -2220,7 +2220,9 @@ private module Stage4 {
bindingset[node, cc, config]
private LocalCc getLocalCc(Node node, Cc cc, Configuration config) {
localFlowEntry(node, config) and
result = getLocalCallContext(pragma[only_bind_out](cc), getNodeEnclosingCallable(node))
result =
getLocalCallContext(pragma[only_bind_into](pragma[only_bind_out](cc)),
getNodeEnclosingCallable(node))
}
private predicate localStep(
@@ -3255,7 +3257,9 @@ private predicate pathStep(PathNodeMid mid, Node node, CallContext cc, SummaryCt
conf = mid.getConfiguration() and
cc = mid.getCallContext() and
sc = mid.getSummaryCtx() and
localCC = getLocalCallContext(pragma[only_bind_out](cc), getNodeEnclosingCallable(midnode)) and
localCC =
getLocalCallContext(pragma[only_bind_into](pragma[only_bind_out](cc)),
getNodeEnclosingCallable(midnode)) and
ap0 = mid.getAp()
|
localFlowBigStep(midnode, node, true, _, conf, localCC) and

8
python/ql/src/semmle/python/dataflow/new/internal/DataFlowImpl4.qll
View File

@@ -2220,7 +2220,9 @@ private module Stage4 {
bindingset[node, cc, config]
private LocalCc getLocalCc(Node node, Cc cc, Configuration config) {
localFlowEntry(node, config) and
result = getLocalCallContext(pragma[only_bind_out](cc), getNodeEnclosingCallable(node))
result =
getLocalCallContext(pragma[only_bind_into](pragma[only_bind_out](cc)),
getNodeEnclosingCallable(node))
}
private predicate localStep(
@@ -3255,7 +3257,9 @@ private predicate pathStep(PathNodeMid mid, Node node, CallContext cc, SummaryCt
conf = mid.getConfiguration() and
cc = mid.getCallContext() and
sc = mid.getSummaryCtx() and
localCC = getLocalCallContext(pragma[only_bind_out](cc), getNodeEnclosingCallable(midnode)) and
localCC =
getLocalCallContext(pragma[only_bind_into](pragma[only_bind_out](cc)),
getNodeEnclosingCallable(midnode)) and
ap0 = mid.getAp()
|
localFlowBigStep(midnode, node, true, _, conf, localCC) and

8
python/ql/src/semmle/python/dataflow/new/internal/DataFlowImplConsistency.qll
View File

@@ -168,7 +168,13 @@ module Consistency {
msg = "ArgumentNode is missing PostUpdateNode."
}
query predicate postWithInFlow(PostUpdateNode n, string msg) {
// This predicate helps the compiler forget that in some languages
// it is impossible for a `PostUpdateNode` to be the target of
// `simpleLocalFlowStep`.
private predicate isPostUpdateNode(Node n) { n instanceof PostUpdateNode or none() }
query predicate postWithInFlow(Node n, string msg) {
isPostUpdateNode(n) and
simpleLocalFlowStep(_, n) and
msg = "PostUpdateNode should not be the target of local flow."
}

39
python/ql/src/semmle/python/dataflow/new/internal/DataFlowPublic.qll
View File

@@ -183,6 +183,45 @@ class CallCfgNode extends CfgNode, LocalSourceNode {
Node getArgByName(string name) { result.asCfgNode() = node.getArgByName(name) }
}
/**
* A data-flow node corresponding to a method call, that is `foo.bar(...)`.
*
* Also covers the case where the method lookup is done separately from the call itself, as in
* `temp = foo.bar; temp(...)`. Note that this is only tracked through local scope.
*/
class MethodCallNode extends CallCfgNode {
AttrRead method_lookup;
MethodCallNode() { method_lookup = this.getFunction().getALocalSource() }
/**
* Gets the name of the method being invoked (the `bar` in `foo.bar(...)`) if it can be determined.
*
* Note that this method may have multiple results if a single call node represents calls to
* multiple different objects and methods. If you want to link up objects and method names
* accurately, use the `calls` method instead.
*/
string getMethodName() { result = method_lookup.getAttributeName() }
/**
* Gets the data-flow node corresponding to the object receiving this call. That is, the `foo` in
* `foo.bar(...)`.
*
* Note that this method may have multiple results if a single call node represents calls to
* multiple different objects and methods. If you want to link up objects and method names
* accurately, use the `calls` method instead.
*/
Node getObject() { result = method_lookup.getObject() }
/** Holds if this data-flow node calls method `methodName` on the object node `object`. */
predicate calls(Node object, string methodName) {
// As `getObject` and `getMethodName` may both have multiple results, we must look up the object
// and method name directly on `method_lookup`.
object = method_lookup.getObject() and
methodName = method_lookup.getAttributeName()
}
}
/**
* An expression, viewed as a node in a data flow graph.
*

10
python/ql/src/semmle/python/dataflow/new/internal/LocalSources.qll
View File

@@ -78,6 +78,16 @@ class LocalSourceNode extends Node {
*/
CallCfgNode getACall() { Cached::call(this, result) }
/**
* Gets a call to the method `methodName` on this node.
*
* Includes both calls that have the syntactic shape of a method call (as in `obj.m(...)`), and
* calls where the callee undergoes some additional local data flow (as in `tmp = obj.m; m(...)`).
*/
MethodCallNode getAMethodCall(string methodName) {
result = this.getAnAttributeRead(methodName).getACall()
}
/**
* Gets a node that this node may flow to using one heap and/or interprocedural step.
*

135
python/ql/src/experimental/typetracking/TypeTracker.qll → python/ql/src/semmle/python/dataflow/new/internal/TypeTracker.qll
View File

@@ -23,15 +23,57 @@ class OptionalContentName extends string {
OptionalContentName() { this instanceof ContentName or this = "" }
}
/**
* A description of a step on an inter-procedural data flow path.
*/
private newtype TStepSummary =
LevelStep() or
CallStep() or
ReturnStep() or
StoreStep(ContentName content) or
LoadStep(ContentName content)
cached
private module Cached {
/**
* A description of a step on an inter-procedural data flow path.
*/
cached
newtype TStepSummary =
LevelStep() or
CallStep() or
ReturnStep() or
StoreStep(ContentName content) or
LoadStep(ContentName content)
/** Gets the summary resulting from appending `step` to type-tracking summary `tt`. */
cached
TypeTracker append(TypeTracker tt, StepSummary step) {
exists(Boolean hasCall, OptionalContentName content | tt = MkTypeTracker(hasCall, content) |
step = LevelStep() and result = tt
or
step = CallStep() and result = MkTypeTracker(true, content)
or
step = ReturnStep() and hasCall = false and result = tt
or
step = LoadStep(content) and result = MkTypeTracker(hasCall, "")
or
exists(string p | step = StoreStep(p) and content = "" and result = MkTypeTracker(hasCall, p))
)
}
/**
* Gets the summary that corresponds to having taken a forwards
* heap and/or intra-procedural step from `nodeFrom` to `nodeTo`.
*
* Steps contained in this predicate should _not_ depend on the call graph.
*/
cached
predicate stepNoCall(LocalSourceNode nodeFrom, LocalSourceNode nodeTo, StepSummary summary) {
exists(Node mid | nodeFrom.flowsTo(mid) and smallstepNoCall(mid, nodeTo, summary))
}
/**
* Gets the summary that corresponds to having taken a forwards
* inter-procedural step from `nodeFrom` to `nodeTo`.
*/
cached
predicate stepCall(LocalSourceNode nodeFrom, LocalSourceNode nodeTo, StepSummary summary) {
exists(Node mid | nodeFrom.flowsTo(mid) and smallstepCall(mid, nodeTo, summary))
}
}
private import Cached
/**
* INTERNAL: Use `TypeTracker` or `TypeBackTracker` instead.
@@ -53,28 +95,29 @@ class StepSummary extends TStepSummary {
}
}
pragma[noinline]
private predicate smallstepNoCall(Node nodeFrom, LocalSourceNode nodeTo, StepSummary summary) {
jumpStep(nodeFrom, nodeTo) and
summary = LevelStep()
or
exists(string content |
StepSummary::localSourceStoreStep(nodeFrom, nodeTo, content) and
summary = StoreStep(content)
or
basicLoadStep(nodeFrom, nodeTo, content) and summary = LoadStep(content)
)
}
pragma[noinline]
private predicate smallstepCall(Node nodeFrom, LocalSourceNode nodeTo, StepSummary summary) {
callStep(nodeFrom, nodeTo) and summary = CallStep()
or
returnStep(nodeFrom, nodeTo) and
summary = ReturnStep()
}
/** Provides predicates for updating step summaries (`StepSummary`s). */
module StepSummary {
/**
* Gets the summary that corresponds to having taken a forwards
* heap and/or intra-procedural step from `nodeFrom` to `nodeTo`.
*
* Steps contained in this predicate should _not_ depend on the call graph.
*/
cached
private predicate stepNoCall(LocalSourceNode nodeFrom, LocalSourceNode nodeTo, StepSummary summary) {
exists(Node mid | nodeFrom.flowsTo(mid) and smallstepNoCall(mid, nodeTo, summary))
}
/**
* Gets the summary that corresponds to having taken a forwards
* inter-procedural step from `nodeFrom` to `nodeTo`.
*/
cached
private predicate stepCall(LocalSourceNode nodeFrom, LocalSourceNode nodeTo, StepSummary summary) {
exists(Node mid | nodeFrom.flowsTo(mid) and smallstepCall(mid, nodeTo, summary))
}
/**
* Gets the summary that corresponds to having taken a forwards
* heap and/or inter-procedural step from `nodeFrom` to `nodeTo`.
@@ -92,27 +135,6 @@ module StepSummary {
stepCall(nodeFrom, nodeTo, summary)
}
pragma[noinline]
private predicate smallstepNoCall(Node nodeFrom, LocalSourceNode nodeTo, StepSummary summary) {
jumpStep(nodeFrom, nodeTo) and
summary = LevelStep()
or
exists(string content |
localSourceStoreStep(nodeFrom, nodeTo, content) and
summary = StoreStep(content)
or
basicLoadStep(nodeFrom, nodeTo, content) and summary = LoadStep(content)
)
}
pragma[noinline]
private predicate smallstepCall(Node nodeFrom, LocalSourceNode nodeTo, StepSummary summary) {
callStep(nodeFrom, nodeTo) and summary = CallStep()
or
returnStep(nodeFrom, nodeTo) and
summary = ReturnStep()
}
/**
* Gets the summary that corresponds to having taken a forwards
* local, heap and/or inter-procedural step from `nodeFrom` to `nodeTo`.
@@ -193,18 +215,7 @@ class TypeTracker extends TTypeTracker {
TypeTracker() { this = MkTypeTracker(hasCall, content) }
/** Gets the summary resulting from appending `step` to this type-tracking summary. */
cached
TypeTracker append(StepSummary step) {
step = LevelStep() and result = this
or
step = CallStep() and result = MkTypeTracker(true, content)
or
step = ReturnStep() and hasCall = false and result = this
or
step = LoadStep(content) and result = MkTypeTracker(hasCall, "")
or
exists(string p | step = StoreStep(p) and content = "" and result = MkTypeTracker(hasCall, p))
}
TypeTracker append(StepSummary step) { result = append(this, step) }
/** Gets a textual representation of this summary. */
string toString() {

0
python/ql/src/experimental/typetracking/TypeTrackerSpecific.qll → python/ql/src/semmle/python/dataflow/new/internal/TypeTrackerSpecific.qll
View File

52
python/ql/src/semmle/python/frameworks/Aioch.qll Normal file
View File

@@ -0,0 +1,52 @@
/**
* Provides classes modeling security-relevant aspects of the `aioch` PyPI package (an
* async-io version of the `clickhouse-driver` PyPI package).
*
* See https://pypi.org/project/aioch/
*/
private import python
private import semmle.python.Concepts
private import semmle.python.ApiGraphs
private import semmle.python.frameworks.PEP249
private import semmle.python.frameworks.ClickhouseDriver
/**
* INTERNAL: Do not use.
*
* Provides models for `aioch` PyPI package (an async-io version of the
* `clickhouse-driver` PyPI package).
*
* See https://pypi.org/project/aioch/
*/
module Aioch {
/** Provides models for `aioch.Client` class and subclasses. */
module Client {
/** Gets a reference to the `aioch.Client` class or any subclass. */
API::Node subclassRef() {
result = API::moduleImport("aioch").getMember("Client").getASubclass*()
}
/** Gets a reference to an instance of `clickhouse_driver.Client` or any subclass. */
API::Node instance() { result = subclassRef().getReturn() }
}
/**
* A call to any of the the execute methods on a `aioch.Client`, which are just async
* versions of the methods in the `clickhouse-driver` PyPI package.
*
* See
* - https://clickhouse-driver.readthedocs.io/en/latest/api.html#clickhouse_driver.Client.execute
* - https://clickhouse-driver.readthedocs.io/en/latest/api.html#clickhouse_driver.Client.execute_iter
* - https://clickhouse-driver.readthedocs.io/en/latest/api.html#clickhouse_driver.Client.execute_with_progress
*/
class ClientExecuteCall extends SqlExecution::Range, DataFlow::CallCfgNode {
ClientExecuteCall() {
exists(string methodName | methodName = ClickhouseDriver::getExecuteMethodName() |
this = Client::instance().getMember(methodName).getACall()
)
}
override DataFlow::Node getSql() { result in [this.getArg(0), this.getArgByName("query")] }
}
}

65
python/ql/src/semmle/python/frameworks/ClickhouseDriver.qll Normal file
View File

@@ -0,0 +1,65 @@
/**
* Provides classes modeling security-relevant aspects of the `clickhouse-driver` PyPI package.
* See
* - https://pypi.org/project/clickhouse-driver/
* - https://clickhouse-driver.readthedocs.io/en/latest/
*/
private import python
private import semmle.python.Concepts
private import semmle.python.ApiGraphs
private import semmle.python.frameworks.PEP249
/**
* INTERNAL: Do not use.
*
* Provides models for `clickhouse-driver` PyPI package (imported as `clickhouse_driver`).
* See
* - https://pypi.org/project/clickhouse-driver/
* - https://clickhouse-driver.readthedocs.io/en/latest/
*/
module ClickhouseDriver {
/**
* `clickhouse_driver` implements PEP249,
* providing ways to execute SQL statements against a database.
*/
class ClickHouseDriverPEP249 extends PEP249ModuleApiNode {
ClickHouseDriverPEP249() { this = API::moduleImport("clickhouse_driver") }
}
/** Provides models for `clickhouse_driver.Client` class and subclasses. */
module Client {
/** Gets a reference to the `clickhouse_driver.Client` class or any subclass. */
API::Node subclassRef() {
exists(API::Node classRef |
// canonical definition
classRef = API::moduleImport("clickhouse_driver").getMember("client").getMember("Client")
or
// commonly used alias
classRef = API::moduleImport("clickhouse_driver").getMember("Client")
|
result = classRef.getASubclass*()
)
}
/** Gets a reference to an instance of `clickhouse_driver.Client` or any subclass. */
API::Node instance() { result = subclassRef().getReturn() }
}
/** `clickhouse_driver.Client` execute method names */
string getExecuteMethodName() { result in ["execute_with_progress", "execute", "execute_iter"] }
/**
* A call to any of the the execute methods on a `clickhouse_driver.Client` method
*
* See
* - https://clickhouse-driver.readthedocs.io/en/latest/api.html#clickhouse_driver.Client.execute
* - https://clickhouse-driver.readthedocs.io/en/latest/api.html#clickhouse_driver.Client.execute_iter
* - https://clickhouse-driver.readthedocs.io/en/latest/api.html#clickhouse_driver.Client.execute_with_progress
*/
class ClientExecuteCall extends SqlExecution::Range, DataFlow::CallCfgNode {
ClientExecuteCall() { this = Client::instance().getMember(getExecuteMethodName()).getACall() }
override DataFlow::Node getSql() { result in [this.getArg(0), this.getArgByName("query")] }
}
}

36
python/ql/src/semmle/python/frameworks/Cryptography.qll
View File

@@ -228,11 +228,7 @@ private module CryptographyModel {
/** Gets a reference to the encryptor of a Cipher instance using algorithm with `algorithmName`. */
DataFlow::LocalSourceNode cipherEncryptor(DataFlow::TypeTracker t, string algorithmName) {
t.start() and
exists(DataFlow::AttrRead attr |
result.(DataFlow::CallCfgNode).getFunction() = attr and
attr.getAttributeName() = "encryptor" and
attr.getObject() = cipherInstance(algorithmName)
)
result.(DataFlow::MethodCallNode).calls(cipherInstance(algorithmName), "encryptor")
or
exists(DataFlow::TypeTracker t2 | result = cipherEncryptor(t2, algorithmName).track(t2, t))
}
@@ -249,11 +245,7 @@ private module CryptographyModel {
/** Gets a reference to the dncryptor of a Cipher instance using algorithm with `algorithmName`. */
DataFlow::LocalSourceNode cipherDecryptor(DataFlow::TypeTracker t, string algorithmName) {
t.start() and
exists(DataFlow::AttrRead attr |
result.(DataFlow::CallCfgNode).getFunction() = attr and
attr.getAttributeName() = "decryptor" and
attr.getObject() = cipherInstance(algorithmName)
)
result.(DataFlow::MethodCallNode).calls(cipherInstance(algorithmName), "decryptor")
or
exists(DataFlow::TypeTracker t2 | result = cipherDecryptor(t2, algorithmName).track(t2, t))
}
@@ -271,18 +263,14 @@ private module CryptographyModel {
* An encrypt or decrypt operation from `cryptography.hazmat.primitives.ciphers`.
*/
class CryptographyGenericCipherOperation extends Cryptography::CryptographicOperation::Range,
DataFlow::CallCfgNode {
DataFlow::MethodCallNode {
string algorithmName;
CryptographyGenericCipherOperation() {
exists(DataFlow::AttrRead attr |
this.getFunction() = attr and
attr.getAttributeName() = ["update", "update_into"] and
(
attr.getObject() = cipherEncryptor(algorithmName)
or
attr.getObject() = cipherDecryptor(algorithmName)
)
exists(DataFlow::Node object, string method |
object in [cipherEncryptor(algorithmName), cipherDecryptor(algorithmName)] and
method in ["update", "update_into"] and
this.calls(object, method)
)
}
@@ -337,16 +325,10 @@ private module CryptographyModel {
* An hashing operation from `cryptography.hazmat.primitives.hashes`.
*/
class CryptographyGenericHashOperation extends Cryptography::CryptographicOperation::Range,
DataFlow::CallCfgNode {
DataFlow::MethodCallNode {
string algorithmName;
CryptographyGenericHashOperation() {
exists(DataFlow::AttrRead attr |
this.getFunction() = attr and
attr.getAttributeName() = "update" and
attr.getObject() = hashInstance(algorithmName)
)
}
CryptographyGenericHashOperation() { this.calls(hashInstance(algorithmName), "update") }
override Cryptography::CryptographicAlgorithm getAlgorithm() {
result.matchesName(algorithmName)

6
python/ql/src/semmle/python/frameworks/MySQLdb.qll
View File

@@ -1,5 +1,7 @@
/**
* Provides classes modeling security-relevant aspects of the `MySQLdb` PyPI package.
* Provides classes modeling security-relevant aspects of the `MySQL-python` PyPI package
* (imported as `MySQLdb`).
*
* See
* - https://mysqlclient.readthedocs.io/index.html
* - https://pypi.org/project/MySQL-python/
@@ -13,7 +15,7 @@ private import semmle.python.ApiGraphs
private import semmle.python.frameworks.PEP249
/**
* Provides models for the `MySQLdb` PyPI package.
* Provides models for the `MySQL-python` PyPI package (imported as `MySQLdb`).
* See
* - https://mysqlclient.readthedocs.io/index.html
* - https://pypi.org/project/MySQL-python/

8
python/ql/src/semmle/python/frameworks/MysqlConnectorPython.qll → python/ql/src/semmle/python/frameworks/Mysql.qll
View File

@@ -1,5 +1,6 @@
/**
* Provides classes modeling security-relevant aspects of the `mysql-connector-python` package.
* Provides classes modeling security-relevant aspects of the `mysql-connector-python`
* and `mysql-connector` (old package name) PyPI packages (imported as `mysql`).
* See
* - https://dev.mysql.com/doc/connector-python/en/
* - https://dev.mysql.com/doc/connector-python/en/connector-python-example-connecting.html
@@ -13,12 +14,13 @@ private import semmle.python.ApiGraphs
private import semmle.python.frameworks.PEP249
/**
* Provides models for the `mysql-connector-python` package.
* Provides classes modeling security-relevant aspects of the `mysql-connector-python`
* and `mysql-connector` (old package name) PyPI packages (imported as `mysql`).
* See
* - https://dev.mysql.com/doc/connector-python/en/
* - https://dev.mysql.com/doc/connector-python/en/connector-python-example-connecting.html
*/
private module MysqlConnectorPython {
private module Mysql {
// ---------------------------------------------------------------------------
// mysql
// ---------------------------------------------------------------------------

250
python/ql/src/semmle/python/frameworks/Twisted.qll Normal file
View File

@@ -0,0 +1,250 @@
/**
* Provides classes modeling security-relevant aspects of the `twisted` PyPI package.
* See https://twistedmatrix.com/.
*/
private import python
private import semmle.python.dataflow.new.DataFlow
private import semmle.python.dataflow.new.RemoteFlowSources
private import semmle.python.dataflow.new.TaintTracking
private import semmle.python.Concepts
private import semmle.python.ApiGraphs
/**
* Provides models for the `twisted` PyPI package.
* See https://twistedmatrix.com/.
*/
private module Twisted {
// ---------------------------------------------------------------------------
// request handler modeling
// ---------------------------------------------------------------------------
/**
* A class that is a subclass of `twisted.web.resource.Resource`, thereby
* being able to handle HTTP requests.
*
* See https://twistedmatrix.com/documents/21.2.0/api/twisted.web.resource.Resource.html
*/
class TwistedResourceSubclass extends Class {
TwistedResourceSubclass() {
this.getABase() =
API::moduleImport("twisted")
.getMember("web")
.getMember("resource")
.getMember("Resource")
.getASubclass*()
.getAUse()
.asExpr()
}
/** Gets a function that could handle incoming requests, if any. */
Function getARequestHandler() {
// TODO: This doesn't handle attribute assignment. Should be OK, but analysis is not as complete as with
// points-to and `.lookup`, which would handle `post = my_post_handler` inside class def
result = this.getAMethod() and
exists(getRequestParamIndex(result.getName()))
}
}
/**
* Gets the index the request parameter is supposed to be at for the method named
* `methodName` in a `twisted.web.resource.Resource` subclass.
*/
bindingset[methodName]
private int getRequestParamIndex(string methodName) {
methodName.matches("render_%") and result = 1
or
methodName in ["render", "listDynamicEntities", "getChildForRequest"] and result = 1
or
methodName = ["getDynamicEntity", "getChild", "getChildWithDefault"] and result = 2
}
/** A method that handles incoming requests, on a `twisted.web.resource.Resource` subclass. */
class TwistedResourceRequestHandler extends HTTP::Server::RequestHandler::Range {
TwistedResourceRequestHandler() { this = any(TwistedResourceSubclass cls).getARequestHandler() }
Parameter getRequestParameter() { result = this.getArg(getRequestParamIndex(this.getName())) }
override Parameter getARoutedParameter() { none() }
override string getFramework() { result = "twisted" }
}
/**
* A "render" method on a `twisted.web.resource.Resource` subclass, whose return value
* is written as the body of the HTTP response.
*/
class TwistedResourceRenderMethod extends TwistedResourceRequestHandler {
TwistedResourceRenderMethod() {
this.getName() = "render" or this.getName().matches("render_%")
}
}
// ---------------------------------------------------------------------------
// request modeling
// ---------------------------------------------------------------------------
/**
* Provides models for the `twisted.web.server.Request` class
*
* See https://twistedmatrix.com/documents/21.2.0/api/twisted.web.server.Request.html
*/
module Request {
/**
* A source of instances of `twisted.web.server.Request`, extend this class to model new instances.
*
* This can include instantiations of the class, return values from function
* calls, or a special parameter that will be set when functions are called by an external
* library.
*
* Use `Request::instance()` predicate to get
* references to instances of `twisted.web.server.Request`.
*/
abstract class InstanceSource extends DataFlow::LocalSourceNode { }
/** Gets a reference to an instance of `twisted.web.server.Request`. */
private DataFlow::LocalSourceNode instance(DataFlow::TypeTracker t) {
t.start() and
result instanceof InstanceSource
or
exists(DataFlow::TypeTracker t2 | result = instance(t2).track(t2, t))
}
/** Gets a reference to an instance of `twisted.web.server.Request`. */
DataFlow::Node instance() { instance(DataFlow::TypeTracker::end()).flowsTo(result) }
}
/**
* A parameter that will receive a `twisted.web.server.Request` instance,
* when a twisted request handler is called.
*/
class TwistedResourceRequestHandlerRequestParam extends RemoteFlowSource::Range,
Request::InstanceSource, DataFlow::ParameterNode {
TwistedResourceRequestHandlerRequestParam() {
this.getParameter() = any(TwistedResourceRequestHandler handler).getRequestParameter()
}
override string getSourceType() { result = "twisted.web.server.Request" }
}
/**
* Taint propagation for `twisted.web.server.Request`.
*/
private class TwistedRequestAdditionalTaintStep extends TaintTracking::AdditionalTaintStep {
override predicate step(DataFlow::Node nodeFrom, DataFlow::Node nodeTo) {
// Methods
//
// TODO: When we have tools that make it easy, model these properly to handle
// `meth = obj.meth; meth()`. Until then, we'll use this more syntactic approach
// (since it allows us to at least capture the most common cases).
nodeFrom = Request::instance() and
exists(DataFlow::AttrRead attr | attr.getObject() = nodeFrom |
// normal (non-async) methods
attr.getAttributeName() in [
"getCookie", "getHeader", "getAllHeaders", "getUser", "getPassword", "getHost",
"getRequestHostname"
] and
nodeTo.(DataFlow::CallCfgNode).getFunction() = attr
)
or
// Attributes
nodeFrom = Request::instance() and
nodeTo.(DataFlow::AttrRead).getObject() = nodeFrom and
nodeTo.(DataFlow::AttrRead).getAttributeName() in [
"uri", "path", "prepath", "postpath", "content", "args", "received_cookies",
"requestHeaders", "user", "password", "host"
]
}
}
/**
* A parameter of a request handler method (on a `twisted.web.resource.Resource` subclass)
* that is also given remote user input. (a bit like RoutedParameter).
*/
class TwistedResourceRequestHandlerExtraSources extends RemoteFlowSource::Range,
DataFlow::ParameterNode {
TwistedResourceRequestHandlerExtraSources() {
exists(TwistedResourceRequestHandler func, int i |
func.getName() in ["getChild", "getChildWithDefault"] and i = 1
or
func.getName() = "getDynamicEntity" and i = 1
|
this.getParameter() = func.getArg(i)
)
}
override string getSourceType() { result = "twisted Resource method extra parameter" }
}
// ---------------------------------------------------------------------------
// response modeling
// ---------------------------------------------------------------------------
/**
* Implicit response from returns of render methods.
*/
private class TwistedResourceRenderMethodReturn extends HTTP::Server::HttpResponse::Range,
DataFlow::CfgNode {
TwistedResourceRenderMethodReturn() {
this.asCfgNode() = any(TwistedResourceRenderMethod meth).getAReturnValueFlowNode()
}
override DataFlow::Node getBody() { result = this }
override DataFlow::Node getMimetypeOrContentTypeArg() { none() }
override string getMimetypeDefault() { result = "text/html" }
}
/**
* A call to the `twisted.web.server.Request.write` function.
*
* See https://twistedmatrix.com/documents/21.2.0/api/twisted.web.server.Request.html#write
*/
class TwistedRequestWriteCall extends HTTP::Server::HttpResponse::Range, DataFlow::CallCfgNode {
TwistedRequestWriteCall() {
// TODO: When we have tools that make it easy, model these properly to handle
// `meth = obj.meth; meth()`. Until then, we'll use this more syntactic approach
// (since it allows us to at least capture the most common cases).
exists(DataFlow::AttrRead read |
this.getFunction() = read and
read.getObject() = Request::instance() and
read.getAttributeName() = "write"
)
}
override DataFlow::Node getBody() {
result.asCfgNode() in [node.getArg(0), node.getArgByName("data")]
}
override DataFlow::Node getMimetypeOrContentTypeArg() { none() }
override string getMimetypeDefault() { result = "text/html" }
}
/**
* A call to the `redirect` function on a twisted request.
*
* See https://twistedmatrix.com/documents/21.2.0/api/twisted.web.http.Request.html#redirect
*/
class TwistedRequestRedirectCall extends HTTP::Server::HttpRedirectResponse::Range,
DataFlow::CallCfgNode {
TwistedRequestRedirectCall() {
// TODO: When we have tools that make it easy, model these properly to handle
// `meth = obj.meth; meth()`. Until then, we'll use this more syntactic approach
// (since it allows us to at least capture the most common cases).
exists(DataFlow::AttrRead read |
this.getFunction() = read and
read.getObject() = Request::instance() and
read.getAttributeName() = "redirect"
)
}
override DataFlow::Node getBody() { none() }
override DataFlow::Node getRedirectLocation() {
result.asCfgNode() in [node.getArg(0), node.getArgByName("url")]
}
override DataFlow::Node getMimetypeOrContentTypeArg() { none() }
override string getMimetypeDefault() { result = "text/html" }
}
}

9
python/ql/src/semmle/python/security/dataflow/WeakSensitiveDataHashing.qll
View File

@@ -13,6 +13,7 @@ private import semmle.python.dataflow.new.TaintTracking
private import semmle.python.Concepts
private import semmle.python.dataflow.new.RemoteFlowSources
private import semmle.python.dataflow.new.BarrierGuards
private import semmle.python.dataflow.new.SensitiveDataSources
/**
* Provides a taint-tracking configuration for detecting use of a broken or weak
@@ -38,6 +39,10 @@ module NormalHashFunction {
or
node instanceof Sanitizer
}
override predicate isAdditionalTaintStep(DataFlow::Node node1, DataFlow::Node node2) {
sensitiveDataExtraStepForCalls(node1, node2)
}
}
}
@@ -70,5 +75,9 @@ module ComputationallyExpensiveHashFunction {
or
node instanceof Sanitizer
}
override predicate isAdditionalTaintStep(DataFlow::Node node1, DataFlow::Node node2) {
sensitiveDataExtraStepForCalls(node1, node2)
}
}
}

12
python/ql/src/semmle/python/security/dataflow/WeakSensitiveDataHashingCustomizations.qll
View File

@@ -52,7 +52,9 @@ module NormalHashFunction {
* A source of sensitive data, considered as a flow source.
*/
class SensitiveDataSourceAsSource extends Source, SensitiveDataSource {
override string getClassification() { result = SensitiveDataSource.super.getClassification() }
override SensitiveDataClassification getClassification() {
result = SensitiveDataSource.super.getClassification()
}
}
/** The input to a hashing operation using a weak algorithm, considered as a flow sink. */
@@ -120,12 +122,12 @@ module ComputationallyExpensiveHashFunction {
*/
class PasswordSourceAsSource extends Source, SensitiveDataSource {
PasswordSourceAsSource() {
// TODO: once https://github.com/github/codeql/pull/5739 has been merged,
// don't use hardcoded value anymore
SensitiveDataSource.super.getClassification() = "password"
SensitiveDataSource.super.getClassification() = SensitiveDataClassification::password()
}
override string getClassification() { result = SensitiveDataSource.super.getClassification() }
override SensitiveDataClassification getClassification() {
result = SensitiveDataSource.super.getClassification()
}
}
/**

4
python/ql/test/TestUtilities/InlineExpectationsTest.qll
View File

@@ -181,14 +181,14 @@ private int getEndOfColumnPosition(int start, string content) {
min(string name, int cand |
exists(TNamedColumn(name)) and
cand = content.indexOf(name + ":") and
cand > start
cand >= start
|
cand
)
or
not exists(string name |
exists(TNamedColumn(name)) and
content.indexOf(name + ":") > start
content.indexOf(name + ":") >= start
) and
result = content.length()
}

17
python/ql/test/experimental/dataflow/ApiGraphs/async_test.py Normal file
View File

@@ -0,0 +1,17 @@
import pkg # $ use=moduleImport("pkg")
async def foo():
coro = pkg.async_func() # $ use=moduleImport("pkg").getMember("async_func").getReturn()
coro # $ use=moduleImport("pkg").getMember("async_func").getReturn()
result = await coro # $ use=moduleImport("pkg").getMember("async_func").getReturn().getAwaited()
result # $ use=moduleImport("pkg").getMember("async_func").getReturn().getAwaited()
return result # $ use=moduleImport("pkg").getMember("async_func").getReturn().getAwaited()
async def bar():
result = await pkg.async_func() # $ use=moduleImport("pkg").getMember("async_func").getReturn().getAwaited()
return result # $ use=moduleImport("pkg").getMember("async_func").getReturn().getAwaited()
def check_annotations():
# Just to make sure how annotations should look like :)
result = pkg.sync_func() # $ use=moduleImport("pkg").getMember("sync_func").getReturn()
return result # $ use=moduleImport("pkg").getMember("sync_func").getReturn()

2
python/ql/test/experimental/dataflow/ApiGraphs/options
View File

@@ -1 +1 @@
semmle-extractor-options: --lang=3
semmle-extractor-options: --lang=3 --max-import-depth=1

3
python/ql/test/experimental/dataflow/ApiGraphs/use.ql
View File

@@ -13,7 +13,8 @@ class ApiUseTest extends InlineExpectationsTest {
l = n.getLocation() and
// Module variable nodes have no suitable location, so it's best to simply exclude them entirely
// from the inline tests.
not n instanceof DataFlow::ModuleVariableNode
not n instanceof DataFlow::ModuleVariableNode and
exists(l.getFile().getRelativePath())
}
override predicate hasActualResult(Location location, string element, string tag, string value) {

8
python/ql/test/experimental/dataflow/method-calls/test.expected Normal file
View File

@@ -0,0 +1,8 @@
conjunctive_lookup
| test.py:6:1:6:6 | ControlFlowNode for meth() | meth() | obj1 | bar |
| test.py:6:1:6:6 | ControlFlowNode for meth() | meth() | obj1 | foo |
| test.py:6:1:6:6 | ControlFlowNode for meth() | meth() | obj2 | bar |
| test.py:6:1:6:6 | ControlFlowNode for meth() | meth() | obj2 | foo |
calls_lookup
| test.py:6:1:6:6 | ControlFlowNode for meth() | meth() | obj1 | foo |
| test.py:6:1:6:6 | ControlFlowNode for meth() | meth() | obj2 | bar |

6
python/ql/test/experimental/dataflow/method-calls/test.py Normal file
View File

@@ -0,0 +1,6 @@
if cond:
meth = obj1.foo
else:
meth = obj2.bar
meth()

18
python/ql/test/experimental/dataflow/method-calls/test.ql Normal file
View File

@@ -0,0 +1,18 @@
import python
import semmle.python.dataflow.new.DataFlow
import experimental.dataflow.TestUtil.PrintNode
query predicate conjunctive_lookup(
DataFlow::MethodCallNode methCall, string call, string object, string methodName
) {
call = prettyNode(methCall) and
object = prettyNode(methCall.getObject()) and
methodName = methCall.getMethodName()
}
query predicate calls_lookup(
DataFlow::MethodCallNode methCall, string call, string object, string methodName
) {
call = prettyNode(methCall) and
exists(DataFlow::Node o | methCall.calls(o, methodName) and object = prettyNode(o))
}

33
python/ql/test/experimental/dataflow/sensitive-data/TestSensitiveDataSources.ql
View File

@@ -1,12 +1,17 @@
// /**
// * @kind path-problem
// */
import python
import semmle.python.dataflow.new.DataFlow
import semmle.python.dataflow.new.TaintTracking
import TestUtilities.InlineExpectationsTest
import semmle.python.dataflow.new.SensitiveDataSources
private import semmle.python.ApiGraphs
class SensitiveDataSourcesTest extends InlineExpectationsTest {
SensitiveDataSourcesTest() { this = "SensitiveDataSourcesTest" }
override string getARelevantTag() { result = "SensitiveDataSource" }
override string getARelevantTag() { result in ["SensitiveDataSource", "SensitiveUse"] }
override predicate hasActualResult(Location location, string element, string tag, string value) {
exists(location.getFile().getRelativePath()) and
@@ -15,6 +20,32 @@ class SensitiveDataSourcesTest extends InlineExpectationsTest {
element = source.toString() and
value = source.getClassification() and
tag = "SensitiveDataSource"
or
exists(DataFlow::Node use |
any(SensitiveUseConfiguration config).hasFlow(source, use) and
location = use.getLocation() and
element = use.toString() and
value = source.getClassification() and
tag = "SensitiveUse"
)
)
}
}
class SensitiveUseConfiguration extends TaintTracking::Configuration {
SensitiveUseConfiguration() { this = "SensitiveUseConfiguration" }
override predicate isSource(DataFlow::Node node) { node instanceof SensitiveDataSource }
override predicate isSink(DataFlow::Node node) {
node = API::builtin("print").getACall().getArg(_)
}
override predicate isAdditionalTaintStep(DataFlow::Node node1, DataFlow::Node node2) {
sensitiveDataExtraStepForCalls(node1, node2)
}
}
// import DataFlow::PathGraph
// from SensitiveUseConfiguration cfg, DataFlow::PathNode source, DataFlow::PathNode sink
// where cfg.hasFlowPath(source, sink)
// select sink, source, sink, "taint from $@", source.getNode(), "here"

51
python/ql/test/experimental/dataflow/sensitive-data/test.py
View File

@@ -1,5 +1,6 @@
from not_found import get_passwd, account_id
from not_found import get_passwd # $ SensitiveDataSource=password
from not_found import account_id # $ SensitiveDataSource=id
def get_password():
pass
@@ -20,14 +21,60 @@ fetch_certificate() # $ SensitiveDataSource=certificate
account_id() # $ SensitiveDataSource=id
safe_to_store = encrypt_password(pwd)
f = get_password
f() # $ SensitiveDataSource=password
# more tests of functions we don't have definition for
x = unkown_func_not_even_imported_get_password() # $ SensitiveDataSource=password
print(x) # $ SensitiveUse=password
f = get_passwd
x = f()
print(x) # $ SensitiveUse=password
import not_found
f = not_found.get_passwd # $ SensitiveDataSource=password
x = f()
print(x) # $ SensitiveUse=password
def my_func(non_sensitive_name):
x = non_sensitive_name()
print(x) # $ SensitiveUse=password
f = not_found.get_passwd # $ SensitiveDataSource=password
my_func(f)
# attributes
foo = ObjectFromDatabase()
foo.secret # $ SensitiveDataSource=secret
foo.username # $ SensitiveDataSource=id
getattr(foo, "password") # $ SensitiveDataSource=password
x = "password"
getattr(foo, x) # $ SensitiveDataSource=password
# based on variable/parameter names
def my_func(password): # $ SensitiveDataSource=password
print(password) # $ SensitiveUse=password
password = some_function() # $ SensitiveDataSource=password
print(password) # $ SensitiveUse=password
for password in some_function2(): # $ SensitiveDataSource=password
print(password) # $ SensitiveUse=password
with some_function3() as password: # $ SensitiveDataSource=password
print(password) # $ SensitiveUse=password
# Special handling of lookups of sensitive properties
request.args["password"], # $ MISSING: SensitiveDataSource=password
request.args["password"], # $ SensitiveDataSource=password
request.args.get("password") # $ SensitiveDataSource=password
x = "password"
request.args.get(x) # $ SensitiveDataSource=password
# I don't think handling `getlist` is super important, just included it to show what we don't handle
request.args.getlist("password")[0] # $ MISSING: SensitiveDataSource=password
from not_found import password2 as foo # $ SensitiveDataSource=password
print(foo) # $ SensitiveUse=password

5
python/ql/test/experimental/semmle/python/frameworks/clickhouse-driver/ClickHouseDriver.expected
View File

@@ -1,5 +0,0 @@
| ClickHouseDriver.py:15:22:15:106 | ControlFlowNode for Attribute() | ClickHouseDriver.py:15:52:15:105 | ControlFlowNode for BinaryExpr |
| ClickHouseDriver.py:18:5:18:87 | ControlFlowNode for Attribute() | ClickHouseDriver.py:18:33:18:86 | ControlFlowNode for BinaryExpr |
| ClickHouseDriver.py:22:5:22:62 | ControlFlowNode for Attribute() | ClickHouseDriver.py:22:33:22:37 | ControlFlowNode for query |
| ClickHouseDriver.py:27:5:27:74 | ControlFlowNode for Attribute() | ClickHouseDriver.py:27:20:27:73 | ControlFlowNode for BinaryExpr |
| ClickHouseDriver.py:30:5:30:89 | ControlFlowNode for Attribute() | ClickHouseDriver.py:30:35:30:88 | ControlFlowNode for BinaryExpr |

32
python/ql/test/experimental/semmle/python/frameworks/clickhouse-driver/ClickHouseDriver.py
View File

@@ -1,32 +0,0 @@
from django.conf.urls import url
from clickhouse_driver import Client
from clickhouse_driver import connect
from aioch import Client as aiochClient
# Dummy Client subclass
class MyClient(Client):
def dummy(self):
return None
def show_user(request, username):
# BAD -- Untrusted user input is directly injected into the sql query using async library 'aioch'
aclient = aiochClient("localhost")
progress = await aclient.execute_with_progress("SELECT * FROM users WHERE username = '%s'" % username)
# BAD -- Untrusted user input is directly injected into the sql query using native client of library 'clickhouse_driver'
Client('localhost').execute("SELECT * FROM users WHERE username = '%s'" % username)
# GOOD -- query uses prepared statements
query = "SELECT * FROM users WHERE username = %(username)s"
Client('localhost').execute(query, {"username": username})
# BAD -- Untrusted user input is directly injected into the sql query using PEP249 interface
conn = connect('clickhouse://localhost')
cursor = conn.cursor()
cursor.execute("SELECT * FROM users WHERE username = '%s'" % username)
# BAD -- Untrusted user input is directly injected into the sql query using MyClient, which is a subclass of Client
MyClient('localhost').execute("SELECT * FROM users WHERE username = '%s'" % username)
urlpatterns = [url(r'^users/(?P<username>[^/]+)$', show_user)]

6
python/ql/test/experimental/semmle/python/frameworks/clickhouse-driver/ClickHouseDriver.ql
View File

@@ -1,6 +0,0 @@
import python
import experimental.semmle.python.frameworks.ClickHouseDriver
import semmle.python.Concepts
from SqlExecution s
select s, s.getSql()

0
python/ql/test/library-tests/frameworks/aioch/ConceptsTest.expected Normal file
View File

2
python/ql/test/library-tests/frameworks/aioch/ConceptsTest.ql Normal file
View File

@@ -0,0 +1,2 @@
import python
import experimental.meta.ConceptsTest

1
python/ql/test/library-tests/frameworks/aioch/options Normal file
View File

@@ -0,0 +1 @@
semmle-extractor-options: --max-import-depth=1 --lang=3

30
python/ql/test/library-tests/frameworks/aioch/sql_test.py Normal file
View File

@@ -0,0 +1,30 @@
import aioch
SQL = "SOME SQL"
async def aioch_test():
client = aioch.Client("localhost")
await client.execute(SQL) # $ getSql=SQL
await client.execute(query=SQL) # $ getSql=SQL
await client.execute_with_progress(SQL) # $ getSql=SQL
await client.execute_with_progress(query=SQL) # $ getSql=SQL
await client.execute_iter(SQL) # $ getSql=SQL
await client.execute_iter(query=SQL) # $ getSql=SQL
# Using custom client (this has been seen done for the blocking version in
# `clickhouse_driver` PyPI package)
class MyClient(aioch.Client):
pass
async def test_custom_client():
client = MyClient("localhost")
await client.execute(SQL) # $ getSql=SQL

0
python/ql/test/library-tests/frameworks/clickhouse_driver/ConceptsTest.expected Normal file
View File

2
python/ql/test/library-tests/frameworks/clickhouse_driver/ConceptsTest.ql Normal file
View File

@@ -0,0 +1,2 @@
import python
import experimental.meta.ConceptsTest

42
python/ql/test/library-tests/frameworks/clickhouse_driver/sql_test.py Normal file
View File

@@ -0,0 +1,42 @@
import clickhouse_driver
SQL = "SOME SQL"
# Normal operation
client = clickhouse_driver.client.Client("localhost")
client.execute(SQL) # $ getSql=SQL
client.execute(query=SQL) # $ getSql=SQL
client.execute_with_progress(SQL) # $ getSql=SQL
client.execute_with_progress(query=SQL) # $ getSql=SQL
client.execute_iter(SQL) # $ getSql=SQL
client.execute_iter(query=SQL) # $ getSql=SQL
# commonly used alias
client = clickhouse_driver.Client("localhost")
client.execute(SQL) # $ getSql=SQL
# Using PEP249 interface
conn = clickhouse_driver.connect('clickhouse://localhost')
cursor = conn.cursor()
cursor.execute(SQL) # $ getSql=SQL
# Using custom client
#
# examples from real world code
# https://github.com/Altinity/clickhouse-mysql-data-reader/blob/3b1b7088751b05e5bbf45890c5949b58208c2343/clickhouse_mysql/dbclient/chclient.py#L10
# https://github.com/Felixoid/clickhouse-plantuml/blob/d8b2ba7d164a836770ec21f5e4035dfb04c41d9c/clickhouse_plantuml/client.py#L9
class MyClient(clickhouse_driver.Client):
pass
MyClient("localhost").execute(SQL) # $ getSql=SQL

0
python/ql/test/library-tests/frameworks/twisted/ConceptsTest.expected Normal file
View File

12
python/ql/test/library-tests/frameworks/twisted/ConceptsTest.ql Normal file
View File

@@ -0,0 +1,12 @@
import python
import experimental.meta.ConceptsTest
class DedicatedResponseTest extends HttpServerHttpResponseTest {
DedicatedResponseTest() { file.getShortName() = "response_test.py" }
}
class OtherResponseTest extends HttpServerHttpResponseTest {
OtherResponseTest() { not this instanceof DedicatedResponseTest }
override string getARelevantTag() { result = "HttpResponse" }
}

3
python/ql/test/library-tests/frameworks/twisted/InlineTaintTest.expected Normal file
View File

@@ -0,0 +1,3 @@
argumentToEnsureNotTaintedNotMarkedAsSpurious
untaintedArgumentToEnsureTaintedNotMarkedAsMissing
failures

1
python/ql/test/library-tests/frameworks/twisted/InlineTaintTest.ql Normal file
View File

@@ -0,0 +1 @@
import experimental.meta.InlineTaintTest

75
python/ql/test/library-tests/frameworks/twisted/response_test.py Normal file
View File

@@ -0,0 +1,75 @@
from twisted.web.server import Site, Request, NOT_DONE_YET
from twisted.web.resource import Resource
from twisted.internet import reactor, endpoints, defer
root = Resource()
class Now(Resource):
def render(self, request: Request): # $ requestHandler
return b"now" # $ HttpResponse mimetype=text/html responseBody=b"now"
class AlsoNow(Resource):
def render(self, request: Request): # $ requestHandler
request.write(b"also now") # $ HttpResponse mimetype=text/html responseBody=b"also now"
return b"" # $ HttpResponse mimetype=text/html responseBody=b""
def process_later(request: Request):
print("process_later called")
request.write(b"later") # $ MISSING: responseBody=b"later"
request.finish()
class Later(Resource):
def render(self, request: Request): # $ requestHandler
# process the request in 1 second
print("setting up callback for process_later")
reactor.callLater(1, process_later, request)
return NOT_DONE_YET # $ SPURIOUS: HttpResponse mimetype=text/html responseBody=NOT_DONE_YET
class PlainText(Resource):
def render(self, request: Request): # $ requestHandler
request.setHeader(b"content-type", "text/plain")
return b"this is plain text" # $ HttpResponse responseBody=b"this is plain text" SPURIOUS: mimetype=text/html MISSING: mimetype=text/plain
class Redirect(Resource):
def render_GET(self, request: Request): # $ requestHandler
request.redirect("/new-location") # $ HttpRedirectResponse redirectLocation="/new-location" HttpResponse mimetype=text/html
# By default, this `hello` output is not returned... not even when
# requested with curl.
return b"hello" # $ SPURIOUS: HttpResponse mimetype=text/html responseBody=b"hello"
class NonHttpBodyOutput(Resource):
"""Examples of providing values in response that is not in the body
"""
def render_GET(self, request: Request): # $ requestHandler
request.responseHeaders.addRawHeader("key", "value")
request.setHeader("key2", "value")
request.addCookie("key", "value")
request.cookies.append(b"key2=value")
return b"" # $ HttpResponse mimetype=text/html responseBody=b""
root.putChild(b"now", Now())
root.putChild(b"also-now", AlsoNow())
root.putChild(b"later", Later())
root.putChild(b"plain-text", PlainText())
root.putChild(b"redirect", Redirect())
root.putChild(b"non-body", NonHttpBodyOutput())
if __name__ == "__main__":
factory = Site(root)
endpoint = endpoints.TCP4ServerEndpoint(reactor, 8880)
endpoint.listen(factory)
print("Will run on http://localhost:8880")
reactor.run()

47
python/ql/test/library-tests/frameworks/twisted/routing_test.py Normal file
View File

@@ -0,0 +1,47 @@
from twisted.web.server import Site, Request
from twisted.web.resource import Resource
from twisted.internet import reactor, endpoints
root = Resource()
class Foo(Resource):
def render(self, request: Request): # $ requestHandler
print(f"{request.content=}")
print(f"{request.cookies=}")
print(f"{request.received_cookies=}")
return b"I am Foo" # $ HttpResponse
root.putChild(b"foo", Foo())
class Child(Resource):
def __init__(self, name):
self.name = name.decode("utf-8")
def render_GET(self, request): # $ requestHandler
return f"Hi, I'm child '{self.name}'".encode("utf-8") # $ HttpResponse
class Parent(Resource):
def getChild(self, path, request): # $ requestHandler
print(path, type(path))
return Child(path)
def render_GET(self, request): # $ requestHandler
return b"Hi, I'm parent" # $ HttpResponse
root.putChild(b"parent", Parent())
if __name__ == "__main__":
factory = Site(root)
endpoint = endpoints.TCP4ServerEndpoint(reactor, 8880)
endpoint.listen(factory)
print("Will run on http://localhost:8880")
reactor.run()

70
python/ql/test/library-tests/frameworks/twisted/taint_test.py Normal file
View File

@@ -0,0 +1,70 @@
from twisted.web.resource import Resource
from twisted.web.server import Request
class MyTaintTest(Resource):
def getChild(self, path, request): # $ requestHandler
ensure_tainted(path, request) # $ tainted
def render(self, request): # $ requestHandler
ensure_tainted(request) # $ tainted
def render_GET(self, request: Request): # $ requestHandler
# see https://twistedmatrix.com/documents/21.2.0/api/twisted.web.server.Request.html
ensure_tainted(
request, # $ tainted
request.uri, # $ tainted
request.path, # $ tainted
request.prepath, # $ tainted
request.postpath, # $ tainted
# file-like
request.content, # $ tainted
request.content.read(), # $ MISSING: tainted
# Dict[bytes, List[bytes]] (for query args)
request.args, # $ tainted
request.args[b"key"], # $ tainted
request.args[b"key"][0], # $ tainted
request.args.get(b"key"), # $ tainted
request.args.get(b"key")[0], # $ tainted
request.received_cookies, # $ tainted
request.received_cookies["key"], # $ tainted
request.received_cookies.get("key"), # $ tainted
request.getCookie(b"key"), # $ tainted
# twisted.web.http_headers.Headers
# see https://twistedmatrix.com/documents/21.2.0/api/twisted.web.http_headers.Headers.html
request.requestHeaders, # $ tainted
request.requestHeaders.getRawHeaders("key"), # $ MISSING: tainted
request.requestHeaders.getRawHeaders("key")[0], # $ MISSING: tainted
request.requestHeaders.getAllRawHeaders(), # $ MISSING: tainted
list(request.requestHeaders.getAllRawHeaders()), # $ MISSING: tainted
request.getHeader("key"), # $ tainted
request.getAllHeaders(), # $ tainted
request.getAllHeaders()["key"], # $ tainted
request.user, # $ tainted
request.getUser(), # $ tainted
request.password, # $ tainted
request.getPassword(), # $ tainted
request.host, # $ tainted
request.getHost(), # $ tainted
request.getRequestHostname(), # $ tainted
)
# technically user-controlled, but unlikely to lead to vulnerabilities.
ensure_not_tainted(
request.method,
)
# not tainted at all
ensure_not_tainted(
# outgoing things
request.cookies,
request.responseHeaders,
)

44
python/ql/test/query-tests/Security/CWE-327-WeakSensitiveDataHashing/WeakSensitiveDataHashing.expected
View File

@@ -1,27 +1,71 @@
edges
| test_cryptodome.py:0:0:0:0 | ModuleVariableNode for Global Variable get_certificate in Module test_cryptodome | test_cryptodome.py:6:17:6:31 | ControlFlowNode for get_certificate |
| test_cryptodome.py:0:0:0:0 | ModuleVariableNode for Global Variable get_password in Module test_cryptodome | test_cryptodome.py:13:17:13:28 | ControlFlowNode for get_password |
| test_cryptodome.py:0:0:0:0 | ModuleVariableNode for Global Variable get_password in Module test_cryptodome | test_cryptodome.py:20:17:20:28 | ControlFlowNode for get_password |
| test_cryptodome.py:2:23:2:34 | ControlFlowNode for ImportMember | test_cryptodome.py:2:23:2:34 | GSSA Variable get_password |
| test_cryptodome.py:2:23:2:34 | GSSA Variable get_password | test_cryptodome.py:0:0:0:0 | ModuleVariableNode for Global Variable get_password in Module test_cryptodome |
| test_cryptodome.py:2:37:2:51 | ControlFlowNode for ImportMember | test_cryptodome.py:2:37:2:51 | GSSA Variable get_certificate |
| test_cryptodome.py:2:37:2:51 | GSSA Variable get_certificate | test_cryptodome.py:0:0:0:0 | ModuleVariableNode for Global Variable get_certificate in Module test_cryptodome |
| test_cryptodome.py:6:17:6:31 | ControlFlowNode for get_certificate | test_cryptodome.py:8:19:8:27 | ControlFlowNode for dangerous |
| test_cryptodome.py:6:17:6:33 | ControlFlowNode for get_certificate() | test_cryptodome.py:8:19:8:27 | ControlFlowNode for dangerous |
| test_cryptodome.py:13:17:13:28 | ControlFlowNode for get_password | test_cryptodome.py:15:19:15:27 | ControlFlowNode for dangerous |
| test_cryptodome.py:13:17:13:30 | ControlFlowNode for get_password() | test_cryptodome.py:15:19:15:27 | ControlFlowNode for dangerous |
| test_cryptodome.py:20:17:20:28 | ControlFlowNode for get_password | test_cryptodome.py:24:19:24:27 | ControlFlowNode for dangerous |
| test_cryptodome.py:20:17:20:30 | ControlFlowNode for get_password() | test_cryptodome.py:24:19:24:27 | ControlFlowNode for dangerous |
| test_cryptography.py:0:0:0:0 | ModuleVariableNode for Global Variable get_certificate in Module test_cryptography | test_cryptography.py:7:17:7:31 | ControlFlowNode for get_certificate |
| test_cryptography.py:0:0:0:0 | ModuleVariableNode for Global Variable get_password in Module test_cryptography | test_cryptography.py:15:17:15:28 | ControlFlowNode for get_password |
| test_cryptography.py:0:0:0:0 | ModuleVariableNode for Global Variable get_password in Module test_cryptography | test_cryptography.py:23:17:23:28 | ControlFlowNode for get_password |
| test_cryptography.py:3:23:3:34 | ControlFlowNode for ImportMember | test_cryptography.py:3:23:3:34 | GSSA Variable get_password |
| test_cryptography.py:3:23:3:34 | GSSA Variable get_password | test_cryptography.py:0:0:0:0 | ModuleVariableNode for Global Variable get_password in Module test_cryptography |
| test_cryptography.py:3:37:3:51 | ControlFlowNode for ImportMember | test_cryptography.py:3:37:3:51 | GSSA Variable get_certificate |
| test_cryptography.py:3:37:3:51 | GSSA Variable get_certificate | test_cryptography.py:0:0:0:0 | ModuleVariableNode for Global Variable get_certificate in Module test_cryptography |
| test_cryptography.py:7:17:7:31 | ControlFlowNode for get_certificate | test_cryptography.py:9:19:9:27 | ControlFlowNode for dangerous |
| test_cryptography.py:7:17:7:33 | ControlFlowNode for get_certificate() | test_cryptography.py:9:19:9:27 | ControlFlowNode for dangerous |
| test_cryptography.py:15:17:15:28 | ControlFlowNode for get_password | test_cryptography.py:17:19:17:27 | ControlFlowNode for dangerous |
| test_cryptography.py:15:17:15:30 | ControlFlowNode for get_password() | test_cryptography.py:17:19:17:27 | ControlFlowNode for dangerous |
| test_cryptography.py:23:17:23:28 | ControlFlowNode for get_password | test_cryptography.py:27:19:27:27 | ControlFlowNode for dangerous |
| test_cryptography.py:23:17:23:30 | ControlFlowNode for get_password() | test_cryptography.py:27:19:27:27 | ControlFlowNode for dangerous |
nodes
| test_cryptodome.py:0:0:0:0 | ModuleVariableNode for Global Variable get_certificate in Module test_cryptodome | semmle.label | ModuleVariableNode for Global Variable get_certificate in Module test_cryptodome |
| test_cryptodome.py:0:0:0:0 | ModuleVariableNode for Global Variable get_password in Module test_cryptodome | semmle.label | ModuleVariableNode for Global Variable get_password in Module test_cryptodome |
| test_cryptodome.py:2:23:2:34 | ControlFlowNode for ImportMember | semmle.label | ControlFlowNode for ImportMember |
| test_cryptodome.py:2:23:2:34 | GSSA Variable get_password | semmle.label | GSSA Variable get_password |
| test_cryptodome.py:2:37:2:51 | ControlFlowNode for ImportMember | semmle.label | ControlFlowNode for ImportMember |
| test_cryptodome.py:2:37:2:51 | GSSA Variable get_certificate | semmle.label | GSSA Variable get_certificate |
| test_cryptodome.py:6:17:6:31 | ControlFlowNode for get_certificate | semmle.label | ControlFlowNode for get_certificate |
| test_cryptodome.py:6:17:6:33 | ControlFlowNode for get_certificate() | semmle.label | ControlFlowNode for get_certificate() |
| test_cryptodome.py:8:19:8:27 | ControlFlowNode for dangerous | semmle.label | ControlFlowNode for dangerous |
| test_cryptodome.py:13:17:13:28 | ControlFlowNode for get_password | semmle.label | ControlFlowNode for get_password |
| test_cryptodome.py:13:17:13:30 | ControlFlowNode for get_password() | semmle.label | ControlFlowNode for get_password() |
| test_cryptodome.py:15:19:15:27 | ControlFlowNode for dangerous | semmle.label | ControlFlowNode for dangerous |
| test_cryptodome.py:20:17:20:28 | ControlFlowNode for get_password | semmle.label | ControlFlowNode for get_password |
| test_cryptodome.py:20:17:20:30 | ControlFlowNode for get_password() | semmle.label | ControlFlowNode for get_password() |
| test_cryptodome.py:24:19:24:27 | ControlFlowNode for dangerous | semmle.label | ControlFlowNode for dangerous |
| test_cryptography.py:0:0:0:0 | ModuleVariableNode for Global Variable get_certificate in Module test_cryptography | semmle.label | ModuleVariableNode for Global Variable get_certificate in Module test_cryptography |
| test_cryptography.py:0:0:0:0 | ModuleVariableNode for Global Variable get_password in Module test_cryptography | semmle.label | ModuleVariableNode for Global Variable get_password in Module test_cryptography |
| test_cryptography.py:3:23:3:34 | ControlFlowNode for ImportMember | semmle.label | ControlFlowNode for ImportMember |
| test_cryptography.py:3:23:3:34 | GSSA Variable get_password | semmle.label | GSSA Variable get_password |
| test_cryptography.py:3:37:3:51 | ControlFlowNode for ImportMember | semmle.label | ControlFlowNode for ImportMember |
| test_cryptography.py:3:37:3:51 | GSSA Variable get_certificate | semmle.label | GSSA Variable get_certificate |
| test_cryptography.py:7:17:7:31 | ControlFlowNode for get_certificate | semmle.label | ControlFlowNode for get_certificate |
| test_cryptography.py:7:17:7:33 | ControlFlowNode for get_certificate() | semmle.label | ControlFlowNode for get_certificate() |
| test_cryptography.py:9:19:9:27 | ControlFlowNode for dangerous | semmle.label | ControlFlowNode for dangerous |
| test_cryptography.py:15:17:15:28 | ControlFlowNode for get_password | semmle.label | ControlFlowNode for get_password |
| test_cryptography.py:15:17:15:30 | ControlFlowNode for get_password() | semmle.label | ControlFlowNode for get_password() |
| test_cryptography.py:17:19:17:27 | ControlFlowNode for dangerous | semmle.label | ControlFlowNode for dangerous |
| test_cryptography.py:23:17:23:28 | ControlFlowNode for get_password | semmle.label | ControlFlowNode for get_password |
| test_cryptography.py:23:17:23:30 | ControlFlowNode for get_password() | semmle.label | ControlFlowNode for get_password() |
| test_cryptography.py:27:19:27:27 | ControlFlowNode for dangerous | semmle.label | ControlFlowNode for dangerous |
#select
| test_cryptodome.py:8:19:8:27 | ControlFlowNode for dangerous | test_cryptodome.py:2:37:2:51 | ControlFlowNode for ImportMember | test_cryptodome.py:8:19:8:27 | ControlFlowNode for dangerous | $@ is used in a hashing algorithm (MD5) that is insecure. | test_cryptodome.py:2:37:2:51 | ControlFlowNode for ImportMember | Sensitive data (certificate) |
| test_cryptodome.py:8:19:8:27 | ControlFlowNode for dangerous | test_cryptodome.py:6:17:6:33 | ControlFlowNode for get_certificate() | test_cryptodome.py:8:19:8:27 | ControlFlowNode for dangerous | $@ is used in a hashing algorithm (MD5) that is insecure. | test_cryptodome.py:6:17:6:33 | ControlFlowNode for get_certificate() | Sensitive data (certificate) |
| test_cryptodome.py:15:19:15:27 | ControlFlowNode for dangerous | test_cryptodome.py:2:23:2:34 | ControlFlowNode for ImportMember | test_cryptodome.py:15:19:15:27 | ControlFlowNode for dangerous | $@ is used in a hashing algorithm (MD5) that is insecure for password hashing, since it is not a computationally expensive hash function. | test_cryptodome.py:2:23:2:34 | ControlFlowNode for ImportMember | Sensitive data (password) |
| test_cryptodome.py:15:19:15:27 | ControlFlowNode for dangerous | test_cryptodome.py:13:17:13:30 | ControlFlowNode for get_password() | test_cryptodome.py:15:19:15:27 | ControlFlowNode for dangerous | $@ is used in a hashing algorithm (MD5) that is insecure for password hashing, since it is not a computationally expensive hash function. | test_cryptodome.py:13:17:13:30 | ControlFlowNode for get_password() | Sensitive data (password) |
| test_cryptodome.py:24:19:24:27 | ControlFlowNode for dangerous | test_cryptodome.py:2:23:2:34 | ControlFlowNode for ImportMember | test_cryptodome.py:24:19:24:27 | ControlFlowNode for dangerous | $@ is used in a hashing algorithm (SHA256) that is insecure for password hashing, since it is not a computationally expensive hash function. | test_cryptodome.py:2:23:2:34 | ControlFlowNode for ImportMember | Sensitive data (password) |
| test_cryptodome.py:24:19:24:27 | ControlFlowNode for dangerous | test_cryptodome.py:20:17:20:30 | ControlFlowNode for get_password() | test_cryptodome.py:24:19:24:27 | ControlFlowNode for dangerous | $@ is used in a hashing algorithm (SHA256) that is insecure for password hashing, since it is not a computationally expensive hash function. | test_cryptodome.py:20:17:20:30 | ControlFlowNode for get_password() | Sensitive data (password) |
| test_cryptography.py:9:19:9:27 | ControlFlowNode for dangerous | test_cryptography.py:3:37:3:51 | ControlFlowNode for ImportMember | test_cryptography.py:9:19:9:27 | ControlFlowNode for dangerous | $@ is used in a hashing algorithm (MD5) that is insecure. | test_cryptography.py:3:37:3:51 | ControlFlowNode for ImportMember | Sensitive data (certificate) |
| test_cryptography.py:9:19:9:27 | ControlFlowNode for dangerous | test_cryptography.py:7:17:7:33 | ControlFlowNode for get_certificate() | test_cryptography.py:9:19:9:27 | ControlFlowNode for dangerous | $@ is used in a hashing algorithm (MD5) that is insecure. | test_cryptography.py:7:17:7:33 | ControlFlowNode for get_certificate() | Sensitive data (certificate) |
| test_cryptography.py:17:19:17:27 | ControlFlowNode for dangerous | test_cryptography.py:3:23:3:34 | ControlFlowNode for ImportMember | test_cryptography.py:17:19:17:27 | ControlFlowNode for dangerous | $@ is used in a hashing algorithm (MD5) that is insecure for password hashing, since it is not a computationally expensive hash function. | test_cryptography.py:3:23:3:34 | ControlFlowNode for ImportMember | Sensitive data (password) |
| test_cryptography.py:17:19:17:27 | ControlFlowNode for dangerous | test_cryptography.py:15:17:15:30 | ControlFlowNode for get_password() | test_cryptography.py:17:19:17:27 | ControlFlowNode for dangerous | $@ is used in a hashing algorithm (MD5) that is insecure for password hashing, since it is not a computationally expensive hash function. | test_cryptography.py:15:17:15:30 | ControlFlowNode for get_password() | Sensitive data (password) |
| test_cryptography.py:27:19:27:27 | ControlFlowNode for dangerous | test_cryptography.py:3:23:3:34 | ControlFlowNode for ImportMember | test_cryptography.py:27:19:27:27 | ControlFlowNode for dangerous | $@ is used in a hashing algorithm (SHA256) that is insecure for password hashing, since it is not a computationally expensive hash function. | test_cryptography.py:3:23:3:34 | ControlFlowNode for ImportMember | Sensitive data (password) |
| test_cryptography.py:27:19:27:27 | ControlFlowNode for dangerous | test_cryptography.py:23:17:23:30 | ControlFlowNode for get_password() | test_cryptography.py:27:19:27:27 | ControlFlowNode for dangerous | $@ is used in a hashing algorithm (SHA256) that is insecure for password hashing, since it is not a computationally expensive hash function. | test_cryptography.py:23:17:23:30 | ControlFlowNode for get_password() | Sensitive data (password) |