hohn/codeql
1
0
Fork 0
mirror of https://github.com/github/codeql.git synced 2026-04-30 19:26:02 +02:00
Code Issues Packages Projects Releases Wiki Activity

resolve merge conflict

Browse Source
This commit is contained in:
jorgectf
2021-06-15 01:22:11 +02:00
parent ae806cd445 cce8eac0a7
commit 1662c5d113
1305 changed files with 44325 additions and 13607 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
python/change-notes/2021-04-09-split-weak-crypto-query.md Normal file
View File

@@ -0,0 +1,2 @@
lgtm,codescanning
* Updated the _Use of a broken or weak cryptographic algorithm_ (`py/weak-cryptographic-algorithm`) query, so it alerts on any use of a weak cryptographic non-hashing algorithm. Introduced a new query _Use of a broken or weak cryptographic hashing algorithm on sensitive data_ (`py/weak-sensitive-data-hashing`) to handle weak cryptographic hashing algorithms, which only alerts when used on sensitive data.

2
python/change-notes/2021-05-10-idna-add-modeling.md Normal file
View File

@@ -0,0 +1,2 @@
lgtm,codescanning
* Added modeling of the PyPI package `idna`, for encoding/decoding Internationalised Domain Names in Applications.

2
python/change-notes/2021-05-10-simplejson-add-modeling.md Normal file
View File

@@ -0,0 +1,2 @@
lgtm,codescanning
* Added modeling of the PyPI package `simplejson`.

2
python/change-notes/2021-05-10-ujson-add-modeling.md Normal file
View File

@@ -0,0 +1,2 @@
lgtm,codescanning
* Added modeling of the PyPI package `ujson`.

2
python/change-notes/2021-06-03-aiohttp-webserver-modeling.md Normal file
View File

@@ -0,0 +1,2 @@
lgtm,codescanning
* Added modeling of sources/sinks when using the `aiohttp.web` web framework.

13
python/ql/src/Expressions/UseofInput.ql
View File

@@ -4,18 +4,21 @@
* @kind problem
* @tags security
* correctness
* security/cwe/cwe-78
* @problem.severity error
* @security-severity 5.9
* @sub-severity high
* @precision high
* @id py/use-of-input
*/
import python
import semmle.python.dataflow.new.DataFlow
import semmle.python.ApiGraphs
from CallNode call, Context context, ControlFlowNode func
from DataFlow::CallCfgNode call
where
context.getAVersion().includes(2, _) and
call.getFunction() = func and
func.pointsTo(context, Value::named("input"), _) and
not func.pointsTo(context, Value::named("raw_input"), _)
major_version() = 2 and
call = API::builtin("input").getACall() and
call != API::builtin("raw_input").getACall()
select call, "The unsafe built-in function 'input' is used in Python 2."

2
python/ql/src/Functions/ReturnValueIgnored.qhelp
View File

@@ -9,7 +9,7 @@ information being thrown away.</p>
<p>A return value is considered to be trivial if it is <code>None</code> or it is a parameter (parameters, usually <code>self</code> are often
returned to assist with method chaining, but can be ignored).
A return value is also assumed to be trivial if it is ignored for 75% or more of calls.
A return value is also assumed to be trivial if it is ignored for more than 25% of calls.
</p>
</overview>

2
python/ql/src/Security/CVE-2018-1281/BindToAllInterfaces.ql
View File

@@ -4,7 +4,9 @@
* and is therefore associated with security risks.
* @kind problem
* @tags security
* external/cwe/cwe-200
* @problem.severity error
* @security-severity 3.6
* @sub-severity low
* @precision high
* @id py/bind-socket-all-network-interfaces

1
python/ql/src/Security/CWE-020-ExternalAPIs/UntrustedDataToExternalAPI.ql
View File

@@ -5,6 +5,7 @@
* @kind path-problem
* @precision low
* @problem.severity error
* @security-severity 5.9
* @tags security external/cwe/cwe-20
*/

1
python/ql/src/Security/CWE-020/IncompleteHostnameRegExp.ql
View File

@@ -3,6 +3,7 @@
* @description Matching a URL or hostname against a regular expression that contains an unescaped dot as part of the hostname might match more hostnames than expected.
* @kind problem
* @problem.severity warning
* @security-severity 5.9
* @precision high
* @id py/incomplete-hostname-regexp
* @tags correctness

1
python/ql/src/Security/CWE-020/IncompleteUrlSubstringSanitization.ql
View File

@@ -3,6 +3,7 @@
* @description Security checks on the substrings of an unparsed URL are often vulnerable to bypassing.
* @kind problem
* @problem.severity warning
* @security-severity 5.9
* @precision high
* @id py/incomplete-url-substring-sanitization
* @tags correctness

1
python/ql/src/Security/CWE-022/PathInjection.ql
View File

@@ -3,6 +3,7 @@
* @description Accessing paths influenced by users can allow an attacker to access unexpected resources.
* @kind path-problem
* @problem.severity error
* @security-severity 6.4
* @sub-severity high
* @precision high
* @id py/path-injection

1
python/ql/src/Security/CWE-022/TarSlip.ql
View File

@@ -6,6 +6,7 @@
* @kind path-problem
* @id py/tarslip
* @problem.severity error
* @security-severity 6.4
* @precision medium
* @tags security
* external/cwe/cwe-022

1
python/ql/src/Security/CWE-078/CommandInjection.ql
View File

@@ -4,6 +4,7 @@
* user to change the meaning of the command.
* @kind path-problem
* @problem.severity error
* @security-severity 5.9
* @sub-severity high
* @precision high
* @id py/command-line-injection

1
python/ql/src/Security/CWE-079/Jinja2WithoutEscaping.ql
View File

@@ -4,6 +4,7 @@
* cause a cross-site scripting vulnerability.
* @kind problem
* @problem.severity error
* @security-severity 2.9
* @precision medium
* @id py/jinja2/autoescape-false
* @tags security

1
python/ql/src/Security/CWE-079/ReflectedXss.ql
View File

@@ -4,6 +4,7 @@
* allows for a cross-site scripting vulnerability.
* @kind path-problem
* @problem.severity error
* @security-severity 2.9
* @sub-severity high
* @precision high
* @id py/reflective-xss

1
python/ql/src/Security/CWE-089/SqlInjection.ql
View File

@@ -4,6 +4,7 @@
* malicious SQL code by the user.
* @kind path-problem
* @problem.severity error
* @security-severity 6.4
* @precision high
* @id py/sql-injection
* @tags security

1
python/ql/src/Security/CWE-094/CodeInjection.ql
View File

@@ -4,6 +4,7 @@
* code execution.
* @kind path-problem
* @problem.severity error
* @security-severity 10.0
* @sub-severity high
* @precision high
* @id py/code-injection

1
python/ql/src/Security/CWE-209/StackTraceExposure.ql
View File

@@ -5,6 +5,7 @@
* developing a subsequent exploit.
* @kind path-problem
* @problem.severity error
* @security-severity 3.6
* @precision high
* @id py/stack-trace-exposure
* @tags security

1
python/ql/src/Security/CWE-215/FlaskDebug.ql
View File

@@ -3,6 +3,7 @@
* @description Running a Flask app in debug mode may allow an attacker to run arbitrary code through the Werkzeug debugger.
* @kind problem
* @problem.severity error
* @security-severity 6.4
* @precision high
* @id py/flask-debug
* @tags security

1
python/ql/src/Security/CWE-295/MissingHostKeyValidation.ql
View File

@@ -3,6 +3,7 @@
* @description Accepting unknown host keys can allow man-in-the-middle attacks.
* @kind problem
* @problem.severity error
* @security-severity 5.2
* @precision high
* @id py/paramiko-missing-host-key-validation
* @tags security

1
python/ql/src/Security/CWE-295/RequestWithoutValidation.ql
View File

@@ -3,6 +3,7 @@
* @description Making a request without certificate validation can allow man-in-the-middle attacks.
* @kind problem
* @problem.severity error
* @security-severity 5.2
* @precision medium
* @id py/request-without-cert-validation
* @tags security

1
python/ql/src/Security/CWE-312/CleartextLogging.ql
View File

@@ -4,6 +4,7 @@
* expose it to an attacker.
* @kind path-problem
* @problem.severity error
* @security-severity 5.9
* @precision high
* @id py/clear-text-logging-sensitive-data
* @tags security

1
python/ql/src/Security/CWE-312/CleartextStorage.ql
View File

@@ -4,6 +4,7 @@
* attacker.
* @kind path-problem
* @problem.severity error
* @security-severity 5.9
* @precision high
* @id py/clear-text-storage-sensitive-data
* @tags security

1
python/ql/src/Security/CWE-326/WeakCryptoKey.ql
View File

@@ -3,6 +3,7 @@
* @description Use of a cryptographic key that is too small may allow the encryption to be broken.
* @kind problem
* @problem.severity error
* @security-severity 5.2
* @precision high
* @id py/weak-crypto-key
* @tags security

22
python/ql/src/Security/CWE-327/BrokenCryptoAlgorithm.qhelp
View File

@@ -15,22 +15,28 @@
secure than it appears to be.
</p>
<p>
This query alerts on any use of a weak cryptographic algorithm, that is
not a hashing algorithm. Use of broken or weak cryptographic hash
functions are handled by the
<code>py/weak-sensitive-data-hashing</code> query.
</p>
</overview>
<recommendation>
<p>
Ensure that you use a strong, modern cryptographic
algorithm. Use at least AES-128 or RSA-2048 for
encryption, and SHA-2 or SHA-3 for secure hashing.
algorithm, such as AES-128 or RSA-2048.
</p>
</recommendation>
<example>
<p>
The following code uses the <code>pycrypto</code>
The following code uses the <code>pycryptodome</code>
library to encrypt some secret data. When you create a cipher using
<code>pycrypto</code> you must specify the encryption
<code>pycryptodome</code> you must specify the encryption
algorithm to use. The first example uses DES, which is an
older algorithm that is now considered weak. The second
example uses AES, which is a stronger modern algorithm.
@@ -39,8 +45,12 @@
<sample src="examples/broken_crypto.py" />
<p>
WARNING: Although the second example above is more robust,
pycrypto is no longer actively maintained so we recommend using <code>cryptography</code> instead.
NOTICE: the original
<code><a href="https://pypi.org/project/pycrypto/">pycrypto</a></code>
PyPI package that provided the <code>Crypto</code> module is not longer
actively maintained, so you should use the
<code><a href="https://pypi.org/project/pycryptodome/">pycryptodome</a></code>
PyPI package instead (which has a compatible API).
</p>
</example>

31
python/ql/src/Security/CWE-327/BrokenCryptoAlgorithm.ql
View File

@@ -1,8 +1,9 @@
/**
* @name Use of a broken or weak cryptographic algorithm
* @description Using broken or weak cryptographic algorithms can compromise security.
* @kind path-problem
* @kind problem
* @problem.severity warning
* @security-severity 5.2
* @precision high
* @id py/weak-cryptographic-algorithm
* @tags security
@@ -10,21 +11,15 @@
*/
import python
import semmle.python.security.Paths
import semmle.python.security.SensitiveData
import semmle.python.security.Crypto
import semmle.python.Concepts
class BrokenCryptoConfiguration extends TaintTracking::Configuration {
BrokenCryptoConfiguration() { this = "Broken crypto configuration" }
override predicate isSource(TaintTracking::Source source) {
source instanceof SensitiveDataSource
}
override predicate isSink(TaintTracking::Sink sink) { sink instanceof WeakCryptoSink }
}
from BrokenCryptoConfiguration config, TaintedPathSource src, TaintedPathSink sink
where config.hasFlowPath(src, sink)
select sink.getSink(), src, sink, "$@ is used in a broken or weak cryptographic algorithm.",
src.getSource(), "Sensitive data"
from Cryptography::CryptographicOperation operation, Cryptography::CryptographicAlgorithm algorithm
where
algorithm = operation.getAlgorithm() and
algorithm.isWeak() and
// `Cryptography::HashingAlgorithm` and `Cryptography::PasswordHashingAlgorithm` are
// handled by `py/weak-sensitive-data-hashing`
algorithm instanceof Cryptography::EncryptionAlgorithm
select operation,
"The cryptographic algorithm " + algorithm.getName() +
" is broken or weak, and should not be used."

1
python/ql/src/Security/CWE-327/InsecureDefaultProtocol.ql
View File

@@ -5,6 +5,7 @@
* @id py/insecure-default-protocol
* @kind problem
* @problem.severity warning
* @security-severity 5.2
* @precision high
* @tags security
* external/cwe/cwe-327

1
python/ql/src/Security/CWE-327/InsecureProtocol.ql
View File

@@ -4,6 +4,7 @@
* @id py/insecure-protocol
* @kind problem
* @problem.severity warning
* @security-severity 5.2
* @precision high
* @tags security
* external/cwe/cwe-327

104
python/ql/src/Security/CWE-327/WeakSensitiveDataHashing.qhelp Normal file
View File

@@ -0,0 +1,104 @@
<!DOCTYPE qhelp PUBLIC
"-//Semmle//qhelp//EN"
"qhelp.dtd">
<qhelp>
<overview>
<p>
Using a broken or weak cryptographic hash function can leave data
vulnerable, and should not be used in security related code.
</p>
<p>
A strong cryptographic hash function should be resistant to:
</p>
<ul>
<li>
pre-image attacks: if you know a hash value <code>h(x)</code>,
you should not be able to easily find the input <code>x</code>.
</li>
<li>
collision attacks: if you know a hash value <code>h(x)</code>,
you should not be able to easily find a different input <code>y</code>
with the same hash value <code>h(x) = h(y)</code>.
</li>
</ul>
<p>
In cases with a limited input space, such as for passwords, the hash
function also needs to be computationally expensive to be resistant to
brute-force attacks. Passwords should also have an unique salt applied
before hashing, but that is not considered by this query.
</p>
<p>
As an example, both MD5 and SHA-1 are known to be vulnerable to collision attacks.
</p>
<p>
Since it's OK to use a weak cryptographic hash function in a non-security
context, this query only alerts when these are used to hash sensitive
data (such as passwords, certificates, usernames).
</p>
<p>
Use of broken or weak cryptographic algorithms that are not hashing algorithms, is
handled by the <code>py/weak-cryptographic-algorithm</code> query.
</p>
</overview>
<recommendation>
<p>
Ensure that you use a strong, modern cryptographic hash function:
</p>
<ul>
<li>
such as Argon2, scrypt, bcrypt, or PBKDF2 for passwords and other data with limited input space.
</li>
<li>
such as SHA-2, or SHA-3 in other cases.
</li>
</ul>
</recommendation>
<example>
<p>
The following example shows two functions for checking whether the hash
of a certificate matches a known value -- to prevent tampering.
The first function uses MD5 that is known to be vulnerable to collision attacks.
The second function uses SHA-256 that is a strong cryptographic hashing function.
</p>
<sample src="examples/weak_certificate_hashing.py" />
</example>
<example>
<p>
The following example shows two functions for hashing passwords.
The first function uses SHA-256 to hash passwords. Although SHA-256 is a
strong cryptographic hash function, it is not suitable for password
hashing since it is not computationally expensive.
</p>
<sample src="examples/weak_password_hashing_bad.py" />
<p>
The second function uses Argon2 (through the <code>argon2-cffi</code>
PyPI package), which is a strong password hashing algorithm (and
includes a per-password salt by default).
</p>
<sample src="examples/weak_password_hashing_good.py" />
</example>
<references>
<li>OWASP: <a href="https://cheatsheetseries.owasp.org/cheatsheets/Password_Storage_Cheat_Sheet.html">Password Storage Cheat Sheet</a></li>
</references>
</qhelp>

48
python/ql/src/Security/CWE-327/WeakSensitiveDataHashing.ql Normal file
View File

@@ -0,0 +1,48 @@
/**
* @name Use of a broken or weak cryptographic hashing algorithm on sensitive data
* @description Using broken or weak cryptographic hashing algorithms can compromise security.
* @kind path-problem
* @problem.severity warning
* @security-severity 5.9
* @precision high
* @id py/weak-sensitive-data-hashing
* @tags security
* external/cwe/cwe-327
* external/cwe/cwe-916
*/
import python
import semmle.python.security.dataflow.WeakSensitiveDataHashing
import semmle.python.dataflow.new.DataFlow
import semmle.python.dataflow.new.TaintTracking
import DataFlow::PathGraph
from
DataFlow::PathNode source, DataFlow::PathNode sink, string ending, string algorithmName,
string classification
where
exists(NormalHashFunction::Configuration config |
config.hasFlowPath(source, sink) and
algorithmName = sink.getNode().(NormalHashFunction::Sink).getAlgorithmName() and
classification = source.getNode().(NormalHashFunction::Source).getClassification() and
ending = "."
)
or
exists(ComputationallyExpensiveHashFunction::Configuration config |
config.hasFlowPath(source, sink) and
algorithmName = sink.getNode().(ComputationallyExpensiveHashFunction::Sink).getAlgorithmName() and
classification =
source.getNode().(ComputationallyExpensiveHashFunction::Source).getClassification() and
(
sink.getNode().(ComputationallyExpensiveHashFunction::Sink).isComputationallyExpensive() and
ending = "."
or
not sink.getNode().(ComputationallyExpensiveHashFunction::Sink).isComputationallyExpensive() and
ending =
" for " + classification +
" hashing, since it is not a computationally expensive hash function."
)
)
select sink.getNode(), source, sink,
"$@ is used in a hashing algorithm (" + algorithmName + ") that is insecure" + ending,
source.getNode(), "Sensitive data (" + classification + ")"

9
python/ql/src/Security/CWE-327/examples/weak_certificate_hashing.py Normal file
View File

@@ -0,0 +1,9 @@
import hashlib
def certificate_matches_known_hash_bad(certificate, known_hash):
hash = hashlib.md5(certificate).hexdigest() # BAD
return hash == known_hash
def certificate_matches_known_hash_good(certificate, known_hash):
hash = hashlib.sha256(certificate).hexdigest() # GOOD
return hash == known_hash

4
python/ql/src/Security/CWE-327/examples/weak_password_hashing_bad.py Normal file
View File

@@ -0,0 +1,4 @@
import hashlib
def get_password_hash(password: str, salt: str):
return hashlib.sha256(password + salt).hexdigest() # BAD

9
python/ql/src/Security/CWE-327/examples/weak_password_hashing_good.py Normal file
View File

@@ -0,0 +1,9 @@
from argon2 import PasswordHasher
def get_initial_hash(password: str):
ph = PasswordHasher()
return ph.hash(password) # GOOD
def check_password(password: str, known_hash):
ph = PasswordHasher()
return ph.verify(known_hash, password) # GOOD

1
python/ql/src/Security/CWE-377/InsecureTemporaryFile.ql
View File

@@ -4,6 +4,7 @@
* @kind problem
* @id py/insecure-temporary-file
* @problem.severity error
* @security-severity 5.9
* @sub-severity high
* @precision high
* @tags external/cwe/cwe-377

1
python/ql/src/Security/CWE-502/UnsafeDeserialization.ql
View File

@@ -4,6 +4,7 @@
* @kind path-problem
* @id py/unsafe-deserialization
* @problem.severity error
* @security-severity 5.9
* @sub-severity high
* @precision high
* @tags external/cwe/cwe-502

1
python/ql/src/Security/CWE-601/UrlRedirect.ql
View File

@@ -4,6 +4,7 @@
* may cause redirection to malicious web sites.
* @kind path-problem
* @problem.severity error
* @security-severity 2.7
* @sub-severity low
* @id py/url-redirection
* @tags security

1
python/ql/src/Security/CWE-732/WeakFilePermissions.ql
View File

@@ -4,6 +4,7 @@
* @kind problem
* @id py/overly-permissive-file
* @problem.severity warning
* @security-severity 5.9
* @sub-severity high
* @precision medium
* @tags external/cwe/cwe-732

1
python/ql/src/Security/CWE-798/HardcodedCredentials.ql
View File

@@ -3,6 +3,7 @@
* @description Credentials are hard coded in the source code of the application.
* @kind path-problem
* @problem.severity error
* @security-severity 5.9
* @precision medium
* @id py/hardcoded-credentials
* @tags security

1
python/ql/src/Statements/ExecUsed.ql
View File

@@ -5,6 +5,7 @@
* @tags security
* correctness
* @problem.severity error
* @security-severity 4.2
* @sub-severity high
* @precision low
* @id py/use-of-exec

1
python/ql/src/Summary/LinesOfUserCode.ql
View File

@@ -7,6 +7,7 @@
* be counted as user written code.
* @kind metric
* @tags summary
* lines-of-code
* @id py/summary/lines-of-user-code
*/

28
python/ql/src/experimental/Security-old-dataflow/CWE-327/BrokenCryptoAlgorithm.ql Normal file
View File

@@ -0,0 +1,28 @@
/**
* @name OLD QUERY: Use of a broken or weak cryptographic algorithm
* @description Using broken or weak cryptographic algorithms can compromise security.
* @kind path-problem
* @problem.severity warning
* @id py/old/weak-cryptographic-algorithm
* @deprecated
*/
import python
import semmle.python.security.Paths
import semmle.python.security.SensitiveData
import semmle.python.security.Crypto
class BrokenCryptoConfiguration extends TaintTracking::Configuration {
BrokenCryptoConfiguration() { this = "Broken crypto configuration" }
override predicate isSource(TaintTracking::Source source) {
source instanceof SensitiveDataSource
}
override predicate isSink(TaintTracking::Sink sink) { sink instanceof WeakCryptoSink }
}
from BrokenCryptoConfiguration config, TaintedPathSource src, TaintedPathSink sink
where config.hasFlowPath(src, sink)
select sink.getSink(), src, sink, "$@ is used in a broken or weak cryptographic algorithm.",
src.getSource(), "Sensitive data"

28
python/ql/src/experimental/Security/CWE-089/ClickHouseSQLInjection.py Normal file
View File

@@ -0,0 +1,28 @@
from django.conf.urls import url
from clickhouse_driver import Client
from clickhouse_driver import connect
from aioch import Client as aiochClient
class MyClient(Client):
def dummy(self):
return None
def show_user(request, username):
# BAD -- Untrusted user input is directly injected into the sql query using async library 'aioch'
aclient = aiochClient("localhost")
progress = await aclient.execute_with_progress("SELECT * FROM users WHERE username = '%s'" % username)
# BAD -- Untrusted user input is directly injected into the sql query using native client of library 'clickhouse_driver'
Client('localhost').execute("SELECT * FROM users WHERE username = '%s'" % username)
# GOOD -- query uses prepared statements
query = "SELECT * FROM users WHERE username = %(username)s"
Client('localhost').execute(query, {"username": username})
# BAD -- PEP249 interface
conn = connect('clickhouse://localhost')
cursor = conn.cursor()
cursor.execute("SELECT * FROM users WHERE username = '%s'" % username)
urlpatterns = [url(r'^users/(?P<username>[^/]+)$', show_user)]

59
python/ql/src/experimental/Security/CWE-089/ClickHouseSQLInjection.qhelp Normal file
View File

@@ -0,0 +1,59 @@
<!DOCTYPE qhelp PUBLIC
"-//Semmle//qhelp//EN"
"qhelp.dtd">
<qhelp>
<overview>
<p>
If a database query (such as a SQL or NoSQL query) is built from
user-provided data without sufficient sanitization, a user
may be able to run malicious database queries.
</p>
</overview>
<recommendation>
<p>
Most database connector libraries offer a way of safely
embedding untrusted data into a query by means of query parameters
or prepared statements.
</p>
</recommendation>
<example>
<p>
In the following snippet, a user is fetched from a <code>ClickHouse</code> database
using five different queries. In the "BAD" cases the query is built directly from user-controlled data.
In the "GOOD" case the user-controlled data is safely embedded into the query by using query parameters.
</p>
<p>
In the first case, the query executed via aioch Client. aioch - is a module
for asynchronous queries to database.
</p>
<p>
In the second and third cases, the connection is established via `Client` class.
This class implement different method to execute a query.
</p>
<p>
In the forth case, good pattern is presented. Query parameters are send through
second dict-like argument.
</p>
<p>
In the fifth case, there is example of PEP249 interface usage.
</p>
<p>
In the sixth case, there is custom Class usge which is a subclass of default Client.
</p>
<sample src="ClickHouseSQLInjection.py" />
</example>
<references>
<li>Wikipedia: <a href="https://en.wikipedia.org/wiki/SQL_injection">SQL injection</a>.</li>
<li>OWASP: <a href="https://cheatsheetseries.owasp.org/cheatsheets/SQL_Injection_Prevention_Cheat_Sheet.html">SQL Injection Prevention Cheat Sheet</a>.</li>
</references>
</qhelp>

22
python/ql/src/experimental/Security/CWE-089/ClickHouseSQLInjection.ql Normal file
View File

@@ -0,0 +1,22 @@
/**
* @id py/yandex/clickhouse-sql-injection
* @name Clickhouse SQL query built from user-controlled sources
* @description Building a SQL query from user-controlled sources is vulnerable to insertion of
* malicious SQL code by the user.
* @kind path-problem
* @problem.severity error
* @precision high
* @tags security
* external/cwe/cwe-089
* external/owasp/owasp-a1
*/
import python
import experimental.semmle.python.frameworks.ClickHouseDriver
import semmle.python.security.dataflow.SqlInjection
import DataFlow::PathGraph
from SQLInjectionConfiguration config, DataFlow::PathNode source, DataFlow::PathNode sink
where config.hasFlowPath(source, sink)
select sink.getNode(), source, sink, "This SQL query depends on $@.", source.getNode(),
"a user-provided value"

50
python/ql/src/experimental/Security/CWE-090/LDAPInjection.qhelp Normal file
View File

@@ -0,0 +1,50 @@
<!DOCTYPE qhelp PUBLIC
"-//Semmle//qhelp//EN"
"qhelp.dtd">
<qhelp>
<overview>
<p>If an LDAP query or DN is built using string concatenation or string formatting, and the
components of the concatenation include user input without any proper sanitization, a user
is likely to be able to run malicious LDAP queries.</p>
</overview>
<recommendation>
<p>If user input must be included in an LDAP query or DN, it should be escaped to
avoid a malicious user providing special characters that change the meaning
of the query. In Python2, user input should be escaped with <code>ldap.dn.escape_dn_chars</code>
or <code>ldap.filter.escape_filter_chars</code>, while in Python3, user input should be escaped with
<code>ldap3.utils.dn.escape_rdn</code> or <code>ldap3.utils.conv.escape_filter_chars</code>
depending on the component tainted by the user. A good practice is to escape filter characters
that could change the meaning of the query (https://tools.ietf.org/search/rfc4515#section-3).</p>
</recommendation>
<example>
<p>In the following examples, the code accepts both <code>username</code> and <code>dc</code> from the user,
which it then uses to build a LDAP query and DN.</p>
<p>The first and the second example uses the unsanitized user input directly
in the search filter and DN for the LDAP query.
A malicious user could provide special characters to change the meaning of these
components, and search for a completely different set of values.</p>
<sample src="examples/example_bad1.py" />
<sample src="examples/example_bad2.py" />
<p>In the third and four example, the input provided by the user is sanitized before it is included in the search filter or DN.
This ensures the meaning of the query cannot be changed by a malicious user.</p>
<sample src="examples/example_good1.py" />
<sample src="examples/example_good2.py" />
</example>
<references>
<li>OWASP: <a href="https://cheatsheetseries.owasp.org/cheatsheets/LDAP_Injection_Prevention_Cheat_Sheet.html">LDAP Injection Prevention Cheat Sheet</a>.</li>
<li>OWASP: <a href="https://owasp.org/www-community/attacks/LDAP_Injection">LDAP Injection</a>.</li>
<li>SonarSource: <a href="https://rules.sonarsource.com/python/RSPEC-2078">RSPEC-2078</a>.</li>
<li>Python2: <a href="https://www.python-ldap.org/en/python-ldap-3.3.0/reference/ldap.html">LDAP Documentation</a>.</li>
<li>Python3: <a href="https://ldap3.readthedocs.io/en/latest/">LDAP Documentation</a>.</li>
<li>Wikipedia: <a href="https://en.wikipedia.org/wiki/LDAP_injection">LDAP injection</a>.</li>
<li>BlackHat: <a href="https://www.blackhat.com/presentations/bh-europe-08/Alonso-Parada/Whitepaper/bh-eu-08-alonso-parada-WP.pdf">LDAP Injection and Blind LDAP Injection</a>.</li>
<li>LDAP: <a href="https://ldap.com/2018/05/04/understanding-and-defending-against-ldap-injection-attacks/">Understanding and Defending Against LDAP Injection Attacks</a>.</li>
</references>
</qhelp>

21
python/ql/src/experimental/Security/CWE-090/LDAPInjection.ql Normal file
View File

@@ -0,0 +1,21 @@
/**
* @name LDAP query built from user-controlled sources
* @description Building an LDAP query from user-controlled sources is vulnerable to insertion of
* malicious LDAP code by the user.
* @kind path-problem
* @problem.severity error
* @id py/ldap-injection
* @tags experimental
* security
* external/cwe/cwe-090
*/
// Determine precision above
import python
import experimental.semmle.python.security.injection.LDAP
import DataFlow::PathGraph
from LDAPInjectionFlowConfig config, DataFlow::PathNode source, DataFlow::PathNode sink
where config.hasFlowPath(source, sink)
select sink.getNode(), source, sink, "$@ LDAP query parameter comes from $@.", sink.getNode(),
"This", source.getNode(), "a user-provided value"

15
python/ql/src/experimental/Security/CWE-090/examples/example_bad1.py Normal file
View File

@@ -0,0 +1,15 @@
from flask import request, Flask
import ldap
@app.route("/normal")
def normal():
unsafe_dc = request.args['dc']
unsafe_filter = request.args['username']
dn = "dc={}".format(unsafe_dc)
search_filter = "(user={})".format(unsafe_filter)
ldap_connection = ldap.initialize("ldap://127.0.0.1")
user = ldap_connection.search_s(
dn, ldap.SCOPE_SUBTREE, search_filter)

15
python/ql/src/experimental/Security/CWE-090/examples/example_bad2.py Normal file
View File

@@ -0,0 +1,15 @@
from flask import request, Flask
import ldap3
@app.route("/normal")
def normal():
unsafe_dc = request.args['dc']
unsafe_filter = request.args['username']
dn = "dc={}".format(unsafe_dc)
search_filter = "(user={})".format(unsafe_filter)
srv = ldap3.Server('ldap://127.0.0.1')
conn = ldap3.Connection(srv, user=dn, auto_bind=True)
conn.search(dn, search_filter)

20
python/ql/src/experimental/Security/CWE-090/examples/example_good1.py Normal file
View File

@@ -0,0 +1,20 @@
from flask import request, Flask
import ldap
import ldap.filter
import ldap.dn
@app.route("/normal")
def normal():
unsafe_dc = request.args['dc']
unsafe_filter = request.args['username']
safe_dc = ldap.dn.escape_dn_chars(unsafe_dc)
safe_filter = ldap.filter.escape_filter_chars(unsafe_filter)
dn = "dc={}".format(safe_dc)
search_filter = "(user={})".format(safe_filter)
ldap_connection = ldap.initialize("ldap://127.0.0.1")
user = ldap_connection.search_s(
dn, ldap.SCOPE_SUBTREE, search_filter)

20
python/ql/src/experimental/Security/CWE-090/examples/example_good2.py Normal file
View File

@@ -0,0 +1,20 @@
from flask import request, Flask
import ldap3
from ldap3.utils.dn import escape_rdn
from ldap3.utils.conv import escape_filter_chars
@app.route("/normal")
def normal():
unsafe_dc = request.args['dc']
unsafe_filter = request.args['username']
safe_dc = escape_rdn(unsafe_dc)
safe_filter = escape_filter_chars(unsafe_filter)
dn = "dc={}".format(safe_dc)
search_filter = "(user={})".format(safe_filter)
srv = ldap3.Server('ldap://127.0.0.1')
conn = ldap3.Connection(srv, user=dn, auto_bind=True)
conn.search(dn, search_filter)

45
python/ql/src/experimental/Security/CWE-730/RegexInjection.qhelp Normal file
View File

@@ -0,0 +1,45 @@
<!DOCTYPE qhelp PUBLIC
"-//Semmle//qhelp//EN"
"qhelp.dtd">
<qhelp>
<overview>
<p>
Constructing a regular expression with unsanitized user input is dangerous as a malicious user may
be able to modify the meaning of the expression. In particular, such a user may be able to provide
a regular expression fragment that takes exponential time in the worst case, and use that to
perform a Denial of Service attack.
</p>
</overview>
<recommendation>
<p>
Before embedding user input into a regular expression, use a sanitization function such as
<code>re.escape</code> to escape meta-characters that have a special meaning regarding
regular expressions' syntax.
</p>
</recommendation>
<example>
<p>
The following examples are based on a simple Flask web server environment.
</p>
<p>
The following example shows a HTTP request parameter that is used to construct a regular expression
without sanitizing it first:
</p>
<sample src="re_bad.py" />
<p>
Instead, the request parameter should be sanitized first, for example using the function
<code>re.escape</code>. This ensures that the user cannot insert characters which have a
special meaning in regular expressions.
</p>
<sample src="re_good.py" />
</example>
<references>
<li>OWASP: <a href="https://www.owasp.org/index.php/Regular_expression_Denial_of_Service_-_ReDoS">Regular expression Denial of Service - ReDoS</a>.</li>
<li>Wikipedia: <a href="https://en.wikipedia.org/wiki/ReDoS">ReDoS</a>.</li>
<li>Python docs: <a href="https://docs.python.org/3/library/re.html">re</a>.</li>
<li>SonarSource: <a href="https://rules.sonarsource.com/python/type/Vulnerability/RSPEC-2631">RSPEC-2631</a>.</li>
</references>
</qhelp>

29
python/ql/src/experimental/Security/CWE-730/RegexInjection.ql Normal file
View File

@@ -0,0 +1,29 @@
/**
* @name Regular expression injection
* @description User input should not be used in regular expressions without first being escaped,
* otherwise a malicious user may be able to inject an expression that could require
* exponential time on certain inputs.
* @kind path-problem
* @problem.severity error
* @id py/regex-injection
* @tags security
* external/cwe/cwe-730
* external/cwe/cwe-400
*/
// determine precision above
import python
import experimental.semmle.python.security.injection.RegexInjection
import DataFlow::PathGraph
from
RegexInjectionFlowConfig config, DataFlow::PathNode source, DataFlow::PathNode sink,
RegexInjectionSink regexInjectionSink, Attribute methodAttribute
where
config.hasFlowPath(source, sink) and
regexInjectionSink = sink.getNode() and
methodAttribute = regexInjectionSink.getRegexMethod()
select sink.getNode(), source, sink,
"$@ regular expression is constructed from a $@ and executed by $@.", sink.getNode(), "This",
source.getNode(), "user-provided value", methodAttribute,
regexInjectionSink.getRegexModule() + "." + methodAttribute.getName()

15
python/ql/src/experimental/Security/CWE-730/re_bad.py Normal file
View File

@@ -0,0 +1,15 @@
from flask import request, Flask
import re
@app.route("/direct")
def direct():
unsafe_pattern = request.args["pattern"]
re.search(unsafe_pattern, "")
@app.route("/compile")
def compile():
unsafe_pattern = request.args["pattern"]
compiled_pattern = re.compile(unsafe_pattern)
compiled_pattern.search("")

17
python/ql/src/experimental/Security/CWE-730/re_good.py Normal file
View File

@@ -0,0 +1,17 @@
from flask import request, Flask
import re
@app.route("/direct")
def direct():
unsafe_pattern = request.args['pattern']
safe_pattern = re.escape(unsafe_pattern)
re.search(safe_pattern, "")
@app.route("/compile")
def compile():
unsafe_pattern = request.args['pattern']
safe_pattern = re.escape(unsafe_pattern)
compiled_pattern = re.compile(safe_pattern)
compiled_pattern.search("")

133
python/ql/src/experimental/semmle/python/Concepts.qll
View File

@@ -14,6 +14,139 @@ private import semmle.python.dataflow.new.RemoteFlowSources
private import semmle.python.dataflow.new.TaintTracking
private import experimental.semmle.python.Frameworks
/** Provides classes for modeling Regular Expression-related APIs. */
module RegexExecution {
/**
* A data-flow node that executes a regular expression.
*
* Extend this class to model new APIs. If you want to refine existing API models,
* extend `RegexExecution` instead.
*/
abstract class Range extends DataFlow::Node {
/**
* Gets the argument containing the executed expression.
*/
abstract DataFlow::Node getRegexNode();
/**
* Gets the library used to execute the regular expression.
*/
abstract string getRegexModule();
}
}
/**
* A data-flow node that executes a regular expression.
*
* Extend this class to refine existing API models. If you want to model new APIs,
* extend `RegexExecution::Range` instead.
*/
class RegexExecution extends DataFlow::Node {
RegexExecution::Range range;
RegexExecution() { this = range }
DataFlow::Node getRegexNode() { result = range.getRegexNode() }
string getRegexModule() { result = range.getRegexModule() }
}
/** Provides classes for modeling Regular Expression escape-related APIs. */
module RegexEscape {
/**
* A data-flow node that escapes a regular expression.
*
* Extend this class to model new APIs. If you want to refine existing API models,
* extend `RegexEscape` instead.
*/
abstract class Range extends DataFlow::Node {
/**
* Gets the argument containing the escaped expression.
*/
abstract DataFlow::Node getRegexNode();
}
}
/**
* A data-flow node that escapes a regular expression.
*
* Extend this class to refine existing API models. If you want to model new APIs,
* extend `RegexEscape::Range` instead.
*/
class RegexEscape extends DataFlow::Node {
RegexEscape::Range range;
RegexEscape() { this = range }
DataFlow::Node getRegexNode() { result = range.getRegexNode() }
}
/** Provides classes for modeling LDAP query execution-related APIs. */
module LDAPQuery {
/**
* A data-flow node that collects methods executing a LDAP query.
*
* Extend this class to model new APIs. If you want to refine existing API models,
* extend `LDAPQuery` instead.
*/
abstract class Range extends DataFlow::Node {
/**
* Gets the argument containing the executed expression.
*/
abstract DataFlow::Node getQuery();
}
}
/**
* A data-flow node that collect methods executing a LDAP query.
*
* Extend this class to refine existing API models. If you want to model new APIs,
* extend `LDAPQuery::Range` instead.
*/
class LDAPQuery extends DataFlow::Node {
LDAPQuery::Range range;
LDAPQuery() { this = range }
/**
* Gets the argument containing the executed expression.
*/
DataFlow::Node getQuery() { result = range.getQuery() }
}
/** Provides classes for modeling LDAP components escape-related APIs. */
module LDAPEscape {
/**
* A data-flow node that collects functions escaping LDAP components.
*
* Extend this class to model new APIs. If you want to refine existing API models,
* extend `LDAPEscape` instead.
*/
abstract class Range extends DataFlow::Node {
/**
* Gets the argument containing the escaped expression.
*/
abstract DataFlow::Node getAnInput();
}
}
/**
* A data-flow node that collects functions escaping LDAP components.
*
* Extend this class to refine existing API models. If you want to model new APIs,
* extend `LDAPEscape::Range` instead.
*/
class LDAPEscape extends DataFlow::Node {
LDAPEscape::Range range;
LDAPEscape() { this = range }
/**
* Gets the argument containing the escaped expression.
*/
DataFlow::Node getAnInput() { result = range.getAnInput() }
}
/** Provides classes for modeling LDAP bind-related APIs. */
module LDAPBind {
/**

1
python/ql/src/experimental/semmle/python/Frameworks.qll
View File

@@ -3,3 +3,4 @@
*/
private import experimental.semmle.python.frameworks.Stdlib
private import experimental.semmle.python.frameworks.LDAP

85
python/ql/src/experimental/semmle/python/frameworks/ClickHouseDriver.qll Normal file
View File

@@ -0,0 +1,85 @@
/**
* Provides classes modeling security-relevant aspects of `clickhouse-driver` and `aioch` PyPI packages.
* See
* - https://pypi.org/project/clickhouse-driver/
* - https://pypi.org/project/aioch/
* - https://clickhouse-driver.readthedocs.io/en/latest/
*/
private import python
private import semmle.python.Concepts
private import semmle.python.ApiGraphs
private import semmle.python.frameworks.PEP249
/**
* Provides models for `clickhouse-driver` and `aioch` PyPI packages.
* See
* - https://pypi.org/project/clickhouse-driver/
* - https://pypi.org/project/aioch/
* - https://clickhouse-driver.readthedocs.io/en/latest/
*/
module ClickHouseDriver {
/** Gets a reference to the `clickhouse_driver` module. */
API::Node clickhouse_driver() { result = API::moduleImport("clickhouse_driver") }
/** Gets a reference to the `aioch` module. This module allows to make async db queries. */
API::Node aioch() { result = API::moduleImport("aioch") }
/**
* `clickhouse_driver` implements PEP249,
* providing ways to execute SQL statements against a database.
*/
class ClickHouseDriverPEP249 extends PEP249ModuleApiNode {
ClickHouseDriverPEP249() { this = clickhouse_driver() }
}
module Client {
/** Gets a reference to a Client call. */
private DataFlow::Node client_ref() {
result = clickhouse_driver().getMember("Client").getASubclass*().getAUse()
or
result = aioch().getMember("Client").getASubclass*().getAUse()
}
/** A direct instantiation of `clickhouse_driver.Client`. */
private class ClientInstantiation extends DataFlow::CallCfgNode {
ClientInstantiation() { this.getFunction() = client_ref() }
}
/** Gets a reference to an instance of `clickhouse_driver.Client`. */
private DataFlow::LocalSourceNode instance(DataFlow::TypeTracker t) {
t.start() and
result instanceof ClientInstantiation
or
exists(DataFlow::TypeTracker t2 | result = instance(t2).track(t2, t))
}
/** Gets a reference to an instance of `clickhouse_driver.Client`. */
DataFlow::Node instance() { instance(DataFlow::TypeTracker::end()).flowsTo(result) }
}
/** clickhouse_driver.Client execute methods */
private string execute_function() {
result in ["execute_with_progress", "execute", "execute_iter"]
}
/** Gets a reference to the `clickhouse_driver.Client.execute` method */
private DataFlow::LocalSourceNode clickhouse_execute(DataFlow::TypeTracker t) {
t.startInAttr(execute_function()) and
result = Client::instance()
or
exists(DataFlow::TypeTracker t2 | result = clickhouse_execute(t2).track(t2, t))
}
/** Gets a reference to the `clickhouse_driver.Client.execute` method */
DataFlow::Node clickhouse_execute() {
clickhouse_execute(DataFlow::TypeTracker::end()).flowsTo(result)
}
/** A call to the `clickhouse_driver.Client.execute` method */
private class ExecuteCall extends SqlExecution::Range, DataFlow::CallCfgNode {
ExecuteCall() { this.getFunction() = clickhouse_execute() }
override DataFlow::Node getSql() { result.asCfgNode() = node.getArg(0) }
}
}

153
python/ql/src/experimental/semmle/python/frameworks/LDAP.qll Normal file
View File

@@ -0,0 +1,153 @@
/**
* Provides classes modeling security-relevant aspects of the LDAP libraries.
*/
private import python
private import semmle.python.dataflow.new.DataFlow
private import semmle.python.dataflow.new.TaintTracking
private import semmle.python.dataflow.new.RemoteFlowSources
private import experimental.semmle.python.Concepts
private import semmle.python.ApiGraphs
/**
* Provides models for Python's ldap-related libraries.
*/
private module LDAP {
/**
* Provides models for the `python-ldap` PyPI package (imported as `ldap`).
*
* See https://www.python-ldap.org/en/python-ldap-3.3.0/index.html
*/
private module LDAP2 {
/**
* List of `ldap` methods used to execute a query.
*
* See https://www.python-ldap.org/en/python-ldap-3.3.0/reference/ldap.html#functions
*/
private class LDAP2QueryMethods extends string {
LDAP2QueryMethods() {
this in ["search", "search_s", "search_st", "search_ext", "search_ext_s"]
}
}
/**
* A class to find `ldap` methods executing a query.
*
* See `LDAP2QueryMethods`
*/
private class LDAP2Query extends DataFlow::CallCfgNode, LDAPQuery::Range {
DataFlow::Node ldapQuery;
LDAP2Query() {
exists(DataFlow::AttrRead searchMethod |
this.getFunction() = searchMethod and
API::moduleImport("ldap").getMember("initialize").getACall() =
searchMethod.getObject().getALocalSource() and
searchMethod.getAttributeName() instanceof LDAP2QueryMethods and
(
ldapQuery = this.getArg(0)
or
(
ldapQuery = this.getArg(2) or
ldapQuery = this.getArgByName("filterstr")
)
)
)
}
override DataFlow::Node getQuery() { result = ldapQuery }
}
/**
* A class to find calls to `ldap.dn.escape_dn_chars`.
*
* See https://github.com/python-ldap/python-ldap/blob/7ce471e238cdd9a4dd8d17baccd1c9e05e6f894a/Lib/ldap/dn.py#L17
*/
private class LDAP2EscapeDNCall extends DataFlow::CallCfgNode, LDAPEscape::Range {
LDAP2EscapeDNCall() {
this = API::moduleImport("ldap").getMember("dn").getMember("escape_dn_chars").getACall()
}
override DataFlow::Node getAnInput() { result = this.getArg(0) }
}
/**
* A class to find calls to `ldap.filter.escape_filter_chars`.
*
* See https://www.python-ldap.org/en/python-ldap-3.3.0/reference/ldap-filter.html#ldap.filter.escape_filter_chars
*/
private class LDAP2EscapeFilterCall extends DataFlow::CallCfgNode, LDAPEscape::Range {
LDAP2EscapeFilterCall() {
this =
API::moduleImport("ldap").getMember("filter").getMember("escape_filter_chars").getACall()
}
override DataFlow::Node getAnInput() { result = this.getArg(0) }
}
}
/**
* Provides models for the `ldap3` PyPI package
*
* See https://pypi.org/project/ldap3/
*/
private module LDAP3 {
/**
* A class to find `ldap3` methods executing a query.
*/
private class LDAP3Query extends DataFlow::CallCfgNode, LDAPQuery::Range {
DataFlow::Node ldapQuery;
LDAP3Query() {
exists(DataFlow::AttrRead searchMethod |
this.getFunction() = searchMethod and
API::moduleImport("ldap3").getMember("Connection").getACall() =
searchMethod.getObject().getALocalSource() and
searchMethod.getAttributeName() = "search" and
(
ldapQuery = this.getArg(0) or
ldapQuery = this.getArg(1)
)
)
}
override DataFlow::Node getQuery() { result = ldapQuery }
}
/**
* A class to find calls to `ldap3.utils.dn.escape_rdn`.
*
* See https://github.com/cannatag/ldap3/blob/4d33166f0869b929f59c6e6825a1b9505eb99967/ldap3/utils/dn.py#L390
*/
private class LDAP3EscapeDNCall extends DataFlow::CallCfgNode, LDAPEscape::Range {
LDAP3EscapeDNCall() {
this =
API::moduleImport("ldap3")
.getMember("utils")
.getMember("dn")
.getMember("escape_rdn")
.getACall()
}
override DataFlow::Node getAnInput() { result = this.getArg(0) }
}
/**
* A class to find calls to `ldap3.utils.conv.escape_filter_chars`.
*
* See https://github.com/cannatag/ldap3/blob/4d33166f0869b929f59c6e6825a1b9505eb99967/ldap3/utils/conv.py#L91
*/
private class LDAP3EscapeFilterCall extends DataFlow::CallCfgNode, LDAPEscape::Range {
LDAP3EscapeFilterCall() {
this =
API::moduleImport("ldap3")
.getMember("utils")
.getMember("conv")
.getMember("escape_filter_chars")
.getACall()
}
override DataFlow::Node getAnInput() { result = this.getArg(0) }
}
}
}

89
python/ql/src/experimental/semmle/python/frameworks/Stdlib.qll
View File

@@ -10,6 +10,95 @@ private import semmle.python.dataflow.new.RemoteFlowSources
private import experimental.semmle.python.Concepts
private import semmle.python.ApiGraphs
/**
* Provides models for Python's `re` library.
*
* See https://docs.python.org/3/library/re.html
*/
private module Re {
/**
* List of `re` methods immediately executing an expression.
*
* See https://docs.python.org/3/library/re.html#module-contents
*/
private class RegexExecutionMethods extends string {
RegexExecutionMethods() {
this in ["match", "fullmatch", "search", "split", "findall", "finditer", "sub", "subn"]
}
}
/**
* A class to find `re` methods immediately executing an expression.
*
* See `RegexExecutionMethods`
*/
private class DirectRegex extends DataFlow::CallCfgNode, RegexExecution::Range {
DataFlow::Node regexNode;
DirectRegex() {
this = API::moduleImport("re").getMember(any(RegexExecutionMethods m)).getACall() and
regexNode = this.getArg(0)
}
override DataFlow::Node getRegexNode() { result = regexNode }
override string getRegexModule() { result = "re" }
}
/**
* A class to find `re` methods immediately executing a compiled expression by `re.compile`.
*
* Given the following example:
*
* ```py
* pattern = re.compile(input)
* pattern.match(s)
* ```
*
* This class will identify that `re.compile` compiles `input` and afterwards
* executes `re`'s `match`. As a result, `this` will refer to `pattern.match(s)`
* and `this.getRegexNode()` will return the node for `input` (`re.compile`'s first argument)
*
*
* See `RegexExecutionMethods`
*
* See https://docs.python.org/3/library/re.html#regular-expression-objects
*/
private class CompiledRegex extends DataFlow::CallCfgNode, RegexExecution::Range {
DataFlow::Node regexNode;
CompiledRegex() {
exists(DataFlow::CallCfgNode patternCall, DataFlow::AttrRead reMethod |
this.getFunction() = reMethod and
patternCall = API::moduleImport("re").getMember("compile").getACall() and
patternCall.flowsTo(reMethod.getObject()) and
reMethod.getAttributeName() instanceof RegexExecutionMethods and
regexNode = patternCall.getArg(0)
)
}
override DataFlow::Node getRegexNode() { result = regexNode }
override string getRegexModule() { result = "re" }
}
/**
* A class to find `re` methods escaping an expression.
*
* See https://docs.python.org/3/library/re.html#re.escape
*/
class ReEscape extends DataFlow::CallCfgNode, RegexEscape::Range {
DataFlow::Node regexNode;
ReEscape() {
this = API::moduleImport("re").getMember("escape").getACall() and
regexNode = this.getArg(0)
}
override DataFlow::Node getRegexNode() { result = regexNode }
}
}
/**
* Provides models for Python's ldap-related libraries.
*/

24
python/ql/src/experimental/semmle/python/security/injection/LDAP.qll Normal file
View File

@@ -0,0 +1,24 @@
/**
* Provides a taint-tracking configuration for detecting LDAP injection vulnerabilities
*/
import python
import experimental.semmle.python.Concepts
import semmle.python.dataflow.new.DataFlow
import semmle.python.dataflow.new.TaintTracking
import semmle.python.dataflow.new.RemoteFlowSources
/**
* A taint-tracking configuration for detecting LDAP injections.
*/
class LDAPInjectionFlowConfig extends TaintTracking::Configuration {
LDAPInjectionFlowConfig() { this = "LDAPInjectionFlowConfig" }
override predicate isSource(DataFlow::Node source) { source instanceof RemoteFlowSource }
override predicate isSink(DataFlow::Node sink) { sink = any(LDAPQuery ldapQuery).getQuery() }
override predicate isSanitizer(DataFlow::Node sanitizer) {
sanitizer = any(LDAPEscape ldapEsc).getAnInput()
}
}

53
python/ql/src/experimental/semmle/python/security/injection/RegexInjection.qll Normal file
View File

@@ -0,0 +1,53 @@
/**
* Provides a taint-tracking configuration for detecting regular expression injection
* vulnerabilities.
*/
import python
import experimental.semmle.python.Concepts
import semmle.python.dataflow.new.DataFlow
import semmle.python.dataflow.new.TaintTracking
import semmle.python.dataflow.new.RemoteFlowSources
/**
* A class to find methods executing regular expressions.
*
* See `RegexExecution`
*/
class RegexInjectionSink extends DataFlow::Node {
string regexModule;
Attribute regexMethod;
RegexInjectionSink() {
exists(RegexExecution reExec |
this = reExec.getRegexNode() and
regexModule = reExec.getRegexModule() and
regexMethod = reExec.(DataFlow::CallCfgNode).getFunction().asExpr().(Attribute)
)
}
/**
* Gets the argument containing the executed expression.
*/
string getRegexModule() { result = regexModule }
/**
* Gets the method used to execute the regular expression.
*/
Attribute getRegexMethod() { result = regexMethod }
}
/**
* A taint-tracking configuration for detecting regular expression injections.
*/
class RegexInjectionFlowConfig extends TaintTracking::Configuration {
RegexInjectionFlowConfig() { this = "RegexInjectionFlowConfig" }
override predicate isSource(DataFlow::Node source) { source instanceof RemoteFlowSource }
override predicate isSink(DataFlow::Node sink) { sink instanceof RegexInjectionSink }
override predicate isSanitizer(DataFlow::Node sanitizer) {
sanitizer = any(RegexEscape reEscape).getRegexNode()
}
}

175
python/ql/src/semmle/crypto/Crypto.qll
View File

@@ -1,174 +1,3 @@
/**
* Provides classes modeling cryptographic algorithms, separated into strong and weak variants.
*
* The classification into strong and weak are based on Wikipedia, OWASP and google (2017).
*/
/** DEPRECATED: Use `semmle.python.concepts.CryptoAlgorithms` instead. */
/**
* Names of cryptographic algorithms, separated into strong and weak variants.
*
* The names are normalized: upper-case, no spaces, dashes or underscores.
*
* The names are inspired by the names used in real world crypto libraries.
*
* The classification into strong and weak are based on Wikipedia, OWASP and google (2017).
*/
private module AlgorithmNames {
predicate isStrongHashingAlgorithm(string name) {
name = "DSA" or
name = "ED25519" or
name = "ES256" or
name = "ECDSA256" or
name = "ES384" or
name = "ECDSA384" or
name = "ES512" or
name = "ECDSA512" or
name = "SHA2" or
name = "SHA224" or
name = "SHA256" or
name = "SHA384" or
name = "SHA512" or
name = "SHA3"
}
predicate isWeakHashingAlgorithm(string name) {
name = "HAVEL128" or
name = "MD2" or
name = "MD4" or
name = "MD5" or
name = "PANAMA" or
name = "RIPEMD" or
name = "RIPEMD128" or
name = "RIPEMD256" or
name = "RIPEMD160" or
name = "RIPEMD320" or
name = "SHA0" or
name = "SHA1"
}
predicate isStrongEncryptionAlgorithm(string name) {
name = "AES" or
name = "AES128" or
name = "AES192" or
name = "AES256" or
name = "AES512" or
name = "RSA" or
name = "RABBIT" or
name = "BLOWFISH"
}
predicate isWeakEncryptionAlgorithm(string name) {
name = "DES" or
name = "3DES" or
name = "TRIPLEDES" or
name = "TDEA" or
name = "TRIPLEDEA" or
name = "ARC2" or
name = "RC2" or
name = "ARC4" or
name = "RC4" or
name = "ARCFOUR" or
name = "ARC5" or
name = "RC5"
}
predicate isStrongPasswordHashingAlgorithm(string name) {
name = "ARGON2" or
name = "PBKDF2" or
name = "BCRYPT" or
name = "SCRYPT"
}
predicate isWeakPasswordHashingAlgorithm(string name) { none() }
}
private import AlgorithmNames
/**
* A cryptographic algorithm.
*/
private newtype TCryptographicAlgorithm =
MkHashingAlgorithm(string name, boolean isWeak) {
isStrongHashingAlgorithm(name) and isWeak = false
or
isWeakHashingAlgorithm(name) and isWeak = true
} or
MkEncryptionAlgorithm(string name, boolean isWeak) {
isStrongEncryptionAlgorithm(name) and isWeak = false
or
isWeakEncryptionAlgorithm(name) and isWeak = true
} or
MkPasswordHashingAlgorithm(string name, boolean isWeak) {
isStrongPasswordHashingAlgorithm(name) and isWeak = false
or
isWeakPasswordHashingAlgorithm(name) and isWeak = true
}
/**
* A cryptographic algorithm.
*/
abstract class CryptographicAlgorithm extends TCryptographicAlgorithm {
/** Gets a textual representation of this element. */
string toString() { result = getName() }
/**
* Gets the normalized name of this algorithm (upper-case, no spaces, dashes or underscores).
*/
abstract string getName();
/**
* Holds if the name of this algorithm matches `name` modulo case,
* white space, dashes, and underscores.
*/
bindingset[name]
predicate matchesName(string name) {
name.toUpperCase().regexpReplaceAll("[-_ ]", "") = getName()
}
/**
* Holds if this algorithm is weak.
*/
abstract predicate isWeak();
}
/**
* A hashing algorithm such as `MD5` or `SHA512`.
*/
class HashingAlgorithm extends MkHashingAlgorithm, CryptographicAlgorithm {
string name;
boolean isWeak;
HashingAlgorithm() { this = MkHashingAlgorithm(name, isWeak) }
override string getName() { result = name }
override predicate isWeak() { isWeak = true }
}
/**
* An encryption algorithm such as `DES` or `AES512`.
*/
class EncryptionAlgorithm extends MkEncryptionAlgorithm, CryptographicAlgorithm {
string name;
boolean isWeak;
EncryptionAlgorithm() { this = MkEncryptionAlgorithm(name, isWeak) }
override string getName() { result = name }
override predicate isWeak() { isWeak = true }
}
/**
* A password hashing algorithm such as `PBKDF2` or `SCRYPT`.
*/
class PasswordHashingAlgorithm extends MkPasswordHashingAlgorithm, CryptographicAlgorithm {
string name;
boolean isWeak;
PasswordHashingAlgorithm() { this = MkPasswordHashingAlgorithm(name, isWeak) }
override string getName() { result = name }
override predicate isWeak() { isWeak = true }
}
import semmle.python.concepts.CryptoAlgorithms

91
python/ql/src/semmle/python/ApiGraphs.qll
View File

@@ -349,22 +349,95 @@ module API {
)
}
private import semmle.python.types.Builtins as Builtins
/** Gets the name of a known built-in. */
private string getBuiltInName() {
// These lists were created by inspecting the `builtins` and `__builtin__` modules in
// Python 3 and 2 respectively, using the `dir` built-in.
// Built-in functions and exceptions shared between Python 2 and 3
result in [
"abs", "all", "any", "bin", "bool", "bytearray", "callable", "chr", "classmethod",
"compile", "complex", "delattr", "dict", "dir", "divmod", "enumerate", "eval", "filter",
"float", "format", "frozenset", "getattr", "globals", "hasattr", "hash", "help", "hex",
"id", "input", "int", "isinstance", "issubclass", "iter", "len", "list", "locals", "map",
"max", "memoryview", "min", "next", "object", "oct", "open", "ord", "pow", "print",
"property", "range", "repr", "reversed", "round", "set", "setattr", "slice", "sorted",
"staticmethod", "str", "sum", "super", "tuple", "type", "vars", "zip", "__import__",
// Exceptions
"ArithmeticError", "AssertionError", "AttributeError", "BaseException", "BufferError",
"BytesWarning", "DeprecationWarning", "EOFError", "EnvironmentError", "Exception",
"FloatingPointError", "FutureWarning", "GeneratorExit", "IOError", "ImportError",
"ImportWarning", "IndentationError", "IndexError", "KeyError", "KeyboardInterrupt",
"LookupError", "MemoryError", "NameError", "NotImplemented", "NotImplementedError",
"OSError", "OverflowError", "PendingDeprecationWarning", "ReferenceError", "RuntimeError",
"RuntimeWarning", "StandardError", "StopIteration", "SyntaxError", "SyntaxWarning",
"SystemError", "SystemExit", "TabError", "TypeError", "UnboundLocalError",
"UnicodeDecodeError", "UnicodeEncodeError", "UnicodeError", "UnicodeTranslateError",
"UnicodeWarning", "UserWarning", "ValueError", "Warning", "ZeroDivisionError",
// Added for compatibility
"exec"
]
or
// Built-in constants shared between Python 2 and 3
result in ["False", "True", "None", "NotImplemented", "Ellipsis", "__debug__"]
or
// Python 3 only
result in [
"ascii", "breakpoint", "bytes", "exec",
// Exceptions
"BlockingIOError", "BrokenPipeError", "ChildProcessError", "ConnectionAbortedError",
"ConnectionError", "ConnectionRefusedError", "ConnectionResetError", "FileExistsError",
"FileNotFoundError", "InterruptedError", "IsADirectoryError", "ModuleNotFoundError",
"NotADirectoryError", "PermissionError", "ProcessLookupError", "RecursionError",
"ResourceWarning", "StopAsyncIteration", "TimeoutError"
]
or
// Python 2 only
result in [
"basestring", "cmp", "execfile", "file", "long", "raw_input", "reduce", "reload",
"unichr", "unicode", "xrange"
]
}
/**
* Gets a data flow node that is likely to refer to a built-in with the name `name`.
*
* Currently this is an over-approximation, and does not account for things like overwriting a
* Currently this is an over-approximation, and may not account for things like overwriting a
* built-in with a different value.
*/
private DataFlow::Node likely_builtin(string name) {
result.asCfgNode() =
any(NameNode n |
n.isGlobal() and
n.isLoad() and
name = n.getId() and
name in [any(Builtins::Builtin b).getName(), "None", "True", "False"]
)
exists(Module m |
result.asCfgNode() =
any(NameNode n |
possible_builtin_accessed_in_module(n, name, m) and
not possible_builtin_defined_in_module(name, m)
)
)
}
/**
* Holds if a global variable called `name` (which is also the name of a built-in) is assigned
* a value in the module `m`.
*/
private predicate possible_builtin_defined_in_module(string name, Module m) {
exists(NameNode n |
not exists(LocalVariable v | n.defines(v)) and
n.isStore() and
name = n.getId() and
name = getBuiltInName() and
m = n.getEnclosingModule()
)
}
/**
* Holds if `n` is an access of a global variable called `name` (which is also the name of a
* built-in) inside the module `m`.
*/
private predicate possible_builtin_accessed_in_module(NameNode n, string name, Module m) {
n.isGlobal() and
n.isLoad() and
name = n.getId() and
name = getBuiltInName() and
m = n.getEnclosingModule()
}
/**

48
python/ql/src/semmle/python/Concepts.qll
View File

@@ -527,7 +527,14 @@ module HTTP {
}
}
/** Provides models for cryptographic things. */
/**
* Provides models for cryptographic things.
*
* Note: The `CryptographicAlgorithm` class currently doesn't take weak keys into
* consideration for the `isWeak` member predicate. So RSA is always considered
* secure, although using a low number of bits will actually make it insecure. We plan
* to improve our libraries in the future to more precisely capture this aspect.
*/
module Cryptography {
/** Provides models for public-key cryptography, also called asymmetric cryptography. */
module PublicKey {
@@ -626,4 +633,43 @@ module Cryptography {
}
}
}
import semmle.python.concepts.CryptoAlgorithms
/**
* A data-flow node that is an application of a cryptographic algorithm. For example,
* encryption, decryption, signature-validation.
*
* Extend this class to refine existing API models. If you want to model new APIs,
* extend `CryptographicOperation::Range` instead.
*/
class CryptographicOperation extends DataFlow::Node {
CryptographicOperation::Range range;
CryptographicOperation() { this = range }
/** Gets the algorithm used, if it matches a known `CryptographicAlgorithm`. */
CryptographicAlgorithm getAlgorithm() { result = range.getAlgorithm() }
/** Gets an input the algorithm is used on, for example the plain text input to be encrypted. */
DataFlow::Node getAnInput() { result = range.getAnInput() }
}
/** Provides classes for modeling new applications of a cryptographic algorithms. */
module CryptographicOperation {
/**
* A data-flow node that is an application of a cryptographic algorithm. For example,
* encryption, decryption, signature-validation.
*
* Extend this class to model new APIs. If you want to refine existing API models,
* extend `CryptographicOperation` instead.
*/
abstract class Range extends DataFlow::Node {
/** Gets the algorithm used, if it matches a known `CryptographicAlgorithm`. */
abstract CryptographicAlgorithm getAlgorithm();
/** Gets an input the algorithm is used on, for example the plain text input to be encrypted. */
abstract DataFlow::Node getAnInput();
}
}
}

8
python/ql/src/semmle/python/Frameworks.qll
View File

@@ -2,17 +2,25 @@
* Helper file that imports all framework modeling.
*/
// If you add modeling of a new framework/library, remember to add it it to the docs in
// `docs/codeql/support/reusables/frameworks.rst`
private import semmle.python.frameworks.Aiohttp
private import semmle.python.frameworks.Cryptodome
private import semmle.python.frameworks.Cryptography
private import semmle.python.frameworks.Dill
private import semmle.python.frameworks.Django
private import semmle.python.frameworks.Fabric
private import semmle.python.frameworks.Flask
private import semmle.python.frameworks.Idna
private import semmle.python.frameworks.Invoke
private import semmle.python.frameworks.Multidict
private import semmle.python.frameworks.MysqlConnectorPython
private import semmle.python.frameworks.MySQLdb
private import semmle.python.frameworks.Psycopg2
private import semmle.python.frameworks.PyMySQL
private import semmle.python.frameworks.Simplejson
private import semmle.python.frameworks.Stdlib
private import semmle.python.frameworks.Tornado
private import semmle.python.frameworks.Ujson
private import semmle.python.frameworks.Yaml
private import semmle.python.frameworks.Yarl

174
python/ql/src/semmle/python/concepts/CryptoAlgorithms.qll Normal file
View File

@@ -0,0 +1,174 @@
/**
* Provides classes modeling cryptographic algorithms, separated into strong and weak variants.
*
* The classification into strong and weak are based on Wikipedia, OWASP and google (2017).
*/
/**
* Names of cryptographic algorithms, separated into strong and weak variants.
*
* The names are normalized: upper-case, no spaces, dashes or underscores.
*
* The names are inspired by the names used in real world crypto libraries.
*
* The classification into strong and weak are based on Wikipedia, OWASP and google (2017).
*/
private module AlgorithmNames {
predicate isStrongHashingAlgorithm(string name) {
name = "DSA" or
name = "ED25519" or
name = "ES256" or
name = "ECDSA256" or
name = "ES384" or
name = "ECDSA384" or
name = "ES512" or
name = "ECDSA512" or
name = "SHA2" or
name = "SHA224" or
name = "SHA256" or
name = "SHA384" or
name = "SHA512" or
name = "SHA3"
}
predicate isWeakHashingAlgorithm(string name) {
name = "HAVEL128" or
name = "MD2" or
name = "MD4" or
name = "MD5" or
name = "PANAMA" or
name = "RIPEMD" or
name = "RIPEMD128" or
name = "RIPEMD256" or
name = "RIPEMD160" or
name = "RIPEMD320" or
name = "SHA0" or
name = "SHA1"
}
predicate isStrongEncryptionAlgorithm(string name) {
name = "AES" or
name = "AES128" or
name = "AES192" or
name = "AES256" or
name = "AES512" or
name = "RSA" or
name = "RABBIT" or
name = "BLOWFISH"
}
predicate isWeakEncryptionAlgorithm(string name) {
name = "DES" or
name = "3DES" or
name = "TRIPLEDES" or
name = "TDEA" or
name = "TRIPLEDEA" or
name = "ARC2" or
name = "RC2" or
name = "ARC4" or
name = "RC4" or
name = "ARCFOUR" or
name = "ARC5" or
name = "RC5"
}
predicate isStrongPasswordHashingAlgorithm(string name) {
name = "ARGON2" or
name = "PBKDF2" or
name = "BCRYPT" or
name = "SCRYPT"
}
predicate isWeakPasswordHashingAlgorithm(string name) { none() }
}
private import AlgorithmNames
/**
* A cryptographic algorithm.
*/
private newtype TCryptographicAlgorithm =
MkHashingAlgorithm(string name, boolean isWeak) {
isStrongHashingAlgorithm(name) and isWeak = false
or
isWeakHashingAlgorithm(name) and isWeak = true
} or
MkEncryptionAlgorithm(string name, boolean isWeak) {
isStrongEncryptionAlgorithm(name) and isWeak = false
or
isWeakEncryptionAlgorithm(name) and isWeak = true
} or
MkPasswordHashingAlgorithm(string name, boolean isWeak) {
isStrongPasswordHashingAlgorithm(name) and isWeak = false
or
isWeakPasswordHashingAlgorithm(name) and isWeak = true
}
/**
* A cryptographic algorithm.
*/
abstract class CryptographicAlgorithm extends TCryptographicAlgorithm {
/** Gets a textual representation of this element. */
string toString() { result = getName() }
/**
* Gets the normalized name of this algorithm (upper-case, no spaces, dashes or underscores).
*/
abstract string getName();
/**
* Holds if the name of this algorithm matches `name` modulo case,
* white space, dashes, and underscores.
*/
bindingset[name]
predicate matchesName(string name) {
name.toUpperCase().regexpReplaceAll("[-_ ]", "") = getName()
}
/**
* Holds if this algorithm is weak.
*/
abstract predicate isWeak();
}
/**
* A hashing algorithm such as `MD5` or `SHA512`.
*/
class HashingAlgorithm extends MkHashingAlgorithm, CryptographicAlgorithm {
string name;
boolean isWeak;
HashingAlgorithm() { this = MkHashingAlgorithm(name, isWeak) }
override string getName() { result = name }
override predicate isWeak() { isWeak = true }
}
/**
* An encryption algorithm such as `DES` or `AES512`.
*/
class EncryptionAlgorithm extends MkEncryptionAlgorithm, CryptographicAlgorithm {
string name;
boolean isWeak;
EncryptionAlgorithm() { this = MkEncryptionAlgorithm(name, isWeak) }
override string getName() { result = name }
override predicate isWeak() { isWeak = true }
}
/**
* A password hashing algorithm such as `PBKDF2` or `SCRYPT`.
*/
class PasswordHashingAlgorithm extends MkPasswordHashingAlgorithm, CryptographicAlgorithm {
string name;
boolean isWeak;
PasswordHashingAlgorithm() { this = MkPasswordHashingAlgorithm(name, isWeak) }
override string getName() { result = name }
override predicate isWeak() { isWeak = true }
}

66
python/ql/src/semmle/python/dataflow/new/SensitiveDataSources.qll Normal file
View File

@@ -0,0 +1,66 @@
/**
* Provides an extension point for for modeling sensitive data, such as secrets, certificates, or passwords.
* Sensitive data can be interesting to use as data-flow sources in security queries.
*/
private import python
private import semmle.python.dataflow.new.DataFlow
// Need to import since frameworks can extend `RemoteFlowSource::Range`
private import semmle.python.Frameworks
private import semmle.python.Concepts
private import semmle.python.security.SensitiveData as OldSensitiveData
/**
* A data flow source of sensitive data, such as secrets, certificates, or passwords.
*
* Extend this class to refine existing API models. If you want to model new APIs,
* extend `SensitiveDataSource::Range` instead.
*/
class SensitiveDataSource extends DataFlow::Node {
SensitiveDataSource::Range range;
SensitiveDataSource() { this = range }
/**
* INTERNAL: Do not use.
*
* This will be rewritten to have better types soon, and therefore should only be used internally until then.
*
* Gets the classification of the sensitive data.
*/
string getClassification() { result = range.getClassification() }
}
/** Provides a class for modeling new sources of sensitive data, such as secrets, certificates, or passwords. */
module SensitiveDataSource {
/**
* A data flow source of sensitive data, such as secrets, certificates, or passwords.
*
* Extend this class to model new APIs. If you want to refine existing API models,
* extend `SensitiveDataSource` instead.
*/
abstract class Range extends DataFlow::Node {
/**
* INTERNAL: Do not use.
*
* This will be rewritten to have better types soon, and therefore should only be used internally until then.
*
* Gets the classification of the sensitive data.
*/
abstract string getClassification();
}
}
private class PortOfOldModeling extends SensitiveDataSource::Range {
OldSensitiveData::SensitiveData::Source oldSensitiveSource;
PortOfOldModeling() { this.asCfgNode() = oldSensitiveSource }
override string getClassification() {
exists(OldSensitiveData::SensitiveData classification |
oldSensitiveSource.isSourceOf(classification)
|
classification = "sensitive.data." + result
)
}
}

432
python/ql/src/semmle/python/dataflow/new/TypeTracker.qll
View File

@@ -1,167 +1,16 @@
/** Step Summaries and Type Tracking */
/**
* This file acts as a wrapper for `internal.TypeTracker`, exposing some of the functionality with
* names that are more appropriate for Python.
*/
private import python
private import internal.DataFlowPublic
private import internal.DataFlowPrivate
private import internal.TypeTracker as Internal
/** Any string that may appear as the name of an attribute or access path. */
class AttributeName extends string {
AttributeName() { this = any(AttrRef a).getAttributeName() }
}
class AttributeName = Internal::ContentName;
/** Either an attribute name, or the empty string (representing no attribute). */
class OptionalAttributeName extends string {
OptionalAttributeName() { this instanceof AttributeName or this = "" }
}
/**
* A description of a step on an inter-procedural data flow path.
*/
private newtype TStepSummary =
LevelStep() or
CallStep() or
ReturnStep() or
StoreStep(AttributeName attr) or
LoadStep(AttributeName attr)
/**
* INTERNAL: Use `TypeTracker` or `TypeBackTracker` instead.
*
* A description of a step on an inter-procedural data flow path.
*/
class StepSummary extends TStepSummary {
/** Gets a textual representation of this step summary. */
string toString() {
this instanceof LevelStep and result = "level"
or
this instanceof CallStep and result = "call"
or
this instanceof ReturnStep and result = "return"
or
exists(string attr | this = StoreStep(attr) | result = "store " + attr)
or
exists(string attr | this = LoadStep(attr) | result = "load " + attr)
}
}
/** Provides predicates for updating step summaries (`StepSummary`s). */
module StepSummary {
/**
* Gets the summary that corresponds to having taken a forwards
* heap and/or inter-procedural step from `nodeFrom` to `nodeTo`.
*/
cached
predicate step(LocalSourceNode nodeFrom, LocalSourceNode nodeTo, StepSummary summary) {
exists(Node mid | nodeFrom.flowsTo(mid) and smallstep(mid, nodeTo, summary))
}
/**
* Gets the summary that corresponds to having taken a forwards
* local, heap and/or inter-procedural step from `nodeFrom` to `nodeTo`.
*
* Unlike `StepSummary::step`, this predicate does not compress
* type-preserving steps.
*/
predicate smallstep(Node nodeFrom, Node nodeTo, StepSummary summary) {
jumpStep(nodeFrom, nodeTo) and
summary = LevelStep()
or
callStep(nodeFrom, nodeTo) and summary = CallStep()
or
returnStep(nodeFrom, nodeTo) and
summary = ReturnStep()
or
exists(string attr |
basicStoreStep(nodeFrom, nodeTo, attr) and
summary = StoreStep(attr)
or
basicLoadStep(nodeFrom, nodeTo, attr) and summary = LoadStep(attr)
)
}
}
/**
* Gets a callable for the call where `nodeFrom` is used as the `i`'th argument.
*
* Helper predicate to avoid bad join order experienced in `callStep`.
* This happened when `isParameterOf` was joined _before_ `getCallable`.
*/
pragma[nomagic]
private DataFlowCallable getCallableForArgument(ArgumentNode nodeFrom, int i) {
exists(DataFlowCall call |
nodeFrom.argumentOf(call, i) and
result = call.getCallable()
)
}
/** Holds if `nodeFrom` steps to `nodeTo` by being passed as a parameter in a call. */
predicate callStep(ArgumentNode nodeFrom, ParameterNode nodeTo) {
// TODO: Support special methods?
exists(DataFlowCallable callable, int i |
callable = getCallableForArgument(nodeFrom, i) and
nodeTo.isParameterOf(callable, i)
)
}
/** Holds if `nodeFrom` steps to `nodeTo` by being returned from a call. */
predicate returnStep(ReturnNode nodeFrom, Node nodeTo) {
exists(DataFlowCall call |
nodeFrom.getEnclosingCallable() = call.getCallable() and nodeTo.asCfgNode() = call.getNode()
)
}
/**
* Holds if `nodeFrom` is being written to the `attr` attribute of the object in `nodeTo`.
*
* Note that the choice of `nodeTo` does not have to make sense "chronologically".
* All we care about is whether the `attr` attribute of `nodeTo` can have a specific type,
* and the assumption is that if a specific type appears here, then any access of that
* particular attribute can yield something of that particular type.
*
* Thus, in an example such as
*
* ```python
* def foo(y):
* x = Foo()
* bar(x)
* x.attr = y
* baz(x)
*
* def bar(x):
* z = x.attr
* ```
* for the attribute write `x.attr = y`, we will have `attr` being the literal string `"attr"`,
* `nodeFrom` will be `y`, and `nodeTo` will be the object `Foo()` created on the first line of the
* function. This means we will track the fact that `x.attr` can have the type of `y` into the
* assignment to `z` inside `bar`, even though this attribute write happens _after_ `bar` is called.
*/
predicate basicStoreStep(Node nodeFrom, LocalSourceNode nodeTo, string attr) {
exists(AttrWrite a |
a.mayHaveAttributeName(attr) and
nodeFrom = a.getValue() and
nodeTo.flowsTo(a.getObject())
)
}
/**
* Holds if `nodeTo` is the result of accessing the `attr` attribute of `nodeFrom`.
*/
predicate basicLoadStep(Node nodeFrom, Node nodeTo, string attr) {
exists(AttrRead a |
a.mayHaveAttributeName(attr) and
nodeFrom = a.getObject() and
nodeTo = a
)
}
/**
* A utility class that is equivalent to `boolean` but does not require type joining.
*/
private class Boolean extends boolean {
Boolean() { this = true or this = false }
}
private newtype TTypeTracker = MkTypeTracker(Boolean hasCall, OptionalAttributeName attr)
class OptionalAttributeName = Internal::OptionalContentName;
/**
* Summary of the steps needed to track a value to a given dataflow node.
@@ -173,8 +22,8 @@ private newtype TTypeTracker = MkTypeTracker(Boolean hasCall, OptionalAttributeN
*
* It is recommended that all uses of this type are written in the following form,
* for tracking some type `myType`:
* ```
* private DataFlow::LocalSourceNode myType(DataFlow::TypeTracker t) {
* ```ql
* DataFlow::LocalSourceNode myType(DataFlow::TypeTracker t) {
* t.start() and
* result = < source of myType >
* or
@@ -183,279 +32,34 @@ private newtype TTypeTracker = MkTypeTracker(Boolean hasCall, OptionalAttributeN
* )
* }
*
* DataFlow::Node myType() { myType(DataFlow::TypeTracker::end()).flowsTo(result) }
* DataFlow::LocalSourceNode myType() { myType(DataFlow::TypeTracker::end()) }
* ```
*
* Instead of `result = myType(t2).track(t2, t)`, you can also use the equivalent
* `t = t2.step(myType(t2), result)`. If you additionally want to track individual
* intra-procedural steps, use `t = t2.smallstep(myCallback(t2), result)`.
*/
class TypeTracker extends TTypeTracker {
Boolean hasCall;
OptionalAttributeName attr;
TypeTracker() { this = MkTypeTracker(hasCall, attr) }
/** Gets the summary resulting from appending `step` to this type-tracking summary. */
cached
TypeTracker append(StepSummary step) {
step = LevelStep() and result = this
or
step = CallStep() and result = MkTypeTracker(true, attr)
or
step = ReturnStep() and hasCall = false and result = this
or
step = LoadStep(attr) and result = MkTypeTracker(hasCall, "")
or
exists(string p | step = StoreStep(p) and attr = "" and result = MkTypeTracker(hasCall, p))
}
/** Gets a textual representation of this summary. */
string toString() {
exists(string withCall, string withAttr |
(if hasCall = true then withCall = "with" else withCall = "without") and
(if attr != "" then withAttr = " with attribute " + attr else withAttr = "") and
result = "type tracker " + withCall + " call steps" + withAttr
)
}
/**
* Holds if this is the starting point of type tracking.
*/
predicate start() { hasCall = false and attr = "" }
class TypeTracker extends Internal::TypeTracker {
/**
* Holds if this is the starting point of type tracking, and the value starts in the attribute named `attrName`.
* The type tracking only ends after the attribute has been loaded.
*/
predicate startInAttr(AttributeName attrName) { hasCall = false and attr = attrName }
/**
* Holds if this is the starting point of type tracking
* when tracking a parameter into a call, but not out of it.
*/
predicate call() { hasCall = true and attr = "" }
/**
* Holds if this is the end point of type tracking.
*/
predicate end() { attr = "" }
/**
* INTERNAL. DO NOT USE.
*
* Holds if this type has been tracked into a call.
*/
boolean hasCall() { result = hasCall }
predicate startInAttr(string attrName) { this.startInContent(attrName) }
/**
* INTERNAL. DO NOT USE.
*
* Gets the attribute associated with this type tracker.
*/
string getAttr() { result = attr }
/**
* Gets a type tracker that starts where this one has left off to allow continued
* tracking.
*
* This predicate is only defined if the type has not been tracked into an attribute.
*/
TypeTracker continue() { attr = "" and result = this }
/**
* Gets the summary that corresponds to having taken a forwards
* heap and/or inter-procedural step from `nodeFrom` to `nodeTo`.
*/
pragma[inline]
TypeTracker step(LocalSourceNode nodeFrom, LocalSourceNode nodeTo) {
exists(StepSummary summary |
StepSummary::step(nodeFrom, pragma[only_bind_out](nodeTo), pragma[only_bind_into](summary)) and
result = this.append(pragma[only_bind_into](summary))
)
}
/**
* Gets the summary that corresponds to having taken a forwards
* local, heap and/or inter-procedural step from `nodeFrom` to `nodeTo`.
*
* Unlike `TypeTracker::step`, this predicate exposes all edges
* in the flow graph, and not just the edges between `Node`s.
* It may therefore be less performant.
*
* Type tracking predicates using small steps typically take the following form:
* ```ql
* DataFlow::Node myType(DataFlow::TypeTracker t) {
* t.start() and
* result = < source of myType >
* or
* exists (DataFlow::TypeTracker t2 |
* t = t2.smallstep(myType(t2), result)
* )
* }
*
* DataFlow::Node myType() {
* result = myType(DataFlow::TypeTracker::end())
* }
* ```
*/
pragma[inline]
TypeTracker smallstep(Node nodeFrom, Node nodeTo) {
exists(StepSummary summary |
StepSummary::smallstep(nodeFrom, nodeTo, summary) and
result = this.append(summary)
)
or
simpleLocalFlowStep(nodeFrom, nodeTo) and
result = this
}
string getAttr() { result = this.getContent() }
}
/** Provides predicates for implementing custom `TypeTracker`s. */
module TypeTracker {
/**
* Gets a valid end point of type tracking.
*/
TypeTracker end() { result.end() }
}
module TypeTracker = Internal::TypeTracker;
private newtype TTypeBackTracker = MkTypeBackTracker(Boolean hasReturn, OptionalAttributeName attr)
class StepSummary = Internal::StepSummary;
/**
* Summary of the steps needed to back-track a use of a value to a given dataflow node.
*
* This can for example be used to track callbacks that are passed to a certain API,
* so we can model specific parameters of that callback as having a certain type.
*
* Note that type back-tracking does not provide a source/sink relation, that is,
* it may determine that a node will be used in an API call somewhere, but it won't
* determine exactly where that use was, or the path that led to the use.
*
* It is recommended that all uses of this type are written in the following form,
* for back-tracking some callback type `myCallback`:
*
* ```
* private DataFlow::LocalSourceNode myCallback(DataFlow::TypeBackTracker t) {
* t.start() and
* result = (< some API call >).getArgument(< n >).getALocalSource()
* or
* exists (DataFlow::TypeBackTracker t2 |
* result = myCallback(t2).backtrack(t2, t)
* )
* }
*
* DataFlow::LocalSourceNode myCallback() { result = myCallback(DataFlow::TypeBackTracker::end()) }
* ```
*
* Instead of `result = myCallback(t2).backtrack(t2, t)`, you can also use the equivalent
* `t2 = t.step(result, myCallback(t2))`. If you additionally want to track individual
* intra-procedural steps, use `t2 = t.smallstep(result, myCallback(t2))`.
*/
class TypeBackTracker extends TTypeBackTracker {
Boolean hasReturn;
string attr;
module StepSummary = Internal::StepSummary;
TypeBackTracker() { this = MkTypeBackTracker(hasReturn, attr) }
class TypeBackTracker = Internal::TypeBackTracker;
/** Gets the summary resulting from prepending `step` to this type-tracking summary. */
TypeBackTracker prepend(StepSummary step) {
step = LevelStep() and result = this
or
step = CallStep() and hasReturn = false and result = this
or
step = ReturnStep() and result = MkTypeBackTracker(true, attr)
or
exists(string p | step = LoadStep(p) and attr = "" and result = MkTypeBackTracker(hasReturn, p))
or
step = StoreStep(attr) and result = MkTypeBackTracker(hasReturn, "")
}
/** Gets a textual representation of this summary. */
string toString() {
exists(string withReturn, string withAttr |
(if hasReturn = true then withReturn = "with" else withReturn = "without") and
(if attr != "" then withAttr = " with attribute " + attr else withAttr = "") and
result = "type back-tracker " + withReturn + " return steps" + withAttr
)
}
/**
* Holds if this is the starting point of type tracking.
*/
predicate start() { hasReturn = false and attr = "" }
/**
* Holds if this is the end point of type tracking.
*/
predicate end() { attr = "" }
/**
* INTERNAL. DO NOT USE.
*
* Holds if this type has been back-tracked into a call through return edge.
*/
boolean hasReturn() { result = hasReturn }
/**
* Gets a type tracker that starts where this one has left off to allow continued
* tracking.
*
* This predicate is only defined if the type has not been tracked into an attribute.
*/
TypeBackTracker continue() { attr = "" and result = this }
/**
* Gets the summary that corresponds to having taken a backwards
* heap and/or inter-procedural step from `nodeTo` to `nodeFrom`.
*/
pragma[inline]
TypeBackTracker step(LocalSourceNode nodeFrom, LocalSourceNode nodeTo) {
exists(StepSummary summary |
StepSummary::step(pragma[only_bind_out](nodeFrom), nodeTo, pragma[only_bind_into](summary)) and
this = result.prepend(pragma[only_bind_into](summary))
)
}
/**
* Gets the summary that corresponds to having taken a backwards
* local, heap and/or inter-procedural step from `nodeTo` to `nodeFrom`.
*
* Unlike `TypeBackTracker::step`, this predicate exposes all edges
* in the flowgraph, and not just the edges between
* `LocalSourceNode`s. It may therefore be less performant.
*
* Type tracking predicates using small steps typically take the following form:
* ```ql
* DataFlow::Node myType(DataFlow::TypeBackTracker t) {
* t.start() and
* result = < some API call >.getArgument(< n >)
* or
* exists (DataFlow::TypeBackTracker t2 |
* t = t2.smallstep(result, myType(t2))
* )
* }
*
* DataFlow::Node myType() {
* result = myType(DataFlow::TypeBackTracker::end())
* }
* ```
*/
pragma[inline]
TypeBackTracker smallstep(Node nodeFrom, Node nodeTo) {
exists(StepSummary summary |
StepSummary::smallstep(nodeFrom, nodeTo, summary) and
this = result.prepend(summary)
)
or
simpleLocalFlowStep(nodeFrom, nodeTo) and
this = result
}
}
/** Provides predicates for implementing custom `TypeBackTracker`s. */
module TypeBackTracker {
/**
* Gets a valid end point of type back-tracking.
*/
TypeBackTracker end() { result.end() }
}
module TypeBackTracker = Internal::TypeBackTracker;

476
python/ql/src/semmle/python/dataflow/new/internal/DataFlowImpl.qll
View File

@@ -211,10 +211,7 @@ private predicate fullBarrier(Node node, Configuration config) {
* Holds if data can flow in one local step from `node1` to `node2`.
*/
private predicate localFlowStep(Node node1, Node node2, Configuration config) {
(
simpleLocalFlowStep(node1, node2) or
reverseStepThroughInputOutputAlias(node1, node2)
) and
simpleLocalFlowStepExt(node1, node2) and
not outBarrier(node1, config) and
not inBarrier(node2, config) and
not fullBarrier(node1, config) and
@@ -237,7 +234,7 @@ private predicate additionalLocalFlowStep(Node node1, Node node2, Configuration
* Holds if data can flow from `node1` to `node2` in a way that discards call contexts.
*/
private predicate jumpStep(Node node1, Node node2, Configuration config) {
jumpStep(node1, node2) and
jumpStepCached(node1, node2) and
not outBarrier(node1, config) and
not inBarrier(node2, config) and
not fullBarrier(node1, config) and
@@ -388,7 +385,7 @@ private module Stage1 {
*/
pragma[nomagic]
private predicate fwdFlowIsEntered(DataFlowCall call, Cc cc, Configuration config) {
exists(ArgumentNode arg |
exists(ArgNode arg |
fwdFlow(arg, cc, config) and
viableParamArg(call, _, arg)
)
@@ -515,29 +512,29 @@ private module Stage1 {
pragma[nomagic]
predicate viableParamArgNodeCandFwd1(
DataFlowCall call, ParameterNode p, ArgumentNode arg, Configuration config
DataFlowCall call, ParamNode p, ArgNode arg, Configuration config
) {
viableParamArg(call, p, arg) and
fwdFlow(arg, config)
}
pragma[nomagic]
private predicate revFlowIn(
DataFlowCall call, ArgumentNode arg, boolean toReturn, Configuration config
) {
exists(ParameterNode p |
private predicate revFlowIn(DataFlowCall call, ArgNode arg, boolean toReturn, Configuration config) {
exists(ParamNode p |
revFlow(p, toReturn, config) and
viableParamArgNodeCandFwd1(call, p, arg, config)
)
}
pragma[nomagic]
private predicate revFlowInToReturn(DataFlowCall call, ArgumentNode arg, Configuration config) {
private predicate revFlowInToReturn(DataFlowCall call, ArgNode arg, Configuration config) {
revFlowIn(call, arg, true, config)
}
/**
* Holds if an output from `call` is reached in the flow covered by `revFlow`.
* Holds if an output from `call` is reached in the flow covered by `revFlow`
* and data might flow through the target callable resulting in reverse flow
* reaching an argument of `call`.
*/
pragma[nomagic]
private predicate revFlowIsReturned(DataFlowCall call, boolean toReturn, Configuration config) {
@@ -597,7 +594,7 @@ private module Stage1 {
* Holds if flow may enter through `p` and reach a return node making `p` a
* candidate for the origin of a summary.
*/
predicate parameterMayFlowThrough(ParameterNode p, DataFlowCallable c, Ap ap, Configuration config) {
predicate parameterMayFlowThrough(ParamNode p, DataFlowCallable c, Ap ap, Configuration config) {
exists(ReturnKindExt kind |
throughFlowNodeCand(p, config) and
returnFlowCallableNodeCand(c, kind, config) and
@@ -610,6 +607,15 @@ private module Stage1 {
)
}
pragma[nomagic]
predicate callMayFlowThroughRev(DataFlowCall call, Configuration config) {
exists(ArgNode arg, boolean toReturn |
revFlow(arg, toReturn, config) and
revFlowInToReturn(call, arg, config) and
revFlowIsReturned(call, toReturn, config)
)
}
predicate stats(boolean fwd, int nodes, int fields, int conscand, int tuples, Configuration config) {
fwd = true and
nodes = count(Node node | fwdFlow(node, config)) and
@@ -663,7 +669,7 @@ private predicate flowOutOfCallNodeCand1(
pragma[nomagic]
private predicate viableParamArgNodeCand1(
DataFlowCall call, ParameterNode p, ArgumentNode arg, Configuration config
DataFlowCall call, ParamNode p, ArgNode arg, Configuration config
) {
Stage1::viableParamArgNodeCandFwd1(call, p, arg, config) and
Stage1::revFlow(arg, config)
@@ -675,7 +681,7 @@ private predicate viableParamArgNodeCand1(
*/
pragma[nomagic]
private predicate flowIntoCallNodeCand1(
DataFlowCall call, ArgumentNode arg, ParameterNode p, Configuration config
DataFlowCall call, ArgNode arg, ParamNode p, Configuration config
) {
viableParamArgNodeCand1(call, p, arg, config) and
Stage1::revFlow(p, config) and
@@ -735,8 +741,7 @@ private predicate flowOutOfCallNodeCand1(
*/
pragma[nomagic]
private predicate flowIntoCallNodeCand1(
DataFlowCall call, ArgumentNode arg, ParameterNode p, boolean allowsFieldFlow,
Configuration config
DataFlowCall call, ArgNode arg, ParamNode p, boolean allowsFieldFlow, Configuration config
) {
flowIntoCallNodeCand1(call, arg, p, config) and
exists(int b, int j |
@@ -833,6 +838,16 @@ private module Stage2 {
PrevStage::revFlow(node, _, _, apa, config)
}
pragma[nomagic]
private predicate flowThroughOutOfCall(
DataFlowCall call, ReturnNodeExt ret, Node out, boolean allowsFieldFlow, Configuration config
) {
flowOutOfCall(call, ret, out, allowsFieldFlow, pragma[only_bind_into](config)) and
PrevStage::callMayFlowThroughRev(call, pragma[only_bind_into](config)) and
PrevStage::parameterMayFlowThrough(_, getNodeEnclosingCallable(ret), _,
pragma[only_bind_into](config))
}
/**
* Holds if `node` is reachable with access path `ap` from a source in the
* configuration `config`.
@@ -899,13 +914,11 @@ private module Stage2 {
)
or
// flow out of a callable
exists(DataFlowCall call |
fwdFlowOut(call, node, any(CcNoCall innercc), cc, argAp, ap, config)
or
exists(Ap argAp0 |
fwdFlowOutFromArg(call, node, argAp0, ap, config) and
fwdFlowIsEntered(call, cc, argAp, argAp0, config)
)
fwdFlowOutNotFromArg(node, cc, argAp, ap, config)
or
exists(DataFlowCall call, Ap argAp0 |
fwdFlowOutFromArg(call, node, argAp0, ap, config) and
fwdFlowIsEntered(call, cc, argAp, argAp0, config)
)
}
@@ -944,10 +957,10 @@ private module Stage2 {
pragma[nomagic]
private predicate fwdFlowIn(
DataFlowCall call, ParameterNode p, Cc outercc, Cc innercc, ApOption argAp, Ap ap,
DataFlowCall call, ParamNode p, Cc outercc, Cc innercc, ApOption argAp, Ap ap,
Configuration config
) {
exists(ArgumentNode arg, boolean allowsFieldFlow |
exists(ArgNode arg, boolean allowsFieldFlow |
fwdFlow(arg, outercc, argAp, ap, config) and
flowIntoCall(call, arg, p, allowsFieldFlow, config) and
innercc = getCallContextCall(call, getNodeEnclosingCallable(p), outercc)
@@ -956,17 +969,14 @@ private module Stage2 {
)
}
/**
* Holds if flow may exit from `call` at `out` with access path `ap`. The
* inner call context is `innercc`, but `ccOut` is just the call context
* based on the return step. In the case of through-flow `ccOut` is discarded
* and replaced by the outer call context as tracked by `fwdFlowIsEntered`.
*/
pragma[nomagic]
private predicate fwdFlowOut(
DataFlowCall call, Node out, Cc innercc, Cc ccOut, ApOption argAp, Ap ap, Configuration config
private predicate fwdFlowOutNotFromArg(
Node out, Cc ccOut, ApOption argAp, Ap ap, Configuration config
) {
exists(ReturnNodeExt ret, boolean allowsFieldFlow, DataFlowCallable inner |
exists(
DataFlowCall call, ReturnNodeExt ret, boolean allowsFieldFlow, CcNoCall innercc,
DataFlowCallable inner
|
fwdFlow(ret, innercc, argAp, ap, config) and
flowOutOfCall(call, ret, out, allowsFieldFlow, config) and
inner = getNodeEnclosingCallable(ret) and
@@ -981,7 +991,13 @@ private module Stage2 {
private predicate fwdFlowOutFromArg(
DataFlowCall call, Node out, Ap argAp, Ap ap, Configuration config
) {
fwdFlowOut(call, out, any(CcCall ccc), _, apSome(argAp), ap, config)
exists(ReturnNodeExt ret, boolean allowsFieldFlow, CcCall ccc |
fwdFlow(ret, ccc, apSome(argAp), ap, config) and
flowThroughOutOfCall(call, ret, out, allowsFieldFlow, config) and
ccc.matchesCall(call)
|
ap instanceof ApNil or allowsFieldFlow = true
)
}
/**
@@ -992,7 +1008,7 @@ private module Stage2 {
private predicate fwdFlowIsEntered(
DataFlowCall call, Cc cc, ApOption argAp, Ap ap, Configuration config
) {
exists(ParameterNode p |
exists(ParamNode p |
fwdFlowIn(call, p, cc, _, argAp, ap, config) and
PrevStage::parameterMayFlowThrough(p, _, getApprox(ap), config)
)
@@ -1012,6 +1028,27 @@ private module Stage2 {
fwdFlowConsCand(ap1, c, ap2, config)
}
pragma[nomagic]
private predicate callMayFlowThroughFwd(DataFlowCall call, Configuration config) {
exists(Ap argAp0, Node out, Cc cc, ApOption argAp, Ap ap |
fwdFlow(out, pragma[only_bind_into](cc), pragma[only_bind_into](argAp), ap,
pragma[only_bind_into](config)) and
fwdFlowOutFromArg(call, out, argAp0, ap, config) and
fwdFlowIsEntered(call, pragma[only_bind_into](cc), pragma[only_bind_into](argAp), argAp0,
pragma[only_bind_into](config))
)
}
pragma[nomagic]
private predicate flowThroughIntoCall(
DataFlowCall call, ArgNode arg, ParamNode p, boolean allowsFieldFlow, Configuration config
) {
flowIntoCall(call, arg, p, allowsFieldFlow, config) and
fwdFlow(arg, _, _, _, pragma[only_bind_into](config)) and
PrevStage::parameterMayFlowThrough(p, _, _, pragma[only_bind_into](config)) and
callMayFlowThroughFwd(call, pragma[only_bind_into](config))
}
/**
* Holds if `node` with access path `ap` is part of a path from a source to a
* sink in the configuration `config`.
@@ -1077,14 +1114,12 @@ private module Stage2 {
)
or
// flow into a callable
exists(DataFlowCall call |
revFlowIn(call, node, toReturn, returnAp, ap, config) and
toReturn = false
or
exists(Ap returnAp0 |
revFlowInToReturn(call, node, returnAp0, ap, config) and
revFlowIsReturned(call, toReturn, returnAp, returnAp0, config)
)
revFlowInNotToReturn(node, returnAp, ap, config) and
toReturn = false
or
exists(DataFlowCall call, Ap returnAp0 |
revFlowInToReturn(call, node, returnAp0, ap, config) and
revFlowIsReturned(call, toReturn, returnAp, returnAp0, config)
)
or
// flow out of a callable
@@ -1132,13 +1167,10 @@ private module Stage2 {
}
pragma[nomagic]
private predicate revFlowIn(
DataFlowCall call, ArgumentNode arg, boolean toReturn, ApOption returnAp, Ap ap,
Configuration config
) {
exists(ParameterNode p, boolean allowsFieldFlow |
revFlow(p, toReturn, returnAp, ap, config) and
flowIntoCall(call, arg, p, allowsFieldFlow, config)
private predicate revFlowInNotToReturn(ArgNode arg, ApOption returnAp, Ap ap, Configuration config) {
exists(ParamNode p, boolean allowsFieldFlow |
revFlow(p, false, returnAp, ap, config) and
flowIntoCall(_, arg, p, allowsFieldFlow, config)
|
ap instanceof ApNil or allowsFieldFlow = true
)
@@ -1146,9 +1178,14 @@ private module Stage2 {
pragma[nomagic]
private predicate revFlowInToReturn(
DataFlowCall call, ArgumentNode arg, Ap returnAp, Ap ap, Configuration config
DataFlowCall call, ArgNode arg, Ap returnAp, Ap ap, Configuration config
) {
revFlowIn(call, arg, true, apSome(returnAp), ap, config)
exists(ParamNode p, boolean allowsFieldFlow |
revFlow(p, true, apSome(returnAp), ap, config) and
flowThroughIntoCall(call, arg, p, allowsFieldFlow, config)
|
ap instanceof ApNil or allowsFieldFlow = true
)
}
/**
@@ -1199,13 +1236,13 @@ private module Stage2 {
pragma[noinline]
private predicate parameterFlow(
ParameterNode p, Ap ap, Ap ap0, DataFlowCallable c, Configuration config
ParamNode p, Ap ap, Ap ap0, DataFlowCallable c, Configuration config
) {
revFlow(p, true, apSome(ap0), ap, config) and
c = getNodeEnclosingCallable(p)
}
predicate parameterMayFlowThrough(ParameterNode p, DataFlowCallable c, Ap ap, Configuration config) {
predicate parameterMayFlowThrough(ParamNode p, DataFlowCallable c, Ap ap, Configuration config) {
exists(ReturnNodeExt ret, Ap ap0, ReturnKindExt kind, int pos |
parameterFlow(p, ap, ap0, c, config) and
c = getNodeEnclosingCallable(ret) and
@@ -1218,6 +1255,15 @@ private module Stage2 {
)
}
pragma[nomagic]
predicate callMayFlowThroughRev(DataFlowCall call, Configuration config) {
exists(Ap returnAp0, ArgNode arg, boolean toReturn, ApOption returnAp, Ap ap |
revFlow(arg, toReturn, returnAp, ap, config) and
revFlowInToReturn(call, arg, returnAp0, ap, config) and
revFlowIsReturned(call, toReturn, returnAp, returnAp0, config)
)
}
predicate stats(boolean fwd, int nodes, int fields, int conscand, int tuples, Configuration config) {
fwd = true and
nodes = count(Node node | fwdFlow(node, _, _, _, config)) and
@@ -1245,8 +1291,7 @@ private predicate flowOutOfCallNodeCand2(
pragma[nomagic]
private predicate flowIntoCallNodeCand2(
DataFlowCall call, ArgumentNode node1, ParameterNode node2, boolean allowsFieldFlow,
Configuration config
DataFlowCall call, ArgNode node1, ParamNode node2, boolean allowsFieldFlow, Configuration config
) {
flowIntoCallNodeCand1(call, node1, node2, allowsFieldFlow, config) and
Stage2::revFlow(node2, pragma[only_bind_into](config)) and
@@ -1260,8 +1305,8 @@ private module LocalFlowBigStep {
*/
private class FlowCheckNode extends Node {
FlowCheckNode() {
this instanceof CastNode or
clearsContent(this, _)
castNode(this) or
clearsContentCached(this, _)
}
}
@@ -1275,7 +1320,7 @@ private module LocalFlowBigStep {
config.isSource(node) or
jumpStep(_, node, config) or
additionalJumpStep(_, node, config) or
node instanceof ParameterNode or
node instanceof ParamNode or
node instanceof OutNodeExt or
store(_, _, node, _) or
read(_, _, node) or
@@ -1321,21 +1366,21 @@ private module LocalFlowBigStep {
Node node1, Node node2, boolean preservesValue, DataFlowType t, Configuration config,
LocalCallContext cc
) {
not isUnreachableInCall(node2, cc.(LocalCallContextSpecificCall).getCall()) and
not isUnreachableInCallCached(node2, cc.(LocalCallContextSpecificCall).getCall()) and
(
localFlowEntry(node1, pragma[only_bind_into](config)) and
(
localFlowStepNodeCand1(node1, node2, config) and
preservesValue = true and
t = getNodeType(node1)
t = getNodeDataFlowType(node1)
or
additionalLocalFlowStepNodeCand2(node1, node2, config) and
preservesValue = false and
t = getNodeType(node2)
t = getNodeDataFlowType(node2)
) and
node1 != node2 and
cc.relevantFor(getNodeEnclosingCallable(node1)) and
not isUnreachableInCall(node1, cc.(LocalCallContextSpecificCall).getCall()) and
not isUnreachableInCallCached(node1, cc.(LocalCallContextSpecificCall).getCall()) and
Stage2::revFlow(node2, pragma[only_bind_into](config))
or
exists(Node mid |
@@ -1350,7 +1395,7 @@ private module LocalFlowBigStep {
additionalLocalFlowStepNodeCand2(mid, node2, config) and
not mid instanceof FlowCheckNode and
preservesValue = false and
t = getNodeType(node2) and
t = getNodeDataFlowType(node2) and
Stage2::revFlow(node2, pragma[only_bind_into](config))
)
)
@@ -1384,7 +1429,7 @@ private module Stage3 {
private ApApprox getApprox(Ap ap) { result = ap.toBoolNonEmpty() }
private ApNil getApNil(Node node) {
PrevStage::revFlow(node, _) and result = TFrontNil(getNodeType(node))
PrevStage::revFlow(node, _) and result = TFrontNil(getNodeDataFlowType(node))
}
bindingset[tc, tail]
@@ -1443,7 +1488,9 @@ private module Stage3 {
bindingset[node, ap]
private predicate filter(Node node, Ap ap) {
not ap.isClearedAt(node) and
if node instanceof CastingNode then compatibleTypes(getNodeType(node), ap.getType()) else any()
if node instanceof CastingNode
then compatibleTypes(getNodeDataFlowType(node), ap.getType())
else any()
}
bindingset[ap, contentType]
@@ -1465,6 +1512,16 @@ private module Stage3 {
)
}
pragma[nomagic]
private predicate flowThroughOutOfCall(
DataFlowCall call, ReturnNodeExt ret, Node out, boolean allowsFieldFlow, Configuration config
) {
flowOutOfCall(call, ret, out, allowsFieldFlow, pragma[only_bind_into](config)) and
PrevStage::callMayFlowThroughRev(call, pragma[only_bind_into](config)) and
PrevStage::parameterMayFlowThrough(_, getNodeEnclosingCallable(ret), _,
pragma[only_bind_into](config))
}
/**
* Holds if `node` is reachable with access path `ap` from a source in the
* configuration `config`.
@@ -1538,13 +1595,11 @@ private module Stage3 {
)
or
// flow out of a callable
exists(DataFlowCall call |
fwdFlowOut(call, node, any(CcNoCall innercc), cc, argAp, ap, config)
or
exists(Ap argAp0 |
fwdFlowOutFromArg(call, node, argAp0, ap, config) and
fwdFlowIsEntered(call, cc, argAp, argAp0, config)
)
fwdFlowOutNotFromArg(node, cc, argAp, ap, config)
or
exists(DataFlowCall call, Ap argAp0 |
fwdFlowOutFromArg(call, node, argAp0, ap, config) and
fwdFlowIsEntered(call, cc, argAp, argAp0, config)
)
}
@@ -1583,10 +1638,10 @@ private module Stage3 {
pragma[nomagic]
private predicate fwdFlowIn(
DataFlowCall call, ParameterNode p, Cc outercc, Cc innercc, ApOption argAp, Ap ap,
DataFlowCall call, ParamNode p, Cc outercc, Cc innercc, ApOption argAp, Ap ap,
Configuration config
) {
exists(ArgumentNode arg, boolean allowsFieldFlow |
exists(ArgNode arg, boolean allowsFieldFlow |
fwdFlow(arg, outercc, argAp, ap, config) and
flowIntoCall(call, arg, p, allowsFieldFlow, config) and
innercc = getCallContextCall(call, getNodeEnclosingCallable(p), outercc)
@@ -1595,17 +1650,14 @@ private module Stage3 {
)
}
/**
* Holds if flow may exit from `call` at `out` with access path `ap`. The
* inner call context is `innercc`, but `ccOut` is just the call context
* based on the return step. In the case of through-flow `ccOut` is discarded
* and replaced by the outer call context as tracked by `fwdFlowIsEntered`.
*/
pragma[nomagic]
private predicate fwdFlowOut(
DataFlowCall call, Node out, Cc innercc, Cc ccOut, ApOption argAp, Ap ap, Configuration config
private predicate fwdFlowOutNotFromArg(
Node out, Cc ccOut, ApOption argAp, Ap ap, Configuration config
) {
exists(ReturnNodeExt ret, boolean allowsFieldFlow, DataFlowCallable inner |
exists(
DataFlowCall call, ReturnNodeExt ret, boolean allowsFieldFlow, CcNoCall innercc,
DataFlowCallable inner
|
fwdFlow(ret, innercc, argAp, ap, config) and
flowOutOfCall(call, ret, out, allowsFieldFlow, config) and
inner = getNodeEnclosingCallable(ret) and
@@ -1620,7 +1672,13 @@ private module Stage3 {
private predicate fwdFlowOutFromArg(
DataFlowCall call, Node out, Ap argAp, Ap ap, Configuration config
) {
fwdFlowOut(call, out, any(CcCall ccc), _, apSome(argAp), ap, config)
exists(ReturnNodeExt ret, boolean allowsFieldFlow, CcCall ccc |
fwdFlow(ret, ccc, apSome(argAp), ap, config) and
flowThroughOutOfCall(call, ret, out, allowsFieldFlow, config) and
ccc.matchesCall(call)
|
ap instanceof ApNil or allowsFieldFlow = true
)
}
/**
@@ -1631,7 +1689,7 @@ private module Stage3 {
private predicate fwdFlowIsEntered(
DataFlowCall call, Cc cc, ApOption argAp, Ap ap, Configuration config
) {
exists(ParameterNode p |
exists(ParamNode p |
fwdFlowIn(call, p, cc, _, argAp, ap, config) and
PrevStage::parameterMayFlowThrough(p, _, unbindApa(getApprox(ap)), config)
)
@@ -1651,6 +1709,27 @@ private module Stage3 {
fwdFlowConsCand(ap1, c, ap2, config)
}
pragma[nomagic]
private predicate callMayFlowThroughFwd(DataFlowCall call, Configuration config) {
exists(Ap argAp0, Node out, Cc cc, ApOption argAp, Ap ap |
fwdFlow(out, pragma[only_bind_into](cc), pragma[only_bind_into](argAp), ap,
pragma[only_bind_into](config)) and
fwdFlowOutFromArg(call, out, argAp0, ap, config) and
fwdFlowIsEntered(call, pragma[only_bind_into](cc), pragma[only_bind_into](argAp), argAp0,
pragma[only_bind_into](config))
)
}
pragma[nomagic]
private predicate flowThroughIntoCall(
DataFlowCall call, ArgNode arg, ParamNode p, boolean allowsFieldFlow, Configuration config
) {
flowIntoCall(call, arg, p, allowsFieldFlow, config) and
fwdFlow(arg, _, _, _, pragma[only_bind_into](config)) and
PrevStage::parameterMayFlowThrough(p, _, _, pragma[only_bind_into](config)) and
callMayFlowThroughFwd(call, pragma[only_bind_into](config))
}
/**
* Holds if `node` with access path `ap` is part of a path from a source to a
* sink in the configuration `config`.
@@ -1716,14 +1795,12 @@ private module Stage3 {
)
or
// flow into a callable
exists(DataFlowCall call |
revFlowIn(call, node, toReturn, returnAp, ap, config) and
toReturn = false
or
exists(Ap returnAp0 |
revFlowInToReturn(call, node, returnAp0, ap, config) and
revFlowIsReturned(call, toReturn, returnAp, returnAp0, config)
)
revFlowInNotToReturn(node, returnAp, ap, config) and
toReturn = false
or
exists(DataFlowCall call, Ap returnAp0 |
revFlowInToReturn(call, node, returnAp0, ap, config) and
revFlowIsReturned(call, toReturn, returnAp, returnAp0, config)
)
or
// flow out of a callable
@@ -1771,13 +1848,10 @@ private module Stage3 {
}
pragma[nomagic]
private predicate revFlowIn(
DataFlowCall call, ArgumentNode arg, boolean toReturn, ApOption returnAp, Ap ap,
Configuration config
) {
exists(ParameterNode p, boolean allowsFieldFlow |
revFlow(p, toReturn, returnAp, ap, config) and
flowIntoCall(call, arg, p, allowsFieldFlow, config)
private predicate revFlowInNotToReturn(ArgNode arg, ApOption returnAp, Ap ap, Configuration config) {
exists(ParamNode p, boolean allowsFieldFlow |
revFlow(p, false, returnAp, ap, config) and
flowIntoCall(_, arg, p, allowsFieldFlow, config)
|
ap instanceof ApNil or allowsFieldFlow = true
)
@@ -1785,9 +1859,14 @@ private module Stage3 {
pragma[nomagic]
private predicate revFlowInToReturn(
DataFlowCall call, ArgumentNode arg, Ap returnAp, Ap ap, Configuration config
DataFlowCall call, ArgNode arg, Ap returnAp, Ap ap, Configuration config
) {
revFlowIn(call, arg, true, apSome(returnAp), ap, config)
exists(ParamNode p, boolean allowsFieldFlow |
revFlow(p, true, apSome(returnAp), ap, config) and
flowThroughIntoCall(call, arg, p, allowsFieldFlow, config)
|
ap instanceof ApNil or allowsFieldFlow = true
)
}
/**
@@ -1838,13 +1917,13 @@ private module Stage3 {
pragma[noinline]
private predicate parameterFlow(
ParameterNode p, Ap ap, Ap ap0, DataFlowCallable c, Configuration config
ParamNode p, Ap ap, Ap ap0, DataFlowCallable c, Configuration config
) {
revFlow(p, true, apSome(ap0), ap, config) and
c = getNodeEnclosingCallable(p)
}
predicate parameterMayFlowThrough(ParameterNode p, DataFlowCallable c, Ap ap, Configuration config) {
predicate parameterMayFlowThrough(ParamNode p, DataFlowCallable c, Ap ap, Configuration config) {
exists(ReturnNodeExt ret, Ap ap0, ReturnKindExt kind, int pos |
parameterFlow(p, ap, ap0, c, config) and
c = getNodeEnclosingCallable(ret) and
@@ -1857,6 +1936,15 @@ private module Stage3 {
)
}
pragma[nomagic]
predicate callMayFlowThroughRev(DataFlowCall call, Configuration config) {
exists(Ap returnAp0, ArgNode arg, boolean toReturn, ApOption returnAp, Ap ap |
revFlow(arg, toReturn, returnAp, ap, config) and
revFlowInToReturn(call, arg, returnAp0, ap, config) and
revFlowIsReturned(call, toReturn, returnAp, returnAp0, config)
)
}
predicate stats(boolean fwd, int nodes, int fields, int conscand, int tuples, Configuration config) {
fwd = true and
nodes = count(Node node | fwdFlow(node, _, _, _, config)) and
@@ -2088,7 +2176,7 @@ private module Stage4 {
private ApApprox getApprox(Ap ap) { result = ap.getFront() }
private ApNil getApNil(Node node) {
PrevStage::revFlow(node, _) and result = TNil(getNodeType(node))
PrevStage::revFlow(node, _) and result = TNil(getNodeDataFlowType(node))
}
bindingset[tc, tail]
@@ -2127,8 +2215,6 @@ private module Stage4 {
bindingset[innercc, inner, call]
private predicate checkCallContextReturn(Cc innercc, DataFlowCallable inner, DataFlowCall call) {
resolveReturn(innercc, inner, call)
or
innercc.(CallContextCall).matchesCall(call)
}
bindingset[node, cc, config]
@@ -2155,8 +2241,7 @@ private module Stage4 {
pragma[nomagic]
private predicate flowIntoCall(
DataFlowCall call, ArgumentNode node1, ParameterNode node2, boolean allowsFieldFlow,
Configuration config
DataFlowCall call, ArgNode node1, ParamNode node2, boolean allowsFieldFlow, Configuration config
) {
flowIntoCallNodeCand2(call, node1, node2, allowsFieldFlow, config) and
PrevStage::revFlow(node2, _, _, _, pragma[only_bind_into](config)) and
@@ -2182,6 +2267,16 @@ private module Stage4 {
)
}
pragma[nomagic]
private predicate flowThroughOutOfCall(
DataFlowCall call, ReturnNodeExt ret, Node out, boolean allowsFieldFlow, Configuration config
) {
flowOutOfCall(call, ret, out, allowsFieldFlow, pragma[only_bind_into](config)) and
PrevStage::callMayFlowThroughRev(call, pragma[only_bind_into](config)) and
PrevStage::parameterMayFlowThrough(_, getNodeEnclosingCallable(ret), _,
pragma[only_bind_into](config))
}
/**
* Holds if `node` is reachable with access path `ap` from a source in the
* configuration `config`.
@@ -2255,13 +2350,11 @@ private module Stage4 {
)
or
// flow out of a callable
exists(DataFlowCall call |
fwdFlowOut(call, node, any(CcNoCall innercc), cc, argAp, ap, config)
or
exists(Ap argAp0 |
fwdFlowOutFromArg(call, node, argAp0, ap, config) and
fwdFlowIsEntered(call, cc, argAp, argAp0, config)
)
fwdFlowOutNotFromArg(node, cc, argAp, ap, config)
or
exists(DataFlowCall call, Ap argAp0 |
fwdFlowOutFromArg(call, node, argAp0, ap, config) and
fwdFlowIsEntered(call, cc, argAp, argAp0, config)
)
}
@@ -2300,10 +2393,10 @@ private module Stage4 {
pragma[nomagic]
private predicate fwdFlowIn(
DataFlowCall call, ParameterNode p, Cc outercc, Cc innercc, ApOption argAp, Ap ap,
DataFlowCall call, ParamNode p, Cc outercc, Cc innercc, ApOption argAp, Ap ap,
Configuration config
) {
exists(ArgumentNode arg, boolean allowsFieldFlow |
exists(ArgNode arg, boolean allowsFieldFlow |
fwdFlow(arg, outercc, argAp, ap, config) and
flowIntoCall(call, arg, p, allowsFieldFlow, config) and
innercc = getCallContextCall(call, getNodeEnclosingCallable(p), outercc)
@@ -2312,17 +2405,14 @@ private module Stage4 {
)
}
/**
* Holds if flow may exit from `call` at `out` with access path `ap`. The
* inner call context is `innercc`, but `ccOut` is just the call context
* based on the return step. In the case of through-flow `ccOut` is discarded
* and replaced by the outer call context as tracked by `fwdFlowIsEntered`.
*/
pragma[nomagic]
private predicate fwdFlowOut(
DataFlowCall call, Node out, Cc innercc, Cc ccOut, ApOption argAp, Ap ap, Configuration config
private predicate fwdFlowOutNotFromArg(
Node out, Cc ccOut, ApOption argAp, Ap ap, Configuration config
) {
exists(ReturnNodeExt ret, boolean allowsFieldFlow, DataFlowCallable inner |
exists(
DataFlowCall call, ReturnNodeExt ret, boolean allowsFieldFlow, CcNoCall innercc,
DataFlowCallable inner
|
fwdFlow(ret, innercc, argAp, ap, config) and
flowOutOfCall(call, ret, out, allowsFieldFlow, config) and
inner = getNodeEnclosingCallable(ret) and
@@ -2337,7 +2427,13 @@ private module Stage4 {
private predicate fwdFlowOutFromArg(
DataFlowCall call, Node out, Ap argAp, Ap ap, Configuration config
) {
fwdFlowOut(call, out, any(CcCall ccc), _, apSome(argAp), ap, config)
exists(ReturnNodeExt ret, boolean allowsFieldFlow, CcCall ccc |
fwdFlow(ret, ccc, apSome(argAp), ap, config) and
flowThroughOutOfCall(call, ret, out, allowsFieldFlow, config) and
ccc.matchesCall(call)
|
ap instanceof ApNil or allowsFieldFlow = true
)
}
/**
@@ -2348,7 +2444,7 @@ private module Stage4 {
private predicate fwdFlowIsEntered(
DataFlowCall call, Cc cc, ApOption argAp, Ap ap, Configuration config
) {
exists(ParameterNode p |
exists(ParamNode p |
fwdFlowIn(call, p, cc, _, argAp, ap, config) and
PrevStage::parameterMayFlowThrough(p, _, unbindApa(getApprox(ap)), config)
)
@@ -2368,6 +2464,27 @@ private module Stage4 {
fwdFlowConsCand(ap1, c, ap2, config)
}
pragma[nomagic]
private predicate callMayFlowThroughFwd(DataFlowCall call, Configuration config) {
exists(Ap argAp0, Node out, Cc cc, ApOption argAp, Ap ap |
fwdFlow(out, pragma[only_bind_into](cc), pragma[only_bind_into](argAp), ap,
pragma[only_bind_into](config)) and
fwdFlowOutFromArg(call, out, argAp0, ap, config) and
fwdFlowIsEntered(call, pragma[only_bind_into](cc), pragma[only_bind_into](argAp), argAp0,
pragma[only_bind_into](config))
)
}
pragma[nomagic]
private predicate flowThroughIntoCall(
DataFlowCall call, ArgNode arg, ParamNode p, boolean allowsFieldFlow, Configuration config
) {
flowIntoCall(call, arg, p, allowsFieldFlow, config) and
fwdFlow(arg, _, _, _, pragma[only_bind_into](config)) and
PrevStage::parameterMayFlowThrough(p, _, _, pragma[only_bind_into](config)) and
callMayFlowThroughFwd(call, pragma[only_bind_into](config))
}
/**
* Holds if `node` with access path `ap` is part of a path from a source to a
* sink in the configuration `config`.
@@ -2433,14 +2550,12 @@ private module Stage4 {
)
or
// flow into a callable
exists(DataFlowCall call |
revFlowIn(call, node, toReturn, returnAp, ap, config) and
toReturn = false
or
exists(Ap returnAp0 |
revFlowInToReturn(call, node, returnAp0, ap, config) and
revFlowIsReturned(call, toReturn, returnAp, returnAp0, config)
)
revFlowInNotToReturn(node, returnAp, ap, config) and
toReturn = false
or
exists(DataFlowCall call, Ap returnAp0 |
revFlowInToReturn(call, node, returnAp0, ap, config) and
revFlowIsReturned(call, toReturn, returnAp, returnAp0, config)
)
or
// flow out of a callable
@@ -2488,13 +2603,10 @@ private module Stage4 {
}
pragma[nomagic]
private predicate revFlowIn(
DataFlowCall call, ArgumentNode arg, boolean toReturn, ApOption returnAp, Ap ap,
Configuration config
) {
exists(ParameterNode p, boolean allowsFieldFlow |
revFlow(p, toReturn, returnAp, ap, config) and
flowIntoCall(call, arg, p, allowsFieldFlow, config)
private predicate revFlowInNotToReturn(ArgNode arg, ApOption returnAp, Ap ap, Configuration config) {
exists(ParamNode p, boolean allowsFieldFlow |
revFlow(p, false, returnAp, ap, config) and
flowIntoCall(_, arg, p, allowsFieldFlow, config)
|
ap instanceof ApNil or allowsFieldFlow = true
)
@@ -2502,9 +2614,14 @@ private module Stage4 {
pragma[nomagic]
private predicate revFlowInToReturn(
DataFlowCall call, ArgumentNode arg, Ap returnAp, Ap ap, Configuration config
DataFlowCall call, ArgNode arg, Ap returnAp, Ap ap, Configuration config
) {
revFlowIn(call, arg, true, apSome(returnAp), ap, config)
exists(ParamNode p, boolean allowsFieldFlow |
revFlow(p, true, apSome(returnAp), ap, config) and
flowThroughIntoCall(call, arg, p, allowsFieldFlow, config)
|
ap instanceof ApNil or allowsFieldFlow = true
)
}
/**
@@ -2555,13 +2672,13 @@ private module Stage4 {
pragma[noinline]
private predicate parameterFlow(
ParameterNode p, Ap ap, Ap ap0, DataFlowCallable c, Configuration config
ParamNode p, Ap ap, Ap ap0, DataFlowCallable c, Configuration config
) {
revFlow(p, true, apSome(ap0), ap, config) and
c = getNodeEnclosingCallable(p)
}
predicate parameterMayFlowThrough(ParameterNode p, DataFlowCallable c, Ap ap, Configuration config) {
predicate parameterMayFlowThrough(ParamNode p, DataFlowCallable c, Ap ap, Configuration config) {
exists(ReturnNodeExt ret, Ap ap0, ReturnKindExt kind, int pos |
parameterFlow(p, ap, ap0, c, config) and
c = getNodeEnclosingCallable(ret) and
@@ -2574,6 +2691,15 @@ private module Stage4 {
)
}
pragma[nomagic]
predicate callMayFlowThroughRev(DataFlowCall call, Configuration config) {
exists(Ap returnAp0, ArgNode arg, boolean toReturn, ApOption returnAp, Ap ap |
revFlow(arg, toReturn, returnAp, ap, config) and
revFlowInToReturn(call, arg, returnAp0, ap, config) and
revFlowIsReturned(call, toReturn, returnAp, returnAp0, config)
)
}
predicate stats(boolean fwd, int nodes, int fields, int conscand, int tuples, Configuration config) {
fwd = true and
nodes = count(Node node | fwdFlow(node, _, _, _, config)) and
@@ -2606,7 +2732,7 @@ private predicate nodeMayUseSummary(Node n, AccessPathApprox apa, Configuration
private newtype TSummaryCtx =
TSummaryCtxNone() or
TSummaryCtxSome(ParameterNode p, AccessPath ap) {
TSummaryCtxSome(ParamNode p, AccessPath ap) {
Stage4::parameterMayFlowThrough(p, _, ap.getApprox(), _)
}
@@ -2627,7 +2753,7 @@ private class SummaryCtxNone extends SummaryCtx, TSummaryCtxNone {
/** A summary context from which a flow summary can be generated. */
private class SummaryCtxSome extends SummaryCtx, TSummaryCtxSome {
private ParameterNode p;
private ParamNode p;
private AccessPath ap;
SummaryCtxSome() { this = TSummaryCtxSome(p, ap) }
@@ -2758,7 +2884,7 @@ private newtype TPathNode =
config.isSource(node) and
cc instanceof CallContextAny and
sc instanceof SummaryCtxNone and
ap = TAccessPathNil(getNodeType(node))
ap = TAccessPathNil(getNodeDataFlowType(node))
or
// ... or a step from an existing PathNode to another node.
exists(PathNodeMid mid |
@@ -2979,7 +3105,7 @@ class PathNode extends TPathNode {
Configuration getConfiguration() { none() }
private predicate isHidden() {
nodeIsHidden(this.getNode()) and
hiddenNode(this.getNode()) and
not this.isSource() and
not this instanceof PathNodeSink
}
@@ -3148,7 +3274,7 @@ private predicate pathStep(PathNodeMid mid, Node node, CallContext cc, SummaryCt
cc instanceof CallContextAny and
sc instanceof SummaryCtxNone and
mid.getAp() instanceof AccessPathNil and
ap = TAccessPathNil(getNodeType(node))
ap = TAccessPathNil(getNodeDataFlowType(node))
or
exists(TypedContent tc | pathStoreStep(mid, node, ap.pop(tc), tc, cc)) and
sc = mid.getSummaryCtx()
@@ -3235,7 +3361,7 @@ pragma[noinline]
private predicate pathIntoArg(
PathNodeMid mid, int i, CallContext cc, DataFlowCall call, AccessPath ap, AccessPathApprox apa
) {
exists(ArgumentNode arg |
exists(ArgNode arg |
arg = mid.getNode() and
cc = mid.getCallContext() and
arg.argumentOf(call, i) and
@@ -3248,7 +3374,7 @@ pragma[noinline]
private predicate parameterCand(
DataFlowCallable callable, int i, AccessPathApprox apa, Configuration config
) {
exists(ParameterNode p |
exists(ParamNode p |
Stage4::revFlow(p, _, _, apa, config) and
p.isParameterOf(callable, i)
)
@@ -3272,7 +3398,7 @@ private predicate pathIntoCallable0(
* respectively.
*/
private predicate pathIntoCallable(
PathNodeMid mid, ParameterNode p, CallContext outercc, CallContextCall innercc, SummaryCtx sc,
PathNodeMid mid, ParamNode p, CallContext outercc, CallContextCall innercc, SummaryCtx sc,
DataFlowCall call
) {
exists(int i, DataFlowCallable callable, AccessPath ap |
@@ -3568,7 +3694,7 @@ private module FlowExploration {
private newtype TSummaryCtx1 =
TSummaryCtx1None() or
TSummaryCtx1Param(ParameterNode p)
TSummaryCtx1Param(ParamNode p)
private newtype TSummaryCtx2 =
TSummaryCtx2None() or
@@ -3591,7 +3717,7 @@ private module FlowExploration {
cc instanceof CallContextAny and
sc1 = TSummaryCtx1None() and
sc2 = TSummaryCtx2None() and
ap = TPartialNil(getNodeType(node)) and
ap = TPartialNil(getNodeDataFlowType(node)) and
not fullBarrier(node, config) and
exists(config.explorationLimit())
or
@@ -3611,7 +3737,7 @@ private module FlowExploration {
or
exists(PartialPathNodeRev mid |
revPartialPathStep(mid, node, sc1, sc2, ap, config) and
not clearsContent(node, ap.getHead()) and
not clearsContentCached(node, ap.getHead()) and
not fullBarrier(node, config) and
distSink(getNodeEnclosingCallable(node), config) <= config.explorationLimit()
)
@@ -3625,9 +3751,9 @@ private module FlowExploration {
exists(PartialPathNodeFwd mid |
partialPathStep(mid, node, cc, sc1, sc2, ap, config) and
not fullBarrier(node, config) and
not clearsContent(node, ap.getHead().getContent()) and
not clearsContentCached(node, ap.getHead().getContent()) and
if node instanceof CastingNode
then compatibleTypes(getNodeType(node), ap.getType())
then compatibleTypes(getNodeDataFlowType(node), ap.getType())
else any()
)
}
@@ -3783,7 +3909,7 @@ private module FlowExploration {
PartialPathNodeFwd mid, Node node, CallContext cc, TSummaryCtx1 sc1, TSummaryCtx2 sc2,
PartialAccessPath ap, Configuration config
) {
not isUnreachableInCall(node, cc.(CallContextSpecificCall).getCall()) and
not isUnreachableInCallCached(node, cc.(CallContextSpecificCall).getCall()) and
(
localFlowStep(mid.getNode(), node, config) and
cc = mid.getCallContext() and
@@ -3797,7 +3923,7 @@ private module FlowExploration {
sc1 = mid.getSummaryCtx1() and
sc2 = mid.getSummaryCtx2() and
mid.getAp() instanceof PartialAccessPathNil and
ap = TPartialNil(getNodeType(node)) and
ap = TPartialNil(getNodeDataFlowType(node)) and
config = mid.getConfiguration()
)
or
@@ -3813,7 +3939,7 @@ private module FlowExploration {
sc1 = TSummaryCtx1None() and
sc2 = TSummaryCtx2None() and
mid.getAp() instanceof PartialAccessPathNil and
ap = TPartialNil(getNodeType(node)) and
ap = TPartialNil(getNodeDataFlowType(node)) and
config = mid.getConfiguration()
or
partialPathStoreStep(mid, _, _, node, ap) and
@@ -3827,7 +3953,7 @@ private module FlowExploration {
sc1 = mid.getSummaryCtx1() and
sc2 = mid.getSummaryCtx2() and
apConsFwd(ap, tc, ap0, config) and
compatibleTypes(ap.getType(), getNodeType(node))
compatibleTypes(ap.getType(), getNodeDataFlowType(node))
)
or
partialPathIntoCallable(mid, node, _, cc, sc1, sc2, _, ap, config)
@@ -3924,7 +4050,7 @@ private module FlowExploration {
PartialPathNodeFwd mid, int i, CallContext cc, DataFlowCall call, PartialAccessPath ap,
Configuration config
) {
exists(ArgumentNode arg |
exists(ArgNode arg |
arg = mid.getNode() and
cc = mid.getCallContext() and
arg.argumentOf(call, i) and
@@ -3943,7 +4069,7 @@ private module FlowExploration {
}
private predicate partialPathIntoCallable(
PartialPathNodeFwd mid, ParameterNode p, CallContext outercc, CallContextCall innercc,
PartialPathNodeFwd mid, ParamNode p, CallContext outercc, CallContextCall innercc,
TSummaryCtx1 sc1, TSummaryCtx2 sc2, DataFlowCall call, PartialAccessPath ap,
Configuration config
) {
@@ -3980,7 +4106,7 @@ private module FlowExploration {
DataFlowCall call, PartialPathNodeFwd mid, ReturnKindExt kind, CallContext cc,
PartialAccessPath ap, Configuration config
) {
exists(ParameterNode p, CallContext innercc, TSummaryCtx1 sc1, TSummaryCtx2 sc2 |
exists(ParamNode p, CallContext innercc, TSummaryCtx1 sc1, TSummaryCtx2 sc2 |
partialPathIntoCallable(mid, p, cc, innercc, sc1, sc2, call, _, config) and
paramFlowsThroughInPartialPath(kind, innercc, sc1, sc2, ap, config)
)
@@ -4037,7 +4163,7 @@ private module FlowExploration {
apConsRev(ap, c, ap0, config)
)
or
exists(ParameterNode p |
exists(ParamNode p |
mid.getNode() = p and
viableParamArg(_, p, node) and
sc1 = mid.getSummaryCtx1() and
@@ -4115,7 +4241,7 @@ private module FlowExploration {
int pos, TRevSummaryCtx1Some sc1, TRevSummaryCtx2Some sc2, RevPartialAccessPath ap,
Configuration config
) {
exists(PartialPathNodeRev mid, ParameterNode p |
exists(PartialPathNodeRev mid, ParamNode p |
mid.getNode() = p and
p.isParameterOf(_, pos) and
sc1 = mid.getSummaryCtx1() and
@@ -4138,7 +4264,7 @@ private module FlowExploration {
pragma[nomagic]
private predicate revPartialPathThroughCallable(
PartialPathNodeRev mid, ArgumentNode node, RevPartialAccessPath ap, Configuration config
PartialPathNodeRev mid, ArgNode node, RevPartialAccessPath ap, Configuration config
) {
exists(DataFlowCall call, int pos |
revPartialPathThroughCallable0(call, mid, pos, ap, config) and

476
python/ql/src/semmle/python/dataflow/new/internal/DataFlowImpl2.qll
View File

@@ -211,10 +211,7 @@ private predicate fullBarrier(Node node, Configuration config) {
* Holds if data can flow in one local step from `node1` to `node2`.
*/
private predicate localFlowStep(Node node1, Node node2, Configuration config) {
(
simpleLocalFlowStep(node1, node2) or
reverseStepThroughInputOutputAlias(node1, node2)
) and
simpleLocalFlowStepExt(node1, node2) and
not outBarrier(node1, config) and
not inBarrier(node2, config) and
not fullBarrier(node1, config) and
@@ -237,7 +234,7 @@ private predicate additionalLocalFlowStep(Node node1, Node node2, Configuration
* Holds if data can flow from `node1` to `node2` in a way that discards call contexts.
*/
private predicate jumpStep(Node node1, Node node2, Configuration config) {
jumpStep(node1, node2) and
jumpStepCached(node1, node2) and
not outBarrier(node1, config) and
not inBarrier(node2, config) and
not fullBarrier(node1, config) and
@@ -388,7 +385,7 @@ private module Stage1 {
*/
pragma[nomagic]
private predicate fwdFlowIsEntered(DataFlowCall call, Cc cc, Configuration config) {
exists(ArgumentNode arg |
exists(ArgNode arg |
fwdFlow(arg, cc, config) and
viableParamArg(call, _, arg)
)
@@ -515,29 +512,29 @@ private module Stage1 {
pragma[nomagic]
predicate viableParamArgNodeCandFwd1(
DataFlowCall call, ParameterNode p, ArgumentNode arg, Configuration config
DataFlowCall call, ParamNode p, ArgNode arg, Configuration config
) {
viableParamArg(call, p, arg) and
fwdFlow(arg, config)
}
pragma[nomagic]
private predicate revFlowIn(
DataFlowCall call, ArgumentNode arg, boolean toReturn, Configuration config
) {
exists(ParameterNode p |
private predicate revFlowIn(DataFlowCall call, ArgNode arg, boolean toReturn, Configuration config) {
exists(ParamNode p |
revFlow(p, toReturn, config) and
viableParamArgNodeCandFwd1(call, p, arg, config)
)
}
pragma[nomagic]
private predicate revFlowInToReturn(DataFlowCall call, ArgumentNode arg, Configuration config) {
private predicate revFlowInToReturn(DataFlowCall call, ArgNode arg, Configuration config) {
revFlowIn(call, arg, true, config)
}
/**
* Holds if an output from `call` is reached in the flow covered by `revFlow`.
* Holds if an output from `call` is reached in the flow covered by `revFlow`
* and data might flow through the target callable resulting in reverse flow
* reaching an argument of `call`.
*/
pragma[nomagic]
private predicate revFlowIsReturned(DataFlowCall call, boolean toReturn, Configuration config) {
@@ -597,7 +594,7 @@ private module Stage1 {
* Holds if flow may enter through `p` and reach a return node making `p` a
* candidate for the origin of a summary.
*/
predicate parameterMayFlowThrough(ParameterNode p, DataFlowCallable c, Ap ap, Configuration config) {
predicate parameterMayFlowThrough(ParamNode p, DataFlowCallable c, Ap ap, Configuration config) {
exists(ReturnKindExt kind |
throughFlowNodeCand(p, config) and
returnFlowCallableNodeCand(c, kind, config) and
@@ -610,6 +607,15 @@ private module Stage1 {
)
}
pragma[nomagic]
predicate callMayFlowThroughRev(DataFlowCall call, Configuration config) {
exists(ArgNode arg, boolean toReturn |
revFlow(arg, toReturn, config) and
revFlowInToReturn(call, arg, config) and
revFlowIsReturned(call, toReturn, config)
)
}
predicate stats(boolean fwd, int nodes, int fields, int conscand, int tuples, Configuration config) {
fwd = true and
nodes = count(Node node | fwdFlow(node, config)) and
@@ -663,7 +669,7 @@ private predicate flowOutOfCallNodeCand1(
pragma[nomagic]
private predicate viableParamArgNodeCand1(
DataFlowCall call, ParameterNode p, ArgumentNode arg, Configuration config
DataFlowCall call, ParamNode p, ArgNode arg, Configuration config
) {
Stage1::viableParamArgNodeCandFwd1(call, p, arg, config) and
Stage1::revFlow(arg, config)
@@ -675,7 +681,7 @@ private predicate viableParamArgNodeCand1(
*/
pragma[nomagic]
private predicate flowIntoCallNodeCand1(
DataFlowCall call, ArgumentNode arg, ParameterNode p, Configuration config
DataFlowCall call, ArgNode arg, ParamNode p, Configuration config
) {
viableParamArgNodeCand1(call, p, arg, config) and
Stage1::revFlow(p, config) and
@@ -735,8 +741,7 @@ private predicate flowOutOfCallNodeCand1(
*/
pragma[nomagic]
private predicate flowIntoCallNodeCand1(
DataFlowCall call, ArgumentNode arg, ParameterNode p, boolean allowsFieldFlow,
Configuration config
DataFlowCall call, ArgNode arg, ParamNode p, boolean allowsFieldFlow, Configuration config
) {
flowIntoCallNodeCand1(call, arg, p, config) and
exists(int b, int j |
@@ -833,6 +838,16 @@ private module Stage2 {
PrevStage::revFlow(node, _, _, apa, config)
}
pragma[nomagic]
private predicate flowThroughOutOfCall(
DataFlowCall call, ReturnNodeExt ret, Node out, boolean allowsFieldFlow, Configuration config
) {
flowOutOfCall(call, ret, out, allowsFieldFlow, pragma[only_bind_into](config)) and
PrevStage::callMayFlowThroughRev(call, pragma[only_bind_into](config)) and
PrevStage::parameterMayFlowThrough(_, getNodeEnclosingCallable(ret), _,
pragma[only_bind_into](config))
}
/**
* Holds if `node` is reachable with access path `ap` from a source in the
* configuration `config`.
@@ -899,13 +914,11 @@ private module Stage2 {
)
or
// flow out of a callable
exists(DataFlowCall call |
fwdFlowOut(call, node, any(CcNoCall innercc), cc, argAp, ap, config)
or
exists(Ap argAp0 |
fwdFlowOutFromArg(call, node, argAp0, ap, config) and
fwdFlowIsEntered(call, cc, argAp, argAp0, config)
)
fwdFlowOutNotFromArg(node, cc, argAp, ap, config)
or
exists(DataFlowCall call, Ap argAp0 |
fwdFlowOutFromArg(call, node, argAp0, ap, config) and
fwdFlowIsEntered(call, cc, argAp, argAp0, config)
)
}
@@ -944,10 +957,10 @@ private module Stage2 {
pragma[nomagic]
private predicate fwdFlowIn(
DataFlowCall call, ParameterNode p, Cc outercc, Cc innercc, ApOption argAp, Ap ap,
DataFlowCall call, ParamNode p, Cc outercc, Cc innercc, ApOption argAp, Ap ap,
Configuration config
) {
exists(ArgumentNode arg, boolean allowsFieldFlow |
exists(ArgNode arg, boolean allowsFieldFlow |
fwdFlow(arg, outercc, argAp, ap, config) and
flowIntoCall(call, arg, p, allowsFieldFlow, config) and
innercc = getCallContextCall(call, getNodeEnclosingCallable(p), outercc)
@@ -956,17 +969,14 @@ private module Stage2 {
)
}
/**
* Holds if flow may exit from `call` at `out` with access path `ap`. The
* inner call context is `innercc`, but `ccOut` is just the call context
* based on the return step. In the case of through-flow `ccOut` is discarded
* and replaced by the outer call context as tracked by `fwdFlowIsEntered`.
*/
pragma[nomagic]
private predicate fwdFlowOut(
DataFlowCall call, Node out, Cc innercc, Cc ccOut, ApOption argAp, Ap ap, Configuration config
private predicate fwdFlowOutNotFromArg(
Node out, Cc ccOut, ApOption argAp, Ap ap, Configuration config
) {
exists(ReturnNodeExt ret, boolean allowsFieldFlow, DataFlowCallable inner |
exists(
DataFlowCall call, ReturnNodeExt ret, boolean allowsFieldFlow, CcNoCall innercc,
DataFlowCallable inner
|
fwdFlow(ret, innercc, argAp, ap, config) and
flowOutOfCall(call, ret, out, allowsFieldFlow, config) and
inner = getNodeEnclosingCallable(ret) and
@@ -981,7 +991,13 @@ private module Stage2 {
private predicate fwdFlowOutFromArg(
DataFlowCall call, Node out, Ap argAp, Ap ap, Configuration config
) {
fwdFlowOut(call, out, any(CcCall ccc), _, apSome(argAp), ap, config)
exists(ReturnNodeExt ret, boolean allowsFieldFlow, CcCall ccc |
fwdFlow(ret, ccc, apSome(argAp), ap, config) and
flowThroughOutOfCall(call, ret, out, allowsFieldFlow, config) and
ccc.matchesCall(call)
|
ap instanceof ApNil or allowsFieldFlow = true
)
}
/**
@@ -992,7 +1008,7 @@ private module Stage2 {
private predicate fwdFlowIsEntered(
DataFlowCall call, Cc cc, ApOption argAp, Ap ap, Configuration config
) {
exists(ParameterNode p |
exists(ParamNode p |
fwdFlowIn(call, p, cc, _, argAp, ap, config) and
PrevStage::parameterMayFlowThrough(p, _, getApprox(ap), config)
)
@@ -1012,6 +1028,27 @@ private module Stage2 {
fwdFlowConsCand(ap1, c, ap2, config)
}
pragma[nomagic]
private predicate callMayFlowThroughFwd(DataFlowCall call, Configuration config) {
exists(Ap argAp0, Node out, Cc cc, ApOption argAp, Ap ap |
fwdFlow(out, pragma[only_bind_into](cc), pragma[only_bind_into](argAp), ap,
pragma[only_bind_into](config)) and
fwdFlowOutFromArg(call, out, argAp0, ap, config) and
fwdFlowIsEntered(call, pragma[only_bind_into](cc), pragma[only_bind_into](argAp), argAp0,
pragma[only_bind_into](config))
)
}
pragma[nomagic]
private predicate flowThroughIntoCall(
DataFlowCall call, ArgNode arg, ParamNode p, boolean allowsFieldFlow, Configuration config
) {
flowIntoCall(call, arg, p, allowsFieldFlow, config) and
fwdFlow(arg, _, _, _, pragma[only_bind_into](config)) and
PrevStage::parameterMayFlowThrough(p, _, _, pragma[only_bind_into](config)) and
callMayFlowThroughFwd(call, pragma[only_bind_into](config))
}
/**
* Holds if `node` with access path `ap` is part of a path from a source to a
* sink in the configuration `config`.
@@ -1077,14 +1114,12 @@ private module Stage2 {
)
or
// flow into a callable
exists(DataFlowCall call |
revFlowIn(call, node, toReturn, returnAp, ap, config) and
toReturn = false
or
exists(Ap returnAp0 |
revFlowInToReturn(call, node, returnAp0, ap, config) and
revFlowIsReturned(call, toReturn, returnAp, returnAp0, config)
)
revFlowInNotToReturn(node, returnAp, ap, config) and
toReturn = false
or
exists(DataFlowCall call, Ap returnAp0 |
revFlowInToReturn(call, node, returnAp0, ap, config) and
revFlowIsReturned(call, toReturn, returnAp, returnAp0, config)
)
or
// flow out of a callable
@@ -1132,13 +1167,10 @@ private module Stage2 {
}
pragma[nomagic]
private predicate revFlowIn(
DataFlowCall call, ArgumentNode arg, boolean toReturn, ApOption returnAp, Ap ap,
Configuration config
) {
exists(ParameterNode p, boolean allowsFieldFlow |
revFlow(p, toReturn, returnAp, ap, config) and
flowIntoCall(call, arg, p, allowsFieldFlow, config)
private predicate revFlowInNotToReturn(ArgNode arg, ApOption returnAp, Ap ap, Configuration config) {
exists(ParamNode p, boolean allowsFieldFlow |
revFlow(p, false, returnAp, ap, config) and
flowIntoCall(_, arg, p, allowsFieldFlow, config)
|
ap instanceof ApNil or allowsFieldFlow = true
)
@@ -1146,9 +1178,14 @@ private module Stage2 {
pragma[nomagic]
private predicate revFlowInToReturn(
DataFlowCall call, ArgumentNode arg, Ap returnAp, Ap ap, Configuration config
DataFlowCall call, ArgNode arg, Ap returnAp, Ap ap, Configuration config
) {
revFlowIn(call, arg, true, apSome(returnAp), ap, config)
exists(ParamNode p, boolean allowsFieldFlow |
revFlow(p, true, apSome(returnAp), ap, config) and
flowThroughIntoCall(call, arg, p, allowsFieldFlow, config)
|
ap instanceof ApNil or allowsFieldFlow = true
)
}
/**
@@ -1199,13 +1236,13 @@ private module Stage2 {
pragma[noinline]
private predicate parameterFlow(
ParameterNode p, Ap ap, Ap ap0, DataFlowCallable c, Configuration config
ParamNode p, Ap ap, Ap ap0, DataFlowCallable c, Configuration config
) {
revFlow(p, true, apSome(ap0), ap, config) and
c = getNodeEnclosingCallable(p)
}
predicate parameterMayFlowThrough(ParameterNode p, DataFlowCallable c, Ap ap, Configuration config) {
predicate parameterMayFlowThrough(ParamNode p, DataFlowCallable c, Ap ap, Configuration config) {
exists(ReturnNodeExt ret, Ap ap0, ReturnKindExt kind, int pos |
parameterFlow(p, ap, ap0, c, config) and
c = getNodeEnclosingCallable(ret) and
@@ -1218,6 +1255,15 @@ private module Stage2 {
)
}
pragma[nomagic]
predicate callMayFlowThroughRev(DataFlowCall call, Configuration config) {
exists(Ap returnAp0, ArgNode arg, boolean toReturn, ApOption returnAp, Ap ap |
revFlow(arg, toReturn, returnAp, ap, config) and
revFlowInToReturn(call, arg, returnAp0, ap, config) and
revFlowIsReturned(call, toReturn, returnAp, returnAp0, config)
)
}
predicate stats(boolean fwd, int nodes, int fields, int conscand, int tuples, Configuration config) {
fwd = true and
nodes = count(Node node | fwdFlow(node, _, _, _, config)) and
@@ -1245,8 +1291,7 @@ private predicate flowOutOfCallNodeCand2(
pragma[nomagic]
private predicate flowIntoCallNodeCand2(
DataFlowCall call, ArgumentNode node1, ParameterNode node2, boolean allowsFieldFlow,
Configuration config
DataFlowCall call, ArgNode node1, ParamNode node2, boolean allowsFieldFlow, Configuration config
) {
flowIntoCallNodeCand1(call, node1, node2, allowsFieldFlow, config) and
Stage2::revFlow(node2, pragma[only_bind_into](config)) and
@@ -1260,8 +1305,8 @@ private module LocalFlowBigStep {
*/
private class FlowCheckNode extends Node {
FlowCheckNode() {
this instanceof CastNode or
clearsContent(this, _)
castNode(this) or
clearsContentCached(this, _)
}
}
@@ -1275,7 +1320,7 @@ private module LocalFlowBigStep {
config.isSource(node) or
jumpStep(_, node, config) or
additionalJumpStep(_, node, config) or
node instanceof ParameterNode or
node instanceof ParamNode or
node instanceof OutNodeExt or
store(_, _, node, _) or
read(_, _, node) or
@@ -1321,21 +1366,21 @@ private module LocalFlowBigStep {
Node node1, Node node2, boolean preservesValue, DataFlowType t, Configuration config,
LocalCallContext cc
) {
not isUnreachableInCall(node2, cc.(LocalCallContextSpecificCall).getCall()) and
not isUnreachableInCallCached(node2, cc.(LocalCallContextSpecificCall).getCall()) and
(
localFlowEntry(node1, pragma[only_bind_into](config)) and
(
localFlowStepNodeCand1(node1, node2, config) and
preservesValue = true and
t = getNodeType(node1)
t = getNodeDataFlowType(node1)
or
additionalLocalFlowStepNodeCand2(node1, node2, config) and
preservesValue = false and
t = getNodeType(node2)
t = getNodeDataFlowType(node2)
) and
node1 != node2 and
cc.relevantFor(getNodeEnclosingCallable(node1)) and
not isUnreachableInCall(node1, cc.(LocalCallContextSpecificCall).getCall()) and
not isUnreachableInCallCached(node1, cc.(LocalCallContextSpecificCall).getCall()) and
Stage2::revFlow(node2, pragma[only_bind_into](config))
or
exists(Node mid |
@@ -1350,7 +1395,7 @@ private module LocalFlowBigStep {
additionalLocalFlowStepNodeCand2(mid, node2, config) and
not mid instanceof FlowCheckNode and
preservesValue = false and
t = getNodeType(node2) and
t = getNodeDataFlowType(node2) and
Stage2::revFlow(node2, pragma[only_bind_into](config))
)
)
@@ -1384,7 +1429,7 @@ private module Stage3 {
private ApApprox getApprox(Ap ap) { result = ap.toBoolNonEmpty() }
private ApNil getApNil(Node node) {
PrevStage::revFlow(node, _) and result = TFrontNil(getNodeType(node))
PrevStage::revFlow(node, _) and result = TFrontNil(getNodeDataFlowType(node))
}
bindingset[tc, tail]
@@ -1443,7 +1488,9 @@ private module Stage3 {
bindingset[node, ap]
private predicate filter(Node node, Ap ap) {
not ap.isClearedAt(node) and
if node instanceof CastingNode then compatibleTypes(getNodeType(node), ap.getType()) else any()
if node instanceof CastingNode
then compatibleTypes(getNodeDataFlowType(node), ap.getType())
else any()
}
bindingset[ap, contentType]
@@ -1465,6 +1512,16 @@ private module Stage3 {
)
}
pragma[nomagic]
private predicate flowThroughOutOfCall(
DataFlowCall call, ReturnNodeExt ret, Node out, boolean allowsFieldFlow, Configuration config
) {
flowOutOfCall(call, ret, out, allowsFieldFlow, pragma[only_bind_into](config)) and
PrevStage::callMayFlowThroughRev(call, pragma[only_bind_into](config)) and
PrevStage::parameterMayFlowThrough(_, getNodeEnclosingCallable(ret), _,
pragma[only_bind_into](config))
}
/**
* Holds if `node` is reachable with access path `ap` from a source in the
* configuration `config`.
@@ -1538,13 +1595,11 @@ private module Stage3 {
)
or
// flow out of a callable
exists(DataFlowCall call |
fwdFlowOut(call, node, any(CcNoCall innercc), cc, argAp, ap, config)
or
exists(Ap argAp0 |
fwdFlowOutFromArg(call, node, argAp0, ap, config) and
fwdFlowIsEntered(call, cc, argAp, argAp0, config)
)
fwdFlowOutNotFromArg(node, cc, argAp, ap, config)
or
exists(DataFlowCall call, Ap argAp0 |
fwdFlowOutFromArg(call, node, argAp0, ap, config) and
fwdFlowIsEntered(call, cc, argAp, argAp0, config)
)
}
@@ -1583,10 +1638,10 @@ private module Stage3 {
pragma[nomagic]
private predicate fwdFlowIn(
DataFlowCall call, ParameterNode p, Cc outercc, Cc innercc, ApOption argAp, Ap ap,
DataFlowCall call, ParamNode p, Cc outercc, Cc innercc, ApOption argAp, Ap ap,
Configuration config
) {
exists(ArgumentNode arg, boolean allowsFieldFlow |
exists(ArgNode arg, boolean allowsFieldFlow |
fwdFlow(arg, outercc, argAp, ap, config) and
flowIntoCall(call, arg, p, allowsFieldFlow, config) and
innercc = getCallContextCall(call, getNodeEnclosingCallable(p), outercc)
@@ -1595,17 +1650,14 @@ private module Stage3 {
)
}
/**
* Holds if flow may exit from `call` at `out` with access path `ap`. The
* inner call context is `innercc`, but `ccOut` is just the call context
* based on the return step. In the case of through-flow `ccOut` is discarded
* and replaced by the outer call context as tracked by `fwdFlowIsEntered`.
*/
pragma[nomagic]
private predicate fwdFlowOut(
DataFlowCall call, Node out, Cc innercc, Cc ccOut, ApOption argAp, Ap ap, Configuration config
private predicate fwdFlowOutNotFromArg(
Node out, Cc ccOut, ApOption argAp, Ap ap, Configuration config
) {
exists(ReturnNodeExt ret, boolean allowsFieldFlow, DataFlowCallable inner |
exists(
DataFlowCall call, ReturnNodeExt ret, boolean allowsFieldFlow, CcNoCall innercc,
DataFlowCallable inner
|
fwdFlow(ret, innercc, argAp, ap, config) and
flowOutOfCall(call, ret, out, allowsFieldFlow, config) and
inner = getNodeEnclosingCallable(ret) and
@@ -1620,7 +1672,13 @@ private module Stage3 {
private predicate fwdFlowOutFromArg(
DataFlowCall call, Node out, Ap argAp, Ap ap, Configuration config
) {
fwdFlowOut(call, out, any(CcCall ccc), _, apSome(argAp), ap, config)
exists(ReturnNodeExt ret, boolean allowsFieldFlow, CcCall ccc |
fwdFlow(ret, ccc, apSome(argAp), ap, config) and
flowThroughOutOfCall(call, ret, out, allowsFieldFlow, config) and
ccc.matchesCall(call)
|
ap instanceof ApNil or allowsFieldFlow = true
)
}
/**
@@ -1631,7 +1689,7 @@ private module Stage3 {
private predicate fwdFlowIsEntered(
DataFlowCall call, Cc cc, ApOption argAp, Ap ap, Configuration config
) {
exists(ParameterNode p |
exists(ParamNode p |
fwdFlowIn(call, p, cc, _, argAp, ap, config) and
PrevStage::parameterMayFlowThrough(p, _, unbindApa(getApprox(ap)), config)
)
@@ -1651,6 +1709,27 @@ private module Stage3 {
fwdFlowConsCand(ap1, c, ap2, config)
}
pragma[nomagic]
private predicate callMayFlowThroughFwd(DataFlowCall call, Configuration config) {
exists(Ap argAp0, Node out, Cc cc, ApOption argAp, Ap ap |
fwdFlow(out, pragma[only_bind_into](cc), pragma[only_bind_into](argAp), ap,
pragma[only_bind_into](config)) and
fwdFlowOutFromArg(call, out, argAp0, ap, config) and
fwdFlowIsEntered(call, pragma[only_bind_into](cc), pragma[only_bind_into](argAp), argAp0,
pragma[only_bind_into](config))
)
}
pragma[nomagic]
private predicate flowThroughIntoCall(
DataFlowCall call, ArgNode arg, ParamNode p, boolean allowsFieldFlow, Configuration config
) {
flowIntoCall(call, arg, p, allowsFieldFlow, config) and
fwdFlow(arg, _, _, _, pragma[only_bind_into](config)) and
PrevStage::parameterMayFlowThrough(p, _, _, pragma[only_bind_into](config)) and
callMayFlowThroughFwd(call, pragma[only_bind_into](config))
}
/**
* Holds if `node` with access path `ap` is part of a path from a source to a
* sink in the configuration `config`.
@@ -1716,14 +1795,12 @@ private module Stage3 {
)
or
// flow into a callable
exists(DataFlowCall call |
revFlowIn(call, node, toReturn, returnAp, ap, config) and
toReturn = false
or
exists(Ap returnAp0 |
revFlowInToReturn(call, node, returnAp0, ap, config) and
revFlowIsReturned(call, toReturn, returnAp, returnAp0, config)
)
revFlowInNotToReturn(node, returnAp, ap, config) and
toReturn = false
or
exists(DataFlowCall call, Ap returnAp0 |
revFlowInToReturn(call, node, returnAp0, ap, config) and
revFlowIsReturned(call, toReturn, returnAp, returnAp0, config)
)
or
// flow out of a callable
@@ -1771,13 +1848,10 @@ private module Stage3 {
}
pragma[nomagic]
private predicate revFlowIn(
DataFlowCall call, ArgumentNode arg, boolean toReturn, ApOption returnAp, Ap ap,
Configuration config
) {
exists(ParameterNode p, boolean allowsFieldFlow |
revFlow(p, toReturn, returnAp, ap, config) and
flowIntoCall(call, arg, p, allowsFieldFlow, config)
private predicate revFlowInNotToReturn(ArgNode arg, ApOption returnAp, Ap ap, Configuration config) {
exists(ParamNode p, boolean allowsFieldFlow |
revFlow(p, false, returnAp, ap, config) and
flowIntoCall(_, arg, p, allowsFieldFlow, config)
|
ap instanceof ApNil or allowsFieldFlow = true
)
@@ -1785,9 +1859,14 @@ private module Stage3 {
pragma[nomagic]
private predicate revFlowInToReturn(
DataFlowCall call, ArgumentNode arg, Ap returnAp, Ap ap, Configuration config
DataFlowCall call, ArgNode arg, Ap returnAp, Ap ap, Configuration config
) {
revFlowIn(call, arg, true, apSome(returnAp), ap, config)
exists(ParamNode p, boolean allowsFieldFlow |
revFlow(p, true, apSome(returnAp), ap, config) and
flowThroughIntoCall(call, arg, p, allowsFieldFlow, config)
|
ap instanceof ApNil or allowsFieldFlow = true
)
}
/**
@@ -1838,13 +1917,13 @@ private module Stage3 {
pragma[noinline]
private predicate parameterFlow(
ParameterNode p, Ap ap, Ap ap0, DataFlowCallable c, Configuration config
ParamNode p, Ap ap, Ap ap0, DataFlowCallable c, Configuration config
) {
revFlow(p, true, apSome(ap0), ap, config) and
c = getNodeEnclosingCallable(p)
}
predicate parameterMayFlowThrough(ParameterNode p, DataFlowCallable c, Ap ap, Configuration config) {
predicate parameterMayFlowThrough(ParamNode p, DataFlowCallable c, Ap ap, Configuration config) {
exists(ReturnNodeExt ret, Ap ap0, ReturnKindExt kind, int pos |
parameterFlow(p, ap, ap0, c, config) and
c = getNodeEnclosingCallable(ret) and
@@ -1857,6 +1936,15 @@ private module Stage3 {
)
}
pragma[nomagic]
predicate callMayFlowThroughRev(DataFlowCall call, Configuration config) {
exists(Ap returnAp0, ArgNode arg, boolean toReturn, ApOption returnAp, Ap ap |
revFlow(arg, toReturn, returnAp, ap, config) and
revFlowInToReturn(call, arg, returnAp0, ap, config) and
revFlowIsReturned(call, toReturn, returnAp, returnAp0, config)
)
}
predicate stats(boolean fwd, int nodes, int fields, int conscand, int tuples, Configuration config) {
fwd = true and
nodes = count(Node node | fwdFlow(node, _, _, _, config)) and
@@ -2088,7 +2176,7 @@ private module Stage4 {
private ApApprox getApprox(Ap ap) { result = ap.getFront() }
private ApNil getApNil(Node node) {
PrevStage::revFlow(node, _) and result = TNil(getNodeType(node))
PrevStage::revFlow(node, _) and result = TNil(getNodeDataFlowType(node))
}
bindingset[tc, tail]
@@ -2127,8 +2215,6 @@ private module Stage4 {
bindingset[innercc, inner, call]
private predicate checkCallContextReturn(Cc innercc, DataFlowCallable inner, DataFlowCall call) {
resolveReturn(innercc, inner, call)
or
innercc.(CallContextCall).matchesCall(call)
}
bindingset[node, cc, config]
@@ -2155,8 +2241,7 @@ private module Stage4 {
pragma[nomagic]
private predicate flowIntoCall(
DataFlowCall call, ArgumentNode node1, ParameterNode node2, boolean allowsFieldFlow,
Configuration config
DataFlowCall call, ArgNode node1, ParamNode node2, boolean allowsFieldFlow, Configuration config
) {
flowIntoCallNodeCand2(call, node1, node2, allowsFieldFlow, config) and
PrevStage::revFlow(node2, _, _, _, pragma[only_bind_into](config)) and
@@ -2182,6 +2267,16 @@ private module Stage4 {
)
}
pragma[nomagic]
private predicate flowThroughOutOfCall(
DataFlowCall call, ReturnNodeExt ret, Node out, boolean allowsFieldFlow, Configuration config
) {
flowOutOfCall(call, ret, out, allowsFieldFlow, pragma[only_bind_into](config)) and
PrevStage::callMayFlowThroughRev(call, pragma[only_bind_into](config)) and
PrevStage::parameterMayFlowThrough(_, getNodeEnclosingCallable(ret), _,
pragma[only_bind_into](config))
}
/**
* Holds if `node` is reachable with access path `ap` from a source in the
* configuration `config`.
@@ -2255,13 +2350,11 @@ private module Stage4 {
)
or
// flow out of a callable
exists(DataFlowCall call |
fwdFlowOut(call, node, any(CcNoCall innercc), cc, argAp, ap, config)
or
exists(Ap argAp0 |
fwdFlowOutFromArg(call, node, argAp0, ap, config) and
fwdFlowIsEntered(call, cc, argAp, argAp0, config)
)
fwdFlowOutNotFromArg(node, cc, argAp, ap, config)
or
exists(DataFlowCall call, Ap argAp0 |
fwdFlowOutFromArg(call, node, argAp0, ap, config) and
fwdFlowIsEntered(call, cc, argAp, argAp0, config)
)
}
@@ -2300,10 +2393,10 @@ private module Stage4 {
pragma[nomagic]
private predicate fwdFlowIn(
DataFlowCall call, ParameterNode p, Cc outercc, Cc innercc, ApOption argAp, Ap ap,
DataFlowCall call, ParamNode p, Cc outercc, Cc innercc, ApOption argAp, Ap ap,
Configuration config
) {
exists(ArgumentNode arg, boolean allowsFieldFlow |
exists(ArgNode arg, boolean allowsFieldFlow |
fwdFlow(arg, outercc, argAp, ap, config) and
flowIntoCall(call, arg, p, allowsFieldFlow, config) and
innercc = getCallContextCall(call, getNodeEnclosingCallable(p), outercc)
@@ -2312,17 +2405,14 @@ private module Stage4 {
)
}
/**
* Holds if flow may exit from `call` at `out` with access path `ap`. The
* inner call context is `innercc`, but `ccOut` is just the call context
* based on the return step. In the case of through-flow `ccOut` is discarded
* and replaced by the outer call context as tracked by `fwdFlowIsEntered`.
*/
pragma[nomagic]
private predicate fwdFlowOut(
DataFlowCall call, Node out, Cc innercc, Cc ccOut, ApOption argAp, Ap ap, Configuration config
private predicate fwdFlowOutNotFromArg(
Node out, Cc ccOut, ApOption argAp, Ap ap, Configuration config
) {
exists(ReturnNodeExt ret, boolean allowsFieldFlow, DataFlowCallable inner |
exists(
DataFlowCall call, ReturnNodeExt ret, boolean allowsFieldFlow, CcNoCall innercc,
DataFlowCallable inner
|
fwdFlow(ret, innercc, argAp, ap, config) and
flowOutOfCall(call, ret, out, allowsFieldFlow, config) and
inner = getNodeEnclosingCallable(ret) and
@@ -2337,7 +2427,13 @@ private module Stage4 {
private predicate fwdFlowOutFromArg(
DataFlowCall call, Node out, Ap argAp, Ap ap, Configuration config
) {
fwdFlowOut(call, out, any(CcCall ccc), _, apSome(argAp), ap, config)
exists(ReturnNodeExt ret, boolean allowsFieldFlow, CcCall ccc |
fwdFlow(ret, ccc, apSome(argAp), ap, config) and
flowThroughOutOfCall(call, ret, out, allowsFieldFlow, config) and
ccc.matchesCall(call)
|
ap instanceof ApNil or allowsFieldFlow = true
)
}
/**
@@ -2348,7 +2444,7 @@ private module Stage4 {
private predicate fwdFlowIsEntered(
DataFlowCall call, Cc cc, ApOption argAp, Ap ap, Configuration config
) {
exists(ParameterNode p |
exists(ParamNode p |
fwdFlowIn(call, p, cc, _, argAp, ap, config) and
PrevStage::parameterMayFlowThrough(p, _, unbindApa(getApprox(ap)), config)
)
@@ -2368,6 +2464,27 @@ private module Stage4 {
fwdFlowConsCand(ap1, c, ap2, config)
}
pragma[nomagic]
private predicate callMayFlowThroughFwd(DataFlowCall call, Configuration config) {
exists(Ap argAp0, Node out, Cc cc, ApOption argAp, Ap ap |
fwdFlow(out, pragma[only_bind_into](cc), pragma[only_bind_into](argAp), ap,
pragma[only_bind_into](config)) and
fwdFlowOutFromArg(call, out, argAp0, ap, config) and
fwdFlowIsEntered(call, pragma[only_bind_into](cc), pragma[only_bind_into](argAp), argAp0,
pragma[only_bind_into](config))
)
}
pragma[nomagic]
private predicate flowThroughIntoCall(
DataFlowCall call, ArgNode arg, ParamNode p, boolean allowsFieldFlow, Configuration config
) {
flowIntoCall(call, arg, p, allowsFieldFlow, config) and
fwdFlow(arg, _, _, _, pragma[only_bind_into](config)) and
PrevStage::parameterMayFlowThrough(p, _, _, pragma[only_bind_into](config)) and
callMayFlowThroughFwd(call, pragma[only_bind_into](config))
}
/**
* Holds if `node` with access path `ap` is part of a path from a source to a
* sink in the configuration `config`.
@@ -2433,14 +2550,12 @@ private module Stage4 {
)
or
// flow into a callable
exists(DataFlowCall call |
revFlowIn(call, node, toReturn, returnAp, ap, config) and
toReturn = false
or
exists(Ap returnAp0 |
revFlowInToReturn(call, node, returnAp0, ap, config) and
revFlowIsReturned(call, toReturn, returnAp, returnAp0, config)
)
revFlowInNotToReturn(node, returnAp, ap, config) and
toReturn = false
or
exists(DataFlowCall call, Ap returnAp0 |
revFlowInToReturn(call, node, returnAp0, ap, config) and
revFlowIsReturned(call, toReturn, returnAp, returnAp0, config)
)
or
// flow out of a callable
@@ -2488,13 +2603,10 @@ private module Stage4 {
}
pragma[nomagic]
private predicate revFlowIn(
DataFlowCall call, ArgumentNode arg, boolean toReturn, ApOption returnAp, Ap ap,
Configuration config
) {
exists(ParameterNode p, boolean allowsFieldFlow |
revFlow(p, toReturn, returnAp, ap, config) and
flowIntoCall(call, arg, p, allowsFieldFlow, config)
private predicate revFlowInNotToReturn(ArgNode arg, ApOption returnAp, Ap ap, Configuration config) {
exists(ParamNode p, boolean allowsFieldFlow |
revFlow(p, false, returnAp, ap, config) and
flowIntoCall(_, arg, p, allowsFieldFlow, config)
|
ap instanceof ApNil or allowsFieldFlow = true
)
@@ -2502,9 +2614,14 @@ private module Stage4 {
pragma[nomagic]
private predicate revFlowInToReturn(
DataFlowCall call, ArgumentNode arg, Ap returnAp, Ap ap, Configuration config
DataFlowCall call, ArgNode arg, Ap returnAp, Ap ap, Configuration config
) {
revFlowIn(call, arg, true, apSome(returnAp), ap, config)
exists(ParamNode p, boolean allowsFieldFlow |
revFlow(p, true, apSome(returnAp), ap, config) and
flowThroughIntoCall(call, arg, p, allowsFieldFlow, config)
|
ap instanceof ApNil or allowsFieldFlow = true
)
}
/**
@@ -2555,13 +2672,13 @@ private module Stage4 {
pragma[noinline]
private predicate parameterFlow(
ParameterNode p, Ap ap, Ap ap0, DataFlowCallable c, Configuration config
ParamNode p, Ap ap, Ap ap0, DataFlowCallable c, Configuration config
) {
revFlow(p, true, apSome(ap0), ap, config) and
c = getNodeEnclosingCallable(p)
}
predicate parameterMayFlowThrough(ParameterNode p, DataFlowCallable c, Ap ap, Configuration config) {
predicate parameterMayFlowThrough(ParamNode p, DataFlowCallable c, Ap ap, Configuration config) {
exists(ReturnNodeExt ret, Ap ap0, ReturnKindExt kind, int pos |
parameterFlow(p, ap, ap0, c, config) and
c = getNodeEnclosingCallable(ret) and
@@ -2574,6 +2691,15 @@ private module Stage4 {
)
}
pragma[nomagic]
predicate callMayFlowThroughRev(DataFlowCall call, Configuration config) {
exists(Ap returnAp0, ArgNode arg, boolean toReturn, ApOption returnAp, Ap ap |
revFlow(arg, toReturn, returnAp, ap, config) and
revFlowInToReturn(call, arg, returnAp0, ap, config) and
revFlowIsReturned(call, toReturn, returnAp, returnAp0, config)
)
}
predicate stats(boolean fwd, int nodes, int fields, int conscand, int tuples, Configuration config) {
fwd = true and
nodes = count(Node node | fwdFlow(node, _, _, _, config)) and
@@ -2606,7 +2732,7 @@ private predicate nodeMayUseSummary(Node n, AccessPathApprox apa, Configuration
private newtype TSummaryCtx =
TSummaryCtxNone() or
TSummaryCtxSome(ParameterNode p, AccessPath ap) {
TSummaryCtxSome(ParamNode p, AccessPath ap) {
Stage4::parameterMayFlowThrough(p, _, ap.getApprox(), _)
}
@@ -2627,7 +2753,7 @@ private class SummaryCtxNone extends SummaryCtx, TSummaryCtxNone {
/** A summary context from which a flow summary can be generated. */
private class SummaryCtxSome extends SummaryCtx, TSummaryCtxSome {
private ParameterNode p;
private ParamNode p;
private AccessPath ap;
SummaryCtxSome() { this = TSummaryCtxSome(p, ap) }
@@ -2758,7 +2884,7 @@ private newtype TPathNode =
config.isSource(node) and
cc instanceof CallContextAny and
sc instanceof SummaryCtxNone and
ap = TAccessPathNil(getNodeType(node))
ap = TAccessPathNil(getNodeDataFlowType(node))
or
// ... or a step from an existing PathNode to another node.
exists(PathNodeMid mid |
@@ -2979,7 +3105,7 @@ class PathNode extends TPathNode {
Configuration getConfiguration() { none() }
private predicate isHidden() {
nodeIsHidden(this.getNode()) and
hiddenNode(this.getNode()) and
not this.isSource() and
not this instanceof PathNodeSink
}
@@ -3148,7 +3274,7 @@ private predicate pathStep(PathNodeMid mid, Node node, CallContext cc, SummaryCt
cc instanceof CallContextAny and
sc instanceof SummaryCtxNone and
mid.getAp() instanceof AccessPathNil and
ap = TAccessPathNil(getNodeType(node))
ap = TAccessPathNil(getNodeDataFlowType(node))
or
exists(TypedContent tc | pathStoreStep(mid, node, ap.pop(tc), tc, cc)) and
sc = mid.getSummaryCtx()
@@ -3235,7 +3361,7 @@ pragma[noinline]
private predicate pathIntoArg(
PathNodeMid mid, int i, CallContext cc, DataFlowCall call, AccessPath ap, AccessPathApprox apa
) {
exists(ArgumentNode arg |
exists(ArgNode arg |
arg = mid.getNode() and
cc = mid.getCallContext() and
arg.argumentOf(call, i) and
@@ -3248,7 +3374,7 @@ pragma[noinline]
private predicate parameterCand(
DataFlowCallable callable, int i, AccessPathApprox apa, Configuration config
) {
exists(ParameterNode p |
exists(ParamNode p |
Stage4::revFlow(p, _, _, apa, config) and
p.isParameterOf(callable, i)
)
@@ -3272,7 +3398,7 @@ private predicate pathIntoCallable0(
* respectively.
*/
private predicate pathIntoCallable(
PathNodeMid mid, ParameterNode p, CallContext outercc, CallContextCall innercc, SummaryCtx sc,
PathNodeMid mid, ParamNode p, CallContext outercc, CallContextCall innercc, SummaryCtx sc,
DataFlowCall call
) {
exists(int i, DataFlowCallable callable, AccessPath ap |
@@ -3568,7 +3694,7 @@ private module FlowExploration {
private newtype TSummaryCtx1 =
TSummaryCtx1None() or
TSummaryCtx1Param(ParameterNode p)
TSummaryCtx1Param(ParamNode p)
private newtype TSummaryCtx2 =
TSummaryCtx2None() or
@@ -3591,7 +3717,7 @@ private module FlowExploration {
cc instanceof CallContextAny and
sc1 = TSummaryCtx1None() and
sc2 = TSummaryCtx2None() and
ap = TPartialNil(getNodeType(node)) and
ap = TPartialNil(getNodeDataFlowType(node)) and
not fullBarrier(node, config) and
exists(config.explorationLimit())
or
@@ -3611,7 +3737,7 @@ private module FlowExploration {
or
exists(PartialPathNodeRev mid |
revPartialPathStep(mid, node, sc1, sc2, ap, config) and
not clearsContent(node, ap.getHead()) and
not clearsContentCached(node, ap.getHead()) and
not fullBarrier(node, config) and
distSink(getNodeEnclosingCallable(node), config) <= config.explorationLimit()
)
@@ -3625,9 +3751,9 @@ private module FlowExploration {
exists(PartialPathNodeFwd mid |
partialPathStep(mid, node, cc, sc1, sc2, ap, config) and
not fullBarrier(node, config) and
not clearsContent(node, ap.getHead().getContent()) and
not clearsContentCached(node, ap.getHead().getContent()) and
if node instanceof CastingNode
then compatibleTypes(getNodeType(node), ap.getType())
then compatibleTypes(getNodeDataFlowType(node), ap.getType())
else any()
)
}
@@ -3783,7 +3909,7 @@ private module FlowExploration {
PartialPathNodeFwd mid, Node node, CallContext cc, TSummaryCtx1 sc1, TSummaryCtx2 sc2,
PartialAccessPath ap, Configuration config
) {
not isUnreachableInCall(node, cc.(CallContextSpecificCall).getCall()) and
not isUnreachableInCallCached(node, cc.(CallContextSpecificCall).getCall()) and
(
localFlowStep(mid.getNode(), node, config) and
cc = mid.getCallContext() and
@@ -3797,7 +3923,7 @@ private module FlowExploration {
sc1 = mid.getSummaryCtx1() and
sc2 = mid.getSummaryCtx2() and
mid.getAp() instanceof PartialAccessPathNil and
ap = TPartialNil(getNodeType(node)) and
ap = TPartialNil(getNodeDataFlowType(node)) and
config = mid.getConfiguration()
)
or
@@ -3813,7 +3939,7 @@ private module FlowExploration {
sc1 = TSummaryCtx1None() and
sc2 = TSummaryCtx2None() and
mid.getAp() instanceof PartialAccessPathNil and
ap = TPartialNil(getNodeType(node)) and
ap = TPartialNil(getNodeDataFlowType(node)) and
config = mid.getConfiguration()
or
partialPathStoreStep(mid, _, _, node, ap) and
@@ -3827,7 +3953,7 @@ private module FlowExploration {
sc1 = mid.getSummaryCtx1() and
sc2 = mid.getSummaryCtx2() and
apConsFwd(ap, tc, ap0, config) and
compatibleTypes(ap.getType(), getNodeType(node))
compatibleTypes(ap.getType(), getNodeDataFlowType(node))
)
or
partialPathIntoCallable(mid, node, _, cc, sc1, sc2, _, ap, config)
@@ -3924,7 +4050,7 @@ private module FlowExploration {
PartialPathNodeFwd mid, int i, CallContext cc, DataFlowCall call, PartialAccessPath ap,
Configuration config
) {
exists(ArgumentNode arg |
exists(ArgNode arg |
arg = mid.getNode() and
cc = mid.getCallContext() and
arg.argumentOf(call, i) and
@@ -3943,7 +4069,7 @@ private module FlowExploration {
}
private predicate partialPathIntoCallable(
PartialPathNodeFwd mid, ParameterNode p, CallContext outercc, CallContextCall innercc,
PartialPathNodeFwd mid, ParamNode p, CallContext outercc, CallContextCall innercc,
TSummaryCtx1 sc1, TSummaryCtx2 sc2, DataFlowCall call, PartialAccessPath ap,
Configuration config
) {
@@ -3980,7 +4106,7 @@ private module FlowExploration {
DataFlowCall call, PartialPathNodeFwd mid, ReturnKindExt kind, CallContext cc,
PartialAccessPath ap, Configuration config
) {
exists(ParameterNode p, CallContext innercc, TSummaryCtx1 sc1, TSummaryCtx2 sc2 |
exists(ParamNode p, CallContext innercc, TSummaryCtx1 sc1, TSummaryCtx2 sc2 |
partialPathIntoCallable(mid, p, cc, innercc, sc1, sc2, call, _, config) and
paramFlowsThroughInPartialPath(kind, innercc, sc1, sc2, ap, config)
)
@@ -4037,7 +4163,7 @@ private module FlowExploration {
apConsRev(ap, c, ap0, config)
)
or
exists(ParameterNode p |
exists(ParamNode p |
mid.getNode() = p and
viableParamArg(_, p, node) and
sc1 = mid.getSummaryCtx1() and
@@ -4115,7 +4241,7 @@ private module FlowExploration {
int pos, TRevSummaryCtx1Some sc1, TRevSummaryCtx2Some sc2, RevPartialAccessPath ap,
Configuration config
) {
exists(PartialPathNodeRev mid, ParameterNode p |
exists(PartialPathNodeRev mid, ParamNode p |
mid.getNode() = p and
p.isParameterOf(_, pos) and
sc1 = mid.getSummaryCtx1() and
@@ -4138,7 +4264,7 @@ private module FlowExploration {
pragma[nomagic]
private predicate revPartialPathThroughCallable(
PartialPathNodeRev mid, ArgumentNode node, RevPartialAccessPath ap, Configuration config
PartialPathNodeRev mid, ArgNode node, RevPartialAccessPath ap, Configuration config
) {
exists(DataFlowCall call, int pos |
revPartialPathThroughCallable0(call, mid, pos, ap, config) and

476
python/ql/src/semmle/python/dataflow/new/internal/DataFlowImpl3.qll
View File

@@ -211,10 +211,7 @@ private predicate fullBarrier(Node node, Configuration config) {
* Holds if data can flow in one local step from `node1` to `node2`.
*/
private predicate localFlowStep(Node node1, Node node2, Configuration config) {
(
simpleLocalFlowStep(node1, node2) or
reverseStepThroughInputOutputAlias(node1, node2)
) and
simpleLocalFlowStepExt(node1, node2) and
not outBarrier(node1, config) and
not inBarrier(node2, config) and
not fullBarrier(node1, config) and
@@ -237,7 +234,7 @@ private predicate additionalLocalFlowStep(Node node1, Node node2, Configuration
* Holds if data can flow from `node1` to `node2` in a way that discards call contexts.
*/
private predicate jumpStep(Node node1, Node node2, Configuration config) {
jumpStep(node1, node2) and
jumpStepCached(node1, node2) and
not outBarrier(node1, config) and
not inBarrier(node2, config) and
not fullBarrier(node1, config) and
@@ -388,7 +385,7 @@ private module Stage1 {
*/
pragma[nomagic]
private predicate fwdFlowIsEntered(DataFlowCall call, Cc cc, Configuration config) {
exists(ArgumentNode arg |
exists(ArgNode arg |
fwdFlow(arg, cc, config) and
viableParamArg(call, _, arg)
)
@@ -515,29 +512,29 @@ private module Stage1 {
pragma[nomagic]
predicate viableParamArgNodeCandFwd1(
DataFlowCall call, ParameterNode p, ArgumentNode arg, Configuration config
DataFlowCall call, ParamNode p, ArgNode arg, Configuration config
) {
viableParamArg(call, p, arg) and
fwdFlow(arg, config)
}
pragma[nomagic]
private predicate revFlowIn(
DataFlowCall call, ArgumentNode arg, boolean toReturn, Configuration config
) {
exists(ParameterNode p |
private predicate revFlowIn(DataFlowCall call, ArgNode arg, boolean toReturn, Configuration config) {
exists(ParamNode p |
revFlow(p, toReturn, config) and
viableParamArgNodeCandFwd1(call, p, arg, config)
)
}
pragma[nomagic]
private predicate revFlowInToReturn(DataFlowCall call, ArgumentNode arg, Configuration config) {
private predicate revFlowInToReturn(DataFlowCall call, ArgNode arg, Configuration config) {
revFlowIn(call, arg, true, config)
}
/**
* Holds if an output from `call` is reached in the flow covered by `revFlow`.
* Holds if an output from `call` is reached in the flow covered by `revFlow`
* and data might flow through the target callable resulting in reverse flow
* reaching an argument of `call`.
*/
pragma[nomagic]
private predicate revFlowIsReturned(DataFlowCall call, boolean toReturn, Configuration config) {
@@ -597,7 +594,7 @@ private module Stage1 {
* Holds if flow may enter through `p` and reach a return node making `p` a
* candidate for the origin of a summary.
*/
predicate parameterMayFlowThrough(ParameterNode p, DataFlowCallable c, Ap ap, Configuration config) {
predicate parameterMayFlowThrough(ParamNode p, DataFlowCallable c, Ap ap, Configuration config) {
exists(ReturnKindExt kind |
throughFlowNodeCand(p, config) and
returnFlowCallableNodeCand(c, kind, config) and
@@ -610,6 +607,15 @@ private module Stage1 {
)
}
pragma[nomagic]
predicate callMayFlowThroughRev(DataFlowCall call, Configuration config) {
exists(ArgNode arg, boolean toReturn |
revFlow(arg, toReturn, config) and
revFlowInToReturn(call, arg, config) and
revFlowIsReturned(call, toReturn, config)
)
}
predicate stats(boolean fwd, int nodes, int fields, int conscand, int tuples, Configuration config) {
fwd = true and
nodes = count(Node node | fwdFlow(node, config)) and
@@ -663,7 +669,7 @@ private predicate flowOutOfCallNodeCand1(
pragma[nomagic]
private predicate viableParamArgNodeCand1(
DataFlowCall call, ParameterNode p, ArgumentNode arg, Configuration config
DataFlowCall call, ParamNode p, ArgNode arg, Configuration config
) {
Stage1::viableParamArgNodeCandFwd1(call, p, arg, config) and
Stage1::revFlow(arg, config)
@@ -675,7 +681,7 @@ private predicate viableParamArgNodeCand1(
*/
pragma[nomagic]
private predicate flowIntoCallNodeCand1(
DataFlowCall call, ArgumentNode arg, ParameterNode p, Configuration config
DataFlowCall call, ArgNode arg, ParamNode p, Configuration config
) {
viableParamArgNodeCand1(call, p, arg, config) and
Stage1::revFlow(p, config) and
@@ -735,8 +741,7 @@ private predicate flowOutOfCallNodeCand1(
*/
pragma[nomagic]
private predicate flowIntoCallNodeCand1(
DataFlowCall call, ArgumentNode arg, ParameterNode p, boolean allowsFieldFlow,
Configuration config
DataFlowCall call, ArgNode arg, ParamNode p, boolean allowsFieldFlow, Configuration config
) {
flowIntoCallNodeCand1(call, arg, p, config) and
exists(int b, int j |
@@ -833,6 +838,16 @@ private module Stage2 {
PrevStage::revFlow(node, _, _, apa, config)
}
pragma[nomagic]
private predicate flowThroughOutOfCall(
DataFlowCall call, ReturnNodeExt ret, Node out, boolean allowsFieldFlow, Configuration config
) {
flowOutOfCall(call, ret, out, allowsFieldFlow, pragma[only_bind_into](config)) and
PrevStage::callMayFlowThroughRev(call, pragma[only_bind_into](config)) and
PrevStage::parameterMayFlowThrough(_, getNodeEnclosingCallable(ret), _,
pragma[only_bind_into](config))
}
/**
* Holds if `node` is reachable with access path `ap` from a source in the
* configuration `config`.
@@ -899,13 +914,11 @@ private module Stage2 {
)
or
// flow out of a callable
exists(DataFlowCall call |
fwdFlowOut(call, node, any(CcNoCall innercc), cc, argAp, ap, config)
or
exists(Ap argAp0 |
fwdFlowOutFromArg(call, node, argAp0, ap, config) and
fwdFlowIsEntered(call, cc, argAp, argAp0, config)
)
fwdFlowOutNotFromArg(node, cc, argAp, ap, config)
or
exists(DataFlowCall call, Ap argAp0 |
fwdFlowOutFromArg(call, node, argAp0, ap, config) and
fwdFlowIsEntered(call, cc, argAp, argAp0, config)
)
}
@@ -944,10 +957,10 @@ private module Stage2 {
pragma[nomagic]
private predicate fwdFlowIn(
DataFlowCall call, ParameterNode p, Cc outercc, Cc innercc, ApOption argAp, Ap ap,
DataFlowCall call, ParamNode p, Cc outercc, Cc innercc, ApOption argAp, Ap ap,
Configuration config
) {
exists(ArgumentNode arg, boolean allowsFieldFlow |
exists(ArgNode arg, boolean allowsFieldFlow |
fwdFlow(arg, outercc, argAp, ap, config) and
flowIntoCall(call, arg, p, allowsFieldFlow, config) and
innercc = getCallContextCall(call, getNodeEnclosingCallable(p), outercc)
@@ -956,17 +969,14 @@ private module Stage2 {
)
}
/**
* Holds if flow may exit from `call` at `out` with access path `ap`. The
* inner call context is `innercc`, but `ccOut` is just the call context
* based on the return step. In the case of through-flow `ccOut` is discarded
* and replaced by the outer call context as tracked by `fwdFlowIsEntered`.
*/
pragma[nomagic]
private predicate fwdFlowOut(
DataFlowCall call, Node out, Cc innercc, Cc ccOut, ApOption argAp, Ap ap, Configuration config
private predicate fwdFlowOutNotFromArg(
Node out, Cc ccOut, ApOption argAp, Ap ap, Configuration config
) {
exists(ReturnNodeExt ret, boolean allowsFieldFlow, DataFlowCallable inner |
exists(
DataFlowCall call, ReturnNodeExt ret, boolean allowsFieldFlow, CcNoCall innercc,
DataFlowCallable inner
|
fwdFlow(ret, innercc, argAp, ap, config) and
flowOutOfCall(call, ret, out, allowsFieldFlow, config) and
inner = getNodeEnclosingCallable(ret) and
@@ -981,7 +991,13 @@ private module Stage2 {
private predicate fwdFlowOutFromArg(
DataFlowCall call, Node out, Ap argAp, Ap ap, Configuration config
) {
fwdFlowOut(call, out, any(CcCall ccc), _, apSome(argAp), ap, config)
exists(ReturnNodeExt ret, boolean allowsFieldFlow, CcCall ccc |
fwdFlow(ret, ccc, apSome(argAp), ap, config) and
flowThroughOutOfCall(call, ret, out, allowsFieldFlow, config) and
ccc.matchesCall(call)
|
ap instanceof ApNil or allowsFieldFlow = true
)
}
/**
@@ -992,7 +1008,7 @@ private module Stage2 {
private predicate fwdFlowIsEntered(
DataFlowCall call, Cc cc, ApOption argAp, Ap ap, Configuration config
) {
exists(ParameterNode p |
exists(ParamNode p |
fwdFlowIn(call, p, cc, _, argAp, ap, config) and
PrevStage::parameterMayFlowThrough(p, _, getApprox(ap), config)
)
@@ -1012,6 +1028,27 @@ private module Stage2 {
fwdFlowConsCand(ap1, c, ap2, config)
}
pragma[nomagic]
private predicate callMayFlowThroughFwd(DataFlowCall call, Configuration config) {
exists(Ap argAp0, Node out, Cc cc, ApOption argAp, Ap ap |
fwdFlow(out, pragma[only_bind_into](cc), pragma[only_bind_into](argAp), ap,
pragma[only_bind_into](config)) and
fwdFlowOutFromArg(call, out, argAp0, ap, config) and
fwdFlowIsEntered(call, pragma[only_bind_into](cc), pragma[only_bind_into](argAp), argAp0,
pragma[only_bind_into](config))
)
}
pragma[nomagic]
private predicate flowThroughIntoCall(
DataFlowCall call, ArgNode arg, ParamNode p, boolean allowsFieldFlow, Configuration config
) {
flowIntoCall(call, arg, p, allowsFieldFlow, config) and
fwdFlow(arg, _, _, _, pragma[only_bind_into](config)) and
PrevStage::parameterMayFlowThrough(p, _, _, pragma[only_bind_into](config)) and
callMayFlowThroughFwd(call, pragma[only_bind_into](config))
}
/**
* Holds if `node` with access path `ap` is part of a path from a source to a
* sink in the configuration `config`.
@@ -1077,14 +1114,12 @@ private module Stage2 {
)
or
// flow into a callable
exists(DataFlowCall call |
revFlowIn(call, node, toReturn, returnAp, ap, config) and
toReturn = false
or
exists(Ap returnAp0 |
revFlowInToReturn(call, node, returnAp0, ap, config) and
revFlowIsReturned(call, toReturn, returnAp, returnAp0, config)
)
revFlowInNotToReturn(node, returnAp, ap, config) and
toReturn = false
or
exists(DataFlowCall call, Ap returnAp0 |
revFlowInToReturn(call, node, returnAp0, ap, config) and
revFlowIsReturned(call, toReturn, returnAp, returnAp0, config)
)
or
// flow out of a callable
@@ -1132,13 +1167,10 @@ private module Stage2 {
}
pragma[nomagic]
private predicate revFlowIn(
DataFlowCall call, ArgumentNode arg, boolean toReturn, ApOption returnAp, Ap ap,
Configuration config
) {
exists(ParameterNode p, boolean allowsFieldFlow |
revFlow(p, toReturn, returnAp, ap, config) and
flowIntoCall(call, arg, p, allowsFieldFlow, config)
private predicate revFlowInNotToReturn(ArgNode arg, ApOption returnAp, Ap ap, Configuration config) {
exists(ParamNode p, boolean allowsFieldFlow |
revFlow(p, false, returnAp, ap, config) and
flowIntoCall(_, arg, p, allowsFieldFlow, config)
|
ap instanceof ApNil or allowsFieldFlow = true
)
@@ -1146,9 +1178,14 @@ private module Stage2 {
pragma[nomagic]
private predicate revFlowInToReturn(
DataFlowCall call, ArgumentNode arg, Ap returnAp, Ap ap, Configuration config
DataFlowCall call, ArgNode arg, Ap returnAp, Ap ap, Configuration config
) {
revFlowIn(call, arg, true, apSome(returnAp), ap, config)
exists(ParamNode p, boolean allowsFieldFlow |
revFlow(p, true, apSome(returnAp), ap, config) and
flowThroughIntoCall(call, arg, p, allowsFieldFlow, config)
|
ap instanceof ApNil or allowsFieldFlow = true
)
}
/**
@@ -1199,13 +1236,13 @@ private module Stage2 {
pragma[noinline]
private predicate parameterFlow(
ParameterNode p, Ap ap, Ap ap0, DataFlowCallable c, Configuration config
ParamNode p, Ap ap, Ap ap0, DataFlowCallable c, Configuration config
) {
revFlow(p, true, apSome(ap0), ap, config) and
c = getNodeEnclosingCallable(p)
}
predicate parameterMayFlowThrough(ParameterNode p, DataFlowCallable c, Ap ap, Configuration config) {
predicate parameterMayFlowThrough(ParamNode p, DataFlowCallable c, Ap ap, Configuration config) {
exists(ReturnNodeExt ret, Ap ap0, ReturnKindExt kind, int pos |
parameterFlow(p, ap, ap0, c, config) and
c = getNodeEnclosingCallable(ret) and
@@ -1218,6 +1255,15 @@ private module Stage2 {
)
}
pragma[nomagic]
predicate callMayFlowThroughRev(DataFlowCall call, Configuration config) {
exists(Ap returnAp0, ArgNode arg, boolean toReturn, ApOption returnAp, Ap ap |
revFlow(arg, toReturn, returnAp, ap, config) and
revFlowInToReturn(call, arg, returnAp0, ap, config) and
revFlowIsReturned(call, toReturn, returnAp, returnAp0, config)
)
}
predicate stats(boolean fwd, int nodes, int fields, int conscand, int tuples, Configuration config) {
fwd = true and
nodes = count(Node node | fwdFlow(node, _, _, _, config)) and
@@ -1245,8 +1291,7 @@ private predicate flowOutOfCallNodeCand2(
pragma[nomagic]
private predicate flowIntoCallNodeCand2(
DataFlowCall call, ArgumentNode node1, ParameterNode node2, boolean allowsFieldFlow,
Configuration config
DataFlowCall call, ArgNode node1, ParamNode node2, boolean allowsFieldFlow, Configuration config
) {
flowIntoCallNodeCand1(call, node1, node2, allowsFieldFlow, config) and
Stage2::revFlow(node2, pragma[only_bind_into](config)) and
@@ -1260,8 +1305,8 @@ private module LocalFlowBigStep {
*/
private class FlowCheckNode extends Node {
FlowCheckNode() {
this instanceof CastNode or
clearsContent(this, _)
castNode(this) or
clearsContentCached(this, _)
}
}
@@ -1275,7 +1320,7 @@ private module LocalFlowBigStep {
config.isSource(node) or
jumpStep(_, node, config) or
additionalJumpStep(_, node, config) or
node instanceof ParameterNode or
node instanceof ParamNode or
node instanceof OutNodeExt or
store(_, _, node, _) or
read(_, _, node) or
@@ -1321,21 +1366,21 @@ private module LocalFlowBigStep {
Node node1, Node node2, boolean preservesValue, DataFlowType t, Configuration config,
LocalCallContext cc
) {
not isUnreachableInCall(node2, cc.(LocalCallContextSpecificCall).getCall()) and
not isUnreachableInCallCached(node2, cc.(LocalCallContextSpecificCall).getCall()) and
(
localFlowEntry(node1, pragma[only_bind_into](config)) and
(
localFlowStepNodeCand1(node1, node2, config) and
preservesValue = true and
t = getNodeType(node1)
t = getNodeDataFlowType(node1)
or
additionalLocalFlowStepNodeCand2(node1, node2, config) and
preservesValue = false and
t = getNodeType(node2)
t = getNodeDataFlowType(node2)
) and
node1 != node2 and
cc.relevantFor(getNodeEnclosingCallable(node1)) and
not isUnreachableInCall(node1, cc.(LocalCallContextSpecificCall).getCall()) and
not isUnreachableInCallCached(node1, cc.(LocalCallContextSpecificCall).getCall()) and
Stage2::revFlow(node2, pragma[only_bind_into](config))
or
exists(Node mid |
@@ -1350,7 +1395,7 @@ private module LocalFlowBigStep {
additionalLocalFlowStepNodeCand2(mid, node2, config) and
not mid instanceof FlowCheckNode and
preservesValue = false and
t = getNodeType(node2) and
t = getNodeDataFlowType(node2) and
Stage2::revFlow(node2, pragma[only_bind_into](config))
)
)
@@ -1384,7 +1429,7 @@ private module Stage3 {
private ApApprox getApprox(Ap ap) { result = ap.toBoolNonEmpty() }
private ApNil getApNil(Node node) {
PrevStage::revFlow(node, _) and result = TFrontNil(getNodeType(node))
PrevStage::revFlow(node, _) and result = TFrontNil(getNodeDataFlowType(node))
}
bindingset[tc, tail]
@@ -1443,7 +1488,9 @@ private module Stage3 {
bindingset[node, ap]
private predicate filter(Node node, Ap ap) {
not ap.isClearedAt(node) and
if node instanceof CastingNode then compatibleTypes(getNodeType(node), ap.getType()) else any()
if node instanceof CastingNode
then compatibleTypes(getNodeDataFlowType(node), ap.getType())
else any()
}
bindingset[ap, contentType]
@@ -1465,6 +1512,16 @@ private module Stage3 {
)
}
pragma[nomagic]
private predicate flowThroughOutOfCall(
DataFlowCall call, ReturnNodeExt ret, Node out, boolean allowsFieldFlow, Configuration config
) {
flowOutOfCall(call, ret, out, allowsFieldFlow, pragma[only_bind_into](config)) and
PrevStage::callMayFlowThroughRev(call, pragma[only_bind_into](config)) and
PrevStage::parameterMayFlowThrough(_, getNodeEnclosingCallable(ret), _,
pragma[only_bind_into](config))
}
/**
* Holds if `node` is reachable with access path `ap` from a source in the
* configuration `config`.
@@ -1538,13 +1595,11 @@ private module Stage3 {
)
or
// flow out of a callable
exists(DataFlowCall call |
fwdFlowOut(call, node, any(CcNoCall innercc), cc, argAp, ap, config)
or
exists(Ap argAp0 |
fwdFlowOutFromArg(call, node, argAp0, ap, config) and
fwdFlowIsEntered(call, cc, argAp, argAp0, config)
)
fwdFlowOutNotFromArg(node, cc, argAp, ap, config)
or
exists(DataFlowCall call, Ap argAp0 |
fwdFlowOutFromArg(call, node, argAp0, ap, config) and
fwdFlowIsEntered(call, cc, argAp, argAp0, config)
)
}
@@ -1583,10 +1638,10 @@ private module Stage3 {
pragma[nomagic]
private predicate fwdFlowIn(
DataFlowCall call, ParameterNode p, Cc outercc, Cc innercc, ApOption argAp, Ap ap,
DataFlowCall call, ParamNode p, Cc outercc, Cc innercc, ApOption argAp, Ap ap,
Configuration config
) {
exists(ArgumentNode arg, boolean allowsFieldFlow |
exists(ArgNode arg, boolean allowsFieldFlow |
fwdFlow(arg, outercc, argAp, ap, config) and
flowIntoCall(call, arg, p, allowsFieldFlow, config) and
innercc = getCallContextCall(call, getNodeEnclosingCallable(p), outercc)
@@ -1595,17 +1650,14 @@ private module Stage3 {
)
}
/**
* Holds if flow may exit from `call` at `out` with access path `ap`. The
* inner call context is `innercc`, but `ccOut` is just the call context
* based on the return step. In the case of through-flow `ccOut` is discarded
* and replaced by the outer call context as tracked by `fwdFlowIsEntered`.
*/
pragma[nomagic]
private predicate fwdFlowOut(
DataFlowCall call, Node out, Cc innercc, Cc ccOut, ApOption argAp, Ap ap, Configuration config
private predicate fwdFlowOutNotFromArg(
Node out, Cc ccOut, ApOption argAp, Ap ap, Configuration config
) {
exists(ReturnNodeExt ret, boolean allowsFieldFlow, DataFlowCallable inner |
exists(
DataFlowCall call, ReturnNodeExt ret, boolean allowsFieldFlow, CcNoCall innercc,
DataFlowCallable inner
|
fwdFlow(ret, innercc, argAp, ap, config) and
flowOutOfCall(call, ret, out, allowsFieldFlow, config) and
inner = getNodeEnclosingCallable(ret) and
@@ -1620,7 +1672,13 @@ private module Stage3 {
private predicate fwdFlowOutFromArg(
DataFlowCall call, Node out, Ap argAp, Ap ap, Configuration config
) {
fwdFlowOut(call, out, any(CcCall ccc), _, apSome(argAp), ap, config)
exists(ReturnNodeExt ret, boolean allowsFieldFlow, CcCall ccc |
fwdFlow(ret, ccc, apSome(argAp), ap, config) and
flowThroughOutOfCall(call, ret, out, allowsFieldFlow, config) and
ccc.matchesCall(call)
|
ap instanceof ApNil or allowsFieldFlow = true
)
}
/**
@@ -1631,7 +1689,7 @@ private module Stage3 {
private predicate fwdFlowIsEntered(
DataFlowCall call, Cc cc, ApOption argAp, Ap ap, Configuration config
) {
exists(ParameterNode p |
exists(ParamNode p |
fwdFlowIn(call, p, cc, _, argAp, ap, config) and
PrevStage::parameterMayFlowThrough(p, _, unbindApa(getApprox(ap)), config)
)
@@ -1651,6 +1709,27 @@ private module Stage3 {
fwdFlowConsCand(ap1, c, ap2, config)
}
pragma[nomagic]
private predicate callMayFlowThroughFwd(DataFlowCall call, Configuration config) {
exists(Ap argAp0, Node out, Cc cc, ApOption argAp, Ap ap |
fwdFlow(out, pragma[only_bind_into](cc), pragma[only_bind_into](argAp), ap,
pragma[only_bind_into](config)) and
fwdFlowOutFromArg(call, out, argAp0, ap, config) and
fwdFlowIsEntered(call, pragma[only_bind_into](cc), pragma[only_bind_into](argAp), argAp0,
pragma[only_bind_into](config))
)
}
pragma[nomagic]
private predicate flowThroughIntoCall(
DataFlowCall call, ArgNode arg, ParamNode p, boolean allowsFieldFlow, Configuration config
) {
flowIntoCall(call, arg, p, allowsFieldFlow, config) and
fwdFlow(arg, _, _, _, pragma[only_bind_into](config)) and
PrevStage::parameterMayFlowThrough(p, _, _, pragma[only_bind_into](config)) and
callMayFlowThroughFwd(call, pragma[only_bind_into](config))
}
/**
* Holds if `node` with access path `ap` is part of a path from a source to a
* sink in the configuration `config`.
@@ -1716,14 +1795,12 @@ private module Stage3 {
)
or
// flow into a callable
exists(DataFlowCall call |
revFlowIn(call, node, toReturn, returnAp, ap, config) and
toReturn = false
or
exists(Ap returnAp0 |
revFlowInToReturn(call, node, returnAp0, ap, config) and
revFlowIsReturned(call, toReturn, returnAp, returnAp0, config)
)
revFlowInNotToReturn(node, returnAp, ap, config) and
toReturn = false
or
exists(DataFlowCall call, Ap returnAp0 |
revFlowInToReturn(call, node, returnAp0, ap, config) and
revFlowIsReturned(call, toReturn, returnAp, returnAp0, config)
)
or
// flow out of a callable
@@ -1771,13 +1848,10 @@ private module Stage3 {
}
pragma[nomagic]
private predicate revFlowIn(
DataFlowCall call, ArgumentNode arg, boolean toReturn, ApOption returnAp, Ap ap,
Configuration config
) {
exists(ParameterNode p, boolean allowsFieldFlow |
revFlow(p, toReturn, returnAp, ap, config) and
flowIntoCall(call, arg, p, allowsFieldFlow, config)
private predicate revFlowInNotToReturn(ArgNode arg, ApOption returnAp, Ap ap, Configuration config) {
exists(ParamNode p, boolean allowsFieldFlow |
revFlow(p, false, returnAp, ap, config) and
flowIntoCall(_, arg, p, allowsFieldFlow, config)
|
ap instanceof ApNil or allowsFieldFlow = true
)
@@ -1785,9 +1859,14 @@ private module Stage3 {
pragma[nomagic]
private predicate revFlowInToReturn(
DataFlowCall call, ArgumentNode arg, Ap returnAp, Ap ap, Configuration config
DataFlowCall call, ArgNode arg, Ap returnAp, Ap ap, Configuration config
) {
revFlowIn(call, arg, true, apSome(returnAp), ap, config)
exists(ParamNode p, boolean allowsFieldFlow |
revFlow(p, true, apSome(returnAp), ap, config) and
flowThroughIntoCall(call, arg, p, allowsFieldFlow, config)
|
ap instanceof ApNil or allowsFieldFlow = true
)
}
/**
@@ -1838,13 +1917,13 @@ private module Stage3 {
pragma[noinline]
private predicate parameterFlow(
ParameterNode p, Ap ap, Ap ap0, DataFlowCallable c, Configuration config
ParamNode p, Ap ap, Ap ap0, DataFlowCallable c, Configuration config
) {
revFlow(p, true, apSome(ap0), ap, config) and
c = getNodeEnclosingCallable(p)
}
predicate parameterMayFlowThrough(ParameterNode p, DataFlowCallable c, Ap ap, Configuration config) {
predicate parameterMayFlowThrough(ParamNode p, DataFlowCallable c, Ap ap, Configuration config) {
exists(ReturnNodeExt ret, Ap ap0, ReturnKindExt kind, int pos |
parameterFlow(p, ap, ap0, c, config) and
c = getNodeEnclosingCallable(ret) and
@@ -1857,6 +1936,15 @@ private module Stage3 {
)
}
pragma[nomagic]
predicate callMayFlowThroughRev(DataFlowCall call, Configuration config) {
exists(Ap returnAp0, ArgNode arg, boolean toReturn, ApOption returnAp, Ap ap |
revFlow(arg, toReturn, returnAp, ap, config) and
revFlowInToReturn(call, arg, returnAp0, ap, config) and
revFlowIsReturned(call, toReturn, returnAp, returnAp0, config)
)
}
predicate stats(boolean fwd, int nodes, int fields, int conscand, int tuples, Configuration config) {
fwd = true and
nodes = count(Node node | fwdFlow(node, _, _, _, config)) and
@@ -2088,7 +2176,7 @@ private module Stage4 {
private ApApprox getApprox(Ap ap) { result = ap.getFront() }
private ApNil getApNil(Node node) {
PrevStage::revFlow(node, _) and result = TNil(getNodeType(node))
PrevStage::revFlow(node, _) and result = TNil(getNodeDataFlowType(node))
}
bindingset[tc, tail]
@@ -2127,8 +2215,6 @@ private module Stage4 {
bindingset[innercc, inner, call]
private predicate checkCallContextReturn(Cc innercc, DataFlowCallable inner, DataFlowCall call) {
resolveReturn(innercc, inner, call)
or
innercc.(CallContextCall).matchesCall(call)
}
bindingset[node, cc, config]
@@ -2155,8 +2241,7 @@ private module Stage4 {
pragma[nomagic]
private predicate flowIntoCall(
DataFlowCall call, ArgumentNode node1, ParameterNode node2, boolean allowsFieldFlow,
Configuration config
DataFlowCall call, ArgNode node1, ParamNode node2, boolean allowsFieldFlow, Configuration config
) {
flowIntoCallNodeCand2(call, node1, node2, allowsFieldFlow, config) and
PrevStage::revFlow(node2, _, _, _, pragma[only_bind_into](config)) and
@@ -2182,6 +2267,16 @@ private module Stage4 {
)
}
pragma[nomagic]
private predicate flowThroughOutOfCall(
DataFlowCall call, ReturnNodeExt ret, Node out, boolean allowsFieldFlow, Configuration config
) {
flowOutOfCall(call, ret, out, allowsFieldFlow, pragma[only_bind_into](config)) and
PrevStage::callMayFlowThroughRev(call, pragma[only_bind_into](config)) and
PrevStage::parameterMayFlowThrough(_, getNodeEnclosingCallable(ret), _,
pragma[only_bind_into](config))
}
/**
* Holds if `node` is reachable with access path `ap` from a source in the
* configuration `config`.
@@ -2255,13 +2350,11 @@ private module Stage4 {
)
or
// flow out of a callable
exists(DataFlowCall call |
fwdFlowOut(call, node, any(CcNoCall innercc), cc, argAp, ap, config)
or
exists(Ap argAp0 |
fwdFlowOutFromArg(call, node, argAp0, ap, config) and
fwdFlowIsEntered(call, cc, argAp, argAp0, config)
)
fwdFlowOutNotFromArg(node, cc, argAp, ap, config)
or
exists(DataFlowCall call, Ap argAp0 |
fwdFlowOutFromArg(call, node, argAp0, ap, config) and
fwdFlowIsEntered(call, cc, argAp, argAp0, config)
)
}
@@ -2300,10 +2393,10 @@ private module Stage4 {
pragma[nomagic]
private predicate fwdFlowIn(
DataFlowCall call, ParameterNode p, Cc outercc, Cc innercc, ApOption argAp, Ap ap,
DataFlowCall call, ParamNode p, Cc outercc, Cc innercc, ApOption argAp, Ap ap,
Configuration config
) {
exists(ArgumentNode arg, boolean allowsFieldFlow |
exists(ArgNode arg, boolean allowsFieldFlow |
fwdFlow(arg, outercc, argAp, ap, config) and
flowIntoCall(call, arg, p, allowsFieldFlow, config) and
innercc = getCallContextCall(call, getNodeEnclosingCallable(p), outercc)
@@ -2312,17 +2405,14 @@ private module Stage4 {
)
}
/**
* Holds if flow may exit from `call` at `out` with access path `ap`. The
* inner call context is `innercc`, but `ccOut` is just the call context
* based on the return step. In the case of through-flow `ccOut` is discarded
* and replaced by the outer call context as tracked by `fwdFlowIsEntered`.
*/
pragma[nomagic]
private predicate fwdFlowOut(
DataFlowCall call, Node out, Cc innercc, Cc ccOut, ApOption argAp, Ap ap, Configuration config
private predicate fwdFlowOutNotFromArg(
Node out, Cc ccOut, ApOption argAp, Ap ap, Configuration config
) {
exists(ReturnNodeExt ret, boolean allowsFieldFlow, DataFlowCallable inner |
exists(
DataFlowCall call, ReturnNodeExt ret, boolean allowsFieldFlow, CcNoCall innercc,
DataFlowCallable inner
|
fwdFlow(ret, innercc, argAp, ap, config) and
flowOutOfCall(call, ret, out, allowsFieldFlow, config) and
inner = getNodeEnclosingCallable(ret) and
@@ -2337,7 +2427,13 @@ private module Stage4 {
private predicate fwdFlowOutFromArg(
DataFlowCall call, Node out, Ap argAp, Ap ap, Configuration config
) {
fwdFlowOut(call, out, any(CcCall ccc), _, apSome(argAp), ap, config)
exists(ReturnNodeExt ret, boolean allowsFieldFlow, CcCall ccc |
fwdFlow(ret, ccc, apSome(argAp), ap, config) and
flowThroughOutOfCall(call, ret, out, allowsFieldFlow, config) and
ccc.matchesCall(call)
|
ap instanceof ApNil or allowsFieldFlow = true
)
}
/**
@@ -2348,7 +2444,7 @@ private module Stage4 {
private predicate fwdFlowIsEntered(
DataFlowCall call, Cc cc, ApOption argAp, Ap ap, Configuration config
) {
exists(ParameterNode p |
exists(ParamNode p |
fwdFlowIn(call, p, cc, _, argAp, ap, config) and
PrevStage::parameterMayFlowThrough(p, _, unbindApa(getApprox(ap)), config)
)
@@ -2368,6 +2464,27 @@ private module Stage4 {
fwdFlowConsCand(ap1, c, ap2, config)
}
pragma[nomagic]
private predicate callMayFlowThroughFwd(DataFlowCall call, Configuration config) {
exists(Ap argAp0, Node out, Cc cc, ApOption argAp, Ap ap |
fwdFlow(out, pragma[only_bind_into](cc), pragma[only_bind_into](argAp), ap,
pragma[only_bind_into](config)) and
fwdFlowOutFromArg(call, out, argAp0, ap, config) and
fwdFlowIsEntered(call, pragma[only_bind_into](cc), pragma[only_bind_into](argAp), argAp0,
pragma[only_bind_into](config))
)
}
pragma[nomagic]
private predicate flowThroughIntoCall(
DataFlowCall call, ArgNode arg, ParamNode p, boolean allowsFieldFlow, Configuration config
) {
flowIntoCall(call, arg, p, allowsFieldFlow, config) and
fwdFlow(arg, _, _, _, pragma[only_bind_into](config)) and
PrevStage::parameterMayFlowThrough(p, _, _, pragma[only_bind_into](config)) and
callMayFlowThroughFwd(call, pragma[only_bind_into](config))
}
/**
* Holds if `node` with access path `ap` is part of a path from a source to a
* sink in the configuration `config`.
@@ -2433,14 +2550,12 @@ private module Stage4 {
)
or
// flow into a callable
exists(DataFlowCall call |
revFlowIn(call, node, toReturn, returnAp, ap, config) and
toReturn = false
or
exists(Ap returnAp0 |
revFlowInToReturn(call, node, returnAp0, ap, config) and
revFlowIsReturned(call, toReturn, returnAp, returnAp0, config)
)
revFlowInNotToReturn(node, returnAp, ap, config) and
toReturn = false
or
exists(DataFlowCall call, Ap returnAp0 |
revFlowInToReturn(call, node, returnAp0, ap, config) and
revFlowIsReturned(call, toReturn, returnAp, returnAp0, config)
)
or
// flow out of a callable
@@ -2488,13 +2603,10 @@ private module Stage4 {
}
pragma[nomagic]
private predicate revFlowIn(
DataFlowCall call, ArgumentNode arg, boolean toReturn, ApOption returnAp, Ap ap,
Configuration config
) {
exists(ParameterNode p, boolean allowsFieldFlow |
revFlow(p, toReturn, returnAp, ap, config) and
flowIntoCall(call, arg, p, allowsFieldFlow, config)
private predicate revFlowInNotToReturn(ArgNode arg, ApOption returnAp, Ap ap, Configuration config) {
exists(ParamNode p, boolean allowsFieldFlow |
revFlow(p, false, returnAp, ap, config) and
flowIntoCall(_, arg, p, allowsFieldFlow, config)
|
ap instanceof ApNil or allowsFieldFlow = true
)
@@ -2502,9 +2614,14 @@ private module Stage4 {
pragma[nomagic]
private predicate revFlowInToReturn(
DataFlowCall call, ArgumentNode arg, Ap returnAp, Ap ap, Configuration config
DataFlowCall call, ArgNode arg, Ap returnAp, Ap ap, Configuration config
) {
revFlowIn(call, arg, true, apSome(returnAp), ap, config)
exists(ParamNode p, boolean allowsFieldFlow |
revFlow(p, true, apSome(returnAp), ap, config) and
flowThroughIntoCall(call, arg, p, allowsFieldFlow, config)
|
ap instanceof ApNil or allowsFieldFlow = true
)
}
/**
@@ -2555,13 +2672,13 @@ private module Stage4 {
pragma[noinline]
private predicate parameterFlow(
ParameterNode p, Ap ap, Ap ap0, DataFlowCallable c, Configuration config
ParamNode p, Ap ap, Ap ap0, DataFlowCallable c, Configuration config
) {
revFlow(p, true, apSome(ap0), ap, config) and
c = getNodeEnclosingCallable(p)
}
predicate parameterMayFlowThrough(ParameterNode p, DataFlowCallable c, Ap ap, Configuration config) {
predicate parameterMayFlowThrough(ParamNode p, DataFlowCallable c, Ap ap, Configuration config) {
exists(ReturnNodeExt ret, Ap ap0, ReturnKindExt kind, int pos |
parameterFlow(p, ap, ap0, c, config) and
c = getNodeEnclosingCallable(ret) and
@@ -2574,6 +2691,15 @@ private module Stage4 {
)
}
pragma[nomagic]
predicate callMayFlowThroughRev(DataFlowCall call, Configuration config) {
exists(Ap returnAp0, ArgNode arg, boolean toReturn, ApOption returnAp, Ap ap |
revFlow(arg, toReturn, returnAp, ap, config) and
revFlowInToReturn(call, arg, returnAp0, ap, config) and
revFlowIsReturned(call, toReturn, returnAp, returnAp0, config)
)
}
predicate stats(boolean fwd, int nodes, int fields, int conscand, int tuples, Configuration config) {
fwd = true and
nodes = count(Node node | fwdFlow(node, _, _, _, config)) and
@@ -2606,7 +2732,7 @@ private predicate nodeMayUseSummary(Node n, AccessPathApprox apa, Configuration
private newtype TSummaryCtx =
TSummaryCtxNone() or
TSummaryCtxSome(ParameterNode p, AccessPath ap) {
TSummaryCtxSome(ParamNode p, AccessPath ap) {
Stage4::parameterMayFlowThrough(p, _, ap.getApprox(), _)
}
@@ -2627,7 +2753,7 @@ private class SummaryCtxNone extends SummaryCtx, TSummaryCtxNone {
/** A summary context from which a flow summary can be generated. */
private class SummaryCtxSome extends SummaryCtx, TSummaryCtxSome {
private ParameterNode p;
private ParamNode p;
private AccessPath ap;
SummaryCtxSome() { this = TSummaryCtxSome(p, ap) }
@@ -2758,7 +2884,7 @@ private newtype TPathNode =
config.isSource(node) and
cc instanceof CallContextAny and
sc instanceof SummaryCtxNone and
ap = TAccessPathNil(getNodeType(node))
ap = TAccessPathNil(getNodeDataFlowType(node))
or
// ... or a step from an existing PathNode to another node.
exists(PathNodeMid mid |
@@ -2979,7 +3105,7 @@ class PathNode extends TPathNode {
Configuration getConfiguration() { none() }
private predicate isHidden() {
nodeIsHidden(this.getNode()) and
hiddenNode(this.getNode()) and
not this.isSource() and
not this instanceof PathNodeSink
}
@@ -3148,7 +3274,7 @@ private predicate pathStep(PathNodeMid mid, Node node, CallContext cc, SummaryCt
cc instanceof CallContextAny and
sc instanceof SummaryCtxNone and
mid.getAp() instanceof AccessPathNil and
ap = TAccessPathNil(getNodeType(node))
ap = TAccessPathNil(getNodeDataFlowType(node))
or
exists(TypedContent tc | pathStoreStep(mid, node, ap.pop(tc), tc, cc)) and
sc = mid.getSummaryCtx()
@@ -3235,7 +3361,7 @@ pragma[noinline]
private predicate pathIntoArg(
PathNodeMid mid, int i, CallContext cc, DataFlowCall call, AccessPath ap, AccessPathApprox apa
) {
exists(ArgumentNode arg |
exists(ArgNode arg |
arg = mid.getNode() and
cc = mid.getCallContext() and
arg.argumentOf(call, i) and
@@ -3248,7 +3374,7 @@ pragma[noinline]
private predicate parameterCand(
DataFlowCallable callable, int i, AccessPathApprox apa, Configuration config
) {
exists(ParameterNode p |
exists(ParamNode p |
Stage4::revFlow(p, _, _, apa, config) and
p.isParameterOf(callable, i)
)
@@ -3272,7 +3398,7 @@ private predicate pathIntoCallable0(
* respectively.
*/
private predicate pathIntoCallable(
PathNodeMid mid, ParameterNode p, CallContext outercc, CallContextCall innercc, SummaryCtx sc,
PathNodeMid mid, ParamNode p, CallContext outercc, CallContextCall innercc, SummaryCtx sc,
DataFlowCall call
) {
exists(int i, DataFlowCallable callable, AccessPath ap |
@@ -3568,7 +3694,7 @@ private module FlowExploration {
private newtype TSummaryCtx1 =
TSummaryCtx1None() or
TSummaryCtx1Param(ParameterNode p)
TSummaryCtx1Param(ParamNode p)
private newtype TSummaryCtx2 =
TSummaryCtx2None() or
@@ -3591,7 +3717,7 @@ private module FlowExploration {
cc instanceof CallContextAny and
sc1 = TSummaryCtx1None() and
sc2 = TSummaryCtx2None() and
ap = TPartialNil(getNodeType(node)) and
ap = TPartialNil(getNodeDataFlowType(node)) and
not fullBarrier(node, config) and
exists(config.explorationLimit())
or
@@ -3611,7 +3737,7 @@ private module FlowExploration {
or
exists(PartialPathNodeRev mid |
revPartialPathStep(mid, node, sc1, sc2, ap, config) and
not clearsContent(node, ap.getHead()) and
not clearsContentCached(node, ap.getHead()) and
not fullBarrier(node, config) and
distSink(getNodeEnclosingCallable(node), config) <= config.explorationLimit()
)
@@ -3625,9 +3751,9 @@ private module FlowExploration {
exists(PartialPathNodeFwd mid |
partialPathStep(mid, node, cc, sc1, sc2, ap, config) and
not fullBarrier(node, config) and
not clearsContent(node, ap.getHead().getContent()) and
not clearsContentCached(node, ap.getHead().getContent()) and
if node instanceof CastingNode
then compatibleTypes(getNodeType(node), ap.getType())
then compatibleTypes(getNodeDataFlowType(node), ap.getType())
else any()
)
}
@@ -3783,7 +3909,7 @@ private module FlowExploration {
PartialPathNodeFwd mid, Node node, CallContext cc, TSummaryCtx1 sc1, TSummaryCtx2 sc2,
PartialAccessPath ap, Configuration config
) {
not isUnreachableInCall(node, cc.(CallContextSpecificCall).getCall()) and
not isUnreachableInCallCached(node, cc.(CallContextSpecificCall).getCall()) and
(
localFlowStep(mid.getNode(), node, config) and
cc = mid.getCallContext() and
@@ -3797,7 +3923,7 @@ private module FlowExploration {
sc1 = mid.getSummaryCtx1() and
sc2 = mid.getSummaryCtx2() and
mid.getAp() instanceof PartialAccessPathNil and
ap = TPartialNil(getNodeType(node)) and
ap = TPartialNil(getNodeDataFlowType(node)) and
config = mid.getConfiguration()
)
or
@@ -3813,7 +3939,7 @@ private module FlowExploration {
sc1 = TSummaryCtx1None() and
sc2 = TSummaryCtx2None() and
mid.getAp() instanceof PartialAccessPathNil and
ap = TPartialNil(getNodeType(node)) and
ap = TPartialNil(getNodeDataFlowType(node)) and
config = mid.getConfiguration()
or
partialPathStoreStep(mid, _, _, node, ap) and
@@ -3827,7 +3953,7 @@ private module FlowExploration {
sc1 = mid.getSummaryCtx1() and
sc2 = mid.getSummaryCtx2() and
apConsFwd(ap, tc, ap0, config) and
compatibleTypes(ap.getType(), getNodeType(node))
compatibleTypes(ap.getType(), getNodeDataFlowType(node))
)
or
partialPathIntoCallable(mid, node, _, cc, sc1, sc2, _, ap, config)
@@ -3924,7 +4050,7 @@ private module FlowExploration {
PartialPathNodeFwd mid, int i, CallContext cc, DataFlowCall call, PartialAccessPath ap,
Configuration config
) {
exists(ArgumentNode arg |
exists(ArgNode arg |
arg = mid.getNode() and
cc = mid.getCallContext() and
arg.argumentOf(call, i) and
@@ -3943,7 +4069,7 @@ private module FlowExploration {
}
private predicate partialPathIntoCallable(
PartialPathNodeFwd mid, ParameterNode p, CallContext outercc, CallContextCall innercc,
PartialPathNodeFwd mid, ParamNode p, CallContext outercc, CallContextCall innercc,
TSummaryCtx1 sc1, TSummaryCtx2 sc2, DataFlowCall call, PartialAccessPath ap,
Configuration config
) {
@@ -3980,7 +4106,7 @@ private module FlowExploration {
DataFlowCall call, PartialPathNodeFwd mid, ReturnKindExt kind, CallContext cc,
PartialAccessPath ap, Configuration config
) {
exists(ParameterNode p, CallContext innercc, TSummaryCtx1 sc1, TSummaryCtx2 sc2 |
exists(ParamNode p, CallContext innercc, TSummaryCtx1 sc1, TSummaryCtx2 sc2 |
partialPathIntoCallable(mid, p, cc, innercc, sc1, sc2, call, _, config) and
paramFlowsThroughInPartialPath(kind, innercc, sc1, sc2, ap, config)
)
@@ -4037,7 +4163,7 @@ private module FlowExploration {
apConsRev(ap, c, ap0, config)
)
or
exists(ParameterNode p |
exists(ParamNode p |
mid.getNode() = p and
viableParamArg(_, p, node) and
sc1 = mid.getSummaryCtx1() and
@@ -4115,7 +4241,7 @@ private module FlowExploration {
int pos, TRevSummaryCtx1Some sc1, TRevSummaryCtx2Some sc2, RevPartialAccessPath ap,
Configuration config
) {
exists(PartialPathNodeRev mid, ParameterNode p |
exists(PartialPathNodeRev mid, ParamNode p |
mid.getNode() = p and
p.isParameterOf(_, pos) and
sc1 = mid.getSummaryCtx1() and
@@ -4138,7 +4264,7 @@ private module FlowExploration {
pragma[nomagic]
private predicate revPartialPathThroughCallable(
PartialPathNodeRev mid, ArgumentNode node, RevPartialAccessPath ap, Configuration config
PartialPathNodeRev mid, ArgNode node, RevPartialAccessPath ap, Configuration config
) {
exists(DataFlowCall call, int pos |
revPartialPathThroughCallable0(call, mid, pos, ap, config) and

476
python/ql/src/semmle/python/dataflow/new/internal/DataFlowImpl4.qll
View File

@@ -211,10 +211,7 @@ private predicate fullBarrier(Node node, Configuration config) {
* Holds if data can flow in one local step from `node1` to `node2`.
*/
private predicate localFlowStep(Node node1, Node node2, Configuration config) {
(
simpleLocalFlowStep(node1, node2) or
reverseStepThroughInputOutputAlias(node1, node2)
) and
simpleLocalFlowStepExt(node1, node2) and
not outBarrier(node1, config) and
not inBarrier(node2, config) and
not fullBarrier(node1, config) and
@@ -237,7 +234,7 @@ private predicate additionalLocalFlowStep(Node node1, Node node2, Configuration
* Holds if data can flow from `node1` to `node2` in a way that discards call contexts.
*/
private predicate jumpStep(Node node1, Node node2, Configuration config) {
jumpStep(node1, node2) and
jumpStepCached(node1, node2) and
not outBarrier(node1, config) and
not inBarrier(node2, config) and
not fullBarrier(node1, config) and
@@ -388,7 +385,7 @@ private module Stage1 {
*/
pragma[nomagic]
private predicate fwdFlowIsEntered(DataFlowCall call, Cc cc, Configuration config) {
exists(ArgumentNode arg |
exists(ArgNode arg |
fwdFlow(arg, cc, config) and
viableParamArg(call, _, arg)
)
@@ -515,29 +512,29 @@ private module Stage1 {
pragma[nomagic]
predicate viableParamArgNodeCandFwd1(
DataFlowCall call, ParameterNode p, ArgumentNode arg, Configuration config
DataFlowCall call, ParamNode p, ArgNode arg, Configuration config
) {
viableParamArg(call, p, arg) and
fwdFlow(arg, config)
}
pragma[nomagic]
private predicate revFlowIn(
DataFlowCall call, ArgumentNode arg, boolean toReturn, Configuration config
) {
exists(ParameterNode p |
private predicate revFlowIn(DataFlowCall call, ArgNode arg, boolean toReturn, Configuration config) {
exists(ParamNode p |
revFlow(p, toReturn, config) and
viableParamArgNodeCandFwd1(call, p, arg, config)
)
}
pragma[nomagic]
private predicate revFlowInToReturn(DataFlowCall call, ArgumentNode arg, Configuration config) {
private predicate revFlowInToReturn(DataFlowCall call, ArgNode arg, Configuration config) {
revFlowIn(call, arg, true, config)
}
/**
* Holds if an output from `call` is reached in the flow covered by `revFlow`.
* Holds if an output from `call` is reached in the flow covered by `revFlow`
* and data might flow through the target callable resulting in reverse flow
* reaching an argument of `call`.
*/
pragma[nomagic]
private predicate revFlowIsReturned(DataFlowCall call, boolean toReturn, Configuration config) {
@@ -597,7 +594,7 @@ private module Stage1 {
* Holds if flow may enter through `p` and reach a return node making `p` a
* candidate for the origin of a summary.
*/
predicate parameterMayFlowThrough(ParameterNode p, DataFlowCallable c, Ap ap, Configuration config) {
predicate parameterMayFlowThrough(ParamNode p, DataFlowCallable c, Ap ap, Configuration config) {
exists(ReturnKindExt kind |
throughFlowNodeCand(p, config) and
returnFlowCallableNodeCand(c, kind, config) and
@@ -610,6 +607,15 @@ private module Stage1 {
)
}
pragma[nomagic]
predicate callMayFlowThroughRev(DataFlowCall call, Configuration config) {
exists(ArgNode arg, boolean toReturn |
revFlow(arg, toReturn, config) and
revFlowInToReturn(call, arg, config) and
revFlowIsReturned(call, toReturn, config)
)
}
predicate stats(boolean fwd, int nodes, int fields, int conscand, int tuples, Configuration config) {
fwd = true and
nodes = count(Node node | fwdFlow(node, config)) and
@@ -663,7 +669,7 @@ private predicate flowOutOfCallNodeCand1(
pragma[nomagic]
private predicate viableParamArgNodeCand1(
DataFlowCall call, ParameterNode p, ArgumentNode arg, Configuration config
DataFlowCall call, ParamNode p, ArgNode arg, Configuration config
) {
Stage1::viableParamArgNodeCandFwd1(call, p, arg, config) and
Stage1::revFlow(arg, config)
@@ -675,7 +681,7 @@ private predicate viableParamArgNodeCand1(
*/
pragma[nomagic]
private predicate flowIntoCallNodeCand1(
DataFlowCall call, ArgumentNode arg, ParameterNode p, Configuration config
DataFlowCall call, ArgNode arg, ParamNode p, Configuration config
) {
viableParamArgNodeCand1(call, p, arg, config) and
Stage1::revFlow(p, config) and
@@ -735,8 +741,7 @@ private predicate flowOutOfCallNodeCand1(
*/
pragma[nomagic]
private predicate flowIntoCallNodeCand1(
DataFlowCall call, ArgumentNode arg, ParameterNode p, boolean allowsFieldFlow,
Configuration config
DataFlowCall call, ArgNode arg, ParamNode p, boolean allowsFieldFlow, Configuration config
) {
flowIntoCallNodeCand1(call, arg, p, config) and
exists(int b, int j |
@@ -833,6 +838,16 @@ private module Stage2 {
PrevStage::revFlow(node, _, _, apa, config)
}
pragma[nomagic]
private predicate flowThroughOutOfCall(
DataFlowCall call, ReturnNodeExt ret, Node out, boolean allowsFieldFlow, Configuration config
) {
flowOutOfCall(call, ret, out, allowsFieldFlow, pragma[only_bind_into](config)) and
PrevStage::callMayFlowThroughRev(call, pragma[only_bind_into](config)) and
PrevStage::parameterMayFlowThrough(_, getNodeEnclosingCallable(ret), _,
pragma[only_bind_into](config))
}
/**
* Holds if `node` is reachable with access path `ap` from a source in the
* configuration `config`.
@@ -899,13 +914,11 @@ private module Stage2 {
)
or
// flow out of a callable
exists(DataFlowCall call |
fwdFlowOut(call, node, any(CcNoCall innercc), cc, argAp, ap, config)
or
exists(Ap argAp0 |
fwdFlowOutFromArg(call, node, argAp0, ap, config) and
fwdFlowIsEntered(call, cc, argAp, argAp0, config)
)
fwdFlowOutNotFromArg(node, cc, argAp, ap, config)
or
exists(DataFlowCall call, Ap argAp0 |
fwdFlowOutFromArg(call, node, argAp0, ap, config) and
fwdFlowIsEntered(call, cc, argAp, argAp0, config)
)
}
@@ -944,10 +957,10 @@ private module Stage2 {
pragma[nomagic]
private predicate fwdFlowIn(
DataFlowCall call, ParameterNode p, Cc outercc, Cc innercc, ApOption argAp, Ap ap,
DataFlowCall call, ParamNode p, Cc outercc, Cc innercc, ApOption argAp, Ap ap,
Configuration config
) {
exists(ArgumentNode arg, boolean allowsFieldFlow |
exists(ArgNode arg, boolean allowsFieldFlow |
fwdFlow(arg, outercc, argAp, ap, config) and
flowIntoCall(call, arg, p, allowsFieldFlow, config) and
innercc = getCallContextCall(call, getNodeEnclosingCallable(p), outercc)
@@ -956,17 +969,14 @@ private module Stage2 {
)
}
/**
* Holds if flow may exit from `call` at `out` with access path `ap`. The
* inner call context is `innercc`, but `ccOut` is just the call context
* based on the return step. In the case of through-flow `ccOut` is discarded
* and replaced by the outer call context as tracked by `fwdFlowIsEntered`.
*/
pragma[nomagic]
private predicate fwdFlowOut(
DataFlowCall call, Node out, Cc innercc, Cc ccOut, ApOption argAp, Ap ap, Configuration config
private predicate fwdFlowOutNotFromArg(
Node out, Cc ccOut, ApOption argAp, Ap ap, Configuration config
) {
exists(ReturnNodeExt ret, boolean allowsFieldFlow, DataFlowCallable inner |
exists(
DataFlowCall call, ReturnNodeExt ret, boolean allowsFieldFlow, CcNoCall innercc,
DataFlowCallable inner
|
fwdFlow(ret, innercc, argAp, ap, config) and
flowOutOfCall(call, ret, out, allowsFieldFlow, config) and
inner = getNodeEnclosingCallable(ret) and
@@ -981,7 +991,13 @@ private module Stage2 {
private predicate fwdFlowOutFromArg(
DataFlowCall call, Node out, Ap argAp, Ap ap, Configuration config
) {
fwdFlowOut(call, out, any(CcCall ccc), _, apSome(argAp), ap, config)
exists(ReturnNodeExt ret, boolean allowsFieldFlow, CcCall ccc |
fwdFlow(ret, ccc, apSome(argAp), ap, config) and
flowThroughOutOfCall(call, ret, out, allowsFieldFlow, config) and
ccc.matchesCall(call)
|
ap instanceof ApNil or allowsFieldFlow = true
)
}
/**
@@ -992,7 +1008,7 @@ private module Stage2 {
private predicate fwdFlowIsEntered(
DataFlowCall call, Cc cc, ApOption argAp, Ap ap, Configuration config
) {
exists(ParameterNode p |
exists(ParamNode p |
fwdFlowIn(call, p, cc, _, argAp, ap, config) and
PrevStage::parameterMayFlowThrough(p, _, getApprox(ap), config)
)
@@ -1012,6 +1028,27 @@ private module Stage2 {
fwdFlowConsCand(ap1, c, ap2, config)
}
pragma[nomagic]
private predicate callMayFlowThroughFwd(DataFlowCall call, Configuration config) {
exists(Ap argAp0, Node out, Cc cc, ApOption argAp, Ap ap |
fwdFlow(out, pragma[only_bind_into](cc), pragma[only_bind_into](argAp), ap,
pragma[only_bind_into](config)) and
fwdFlowOutFromArg(call, out, argAp0, ap, config) and
fwdFlowIsEntered(call, pragma[only_bind_into](cc), pragma[only_bind_into](argAp), argAp0,
pragma[only_bind_into](config))
)
}
pragma[nomagic]
private predicate flowThroughIntoCall(
DataFlowCall call, ArgNode arg, ParamNode p, boolean allowsFieldFlow, Configuration config
) {
flowIntoCall(call, arg, p, allowsFieldFlow, config) and
fwdFlow(arg, _, _, _, pragma[only_bind_into](config)) and
PrevStage::parameterMayFlowThrough(p, _, _, pragma[only_bind_into](config)) and
callMayFlowThroughFwd(call, pragma[only_bind_into](config))
}
/**
* Holds if `node` with access path `ap` is part of a path from a source to a
* sink in the configuration `config`.
@@ -1077,14 +1114,12 @@ private module Stage2 {
)
or
// flow into a callable
exists(DataFlowCall call |
revFlowIn(call, node, toReturn, returnAp, ap, config) and
toReturn = false
or
exists(Ap returnAp0 |
revFlowInToReturn(call, node, returnAp0, ap, config) and
revFlowIsReturned(call, toReturn, returnAp, returnAp0, config)
)
revFlowInNotToReturn(node, returnAp, ap, config) and
toReturn = false
or
exists(DataFlowCall call, Ap returnAp0 |
revFlowInToReturn(call, node, returnAp0, ap, config) and
revFlowIsReturned(call, toReturn, returnAp, returnAp0, config)
)
or
// flow out of a callable
@@ -1132,13 +1167,10 @@ private module Stage2 {
}
pragma[nomagic]
private predicate revFlowIn(
DataFlowCall call, ArgumentNode arg, boolean toReturn, ApOption returnAp, Ap ap,
Configuration config
) {
exists(ParameterNode p, boolean allowsFieldFlow |
revFlow(p, toReturn, returnAp, ap, config) and
flowIntoCall(call, arg, p, allowsFieldFlow, config)
private predicate revFlowInNotToReturn(ArgNode arg, ApOption returnAp, Ap ap, Configuration config) {
exists(ParamNode p, boolean allowsFieldFlow |
revFlow(p, false, returnAp, ap, config) and
flowIntoCall(_, arg, p, allowsFieldFlow, config)
|
ap instanceof ApNil or allowsFieldFlow = true
)
@@ -1146,9 +1178,14 @@ private module Stage2 {
pragma[nomagic]
private predicate revFlowInToReturn(
DataFlowCall call, ArgumentNode arg, Ap returnAp, Ap ap, Configuration config
DataFlowCall call, ArgNode arg, Ap returnAp, Ap ap, Configuration config
) {
revFlowIn(call, arg, true, apSome(returnAp), ap, config)
exists(ParamNode p, boolean allowsFieldFlow |
revFlow(p, true, apSome(returnAp), ap, config) and
flowThroughIntoCall(call, arg, p, allowsFieldFlow, config)
|
ap instanceof ApNil or allowsFieldFlow = true
)
}
/**
@@ -1199,13 +1236,13 @@ private module Stage2 {
pragma[noinline]
private predicate parameterFlow(
ParameterNode p, Ap ap, Ap ap0, DataFlowCallable c, Configuration config
ParamNode p, Ap ap, Ap ap0, DataFlowCallable c, Configuration config
) {
revFlow(p, true, apSome(ap0), ap, config) and
c = getNodeEnclosingCallable(p)
}
predicate parameterMayFlowThrough(ParameterNode p, DataFlowCallable c, Ap ap, Configuration config) {
predicate parameterMayFlowThrough(ParamNode p, DataFlowCallable c, Ap ap, Configuration config) {
exists(ReturnNodeExt ret, Ap ap0, ReturnKindExt kind, int pos |
parameterFlow(p, ap, ap0, c, config) and
c = getNodeEnclosingCallable(ret) and
@@ -1218,6 +1255,15 @@ private module Stage2 {
)
}
pragma[nomagic]
predicate callMayFlowThroughRev(DataFlowCall call, Configuration config) {
exists(Ap returnAp0, ArgNode arg, boolean toReturn, ApOption returnAp, Ap ap |
revFlow(arg, toReturn, returnAp, ap, config) and
revFlowInToReturn(call, arg, returnAp0, ap, config) and
revFlowIsReturned(call, toReturn, returnAp, returnAp0, config)
)
}
predicate stats(boolean fwd, int nodes, int fields, int conscand, int tuples, Configuration config) {
fwd = true and
nodes = count(Node node | fwdFlow(node, _, _, _, config)) and
@@ -1245,8 +1291,7 @@ private predicate flowOutOfCallNodeCand2(
pragma[nomagic]
private predicate flowIntoCallNodeCand2(
DataFlowCall call, ArgumentNode node1, ParameterNode node2, boolean allowsFieldFlow,
Configuration config
DataFlowCall call, ArgNode node1, ParamNode node2, boolean allowsFieldFlow, Configuration config
) {
flowIntoCallNodeCand1(call, node1, node2, allowsFieldFlow, config) and
Stage2::revFlow(node2, pragma[only_bind_into](config)) and
@@ -1260,8 +1305,8 @@ private module LocalFlowBigStep {
*/
private class FlowCheckNode extends Node {
FlowCheckNode() {
this instanceof CastNode or
clearsContent(this, _)
castNode(this) or
clearsContentCached(this, _)
}
}
@@ -1275,7 +1320,7 @@ private module LocalFlowBigStep {
config.isSource(node) or
jumpStep(_, node, config) or
additionalJumpStep(_, node, config) or
node instanceof ParameterNode or
node instanceof ParamNode or
node instanceof OutNodeExt or
store(_, _, node, _) or
read(_, _, node) or
@@ -1321,21 +1366,21 @@ private module LocalFlowBigStep {
Node node1, Node node2, boolean preservesValue, DataFlowType t, Configuration config,
LocalCallContext cc
) {
not isUnreachableInCall(node2, cc.(LocalCallContextSpecificCall).getCall()) and
not isUnreachableInCallCached(node2, cc.(LocalCallContextSpecificCall).getCall()) and
(
localFlowEntry(node1, pragma[only_bind_into](config)) and
(
localFlowStepNodeCand1(node1, node2, config) and
preservesValue = true and
t = getNodeType(node1)
t = getNodeDataFlowType(node1)
or
additionalLocalFlowStepNodeCand2(node1, node2, config) and
preservesValue = false and
t = getNodeType(node2)
t = getNodeDataFlowType(node2)
) and
node1 != node2 and
cc.relevantFor(getNodeEnclosingCallable(node1)) and
not isUnreachableInCall(node1, cc.(LocalCallContextSpecificCall).getCall()) and
not isUnreachableInCallCached(node1, cc.(LocalCallContextSpecificCall).getCall()) and
Stage2::revFlow(node2, pragma[only_bind_into](config))
or
exists(Node mid |
@@ -1350,7 +1395,7 @@ private module LocalFlowBigStep {
additionalLocalFlowStepNodeCand2(mid, node2, config) and
not mid instanceof FlowCheckNode and
preservesValue = false and
t = getNodeType(node2) and
t = getNodeDataFlowType(node2) and
Stage2::revFlow(node2, pragma[only_bind_into](config))
)
)
@@ -1384,7 +1429,7 @@ private module Stage3 {
private ApApprox getApprox(Ap ap) { result = ap.toBoolNonEmpty() }
private ApNil getApNil(Node node) {
PrevStage::revFlow(node, _) and result = TFrontNil(getNodeType(node))
PrevStage::revFlow(node, _) and result = TFrontNil(getNodeDataFlowType(node))
}
bindingset[tc, tail]
@@ -1443,7 +1488,9 @@ private module Stage3 {
bindingset[node, ap]
private predicate filter(Node node, Ap ap) {
not ap.isClearedAt(node) and
if node instanceof CastingNode then compatibleTypes(getNodeType(node), ap.getType()) else any()
if node instanceof CastingNode
then compatibleTypes(getNodeDataFlowType(node), ap.getType())
else any()
}
bindingset[ap, contentType]
@@ -1465,6 +1512,16 @@ private module Stage3 {
)
}
pragma[nomagic]
private predicate flowThroughOutOfCall(
DataFlowCall call, ReturnNodeExt ret, Node out, boolean allowsFieldFlow, Configuration config
) {
flowOutOfCall(call, ret, out, allowsFieldFlow, pragma[only_bind_into](config)) and
PrevStage::callMayFlowThroughRev(call, pragma[only_bind_into](config)) and
PrevStage::parameterMayFlowThrough(_, getNodeEnclosingCallable(ret), _,
pragma[only_bind_into](config))
}
/**
* Holds if `node` is reachable with access path `ap` from a source in the
* configuration `config`.
@@ -1538,13 +1595,11 @@ private module Stage3 {
)
or
// flow out of a callable
exists(DataFlowCall call |
fwdFlowOut(call, node, any(CcNoCall innercc), cc, argAp, ap, config)
or
exists(Ap argAp0 |
fwdFlowOutFromArg(call, node, argAp0, ap, config) and
fwdFlowIsEntered(call, cc, argAp, argAp0, config)
)
fwdFlowOutNotFromArg(node, cc, argAp, ap, config)
or
exists(DataFlowCall call, Ap argAp0 |
fwdFlowOutFromArg(call, node, argAp0, ap, config) and
fwdFlowIsEntered(call, cc, argAp, argAp0, config)
)
}
@@ -1583,10 +1638,10 @@ private module Stage3 {
pragma[nomagic]
private predicate fwdFlowIn(
DataFlowCall call, ParameterNode p, Cc outercc, Cc innercc, ApOption argAp, Ap ap,
DataFlowCall call, ParamNode p, Cc outercc, Cc innercc, ApOption argAp, Ap ap,
Configuration config
) {
exists(ArgumentNode arg, boolean allowsFieldFlow |
exists(ArgNode arg, boolean allowsFieldFlow |
fwdFlow(arg, outercc, argAp, ap, config) and
flowIntoCall(call, arg, p, allowsFieldFlow, config) and
innercc = getCallContextCall(call, getNodeEnclosingCallable(p), outercc)
@@ -1595,17 +1650,14 @@ private module Stage3 {
)
}
/**
* Holds if flow may exit from `call` at `out` with access path `ap`. The
* inner call context is `innercc`, but `ccOut` is just the call context
* based on the return step. In the case of through-flow `ccOut` is discarded
* and replaced by the outer call context as tracked by `fwdFlowIsEntered`.
*/
pragma[nomagic]
private predicate fwdFlowOut(
DataFlowCall call, Node out, Cc innercc, Cc ccOut, ApOption argAp, Ap ap, Configuration config
private predicate fwdFlowOutNotFromArg(
Node out, Cc ccOut, ApOption argAp, Ap ap, Configuration config
) {
exists(ReturnNodeExt ret, boolean allowsFieldFlow, DataFlowCallable inner |
exists(
DataFlowCall call, ReturnNodeExt ret, boolean allowsFieldFlow, CcNoCall innercc,
DataFlowCallable inner
|
fwdFlow(ret, innercc, argAp, ap, config) and
flowOutOfCall(call, ret, out, allowsFieldFlow, config) and
inner = getNodeEnclosingCallable(ret) and
@@ -1620,7 +1672,13 @@ private module Stage3 {
private predicate fwdFlowOutFromArg(
DataFlowCall call, Node out, Ap argAp, Ap ap, Configuration config
) {
fwdFlowOut(call, out, any(CcCall ccc), _, apSome(argAp), ap, config)
exists(ReturnNodeExt ret, boolean allowsFieldFlow, CcCall ccc |
fwdFlow(ret, ccc, apSome(argAp), ap, config) and
flowThroughOutOfCall(call, ret, out, allowsFieldFlow, config) and
ccc.matchesCall(call)
|
ap instanceof ApNil or allowsFieldFlow = true
)
}
/**
@@ -1631,7 +1689,7 @@ private module Stage3 {
private predicate fwdFlowIsEntered(
DataFlowCall call, Cc cc, ApOption argAp, Ap ap, Configuration config
) {
exists(ParameterNode p |
exists(ParamNode p |
fwdFlowIn(call, p, cc, _, argAp, ap, config) and
PrevStage::parameterMayFlowThrough(p, _, unbindApa(getApprox(ap)), config)
)
@@ -1651,6 +1709,27 @@ private module Stage3 {
fwdFlowConsCand(ap1, c, ap2, config)
}
pragma[nomagic]
private predicate callMayFlowThroughFwd(DataFlowCall call, Configuration config) {
exists(Ap argAp0, Node out, Cc cc, ApOption argAp, Ap ap |
fwdFlow(out, pragma[only_bind_into](cc), pragma[only_bind_into](argAp), ap,
pragma[only_bind_into](config)) and
fwdFlowOutFromArg(call, out, argAp0, ap, config) and
fwdFlowIsEntered(call, pragma[only_bind_into](cc), pragma[only_bind_into](argAp), argAp0,
pragma[only_bind_into](config))
)
}
pragma[nomagic]
private predicate flowThroughIntoCall(
DataFlowCall call, ArgNode arg, ParamNode p, boolean allowsFieldFlow, Configuration config
) {
flowIntoCall(call, arg, p, allowsFieldFlow, config) and
fwdFlow(arg, _, _, _, pragma[only_bind_into](config)) and
PrevStage::parameterMayFlowThrough(p, _, _, pragma[only_bind_into](config)) and
callMayFlowThroughFwd(call, pragma[only_bind_into](config))
}
/**
* Holds if `node` with access path `ap` is part of a path from a source to a
* sink in the configuration `config`.
@@ -1716,14 +1795,12 @@ private module Stage3 {
)
or
// flow into a callable
exists(DataFlowCall call |
revFlowIn(call, node, toReturn, returnAp, ap, config) and
toReturn = false
or
exists(Ap returnAp0 |
revFlowInToReturn(call, node, returnAp0, ap, config) and
revFlowIsReturned(call, toReturn, returnAp, returnAp0, config)
)
revFlowInNotToReturn(node, returnAp, ap, config) and
toReturn = false
or
exists(DataFlowCall call, Ap returnAp0 |
revFlowInToReturn(call, node, returnAp0, ap, config) and
revFlowIsReturned(call, toReturn, returnAp, returnAp0, config)
)
or
// flow out of a callable
@@ -1771,13 +1848,10 @@ private module Stage3 {
}
pragma[nomagic]
private predicate revFlowIn(
DataFlowCall call, ArgumentNode arg, boolean toReturn, ApOption returnAp, Ap ap,
Configuration config
) {
exists(ParameterNode p, boolean allowsFieldFlow |
revFlow(p, toReturn, returnAp, ap, config) and
flowIntoCall(call, arg, p, allowsFieldFlow, config)
private predicate revFlowInNotToReturn(ArgNode arg, ApOption returnAp, Ap ap, Configuration config) {
exists(ParamNode p, boolean allowsFieldFlow |
revFlow(p, false, returnAp, ap, config) and
flowIntoCall(_, arg, p, allowsFieldFlow, config)
|
ap instanceof ApNil or allowsFieldFlow = true
)
@@ -1785,9 +1859,14 @@ private module Stage3 {
pragma[nomagic]
private predicate revFlowInToReturn(
DataFlowCall call, ArgumentNode arg, Ap returnAp, Ap ap, Configuration config
DataFlowCall call, ArgNode arg, Ap returnAp, Ap ap, Configuration config
) {
revFlowIn(call, arg, true, apSome(returnAp), ap, config)
exists(ParamNode p, boolean allowsFieldFlow |
revFlow(p, true, apSome(returnAp), ap, config) and
flowThroughIntoCall(call, arg, p, allowsFieldFlow, config)
|
ap instanceof ApNil or allowsFieldFlow = true
)
}
/**
@@ -1838,13 +1917,13 @@ private module Stage3 {
pragma[noinline]
private predicate parameterFlow(
ParameterNode p, Ap ap, Ap ap0, DataFlowCallable c, Configuration config
ParamNode p, Ap ap, Ap ap0, DataFlowCallable c, Configuration config
) {
revFlow(p, true, apSome(ap0), ap, config) and
c = getNodeEnclosingCallable(p)
}
predicate parameterMayFlowThrough(ParameterNode p, DataFlowCallable c, Ap ap, Configuration config) {
predicate parameterMayFlowThrough(ParamNode p, DataFlowCallable c, Ap ap, Configuration config) {
exists(ReturnNodeExt ret, Ap ap0, ReturnKindExt kind, int pos |
parameterFlow(p, ap, ap0, c, config) and
c = getNodeEnclosingCallable(ret) and
@@ -1857,6 +1936,15 @@ private module Stage3 {
)
}
pragma[nomagic]
predicate callMayFlowThroughRev(DataFlowCall call, Configuration config) {
exists(Ap returnAp0, ArgNode arg, boolean toReturn, ApOption returnAp, Ap ap |
revFlow(arg, toReturn, returnAp, ap, config) and
revFlowInToReturn(call, arg, returnAp0, ap, config) and
revFlowIsReturned(call, toReturn, returnAp, returnAp0, config)
)
}
predicate stats(boolean fwd, int nodes, int fields, int conscand, int tuples, Configuration config) {
fwd = true and
nodes = count(Node node | fwdFlow(node, _, _, _, config)) and
@@ -2088,7 +2176,7 @@ private module Stage4 {
private ApApprox getApprox(Ap ap) { result = ap.getFront() }
private ApNil getApNil(Node node) {
PrevStage::revFlow(node, _) and result = TNil(getNodeType(node))
PrevStage::revFlow(node, _) and result = TNil(getNodeDataFlowType(node))
}
bindingset[tc, tail]
@@ -2127,8 +2215,6 @@ private module Stage4 {
bindingset[innercc, inner, call]
private predicate checkCallContextReturn(Cc innercc, DataFlowCallable inner, DataFlowCall call) {
resolveReturn(innercc, inner, call)
or
innercc.(CallContextCall).matchesCall(call)
}
bindingset[node, cc, config]
@@ -2155,8 +2241,7 @@ private module Stage4 {
pragma[nomagic]
private predicate flowIntoCall(
DataFlowCall call, ArgumentNode node1, ParameterNode node2, boolean allowsFieldFlow,
Configuration config
DataFlowCall call, ArgNode node1, ParamNode node2, boolean allowsFieldFlow, Configuration config
) {
flowIntoCallNodeCand2(call, node1, node2, allowsFieldFlow, config) and
PrevStage::revFlow(node2, _, _, _, pragma[only_bind_into](config)) and
@@ -2182,6 +2267,16 @@ private module Stage4 {
)
}
pragma[nomagic]
private predicate flowThroughOutOfCall(
DataFlowCall call, ReturnNodeExt ret, Node out, boolean allowsFieldFlow, Configuration config
) {
flowOutOfCall(call, ret, out, allowsFieldFlow, pragma[only_bind_into](config)) and
PrevStage::callMayFlowThroughRev(call, pragma[only_bind_into](config)) and
PrevStage::parameterMayFlowThrough(_, getNodeEnclosingCallable(ret), _,
pragma[only_bind_into](config))
}
/**
* Holds if `node` is reachable with access path `ap` from a source in the
* configuration `config`.
@@ -2255,13 +2350,11 @@ private module Stage4 {
)
or
// flow out of a callable
exists(DataFlowCall call |
fwdFlowOut(call, node, any(CcNoCall innercc), cc, argAp, ap, config)
or
exists(Ap argAp0 |
fwdFlowOutFromArg(call, node, argAp0, ap, config) and
fwdFlowIsEntered(call, cc, argAp, argAp0, config)
)
fwdFlowOutNotFromArg(node, cc, argAp, ap, config)
or
exists(DataFlowCall call, Ap argAp0 |
fwdFlowOutFromArg(call, node, argAp0, ap, config) and
fwdFlowIsEntered(call, cc, argAp, argAp0, config)
)
}
@@ -2300,10 +2393,10 @@ private module Stage4 {
pragma[nomagic]
private predicate fwdFlowIn(
DataFlowCall call, ParameterNode p, Cc outercc, Cc innercc, ApOption argAp, Ap ap,
DataFlowCall call, ParamNode p, Cc outercc, Cc innercc, ApOption argAp, Ap ap,
Configuration config
) {
exists(ArgumentNode arg, boolean allowsFieldFlow |
exists(ArgNode arg, boolean allowsFieldFlow |
fwdFlow(arg, outercc, argAp, ap, config) and
flowIntoCall(call, arg, p, allowsFieldFlow, config) and
innercc = getCallContextCall(call, getNodeEnclosingCallable(p), outercc)
@@ -2312,17 +2405,14 @@ private module Stage4 {
)
}
/**
* Holds if flow may exit from `call` at `out` with access path `ap`. The
* inner call context is `innercc`, but `ccOut` is just the call context
* based on the return step. In the case of through-flow `ccOut` is discarded
* and replaced by the outer call context as tracked by `fwdFlowIsEntered`.
*/
pragma[nomagic]
private predicate fwdFlowOut(
DataFlowCall call, Node out, Cc innercc, Cc ccOut, ApOption argAp, Ap ap, Configuration config
private predicate fwdFlowOutNotFromArg(
Node out, Cc ccOut, ApOption argAp, Ap ap, Configuration config
) {
exists(ReturnNodeExt ret, boolean allowsFieldFlow, DataFlowCallable inner |
exists(
DataFlowCall call, ReturnNodeExt ret, boolean allowsFieldFlow, CcNoCall innercc,
DataFlowCallable inner
|
fwdFlow(ret, innercc, argAp, ap, config) and
flowOutOfCall(call, ret, out, allowsFieldFlow, config) and
inner = getNodeEnclosingCallable(ret) and
@@ -2337,7 +2427,13 @@ private module Stage4 {
private predicate fwdFlowOutFromArg(
DataFlowCall call, Node out, Ap argAp, Ap ap, Configuration config
) {
fwdFlowOut(call, out, any(CcCall ccc), _, apSome(argAp), ap, config)
exists(ReturnNodeExt ret, boolean allowsFieldFlow, CcCall ccc |
fwdFlow(ret, ccc, apSome(argAp), ap, config) and
flowThroughOutOfCall(call, ret, out, allowsFieldFlow, config) and
ccc.matchesCall(call)
|
ap instanceof ApNil or allowsFieldFlow = true
)
}
/**
@@ -2348,7 +2444,7 @@ private module Stage4 {
private predicate fwdFlowIsEntered(
DataFlowCall call, Cc cc, ApOption argAp, Ap ap, Configuration config
) {
exists(ParameterNode p |
exists(ParamNode p |
fwdFlowIn(call, p, cc, _, argAp, ap, config) and
PrevStage::parameterMayFlowThrough(p, _, unbindApa(getApprox(ap)), config)
)
@@ -2368,6 +2464,27 @@ private module Stage4 {
fwdFlowConsCand(ap1, c, ap2, config)
}
pragma[nomagic]
private predicate callMayFlowThroughFwd(DataFlowCall call, Configuration config) {
exists(Ap argAp0, Node out, Cc cc, ApOption argAp, Ap ap |
fwdFlow(out, pragma[only_bind_into](cc), pragma[only_bind_into](argAp), ap,
pragma[only_bind_into](config)) and
fwdFlowOutFromArg(call, out, argAp0, ap, config) and
fwdFlowIsEntered(call, pragma[only_bind_into](cc), pragma[only_bind_into](argAp), argAp0,
pragma[only_bind_into](config))
)
}
pragma[nomagic]
private predicate flowThroughIntoCall(
DataFlowCall call, ArgNode arg, ParamNode p, boolean allowsFieldFlow, Configuration config
) {
flowIntoCall(call, arg, p, allowsFieldFlow, config) and
fwdFlow(arg, _, _, _, pragma[only_bind_into](config)) and
PrevStage::parameterMayFlowThrough(p, _, _, pragma[only_bind_into](config)) and
callMayFlowThroughFwd(call, pragma[only_bind_into](config))
}
/**
* Holds if `node` with access path `ap` is part of a path from a source to a
* sink in the configuration `config`.
@@ -2433,14 +2550,12 @@ private module Stage4 {
)
or
// flow into a callable
exists(DataFlowCall call |
revFlowIn(call, node, toReturn, returnAp, ap, config) and
toReturn = false
or
exists(Ap returnAp0 |
revFlowInToReturn(call, node, returnAp0, ap, config) and
revFlowIsReturned(call, toReturn, returnAp, returnAp0, config)
)
revFlowInNotToReturn(node, returnAp, ap, config) and
toReturn = false
or
exists(DataFlowCall call, Ap returnAp0 |
revFlowInToReturn(call, node, returnAp0, ap, config) and
revFlowIsReturned(call, toReturn, returnAp, returnAp0, config)
)
or
// flow out of a callable
@@ -2488,13 +2603,10 @@ private module Stage4 {
}
pragma[nomagic]
private predicate revFlowIn(
DataFlowCall call, ArgumentNode arg, boolean toReturn, ApOption returnAp, Ap ap,
Configuration config
) {
exists(ParameterNode p, boolean allowsFieldFlow |
revFlow(p, toReturn, returnAp, ap, config) and
flowIntoCall(call, arg, p, allowsFieldFlow, config)
private predicate revFlowInNotToReturn(ArgNode arg, ApOption returnAp, Ap ap, Configuration config) {
exists(ParamNode p, boolean allowsFieldFlow |
revFlow(p, false, returnAp, ap, config) and
flowIntoCall(_, arg, p, allowsFieldFlow, config)
|
ap instanceof ApNil or allowsFieldFlow = true
)
@@ -2502,9 +2614,14 @@ private module Stage4 {
pragma[nomagic]
private predicate revFlowInToReturn(
DataFlowCall call, ArgumentNode arg, Ap returnAp, Ap ap, Configuration config
DataFlowCall call, ArgNode arg, Ap returnAp, Ap ap, Configuration config
) {
revFlowIn(call, arg, true, apSome(returnAp), ap, config)
exists(ParamNode p, boolean allowsFieldFlow |
revFlow(p, true, apSome(returnAp), ap, config) and
flowThroughIntoCall(call, arg, p, allowsFieldFlow, config)
|
ap instanceof ApNil or allowsFieldFlow = true
)
}
/**
@@ -2555,13 +2672,13 @@ private module Stage4 {
pragma[noinline]
private predicate parameterFlow(
ParameterNode p, Ap ap, Ap ap0, DataFlowCallable c, Configuration config
ParamNode p, Ap ap, Ap ap0, DataFlowCallable c, Configuration config
) {
revFlow(p, true, apSome(ap0), ap, config) and
c = getNodeEnclosingCallable(p)
}
predicate parameterMayFlowThrough(ParameterNode p, DataFlowCallable c, Ap ap, Configuration config) {
predicate parameterMayFlowThrough(ParamNode p, DataFlowCallable c, Ap ap, Configuration config) {
exists(ReturnNodeExt ret, Ap ap0, ReturnKindExt kind, int pos |
parameterFlow(p, ap, ap0, c, config) and
c = getNodeEnclosingCallable(ret) and
@@ -2574,6 +2691,15 @@ private module Stage4 {
)
}
pragma[nomagic]
predicate callMayFlowThroughRev(DataFlowCall call, Configuration config) {
exists(Ap returnAp0, ArgNode arg, boolean toReturn, ApOption returnAp, Ap ap |
revFlow(arg, toReturn, returnAp, ap, config) and
revFlowInToReturn(call, arg, returnAp0, ap, config) and
revFlowIsReturned(call, toReturn, returnAp, returnAp0, config)
)
}
predicate stats(boolean fwd, int nodes, int fields, int conscand, int tuples, Configuration config) {
fwd = true and
nodes = count(Node node | fwdFlow(node, _, _, _, config)) and
@@ -2606,7 +2732,7 @@ private predicate nodeMayUseSummary(Node n, AccessPathApprox apa, Configuration
private newtype TSummaryCtx =
TSummaryCtxNone() or
TSummaryCtxSome(ParameterNode p, AccessPath ap) {
TSummaryCtxSome(ParamNode p, AccessPath ap) {
Stage4::parameterMayFlowThrough(p, _, ap.getApprox(), _)
}
@@ -2627,7 +2753,7 @@ private class SummaryCtxNone extends SummaryCtx, TSummaryCtxNone {
/** A summary context from which a flow summary can be generated. */
private class SummaryCtxSome extends SummaryCtx, TSummaryCtxSome {
private ParameterNode p;
private ParamNode p;
private AccessPath ap;
SummaryCtxSome() { this = TSummaryCtxSome(p, ap) }
@@ -2758,7 +2884,7 @@ private newtype TPathNode =
config.isSource(node) and
cc instanceof CallContextAny and
sc instanceof SummaryCtxNone and
ap = TAccessPathNil(getNodeType(node))
ap = TAccessPathNil(getNodeDataFlowType(node))
or
// ... or a step from an existing PathNode to another node.
exists(PathNodeMid mid |
@@ -2979,7 +3105,7 @@ class PathNode extends TPathNode {
Configuration getConfiguration() { none() }
private predicate isHidden() {
nodeIsHidden(this.getNode()) and
hiddenNode(this.getNode()) and
not this.isSource() and
not this instanceof PathNodeSink
}
@@ -3148,7 +3274,7 @@ private predicate pathStep(PathNodeMid mid, Node node, CallContext cc, SummaryCt
cc instanceof CallContextAny and
sc instanceof SummaryCtxNone and
mid.getAp() instanceof AccessPathNil and
ap = TAccessPathNil(getNodeType(node))
ap = TAccessPathNil(getNodeDataFlowType(node))
or
exists(TypedContent tc | pathStoreStep(mid, node, ap.pop(tc), tc, cc)) and
sc = mid.getSummaryCtx()
@@ -3235,7 +3361,7 @@ pragma[noinline]
private predicate pathIntoArg(
PathNodeMid mid, int i, CallContext cc, DataFlowCall call, AccessPath ap, AccessPathApprox apa
) {
exists(ArgumentNode arg |
exists(ArgNode arg |
arg = mid.getNode() and
cc = mid.getCallContext() and
arg.argumentOf(call, i) and
@@ -3248,7 +3374,7 @@ pragma[noinline]
private predicate parameterCand(
DataFlowCallable callable, int i, AccessPathApprox apa, Configuration config
) {
exists(ParameterNode p |
exists(ParamNode p |
Stage4::revFlow(p, _, _, apa, config) and
p.isParameterOf(callable, i)
)
@@ -3272,7 +3398,7 @@ private predicate pathIntoCallable0(
* respectively.
*/
private predicate pathIntoCallable(
PathNodeMid mid, ParameterNode p, CallContext outercc, CallContextCall innercc, SummaryCtx sc,
PathNodeMid mid, ParamNode p, CallContext outercc, CallContextCall innercc, SummaryCtx sc,
DataFlowCall call
) {
exists(int i, DataFlowCallable callable, AccessPath ap |
@@ -3568,7 +3694,7 @@ private module FlowExploration {
private newtype TSummaryCtx1 =
TSummaryCtx1None() or
TSummaryCtx1Param(ParameterNode p)
TSummaryCtx1Param(ParamNode p)
private newtype TSummaryCtx2 =
TSummaryCtx2None() or
@@ -3591,7 +3717,7 @@ private module FlowExploration {
cc instanceof CallContextAny and
sc1 = TSummaryCtx1None() and
sc2 = TSummaryCtx2None() and
ap = TPartialNil(getNodeType(node)) and
ap = TPartialNil(getNodeDataFlowType(node)) and
not fullBarrier(node, config) and
exists(config.explorationLimit())
or
@@ -3611,7 +3737,7 @@ private module FlowExploration {
or
exists(PartialPathNodeRev mid |
revPartialPathStep(mid, node, sc1, sc2, ap, config) and
not clearsContent(node, ap.getHead()) and
not clearsContentCached(node, ap.getHead()) and
not fullBarrier(node, config) and
distSink(getNodeEnclosingCallable(node), config) <= config.explorationLimit()
)
@@ -3625,9 +3751,9 @@ private module FlowExploration {
exists(PartialPathNodeFwd mid |
partialPathStep(mid, node, cc, sc1, sc2, ap, config) and
not fullBarrier(node, config) and
not clearsContent(node, ap.getHead().getContent()) and
not clearsContentCached(node, ap.getHead().getContent()) and
if node instanceof CastingNode
then compatibleTypes(getNodeType(node), ap.getType())
then compatibleTypes(getNodeDataFlowType(node), ap.getType())
else any()
)
}
@@ -3783,7 +3909,7 @@ private module FlowExploration {
PartialPathNodeFwd mid, Node node, CallContext cc, TSummaryCtx1 sc1, TSummaryCtx2 sc2,
PartialAccessPath ap, Configuration config
) {
not isUnreachableInCall(node, cc.(CallContextSpecificCall).getCall()) and
not isUnreachableInCallCached(node, cc.(CallContextSpecificCall).getCall()) and
(
localFlowStep(mid.getNode(), node, config) and
cc = mid.getCallContext() and
@@ -3797,7 +3923,7 @@ private module FlowExploration {
sc1 = mid.getSummaryCtx1() and
sc2 = mid.getSummaryCtx2() and
mid.getAp() instanceof PartialAccessPathNil and
ap = TPartialNil(getNodeType(node)) and
ap = TPartialNil(getNodeDataFlowType(node)) and
config = mid.getConfiguration()
)
or
@@ -3813,7 +3939,7 @@ private module FlowExploration {
sc1 = TSummaryCtx1None() and
sc2 = TSummaryCtx2None() and
mid.getAp() instanceof PartialAccessPathNil and
ap = TPartialNil(getNodeType(node)) and
ap = TPartialNil(getNodeDataFlowType(node)) and
config = mid.getConfiguration()
or
partialPathStoreStep(mid, _, _, node, ap) and
@@ -3827,7 +3953,7 @@ private module FlowExploration {
sc1 = mid.getSummaryCtx1() and
sc2 = mid.getSummaryCtx2() and
apConsFwd(ap, tc, ap0, config) and
compatibleTypes(ap.getType(), getNodeType(node))
compatibleTypes(ap.getType(), getNodeDataFlowType(node))
)
or
partialPathIntoCallable(mid, node, _, cc, sc1, sc2, _, ap, config)
@@ -3924,7 +4050,7 @@ private module FlowExploration {
PartialPathNodeFwd mid, int i, CallContext cc, DataFlowCall call, PartialAccessPath ap,
Configuration config
) {
exists(ArgumentNode arg |
exists(ArgNode arg |
arg = mid.getNode() and
cc = mid.getCallContext() and
arg.argumentOf(call, i) and
@@ -3943,7 +4069,7 @@ private module FlowExploration {
}
private predicate partialPathIntoCallable(
PartialPathNodeFwd mid, ParameterNode p, CallContext outercc, CallContextCall innercc,
PartialPathNodeFwd mid, ParamNode p, CallContext outercc, CallContextCall innercc,
TSummaryCtx1 sc1, TSummaryCtx2 sc2, DataFlowCall call, PartialAccessPath ap,
Configuration config
) {
@@ -3980,7 +4106,7 @@ private module FlowExploration {
DataFlowCall call, PartialPathNodeFwd mid, ReturnKindExt kind, CallContext cc,
PartialAccessPath ap, Configuration config
) {
exists(ParameterNode p, CallContext innercc, TSummaryCtx1 sc1, TSummaryCtx2 sc2 |
exists(ParamNode p, CallContext innercc, TSummaryCtx1 sc1, TSummaryCtx2 sc2 |
partialPathIntoCallable(mid, p, cc, innercc, sc1, sc2, call, _, config) and
paramFlowsThroughInPartialPath(kind, innercc, sc1, sc2, ap, config)
)
@@ -4037,7 +4163,7 @@ private module FlowExploration {
apConsRev(ap, c, ap0, config)
)
or
exists(ParameterNode p |
exists(ParamNode p |
mid.getNode() = p and
viableParamArg(_, p, node) and
sc1 = mid.getSummaryCtx1() and
@@ -4115,7 +4241,7 @@ private module FlowExploration {
int pos, TRevSummaryCtx1Some sc1, TRevSummaryCtx2Some sc2, RevPartialAccessPath ap,
Configuration config
) {
exists(PartialPathNodeRev mid, ParameterNode p |
exists(PartialPathNodeRev mid, ParamNode p |
mid.getNode() = p and
p.isParameterOf(_, pos) and
sc1 = mid.getSummaryCtx1() and
@@ -4138,7 +4264,7 @@ private module FlowExploration {
pragma[nomagic]
private predicate revPartialPathThroughCallable(
PartialPathNodeRev mid, ArgumentNode node, RevPartialAccessPath ap, Configuration config
PartialPathNodeRev mid, ArgNode node, RevPartialAccessPath ap, Configuration config
) {
exists(DataFlowCall call, int pos |
revPartialPathThroughCallable0(call, mid, pos, ap, config) and

309
python/ql/src/semmle/python/dataflow/new/internal/DataFlowImplCommon.qll
View File

@@ -35,22 +35,22 @@ predicate accessPathCostLimits(int apLimit, int tupleLimit) {
* calls. For this reason, we cannot reuse the code from `DataFlowImpl.qll` directly.
*/
private module LambdaFlow {
private predicate viableParamNonLambda(DataFlowCall call, int i, ParameterNode p) {
private predicate viableParamNonLambda(DataFlowCall call, int i, ParamNode p) {
p.isParameterOf(viableCallable(call), i)
}
private predicate viableParamLambda(DataFlowCall call, int i, ParameterNode p) {
private predicate viableParamLambda(DataFlowCall call, int i, ParamNode p) {
p.isParameterOf(viableCallableLambda(call, _), i)
}
private predicate viableParamArgNonLambda(DataFlowCall call, ParameterNode p, ArgumentNode arg) {
private predicate viableParamArgNonLambda(DataFlowCall call, ParamNode p, ArgNode arg) {
exists(int i |
viableParamNonLambda(call, i, p) and
arg.argumentOf(call, i)
)
}
private predicate viableParamArgLambda(DataFlowCall call, ParameterNode p, ArgumentNode arg) {
private predicate viableParamArgLambda(DataFlowCall call, ParamNode p, ArgNode arg) {
exists(int i |
viableParamLambda(call, i, p) and
arg.argumentOf(call, i)
@@ -118,8 +118,8 @@ private module LambdaFlow {
boolean toJump, DataFlowCallOption lastCall
) {
revLambdaFlow0(lambdaCall, kind, node, t, toReturn, toJump, lastCall) and
if node instanceof CastNode or node instanceof ArgumentNode or node instanceof ReturnNode
then compatibleTypes(t, getNodeType(node))
if castNode(node) or node instanceof ArgNode or node instanceof ReturnNode
then compatibleTypes(t, getNodeDataFlowType(node))
else any()
}
@@ -129,7 +129,7 @@ private module LambdaFlow {
boolean toJump, DataFlowCallOption lastCall
) {
lambdaCall(lambdaCall, kind, node) and
t = getNodeType(node) and
t = getNodeDataFlowType(node) and
toReturn = false and
toJump = false and
lastCall = TDataFlowCallNone()
@@ -146,7 +146,7 @@ private module LambdaFlow {
getNodeEnclosingCallable(node) = getNodeEnclosingCallable(mid)
|
preservesValue = false and
t = getNodeType(node)
t = getNodeDataFlowType(node)
or
preservesValue = true and
t = t0
@@ -160,7 +160,7 @@ private module LambdaFlow {
toJump = true and
lastCall = TDataFlowCallNone()
|
jumpStep(node, mid) and
jumpStepCached(node, mid) and
t = t0
or
exists(boolean preservesValue |
@@ -168,7 +168,7 @@ private module LambdaFlow {
getNodeEnclosingCallable(node) != getNodeEnclosingCallable(mid)
|
preservesValue = false and
t = getNodeType(node)
t = getNodeDataFlowType(node)
or
preservesValue = true and
t = t0
@@ -176,7 +176,7 @@ private module LambdaFlow {
)
or
// flow into a callable
exists(ParameterNode p, DataFlowCallOption lastCall0, DataFlowCall call |
exists(ParamNode p, DataFlowCallOption lastCall0, DataFlowCall call |
revLambdaFlowIn(lambdaCall, kind, p, t, toJump, lastCall0) and
(
if lastCall0 = TDataFlowCallNone() and toJump = false
@@ -227,7 +227,7 @@ private module LambdaFlow {
pragma[nomagic]
predicate revLambdaFlowIn(
DataFlowCall lambdaCall, LambdaCallKind kind, ParameterNode p, DataFlowType t, boolean toJump,
DataFlowCall lambdaCall, LambdaCallKind kind, ParamNode p, DataFlowType t, boolean toJump,
DataFlowCallOption lastCall
) {
revLambdaFlow(lambdaCall, kind, p, t, false, toJump, lastCall)
@@ -242,6 +242,89 @@ private DataFlowCallable viableCallableExt(DataFlowCall call) {
cached
private module Cached {
/**
* If needed, call this predicate from `DataFlowImplSpecific.qll` in order to
* force a stage-dependency on the `DataFlowImplCommon.qll` stage and therby
* collapsing the two stages.
*/
cached
predicate forceCachingInSameStage() { any() }
cached
predicate nodeEnclosingCallable(Node n, DataFlowCallable c) { c = n.getEnclosingCallable() }
cached
predicate callEnclosingCallable(DataFlowCall call, DataFlowCallable c) {
c = call.getEnclosingCallable()
}
cached
predicate nodeDataFlowType(Node n, DataFlowType t) { t = getNodeType(n) }
cached
predicate jumpStepCached(Node node1, Node node2) { jumpStep(node1, node2) }
cached
predicate clearsContentCached(Node n, Content c) { clearsContent(n, c) }
cached
predicate isUnreachableInCallCached(Node n, DataFlowCall call) { isUnreachableInCall(n, call) }
cached
predicate outNodeExt(Node n) {
n instanceof OutNode
or
n.(PostUpdateNode).getPreUpdateNode() instanceof ArgNode
}
cached
predicate hiddenNode(Node n) { nodeIsHidden(n) }
cached
OutNodeExt getAnOutNodeExt(DataFlowCall call, ReturnKindExt k) {
result = getAnOutNode(call, k.(ValueReturnKind).getKind())
or
exists(ArgNode arg |
result.(PostUpdateNode).getPreUpdateNode() = arg and
arg.argumentOf(call, k.(ParamUpdateReturnKind).getPosition())
)
}
cached
predicate returnNodeExt(Node n, ReturnKindExt k) {
k = TValueReturn(n.(ReturnNode).getKind())
or
exists(ParamNode p, int pos |
parameterValueFlowsToPreUpdate(p, n) and
p.isParameterOf(_, pos) and
k = TParamUpdate(pos)
)
}
cached
predicate castNode(Node n) { n instanceof CastNode }
cached
predicate castingNode(Node n) {
castNode(n) or
n instanceof ParamNode or
n instanceof OutNodeExt or
// For reads, `x.f`, we want to check that the tracked type after the read (which
// is obtained by popping the head of the access path stack) is compatible with
// the type of `x.f`.
read(_, _, n)
}
cached
predicate parameterNode(Node n, DataFlowCallable c, int i) {
n.(ParameterNode).isParameterOf(c, i)
}
cached
predicate argumentNode(Node n, DataFlowCall call, int pos) {
n.(ArgumentNode).argumentOf(call, pos)
}
/**
* Gets a viable target for the lambda call `call`.
*
@@ -261,7 +344,7 @@ private module Cached {
* The instance parameter is considered to have index `-1`.
*/
pragma[nomagic]
private predicate viableParam(DataFlowCall call, int i, ParameterNode p) {
private predicate viableParam(DataFlowCall call, int i, ParamNode p) {
p.isParameterOf(viableCallableExt(call), i)
}
@@ -270,11 +353,11 @@ private module Cached {
* dispatch into account.
*/
cached
predicate viableParamArg(DataFlowCall call, ParameterNode p, ArgumentNode arg) {
predicate viableParamArg(DataFlowCall call, ParamNode p, ArgNode arg) {
exists(int i |
viableParam(call, i, p) and
arg.argumentOf(call, i) and
compatibleTypes(getNodeType(arg), getNodeType(p))
compatibleTypes(getNodeDataFlowType(arg), getNodeDataFlowType(p))
)
}
@@ -312,7 +395,7 @@ private module Cached {
* `read` indicates whether it is contents of `p` that can flow to `node`.
*/
pragma[nomagic]
private predicate parameterValueFlowCand(ParameterNode p, Node node, boolean read) {
private predicate parameterValueFlowCand(ParamNode p, Node node, boolean read) {
p = node and
read = false
or
@@ -325,30 +408,30 @@ private module Cached {
// read
exists(Node mid |
parameterValueFlowCand(p, mid, false) and
readStep(mid, _, node) and
read(mid, _, node) and
read = true
)
or
// flow through: no prior read
exists(ArgumentNode arg |
exists(ArgNode arg |
parameterValueFlowArgCand(p, arg, false) and
argumentValueFlowsThroughCand(arg, node, read)
)
or
// flow through: no read inside method
exists(ArgumentNode arg |
exists(ArgNode arg |
parameterValueFlowArgCand(p, arg, read) and
argumentValueFlowsThroughCand(arg, node, false)
)
}
pragma[nomagic]
private predicate parameterValueFlowArgCand(ParameterNode p, ArgumentNode arg, boolean read) {
private predicate parameterValueFlowArgCand(ParamNode p, ArgNode arg, boolean read) {
parameterValueFlowCand(p, arg, read)
}
pragma[nomagic]
predicate parameterValueFlowsToPreUpdateCand(ParameterNode p, PostUpdateNode n) {
predicate parameterValueFlowsToPreUpdateCand(ParamNode p, PostUpdateNode n) {
parameterValueFlowCand(p, n.getPreUpdateNode(), false)
}
@@ -360,7 +443,7 @@ private module Cached {
* `read` indicates whether it is contents of `p` that can flow to the return
* node.
*/
predicate parameterValueFlowReturnCand(ParameterNode p, ReturnKind kind, boolean read) {
predicate parameterValueFlowReturnCand(ParamNode p, ReturnKind kind, boolean read) {
exists(ReturnNode ret |
parameterValueFlowCand(p, ret, read) and
kind = ret.getKind()
@@ -369,9 +452,9 @@ private module Cached {
pragma[nomagic]
private predicate argumentValueFlowsThroughCand0(
DataFlowCall call, ArgumentNode arg, ReturnKind kind, boolean read
DataFlowCall call, ArgNode arg, ReturnKind kind, boolean read
) {
exists(ParameterNode param | viableParamArg(call, param, arg) |
exists(ParamNode param | viableParamArg(call, param, arg) |
parameterValueFlowReturnCand(param, kind, read)
)
}
@@ -382,14 +465,14 @@ private module Cached {
*
* `read` indicates whether it is contents of `arg` that can flow to `out`.
*/
predicate argumentValueFlowsThroughCand(ArgumentNode arg, Node out, boolean read) {
predicate argumentValueFlowsThroughCand(ArgNode arg, Node out, boolean read) {
exists(DataFlowCall call, ReturnKind kind |
argumentValueFlowsThroughCand0(call, arg, kind, read) and
out = getAnOutNode(call, kind)
)
}
predicate cand(ParameterNode p, Node n) {
predicate cand(ParamNode p, Node n) {
parameterValueFlowCand(p, n, _) and
(
parameterValueFlowReturnCand(p, _, _)
@@ -416,21 +499,21 @@ private module Cached {
* If a read step was taken, then `read` captures the `Content`, the
* container type, and the content type.
*/
predicate parameterValueFlow(ParameterNode p, Node node, ReadStepTypesOption read) {
predicate parameterValueFlow(ParamNode p, Node node, ReadStepTypesOption read) {
parameterValueFlow0(p, node, read) and
if node instanceof CastingNode
then
// normal flow through
read = TReadStepTypesNone() and
compatibleTypes(getNodeType(p), getNodeType(node))
compatibleTypes(getNodeDataFlowType(p), getNodeDataFlowType(node))
or
// getter
compatibleTypes(read.getContentType(), getNodeType(node))
compatibleTypes(read.getContentType(), getNodeDataFlowType(node))
else any()
}
pragma[nomagic]
private predicate parameterValueFlow0(ParameterNode p, Node node, ReadStepTypesOption read) {
private predicate parameterValueFlow0(ParamNode p, Node node, ReadStepTypesOption read) {
p = node and
Cand::cand(p, _) and
read = TReadStepTypesNone()
@@ -447,7 +530,7 @@ private module Cached {
readStepWithTypes(mid, read.getContainerType(), read.getContent(), node,
read.getContentType()) and
Cand::parameterValueFlowReturnCand(p, _, true) and
compatibleTypes(getNodeType(p), read.getContainerType())
compatibleTypes(getNodeDataFlowType(p), read.getContainerType())
)
or
parameterValueFlow0_0(TReadStepTypesNone(), p, node, read)
@@ -455,34 +538,32 @@ private module Cached {
pragma[nomagic]
private predicate parameterValueFlow0_0(
ReadStepTypesOption mustBeNone, ParameterNode p, Node node, ReadStepTypesOption read
ReadStepTypesOption mustBeNone, ParamNode p, Node node, ReadStepTypesOption read
) {
// flow through: no prior read
exists(ArgumentNode arg |
exists(ArgNode arg |
parameterValueFlowArg(p, arg, mustBeNone) and
argumentValueFlowsThrough(arg, read, node)
)
or
// flow through: no read inside method
exists(ArgumentNode arg |
exists(ArgNode arg |
parameterValueFlowArg(p, arg, read) and
argumentValueFlowsThrough(arg, mustBeNone, node)
)
}
pragma[nomagic]
private predicate parameterValueFlowArg(
ParameterNode p, ArgumentNode arg, ReadStepTypesOption read
) {
private predicate parameterValueFlowArg(ParamNode p, ArgNode arg, ReadStepTypesOption read) {
parameterValueFlow(p, arg, read) and
Cand::argumentValueFlowsThroughCand(arg, _, _)
}
pragma[nomagic]
private predicate argumentValueFlowsThrough0(
DataFlowCall call, ArgumentNode arg, ReturnKind kind, ReadStepTypesOption read
DataFlowCall call, ArgNode arg, ReturnKind kind, ReadStepTypesOption read
) {
exists(ParameterNode param | viableParamArg(call, param, arg) |
exists(ParamNode param | viableParamArg(call, param, arg) |
parameterValueFlowReturn(param, kind, read)
)
}
@@ -496,18 +577,18 @@ private module Cached {
* container type, and the content type.
*/
pragma[nomagic]
predicate argumentValueFlowsThrough(ArgumentNode arg, ReadStepTypesOption read, Node out) {
predicate argumentValueFlowsThrough(ArgNode arg, ReadStepTypesOption read, Node out) {
exists(DataFlowCall call, ReturnKind kind |
argumentValueFlowsThrough0(call, arg, kind, read) and
out = getAnOutNode(call, kind)
|
// normal flow through
read = TReadStepTypesNone() and
compatibleTypes(getNodeType(arg), getNodeType(out))
compatibleTypes(getNodeDataFlowType(arg), getNodeDataFlowType(out))
or
// getter
compatibleTypes(getNodeType(arg), read.getContainerType()) and
compatibleTypes(read.getContentType(), getNodeType(out))
compatibleTypes(getNodeDataFlowType(arg), read.getContainerType()) and
compatibleTypes(read.getContentType(), getNodeDataFlowType(out))
)
}
@@ -516,7 +597,7 @@ private module Cached {
* value-preserving steps and a single read step, not taking call
* contexts into account, thus representing a getter-step.
*/
predicate getterStep(ArgumentNode arg, Content c, Node out) {
predicate getterStep(ArgNode arg, Content c, Node out) {
argumentValueFlowsThrough(arg, TReadStepTypesSome(_, c, _), out)
}
@@ -529,7 +610,7 @@ private module Cached {
* container type, and the content type.
*/
private predicate parameterValueFlowReturn(
ParameterNode p, ReturnKind kind, ReadStepTypesOption read
ParamNode p, ReturnKind kind, ReadStepTypesOption read
) {
exists(ReturnNode ret |
parameterValueFlow(p, ret, read) and
@@ -553,7 +634,7 @@ private module Cached {
private predicate mayBenefitFromCallContextExt(DataFlowCall call, DataFlowCallable callable) {
mayBenefitFromCallContext(call, callable)
or
callable = call.getEnclosingCallable() and
callEnclosingCallable(call, callable) and
exists(viableCallableLambda(call, TDataFlowCallSome(_)))
}
@@ -611,7 +692,7 @@ private module Cached {
mayBenefitFromCallContextExt(call, _) and
c = viableCallableExt(call) and
ctxtgts = count(DataFlowCall ctx | c = viableImplInCallContextExt(call, ctx)) and
tgts = strictcount(DataFlowCall ctx | viableCallableExt(ctx) = call.getEnclosingCallable()) and
tgts = strictcount(DataFlowCall ctx | callEnclosingCallable(call, viableCallableExt(ctx))) and
ctxtgts < tgts
)
}
@@ -635,8 +716,7 @@ private module Cached {
* Holds if `p` can flow to the pre-update node associated with post-update
* node `n`, in the same callable, using only value-preserving steps.
*/
cached
predicate parameterValueFlowsToPreUpdate(ParameterNode p, PostUpdateNode n) {
private predicate parameterValueFlowsToPreUpdate(ParamNode p, PostUpdateNode n) {
parameterValueFlow(p, n.getPreUpdateNode(), TReadStepTypesNone())
}
@@ -644,9 +724,9 @@ private module Cached {
Node node1, Content c, Node node2, DataFlowType contentType, DataFlowType containerType
) {
storeStep(node1, c, node2) and
readStep(_, c, _) and
contentType = getNodeType(node1) and
containerType = getNodeType(node2)
read(_, c, _) and
contentType = getNodeDataFlowType(node1) and
containerType = getNodeDataFlowType(node2)
or
exists(Node n1, Node n2 |
n1 = node1.(PostUpdateNode).getPreUpdateNode() and
@@ -654,12 +734,15 @@ private module Cached {
|
argumentValueFlowsThrough(n2, TReadStepTypesSome(containerType, c, contentType), n1)
or
readStep(n2, c, n1) and
contentType = getNodeType(n1) and
containerType = getNodeType(n2)
read(n2, c, n1) and
contentType = getNodeDataFlowType(n1) and
containerType = getNodeDataFlowType(n2)
)
}
cached
predicate read(Node node1, Content c, Node node2) { readStep(node1, c, node2) }
/**
* Holds if data can flow from `node1` to `node2` via a direct assignment to
* `f`.
@@ -678,8 +761,9 @@ private module Cached {
* are aliases. A typical example is a function returning `this`, implementing a fluent
* interface.
*/
cached
predicate reverseStepThroughInputOutputAlias(PostUpdateNode fromNode, PostUpdateNode toNode) {
private predicate reverseStepThroughInputOutputAlias(
PostUpdateNode fromNode, PostUpdateNode toNode
) {
exists(Node fromPre, Node toPre |
fromPre = fromNode.getPreUpdateNode() and
toPre = toNode.getPreUpdateNode()
@@ -688,14 +772,20 @@ private module Cached {
// Does the language-specific simpleLocalFlowStep already model flow
// from function input to output?
fromPre = getAnOutNode(c, _) and
toPre.(ArgumentNode).argumentOf(c, _) and
simpleLocalFlowStep(toPre.(ArgumentNode), fromPre)
toPre.(ArgNode).argumentOf(c, _) and
simpleLocalFlowStep(toPre.(ArgNode), fromPre)
)
or
argumentValueFlowsThrough(toPre, TReadStepTypesNone(), fromPre)
)
}
cached
predicate simpleLocalFlowStepExt(Node node1, Node node2) {
simpleLocalFlowStep(node1, node2) or
reverseStepThroughInputOutputAlias(node1, node2)
}
/**
* Holds if the call context `call` either improves virtual dispatch in
* `callable` or if it allows us to prune unreachable nodes in `callable`.
@@ -704,7 +794,7 @@ private module Cached {
predicate recordDataFlowCallSite(DataFlowCall call, DataFlowCallable callable) {
reducedViableImplInCallContext(_, callable, call)
or
exists(Node n | getNodeEnclosingCallable(n) = callable | isUnreachableInCall(n, call))
exists(Node n | getNodeEnclosingCallable(n) = callable | isUnreachableInCallCached(n, call))
}
cached
@@ -726,12 +816,12 @@ private module Cached {
cached
newtype TLocalFlowCallContext =
TAnyLocalCall() or
TSpecificLocalCall(DataFlowCall call) { isUnreachableInCall(_, call) }
TSpecificLocalCall(DataFlowCall call) { isUnreachableInCallCached(_, call) }
cached
newtype TReturnKindExt =
TValueReturn(ReturnKind kind) or
TParamUpdate(int pos) { exists(ParameterNode p | p.isParameterOf(_, pos)) }
TParamUpdate(int pos) { exists(ParamNode p | p.isParameterOf(_, pos)) }
cached
newtype TBooleanOption =
@@ -761,23 +851,15 @@ private module Cached {
* A `Node` at which a cast can occur such that the type should be checked.
*/
class CastingNode extends Node {
CastingNode() {
this instanceof ParameterNode or
this instanceof CastNode or
this instanceof OutNodeExt or
// For reads, `x.f`, we want to check that the tracked type after the read (which
// is obtained by popping the head of the access path stack) is compatible with
// the type of `x.f`.
readStep(_, _, this)
}
CastingNode() { castingNode(this) }
}
private predicate readStepWithTypes(
Node n1, DataFlowType container, Content c, Node n2, DataFlowType content
) {
readStep(n1, c, n2) and
container = getNodeType(n1) and
content = getNodeType(n2)
read(n1, c, n2) and
container = getNodeDataFlowType(n1) and
content = getNodeDataFlowType(n2)
}
private newtype TReadStepTypesOption =
@@ -854,7 +936,7 @@ class CallContextSomeCall extends CallContextCall, TSomeCall {
override string toString() { result = "CcSomeCall" }
override predicate relevantFor(DataFlowCallable callable) {
exists(ParameterNode p | getNodeEnclosingCallable(p) = callable)
exists(ParamNode p | getNodeEnclosingCallable(p) = callable)
}
override predicate matchesCall(DataFlowCall call) { any() }
@@ -866,7 +948,7 @@ class CallContextReturn extends CallContextNoCall, TReturn {
}
override predicate relevantFor(DataFlowCallable callable) {
exists(DataFlowCall call | this = TReturn(_, call) and call.getEnclosingCallable() = callable)
exists(DataFlowCall call | this = TReturn(_, call) and callEnclosingCallable(call, callable))
}
}
@@ -899,7 +981,7 @@ class LocalCallContextSpecificCall extends LocalCallContext, TSpecificLocalCall
}
private predicate relevantLocalCCtx(DataFlowCall call, DataFlowCallable callable) {
exists(Node n | getNodeEnclosingCallable(n) = callable and isUnreachableInCall(n, call))
exists(Node n | getNodeEnclosingCallable(n) = callable and isUnreachableInCallCached(n, call))
}
/**
@@ -913,26 +995,37 @@ LocalCallContext getLocalCallContext(CallContext ctx, DataFlowCallable callable)
else result instanceof LocalCallContextAny
}
/**
* The value of a parameter at function entry, viewed as a node in a data
* flow graph.
*/
class ParamNode extends Node {
ParamNode() { parameterNode(this, _, _) }
/**
* Holds if this node is the parameter of callable `c` at the specified
* (zero-based) position.
*/
predicate isParameterOf(DataFlowCallable c, int i) { parameterNode(this, c, i) }
}
/** A data-flow node that represents a call argument. */
class ArgNode extends Node {
ArgNode() { argumentNode(this, _, _) }
/** Holds if this argument occurs at the given position in the given call. */
final predicate argumentOf(DataFlowCall call, int pos) { argumentNode(this, call, pos) }
}
/**
* A node from which flow can return to the caller. This is either a regular
* `ReturnNode` or a `PostUpdateNode` corresponding to the value of a parameter.
*/
class ReturnNodeExt extends Node {
ReturnNodeExt() {
this instanceof ReturnNode or
parameterValueFlowsToPreUpdate(_, this)
}
ReturnNodeExt() { returnNodeExt(this, _) }
/** Gets the kind of this returned value. */
ReturnKindExt getKind() {
result = TValueReturn(this.(ReturnNode).getKind())
or
exists(ParameterNode p, int pos |
parameterValueFlowsToPreUpdate(p, this) and
p.isParameterOf(_, pos) and
result = TParamUpdate(pos)
)
}
ReturnKindExt getKind() { returnNodeExt(this, result) }
}
/**
@@ -940,11 +1033,7 @@ class ReturnNodeExt extends Node {
* or a post-update node associated with a call argument.
*/
class OutNodeExt extends Node {
OutNodeExt() {
this instanceof OutNode
or
this.(PostUpdateNode).getPreUpdateNode() instanceof ArgumentNode
}
OutNodeExt() { outNodeExt(this) }
}
/**
@@ -957,7 +1046,7 @@ abstract class ReturnKindExt extends TReturnKindExt {
abstract string toString();
/** Gets a node corresponding to data flow out of `call`. */
abstract OutNodeExt getAnOutNode(DataFlowCall call);
final OutNodeExt getAnOutNode(DataFlowCall call) { result = getAnOutNodeExt(call, this) }
}
class ValueReturnKind extends ReturnKindExt, TValueReturn {
@@ -968,10 +1057,6 @@ class ValueReturnKind extends ReturnKindExt, TValueReturn {
ReturnKind getKind() { result = kind }
override string toString() { result = kind.toString() }
override OutNodeExt getAnOutNode(DataFlowCall call) {
result = getAnOutNode(call, this.getKind())
}
}
class ParamUpdateReturnKind extends ReturnKindExt, TParamUpdate {
@@ -982,13 +1067,6 @@ class ParamUpdateReturnKind extends ReturnKindExt, TParamUpdate {
int getPosition() { result = pos }
override string toString() { result = "param update " + pos }
override OutNodeExt getAnOutNode(DataFlowCall call) {
exists(ArgumentNode arg |
result.(PostUpdateNode).getPreUpdateNode() = arg and
arg.argumentOf(call, this.getPosition())
)
}
}
/** A callable tagged with a relevant return kind. */
@@ -1015,10 +1093,13 @@ class ReturnPosition extends TReturnPosition0 {
*/
pragma[inline]
DataFlowCallable getNodeEnclosingCallable(Node n) {
exists(Node n0 |
pragma[only_bind_into](n0) = n and
pragma[only_bind_into](result) = n0.getEnclosingCallable()
)
nodeEnclosingCallable(pragma[only_bind_out](n), pragma[only_bind_into](result))
}
/** Gets the type of `n` used for type pruning. */
pragma[inline]
DataFlowType getNodeDataFlowType(Node n) {
nodeDataFlowType(pragma[only_bind_out](n), pragma[only_bind_into](result))
}
pragma[noinline]
@@ -1042,7 +1123,7 @@ predicate resolveReturn(CallContext cc, DataFlowCallable callable, DataFlowCall
cc instanceof CallContextAny and callable = viableCallableExt(call)
or
exists(DataFlowCallable c0, DataFlowCall call0 |
call0.getEnclosingCallable() = callable and
callEnclosingCallable(call0, callable) and
cc = TReturn(c0, call0) and
c0 = prunedViableImplInCallContextReverse(call0, call)
)
@@ -1063,8 +1144,6 @@ DataFlowCallable resolveCall(DataFlowCall call, CallContext cc) {
result = viableCallableExt(call) and cc instanceof CallContextReturn
}
predicate read = readStep/3;
/** An optional Boolean value. */
class BooleanOption extends TBooleanOption {
string toString() {
@@ -1116,7 +1195,7 @@ abstract class AccessPathFront extends TAccessPathFront {
TypedContent getHead() { this = TFrontHead(result) }
predicate isClearedAt(Node n) { clearsContent(n, getHead().getContent()) }
predicate isClearedAt(Node n) { clearsContentCached(n, getHead().getContent()) }
}
class AccessPathFrontNil extends AccessPathFront, TFrontNil {

2
python/ql/src/semmle/python/dataflow/new/internal/DataFlowPrivate.qll
View File

@@ -228,7 +228,6 @@ module EssaFlow {
* data flow. It is a strict subset of the `localFlowStep` predicate, as it
* excludes SSA flow through instance fields.
*/
cached
predicate simpleLocalFlowStep(Node nodeFrom, Node nodeTo) {
// If there is ESSA-flow out of a node `node`, we want flow
// both out of `node` and any post-update node of `node`.
@@ -1559,7 +1558,6 @@ predicate kwUnpackReadStep(CfgNode nodeFrom, DictionaryElementContent c, Node no
* any value stored inside `f` is cleared at the pre-update node associated with `x`
* in `x.f = newValue`.
*/
cached
predicate clearsContent(Node n, Content c) {
exists(CallNode call, CallableValue callable, string name |
call_unpacks(call, _, callable, name, _) and

62
python/ql/src/semmle/python/dataflow/new/internal/TaintTrackingPrivate.qll
View File

@@ -9,36 +9,42 @@ private import semmle.python.dataflow.new.internal.TaintTrackingPublic
*/
predicate defaultTaintSanitizer(DataFlow::Node node) { none() }
/**
* Holds if the additional step from `nodeFrom` to `nodeTo` should be included in all
* global taint flow configurations.
*/
predicate defaultAdditionalTaintStep(DataFlow::Node nodeFrom, DataFlow::Node nodeTo) {
localAdditionalTaintStep(nodeFrom, nodeTo)
or
any(AdditionalTaintStep a).step(nodeFrom, nodeTo)
private module Cached {
/**
* Holds if the additional step from `nodeFrom` to `nodeTo` should be included in all
* global taint flow configurations.
*/
cached
predicate defaultAdditionalTaintStep(DataFlow::Node nodeFrom, DataFlow::Node nodeTo) {
localAdditionalTaintStep(nodeFrom, nodeTo)
or
any(AdditionalTaintStep a).step(nodeFrom, nodeTo)
}
/**
* Holds if taint can flow in one local step from `nodeFrom` to `nodeTo` excluding
* local data flow steps. That is, `nodeFrom` and `nodeTo` are likely to represent
* different objects.
*/
cached
predicate localAdditionalTaintStep(DataFlow::Node nodeFrom, DataFlow::Node nodeTo) {
concatStep(nodeFrom, nodeTo)
or
subscriptStep(nodeFrom, nodeTo)
or
stringManipulation(nodeFrom, nodeTo)
or
containerStep(nodeFrom, nodeTo)
or
copyStep(nodeFrom, nodeTo)
or
forStep(nodeFrom, nodeTo)
or
unpackingAssignmentStep(nodeFrom, nodeTo)
}
}
/**
* Holds if taint can flow in one local step from `nodeFrom` to `nodeTo` excluding
* local data flow steps. That is, `nodeFrom` and `nodeTo` are likely to represent
* different objects.
*/
predicate localAdditionalTaintStep(DataFlow::Node nodeFrom, DataFlow::Node nodeTo) {
concatStep(nodeFrom, nodeTo)
or
subscriptStep(nodeFrom, nodeTo)
or
stringManipulation(nodeFrom, nodeTo)
or
containerStep(nodeFrom, nodeTo)
or
copyStep(nodeFrom, nodeTo)
or
forStep(nodeFrom, nodeTo)
or
unpackingAssignmentStep(nodeFrom, nodeTo)
}
import Cached
/**
* Holds if taint can flow from `nodeFrom` to `nodeTo` with a step related to concatenation.

69
python/ql/src/experimental/typetracking/TypeTracker.qll → python/ql/src/semmle/python/dataflow/new/internal/TypeTracker.qll
View File

@@ -57,11 +57,60 @@ class StepSummary extends TStepSummary {
module StepSummary {
/**
* Gets the summary that corresponds to having taken a forwards
* heap and/or inter-procedural step from `nodeFrom` to `nodeTo`.
* heap and/or intra-procedural step from `nodeFrom` to `nodeTo`.
*
* Steps contained in this predicate should _not_ depend on the call graph.
*/
cached
private predicate stepNoCall(LocalSourceNode nodeFrom, LocalSourceNode nodeTo, StepSummary summary) {
exists(Node mid | nodeFrom.flowsTo(mid) and smallstepNoCall(mid, nodeTo, summary))
}
/**
* Gets the summary that corresponds to having taken a forwards
* inter-procedural step from `nodeFrom` to `nodeTo`.
*/
cached
private predicate stepCall(LocalSourceNode nodeFrom, LocalSourceNode nodeTo, StepSummary summary) {
exists(Node mid | nodeFrom.flowsTo(mid) and smallstepCall(mid, nodeTo, summary))
}
/**
* Gets the summary that corresponds to having taken a forwards
* heap and/or inter-procedural step from `nodeFrom` to `nodeTo`.
*
* This predicate is inlined, which enables better join-orders when
* the call graph construction and type tracking are mutually recursive.
* In such cases, non-linear recursion involving `step` will be limited
* to non-linear recursion for the parts of `step` that involve the
* call graph.
*/
pragma[inline]
predicate step(LocalSourceNode nodeFrom, LocalSourceNode nodeTo, StepSummary summary) {
exists(Node mid | nodeFrom.flowsTo(mid) and smallstep(mid, nodeTo, summary))
stepNoCall(nodeFrom, nodeTo, summary)
or
stepCall(nodeFrom, nodeTo, summary)
}
pragma[noinline]
private predicate smallstepNoCall(Node nodeFrom, LocalSourceNode nodeTo, StepSummary summary) {
jumpStep(nodeFrom, nodeTo) and
summary = LevelStep()
or
exists(string content |
localSourceStoreStep(nodeFrom, nodeTo, content) and
summary = StoreStep(content)
or
basicLoadStep(nodeFrom, nodeTo, content) and summary = LoadStep(content)
)
}
pragma[noinline]
private predicate smallstepCall(Node nodeFrom, LocalSourceNode nodeTo, StepSummary summary) {
callStep(nodeFrom, nodeTo) and summary = CallStep()
or
returnStep(nodeFrom, nodeTo) and
summary = ReturnStep()
}
/**
@@ -71,21 +120,11 @@ module StepSummary {
* Unlike `StepSummary::step`, this predicate does not compress
* type-preserving steps.
*/
pragma[inline]
predicate smallstep(Node nodeFrom, LocalSourceNode nodeTo, StepSummary summary) {
jumpStep(nodeFrom, nodeTo) and
summary = LevelStep()
smallstepNoCall(nodeFrom, nodeTo, summary)
or
callStep(nodeFrom, nodeTo) and summary = CallStep()
or
returnStep(nodeFrom, nodeTo) and
summary = ReturnStep()
or
exists(string content |
localSourceStoreStep(nodeFrom, nodeTo, content) and
summary = StoreStep(content)
or
basicLoadStep(nodeFrom, nodeTo, content) and summary = LoadStep(content)
)
smallstepCall(nodeFrom, nodeTo, summary)
}
/**

0
python/ql/src/experimental/typetracking/TypeTrackerSpecific.qll → python/ql/src/semmle/python/dataflow/new/internal/TypeTrackerSpecific.qll
View File

562
python/ql/src/semmle/python/frameworks/Aiohttp.qll Normal file
View File

@@ -0,0 +1,562 @@
/**
* Provides classes modeling security-relevant aspects of the `aiohttp` PyPI package.
* See https://docs.aiohttp.org/en/stable/index.html
*/
private import python
private import semmle.python.dataflow.new.DataFlow
private import semmle.python.dataflow.new.RemoteFlowSources
private import semmle.python.dataflow.new.TaintTracking
private import semmle.python.Concepts
private import semmle.python.ApiGraphs
private import semmle.python.frameworks.internal.PoorMansFunctionResolution
private import semmle.python.frameworks.internal.SelfRefMixin
private import semmle.python.frameworks.Multidict
private import semmle.python.frameworks.Yarl
/**
* INTERNAL: Do not use.
*
* Provides models for the web server part (`aiohttp.web`) of the `aiohttp` PyPI package.
* See https://docs.aiohttp.org/en/stable/web.html
*/
module AiohttpWebModel {
/**
* Provides models for the `aiohttp.web.View` class and subclasses.
*
* See https://docs.aiohttp.org/en/stable/web_reference.html#view.
*/
module View {
/** Gets a reference to the `aiohttp.web.View` class or any subclass. */
API::Node subclassRef() {
result = API::moduleImport("aiohttp").getMember("web").getMember("View").getASubclass*()
}
}
// -- route modeling --
/** Gets a reference to an `aiohttp.web.Application` instance. */
API::Node applicationInstance() {
// Not sure whether you're allowed to add routes _after_ starting the app, for
// example in the middle of handling a http request... but I'm guessing that for 99%
// for all code, not modeling that `request.app` is a reference to an application
// should be good enough for the route-setup part of the modeling :+1:
result = API::moduleImport("aiohttp").getMember("web").getMember("Application").getReturn()
}
/** Gets a reference to an `aiohttp.web.UrlDispatcher` instance. */
API::Node urlDispathcerInstance() {
result = API::moduleImport("aiohttp").getMember("web").getMember("UrlDispatcher").getReturn()
or
result = applicationInstance().getMember("router")
}
/**
* A route setup in `aiohttp.web`. Since all route-setups can technically use either
* coroutines or view-classes as the handler argument (although that's not how you're
* **supposed** to do things), we also need to handle this.
*
* Extend this class to refine existing API models. If you want to model new APIs,
* extend `AiohttpRouteSetup::Range` instead.
*/
class AiohttpRouteSetup extends HTTP::Server::RouteSetup::Range {
AiohttpRouteSetup::Range range;
AiohttpRouteSetup() { this = range }
override Parameter getARoutedParameter() { none() }
override string getFramework() { result = "aiohttp.web" }
/** Gets the argument specifying the handler (either a coroutine or a view-class). */
DataFlow::Node getHandlerArg() { result = range.getHandlerArg() }
override DataFlow::Node getUrlPatternArg() { result = range.getUrlPatternArg() }
/** Gets the view-class that is referenced in the view-class handler argument, if any. */
Class getViewClass() { result = range.getViewClass() }
override Function getARequestHandler() { result = range.getARequestHandler() }
}
/** Provides a class for modeling new aiohttp.web route setups. */
private module AiohttpRouteSetup {
/**
* A route setup in `aiohttp.web`. Since all route-setups can technically use either
* coroutines or view-classes as the handler argument (although that's not how you're
* **supposed** to do things), we also need to handle this.
*
* Extend this class to model new APIs. If you want to refine existing API models,
* extend `AiohttpRouteSetup` instead.
*/
abstract class Range extends DataFlow::Node {
/** Gets the argument used to set the URL pattern. */
abstract DataFlow::Node getUrlPatternArg();
/** Gets the argument specifying the handler (either a coroutine or a view-class). */
abstract DataFlow::Node getHandlerArg();
/** Gets the view-class that is referenced in the view-class handler argument, if any. */
Class getViewClass() { result = getBackTrackedViewClass(this.getHandlerArg()) }
/**
* Gets a function that will handle incoming requests for this route, if any.
*
* NOTE: This will be modified in the near future to have a `RequestHandler` result, instead of a `Function`.
*/
Function getARequestHandler() {
this.getHandlerArg() = poorMansFunctionTracker(result)
or
result = this.getViewClass().(AiohttpViewClass).getARequestHandler()
}
}
/**
* Gets a reference to a class, that has been backtracked from the view-class handler
* argument `origin` (to a route-setup for view-classes).
*/
private DataFlow::LocalSourceNode viewClassBackTracker(
DataFlow::TypeBackTracker t, DataFlow::Node origin
) {
t.start() and
origin = any(Range rs).getHandlerArg() and
result = origin.getALocalSource()
or
exists(DataFlow::TypeBackTracker t2 |
result = viewClassBackTracker(t2, origin).backtrack(t2, t)
)
}
/**
* Gets a reference to a class, that has been backtracked from the view-class handler
* argument `origin` (to a route-setup for view-classes).
*/
DataFlow::LocalSourceNode viewClassBackTracker(DataFlow::Node origin) {
result = viewClassBackTracker(DataFlow::TypeBackTracker::end(), origin)
}
Class getBackTrackedViewClass(DataFlow::Node origin) {
result.getParent() = viewClassBackTracker(origin).asExpr()
}
}
/** An aiohttp route setup that uses coroutines (async function) as request handlers. */
class AiohttpCoroutineRouteSetup extends AiohttpRouteSetup {
AiohttpCoroutineRouteSetup() { this.getHandlerArg() = poorMansFunctionTracker(_) }
}
/** An aiohttp route setup that uses view-classes as request handlers. */
class AiohttpViewRouteSetup extends AiohttpRouteSetup {
AiohttpViewRouteSetup() { exists(this.getViewClass()) }
}
/**
* A route-setup from
* - `add_route`, `add_view`, `add_get`, `add_post`, , etc. on an `aiohttp.web.UrlDispatcher`.
* - `route`, `view`, `get`, `post`, etc. functions from `aiohttp.web`.
*/
class AiohttpAddRouteCall extends AiohttpRouteSetup::Range, DataFlow::CallCfgNode {
/** At what index route arguments starts, so we can handle "route" version together with get/post/... */
int routeArgsStart;
AiohttpAddRouteCall() {
exists(string funcName |
funcName = HTTP::httpVerbLower() and
routeArgsStart = 0
or
funcName = "view" and
routeArgsStart = 0
or
funcName = "route" and
routeArgsStart = 1
|
this = urlDispathcerInstance().getMember("add_" + funcName).getACall()
or
this = API::moduleImport("aiohttp").getMember("web").getMember(funcName).getACall()
)
}
override DataFlow::Node getUrlPatternArg() {
result in [this.getArg(routeArgsStart + 0), this.getArgByName("path")]
}
override DataFlow::Node getHandlerArg() {
result in [this.getArg(routeArgsStart + 1), this.getArgByName("handler")]
}
}
/** A route-setup using a decorator, such as `route`, `view`, `get`, `post`, etc. on an `aiohttp.web.RouteTableDef`. */
class AiohttpDecoratorRouteSetup extends AiohttpRouteSetup::Range, DataFlow::CallCfgNode {
/** At what index route arguments starts, so we can handle "route" version together with get/post/... */
int routeArgsStart;
AiohttpDecoratorRouteSetup() {
exists(string decoratorName |
decoratorName = HTTP::httpVerbLower() and
routeArgsStart = 0
or
decoratorName = "view" and
routeArgsStart = 0
or
decoratorName = "route" and
routeArgsStart = 1
|
this =
API::moduleImport("aiohttp")
.getMember("web")
.getMember("RouteTableDef")
.getReturn()
.getMember(decoratorName)
.getACall()
)
}
override DataFlow::Node getUrlPatternArg() {
result in [this.getArg(routeArgsStart + 0), this.getArgByName("path")]
}
override DataFlow::Node getHandlerArg() { none() }
override Class getViewClass() { result.getADecorator() = this.asExpr() }
override Function getARequestHandler() {
// we're decorating a class
exists(this.getViewClass()) and
result = super.getARequestHandler()
or
// we're decorating a function
not exists(this.getViewClass()) and
result.getADecorator() = this.asExpr()
}
}
/** A class that we consider an aiohttp.web View class. */
abstract class AiohttpViewClass extends Class, SelfRefMixin {
/** Gets a function that could handle incoming requests, if any. */
Function getARequestHandler() {
// TODO: This doesn't handle attribute assignment. Should be OK, but analysis is not as complete as with
// points-to and `.lookup`, which would handle `post = my_post_handler` inside class def
result = this.getAMethod() and
result.getName() = HTTP::httpVerbLower()
}
}
/** A class that has a super-type which is an aiohttp.web View class. */
class AiohttpViewClassFromSuperClass extends AiohttpViewClass {
AiohttpViewClassFromSuperClass() { this.getABase() = View::subclassRef().getAUse().asExpr() }
}
/** A class that is used in a route-setup, therefore being considered an aiohttp.web View class. */
class AiohttpViewClassFromRouteSetup extends AiohttpViewClass {
AiohttpViewClassFromRouteSetup() { this = any(AiohttpRouteSetup rs).getViewClass() }
}
/** A request handler defined in an `aiohttp.web` view class, that has no known route. */
private class AiohttpViewClassRequestHandlerWithoutKnownRoute extends HTTP::Server::RequestHandler::Range {
AiohttpViewClassRequestHandlerWithoutKnownRoute() {
exists(AiohttpViewClass vc | vc.getARequestHandler() = this) and
not exists(AiohttpRouteSetup setup | setup.getARequestHandler() = this)
}
override Parameter getARoutedParameter() { none() }
override string getFramework() { result = "aiohttp.web" }
}
// ---------------------------------------------------------------------------
// aiohttp.web.Request taint modeling
// ---------------------------------------------------------------------------
/**
* Provides models for the `aiohttp.web.Request` class
*
* See https://docs.aiohttp.org/en/stable/web_reference.html#request-and-base-request
*/
module Request {
/**
* A source of instances of `aiohttp.web.Request`, extend this class to model new instances.
*
* This can include instantiations of the class, return values from function
* calls, or a special parameter that will be set when functions are called by an external
* library.
*
* Use `Request::instance()` predicate to get
* references to instances of `aiohttp.web.Request`.
*/
abstract class InstanceSource extends DataFlow::LocalSourceNode { }
/** Gets a reference to an instance of `aiohttp.web.Request`. */
private DataFlow::LocalSourceNode instance(DataFlow::TypeTracker t) {
t.start() and
result instanceof InstanceSource
or
exists(DataFlow::TypeTracker t2 | result = instance(t2).track(t2, t))
}
/** Gets a reference to an instance of `aiohttp.web.Request`. */
DataFlow::Node instance() { instance(DataFlow::TypeTracker::end()).flowsTo(result) }
}
/**
* Provides models for the `aiohttp.StreamReader` class
*
* See https://docs.aiohttp.org/en/stable/streams.html#aiohttp.StreamReader
*/
module StreamReader {
/**
* A source of instances of `aiohttp.StreamReader`, extend this class to model new instances.
*
* This can include instantiations of the class, return values from function
* calls, or a special parameter that will be set when functions are called by an external
* library.
*
* Use `StreamReader::instance()` predicate to get
* references to instances of `aiohttp.StreamReader`.
*/
abstract class InstanceSource extends DataFlow::LocalSourceNode { }
/** Gets a reference to an instance of `aiohttp.StreamReader`. */
private DataFlow::LocalSourceNode instance(DataFlow::TypeTracker t) {
t.start() and
result instanceof InstanceSource
or
exists(DataFlow::TypeTracker t2 | result = instance(t2).track(t2, t))
}
/** Gets a reference to an instance of `aiohttp.StreamReader`. */
DataFlow::Node instance() { instance(DataFlow::TypeTracker::end()).flowsTo(result) }
/**
* Taint propagation for `aiohttp.StreamReader`.
*/
private class AiohttpStreamReaderAdditionalTaintStep extends TaintTracking::AdditionalTaintStep {
override predicate step(DataFlow::Node nodeFrom, DataFlow::Node nodeTo) {
// Methods
//
// TODO: When we have tools that make it easy, model these properly to handle
// `meth = obj.meth; meth()`. Until then, we'll use this more syntactic approach
// (since it allows us to at least capture the most common cases).
nodeFrom = StreamReader::instance() and
exists(DataFlow::AttrRead attr | attr.getObject() = nodeFrom |
// normal methods
attr.getAttributeName() in ["read_nowait"] and
nodeTo.(DataFlow::CallCfgNode).getFunction() = attr
or
// async methods
exists(Await await, DataFlow::CallCfgNode call |
attr.getAttributeName() in [
"read", "readany", "readexactly", "readline", "readchunk", "iter_chunked",
"iter_any", "iter_chunks"
] and
call.getFunction() = attr and
await.getValue() = call.asExpr() and
nodeTo.asExpr() = await
)
)
}
}
}
/**
* A parameter that will receive an `aiohttp.web.Request` instance when a request
* handler is invoked.
*/
class AiohttpRequestHandlerRequestParam extends Request::InstanceSource, RemoteFlowSource::Range,
DataFlow::ParameterNode {
AiohttpRequestHandlerRequestParam() {
exists(Function requestHandler |
requestHandler = any(AiohttpCoroutineRouteSetup setup).getARequestHandler() and
// We select the _last_ parameter for the request since that is what they do in
// `aiohttp-jinja2`.
// https://github.com/aio-libs/aiohttp-jinja2/blob/7fb4daf2c3003921d34031d38c2311ee0e02c18b/aiohttp_jinja2/__init__.py#L235
//
// I assume that is just to handle cases such as the one below
// ```py
// class MyCustomHandlerClass:
// async def foo_handler(self, request):
// ...
//
// my_custom_handler = MyCustomHandlerClass()
// app.router.add_get("/MyCustomHandlerClass/foo", my_custom_handler.foo_handler)
// ```
this.getParameter() =
max(Parameter param, int i | param = requestHandler.getArg(i) | param order by i)
)
}
override string getSourceType() { result = "aiohttp.web.Request" }
}
/**
* A read of the `request` attribute on an instance of an aiohttp.web View class,
* which is the request being processed currently.
*/
class AiohttpViewClassRequestAttributeRead extends Request::InstanceSource,
RemoteFlowSource::Range, DataFlow::Node {
AiohttpViewClassRequestAttributeRead() {
this.(DataFlow::AttrRead).getObject() = any(AiohttpViewClass vc).getASelfRef() and
this.(DataFlow::AttrRead).getAttributeName() = "request"
}
override string getSourceType() {
result = "aiohttp.web.Request from self.request in View class"
}
}
/**
* Taint propagation for `aiohttp.web.Request`.
*
* See https://docs.aiohttp.org/en/stable/web_reference.html#request-and-base-request
*/
private class AiohttpRequestAdditionalTaintStep extends TaintTracking::AdditionalTaintStep {
override predicate step(DataFlow::Node nodeFrom, DataFlow::Node nodeTo) {
// Methods
//
// TODO: When we have tools that make it easy, model these properly to handle
// `meth = obj.meth; meth()`. Until then, we'll use this more syntactic approach
// (since it allows us to at least capture the most common cases).
nodeFrom = Request::instance() and
exists(DataFlow::AttrRead attr | attr.getObject() = nodeFrom |
// normal methods
attr.getAttributeName() in ["clone", "get_extra_info"] and
nodeTo.(DataFlow::CallCfgNode).getFunction() = attr
or
// async methods
exists(Await await, DataFlow::CallCfgNode call |
attr.getAttributeName() in ["read", "text", "json", "multipart", "post"] and
call.getFunction() = attr and
await.getValue() = call.asExpr() and
nodeTo.asExpr() = await
)
)
or
// Attributes
nodeFrom = Request::instance() and
nodeTo.(DataFlow::AttrRead).getObject() = nodeFrom and
nodeTo.(DataFlow::AttrRead).getAttributeName() in [
"url", "rel_url", "forwarded", "host", "remote", "path", "path_qs", "raw_path", "query",
"headers", "transport", "cookies", "content", "_payload", "content_type", "charset",
"http_range", "if_modified_since", "if_unmodified_since", "if_range", "match_info"
]
}
}
/** An attribute read on an `aiohttp.web.Request` that is a `MultiDictProxy` instance. */
class AiohttpRequestMultiDictProxyInstances extends Multidict::MultiDictProxy::InstanceSource {
AiohttpRequestMultiDictProxyInstances() {
this.(DataFlow::AttrRead).getObject() = Request::instance() and
this.(DataFlow::AttrRead).getAttributeName() in ["query", "headers"]
or
// Handle the common case of `x = await request.post()`
// but don't try to handle anything else, since we don't have an easy way to do this yet.
// TODO: more complete handling of `await request.post()`
exists(Await await, DataFlow::CallCfgNode call, DataFlow::AttrRead read |
this.asExpr() = await
|
read.(DataFlow::AttrRead).getObject() = Request::instance() and
read.(DataFlow::AttrRead).getAttributeName() = "post" and
call.getFunction() = read and
await.getValue() = call.asExpr()
)
}
}
/** An attribute read on an `aiohttp.web.Request` that is a `yarl.URL` instance. */
class AiohttpRequestYarlUrlInstances extends Yarl::Url::InstanceSource {
AiohttpRequestYarlUrlInstances() {
this.(DataFlow::AttrRead).getObject() = Request::instance() and
this.(DataFlow::AttrRead).getAttributeName() in ["url", "rel_url"]
}
}
/** An attribute read on an `aiohttp.web.Request` that is a `aiohttp.StreamReader` instance. */
class AiohttpRequestStreamReaderInstances extends StreamReader::InstanceSource {
AiohttpRequestStreamReaderInstances() {
this.(DataFlow::AttrRead).getObject() = Request::instance() and
this.(DataFlow::AttrRead).getAttributeName() in ["content", "_payload"]
}
}
// ---------------------------------------------------------------------------
// aiohttp.web Response modeling
// ---------------------------------------------------------------------------
/**
* An instantiation of `aiohttp.web.Response`.
*
* Note that `aiohttp.web.HTTPException` (and it's subclasses) is a subclass of `aiohttp.web.Response`.
*
* See
* - https://docs.aiohttp.org/en/stable/web_reference.html#aiohttp.web.Response
* - https://docs.aiohttp.org/en/stable/web_quickstart.html#aiohttp-web-exceptions
*/
class AiohttpWebResponseInstantiation extends HTTP::Server::HttpResponse::Range,
DataFlow::CallCfgNode {
AiohttpWebResponseInstantiation() {
this = API::moduleImport("aiohttp").getMember("web").getMember("Response").getACall()
or
exists(string httpExceptionClassName |
httpExceptionClassName in [
"HTTPException", "HTTPSuccessful", "HTTPOk", "HTTPCreated", "HTTPAccepted",
"HTTPNonAuthoritativeInformation", "HTTPNoContent", "HTTPResetContent",
"HTTPPartialContent", "HTTPRedirection", "HTTPMultipleChoices", "HTTPMovedPermanently",
"HTTPFound", "HTTPSeeOther", "HTTPNotModified", "HTTPUseProxy", "HTTPTemporaryRedirect",
"HTTPPermanentRedirect", "HTTPError", "HTTPClientError", "HTTPBadRequest",
"HTTPUnauthorized", "HTTPPaymentRequired", "HTTPForbidden", "HTTPNotFound",
"HTTPMethodNotAllowed", "HTTPNotAcceptable", "HTTPProxyAuthenticationRequired",
"HTTPRequestTimeout", "HTTPConflict", "HTTPGone", "HTTPLengthRequired",
"HTTPPreconditionFailed", "HTTPRequestEntityTooLarge", "HTTPRequestURITooLong",
"HTTPUnsupportedMediaType", "HTTPRequestRangeNotSatisfiable", "HTTPExpectationFailed",
"HTTPMisdirectedRequest", "HTTPUnprocessableEntity", "HTTPFailedDependency",
"HTTPUpgradeRequired", "HTTPPreconditionRequired", "HTTPTooManyRequests",
"HTTPRequestHeaderFieldsTooLarge", "HTTPUnavailableForLegalReasons", "HTTPServerError",
"HTTPInternalServerError", "HTTPNotImplemented", "HTTPBadGateway",
"HTTPServiceUnavailable", "HTTPGatewayTimeout", "HTTPVersionNotSupported",
"HTTPVariantAlsoNegotiates", "HTTPInsufficientStorage", "HTTPNotExtended",
"HTTPNetworkAuthenticationRequired"
] and
this =
API::moduleImport("aiohttp").getMember("web").getMember(httpExceptionClassName).getACall()
)
}
override DataFlow::Node getBody() {
result in [this.getArgByName("text"), this.getArgByName("body")]
}
override DataFlow::Node getMimetypeOrContentTypeArg() {
result = this.getArgByName("content_type")
}
override string getMimetypeDefault() {
exists(this.getArgByName("text")) and
result = "text/plain"
or
not exists(this.getArgByName("text")) and
result = "application/octet-stream"
}
}
/**
* An instantiation of aiohttp.web HTTP redirect exception.
*
* See the part about redirects at https://docs.aiohttp.org/en/stable/web_quickstart.html#aiohttp-web-exceptions
*/
class AiohttpRedirectExceptionInstantiation extends AiohttpWebResponseInstantiation,
HTTP::Server::HttpRedirectResponse::Range {
AiohttpRedirectExceptionInstantiation() {
exists(string httpRedirectExceptionClassName |
httpRedirectExceptionClassName in [
"HTTPMultipleChoices", "HTTPMovedPermanently", "HTTPFound", "HTTPSeeOther",
"HTTPNotModified", "HTTPUseProxy", "HTTPTemporaryRedirect", "HTTPPermanentRedirect"
] and
this =
API::moduleImport("aiohttp")
.getMember("web")
.getMember(httpRedirectExceptionClassName)
.getACall()
)
}
override DataFlow::Node getRedirectLocation() {
result in [this.getArg(0), this.getArgByName("location")]
}
}
}

117
python/ql/src/semmle/python/frameworks/Cryptodome.qll
View File

@@ -17,7 +17,6 @@ private import semmle.python.ApiGraphs
* See https://pycryptodome.readthedocs.io/en/latest/
*/
private module CryptodomeModel {
// ---------------------------------------------------------------------------
/**
* A call to `Cryptodome.PublicKey.RSA.generate`/`Crypto.PublicKey.RSA.generate`
*
@@ -101,4 +100,120 @@ private module CryptodomeModel {
// Note: There is not really a key-size argument, since it's always specified by the curve.
override DataFlow::Node getKeySizeArg() { none() }
}
/**
* A cryptographic operation on an instance from the `Cipher` subpackage of `Cryptodome`/`Crypto`.
*/
class CryptodomeGenericCipherOperation extends Cryptography::CryptographicOperation::Range,
DataFlow::CallCfgNode {
string methodName;
string cipherName;
CryptodomeGenericCipherOperation() {
methodName in [
"encrypt", "decrypt", "verify", "update", "hexverify", "encrypt_and_digest",
"decrypt_and_verify"
] and
this =
API::moduleImport(["Crypto", "Cryptodome"])
.getMember(["Cipher"])
.getMember(cipherName)
.getMember("new")
.getReturn()
.getMember(methodName)
.getACall()
}
override Cryptography::CryptographicAlgorithm getAlgorithm() { result.matchesName(cipherName) }
override DataFlow::Node getAnInput() {
methodName = "encrypt" and
result in [this.getArg(0), this.getArgByName(["message", "plaintext"])]
or
methodName = "decrypt" and
result in [this.getArg(0), this.getArgByName("ciphertext")]
or
// for the following methods, method signatures can be found in
// https://pycryptodome.readthedocs.io/en/latest/src/cipher/modern.html
methodName in ["update"] and
result in [this.getArg(0), this.getArgByName("data")]
or
// although `mac_tag` is used as the parameter name in the spec above, some implementations use `received_mac_tag`, for an example, see
// https://github.com/Legrandin/pycryptodome/blob/5dace638b70ac35bb5d9b565f3e75f7869c9d851/lib/Crypto/Cipher/ChaCha20_Poly1305.py#L207
methodName in ["verify"] and
result in [this.getArg(0), this.getArgByName(["mac_tag", "received_mac_tag"])]
or
methodName in ["hexverify"] and
result in [this.getArg(0), this.getArgByName("mac_tag_hex")]
or
methodName in ["encrypt_and_digest"] and
result in [this.getArg(0), this.getArgByName("plaintext")]
or
methodName in ["decrypt_and_verify"] and
result in [
this.getArg(0), this.getArgByName("ciphertext"), this.getArg(1),
this.getArgByName("mac_tag")
]
}
}
/**
* A cryptographic operation on an instance from the `Signature` subpackage of `Cryptodome`/`Crypto`.
*/
class CryptodomeGenericSignatureOperation extends Cryptography::CryptographicOperation::Range,
DataFlow::CallCfgNode {
string methodName;
string signatureName;
CryptodomeGenericSignatureOperation() {
methodName in ["sign", "verify"] and
this =
API::moduleImport(["Crypto", "Cryptodome"])
.getMember(["Signature"])
.getMember(signatureName)
.getMember("new")
.getReturn()
.getMember(methodName)
.getACall()
}
override Cryptography::CryptographicAlgorithm getAlgorithm() {
result.matchesName(signatureName)
}
override DataFlow::Node getAnInput() {
methodName = "sign" and
result in [this.getArg(0), this.getArgByName("msg_hash")] // Cryptodome.Hash instance
or
methodName in ["verify"] and
(
result in [this.getArg(0), this.getArgByName(["msg_hash"])] // Cryptodome.Hash instance
or
result in [this.getArg(1), this.getArgByName(["signature"])]
)
}
}
/**
* A cryptographic operation on an instance from the `Hash` subpackage of `Cryptodome`/`Crypto`.
*/
class CryptodomeGenericHashOperation extends Cryptography::CryptographicOperation::Range,
DataFlow::CallCfgNode {
string hashName;
CryptodomeGenericHashOperation() {
exists(API::Node hashModule |
hashModule =
API::moduleImport(["Crypto", "Cryptodome"]).getMember(["Hash"]).getMember(hashName)
|
this = hashModule.getMember("new").getACall()
or
this = hashModule.getMember("new").getReturn().getMember("update").getACall()
)
}
override Cryptography::CryptographicAlgorithm getAlgorithm() { result.matchesName(hashName) }
override DataFlow::Node getAnInput() { result in [this.getArg(0), this.getArgByName("data")] }
}
}

194
python/ql/src/semmle/python/frameworks/Cryptography.qll
View File

@@ -22,7 +22,7 @@ private module CryptographyModel {
* Gets a predefined curve class from
* `cryptography.hazmat.primitives.asymmetric.ec` with a specific key size (in bits).
*/
private DataFlow::Node curveClassWithKeySize(int keySize) {
private API::Node predefinedCurveClass(int keySize) {
exists(string curveName |
result =
API::moduleImport("cryptography")
@@ -31,7 +31,6 @@ private module CryptographyModel {
.getMember("asymmetric")
.getMember("ec")
.getMember(curveName)
.getAUse()
|
// obtained by manually looking at source code in
// https://github.com/pyca/cryptography/blob/cba69f1922803f4f29a3fde01741890d88b8e217/src/cryptography/hazmat/primitives/asymmetric/ec.py#L208-L300
@@ -75,13 +74,30 @@ private module CryptographyModel {
)
}
/** Gets a reference to a predefined curve class with a specific key size (in bits), as well as the origin of the class. */
private DataFlow::LocalSourceNode curveClassWithKeySize(
DataFlow::TypeTracker t, int keySize, DataFlow::Node origin
) {
t.start() and
result = predefinedCurveClass(keySize).getAnImmediateUse() and
origin = result
or
exists(DataFlow::TypeTracker t2 |
result = curveClassWithKeySize(t2, keySize, origin).track(t2, t)
)
}
/** Gets a reference to a predefined curve class with a specific key size (in bits), as well as the origin of the class. */
DataFlow::Node curveClassWithKeySize(int keySize, DataFlow::Node origin) {
curveClassWithKeySize(DataFlow::TypeTracker::end(), keySize, origin).flowsTo(result)
}
/** Gets a reference to a predefined curve class instance with a specific key size (in bits), as well as the origin of the class. */
private DataFlow::LocalSourceNode curveClassInstanceWithKeySize(
DataFlow::TypeTracker t, int keySize, DataFlow::Node origin
) {
t.start() and
result.(DataFlow::CallCfgNode).getFunction() = curveClassWithKeySize(keySize) and
origin = result
result.(DataFlow::CallCfgNode).getFunction() = curveClassWithKeySize(keySize, origin)
or
exists(DataFlow::TypeTracker t2 |
result = curveClassInstanceWithKeySize(t2, keySize, origin).track(t2, t)
@@ -164,9 +180,179 @@ private module CryptographyModel {
override int getKeySizeWithOrigin(DataFlow::Node origin) {
this.getCurveArg() = Ecc::curveClassInstanceWithKeySize(result, origin)
or
this.getCurveArg() = Ecc::curveClassWithKeySize(result, origin)
}
// Note: There is not really a key-size argument, since it's always specified by the curve.
override DataFlow::Node getKeySizeArg() { none() }
}
/** Provides models for the `cryptography.hazmat.primitives.ciphers` package */
private module Ciphers {
/** Gets a reference to a `cryptography.hazmat.primitives.ciphers.algorithms` Class */
API::Node algorithmClassRef(string algorithmName) {
result =
API::moduleImport("cryptography")
.getMember("hazmat")
.getMember("primitives")
.getMember("ciphers")
.getMember("algorithms")
.getMember(algorithmName)
}
/** Gets a reference to a Cipher instance using algorithm with `algorithmName`. */
DataFlow::LocalSourceNode cipherInstance(DataFlow::TypeTracker t, string algorithmName) {
t.start() and
exists(DataFlow::CallCfgNode call | result = call |
call =
API::moduleImport("cryptography")
.getMember("hazmat")
.getMember("primitives")
.getMember("ciphers")
.getMember("Cipher")
.getACall() and
algorithmClassRef(algorithmName).getReturn().getAUse() in [
call.getArg(0), call.getArgByName("algorithm")
]
)
or
exists(DataFlow::TypeTracker t2 | result = cipherInstance(t2, algorithmName).track(t2, t))
}
/** Gets a reference to a Cipher instance using algorithm with `algorithmName`. */
DataFlow::Node cipherInstance(string algorithmName) {
cipherInstance(DataFlow::TypeTracker::end(), algorithmName).flowsTo(result)
}
/** Gets a reference to the encryptor of a Cipher instance using algorithm with `algorithmName`. */
DataFlow::LocalSourceNode cipherEncryptor(DataFlow::TypeTracker t, string algorithmName) {
t.start() and
exists(DataFlow::AttrRead attr |
result.(DataFlow::CallCfgNode).getFunction() = attr and
attr.getAttributeName() = "encryptor" and
attr.getObject() = cipherInstance(algorithmName)
)
or
exists(DataFlow::TypeTracker t2 | result = cipherEncryptor(t2, algorithmName).track(t2, t))
}
/**
* Gets a reference to the encryptor of a Cipher instance using algorithm with `algorithmName`.
*
* You obtain an encryptor by using the `encryptor()` method on a Cipher instance.
*/
DataFlow::Node cipherEncryptor(string algorithmName) {
cipherEncryptor(DataFlow::TypeTracker::end(), algorithmName).flowsTo(result)
}
/** Gets a reference to the dncryptor of a Cipher instance using algorithm with `algorithmName`. */
DataFlow::LocalSourceNode cipherDecryptor(DataFlow::TypeTracker t, string algorithmName) {
t.start() and
exists(DataFlow::AttrRead attr |
result.(DataFlow::CallCfgNode).getFunction() = attr and
attr.getAttributeName() = "decryptor" and
attr.getObject() = cipherInstance(algorithmName)
)
or
exists(DataFlow::TypeTracker t2 | result = cipherDecryptor(t2, algorithmName).track(t2, t))
}
/**
* Gets a reference to the decryptor of a Cipher instance using algorithm with `algorithmName`.
*
* You obtain an decryptor by using the `decryptor()` method on a Cipher instance.
*/
DataFlow::Node cipherDecryptor(string algorithmName) {
cipherDecryptor(DataFlow::TypeTracker::end(), algorithmName).flowsTo(result)
}
/**
* An encrypt or decrypt operation from `cryptography.hazmat.primitives.ciphers`.
*/
class CryptographyGenericCipherOperation extends Cryptography::CryptographicOperation::Range,
DataFlow::CallCfgNode {
string algorithmName;
CryptographyGenericCipherOperation() {
exists(DataFlow::AttrRead attr |
this.getFunction() = attr and
attr.getAttributeName() = ["update", "update_into"] and
(
attr.getObject() = cipherEncryptor(algorithmName)
or
attr.getObject() = cipherDecryptor(algorithmName)
)
)
}
override Cryptography::CryptographicAlgorithm getAlgorithm() {
result.matchesName(algorithmName)
}
override DataFlow::Node getAnInput() { result in [this.getArg(0), this.getArgByName("data")] }
}
}
/** Provides models for the `cryptography.hazmat.primitives.hashes` package */
private module Hashes {
/**
* Gets a reference to a `cryptography.hazmat.primitives.hashes` class, representing
* a hashing algorithm.
*/
API::Node algorithmClassRef(string algorithmName) {
result =
API::moduleImport("cryptography")
.getMember("hazmat")
.getMember("primitives")
.getMember("hashes")
.getMember(algorithmName)
}
/** Gets a reference to a Hash instance using algorithm with `algorithmName`. */
private DataFlow::LocalSourceNode hashInstance(DataFlow::TypeTracker t, string algorithmName) {
t.start() and
exists(DataFlow::CallCfgNode call | result = call |
call =
API::moduleImport("cryptography")
.getMember("hazmat")
.getMember("primitives")
.getMember("hashes")
.getMember("Hash")
.getACall() and
algorithmClassRef(algorithmName).getReturn().getAUse() in [
call.getArg(0), call.getArgByName("algorithm")
]
)
or
exists(DataFlow::TypeTracker t2 | result = hashInstance(t2, algorithmName).track(t2, t))
}
/** Gets a reference to a Hash instance using algorithm with `algorithmName`. */
DataFlow::Node hashInstance(string algorithmName) {
hashInstance(DataFlow::TypeTracker::end(), algorithmName).flowsTo(result)
}
/**
* An hashing operation from `cryptography.hazmat.primitives.hashes`.
*/
class CryptographyGenericHashOperation extends Cryptography::CryptographicOperation::Range,
DataFlow::CallCfgNode {
string algorithmName;
CryptographyGenericHashOperation() {
exists(DataFlow::AttrRead attr |
this.getFunction() = attr and
attr.getAttributeName() = "update" and
attr.getObject() = hashInstance(algorithmName)
)
}
override Cryptography::CryptographicAlgorithm getAlgorithm() {
result.matchesName(algorithmName)
}
override DataFlow::Node getAnInput() { result in [this.getArg(0), this.getArgByName("data")] }
}
}
}

80
python/ql/src/semmle/python/frameworks/Django.qll
View File

@@ -11,6 +11,8 @@ private import semmle.python.Concepts
private import semmle.python.ApiGraphs
private import semmle.python.frameworks.PEP249
private import semmle.python.regex
private import semmle.python.frameworks.internal.PoorMansFunctionResolution
private import semmle.python.frameworks.internal.SelfRefMixin
/**
* Provides models for the `django` PyPI package.
@@ -1386,40 +1388,6 @@ private module PrivateDjango {
// ---------------------------------------------------------------------------
// Helpers
// ---------------------------------------------------------------------------
/**
* Gets the last decorator call for the function `func`, if `func` has decorators.
*/
private Expr lastDecoratorCall(Function func) {
result = func.getDefinition().(FunctionExpr).getADecoratorCall() and
not exists(Call other_decorator | other_decorator.getArg(0) = result)
}
/** Adds the `getASelfRef` member predicate when modeling a class. */
abstract private class SelfRefMixin extends Class {
/**
* Gets a reference to instances of this class, originating from a self parameter of
* a method defined on this class.
*
* Note: TODO: This doesn't take MRO into account
* Note: TODO: This doesn't take staticmethod/classmethod into account
*/
private DataFlow::LocalSourceNode getASelfRef(DataFlow::TypeTracker t) {
t.start() and
result.(DataFlow::ParameterNode).getParameter() = this.getAMethod().getArg(0)
or
exists(DataFlow::TypeTracker t2 | result = this.getASelfRef(t2).track(t2, t))
}
/**
* Gets a reference to instances of this class, originating from a self parameter of
* a method defined on this class.
*
* Note: TODO: This doesn't take MRO into account
* Note: TODO: This doesn't take staticmethod/classmethod into account
*/
DataFlow::Node getASelfRef() { this.getASelfRef(DataFlow::TypeTracker::end()).flowsTo(result) }
}
// ---------------------------------------------------------------------------
// Form and form field modeling
// ---------------------------------------------------------------------------
@@ -1487,46 +1455,6 @@ private module PrivateDjango {
// ---------------------------------------------------------------------------
// routing modeling
// ---------------------------------------------------------------------------
/**
* Gets a reference to the Function `func`.
*
* The idea is that this function should be used as a route handler when setting up a
* route, but currently it just tracks all functions, since we can't do type-tracking
* backwards yet (TODO).
*/
private DataFlow::LocalSourceNode djangoRouteHandlerFunctionTracker(
DataFlow::TypeTracker t, Function func
) {
t.start() and
(
not exists(func.getADecorator()) and
result.asExpr() = func.getDefinition()
or
// If the function has decorators, we still want to model the function as being
// the request handler for a route setup. In such situations, we must track the
// last decorator call instead of the function itself.
//
// Note that this means that we blindly ignore what the decorator actually does to
// the function, which seems like an OK tradeoff.
result.asExpr() = lastDecoratorCall(func)
)
or
exists(DataFlow::TypeTracker t2 |
result = djangoRouteHandlerFunctionTracker(t2, func).track(t2, t)
)
}
/**
* Gets a reference to the Function `func`.
*
* The idea is that this function should be used as a route handler when setting up a
* route, but currently it just tracks all functions, since we can't do type-tracking
* backwards yet (TODO).
*/
private DataFlow::Node djangoRouteHandlerFunctionTracker(Function func) {
djangoRouteHandlerFunctionTracker(DataFlow::TypeTracker::end(), func).flowsTo(result)
}
/**
* In order to recognize a class as being a django view class, based on the `as_view`
* call, we need to be able to track such calls on _any_ class. This is provided by
@@ -1613,7 +1541,7 @@ private module PrivateDjango {
*/
private class DjangoRouteHandler extends Function {
DjangoRouteHandler() {
exists(DjangoRouteSetup route | route.getViewArg() = djangoRouteHandlerFunctionTracker(this))
exists(DjangoRouteSetup route | route.getViewArg() = poorMansFunctionTracker(this))
or
any(DjangoViewClass vc).getARequestHandler() = this
}
@@ -1663,7 +1591,7 @@ private module PrivateDjango {
abstract DataFlow::Node getViewArg();
final override DjangoRouteHandler getARequestHandler() {
djangoRouteHandlerFunctionTracker(result) = getViewArg()
poorMansFunctionTracker(result) = getViewArg()
or
exists(DjangoViewClass vc |
getViewArg() = vc.asViewResult() and

40
python/ql/src/semmle/python/frameworks/Idna.qll Normal file
View File

@@ -0,0 +1,40 @@
/**
* Provides classes modeling security-relevant aspects of the `idna` PyPI package.
* See https://pypi.org/project/idna/.
*/
private import python
private import semmle.python.dataflow.new.DataFlow
private import semmle.python.dataflow.new.TaintTracking
private import semmle.python.Concepts
private import semmle.python.ApiGraphs
/**
* Provides models for the `idna` PyPI package.
* See https://pypi.org/project/idna/.
*/
private module IdnaModel {
/** A call to `idna.encode`. */
private class IdnaEncodeCall extends Encoding::Range, DataFlow::CallCfgNode {
IdnaEncodeCall() { this = API::moduleImport("idna").getMember("encode").getACall() }
override DataFlow::Node getAnInput() { result = [this.getArg(0), this.getArgByName("s")] }
override DataFlow::Node getOutput() { result = this }
override string getFormat() { result = "IDNA" }
}
/** A call to `idna.decode`. */
private class IdnaDecodeCall extends Decoding::Range, DataFlow::CallCfgNode {
IdnaDecodeCall() { this = API::moduleImport("idna").getMember("decode").getACall() }
override DataFlow::Node getAnInput() { result = [this.getArg(0), this.getArgByName("s")] }
override DataFlow::Node getOutput() { result = this }
override string getFormat() { result = "IDNA" }
override predicate mayExecuteInput() { none() }
}
}

88
python/ql/src/semmle/python/frameworks/Multidict.qll Normal file
View File

@@ -0,0 +1,88 @@
/**
* Provides classes modeling security-relevant aspects of the `multidict` PyPI package.
* See https://multidict.readthedocs.io/en/stable/.
*/
private import python
private import semmle.python.dataflow.new.DataFlow
private import semmle.python.dataflow.new.TaintTracking
private import semmle.python.Concepts
private import semmle.python.ApiGraphs
/**
* INTERNAL: Do not use.
*
* Provides models for the `multidict` PyPI package.
* See https://multidict.readthedocs.io/en/stable/.
*/
module Multidict {
/**
* Provides models for a `MultiDictProxy` class:
* - `multidict.MultiDictProxy`
* - `multidict.CIMultiDictProxy`
*
* See https://multidict.readthedocs.io/en/stable/multidict.html#multidictproxy
*/
module MultiDictProxy {
/** Gets a reference to a `MultiDictProxy` class. */
API::Node classRef() {
result = API::moduleImport("multidict").getMember(["MultiDictProxy", "CIMultiDictProxy"])
}
/**
* A source of instances of `multidict.MultiDictProxy`, extend this class to model
* new instances.
*
* This can include instantiations of the class, return values from function
* calls, or a special parameter that will be set when functions are called by an external
* library.
*
* Use `MultiDictProxy::instance()` predicate to get
* references to instances of `multidict.MultiDictProxy`.
*/
abstract class InstanceSource extends DataFlow::LocalSourceNode { }
/** A direct instantiation of a `MultiDictProxy` class. */
private class ClassInstantiation extends InstanceSource, DataFlow::CallCfgNode {
ClassInstantiation() { this = classRef().getACall() }
}
/** Gets a reference to an instance of a `MultiDictProxy` class. */
private DataFlow::LocalSourceNode instance(DataFlow::TypeTracker t) {
t.start() and
result instanceof InstanceSource
or
exists(DataFlow::TypeTracker t2 | result = instance(t2).track(t2, t))
}
/** Gets a reference to an instance of a `MultiDictProxy` class. */
DataFlow::Node instance() { instance(DataFlow::TypeTracker::end()).flowsTo(result) }
/**
* Taint propagation for `multidict.MultiDictProxy`.
*
* See https://multidict.readthedocs.io/en/stable/multidict.html#multidictproxy
*/
class MultiDictProxyAdditionalTaintStep extends TaintTracking::AdditionalTaintStep {
override predicate step(DataFlow::Node nodeFrom, DataFlow::Node nodeTo) {
// class instantiation
exists(ClassInstantiation call |
nodeFrom = call.getArg(0) and
nodeTo = call
)
or
// Methods
//
// TODO: When we have tools that make it easy, model these properly to handle
// `meth = obj.meth; meth()`. Until then, we'll use this more syntactic approach
// (since it allows us to at least capture the most common cases).
nodeFrom = instance() and
exists(DataFlow::AttrRead attr | attr.getObject() = nodeFrom |
// methods (non-async)
attr.getAttributeName() in ["getone", "getall"] and
nodeTo.(DataFlow::CallCfgNode).getFunction() = attr
)
}
}
}
}

2
python/ql/src/semmle/python/frameworks/MySQLdb.qll
View File

@@ -10,7 +10,7 @@ private import semmle.python.dataflow.new.DataFlow
private import semmle.python.dataflow.new.RemoteFlowSources
private import semmle.python.Concepts
private import semmle.python.ApiGraphs
private import PEP249
private import semmle.python.frameworks.PEP249
/**
* Provides models for the `MySQLdb` PyPI package.

2
python/ql/src/semmle/python/frameworks/MysqlConnectorPython.qll
View File

@@ -10,7 +10,7 @@ private import semmle.python.dataflow.new.DataFlow
private import semmle.python.dataflow.new.RemoteFlowSources
private import semmle.python.Concepts
private import semmle.python.ApiGraphs
private import PEP249
private import semmle.python.frameworks.PEP249
/**
* Provides models for the `mysql-connector-python` package.

2
python/ql/src/semmle/python/frameworks/Psycopg2.qll
View File

@@ -10,7 +10,7 @@ private import semmle.python.dataflow.new.DataFlow
private import semmle.python.dataflow.new.RemoteFlowSources
private import semmle.python.Concepts
private import semmle.python.ApiGraphs
private import PEP249
private import semmle.python.frameworks.PEP249
/**
* Provides models for the `psycopg2` PyPI package.

2
python/ql/src/semmle/python/frameworks/PyMySQL.qll
View File

@@ -8,7 +8,7 @@ private import semmle.python.dataflow.new.DataFlow
private import semmle.python.dataflow.new.RemoteFlowSources
private import semmle.python.Concepts
private import semmle.python.ApiGraphs
private import PEP249
private import semmle.python.frameworks.PEP249
/**
* Provides models for the `PyMySQL` PyPI package.

84
python/ql/src/semmle/python/frameworks/Simplejson.qll Normal file
View File

@@ -0,0 +1,84 @@
/**
* Provides classes modeling security-relevant aspects of the `simplejson` PyPI package.
* See https://simplejson.readthedocs.io/en/latest/.
*/
private import python
private import semmle.python.dataflow.new.DataFlow
private import semmle.python.dataflow.new.TaintTracking
private import semmle.python.Concepts
private import semmle.python.ApiGraphs
/**
* Provides models for the `simplejson` PyPI package.
* See https://simplejson.readthedocs.io/en/latest/.
*/
private module SimplejsonModel {
/**
* A call to `simplejson.dumps`.
*
* See https://simplejson.readthedocs.io/en/latest/#simplejson.dumps
*/
private class SimplejsonDumpsCall extends Encoding::Range, DataFlow::CallCfgNode {
SimplejsonDumpsCall() { this = API::moduleImport("simplejson").getMember("dumps").getACall() }
override DataFlow::Node getAnInput() { result in [this.getArg(0), this.getArgByName("obj")] }
override DataFlow::Node getOutput() { result = this }
override string getFormat() { result = "JSON" }
}
/**
* A call to `simplejson.dump`.
*
* See https://simplejson.readthedocs.io/en/latest/#simplejson.dump
*/
private class SimplejsonDumpCall extends Encoding::Range, DataFlow::CallCfgNode {
SimplejsonDumpCall() { this = API::moduleImport("simplejson").getMember("dump").getACall() }
override DataFlow::Node getAnInput() { result in [this.getArg(0), this.getArgByName("obj")] }
override DataFlow::Node getOutput() {
result.(DataFlow::PostUpdateNode).getPreUpdateNode() in [
this.getArg(1), this.getArgByName("fp")
]
}
override string getFormat() { result = "JSON" }
}
/**
* A call to `simplejson.loads`.
*
* See https://simplejson.readthedocs.io/en/latest/#simplejson.loads
*/
private class SimplejsonLoadsCall extends Decoding::Range, DataFlow::CallCfgNode {
SimplejsonLoadsCall() { this = API::moduleImport("simplejson").getMember("loads").getACall() }
override DataFlow::Node getAnInput() { result in [this.getArg(0), this.getArgByName("s")] }
override DataFlow::Node getOutput() { result = this }
override string getFormat() { result = "JSON" }
override predicate mayExecuteInput() { none() }
}
/**
* A call to `simplejson.load`.
*
* See https://simplejson.readthedocs.io/en/latest/#simplejson.load
*/
private class SimplejsonLoadCall extends Decoding::Range, DataFlow::CallCfgNode {
SimplejsonLoadCall() { this = API::moduleImport("simplejson").getMember("load").getACall() }
override DataFlow::Node getAnInput() { result in [this.getArg(0), this.getArgByName("fp")] }
override DataFlow::Node getOutput() { result = this }
override string getFormat() { result = "JSON" }
override predicate mayExecuteInput() { none() }
}
}

153
python/ql/src/semmle/python/frameworks/Stdlib.qll
View File

@@ -9,7 +9,7 @@ private import semmle.python.dataflow.new.TaintTracking
private import semmle.python.dataflow.new.RemoteFlowSources
private import semmle.python.Concepts
private import semmle.python.ApiGraphs
private import PEP249
private import semmle.python.frameworks.PEP249
/** Provides models for the Python standard library. */
private module Stdlib {
@@ -511,7 +511,23 @@ private module Stdlib {
override predicate mayExecuteInput() { none() }
override DataFlow::Node getAnInput() { result.asCfgNode() = node.getArg(0) }
override DataFlow::Node getAnInput() { result in [this.getArg(0), this.getArgByName("s")] }
override DataFlow::Node getOutput() { result = this }
override string getFormat() { result = "JSON" }
}
/**
* A call to `json.load`
* See https://docs.python.org/3/library/json.html#json.load
*/
private class JsonLoadCall extends Decoding::Range, DataFlow::CallCfgNode {
JsonLoadCall() { this = json().getMember("load").getACall() }
override predicate mayExecuteInput() { none() }
override DataFlow::Node getAnInput() { result in [this.getArg(0), this.getArgByName("fp")] }
override DataFlow::Node getOutput() { result = this }
@@ -525,13 +541,31 @@ private module Stdlib {
private class JsonDumpsCall extends Encoding::Range, DataFlow::CallCfgNode {
JsonDumpsCall() { this = json().getMember("dumps").getACall() }
override DataFlow::Node getAnInput() { result.asCfgNode() = node.getArg(0) }
override DataFlow::Node getAnInput() { result in [this.getArg(0), this.getArgByName("obj")] }
override DataFlow::Node getOutput() { result = this }
override string getFormat() { result = "JSON" }
}
/**
* A call to `json.dump`
* See https://docs.python.org/3/library/json.html#json.dump
*/
private class JsonDumpCall extends Encoding::Range, DataFlow::CallCfgNode {
JsonDumpCall() { this = json().getMember("dump").getACall() }
override DataFlow::Node getAnInput() { result in [this.getArg(0), this.getArgByName("obj")] }
override DataFlow::Node getOutput() {
result.(DataFlow::PostUpdateNode).getPreUpdateNode() in [
this.getArg(1), this.getArgByName("fp")
]
}
override string getFormat() { result = "JSON" }
}
// ---------------------------------------------------------------------------
// cgi
// ---------------------------------------------------------------------------
@@ -1054,6 +1088,119 @@ private module Stdlib {
}
}
// ---------------------------------------------------------------------------
// hashlib
// ---------------------------------------------------------------------------
/** Gets a call to `hashlib.new` with `algorithmName` as the first argument. */
private DataFlow::CallCfgNode hashlibNewCall(string algorithmName) {
exists(DataFlow::Node nameArg |
result = API::moduleImport("hashlib").getMember("new").getACall() and
nameArg in [result.getArg(0), result.getArgByName("name")] and
exists(StrConst str |
DataFlow::exprNode(str).(DataFlow::LocalSourceNode).flowsTo(nameArg) and
algorithmName = str.getText()
)
)
}
/** Gets a reference to the result of calling `hashlib.new` with `algorithmName` as the first argument. */
private DataFlow::LocalSourceNode hashlibNewResult(DataFlow::TypeTracker t, string algorithmName) {
t.start() and
result = hashlibNewCall(algorithmName)
or
exists(DataFlow::TypeTracker t2 | result = hashlibNewResult(t2, algorithmName).track(t2, t))
}
/** Gets a reference to the result of calling `hashlib.new` with `algorithmName` as the first argument. */
DataFlow::Node hashlibNewResult(string algorithmName) {
hashlibNewResult(DataFlow::TypeTracker::end(), algorithmName).flowsTo(result)
}
/**
* A hashing operation by supplying initial data when calling the `hashlib.new` function.
*/
class HashlibNewCall extends Cryptography::CryptographicOperation::Range, DataFlow::CallCfgNode {
string hashName;
HashlibNewCall() {
this = hashlibNewCall(hashName) and
exists([this.getArg(1), this.getArgByName("data")])
}
override Cryptography::CryptographicAlgorithm getAlgorithm() { result.matchesName(hashName) }
override DataFlow::Node getAnInput() { result in [this.getArg(1), this.getArgByName("data")] }
}
/**
* A hashing operation by using the `update` method on the result of calling the `hashlib.new` function.
*/
class HashlibNewUpdateCall extends Cryptography::CryptographicOperation::Range,
DataFlow::CallCfgNode {
string hashName;
HashlibNewUpdateCall() {
exists(DataFlow::AttrRead attr |
attr.getObject() = hashlibNewResult(hashName) and
this.getFunction() = attr and
attr.getAttributeName() = "update"
)
}
override Cryptography::CryptographicAlgorithm getAlgorithm() { result.matchesName(hashName) }
override DataFlow::Node getAnInput() { result = this.getArg(0) }
}
/**
* A hashing operation from the `hashlib` package using one of the predefined classes
* (such as `hashlib.md5`). `hashlib.new` is not included, since it is handled by
* `HashlibNewCall` and `HashlibNewUpdateCall`.
*/
abstract class HashlibGenericHashOperation extends Cryptography::CryptographicOperation::Range,
DataFlow::CallCfgNode {
string hashName;
API::Node hashClass;
bindingset[this]
HashlibGenericHashOperation() {
not hashName = "new" and
hashClass = API::moduleImport("hashlib").getMember(hashName)
}
override Cryptography::CryptographicAlgorithm getAlgorithm() { result.matchesName(hashName) }
}
/**
* A hashing operation from the `hashlib` package using one of the predefined classes
* (such as `hashlib.md5`), by calling its' `update` mehtod.
*/
class HashlibHashClassUpdateCall extends HashlibGenericHashOperation {
HashlibHashClassUpdateCall() { this = hashClass.getReturn().getMember("update").getACall() }
override DataFlow::Node getAnInput() { result = this.getArg(0) }
}
/**
* A hashing operation from the `hashlib` package using one of the predefined classes
* (such as `hashlib.md5`), by passing data to when instantiating the class.
*/
class HashlibDataPassedToHashClass extends HashlibGenericHashOperation {
HashlibDataPassedToHashClass() {
// we only want to model calls to classes such as `hashlib.md5()` if initial data
// is passed as an argument
this = hashClass.getACall() and
exists([this.getArg(0), this.getArgByName("string")])
}
override DataFlow::Node getAnInput() {
result = this.getArg(0)
or
// in Python 3.9, you are allowed to use `hashlib.md5(string=<bytes-like>)`.
result = this.getArgByName("string")
}
}
// ---------------------------------------------------------------------------
// OTHER
// ---------------------------------------------------------------------------

76
python/ql/src/semmle/python/frameworks/Ujson.qll Normal file
View File

@@ -0,0 +1,76 @@
/**
* Provides classes modeling security-relevant aspects of the `ujson` PyPI package.
* See https://pypi.org/project/ujson/.
*/
private import python
private import semmle.python.dataflow.new.DataFlow
private import semmle.python.dataflow.new.TaintTracking
private import semmle.python.Concepts
private import semmle.python.ApiGraphs
/**
* Provides models for the `ujson` PyPI package.
* See https://pypi.org/project/ujson/.
*/
private module UjsonModel {
/**
* A call to `usjon.dumps` or `ujson.encode`.
*/
private class UjsonDumpsCall extends Encoding::Range, DataFlow::CallCfgNode {
UjsonDumpsCall() { this = API::moduleImport("ujson").getMember(["dumps", "encode"]).getACall() }
override DataFlow::Node getAnInput() { result in [this.getArg(0), this.getArgByName("obj")] }
override DataFlow::Node getOutput() { result = this }
override string getFormat() { result = "JSON" }
}
/**
* A call to `ujson.dump`.
*/
private class UjsonDumpCall extends Encoding::Range, DataFlow::CallCfgNode {
UjsonDumpCall() { this = API::moduleImport("ujson").getMember("dump").getACall() }
override DataFlow::Node getAnInput() { result = this.getArg(0) }
override DataFlow::Node getOutput() {
result.(DataFlow::PostUpdateNode).getPreUpdateNode() = this.getArg(1)
}
override string getFormat() { result = "JSON" }
}
/**
* A call to `ujson.loads` or `ujson.decode`.
*/
private class UjsonLoadsCall extends Decoding::Range, DataFlow::CallCfgNode {
UjsonLoadsCall() { this = API::moduleImport("ujson").getMember(["loads", "decode"]).getACall() }
// Note: Most other JSON libraries allow the keyword argument `s`, but as of version
// 4.0.2 `ujson` uses `obj` instead.
override DataFlow::Node getAnInput() { result in [this.getArg(0), this.getArgByName("obj")] }
override DataFlow::Node getOutput() { result = this }
override string getFormat() { result = "JSON" }
override predicate mayExecuteInput() { none() }
}
/**
* A call to `ujson.load`.
*/
private class UjsonLoadCall extends Decoding::Range, DataFlow::CallCfgNode {
UjsonLoadCall() { this = API::moduleImport("ujson").getMember("load").getACall() }
override DataFlow::Node getAnInput() { result = this.getArg(0) }
override DataFlow::Node getOutput() { result = this }
override string getFormat() { result = "JSON" }
override predicate mayExecuteInput() { none() }
}
}

5
python/ql/src/semmle/python/frameworks/Werkzeug.qll
View File

@@ -1,5 +1,8 @@
/**
* Provides classes modeling security-relevant aspects of the `flask` package.
* Provides classes modeling security-relevant aspects of the `Werkzeug` PyPI package.
* See
* - https://pypi.org/project/Werkzeug/
* - https://werkzeug.palletsprojects.com/en/1.0.x/#werkzeug
*/
private import python

8
python/ql/src/semmle/python/frameworks/Yaml.qll
View File

@@ -1,6 +1,6 @@
/**
* Provides classes modeling security-relevant aspects of the PyYAML package (obtained
* via `import yaml`)
* Provides classes modeling security-relevant aspects of the `PyYAML` PyPI package
* (imported as `yaml`)
*
* See
* - https://pyyaml.org/wiki/PyYAMLDocumentation
@@ -14,8 +14,8 @@ private import semmle.python.Concepts
private import semmle.python.ApiGraphs
/**
* Provides classes modeling security-relevant aspects of the PyYAML package (obtained
* via `import yaml`)
* Provides classes modeling security-relevant aspects of the `PyYAML` PyPI package
* (imported as `yaml`)
*
* See
* - https://pyyaml.org/wiki/PyYAMLDocumentation

122
python/ql/src/semmle/python/frameworks/Yarl.qll Normal file
View File

@@ -0,0 +1,122 @@
/**
* Provides classes modeling security-relevant aspects of the `yarl` PyPI package.
* See https://yarl.readthedocs.io/en/stable/.
*/
private import python
private import semmle.python.dataflow.new.DataFlow
private import semmle.python.dataflow.new.TaintTracking
private import semmle.python.Concepts
private import semmle.python.ApiGraphs
private import semmle.python.frameworks.Multidict
/**
* INTERNAL: Do not use.
*
* Provides models for the `yarl` PyPI package.
* See https://multidict.readthedocs.io/en/stable/.
*/
module Yarl {
/**
* Provides models for a the `yarl.URL` class:
*
* See https://yarl.readthedocs.io/en/stable/api.html#yarl.URL
*/
module Url {
/**
* A source of instances of `yarl.URL`, extend this class to model new instances.
*
* This can include instantiations of the class, return values from function
* calls, or a special parameter that will be set when functions are called by an external
* library.
*
* Use `Url::instance()` predicate to get references to instances of `yarl.URL`.
*/
abstract class InstanceSource extends DataFlow::LocalSourceNode { }
/** A direct instantiation of `yarl.URL`. */
private class ClassInstantiation extends InstanceSource, DataFlow::CallCfgNode {
ClassInstantiation() { this = API::moduleImport("yarl").getMember("URL").getACall() }
}
/** Gets a reference to an instance of `yarl.URL`. */
private DataFlow::LocalSourceNode instance(DataFlow::TypeTracker t) {
t.start() and
result instanceof InstanceSource
or
exists(DataFlow::TypeTracker t2 | result = instance(t2).track(t2, t))
}
/** Gets a reference to an instance of `yarl.URL`. */
DataFlow::Node instance() { instance(DataFlow::TypeTracker::end()).flowsTo(result) }
/**
* Taint propagation for `yarl.URL`.
*
* See https://yarl.readthedocs.io/en/stable/api.html#yarl.URL
*/
class YarlUrlAdditionalTaintStep extends TaintTracking::AdditionalTaintStep {
override predicate step(DataFlow::Node nodeFrom, DataFlow::Node nodeTo) {
// class instantiation
exists(ClassInstantiation call |
nodeFrom in [call.getArg(0), call.getArgByName("val")] and
nodeTo = call
)
or
// Methods
//
// TODO: When we have tools that make it easy, model these properly to handle
// `meth = obj.meth; meth()`. Until then, we'll use this more syntactic approach
// (since it allows us to at least capture the most common cases).
exists(DataFlow::AttrRead attr |
// methods (that replaces part of URL, taken as only arguments)
attr.getAttributeName() in [
"with_scheme", "with_user", "with_password", "with_host", "with_port", "with_path",
"with_query", "with_query", "update_query", "update_query", "with_fragment",
"with_name",
// join is a bit different, but is still correct to add here :+1:
"join"
] and
(
// obj -> obj.meth()
nodeFrom = instance() and
attr.getObject() = nodeFrom and
nodeTo.(DataFlow::CallCfgNode).getFunction() = attr
or
// argument of obj.meth() -> obj.meth()
attr.getObject() = instance() and
nodeTo.(DataFlow::CallCfgNode).getFunction() = attr and
nodeFrom in [
nodeTo.(DataFlow::CallCfgNode).getArg(_),
nodeTo.(DataFlow::CallCfgNode).getArgByName(_)
]
)
or
// other methods
nodeFrom = instance() and
attr.getObject() = nodeFrom and
attr.getAttributeName() in ["human_repr"] and
nodeTo.(DataFlow::CallCfgNode).getFunction() = attr
)
or
// Attributes
nodeFrom = instance() and
nodeTo.(DataFlow::AttrRead).getObject() = nodeFrom and
nodeTo.(DataFlow::AttrRead).getAttributeName() in [
"user", "raw_user", "password", "raw_password", "host", "raw_host", "port",
"explicit_port", "authority", "raw_authority", "path", "raw_path", "path_qs",
"raw_path_qs", "query_string", "raw_query_string", "fragment", "raw_fragment", "parts",
"raw_parts", "name", "raw_name", "query"
]
}
}
/** An attribute read on a `yarl.URL` that is a `MultiDictProxy` instance. */
class YarlUrlMultiDictProxyInstance extends Multidict::MultiDictProxy::InstanceSource {
YarlUrlMultiDictProxyInstance() {
this.(DataFlow::AttrRead).getObject() = Yarl::Url::instance() and
this.(DataFlow::AttrRead).getAttributeName() = "query"
}
}
}
}

86
python/ql/src/semmle/python/frameworks/internal/PoorMansFunctionResolution.qll Normal file
View File

@@ -0,0 +1,86 @@
/**
* INTERNAL: Do not use.
*
* Notice: The predicates provided in this module is a poor mans solution for function
* resolution, and does not handle anything but the most simple cases.
*
* For example, in the code below, we're not able to tell anything about
* `inst.my_method` (which is a bound-method)
* ```py
* class MyClass:
* def my_method(self):
* pass
*
* inst = MyClass()
* print(inst.my_method)
* ```
*/
private import python
private import semmle.python.dataflow.new.DataFlow
/**
* Gets the last decorator call for the function `func`, if `func` has decorators.
*/
private Expr lastDecoratorCall(Function func) {
result = func.getDefinition().(FunctionExpr).getADecoratorCall() and
not exists(Call other_decorator | other_decorator.getArg(0) = result)
}
/**
* Gets a reference to the Function `func`.
*
* Notice: This is a poor mans solution for function resolution, and does not handle
* anything but the most simple cases.
*
* For example, in the code below, we're not able to tell anything about
* `inst.my_method` (which is a bound-method)
* ```py
* class MyClass:
* def my_method(self):
* pass
*
* inst = MyClass()
* print(inst.my_method)
* ```
*/
private DataFlow::LocalSourceNode poorMansFunctionTracker(DataFlow::TypeTracker t, Function func) {
t.start() and
(
not exists(func.getADecorator()) and
result.asExpr() = func.getDefinition()
or
// If the function has decorators, we still want to model the function as being
// the request handler for a route setup. In such situations, we must track the
// last decorator call instead of the function itself.
//
// Note that this means that we blindly ignore what the decorator actually does to
// the function, which seems like an OK tradeoff.
result.asExpr() = lastDecoratorCall(func)
)
or
exists(DataFlow::TypeTracker t2 | result = poorMansFunctionTracker(t2, func).track(t2, t))
}
/**
* INTERNAL: Do not use.
*
* Gets a reference to the Function `func`.
*
* Notice: This is a poor mans solution for function resolution, and does not handle
* anything but the most simple cases.
*
* For example, in the code below, we're not able to tell anything about
* `inst.my_method` (which is a bound-method)
* ```py
* class MyClass:
* def my_method(self):
* pass
*
* inst = MyClass()
* print(inst.my_method)
* ```
*/
DataFlow::Node poorMansFunctionTracker(Function func) {
poorMansFunctionTracker(DataFlow::TypeTracker::end(), func).flowsTo(result)
}

38
python/ql/src/semmle/python/frameworks/internal/SelfRefMixin.qll Normal file
View File

@@ -0,0 +1,38 @@
/**
* INTERNAL: Do not use.
*
* Provides the `SelfRefMixin` class.
*/
private import python
private import semmle.python.dataflow.new.DataFlow
/**
* INTERNAL: Do not use.
*
* Adds the `getASelfRef` member predicate when modeling a class.
*/
abstract class SelfRefMixin extends Class {
/**
* Gets a reference to instances of this class, originating from a self parameter of
* a method defined on this class.
*
* Note: TODO: This doesn't take MRO into account
* Note: TODO: This doesn't take staticmethod/classmethod into account
*/
private DataFlow::LocalSourceNode getASelfRef(DataFlow::TypeTracker t) {
t.start() and
result.(DataFlow::ParameterNode).getParameter() = this.getAMethod().getArg(0)
or
exists(DataFlow::TypeTracker t2 | result = this.getASelfRef(t2).track(t2, t))
}
/**
* Gets a reference to instances of this class, originating from a self parameter of
* a method defined on this class.
*
* Note: TODO: This doesn't take MRO into account
* Note: TODO: This doesn't take staticmethod/classmethod into account
*/
DataFlow::Node getASelfRef() { this.getASelfRef(DataFlow::TypeTracker::end()).flowsTo(result) }
}

93
python/ql/src/semmle/python/security/SensitiveData.qll
View File

@@ -12,76 +12,15 @@
import python
import semmle.python.dataflow.TaintTracking
import semmle.python.web.HttpRequest
/**
* Provides heuristics for identifying names related to sensitive information.
*
* INTERNAL: Do not use directly.
* This is copied from the javascript library, but should be language independent.
*/
private module HeuristicNames {
/**
* Gets a regular expression that identifies strings that may indicate the presence of secret
* or trusted data.
*/
string maybeSecret() { result = "(?is).*((?<!is)secret|(?<!un|is)trusted).*" }
/**
* Gets a regular expression that identifies strings that may indicate the presence of
* user names or other account information.
*/
string maybeAccountInfo() {
result = "(?is).*acc(ou)?nt.*" or
result = "(?is).*(puid|username|userid).*"
}
/**
* Gets a regular expression that identifies strings that may indicate the presence of
* a password or an authorization key.
*/
string maybePassword() {
result = "(?is).*pass(wd|word|code|phrase)(?!.*question).*" or
result = "(?is).*(auth(entication|ori[sz]ation)?)key.*"
}
/**
* Gets a regular expression that identifies strings that may indicate the presence of
* a certificate.
*/
string maybeCertificate() { result = "(?is).*(cert)(?!.*(format|name)).*" }
/**
* Gets a regular expression that identifies strings that may indicate the presence
* of sensitive data, with `classification` describing the kind of sensitive data involved.
*/
string maybeSensitive(SensitiveData data) {
result = maybeSecret() and data instanceof SensitiveData::Secret
or
result = maybeAccountInfo() and data instanceof SensitiveData::Id
or
result = maybePassword() and data instanceof SensitiveData::Password
or
result = maybeCertificate() and data instanceof SensitiveData::Certificate
}
/**
* Gets a regular expression that identifies strings that may indicate the presence of data
* that is hashed or encrypted, and hence rendered non-sensitive.
*/
string notSensitive() {
result = "(?is).*(redact|censor|obfuscate|hash|md5|sha|((?<!un)(en))?(crypt|code)).*"
}
bindingset[name]
SensitiveData getSensitiveDataForName(string name) {
name.regexpMatch(HeuristicNames::maybeSensitive(result)) and
not name.regexpMatch(HeuristicNames::notSensitive())
}
}
import semmle.python.security.internal.SensitiveDataHeuristics
private import HeuristicNames
abstract class SensitiveData extends TaintKind {
bindingset[this]
SensitiveData() { this = this }
/** Gets the classification of this sensitive data taint kind. */
abstract SensitiveDataClassification getClassification();
}
module SensitiveData {
@@ -89,28 +28,44 @@ module SensitiveData {
Secret() { this = "sensitive.data.secret" }
override string repr() { result = "a secret" }
override SensitiveDataClassification getClassification() {
result = SensitiveDataClassification::secret()
}
}
class Id extends SensitiveData {
Id() { this = "sensitive.data.id" }
override string repr() { result = "an ID" }
override SensitiveDataClassification getClassification() {
result = SensitiveDataClassification::id()
}
}
class Password extends SensitiveData {
Password() { this = "sensitive.data.password" }
override string repr() { result = "a password" }
override SensitiveDataClassification getClassification() {
result = SensitiveDataClassification::password()
}
}
class Certificate extends SensitiveData {
Certificate() { this = "sensitive.data.certificate" }
override string repr() { result = "a certificate or key" }
override SensitiveDataClassification getClassification() {
result = SensitiveDataClassification::certificate()
}
}
private SensitiveData fromFunction(Value func) {
result = HeuristicNames::getSensitiveDataForName(func.getName())
nameIndicatesSensitiveData(func.getName(), result.getClassification())
}
abstract class Source extends TaintSource {
@@ -134,7 +89,7 @@ module SensitiveData {
SensitiveData data;
SensitiveVariableAccess() {
data = HeuristicNames::getSensitiveDataForName(this.(AttrNode).getName())
nameIndicatesSensitiveData(this.(AttrNode).getName(), data.getClassification())
}
override predicate isSourceOf(TaintKind kind) { kind = data }
@@ -149,7 +104,7 @@ module SensitiveData {
this.(CallNode).getFunction().(AttrNode).getName() = "get" and
exists(StringValue sensitive |
this.(CallNode).getAnArg().pointsTo(sensitive) and
data = HeuristicNames::getSensitiveDataForName(sensitive.getText())
nameIndicatesSensitiveData(sensitive.getText(), data.getClassification())
)
}

74
python/ql/src/semmle/python/security/dataflow/WeakSensitiveDataHashing.qll Normal file
View File

@@ -0,0 +1,74 @@
/**
* Provides a taint-tracking configuration for detecting use of a broken or weak
* cryptographic hashing algorithm on sensitive data.
*
* Note, for performance reasons: only import this file if
* `WeakSensitiveDataHashing::Configuration` is needed, otherwise
* `WeakSensitiveDataHashingCustomizations` should be imported instead.
*/
private import python
private import semmle.python.dataflow.new.DataFlow
private import semmle.python.dataflow.new.TaintTracking
private import semmle.python.Concepts
private import semmle.python.dataflow.new.RemoteFlowSources
private import semmle.python.dataflow.new.BarrierGuards
/**
* Provides a taint-tracking configuration for detecting use of a broken or weak
* cryptographic hash function on sensitive data, that does NOT require a
* computationally expensive hash function.
*/
module NormalHashFunction {
import WeakSensitiveDataHashingCustomizations::NormalHashFunction
/**
* A taint-tracking configuration for detecting use of a broken or weak
* cryptographic hashing algorithm on sensitive data.
*/
class Configuration extends TaintTracking::Configuration {
Configuration() { this = "NormalHashFunction" }
override predicate isSource(DataFlow::Node source) { source instanceof Source }
override predicate isSink(DataFlow::Node sink) { sink instanceof Sink }
override predicate isSanitizer(DataFlow::Node node) {
super.isSanitizer(node)
or
node instanceof Sanitizer
}
}
}
/**
* Provides a taint-tracking configuration for detecting use of a broken or weak
* cryptographic hashing algorithm on passwords.
*
* Passwords has stricter requirements on the hashing algorithm used (must be
* computationally expensive to prevent brute-force attacks).
*/
module ComputationallyExpensiveHashFunction {
import WeakSensitiveDataHashingCustomizations::ComputationallyExpensiveHashFunction
/**
* A taint-tracking configuration for detecting use of a broken or weak
* cryptographic hashing algorithm on passwords.
*
* Passwords has stricter requirements on the hashing algorithm used (must be
* computationally expensive to prevent brute-force attacks).
*/
class Configuration extends TaintTracking::Configuration {
Configuration() { this = "ComputationallyExpensiveHashFunction" }
override predicate isSource(DataFlow::Node source) { source instanceof Source }
override predicate isSink(DataFlow::Node sink) { sink instanceof Sink }
override predicate isSanitizer(DataFlow::Node node) {
super.isSanitizer(node)
or
node instanceof Sanitizer
}
}
}

157
python/ql/src/semmle/python/security/dataflow/WeakSensitiveDataHashingCustomizations.qll Normal file
View File

@@ -0,0 +1,157 @@
/**
* Provides default sources, sinks and sanitizers for detecting
* "use of a broken or weak cryptographic hashing algorithm on sensitive data"
* vulnerabilities, as well as extension points for adding your own.
*/
private import python
private import semmle.python.dataflow.new.DataFlow
private import semmle.python.dataflow.new.TaintTracking
private import semmle.python.Concepts
private import semmle.python.dataflow.new.SensitiveDataSources
private import semmle.python.dataflow.new.BarrierGuards
/**
* Provides default sources, sinks and sanitizers for detecting
* "use of a broken or weak cryptographic hashing algorithm on sensitive data"
* vulnerabilities on sensitive data that does NOT require computationally expensive
* hashing, as well as extension points for adding your own.
*
* Also see the `ComputationallyExpensiveHashFunction` module.
*/
module NormalHashFunction {
/**
* A data flow source for "use of a broken or weak cryptographic hashing algorithm on
* sensitive data" vulnerabilities.
*/
abstract class Source extends DataFlow::Node {
Source() { not this instanceof ComputationallyExpensiveHashFunction::Source }
/** Gets the classification of the sensitive data. */
abstract string getClassification();
}
/**
* A data flow sink for "use of a broken or weak cryptographic hashing algorithm on
* sensitive data" vulnerabilities.
*/
abstract class Sink extends DataFlow::Node {
/**
* Gets the name of the weak hashing algorithm.
*/
abstract string getAlgorithmName();
}
/**
* A sanitizer for "use of a broken or weak cryptographic hashing algorithm on
* sensitive data" vulnerabilities.
*/
abstract class Sanitizer extends DataFlow::Node { }
/**
* A source of sensitive data, considered as a flow source.
*/
class SensitiveDataSourceAsSource extends Source, SensitiveDataSource {
override string getClassification() { result = SensitiveDataSource.super.getClassification() }
}
/** The input to a hashing operation using a weak algorithm, considered as a flow sink. */
class WeakHashingOperationInputSink extends Sink {
Cryptography::HashingAlgorithm algorithm;
WeakHashingOperationInputSink() {
exists(Cryptography::CryptographicOperation operation |
algorithm = operation.getAlgorithm() and
algorithm.isWeak() and
this = operation.getAnInput()
)
}
override string getAlgorithmName() { result = algorithm.getName() }
}
}
/**
* Provides default sources, sinks and sanitizers for detecting
* "use of a broken or weak cryptographic hashing algorithm on sensitive data"
* vulnerabilities on sensitive data that DOES require computationally expensive
* hashing, as well as extension points for adding your own.
*
* Also see the `NormalHashFunction` module.
*/
module ComputationallyExpensiveHashFunction {
/**
* A data flow source of sensitive data that requires computationally expensive
* hashing for "use of a broken or weak cryptographic hashing algorithm on sensitive
* data" vulnerabilities.
*/
abstract class Source extends DataFlow::Node {
/** Gets the classification of the sensitive data. */
abstract string getClassification();
}
/**
* A data flow sink for sensitive data that requires computationally expensive
* hashing for "use of a broken or weak cryptographic hashing algorithm on sensitive
* data" vulnerabilities.
*/
abstract class Sink extends DataFlow::Node {
/**
* Gets the name of the weak hashing algorithm.
*/
abstract string getAlgorithmName();
/**
* Holds if this sink is for a computationally expensive hash function (meaning that
* hash function is just weak in some other regard.
*/
abstract predicate isComputationallyExpensive();
}
/**
* A sanitizer of sensitive data that requires computationally expensive
* hashing for "use of a broken or weak cryptographic hashing
* algorithm on sensitive data" vulnerabilities.
*/
abstract class Sanitizer extends DataFlow::Node { }
/**
* A source of passwords, considered as a flow source.
*/
class PasswordSourceAsSource extends Source, SensitiveDataSource {
PasswordSourceAsSource() {
// TODO: once https://github.com/github/codeql/pull/5739 has been merged,
// don't use hardcoded value anymore
SensitiveDataSource.super.getClassification() = "password"
}
override string getClassification() { result = SensitiveDataSource.super.getClassification() }
}
/**
* The input to a password hashing operation using a weak algorithm, considered as a
* flow sink.
*/
class WeakPasswordHashingOperationInputSink extends Sink {
Cryptography::CryptographicAlgorithm algorithm;
WeakPasswordHashingOperationInputSink() {
(
algorithm instanceof Cryptography::PasswordHashingAlgorithm and
algorithm.isWeak()
or
algorithm instanceof Cryptography::HashingAlgorithm // Note that HashingAlgorithm and PasswordHashingAlgorithm are disjoint
) and
exists(Cryptography::CryptographicOperation operation |
algorithm = operation.getAlgorithm() and
this = operation.getAnInput()
)
}
override string getAlgorithmName() { result = algorithm.getName() }
override predicate isComputationallyExpensive() {
algorithm instanceof Cryptography::PasswordHashingAlgorithm
}
}
}

129
python/ql/src/semmle/python/security/internal/SensitiveDataHeuristics.qll Normal file
View File

@@ -0,0 +1,129 @@
/**
* INTERNAL: Do not use.
*
* Provides classes and predicates for identifying strings that may indicate the presence of sensitive data.
* Such that we can share this logic across our CodeQL analysis of different languages.
*
* 'Sensitive' data in general is anything that should not be sent around in unencrypted form.
*/
/**
* A classification of different kinds of sensitive data:
*
* - secret: generic secret or trusted data;
* - id: a user name or other account information;
* - password: a password or authorization key;
* - certificate: a certificate.
*
* While classifications are represented as strings, this should not be relied upon.
* Instead, use the predicates in `SensitiveDataClassification::` to work with
* classifications.
*/
class SensitiveDataClassification extends string {
SensitiveDataClassification() { this in ["secret", "id", "password", "certificate"] }
}
/**
* Provides predicates to select the different kinds of sensitive data we support.
*/
module SensitiveDataClassification {
/** Gets the classification for secret or trusted data. */
SensitiveDataClassification secret() { result = "secret" }
/** Gets the classification for user names or other account information. */
SensitiveDataClassification id() { result = "id" }
/** Gets the classification for passwords or authorization keys. */
SensitiveDataClassification password() { result = "password" }
/** Gets the classification for certificates. */
SensitiveDataClassification certificate() { result = "certificate" }
}
/**
* INTERNAL: Do not use.
*
* Provides heuristics for identifying names related to sensitive information.
*/
module HeuristicNames {
/**
* Gets a regular expression that identifies strings that may indicate the presence of secret
* or trusted data.
*/
string maybeSecret() { result = "(?is).*((?<!is)secret|(?<!un|is)trusted).*" }
/**
* Gets a regular expression that identifies strings that may indicate the presence of
* user names or other account information.
*/
string maybeAccountInfo() {
result = "(?is).*acc(ou)?nt.*" or
result = "(?is).*(puid|username|userid).*" or
result = "(?s).*([uU]|^|_|[a-z](?=U))([uU][iI][dD]).*"
}
/**
* Gets a regular expression that identifies strings that may indicate the presence of
* a password or an authorization key.
*/
string maybePassword() {
result = "(?is).*pass(wd|word|code|phrase)(?!.*question).*" or
result = "(?is).*(auth(entication|ori[sz]ation)?)key.*"
}
/**
* Gets a regular expression that identifies strings that may indicate the presence of
* a certificate.
*/
string maybeCertificate() { result = "(?is).*(cert)(?!.*(format|name)).*" }
/**
* Gets a regular expression that identifies strings that may indicate the presence
* of sensitive data, with `classification` describing the kind of sensitive data involved.
*/
string maybeSensitiveRegexp(SensitiveDataClassification classification) {
result = maybeSecret() and classification = SensitiveDataClassification::secret()
or
result = maybeAccountInfo() and classification = SensitiveDataClassification::id()
or
result = maybePassword() and classification = SensitiveDataClassification::password()
or
result = maybeCertificate() and
classification = SensitiveDataClassification::certificate()
}
/**
* Gets a regular expression that identifies strings that may indicate the presence of data
* that is hashed or encrypted, and hence rendered non-sensitive, or contains special characters
* suggesting nouns within the string do not represent the meaning of the whole string (e.g. a URL or a SQL query).
*/
string notSensitiveRegexp() {
result = "(?is).*([^\\w$.-]|redact|censor|obfuscate|hash|md5|sha|((?<!un)(en))?(crypt|code)).*"
}
/**
* DEPRECATED: Use `maybeSensitiveRegexp` instead.
*/
deprecated predicate maybeSensitive = maybeSensitiveRegexp/1;
/**
* DEPRECATED: Use `notSensitiveRegexp` instead.
*/
deprecated predicate notSensitive = notSensitiveRegexp/0;
/**
* Holds if `name` may indicate the presence of sensitive data, and
* `name` does not indicate that the data is in fact non-sensitive (for example since
* it is hashed or encrypted). `classification` describes the kind of sensitive data
* involved.
*
* That is, one of the regexps from `maybeSensitiveRegexp` matches `name` (with the
* given classification), and none of the regexps from `notSensitiveRegexp` matches
* `name`.
*/
bindingset[name]
predicate nameIndicatesSensitiveData(string name, SensitiveDataClassification classification) {
name.regexpMatch(maybeSensitiveRegexp(classification)) and
not name.regexpMatch(notSensitiveRegexp())
}
}

Some files were not shown because too many files have changed in this diff Show More