Merge branch 'main' into jorgectf/python/deserialization

python/ql/src/CHANGELOG.md

@@ -1,3 +1,22 @@
+## 0.0.10
+
+### New Queries
+
+* The query "LDAP query built from user-controlled sources" (`py/ldap-injection`) has been promoted from experimental to the main query pack. Its results will now appear by default. This query was originally [submitted as an experimental query by @jorgectf](https://github.com/github/codeql/pull/5443).
+* The query "Log Injection" (`py/log-injection`) has been promoted from experimental to the main query pack. Its results will now appear when `security-extended` is used. This query was originally [submitted as an experimental query by @haby0](https://github.com/github/codeql/pull/6182).
+
+## 0.0.9
+
+### Bug Fixes
+
+* The [View AST functionality](https://codeql.github.com/docs/codeql-for-visual-studio-code/exploring-the-structure-of-your-source-code/) no longer prints detailed information about regular expressions, greatly improving performance.
+
+## 0.0.8
+
+### Major Analysis Improvements
+
+* User names and other account information is no longer considered to be sensitive data for the queries `py/clear-text-logging-sensitive-data` and `py/clear-text-storage-sensitive-data`, since this lead to many false positives.
+
 ## 0.0.7
 
 ## 0.0.6

python/ql/src/Security/CWE-090/LdapInjection.qhelp

@@ -4,22 +4,22 @@
 <qhelp>
 <overview>
 <p>If an LDAP query or DN is built using string concatenation or string formatting, and the
-components of the concatenation include user input without any proper sanitization, a user 
+components of the concatenation include user input without any proper sanitization, a user
 is likely to be able to run malicious LDAP queries.</p>
 </overview>
 
 <recommendation>
 <p>If user input must be included in an LDAP query or DN, it should be escaped to
 avoid a malicious user providing special characters that change the meaning
-of the query. In Python2, user input should be escaped with <code>ldap.dn.escape_dn_chars</code> 
-or <code>ldap.filter.escape_filter_chars</code>, while in Python3, user input should be escaped with 
+of the query. In Python2, user input should be escaped with <code>ldap.dn.escape_dn_chars</code>
+or <code>ldap.filter.escape_filter_chars</code>, while in Python3, user input should be escaped with
 <code>ldap3.utils.dn.escape_rdn</code> or <code>ldap3.utils.conv.escape_filter_chars</code>
-depending on the component tainted by the user. A good practice is to escape filter characters 
+depending on the component tainted by the user. A good practice is to escape filter characters
 that could change the meaning of the query (https://tools.ietf.org/search/rfc4515#section-3).</p>
 </recommendation>
 
 <example>
-<p>In the following examples, the code accepts both <code>username</code> and <code>dc</code> from the user, 
+<p>In the following examples, the code accepts both <code>username</code> and <code>dc</code> from the user,
 which it then uses to build a LDAP query and DN.</p>
 
 <p>The first and the second example uses the unsanitized user input directly
@@ -30,7 +30,7 @@ components, and search for a completely different set of values.</p>
 <sample src="examples/example_bad1.py" />
 <sample src="examples/example_bad2.py" />
 
-<p>In the third and four example, the input provided by the user is sanitized before it is included in the search filter or DN. 
+<p>In the third and fourth example, the input provided by the user is sanitized before it is included in the search filter or DN.
 This ensures the meaning of the query cannot be changed by a malicious user.</p>
 
 <sample src="examples/example_good1.py" />

python/ql/src/Security/CWE-090/LdapInjection.ql

@@ -0,0 +1,28 @@
+/**
+ * @name LDAP query built from user-controlled sources
+ * @description Building an LDAP query from user-controlled sources is vulnerable to insertion of
+ *              malicious LDAP code by the user.
+ * @kind path-problem
+ * @problem.severity error
+ * @security-severity 9.8
+ * @precision high
+ * @id py/ldap-injection
+ * @tags security
+ *       external/cwe/cwe-090
+ */
+
+// Determine precision above
+import python
+import semmle.python.security.dataflow.LdapInjection
+import DataFlow::PathGraph
+
+from DataFlow::PathNode source, DataFlow::PathNode sink, string parameterName
+where
+  any(LdapInjection::DnConfiguration dnConfig).hasFlowPath(source, sink) and
+  parameterName = "DN"
+  or
+  any(LdapInjection::FilterConfiguration filterConfig).hasFlowPath(source, sink) and
+  parameterName = "filter"
+select sink.getNode(), source, sink,
+  "$@ LDAP query parameter (" + parameterName + ") comes from $@.", sink.getNode(), "This",
+  source.getNode(), "a user-provided value"

python/ql/src/Security/CWE-117/LogInjection.qhelp

@@ -8,8 +8,11 @@
 
 <p>If unsanitized user input is written to a log entry, a malicious user may be able to forge new log entries.</p>
 
-<p>Forgery can occur if a user provides some input creating the appearance of multiple
-  log entries. This can include unescaped new-line characters, or HTML or other markup.</p>
+<p>Forgery can occur if a user provides some input with characters that are interpreted
+when the log output is displayed. If the log is displayed as a plain text file, then new
+line characters can be used by a malicious user to create the appearance of multiple log
+entries. If the log is displayed as HTML, then arbitrary HTML may be included to spoof
+log entries.</p>
 </overview>
 
 <recommendation>
@@ -29,14 +32,14 @@ other forms of HTML injection.
 
 <example>
 <p>
-In the example, the name provided by the user is recorded using the log output function (<code>logging.info</code> or <code>app.logger.info</code>, etc.). 
-In these four cases, the name provided by the user is not provided The processing is recorded. If a malicious user provides <code>Guest%0D%0AUser name: Admin</code> 
+In the example, the name provided by the user is recorded using the log output function (<code>logging.info</code> or <code>app.logger.info</code>, etc.).
+In these four cases, the name provided by the user is not provided The processing is recorded. If a malicious user provides <code>Guest%0D%0AUser name: Admin</code>
 as a parameter, the log entry will be divided into two lines, the first line is <code>User name: Guest</code> code>, the second line is <code>User name: Admin</code>.
 </p>
 <sample src="LogInjectionBad.py" />
 
 <p>
-In a good example, the program uses the <code>replace</code> function to provide parameter processing to the user, and replace <code>\r\n</code> and <code>\n</code> 
+In a good example, the program uses the <code>replace</code> function to provide parameter processing to the user, and replace <code>\r\n</code> and <code>\n</code>
 with empty characters. To a certain extent, the occurrence of log injection vulnerabilities is reduced.
 </p>
 

python/ql/src/Security/CWE-117/LogInjection.ql

@@ -4,17 +4,18 @@
  *              insertion of forged log entries by a malicious user.
  * @kind path-problem
  * @problem.severity error
- * @precision high
+ * @security-severity 7.8
+ * @precision medium
  * @id py/log-injection
  * @tags security
  *       external/cwe/cwe-117
  */
 
 import python
-import experimental.semmle.python.security.injection.LogInjection
+import semmle.python.security.dataflow.LogInjection
 import DataFlow::PathGraph
 
-from LogInjectionFlowConfig config, DataFlow::PathNode source, DataFlow::PathNode sink
+from LogInjection::Configuration config, DataFlow::PathNode source, DataFlow::PathNode sink
 where config.hasFlowPath(source, sink)
 select sink.getNode(), source, sink, "$@ flows to log entry.", source.getNode(),
   "User-provided value"

python/ql/src/change-notes/released/0.0.10.md

@@ -0,0 +1,6 @@
+## 0.0.10
+
+### New Queries
+
+* The query "LDAP query built from user-controlled sources" (`py/ldap-injection`) has been promoted from experimental to the main query pack. Its results will now appear by default. This query was originally [submitted as an experimental query by @jorgectf](https://github.com/github/codeql/pull/5443).
+* The query "Log Injection" (`py/log-injection`) has been promoted from experimental to the main query pack. Its results will now appear when `security-extended` is used. This query was originally [submitted as an experimental query by @haby0](https://github.com/github/codeql/pull/6182).

python/ql/src/change-notes/released/0.0.8.md

@@ -1,4 +1,5 @@
----
-category: majorAnalysis
----
+## 0.0.8
+
+### Major Analysis Improvements
+
 * User names and other account information is no longer considered to be sensitive data for the queries `py/clear-text-logging-sensitive-data` and `py/clear-text-storage-sensitive-data`, since this lead to many false positives.

python/ql/src/change-notes/released/0.0.9.md

@@ -1,4 +1,5 @@
----
-category: fix
----
+## 0.0.9
+
+### Bug Fixes
+
 * The [View AST functionality](https://codeql.github.com/docs/codeql-for-visual-studio-code/exploring-the-structure-of-your-source-code/) no longer prints detailed information about regular expressions, greatly improving performance.

python/ql/src/codeql-pack.release.yml

@@ -1,2 +1,2 @@
 ---
-lastReleaseVersion: 0.0.7
+lastReleaseVersion: 0.0.10

python/ql/src/experimental/Security/CWE-074/TemplateInjection.ql

@@ -21,11 +21,11 @@ import semmle.python.security.strings.Untrusted
 class TemplateInjectionConfiguration extends TaintTracking::Configuration {
   TemplateInjectionConfiguration() { this = "Template injection configuration" }
 
-  override predicate isSource(TaintTracking::Source source) {
+  deprecated override predicate isSource(TaintTracking::Source source) {
     source instanceof HttpRequestTaintSource
   }
 
-  override predicate isSink(TaintTracking::Sink sink) { sink instanceof SSTISink }
+  deprecated override predicate isSink(TaintTracking::Sink sink) { sink instanceof SSTISink }
 }
 
 from TemplateInjectionConfiguration config, TaintedPathSource src, TaintedPathSink sink

python/ql/src/experimental/Security/CWE-090/LDAPInjection.ql

@@ -1,20 +0,0 @@
-/**
- * @name LDAP query built from user-controlled sources
- * @description Building an LDAP query from user-controlled sources is vulnerable to insertion of
- *              malicious LDAP code by the user.
- * @kind path-problem
- * @problem.severity error
- * @id py/ldap-injection
- * @tags security	
- *       external/cwe/cwe-090
- */
-
-// Determine precision above
-import python
-import experimental.semmle.python.security.injection.LDAP
-import DataFlow::PathGraph
-
-from LDAPInjectionFlowConfig config, DataFlow::PathNode source, DataFlow::PathNode sink
-where config.hasFlowPath(source, sink)
-select sink.getNode(), source, sink, "$@ LDAP query parameter comes from $@.", sink.getNode(),
-  "This", source.getNode(), "a user-provided value"

python/ql/src/experimental/Security/CWE-091/Xslt.ql

@@ -20,11 +20,11 @@ import experimental.semmle.python.security.injection.XSLT
 class XSLTInjectionConfiguration extends TaintTracking::Configuration {
   XSLTInjectionConfiguration() { this = "XSLT injection configuration" }
 
-  override predicate isSource(TaintTracking::Source source) {
+  deprecated override predicate isSource(TaintTracking::Source source) {
     source instanceof HttpRequestTaintSource
   }
 
-  override predicate isSink(TaintTracking::Sink sink) {
+  deprecated override predicate isSink(TaintTracking::Sink sink) {
     sink instanceof XSLTInjection::XSLTInjectionSink
   }
 }

python/ql/src/experimental/Security/CWE-338/InsecureRandomness.qhelp

@@ -0,0 +1,48 @@
+<!DOCTYPE qhelp PUBLIC
+"-//Semmle//qhelp//EN"
+"qhelp.dtd">
+<qhelp>
+<overview>
+<p>
+Using a cryptographically weak pseudo-random number generator to generate a security-sensitive value,
+such as a password, makes it easier for an attacker to predict the value.
+</p>
+<p>
+Pseudo-random number generators generate a sequence of numbers that only approximates the properties
+of random numbers. The sequence is not truly random because it is completely determined by a
+relatively small set of initial values, the seed. If the random number generator is
+cryptographically weak, then this sequence may be easily predictable through outside observations.
+</p>
+</overview>
+<recommendation>
+<p>
+Use a cryptographically secure pseudo-random number generator if the output is to be used in a
+security sensitive context. As a rule of thumb, a value should be considered "security sensitive"
+if predicting it would allow the attacker to perform an action that they would otherwise be unable
+to perform. For example, if an attacker could predict the random password generated for a new user,
+they would be able to log in as that new user.
+</p>
+<p>
+For Python, <code>secrets</code> provides a cryptographically secure pseudo-random
+number generator. <code>random</code> is not cryptographically secure, and should be avoided in
+security contexts.
+</p>
+</recommendation>
+
+<example>
+<p>
+The example below uses the <code>random</code> package instead of <code>secrets</code> to generate a password:
+</p>
+<sample src="examples/InsecureRandomness.py" />
+<p>
+Instead, use <code>secrets</code>:
+</p>
+<sample src="examples/InsecureRandomnessGood.py" />
+</example>
+
+<references>
+  <li>Wikipedia. <a href="http://en.wikipedia.org/wiki/Pseudorandom_number_generator">Pseudo-random number generator</a>.</li>
+  <li>OWASP: <a href="https://owasp.org/www-community/vulnerabilities/Insecure_Randomness">Insecure Randomness</a>.</li>
+  <li>OWASP: <a href="https://cheatsheetseries.owasp.org/cheatsheets/Cryptographic_Storage_Cheat_Sheet.html#secure-random-number-generation">Secure Random Number Generation</a>.</li>
+</references>
+</qhelp>

python/ql/src/experimental/Security/CWE-338/InsecureRandomness.ql

@@ -0,0 +1,23 @@
+/**
+ * @name Insecure randomness
+ * @description Using a cryptographically weak pseudo-random number generator to generate a
+ *              security-sensitive value may allow an attacker to predict what value will
+ *              be generated.
+ * @kind path-problem
+ * @problem.severity warning
+ * @security-severity 7.8
+ * @precision high
+ * @id py/insecure-randomness
+ * @tags security
+ *       external/cwe/cwe-338
+ */
+
+import python
+import experimental.semmle.python.security.InsecureRandomness::InsecureRandomness
+import semmle.python.dataflow.new.DataFlow
+import DataFlow::PathGraph
+
+from Configuration cfg, DataFlow::PathNode source, DataFlow::PathNode sink
+where cfg.hasFlowPath(source, sink)
+select sink.getNode(), source, sink, "Cryptographically insecure $@ in a security context.",
+  source.getNode(), "random value"

python/ql/src/experimental/Security/CWE-338/examples/InsecureRandomness.py

@@ -0,0 +1,6 @@
+import random
+
+
+def generatePassword():
+    # BAD: the random is not cryptographically secure
+    return random.random()

python/ql/src/experimental/Security/CWE-338/examples/InsecureRandomnessGood.py

@@ -0,0 +1,7 @@
+import secrets
+
+
+def generatePassword():
+    # GOOD: the random is cryptographically secure
+    secret_generator = secrets.SystemRandom()
+    return secret_generator.random()

python/ql/src/experimental/semmle/python/Concepts.qll

@@ -14,36 +14,6 @@ private import semmle.python.dataflow.new.RemoteFlowSources
 private import semmle.python.dataflow.new.TaintTracking
 private import experimental.semmle.python.Frameworks
 
-/** Provides classes for modeling log related APIs. */
-module LogOutput {
-  /**
-   * A data flow node for log output.
-   *
-   * Extend this class to model new APIs. If you want to refine existing API models,
-   * extend `LogOutput` instead.
-   */
-  abstract class Range extends DataFlow::Node {
-    /**
-     * Get the parameter value of the log output function.
-     */
-    abstract DataFlow::Node getAnInput();
-  }
-}
-
-/**
- * A data flow node for log output.
- *
- * Extend this class to refine existing API models. If you want to model new APIs,
- * extend `LogOutput::Range` instead.
- */
-class LogOutput extends DataFlow::Node {
-  LogOutput::Range range;
-
-  LogOutput() { this = range }
-
-  DataFlow::Node getAnInput() { result = range.getAnInput() }
-}
-
 module XML {
   /**
    * A data-flow node that collects functions parsing XML.

python/ql/src/experimental/semmle/python/Frameworks.qll

@@ -9,8 +9,8 @@ private import experimental.semmle.python.frameworks.Django
 private import experimental.semmle.python.frameworks.Werkzeug
 private import experimental.semmle.python.frameworks.LDAP
 private import experimental.semmle.python.frameworks.NoSQL
-private import experimental.semmle.python.frameworks.Log
 private import experimental.semmle.python.frameworks.JWT
 private import experimental.semmle.python.libraries.PyJWT
+private import experimental.semmle.python.libraries.Python_JWT
 private import experimental.semmle.python.libraries.Authlib
 private import experimental.semmle.python.libraries.PythonJose

python/ql/src/experimental/semmle/python/frameworks/Log.qll

@@ -1,118 +0,0 @@
-/**
- * Provides classes modeling security-relevant aspects of the log libraries.
- */
-
-private import python
-private import semmle.python.dataflow.new.DataFlow
-private import semmle.python.dataflow.new.TaintTracking
-private import semmle.python.dataflow.new.RemoteFlowSources
-private import experimental.semmle.python.Concepts
-private import semmle.python.frameworks.Flask
-private import semmle.python.ApiGraphs
-
-/**
- * Provides models for Python's log-related libraries.
- */
-private module log {
-  /**
-   * Log output method list.
-   *
-   * See https://docs.python.org/3/library/logging.html#logger-objects
-   */
-  private class LogOutputMethods extends string {
-    LogOutputMethods() {
-      this in ["info", "error", "warn", "warning", "debug", "critical", "exception", "log"]
-    }
-  }
-
-  /**
-   * The class used to find the log output method of the `logging` module.
-   *
-   * See `LogOutputMethods`
-   */
-  private class LoggingCall extends DataFlow::CallCfgNode, LogOutput::Range {
-    LoggingCall() {
-      this = API::moduleImport("logging").getMember(any(LogOutputMethods m)).getACall()
-    }
-
-    override DataFlow::Node getAnInput() {
-      this.getFunction().(DataFlow::AttrRead).getAttributeName() != "log" and
-      result in [this.getArg(_), this.getArgByName(_)] // this includes the arg named "msg"
-      or
-      this.getFunction().(DataFlow::AttrRead).getAttributeName() = "log" and
-      result in [this.getArg(any(int i | i > 0)), this.getArgByName(any(string s | s != "level"))]
-    }
-  }
-
-  /**
-   * The class used to find log output methods related to the `logging.getLogger` instance.
-   *
-   * See `LogOutputMethods`
-   */
-  private class LoggerCall extends DataFlow::CallCfgNode, LogOutput::Range {
-    LoggerCall() {
-      this =
-        API::moduleImport("logging")
-            .getMember("getLogger")
-            .getReturn()
-            .getMember(any(LogOutputMethods m))
-            .getACall()
-    }
-
-    override DataFlow::Node getAnInput() {
-      this.getFunction().(DataFlow::AttrRead).getAttributeName() != "log" and
-      result in [this.getArg(_), this.getArgByName(_)] // this includes the arg named "msg"
-      or
-      this.getFunction().(DataFlow::AttrRead).getAttributeName() = "log" and
-      result in [this.getArg(any(int i | i > 0)), this.getArgByName(any(string s | s != "level"))]
-    }
-  }
-
-  /**
-   * The class used to find the relevant log output method of the `flask.Flask.logger` instance (flask application).
-   *
-   * See `LogOutputMethods`
-   */
-  private class FlaskLoggingCall extends DataFlow::CallCfgNode, LogOutput::Range {
-    FlaskLoggingCall() {
-      this =
-        Flask::FlaskApp::instance()
-            .getMember("logger")
-            .getMember(any(LogOutputMethods m))
-            .getACall()
-    }
-
-    override DataFlow::Node getAnInput() {
-      this.getFunction().(DataFlow::AttrRead).getAttributeName() != "log" and
-      result in [this.getArg(_), this.getArgByName(_)] // this includes the arg named "msg"
-      or
-      this.getFunction().(DataFlow::AttrRead).getAttributeName() = "log" and
-      result in [this.getArg(any(int i | i > 0)), this.getArgByName(any(string s | s != "level"))]
-    }
-  }
-
-  /**
-   * The class used to find the relevant log output method of the `django.utils.log.request_logger` instance (django application).
-   *
-   * See `LogOutputMethods`
-   */
-  private class DjangoLoggingCall extends DataFlow::CallCfgNode, LogOutput::Range {
-    DjangoLoggingCall() {
-      this =
-        API::moduleImport("django")
-            .getMember("utils")
-            .getMember("log")
-            .getMember("request_logger")
-            .getMember(any(LogOutputMethods m))
-            .getACall()
-    }
-
-    override DataFlow::Node getAnInput() {
-      this.getFunction().(DataFlow::AttrRead).getAttributeName() != "log" and
-      result in [this.getArg(_), this.getArgByName(_)] // this includes the arg named "msg"
-      or
-      this.getFunction().(DataFlow::AttrRead).getAttributeName() = "log" and
-      result in [this.getArg(any(int i | i > 0)), this.getArgByName(any(string s | s != "level"))]
-    }
-  }
-}

python/ql/src/experimental/semmle/python/libraries/Python_JWT.qll

@@ -0,0 +1,51 @@
+private import python
+private import experimental.semmle.python.Concepts
+private import semmle.python.ApiGraphs
+
+private module Python_JWT {
+  /**
+   * Gets a call to `python_jwt.process_jwt`.
+   *
+   * Given the following example:
+   *
+   * ```py
+   * python_jwt.process_jwt(token)
+   * python_jwt.verify_jwt(token, "key", "HS256")
+   * ```
+   *
+   * * `this` would be `jwt.process_jwt(token)`.
+   * * `getPayload()`'s result would be `token`.
+   * * `getKey()`'s result would be `"key"`.
+   * * `getAlgorithm()`'s result would be `"HS256"`.
+   * * `getAlgorithmstring()`'s result would be `HS256`.
+   * * `getOptions()`'s result would be `none()`.
+   * * `verifiesSignature()` predicate would succeed.
+   */
+  private class PythonJwtProcessCall extends DataFlow::CallCfgNode, JWTDecoding::Range {
+    PythonJwtProcessCall() {
+      this = API::moduleImport("python_jwt").getMember("process_jwt").getACall()
+    }
+
+    private DataFlow::CallCfgNode verifyCall() {
+      result = API::moduleImport("python_jwt").getMember("verify_jwt").getACall() and
+      this.getPayload().getALocalSource() = result.getArg(0).getALocalSource()
+    }
+
+    override DataFlow::Node getPayload() { result = this.getArg(0) }
+
+    override DataFlow::Node getKey() { result = this.verifyCall().getArg(1) }
+
+    override DataFlow::Node getAlgorithm() { result = this.verifyCall().getArg(2) }
+
+    override string getAlgorithmString() {
+      exists(StrConst str |
+        DataFlow::exprNode(str).(DataFlow::LocalSourceNode).flowsTo(this.getAlgorithm()) and
+        result = str.getText()
+      )
+    }
+
+    override DataFlow::Node getOptions() { none() }
+
+    override predicate verifiesSignature() { exists(this.verifyCall()) }
+  }
+}

python/ql/src/experimental/semmle/python/security/InsecureRandomness.qll

@@ -0,0 +1,37 @@
+/**
+ * Provides a taint tracking configuration for reasoning about random
+ * values that are not cryptographically secure.
+ *
+ * Note, for performance reasons: only import this file if
+ * `InsecureRandomness::Configuration` is needed, otherwise
+ * `InsecureRandomnessCustomizations` should be imported instead.
+ */
+
+private import python
+import semmle.python.dataflow.new.DataFlow
+import semmle.python.dataflow.new.TaintTracking
+
+/**
+ * A taint tracking configuration for random values that are not cryptographically secure.
+ */
+module InsecureRandomness {
+  import InsecureRandomnessCustomizations::InsecureRandomness
+
+  /**
+   * A taint-tracking configuration for reasoning about random values that are
+   * not cryptographically secure.
+   */
+  class Configuration extends TaintTracking::Configuration {
+    Configuration() { this = "InsecureRandomness" }
+
+    override predicate isSource(DataFlow::Node source) { source instanceof Source }
+
+    override predicate isSink(DataFlow::Node sink) { sink instanceof Sink }
+
+    override predicate isSanitizer(DataFlow::Node node) { node instanceof Sanitizer }
+
+    override predicate isSanitizerGuard(DataFlow::BarrierGuard guard) {
+      guard instanceof SanitizerGuard
+    }
+  }
+}

python/ql/src/experimental/semmle/python/security/InsecureRandomnessCustomizations.qll

@@ -0,0 +1,79 @@
+/**
+ * Provides default sources, sinks and sanitizers for reasoning about random values that are
+ * not cryptographically secure, as well as extension points for adding your own.
+ */
+
+private import python
+private import semmle.python.ApiGraphs
+private import semmle.python.Concepts
+private import semmle.python.dataflow.new.BarrierGuards
+private import semmle.python.dataflow.new.internal.DataFlowPrivate
+
+/**
+ * Provides default sources, sinks and sanitizers for reasoning about random values that are
+ * not cryptographically secure, as well as extension points for adding your own.
+ */
+module InsecureRandomness {
+  /**
+   * A data flow source for random values that are not cryptographically secure.
+   */
+  abstract class Source extends DataFlow::Node { }
+
+  /**
+   * A data flow sink for random values that are not cryptographically secure.
+   */
+  abstract class Sink extends DataFlow::Node { }
+
+  /**
+   * A sanitizer for random values that are not cryptographically secure.
+   */
+  abstract class Sanitizer extends DataFlow::Node { }
+
+  /**
+   * A sanitizer guard for random values that are not cryptographically secure.
+   */
+  abstract class SanitizerGuard extends DataFlow::BarrierGuard { }
+
+  /**
+   * A random source that is not sufficient for security use. So far this is only made up
+   * of the math package's rand function, more insufficient random sources can be added here.
+   */
+  class InsecureRandomSource extends Source {
+    InsecureRandomSource() {
+      this =
+        API::moduleImport("random")
+            .getMember([
+                "betavariate", "choice", "choices", "expovariate", "gammavariate", "gauss",
+                "getrandbits", "getstate", "lognormvariate", "normalvariate", "paretovariate",
+                "randbytes", "randint", "random", "randrange", "sample", "seed", "setstate",
+                "shuffle", "triangular", "uniform", "vonmisesvariate", "weibullvariate"
+              ])
+            .getACall()
+    }
+  }
+
+  /**
+   * A use in a function that heuristically deals with unsafe random numbers or random strings.
+   */
+  class RandomFnSink extends Sink {
+    RandomFnSink() {
+      exists(DataFlowCallable randomFn |
+        randomFn
+            .getName()
+            .regexpMatch("(?i).*(gen(erate)?|make|mk|create).*(nonce|salt|pepper|Password).*")
+      |
+        this.getEnclosingCallable() = randomFn
+      )
+    }
+  }
+
+  /**
+   * A cryptographic key, considered as a sink for random values that are not cryptographically
+   * secure.
+   */
+  class CryptoKeySink extends Sink {
+    CryptoKeySink() {
+      exists(Cryptography::CryptographicOperation operation | this = operation.getAnInput())
+    }
+  }
+}

python/ql/src/experimental/semmle/python/security/injection/LDAP.qll

@@ -1,24 +0,0 @@
-/**
- * Provides a taint-tracking configuration for detecting LDAP injection vulnerabilities
- */
-
-import python
-import experimental.semmle.python.Concepts
-import semmle.python.dataflow.new.DataFlow
-import semmle.python.dataflow.new.TaintTracking
-import semmle.python.dataflow.new.RemoteFlowSources
-
-/**
- * A taint-tracking configuration for detecting LDAP injections.
- */
-class LDAPInjectionFlowConfig extends TaintTracking::Configuration {
-  LDAPInjectionFlowConfig() { this = "LDAPInjectionFlowConfig" }
-
-  override predicate isSource(DataFlow::Node source) { source instanceof RemoteFlowSource }
-
-  override predicate isSink(DataFlow::Node sink) { sink = any(LDAPQuery ldapQuery).getQuery() }
-
-  override predicate isSanitizer(DataFlow::Node sanitizer) {
-    sanitizer = any(LDAPEscape ldapEsc).getAnInput()
-  }
-}

python/ql/src/experimental/semmle/python/security/injection/LogInjection.qll

@@ -1,24 +0,0 @@
-import python
-import semmle.python.Concepts
-import experimental.semmle.python.Concepts
-import semmle.python.dataflow.new.DataFlow
-import semmle.python.dataflow.new.TaintTracking
-import semmle.python.dataflow.new.RemoteFlowSources
-
-/**
- * A taint-tracking configuration for tracking untrusted user input used in log entries.
- */
-class LogInjectionFlowConfig extends TaintTracking::Configuration {
-  LogInjectionFlowConfig() { this = "LogInjectionFlowConfig" }
-
-  override predicate isSource(DataFlow::Node source) { source instanceof RemoteFlowSource }
-
-  override predicate isSink(DataFlow::Node sink) { sink = any(LogOutput logoutput).getAnInput() }
-
-  override predicate isSanitizer(DataFlow::Node node) {
-    exists(CallNode call |
-      node.asCfgNode() = call.getFunction().(AttrNode).getObject("replace") and
-      call.getArg(0).getNode().(StrConst).getText() in ["\r\n", "\n"]
-    )
-  }
-}

python/ql/src/experimental/semmle/python/security/injection/XSLT.qll

@@ -21,7 +21,7 @@ module XSLTInjection {
   /**
    * A kind of "taint", representing an untrusted XML string
    */
-  private class ExternalXmlStringKind extends ExternalStringKind {
+  deprecated private class ExternalXmlStringKind extends ExternalStringKind {
     ExternalXmlStringKind() { this = "etree.XML string" }
 
     override TaintKind getTaintForFlowStep(ControlFlowNode fromnode, ControlFlowNode tonode) {

python/ql/src/experimental/semmle/python/templates/Airspeed.qll

@@ -5,7 +5,7 @@ import semmle.python.web.HttpRequest
 import experimental.semmle.python.templates.SSTISink
 
 /** returns the ClassValue representing `airspeed.Template` */
-ClassValue theAirspeedTemplateClass() { result = Value::named("airspeed.Template") }
+deprecated ClassValue theAirspeedTemplateClass() { result = Value::named("airspeed.Template") }
 
 /**
  * Sink representing the `airspeed.Template` class instantiation argument.
@@ -13,7 +13,7 @@ ClassValue theAirspeedTemplateClass() { result = Value::named("airspeed.Template
  *  import airspeed
  *  temp = airspeed.Template(`"sink"`)
  */
-class AirspeedTemplateSink extends SSTISink {
+deprecated class AirspeedTemplateSink extends SSTISink {
   override string toString() { result = "argument to airspeed.Template()" }
 
   AirspeedTemplateSink() {

python/ql/src/experimental/semmle/python/templates/Bottle.qll

@@ -5,7 +5,9 @@ import semmle.python.web.HttpRequest
 import experimental.semmle.python.templates.SSTISink
 
 /** returns the ClassValue representing `bottle.SimpleTemplate` */
-ClassValue theBottleSimpleTemplateClass() { result = Value::named("bottle.SimpleTemplate") }
+deprecated ClassValue theBottleSimpleTemplateClass() {
+  result = Value::named("bottle.SimpleTemplate")
+}
 
 /**
  * Sink representing the `bottle.SimpleTemplate` class instantiation argument.
@@ -13,7 +15,7 @@ ClassValue theBottleSimpleTemplateClass() { result = Value::named("bottle.Simple
  *  from bottle import SimpleTemplate
  *  template = SimpleTemplate(`sink`)
  */
-class BottleSimpleTemplateSink extends SSTISink {
+deprecated class BottleSimpleTemplateSink extends SSTISink {
   override string toString() { result = "argument to bottle.SimpleTemplate()" }
 
   BottleSimpleTemplateSink() {
@@ -32,7 +34,7 @@ class BottleSimpleTemplateSink extends SSTISink {
  *  from bottle import template
  *  tmp = template(`sink`)
  */
-class BottleTemplateSink extends SSTISink {
+deprecated class BottleTemplateSink extends SSTISink {
   override string toString() { result = "argument to bottle.template()" }
 
   BottleTemplateSink() {

python/ql/src/experimental/semmle/python/templates/Chameleon.qll

@@ -5,7 +5,9 @@ import semmle.python.web.HttpRequest
 import experimental.semmle.python.templates.SSTISink
 
 /** returns the ClassValue representing `chameleon.PageTemplate` */
-ClassValue theChameleonPageTemplateClass() { result = Value::named("chameleon.PageTemplate") }
+deprecated ClassValue theChameleonPageTemplateClass() {
+  result = Value::named("chameleon.PageTemplate")
+}
 
 /**
  * Sink representing the `chameleon.PageTemplate` class instantiation argument.
@@ -13,7 +15,7 @@ ClassValue theChameleonPageTemplateClass() { result = Value::named("chameleon.Pa
  *  from chameleon import PageTemplate
  *  template = PageTemplate(`sink`)
  */
-class ChameleonTemplateSink extends SSTISink {
+deprecated class ChameleonTemplateSink extends SSTISink {
   override string toString() { result = "argument to Chameleon.PageTemplate()" }
 
   ChameleonTemplateSink() {

python/ql/src/experimental/semmle/python/templates/Cheetah.qll

@@ -5,7 +5,9 @@ import semmle.python.web.HttpRequest
 import experimental.semmle.python.templates.SSTISink
 
 /** returns the ClassValue representing `Cheetah.Template.Template` */
-ClassValue theCheetahTemplateClass() { result = Value::named("Cheetah.Template.Template") }
+deprecated ClassValue theCheetahTemplateClass() {
+  result = Value::named("Cheetah.Template.Template")
+}
 
 /**
  * Sink representing the instantiation argument of any class which derives from
@@ -22,7 +24,7 @@ ClassValue theCheetahTemplateClass() { result = Value::named("Cheetah.Template.T
  *  from Cheetah.Template import Template
  *  t3 = Template("sink")
  */
-class CheetahTemplateInstantiationSink extends SSTISink {
+deprecated class CheetahTemplateInstantiationSink extends SSTISink {
   override string toString() { result = "argument to Cheetah.Template.Template()" }
 
   CheetahTemplateInstantiationSink() {

python/ql/src/experimental/semmle/python/templates/Chevron.qll

@@ -5,7 +5,7 @@ import semmle.python.web.HttpRequest
 import experimental.semmle.python.templates.SSTISink
 
 /** returns the Value representing `chevron.render` function */
-Value theChevronRenderFunc() { result = Value::named("chevron.render") }
+deprecated Value theChevronRenderFunc() { result = Value::named("chevron.render") }
 
 /**
  * Sink representing the `chevron.render` function call argument.
@@ -13,7 +13,7 @@ Value theChevronRenderFunc() { result = Value::named("chevron.render") }
  *  import chevron
  *  tmp = chevron.render(`sink`,{ 'key' : 'value' })
  */
-class ChevronRenderSink extends SSTISink {
+deprecated class ChevronRenderSink extends SSTISink {
   override string toString() { result = "argument to chevron.render()" }
 
   ChevronRenderSink() {

python/ql/src/experimental/semmle/python/templates/DjangoTemplate.qll

@@ -4,7 +4,7 @@ import python
 import semmle.python.web.HttpRequest
 import experimental.semmle.python.templates.SSTISink
 
-ClassValue theDjangoTemplateClass() { result = Value::named("django.template.Template") }
+deprecated ClassValue theDjangoTemplateClass() { result = Value::named("django.template.Template") }
 
 /**
  * Sink representng `django.template.Template` class instantiation argument.
@@ -12,7 +12,7 @@ ClassValue theDjangoTemplateClass() { result = Value::named("django.template.Tem
  *  from django.template import Template
  *  template = Template(`sink`)
  */
-class DjangoTemplateTemplateSink extends SSTISink {
+deprecated class DjangoTemplateTemplateSink extends SSTISink {
   override string toString() { result = "argument to Django.template()" }
 
   DjangoTemplateTemplateSink() {

python/ql/src/experimental/semmle/python/templates/FlaskTemplate.qll

@@ -4,7 +4,9 @@ import python
 import semmle.python.web.HttpRequest
 import experimental.semmle.python.templates.SSTISink
 
-Value theFlaskRenderTemplateClass() { result = Value::named("flask.render_template_string") }
+deprecated Value theFlaskRenderTemplateClass() {
+  result = Value::named("flask.render_template_string")
+}
 
 /**
  * Sink representng `flask.render_template_string` function call argument.
@@ -12,7 +14,7 @@ Value theFlaskRenderTemplateClass() { result = Value::named("flask.render_templa
  *  from flask import render_template_string
  *  render_template_string(`sink`)
  */
-class FlaskTemplateSink extends SSTISink {
+deprecated class FlaskTemplateSink extends SSTISink {
   override string toString() { result = "argument to flask.render_template_string()" }
 
   FlaskTemplateSink() {

python/ql/src/experimental/semmle/python/templates/Genshi.qll

@@ -5,10 +5,12 @@ import semmle.python.web.HttpRequest
 import experimental.semmle.python.templates.SSTISink
 
 /** returns the ClassValue representing `Genshi.template.TextTemplate` */
-ClassValue theGenshiTextTemplateClass() { result = Value::named("genshi.template.TextTemplate") }
+deprecated ClassValue theGenshiTextTemplateClass() {
+  result = Value::named("genshi.template.TextTemplate")
+}
 
 /** returns the ClassValue representing `Genshi.template.MarkupTemplate` */
-ClassValue theGenshiMarkupTemplateClass() {
+deprecated ClassValue theGenshiMarkupTemplateClass() {
   result = Value::named("genshi.template.MarkupTemplate")
 }
 
@@ -18,7 +20,7 @@ ClassValue theGenshiMarkupTemplateClass() {
  *  from genshi.template import TextTemplate
  *  tmpl = TextTemplate('sink')
  */
-class GenshiTextTemplateSink extends SSTISink {
+deprecated class GenshiTextTemplateSink extends SSTISink {
   override string toString() { result = "argument to genshi.template.TextTemplate()" }
 
   GenshiTextTemplateSink() {
@@ -37,7 +39,7 @@ class GenshiTextTemplateSink extends SSTISink {
  *  from genshi.template import MarkupTemplate
  *  tmpl = MarkupTemplate('sink')
  */
-class GenshiMarkupTemplateSink extends SSTISink {
+deprecated class GenshiMarkupTemplateSink extends SSTISink {
   override string toString() { result = "argument to genshi.template.MarkupTemplate()" }
 
   GenshiMarkupTemplateSink() {

python/ql/src/experimental/semmle/python/templates/Jinja.qll

@@ -5,10 +5,10 @@ import semmle.python.web.HttpRequest
 import experimental.semmle.python.templates.SSTISink
 
 /** returns the ClassValue representing `jinja2.Template` */
-ClassValue theJinja2TemplateClass() { result = Value::named("jinja2.Template") }
+deprecated ClassValue theJinja2TemplateClass() { result = Value::named("jinja2.Template") }
 
 /** returns the ClassValue representing `jinja2.Template` */
-Value theJinja2FromStringValue() { result = Value::named("jinja2.from_string") }
+deprecated Value theJinja2FromStringValue() { result = Value::named("jinja2.from_string") }
 
 /**
  * Sink representing the `jinja2.Template` class instantiation argument.
@@ -16,7 +16,7 @@ Value theJinja2FromStringValue() { result = Value::named("jinja2.from_string") }
  *  from jinja2 import Template
  *  template = Template(`sink`)
  */
-class Jinja2TemplateSink extends SSTISink {
+deprecated class Jinja2TemplateSink extends SSTISink {
   override string toString() { result = "argument to jinja2.Template()" }
 
   Jinja2TemplateSink() {
@@ -35,7 +35,7 @@ class Jinja2TemplateSink extends SSTISink {
  *  from jinja2 import from_string
  *  template = from_string(`sink`)
  */
-class Jinja2FromStringSink extends SSTISink {
+deprecated class Jinja2FromStringSink extends SSTISink {
   override string toString() { result = "argument to jinja2.from_string()" }
 
   Jinja2FromStringSink() {

python/ql/src/experimental/semmle/python/templates/Mako.qll

@@ -5,7 +5,7 @@ import semmle.python.web.HttpRequest
 import experimental.semmle.python.templates.SSTISink
 
 /** returns the ClassValue representing `mako.template.Template` */
-ClassValue theMakoTemplateClass() { result = Value::named("mako.template.Template") }
+deprecated ClassValue theMakoTemplateClass() { result = Value::named("mako.template.Template") }
 
 /**
  * Sink representing the `mako.template.Template` class instantiation argument.
@@ -13,7 +13,7 @@ ClassValue theMakoTemplateClass() { result = Value::named("mako.template.Templat
  *  from mako.template import Template
  *  mytemplate = Template("hello world!")
  */
-class MakoTemplateSink extends SSTISink {
+deprecated class MakoTemplateSink extends SSTISink {
   override string toString() { result = "argument to mako.template.Template()" }
 
   MakoTemplateSink() {

python/ql/src/experimental/semmle/python/templates/SSTISink.qll

@@ -4,4 +4,4 @@ import semmle.python.dataflow.TaintTracking
  * A generic taint sink that is vulnerable to template inclusions.
  * The `temp` in `jinja2.Template(temp)` and similar.
  */
-abstract class SSTISink extends TaintSink { }
+abstract deprecated class SSTISink extends TaintSink { }

python/ql/src/experimental/semmle/python/templates/TRender.qll

@@ -5,7 +5,7 @@ import semmle.python.web.HttpRequest
 import experimental.semmle.python.templates.SSTISink
 
 /** returns the ClassValue representing `trender.TRender` */
-ClassValue theTRenderTemplateClass() { result = Value::named("trender.TRender") }
+deprecated ClassValue theTRenderTemplateClass() { result = Value::named("trender.TRender") }
 
 /**
  * Sink representing the `trender.TRender` class instantiation argument.
@@ -13,7 +13,7 @@ ClassValue theTRenderTemplateClass() { result = Value::named("trender.TRender")
  *  from trender import TRender
  *  template = TRender(`sink`)
  */
-class TRenderTemplateSink extends SSTISink {
+deprecated class TRenderTemplateSink extends SSTISink {
   override string toString() { result = "argument to trender.TRender()" }
 
   TRenderTemplateSink() {

python/ql/src/qlpack.yml

@@ -1,5 +1,5 @@
 name: codeql/python-queries
-version: 0.0.8-dev
+version: 0.0.11-dev
 groups: 
   - python
   - queries