Add Unicode Bypass Validation query tests and help

python/ql/src/experimental/Security/CWE-176/UnicodeBypassValidation.qhelp

@@ -0,0 +1,36 @@
+<!DOCTYPE qhelp PUBLIC "-//Semmle//qhelp//EN" "qhelp.dtd">
+<qhelp>
+    <overview>
+        <p>Security checks bypass due to a Unicode transformation</p>
+        <p>
+            If ever a unicode tranformation is performed after some security checks or logical
+            validation, the
+            latter could be bypassed due to a potential Unicode characters collision.
+            The validation of concern are any character escaping, any regex validation or any string
+            verification.
+        </p>
+        <img src="./vulnerability-flow.png" alt="Security checks bypassed" />
+    </overview>
+    <recommendation>
+        <p> Perform a Unicode normalization before the logical validation. </p>
+    </recommendation>
+    <example>
+
+        <p> The following example showcases the bypass of all checks performed by <code>
+            flask.escape()</code> due to a post-unicode normalization.</p>
+        <p>For instance: the character U+FE64 (<code>﹤</code>) is not filtered-out by the flask
+            escape function. But due to the Unicode normalization, the character is transformed and
+            would become U+003C (<code> &lt; </code> ).</p>
+
+        <sample src="escape-bypass.py" />
+
+    </example>
+    <references>
+        <li> Research study: <a
+                href="https://gosecure.github.io/presentations/2021-02-unicode-owasp-toronto/philippe_arteau_owasp_unicode_v4.pdf">
+            Unicode vulnerabilities that could bYte you
+            </a> and <a
+                href="https://gosecure.github.io/unicode-pentester-cheatsheet/">Unicode pentest
+            cheatsheet</a>. </li>
+    </references>
+</qhelp>

python/ql/src/experimental/Security/CWE-176/UnicodeBypassValidation.ql

@@ -0,0 +1,24 @@
+/**
+ * @name Bypass Logical Validation Using Unicode Characters
+ * @description A Unicode transformation is using a remote user-controlled data. The transformation is a Unicode normalization using the algorithms "NFC" or "NFKC". In all cases, the security measures implemented or the logical validation performed to escape any injection characters, to validate using regex patterns or to perform string-based checks, before the Unicode transformation are **bypassable** by special Unicode characters.
+ * @kind path-problem
+ * @id py/unicode-bypass-validation
+ * @precision high
+ * @problem.severity error
+ * @tags security
+ *       experimental
+ *       external/cwe/cwe-176
+ *       external/cwe/cwe-179
+ *       external/cwe/cwe-180
+ */
+
+import python
+import semmle.python.security.dataflow.UnicodeBypassValidationQuery
+import DataFlow::PathGraph
+
+from Configuration config, DataFlow::PathNode source, DataFlow::PathNode sink
+where config.hasFlowPath(source, sink)
+select sink.getNode(), source, sink,
+  "This $@ processes unsafely $@ and any logical validation in-between could be bypassed using special Unicode characters.",
+  sink.getNode(), "Unicode transformation (Unicode normalization)", source.getNode(),
+  "remote user-controlled data"

python/ql/src/experimental/Security/CWE-176/escape-bypass.py

@@ -0,0 +1,11 @@
+import unicodedata
+from flask import Flask, request, escape, render_template
+
+app = Flask(__name__)
+
+
+@app.route("/unsafe1")
+def unsafe1():
+    user_input = escape(request.args.get("ui"))
+    normalized_user_input = unicodedata.normalize("NFKC", user_input)
+    return render_template("result.html", normalized_user_input=normalized_user_input)