Merge remote-tracking branch 'upstream/rc/1.23' into mergeback-20191202

Conflicts solved:
	javascript/extractor/src/com/semmle/js/extractor/Main.java
	javascript/ql/test/query-tests/Statements/UseOfReturnlessFunction/tst.js

change-notes/1.23/analysis-cpp.md

@@ -2,66 +2,65 @@
 
 The following changes in version 1.23 affect C/C++ analysis in all applications.
 
-## General improvements
-
 ## New queries
 
 | **Query**                   | **Tags**  | **Purpose**                                                        |
 |-----------------------------|-----------|--------------------------------------------------------------------|
-| Hard-coded Japanese era start date (`cpp/japanese-era/exact-era-date`) | reliability, japanese-era | This query is a combination of two old queries that were identical in purpose but separate as an implementation detail.  This new query replaces Hard-coded Japanese era start date in call (`cpp/japanese-era/constructor-or-method-with-exact-era-date`) and Hard-coded Japanese era start date in struct (`cpp/japanese-era/struct-with-exact-era-date`). |
+| Hard-coded Japanese era start date (`cpp/japanese-era/exact-era-date`) | reliability, japanese-era | This query is a combination of two old queries that were identical in purpose but separate as an implementation detail.  This new query replaces Hard-coded Japanese era start date in call (`cpp/japanese-era/constructor-or-method-with-exact-era-date`) and Hard-coded Japanese era start date in struct (`cpp/japanese-era/struct-with-exact-era-date`). Results are not shown on LGTM by default. |
-| Signed overflow check (`cpp/signed-overflow-check`) | correctness, security | Finds overflow checks that rely on signed integer addition to overflow, which has undefined behavior. Example: `a + b < a`. |
+| Pointer overflow check (`cpp/pointer-overflow-check`) | correctness, security | Finds overflow checks that rely on pointer addition to overflow, which has undefined behavior. Example: `ptr + a < ptr`. Results are shown on LGTM by default. |
-| Pointer overflow check (`cpp/pointer-overflow-check`) | correctness, security | Finds overflow checks that rely on pointer addition to overflow, which has undefined behavior. Example: `ptr + a < ptr`. |
+| Signed overflow check (`cpp/signed-overflow-check`) | correctness, security | Finds overflow checks that rely on signed integer addition to overflow, which has undefined behavior. Example: `a + b < a`. Results are shown on LGTM by default. |
+
 
 ## Changes to existing queries
 
 | **Query**                  | **Expected impact**    | **Change**                                                       |
 |----------------------------|------------------------|------------------------------------------------------------------|
-| Query name (`query id`) | Expected impact | Message. |
+| Comparison of narrow type with wide type in loop condition (`cpp/comparison-with-wider-type`) | Higher precision | The precision of this query has been increased to "high" as the alerts from this query have proved to be valuable on real-world projects. With this precision, results are now displayed by default in LGTM. |
 | Hard-coded Japanese era start date in call (`cpp/japanese-era/constructor-or-method-with-exact-era-date`) | Deprecated | This query has been deprecated.  Use the new combined query Hard-coded Japanese era start date (`cpp/japanese-era/exact-era-date`) instead. |
 | Hard-coded Japanese era start date in struct (`cpp/japanese-era/struct-with-exact-era-date`) | Deprecated | This query has been deprecated.  Use the new combined query Hard-coded Japanese era start date (`cpp/japanese-era/exact-era-date`) instead. |
 | Hard-coded Japanese era start date (`cpp/japanese-era/exact-era-date`) | More correct results | This query now checks for the beginning date of the Reiwa era (1st May 2019). |
+| Non-constant format string (`cpp/non-constant-format`) | Fewer false positive results | Fixed false positive results triggrered by mismatching declarations of a formatting function. |
 | Sign check of bitwise operation (`cpp/bitwise-sign-check`) | Fewer false positive results | Results involving `>=` or `<=` are no longer reported. |
-| Too few arguments to formatting function (`cpp/wrong-number-format-arguments`) | Fewer false positive results | Fixed false positives resulting from mistmatching declarations of a formatting function. |
+| Too few arguments to formatting function (`cpp/wrong-number-format-arguments`) | Fewer false positive results | Fixed false positive results triggered by mismatching declarations of a formatting function. |
-| Too many arguments to formatting function (`cpp/too-many-format-arguments`) | Fewer false positive results | Fixed false positives resulting from mistmatching declarations of a formatting function. |
+| Too many arguments to formatting function (`cpp/too-many-format-arguments`) | Fewer false positive results | Fixed false positive results triggered by mismatching declarations of a formatting function. |
-| Unclear comparison precedence (`cpp/comparison-precedence`) | Fewer false positive results | False positives involving template classes and functions have been fixed. |
+| Unclear comparison precedence (`cpp/comparison-precedence`) | Fewer false positive results | False positive results involving template classes and functions have been fixed. |
-| Comparison of narrow type with wide type in loop condition (`cpp/comparison-with-wider-type`) | Higher precision | The precision of this query has been increased to "high" as the alerts from this query have proved to be valuable on real-world projects. With this precision, results are now displayed by default in LGTM. |
+| Wrong type of arguments to formatting function (`cpp/wrong-type-format-argument`) | More correct results and fewer false positive results | This query now understands explicitly-specified argument numbers in format strings, such as the `1$` in `%1$s`. |
-| Non-constant format string (`cpp/non-constant-format`) | Fewer false positive results | Fixed false positives resulting from mistmatching declarations of a formatting function. |
-| Wrong type of arguments to formatting function (`cpp/wrong-type-format-argument`) | More correct results and fewer false positive results | This query now understands explicitly specified argument numbers in format strings, such as the `1$` in `%1$s`. |
 
 ## Changes to libraries
 
-* The data-flow library has been extended with a new feature to aid debugging.
+* The data-flow library in `semmle.code.cpp.dataflow.DataFlow` and
-  Instead of specifying `isSink(Node n) { any() }` on a configuration to
+  `semmle.code.cpp.dataflow.TaintTracking` have had extensive changes:
-  explore the possible flow from a source, it is recommended to use the new
+  * Data flow through fields is now more complete and reliable.
-  `Configuration::hasPartialFlow` predicate, as this gives a more complete
+  * The data-flow library has been extended with a new feature to aid debugging. 
-  picture of the partial flow paths from a given source. The feature is
+    Previously, to explore the possible flow from all sources you could specify `isSink(Node n) { any() }` on a configuration. 
-  disabled by default and can be enabled for individual configurations by
+    Now you can use the new `Configuration::hasPartialFlow` predicate, 
-  overriding `int explorationLimit()`.
+    which gives a more complete picture of the partial flow paths from a given source, including flow that doesn't reach any sink.
-* The data-flow library now supports flow out of C++ reference parameters.
+    The feature is disabled by default and can be enabled for individual configurations by overriding `int explorationLimit()`.
-* The data-flow library now allows flow through the address-of operator (`&`).
+  * There is now flow out of C++ reference parameters.
-* The `DataFlow::DefinitionByReferenceNode` class now considers `f(x)` to be a
+  * There is now flow through the address-of operator (`&`).
+  * The `DataFlow::DefinitionByReferenceNode` class now considers `f(x)` to be a
     definition of `x` when `x` is a variable of pointer type. It no longer
     considers deep paths such as `f(&x.myField)` to be definitions of `x`. These
     changes are in line with the user expectations we've observed.
-* The data-flow library now makes it easier to specify barriers/sanitizers
+  * It's now easier to specify barriers/sanitizers
     arising from guards by overriding the predicate
     `isBarrierGuard`/`isSanitizerGuard` on data-flow and taint-tracking
     configurations respectively.
-* There is now a `DataFlow::localExprFlow` predicate and a
+  * There is now a `DataFlow::localExprFlow` predicate and a
     `TaintTracking::localExprTaint` predicate to make it easy to use the most
     common case of local data flow and taint: from one `Expr` to another.
 * The member predicates of the `FunctionInput` and `FunctionOutput` classes have been renamed for
-  clarity (e.g. `isOutReturnPointer()` to `isReturnValueDeref()`). The existing member predicates
+  clarity (for example, `isOutReturnPointer()` to `isReturnValueDeref()`). The existing member predicates
   have been deprecated, and will be removed in a future release. Code that uses the old member
   predicates should be updated to use the corresponding new member predicate.
-* The predicates `Declaration.hasStdName()` and `Declaration.hasGlobalOrStdName`
+* The predicate `Declaration.hasGlobalOrStdName` has been added, making it
-  have been added, simplifying handling of C++ standard library functions.
+  easier to recognize C library functions called from C++.
 * The control-flow graph is now computed in QL, not in the extractor. This can
-  lead to regressions (or improvements) in how queries are optimized because
+  lead to changes in how queries are optimized because
   optimization in QL relies on static size estimates, and the control-flow edge
   relations will now have different size estimates than before.
 * Support has been added for non-type template arguments.  This means that the
   return type of `Declaration::getTemplateArgument()` and
-  `Declaration::getATemplateArgument` have changed to `Locatable`.  See the
+  `Declaration::getATemplateArgument` have changed to `Locatable`.  For details, see the
-  documentation for `Declaration::getTemplateArgument()` and
+  CodeQL library documentation for `Declaration::getTemplateArgument()` and
-  `Declaration::getTemplateArgumentKind()` for details.
+  `Declaration::getTemplateArgumentKind()`.

change-notes/1.23/analysis-java.md

@@ -6,25 +6,23 @@ The following changes in version 1.23 affect Java analysis in all applications.
 
 | **Query**                   | **Tags**  | **Purpose**                                                        |
 |-----------------------------|-----------|--------------------------------------------------------------------|
-| Continue statement that does not continue (`java/continue-in-false-loop`) | correctness | Finds `continue` statements in `do { ... } while (false)` loops. |
+| Continue statement that does not continue (`java/continue-in-false-loop`) | correctness | Finds `continue` statements in `do { ... } while (false)` loops. Results are shown on LGTM by default. |
 
 ## Changes to existing queries
 
 | **Query**                    | **Expected impact**    | **Change**                        |
 |------------------------------|------------------------|-----------------------------------|
-| Dereferenced variable may be null (`java/dereferenced-value-may-be-null`) | Fewer false positives | Certain indirect null guards involving two auxiliary variables known to be equal can now be detected. |
+| Dereferenced variable may be null (`java/dereferenced-value-may-be-null`) | Fewer false positive results | Additional indirect null guards are detected, where two auxiliary variables are known to be equal. |
-| Non-synchronized override of synchronized method (`java/non-sync-override`) | Fewer false positives | Results are now only reported if the immediately overridden method is synchronized. |
+| Non-synchronized override of synchronized method (`java/non-sync-override`) | Fewer false positive results | Results are now only reported if the immediately overridden method is synchronized. |
-| Query built from user-controlled sources (`java/sql-injection`) | More results | The query now identifies arguments to `Statement.executeLargeUpdate` and `Connection.prepareCall` as SQL expressions sinks. |
+| Query built from local-user-controlled sources (`java/sql-injection-local`) | More results | The query now identifies arguments to `Statement.executeLargeUpdate` and `Connection.prepareCall` as sinks for SQL expressions. |
-| Query built from local-user-controlled sources (`java/sql-injection-local`) | More results | The query now identifies arguments to `Statement.executeLargeUpdate` and `Connection.prepareCall` as SQL expressions sinks. |
+| Query built from user-controlled sources (`java/sql-injection`) | More results | The query now identifies arguments to `Statement.executeLargeUpdate` and `Connection.prepareCall` as sinks for SQL expressions. |
-| Query built without neutralizing special characters (`java/concatenated-sql-query`) | More results | The query now identifies arguments to `Statement.executeLargeUpdate` and `Connection.prepareCall` as SQL expressions sinks. |
+| Query built without neutralizing special characters (`java/concatenated-sql-query`) | More results | The query now identifies arguments to `Statement.executeLargeUpdate` and `Connection.prepareCall` as sinks for SQL expressions. |
-| Useless comparison test (`java/constant-comparison`) | Fewer false positives | Additional overflow check patterns are now recognized and no longer reported. |
+| Useless comparison test (`java/constant-comparison`) | Fewer false positive results | Additional overflow check patterns are now recognized and no longer reported. Also, a few bug fixes in the range analysis for floating-point variables gives a further reduction in false positive results. |
 
 ## Changes to libraries
 
-* The data-flow library has been extended with a new feature to aid debugging.
+The data-flow library has been extended with a new feature to aid debugging. 
-  Instead of specifying `isSink(Node n) { any() }` on a configuration to
+Previously, to explore the possible flow from all sources you could specify `isSink(Node n) { any() }` on a configuration. 
-  explore the possible flow from a source, it is recommended to use the new
+Now you can use the new `Configuration::hasPartialFlow` predicate, 
-  `Configuration::hasPartialFlow` predicate, as this gives a more complete
+which gives a more complete picture of the partial flow paths from a given source, including flow that doesn't reach any sink.
-  picture of the partial flow paths from a given source. The feature is
+The feature is disabled by default and can be enabled for individual configurations by overriding `int explorationLimit()`.
-  disabled by default and can be enabled for individual configurations by
-  overriding `int explorationLimit()`.

change-notes/1.23/analysis-javascript.md

@@ -27,7 +27,7 @@
 |---------------------------------------------------------------------------|-------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
 | Ignoring result from pure array method (`js/ignore-array-result`)         | maintainability, correctness                                      | Highlights calls to array methods without side effects where the return value is ignored. Results are shown on LGTM by default. |
 | Incomplete URL scheme check (`js/incomplete-url-scheme-check`)            | security, correctness, external/cwe/cwe-020                       | Highlights checks for `javascript:` URLs that do not take `data:` or `vbscript:` URLs into account. Results are shown on LGTM by default. |
-| Loop bound injection (`js/loop-bound-injection`)                          | security, external/cwe/cwe-834                                      | Highlights loops where a user-controlled object with an arbitrary .length value can trick the server to loop indefinitely. Results are shown on LGTM by default. |
+| Loop bound injection (`js/loop-bound-injection`)                          | security, external/cwe/cwe-834                                      | Highlights loops where a user-controlled object with an arbitrary `.length` value can trick the server into looping indefinitely. Results are shown on LGTM by default. |
 | Shell command built from environment values (`js/shell-command-injection-from-environment`) | correctness, security, external/cwe/cwe-078, external/cwe/cwe-088 | Highlights shell commands that may change behavior inadvertently depending on the execution environment, indicating a possible violation of [CWE-78](https://cwe.mitre.org/data/definitions/78.html). Results are shown on LGTM by default.|
 | Suspicious method name (`js/suspicious-method-name-declaration`)          | correctness, typescript, methods                                  | Highlights suspiciously named methods where the developer likely meant to write a constructor or function. Results are shown on LGTM by default. |
 | Unreachable method overloads (`js/unreachable-method-overloads`)          | correctness, typescript                                           | Highlights method overloads that are impossible to use from client code. Results are shown on LGTM by default. |
@@ -39,19 +39,19 @@
 
 | **Query**                      | **Expected impact**          | **Change**                                                                |
 |--------------------------------|------------------------------|---------------------------------------------------------------------------|
-| Incomplete string escaping or encoding (`js/incomplete-sanitization`) | Fewer false-positive results | This rule now recognizes additional ways delimiters can be stripped away. |
+| Client-side cross-site scripting (`js/xss`) | More results, fewer false positive results | More potential vulnerabilities involving functions that manipulate DOM attributes are now recognized, and more sanitizers are detected. |
-| Client-side cross-site scripting (`js/xss`) | More results, fewer false-positive results | More potential vulnerabilities involving functions that manipulate DOM attributes are now recognized, and more sanitizers are detected. |
 | Code injection (`js/code-injection`) | More results | More potential vulnerabilities involving functions that manipulate DOM event handler attributes are now recognized. |
-| Hard-coded credentials (`js/hardcoded-credentials`) | Fewer false-positive results | This rule now flags fewer password examples. |
+| Hard-coded credentials (`js/hardcoded-credentials`) | Fewer false positive results | This rule now flags fewer password examples. |
-| Illegal invocation (`js/illegal-invocation`) | Fewer false-positive results | This rule now correctly handles methods named `call` and `apply`. |
+| Illegal invocation (`js/illegal-invocation`) | Fewer false positive results | This rule now correctly handles methods named `call` and `apply`. |
-| Incorrect suffix check (`js/incorrect-suffix-check`) | Fewer false-positive results | The query recognizes valid checks in more cases. |
+| Incomplete string escaping or encoding (`js/incomplete-sanitization`) | Fewer false positive results | This rule now recognizes additional ways delimiters can be stripped away. |
-| Network data written to file (`js/http-to-file-access`) | Fewer false-positive results | This query has been renamed to better match its intended purpose, and now only considers network data untrusted. | 
+| Incorrect suffix check (`js/incorrect-suffix-check`) | Fewer false positive results | The query recognizes valid checks in more cases. |
-| Password in configuration file (`js/password-in-configuration-file`) | Fewer false-positive results | This rule now flags fewer password examples. |
+| Network data written to file (`js/http-to-file-access`) | Fewer false positive results | This query has been renamed to better match its intended purpose, and now only considers network data untrusted. | 
+| Password in configuration file (`js/password-in-configuration-file`) | Fewer false positive results | This rule now flags fewer password examples. |
 | Prototype pollution (`js/prototype-pollution`) | More results | The query now highlights vulnerable uses of jQuery and Angular, and the results are shown on LGTM by default. |
-| Reflected cross-site scripting (`js/reflected-xss`) | Fewer false-positive results | The query now recognizes more sanitizers. |
+| Reflected cross-site scripting (`js/reflected-xss`) | Fewer false positive results | The query now recognizes more sanitizers. |
-| Stored cross-site scripting (`js/stored-xss`) | Fewer false-positive results | The query now recognizes more sanitizers. |
+| Stored cross-site scripting (`js/stored-xss`) | Fewer false positive results | The query now recognizes more sanitizers. |
 | Uncontrolled command line (`js/command-line-injection`) | More results | This query now treats responses from servers as untrusted. |
-| Uncontrolled data used in path expression (`js/path-injection`) | Fewer false-positive results | This query now recognizes calls to Express `sendFile` as safe in some cases. |
+| Uncontrolled data used in path expression (`js/path-injection`) | Fewer false positive results | This query now recognizes calls to Express `sendFile` as safe in some cases. |
 | Unknown directive (`js/unknown-directive`) | Fewer false positive results | This query no longer flags uses of ":", which is sometimes used like a directive. |
 
 ## Changes to libraries
@@ -67,16 +67,16 @@
 
 The following queries (deprecated since 1.17) are no longer available in the distribution:
 
-* Builtin redefined (js/builtin-redefinition)
-* Inefficient method definition (js/method-definition-in-constructor)
 * Bad parity check (js/incomplete-parity-check)
-* Potentially misspelled property or variable name (js/wrong-capitalization)
+* Builtin redefined (js/builtin-redefinition)
-* Unknown JSDoc tag (js/jsdoc/unknown-tag-type)
+* Call to parseInt without radix (js/parseint-without-radix)
+* Inefficient method definition (js/method-definition-in-constructor)
 * Invalid JSLint directive (js/jslint/invalid-directive)
 * Malformed JSLint directive (js/jslint/malformed-directive)
-* Use of HTML comments (js/html-comment)
 * Multi-line string literal (js/multi-line-string)
 * Octal literal (js/octal-literal)
+* Potentially misspelled property or variable name (js/wrong-capitalization)
 * Reserved word used as variable name (js/use-of-reserved-word)
 * Trailing comma in array or object expressions (js/trailing-comma-in-array-or-object)
-* Call to parseInt without radix (js/parseint-without-radix)
+* Unknown JSDoc tag (js/jsdoc/unknown-tag-type)
+* Use of HTML comments (js/html-comment)

change-notes/1.23/analysis-python.md

@@ -3,7 +3,19 @@
 
 ## General improvements
 
+### Python 3.8 support
 
+Python 3.8 syntax is now supported. In particular, the following constructs are parsed correctly:
+
+- Assignment expressions using the "walrus" operator, such as `while chunk := file.read(1024): ...`.
+- The positional argument separator `/`, such as in `def foo(a, /, b, *, c): ...`.
+- Self-documenting expressions in f-strings, such as `f"{var=}"`.
+
+### General query improvements
+
+Following the replacement of the `Object` API (for example, `ClassObject`) in favor of the
+`Value` API (for example, `ClassValue`) in the 1.21 release, many of the standard queries have been updated
+to use the `Value` API. This should result in more precise results.
 
 ## New queries
 
@@ -18,10 +30,23 @@
 
 | **Query**                  | **Expected impact**    | **Change** |
 |----------------------------|------------------------|------------|
-| Unreachable code | Fewer false positives | Analysis now accounts for uses of `contextlib.suppress` to suppress exceptions. |
+| Explicit export is undefined (`py/undefined-export`) | Fewer false positive results | Instances where an exported value may be defined in a module that lacks points-to information are no longer flagged. |
-| `__iter__` method returns a non-iterator | Better alert message | Alert now highlights which class is expected to be an iterator. |
+| Module-level cyclic import (`py/unsafe-cyclic-import`) | Fewer false positive results | Instances where one of the links in an import cycle is never actually executed are no longer flagged. |
-
+| Non-iterable used in for loop (`py/non-iterable-in-for-loop`) | Fewer false positive results | `__aiter__` is now recognized as an iterator method. |
+| Unreachable code (`py/unreachable-statement`) | Fewer false positive results | Analysis now accounts for uses of `contextlib.suppress` to suppress exceptions. |
+| Unreachable code (`py/unreachable-statement`) | Fewer false positive results | Unreachable `else` branches that do nothing but `assert` their non-reachability are no longer flagged. |
+| Unused import (`py/unused-import`) | Fewer false positive results | Instances where a module is used in a forward-referenced type annotation, or only during type checking are no longer flagged. |
+| `__iter__` method returns a non-iterator (`py/iter-returns-non-iterator`) | Better alert message | Alert now highlights which class is expected to be an iterator. |
+| `__init__` method returns a value (`py/explicit-return-in-init`) | Fewer false positive results | Instances where the `__init__` method returns the value of a call to a procedure are no longer flagged. |
 
 ## Changes to QL libraries
 
 * Django library now recognizes positional arguments from a `django.conf.urls.url` regex (Django version 1.x)
+* Instances of the `Value` class now support the `isAbsent` method, indicating
+  whether that `Value` lacks points-to information, but inference
+  suggests that it exists. For instance, if a file contains `import
+  django`, but `django` was not extracted properly, there will be a
+  `ModuleValue` corresponding to this "unknown" module, and the `isAbsent`
+  method will hold for this `ModuleValue`.
+* The `Expr` class now has a nullary method `pointsTo` that returns the possible
+  instances of `Value` that this expression may have.

change-notes/1.23/extractor-javascript.md

@@ -7,17 +7,17 @@
 * Asynchronous generator methods are now parsed correctly and no longer cause a spurious syntax error.
 * Files in `node_modules` and `bower_components` folders are no longer extracted by default. If you still want to extract files from these folders, you can add the following filters to your `lgtm.yml` file (or add them to existing filters):
 
-```yaml
+  ```yaml
-extraction:
+  extraction:
     javascript:
       index:
         filters:
           - include: "**/node_modules"
           - include: "**/bower_components"
-```
+  ```
 
+* Additional [Flow](https://flow.org/) syntax is now supported.
 * Recognition of CommonJS modules has improved. As a result, some files that were previously extracted as
   global scripts are now extracted as modules.
 * Top-level `await` is now supported.
-* A bug was fixed in how the TypeScript extractor handles default-exported anonymous classes.
+* Bugs were fixed in how the TypeScript extractor handles default-exported anonymous classes and computed-instance field names.
-* A bug was fixed in how the TypeScript extractor handles computed instance field names.

cpp/ql/src/Likely Bugs/Arithmetic/SignedOverflowCheck.qhelp

@@ -18,49 +18,39 @@ optimizing compiler.
 <recommendation>
 <p>
 Solutions to this problem can be thought of as falling into one of two
-categories: (1) rewrite the signed expression so that overflow cannot occur
+categories:
-but the signedness remains, or (2) rewrite (or cast) the signed expression
-into unsigned form.
 </p>
 
-<p>
+<ol>
-Below we list examples of expressions where signed overflow may
+<li>Rewrite the signed expression so that overflow cannot occur
-occur, along with proposed solutions.  The list should not be
+but the signedness remains.</li>
-considered exhaustive.
+<li>Change the variables and all their uses to be unsigned.</li>
-</p>
+</ol>
 
 <p>
-Given <code>unsigned short i, delta</code> and <code>i + delta &lt; i</code>, 
+The following cases all fall into the first category.
-it is possible to rewrite it as <code>(unsigned short)(i + delta)&nbsp;&lt;&nbsp;i</code>.
-Note that <code>i + delta</code>does not actually overflow, due to <code>int</code> promotion
 </p>
 
-<p>
+<ol>
-Given <code>unsigned short i, delta</code> and <code>i + delta &lt; i</code>,
+<li>
-it is also possible to rewrite it as <code>USHORT_MAX - delta</code>. It must be true
+Given <code>unsigned short n1, delta</code> and <code>n1 + delta &lt; n1</code>,
-that <code>delta &gt; 0</code> and the <code>limits.h</code> or <code>climits</code>
+it is possible to rewrite it as <code>(unsigned short)(n1 + delta)&nbsp;&lt;&nbsp;n1</code>.
+Note that <code>n1 + delta</code> does not actually overflow, due to <code>int</code> promotion.
+</li>
+
+<li>
+Given <code>unsigned short n1, delta</code> and <code>n1 + delta &lt; n1</code>,
+it is also possible to rewrite it as <code>n1 &gt; USHORT_MAX - delta</code>. The
+<code>limits.h</code> or <code>climits</code> header must then be included.
+</li>
+
+<li>
+Given <code>int n1, delta</code> and <code>n1 + delta &lt; n1</code>,
+it is possible to rewrite it as <code>n1 &gt; INT_MAX - delta</code>. It must be true
+that <code>delta &gt;= 0</code> and the <code>limits.h</code> or <code>climits</code>
 header has been included.
-</p>
+</li>
-
+</ol>
-<p>
-Given <code>int i, delta</code> and <code>i + delta &lt; i</code>,
-it is possible to rewrite it as <code>INT_MAX - delta</code>. It must be true
-that <code>delta &gt; 0</code> and the <code>limits.h</code> or <code>climits</code>
-header has been included.
-</p>
-
-<p>
-Given <code>int i, delta</code> and <code>i + delta &lt; i</code>,
-it is also possible to rewrite it as <code>(unsigned)i + delta &lt; i</code>.
-Note that program semantics are affected by this change.
-</p>
-
-<p>
-Given <code>int i, delta</code> and <code>i + delta &lt; i</code>,
-it is also possible to rewrite it as <code>unsigned int i, delta</code> and
-<code>i + delta &lt; i</code>.  Note that program semantics are 
-affected by this change.
-</p>
 </recommendation>
 
 <example>
@@ -98,7 +88,7 @@ hold true, which likely is not what the programmer intended. (see also the
 <sample src="SignedOverflowCheck-bad2.cpp" />
 <p>
 The next example provides a solution to the previous one.  Even though
-<code>i + delta</code> does not overflow, casting it to an
+<code>n1 + delta</code> does not overflow, casting it to an
 <code>unsigned short</code> truncates the addition modulo 2^16,
 so that <code>unsigned short</code> "wrap around" may now be observed.
 Furthermore, since the left-hand side is now of type <code>unsigned short</code>,

cpp/ql/src/Likely Bugs/Arithmetic/SignedOverflowCheck.ql

@@ -14,6 +14,7 @@
 import cpp
 private import semmle.code.cpp.valuenumbering.GlobalValueNumbering
 private import semmle.code.cpp.rangeanalysis.SimpleRangeAnalysis
+private import semmle.code.cpp.commons.Exclusions
 
 from RelationalOperation ro, AddExpr add, Expr expr1, Expr expr2
 where
@@ -22,7 +23,7 @@ where
   ro.getAnOperand() = expr2 and
   globalValueNumber(expr1) = globalValueNumber(expr2) and
   add.getUnspecifiedType().(IntegralType).isSigned() and
-  not exists(MacroInvocation mi | mi.getAnAffectedElement() = add) and
+  not isFromMacroDefinition(ro) and
   exprMightOverflowPositively(add) and
   exists(Compilation c | c.getAFileCompiled() = ro.getFile() |
     not c.getAnArgument() = "-fwrapv" and

cpp/ql/src/Security/CWE/CWE-190/ComparisonWithWiderType.ql

@@ -44,7 +44,7 @@ Element friendlyLoc(Expr e) {
   not e instanceof Access and not e instanceof Call and result = e
 }
 
-from Loop l, RelationalOperation rel, Expr small, Expr large
+from Loop l, RelationalOperation rel, VariableAccess small, Expr large
 where
   small = rel.getLesserOperand() and
   large = rel.getGreaterOperand() and
@@ -60,7 +60,7 @@ where
   not getComparisonSize(large.(SubExpr).getLeftOperand().getExplicitlyConverted()) <= getComparisonSize(small) and
   not getComparisonSize(large.(RShiftExpr).getLeftOperand().getExplicitlyConverted()) <= getComparisonSize(small) and
   // ignore loop-invariant smaller variables
-  loopVariant(small.getAChild*(), l)
+  loopVariant(small, l)
 select rel,
   "Comparison between $@ of type " + small.getType().getName() + " and $@ of wider type " +
     large.getType().getName() + ".", friendlyLoc(small), small.toString(), friendlyLoc(large),

cpp/ql/src/semmle/code/cpp/Enclosing.qll

@@ -13,6 +13,11 @@ Element stmtEnclosingElement(Stmt s) {
 /**
  * Gets the enclosing element of expression `e`.
  */
+// The `pragma[nomagic]` is a workaround to prevent this cached stage (and all
+// subsequent stages) from being evaluated twice. See QL-888. It has the effect
+// of making the `Conversion` class predicate get the same optimization in all
+// queries.
+pragma[nomagic]
 cached
 Element exprEnclosingElement(Expr e) {
   result = exprEnclosingElement(e.getParent())

cpp/ql/src/semmle/code/cpp/commons/Exclusions.qll

@@ -94,11 +94,18 @@ predicate functionContainsPreprocCode(Function f) {
  * ```
  */
 predicate isFromMacroDefinition(Element e) {
-  exists(MacroInvocation mi |
+  exists(MacroInvocation mi, Location eLocation, Location miLocation |
-    // e is in mi
     mi.getAnExpandedElement() = e and
-    // and e was apparently not passed in as a macro parameter
+    eLocation = e.getLocation() and
-    e.getLocation().getStartLine() = mi.getLocation().getStartLine() and
+    miLocation = mi.getLocation() and
-    e.getLocation().getStartColumn() = mi.getLocation().getStartColumn()
+    // If the location of `e` coincides with the macro invocation, then `e` did
+    // not come from a macro argument. The inequalities here could also be
+    // equalities, but that confuses the join orderer into joining on the source
+    // locations too early.
+    // There are cases where the start location of a non-argument element comes
+    // right after the invocation's open parenthesis, so it appears to be more
+    // robust to match on the end location instead.
+    eLocation.getEndLine() >= miLocation.getEndLine() and
+    eLocation.getEndColumn() >= miLocation.getEndColumn()
   )
 }

cpp/ql/src/semmle/code/cpp/dataflow/internal/DataFlowImplCommon.qll

@@ -360,7 +360,7 @@ private module ImplCommon {
      */
     cached
     predicate read(Node node1, Content f, Node node2) {
-      readStep(node1, f, node2) and storeStep(_, f, _)
+      readStep(node1, f, node2)
       or
       exists(DataFlowCall call, ReturnKind kind |
         read0(call, kind, node1, f) and

cpp/ql/src/semmle/code/cpp/ir/dataflow/internal/DataFlowImplCommon.qll

@@ -360,7 +360,7 @@ private module ImplCommon {
      */
     cached
     predicate read(Node node1, Content f, Node node2) {
-      readStep(node1, f, node2) and storeStep(_, f, _)
+      readStep(node1, f, node2)
       or
       exists(DataFlowCall call, ReturnKind kind |
         read0(call, kind, node1, f) and

cpp/ql/src/semmle/code/cpp/ir/implementation/unaliased_ssa/internal/SimpleSSA.qll

@@ -1,5 +1,6 @@
 import AliasAnalysis
 private import SimpleSSAImports
+import SimpleSSAPublicImports
 
 private class IntValue = Ints::IntValue;
 

cpp/ql/src/semmle/code/cpp/ir/implementation/unaliased_ssa/internal/SimpleSSAImports.qll

@@ -2,4 +2,3 @@ import semmle.code.cpp.ir.implementation.raw.IR
 import semmle.code.cpp.ir.internal.IntegerConstant as Ints
 import semmle.code.cpp.ir.implementation.internal.OperandTag
 import semmle.code.cpp.ir.internal.IRCppLanguage as Language
-import semmle.code.cpp.ir.internal.Overlap

cpp/ql/src/semmle/code/cpp/ir/implementation/unaliased_ssa/internal/SimpleSSAPublicImports.qll

@@ -0,0 +1 @@
+import semmle.code.cpp.ir.internal.Overlap

cpp/ql/src/semmle/uml/MagicDraw.qll

@@ -112,7 +112,7 @@ class UMLType extends UMLElement {
     else result = this.getUMLName()
   }
 
-  string toString() { result = this.getUMLName() }
+  override string toString() { result = this.getUMLName() }
 }
 
 /**
@@ -163,7 +163,7 @@ class UMLProperty extends UMLElement {
     result.getDeclaringType() = this.getUMLType().getCType()
   }
 
-  string toString() {
+  override string toString() {
     if this.isEnumConstant()
     then result = "- <<enum constant>> " + this.getUMLName()
     else result = "- " + this.getUMLName()
@@ -196,7 +196,7 @@ class UMLOperation extends UMLElement {
     result.getDeclaringType() = this.getUMLType().getCType()
   }
 
-  string toString() { result = "+ " + this.getUMLName() }
+  override string toString() { result = "+ " + this.getUMLName() }
 }
 
 /**
@@ -221,7 +221,7 @@ class UMLAssociation extends UMLProperty {
   /**
    * Gets the C field corresponding to this property, if any.
    */
-  Field getCField() {
+  override Field getCField() {
     result.hasName(this.getLabel()) and
     result.getDeclaringType() = this.getSource().getCType()
   }
@@ -271,7 +271,7 @@ class UMLInheritance extends UMLElement {
     )
   }
 
-  string toString() {
+  override string toString() {
     result = this.getUMLClient().getUMLName() + " implements " + this.getUMLSupplier().getUMLName()
   }
 }
@@ -303,5 +303,5 @@ class UMLPackage extends UMLElement {
     else result = this.getUMLName()
   }
 
-  string toString() { result = this.getUMLQualifiedName() }
+  override string toString() { result = this.getUMLQualifiedName() }
 }

csharp/ql/src/semmle/code/csharp/dataflow/internal/DataFlowImplCommon.qll

@@ -360,7 +360,7 @@ private module ImplCommon {
      */
     cached
     predicate read(Node node1, Content f, Node node2) {
-      readStep(node1, f, node2) and storeStep(_, f, _)
+      readStep(node1, f, node2)
       or
       exists(DataFlowCall call, ReturnKind kind |
         read0(call, kind, node1, f) and

csharp/ql/src/semmle/code/csharp/ir/implementation/unaliased_ssa/internal/SimpleSSA.qll

@@ -1,5 +1,6 @@
 import AliasAnalysis
 private import SimpleSSAImports
+import SimpleSSAPublicImports
 
 private class IntValue = Ints::IntValue;
 

csharp/ql/src/semmle/code/csharp/ir/implementation/unaliased_ssa/internal/SimpleSSAImports.qll

@@ -2,4 +2,3 @@ import semmle.code.csharp.ir.implementation.raw.IR
 import semmle.code.csharp.ir.internal.IntegerConstant as Ints
 import semmle.code.csharp.ir.implementation.internal.OperandTag
 import semmle.code.csharp.ir.internal.IRCSharpLanguage as Language
-import semmle.code.csharp.ir.internal.Overlap

csharp/ql/src/semmle/code/csharp/ir/implementation/unaliased_ssa/internal/SimpleSSAPublicImports.qll

@@ -0,0 +1 @@
+import semmle.code.csharp.ir.internal.Overlap

docs/language/learn-ql/cpp/dataflow.rst

@@ -244,6 +244,49 @@ The following data flow configuration tracks data flow from environment variable
    select fopen, "This 'fopen' uses data from $@.",
      getenv, "call to 'getenv'"
 
+The following taint-tracking configuration tracks data from a call to ``ntohl`` to an array index operation. It uses the ``Guards`` library to recognize expressions that have been bounds-checked, and defines ``isSanitizer`` to prevent taint from propagating through them. It also uses ``isAdditionalTaintStep`` to add flow from loop bounds to loop indexes.
+
+.. code-block:: ql
+
+  import cpp
+  import semmle.code.cpp.controlflow.Guards
+  import semmle.code.cpp.dataflow.TaintTracking
+
+  class NetworkToBufferSizeConfiguration extends TaintTracking::Configuration {
+    NetworkToBufferSizeConfiguration() { this = "NetworkToBufferSizeConfiguration" }
+
+    override predicate isSource(DataFlow::Node node) {
+      node.asExpr().(FunctionCall).getTarget().hasGlobalName("ntohl")
+    }
+
+    override predicate isSink(DataFlow::Node node) {
+      exists(ArrayExpr ae | node.asExpr() = ae.getArrayOffset())
+    }
+
+    override predicate isAdditionalTaintStep(DataFlow::Node pred, DataFlow::Node succ) {
+      exists(Loop loop, LoopCounter lc |
+        loop = lc.getALoop() and
+        loop.getControllingExpr().(RelationalOperation).getGreaterOperand() = pred.asExpr() |
+        succ.asExpr() = lc.getVariableAccessInLoop(loop)
+      )
+    }
+
+    override predicate isSanitizer(DataFlow::Node node) {
+      exists(GuardCondition gc, Variable v |
+        gc.getAChild*() = v.getAnAccess() and
+        node.asExpr() = v.getAnAccess() and
+        gc.controls(node.asExpr().getBasicBlock(), _)
+      )
+    }
+  }
+
+  from DataFlow::Node ntohl, DataFlow::Node offset, NetworkToBufferSizeConfiguration conf
+  where conf.hasFlow(ntohl, offset)
+  select offset, "This array offset may be influenced by $@.", ntohl,
+    "converted data from the network"
+
+
+
 Exercises
 ~~~~~~~~~
 

docs/language/learn-ql/ql-training.rst

@@ -6,7 +6,7 @@ CodeQL and variant analysis
 
 `Variant analysis <https://semmle.com/variant-analysis>`__ is the process of using a known vulnerability as a seed to find similar problems in your code. Security engineers typically perform variant analysis to identify possible vulnerabilities and to ensure that these threats are properly fixed across multiple code bases.
 
-`CodeQL <https://semmle.com/ql>`__ is the code analysis engine that underpins LGTM, Semmle's community driven security analysis platform. Together, CodeQL and LGTM provide continuous monitoring and scalable variant analysis for your projects, even if you don’t have your own team of dedicated security engineers. You can read more about using CodeQL and LGTM in variant analysis in the `Semmle blog <https://blog.semmle.com/tags/variant-analysis>`__.
+`CodeQL <https://semmle.com/ql>`__ is the code analysis engine that underpins LGTM, Semmle's community driven security analysis platform. Together, CodeQL and LGTM provide continuous monitoring and scalable variant analysis for your projects, even if you don’t have your own team of dedicated security engineers. You can read more about using CodeQL and LGTM in variant analysis on the `Security Lab research page <https://securitylab.github.com/research>`__.
 
 CodeQL is easy to learn, and exploring code using CodeQL is the most efficient way to perform variant analysis. 
 
@@ -62,5 +62,4 @@ More resources
 
 - If you are completely new to CodeQL, look at our introductory topics in :doc:`Learning CodeQL <index>`.
 - To find more detailed information about how to write queries for specific languages, visit the links in :ref:`Writing CodeQL queries <writing-ql-queries>`.
-- To read more about how CodeQL queries have been used in Semmle's security research, and to read about new CodeQL developments, visit the `Semmle blog <https://blog.semmle.com>`__. 
+- To see examples of CodeQL queries that have been used to find security vulnerabilities and bugs in open-source software projects, visit the `GitHub Security Lab website <https://securitylab.github.com/research>`__ and the associated `repository <https://github.com/github/security-lab>`__.
-- Find more examples of queries written by Semmle's own security researchers in the `Semmle Demos repository <https://github.com/semmle/demos>`__ on GitHub.

docs/language/learn-ql/writing-queries/introduction-to-queries.rst

@@ -136,7 +136,7 @@ When you write a custom query, we also recommend that you write a query help fil
 What next?
 ==========
 
-- See the queries used in real-life variant analysis on the `Semmle blog <https://blog.semmle.com/tags/variant-analysis>`__.
+- See the queries used in real-life variant analysis on the `GitHub Security Lab website <https://securitylab.github.com/research>`__.
 - To learn more about writing path queries, see :doc:`Constructing path queries <path-queries>`.
 - Take a look at the `built-in queries <https://help.semmle.com/wiki/display/QL/Built-in+queries>`__ to see examples of the queries included in CodeQL.
 - Explore the `query cookbooks <https://help.semmle.com/wiki/display/QL/QL+cookbooks>`__ to see how to access the basic language elements contained in the CodeQL libraries.

docs/language/learn-ql/writing-queries/path-queries.rst

@@ -37,7 +37,7 @@ The easiest way to get started writing your own path query is to modify one of t
 - `JavaScript path queries <https://help.semmle.com/wiki/label/js/path-problem>`__
 - `Python path queries <https://help.semmle.com/wiki/label/python/path-problem>`__
  
-The Semmle security research team have used path queries to find security vulnerabilities in various open source projects. To see articles describing how these queries were written, as well as other posts describing other aspects of security research such as exploiting vulnerabilities, browse the security posts on the `Semmle blog <https://blog.semmle.com/tags/security>`__.
+The Security Lab researchers have used path queries to find security vulnerabilities in various open source projects. To see articles describing how these queries were written, as well as other posts describing other aspects of security research such as exploiting vulnerabilities, see the `GitHub Security Lab website <https://securitylab.github.com/research>`__.
 
 Constructing a path query
 =========================

docs/language/learn-ql/writing-queries/writing-queries.rst

@@ -33,7 +33,7 @@ Viewing existing CodeQL queries
 The easiest way to get started writing your own queries is to modify an existing query. To see these queries, or to try out the CodeQL query cookbooks, visit `Exploring CodeQL queries <https://help.semmle.com/QL/ql-explore-queries.html>`__.
 You can also find all the CodeQL queries in our `open source repository on GitHub <https://github.com/semmle/ql>`__. 
 
-You can also find examples of queries developed to find security vulnerabilities and bugs in open-source software projects in the `Semmle demos GitHub repository <https://github.com/semmle/demos>`__ and the `Semmle blog <https://blog.semmle.com/tags/security>`__.
+You can also find examples of queries developed to find security vulnerabilities and bugs in open-source software projects on the `GitHub Security Lab website <https://securitylab.github.com/research>`__ and in the associated `repository <https://github.com/github/security-lab>`__.
 
 Contributing queries
 ********************

docs/language/ql-training/cpp/snprintf.rst

@@ -65,7 +65,7 @@ RCE in rsyslog
     }
 
 - Disclosed as `CVE-2018-1000140 <https://nvd.nist.gov/vuln/detail/CVE-2018-1000140>`__.
-- Blog post: `https://blog.semmle.com/librelp-buffer-overflow-cve-2018-1000140/ <https://blog.semmle.com/librelp-buffer-overflow-cve-2018-1000140/>`__.
+- Blog post: https://securitylab.github.com/research/librelp-buffer-overflow-cve-2018-1000140
 
 Finding the RCE yourself
 ========================

docs/language/ql-training/java/apache-struts-java.rst

@@ -58,7 +58,7 @@ RCE in Apache Struts
 
 - Disclosed as `CVE-2017-9805 <http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2017-9805>`__
 
-- Blog post: https://blog.semmle.com/apache-struts-vulnerability-cve-2017-9805/
+- Blog post: https://securitylab.github.com/research/apache-struts-vulnerability-cve-2017-9805
 
 Finding the RCE yourself
 ========================
@@ -134,4 +134,4 @@ Model answer, step 4
      and sink.getNode() instanceof UnsafeDeserializationSink
    select sink.getNode().(UnsafeDeserializationSink).getMethodAccess(), source, sink, "Unsafe    deserialization of $@.", source, "user input"
 
-More full-featured version: https://github.com/Semmle/demos/tree/master/ql_demos/java/Apache_Struts_CVE-2017-9805
+More full-featured version: https://github.com/github/security-lab/tree/master/CodeQL_Queries/java/Apache_Struts_CVE-2017-9805

docs/language/ql-training/java/global-data-flow-java.rst

@@ -53,8 +53,8 @@ Code injection in Apache struts
 
 .. note::
 
-   More details on the CVE can be found here: https://blog.semmle.com/apache-struts-CVE-2018-11776/ and 
+   More details on the CVE can be found here: https://securitylab.github.com/research/apache-struts-CVE-2018-11776 and 
-   https://github.com/Semmle/demos/tree/master/ql_demos/java/Apache_Struts_CVE-2018-11776
+   https://github.com/github/security-lab/tree/master/CodeQL_Queries/java/Apache_Struts_CVE-2018-11776
 
    More details on OGNL can be found here: https://commons.apache.org/proper/commons-ognl/
 

java/ql/src/semmle/code/java/dataflow/internal/DataFlowImplCommon.qll

@@ -360,7 +360,7 @@ private module ImplCommon {
      */
     cached
     predicate read(Node node1, Content f, Node node2) {
-      readStep(node1, f, node2) and storeStep(_, f, _)
+      readStep(node1, f, node2)
       or
       exists(DataFlowCall call, ReturnKind kind |
         read0(call, kind, node1, f) and

java/ql/test/library-tests/dataflow/fields/E.java

@@ -0,0 +1,32 @@
+public class E {
+  static Object src() { return new Object(); }
+  static void sink(Object obj) {}
+
+  static class Buffer { Object content; }
+  static class BufHolder { Buffer buf; }
+  static class Packet { BufHolder data; }
+
+  static void recv(Buffer buf) {
+    buf.content = src();
+  }
+
+  static void foo(Buffer raw, BufHolder bh, Packet p) {
+    recv(raw);
+    recv(bh.buf);
+    recv(p.data.buf);
+
+    sink(raw.content);
+
+    BufHolder bh2 = bh;
+    sink(bh2.buf.content);
+
+    Packet p2 = p;
+    sink(p2.data.buf.content);
+
+    handlepacket(p);
+  }
+
+  static void handlepacket(Packet p) {
+    sink(p.data.buf.content);
+  }
+}

java/ql/test/library-tests/dataflow/fields/flow.expected

@@ -22,3 +22,7 @@
 | D.java:19:14:19:23 | new Elem(...) | D.java:33:10:33:31 | getElem(...) |
 | D.java:26:14:26:23 | new Elem(...) | D.java:33:10:33:31 | getElem(...) |
 | D.java:37:14:37:23 | new Elem(...) | D.java:44:10:44:26 | boxfield.box.elem |
+| E.java:2:32:2:43 | new Object(...) | E.java:18:10:18:20 | raw.content |
+| E.java:2:32:2:43 | new Object(...) | E.java:21:10:21:24 | bh2.buf.content |
+| E.java:2:32:2:43 | new Object(...) | E.java:24:10:24:28 | p2.data.buf.content |
+| E.java:2:32:2:43 | new Object(...) | E.java:30:10:30:27 | p.data.buf.content |

javascript/extractor/src/com/semmle/jcorn/flow/FlowParser.java

@@ -541,7 +541,11 @@ public class FlowParser extends ESNextParser {
         if (isStatic && this.type == TokenType.colon) {
           this.parseIdent(false);
         } else if (allowSpread && this.eat(TokenType.ellipsis)) {
+          boolean hasType =
+              this.type != endDelim && this.type != TokenType.comma && this.type != TokenType.semi;
+          if (hasType) {
             flowParseType();
+          }
           flowObjectTypeSemicolon();
           continue;
         } else {

javascript/extractor/src/com/semmle/js/extractor/Main.java

@@ -37,7 +37,7 @@ public class Main {
    * A version identifier that should be updated every time the extractor changes in such a way that
    * it may produce different tuples for the same file under the same {@link ExtractorConfig}.
    */
-  public static final String EXTRACTOR_VERSION = "2019-11-19";
+  public static final String EXTRACTOR_VERSION = "2019-11-26";
 
   public static final Pattern NEWLINE = Pattern.compile("\n");
 

javascript/extractor/tests/flow/input/objectTypeSpread.js

@@ -1,2 +1,4 @@
 type A = { x: int };
 type B = { ...A, y: string };
+type C<T> = {c: null | T, ...} | ((r: null | T) => mixed);
+type D = { d: D, ..., };

javascript/extractor/tests/flow/output/trap/objectTypeSpread.js.trap

@@ -16,133 +16,345 @@ lines(#20002,#20001,"type A = { x: int };","
 locations_default(#20003,#10000,1,1,1,20)
 hasLocation(#20002,#20003)
 #20004=*
-lines(#20004,#20001,"type B = { ...A, y: string };","")
+lines(#20004,#20001,"type B = { ...A, y: string };","
+")
 #20005=@"loc,{#10000},2,1,2,29"
 locations_default(#20005,#10000,2,1,2,29)
 hasLocation(#20004,#20005)
-numlines(#20001,2,2,0)
 #20006=*
-tokeninfo(#20006,6,#20001,0,"type")
+lines(#20006,#20001,"type C<T> = {c: null | T, ...} | ((r: null | T) => mixed);","
-#20007=@"loc,{#10000},1,1,1,4"
+")
-locations_default(#20007,#10000,1,1,1,4)
+#20007=@"loc,{#10000},3,1,3,58"
+locations_default(#20007,#10000,3,1,3,58)
 hasLocation(#20006,#20007)
 #20008=*
-tokeninfo(#20008,6,#20001,1,"A")
+lines(#20008,#20001,"type D = { d: D, ..., };","")
-#20009=@"loc,{#10000},1,6,1,6"
+#20009=@"loc,{#10000},4,1,4,24"
-locations_default(#20009,#10000,1,6,1,6)
+locations_default(#20009,#10000,4,1,4,24)
 hasLocation(#20008,#20009)
+numlines(#20001,4,4,0)
 #20010=*
-tokeninfo(#20010,8,#20001,2,"=")
+tokeninfo(#20010,6,#20001,0,"type")
-#20011=@"loc,{#10000},1,8,1,8"
+#20011=@"loc,{#10000},1,1,1,4"
-locations_default(#20011,#10000,1,8,1,8)
+locations_default(#20011,#10000,1,1,1,4)
 hasLocation(#20010,#20011)
 #20012=*
-tokeninfo(#20012,8,#20001,3,"{")
+tokeninfo(#20012,6,#20001,1,"A")
-#20013=@"loc,{#10000},1,10,1,10"
+#20013=@"loc,{#10000},1,6,1,6"
-locations_default(#20013,#10000,1,10,1,10)
+locations_default(#20013,#10000,1,6,1,6)
 hasLocation(#20012,#20013)
 #20014=*
-tokeninfo(#20014,6,#20001,4,"x")
+tokeninfo(#20014,8,#20001,2,"=")
-#20015=@"loc,{#10000},1,12,1,12"
+#20015=@"loc,{#10000},1,8,1,8"
-locations_default(#20015,#10000,1,12,1,12)
+locations_default(#20015,#10000,1,8,1,8)
 hasLocation(#20014,#20015)
 #20016=*
-tokeninfo(#20016,8,#20001,5,":")
+tokeninfo(#20016,8,#20001,3,"{")
-#20017=@"loc,{#10000},1,13,1,13"
+#20017=@"loc,{#10000},1,10,1,10"
-locations_default(#20017,#10000,1,13,1,13)
+locations_default(#20017,#10000,1,10,1,10)
 hasLocation(#20016,#20017)
 #20018=*
-tokeninfo(#20018,6,#20001,6,"int")
+tokeninfo(#20018,6,#20001,4,"x")
-#20019=@"loc,{#10000},1,15,1,17"
+#20019=@"loc,{#10000},1,12,1,12"
-locations_default(#20019,#10000,1,15,1,17)
+locations_default(#20019,#10000,1,12,1,12)
 hasLocation(#20018,#20019)
 #20020=*
-tokeninfo(#20020,8,#20001,7,"}")
+tokeninfo(#20020,8,#20001,5,":")
-#20021=@"loc,{#10000},1,19,1,19"
+#20021=@"loc,{#10000},1,13,1,13"
-locations_default(#20021,#10000,1,19,1,19)
+locations_default(#20021,#10000,1,13,1,13)
 hasLocation(#20020,#20021)
 #20022=*
-tokeninfo(#20022,8,#20001,8,";")
+tokeninfo(#20022,6,#20001,6,"int")
-#20023=@"loc,{#10000},1,20,1,20"
+#20023=@"loc,{#10000},1,15,1,17"
-locations_default(#20023,#10000,1,20,1,20)
+locations_default(#20023,#10000,1,15,1,17)
 hasLocation(#20022,#20023)
 #20024=*
-tokeninfo(#20024,6,#20001,9,"type")
+tokeninfo(#20024,8,#20001,7,"}")
-#20025=@"loc,{#10000},2,1,2,4"
+#20025=@"loc,{#10000},1,19,1,19"
-locations_default(#20025,#10000,2,1,2,4)
+locations_default(#20025,#10000,1,19,1,19)
 hasLocation(#20024,#20025)
 #20026=*
-tokeninfo(#20026,6,#20001,10,"B")
+tokeninfo(#20026,8,#20001,8,";")
-#20027=@"loc,{#10000},2,6,2,6"
+#20027=@"loc,{#10000},1,20,1,20"
-locations_default(#20027,#10000,2,6,2,6)
+locations_default(#20027,#10000,1,20,1,20)
 hasLocation(#20026,#20027)
 #20028=*
-tokeninfo(#20028,8,#20001,11,"=")
+tokeninfo(#20028,6,#20001,9,"type")
-#20029=@"loc,{#10000},2,8,2,8"
+#20029=@"loc,{#10000},2,1,2,4"
-locations_default(#20029,#10000,2,8,2,8)
+locations_default(#20029,#10000,2,1,2,4)
 hasLocation(#20028,#20029)
 #20030=*
-tokeninfo(#20030,8,#20001,12,"{")
+tokeninfo(#20030,6,#20001,10,"B")
-#20031=@"loc,{#10000},2,10,2,10"
+#20031=@"loc,{#10000},2,6,2,6"
-locations_default(#20031,#10000,2,10,2,10)
+locations_default(#20031,#10000,2,6,2,6)
 hasLocation(#20030,#20031)
 #20032=*
-tokeninfo(#20032,8,#20001,13,"...")
+tokeninfo(#20032,8,#20001,11,"=")
-#20033=@"loc,{#10000},2,12,2,14"
+#20033=@"loc,{#10000},2,8,2,8"
-locations_default(#20033,#10000,2,12,2,14)
+locations_default(#20033,#10000,2,8,2,8)
 hasLocation(#20032,#20033)
 #20034=*
-tokeninfo(#20034,6,#20001,14,"A")
+tokeninfo(#20034,8,#20001,12,"{")
-#20035=@"loc,{#10000},2,15,2,15"
+#20035=@"loc,{#10000},2,10,2,10"
-locations_default(#20035,#10000,2,15,2,15)
+locations_default(#20035,#10000,2,10,2,10)
 hasLocation(#20034,#20035)
 #20036=*
-tokeninfo(#20036,8,#20001,15,",")
+tokeninfo(#20036,8,#20001,13,"...")
-#20037=@"loc,{#10000},2,16,2,16"
+#20037=@"loc,{#10000},2,12,2,14"
-locations_default(#20037,#10000,2,16,2,16)
+locations_default(#20037,#10000,2,12,2,14)
 hasLocation(#20036,#20037)
 #20038=*
-tokeninfo(#20038,6,#20001,16,"y")
+tokeninfo(#20038,6,#20001,14,"A")
-#20039=@"loc,{#10000},2,18,2,18"
+#20039=@"loc,{#10000},2,15,2,15"
-locations_default(#20039,#10000,2,18,2,18)
+locations_default(#20039,#10000,2,15,2,15)
 hasLocation(#20038,#20039)
 #20040=*
-tokeninfo(#20040,8,#20001,17,":")
+tokeninfo(#20040,8,#20001,15,",")
-#20041=@"loc,{#10000},2,19,2,19"
+#20041=@"loc,{#10000},2,16,2,16"
-locations_default(#20041,#10000,2,19,2,19)
+locations_default(#20041,#10000,2,16,2,16)
 hasLocation(#20040,#20041)
 #20042=*
-tokeninfo(#20042,6,#20001,18,"string")
+tokeninfo(#20042,6,#20001,16,"y")
-#20043=@"loc,{#10000},2,21,2,26"
+#20043=@"loc,{#10000},2,18,2,18"
-locations_default(#20043,#10000,2,21,2,26)
+locations_default(#20043,#10000,2,18,2,18)
 hasLocation(#20042,#20043)
 #20044=*
-tokeninfo(#20044,8,#20001,19,"}")
+tokeninfo(#20044,8,#20001,17,":")
-#20045=@"loc,{#10000},2,28,2,28"
+#20045=@"loc,{#10000},2,19,2,19"
-locations_default(#20045,#10000,2,28,2,28)
+locations_default(#20045,#10000,2,19,2,19)
 hasLocation(#20044,#20045)
 #20046=*
-tokeninfo(#20046,8,#20001,20,";")
+tokeninfo(#20046,6,#20001,18,"string")
-#20047=@"loc,{#10000},2,29,2,29"
+#20047=@"loc,{#10000},2,21,2,26"
-locations_default(#20047,#10000,2,29,2,29)
+locations_default(#20047,#10000,2,21,2,26)
 hasLocation(#20046,#20047)
 #20048=*
-tokeninfo(#20048,0,#20001,21,"")
+tokeninfo(#20048,8,#20001,19,"}")
-#20049=@"loc,{#10000},2,30,2,29"
+#20049=@"loc,{#10000},2,28,2,28"
-locations_default(#20049,#10000,2,30,2,29)
+locations_default(#20049,#10000,2,28,2,28)
 hasLocation(#20048,#20049)
+#20050=*
+tokeninfo(#20050,8,#20001,20,";")
+#20051=@"loc,{#10000},2,29,2,29"
+locations_default(#20051,#10000,2,29,2,29)
+hasLocation(#20050,#20051)
+#20052=*
+tokeninfo(#20052,6,#20001,21,"type")
+#20053=@"loc,{#10000},3,1,3,4"
+locations_default(#20053,#10000,3,1,3,4)
+hasLocation(#20052,#20053)
+#20054=*
+tokeninfo(#20054,6,#20001,22,"C")
+#20055=@"loc,{#10000},3,6,3,6"
+locations_default(#20055,#10000,3,6,3,6)
+hasLocation(#20054,#20055)
+#20056=*
+tokeninfo(#20056,8,#20001,23,"<")
+#20057=@"loc,{#10000},3,7,3,7"
+locations_default(#20057,#10000,3,7,3,7)
+hasLocation(#20056,#20057)
+#20058=*
+tokeninfo(#20058,6,#20001,24,"T")
+#20059=@"loc,{#10000},3,8,3,8"
+locations_default(#20059,#10000,3,8,3,8)
+hasLocation(#20058,#20059)
+#20060=*
+tokeninfo(#20060,8,#20001,25,">")
+#20061=@"loc,{#10000},3,9,3,9"
+locations_default(#20061,#10000,3,9,3,9)
+hasLocation(#20060,#20061)
+#20062=*
+tokeninfo(#20062,8,#20001,26,"=")
+#20063=@"loc,{#10000},3,11,3,11"
+locations_default(#20063,#10000,3,11,3,11)
+hasLocation(#20062,#20063)
+#20064=*
+tokeninfo(#20064,8,#20001,27,"{")
+#20065=@"loc,{#10000},3,13,3,13"
+locations_default(#20065,#10000,3,13,3,13)
+hasLocation(#20064,#20065)
+#20066=*
+tokeninfo(#20066,6,#20001,28,"c")
+#20067=@"loc,{#10000},3,14,3,14"
+locations_default(#20067,#10000,3,14,3,14)
+hasLocation(#20066,#20067)
+#20068=*
+tokeninfo(#20068,8,#20001,29,":")
+#20069=@"loc,{#10000},3,15,3,15"
+locations_default(#20069,#10000,3,15,3,15)
+hasLocation(#20068,#20069)
+#20070=*
+tokeninfo(#20070,1,#20001,30,"null")
+#20071=@"loc,{#10000},3,17,3,20"
+locations_default(#20071,#10000,3,17,3,20)
+hasLocation(#20070,#20071)
+#20072=*
+tokeninfo(#20072,8,#20001,31,"|")
+#20073=@"loc,{#10000},3,22,3,22"
+locations_default(#20073,#10000,3,22,3,22)
+hasLocation(#20072,#20073)
+#20074=*
+tokeninfo(#20074,6,#20001,32,"T")
+#20075=@"loc,{#10000},3,24,3,24"
+locations_default(#20075,#10000,3,24,3,24)
+hasLocation(#20074,#20075)
+#20076=*
+tokeninfo(#20076,8,#20001,33,",")
+#20077=@"loc,{#10000},3,25,3,25"
+locations_default(#20077,#10000,3,25,3,25)
+hasLocation(#20076,#20077)
+#20078=*
+tokeninfo(#20078,8,#20001,34,"...")
+#20079=@"loc,{#10000},3,27,3,29"
+locations_default(#20079,#10000,3,27,3,29)
+hasLocation(#20078,#20079)
+#20080=*
+tokeninfo(#20080,8,#20001,35,"}")
+#20081=@"loc,{#10000},3,30,3,30"
+locations_default(#20081,#10000,3,30,3,30)
+hasLocation(#20080,#20081)
+#20082=*
+tokeninfo(#20082,8,#20001,36,"|")
+#20083=@"loc,{#10000},3,32,3,32"
+locations_default(#20083,#10000,3,32,3,32)
+hasLocation(#20082,#20083)
+#20084=*
+tokeninfo(#20084,8,#20001,37,"(")
+#20085=@"loc,{#10000},3,34,3,34"
+locations_default(#20085,#10000,3,34,3,34)
+hasLocation(#20084,#20085)
+#20086=*
+tokeninfo(#20086,8,#20001,38,"(")
+#20087=@"loc,{#10000},3,35,3,35"
+locations_default(#20087,#10000,3,35,3,35)
+hasLocation(#20086,#20087)
+#20088=*
+tokeninfo(#20088,6,#20001,39,"r")
+#20089=@"loc,{#10000},3,36,3,36"
+locations_default(#20089,#10000,3,36,3,36)
+hasLocation(#20088,#20089)
+#20090=*
+tokeninfo(#20090,8,#20001,40,":")
+#20091=@"loc,{#10000},3,37,3,37"
+locations_default(#20091,#10000,3,37,3,37)
+hasLocation(#20090,#20091)
+#20092=*
+tokeninfo(#20092,1,#20001,41,"null")
+#20093=@"loc,{#10000},3,39,3,42"
+locations_default(#20093,#10000,3,39,3,42)
+hasLocation(#20092,#20093)
+#20094=*
+tokeninfo(#20094,8,#20001,42,"|")
+#20095=@"loc,{#10000},3,44,3,44"
+locations_default(#20095,#10000,3,44,3,44)
+hasLocation(#20094,#20095)
+#20096=*
+tokeninfo(#20096,6,#20001,43,"T")
+#20097=@"loc,{#10000},3,46,3,46"
+locations_default(#20097,#10000,3,46,3,46)
+hasLocation(#20096,#20097)
+#20098=*
+tokeninfo(#20098,8,#20001,44,")")
+#20099=@"loc,{#10000},3,47,3,47"
+locations_default(#20099,#10000,3,47,3,47)
+hasLocation(#20098,#20099)
+#20100=*
+tokeninfo(#20100,8,#20001,45,"=>")
+#20101=@"loc,{#10000},3,49,3,50"
+locations_default(#20101,#10000,3,49,3,50)
+hasLocation(#20100,#20101)
+#20102=*
+tokeninfo(#20102,6,#20001,46,"mixed")
+#20103=@"loc,{#10000},3,52,3,56"
+locations_default(#20103,#10000,3,52,3,56)
+hasLocation(#20102,#20103)
+#20104=*
+tokeninfo(#20104,8,#20001,47,")")
+#20105=@"loc,{#10000},3,57,3,57"
+locations_default(#20105,#10000,3,57,3,57)
+hasLocation(#20104,#20105)
+#20106=*
+tokeninfo(#20106,8,#20001,48,";")
+#20107=@"loc,{#10000},3,58,3,58"
+locations_default(#20107,#10000,3,58,3,58)
+hasLocation(#20106,#20107)
+#20108=*
+tokeninfo(#20108,6,#20001,49,"type")
+#20109=@"loc,{#10000},4,1,4,4"
+locations_default(#20109,#10000,4,1,4,4)
+hasLocation(#20108,#20109)
+#20110=*
+tokeninfo(#20110,6,#20001,50,"D")
+#20111=@"loc,{#10000},4,6,4,6"
+locations_default(#20111,#10000,4,6,4,6)
+hasLocation(#20110,#20111)
+#20112=*
+tokeninfo(#20112,8,#20001,51,"=")
+#20113=@"loc,{#10000},4,8,4,8"
+locations_default(#20113,#10000,4,8,4,8)
+hasLocation(#20112,#20113)
+#20114=*
+tokeninfo(#20114,8,#20001,52,"{")
+#20115=@"loc,{#10000},4,10,4,10"
+locations_default(#20115,#10000,4,10,4,10)
+hasLocation(#20114,#20115)
+#20116=*
+tokeninfo(#20116,6,#20001,53,"d")
+#20117=@"loc,{#10000},4,12,4,12"
+locations_default(#20117,#10000,4,12,4,12)
+hasLocation(#20116,#20117)
+#20118=*
+tokeninfo(#20118,8,#20001,54,":")
+#20119=@"loc,{#10000},4,13,4,13"
+locations_default(#20119,#10000,4,13,4,13)
+hasLocation(#20118,#20119)
+#20120=*
+tokeninfo(#20120,6,#20001,55,"D")
+#20121=@"loc,{#10000},4,15,4,15"
+locations_default(#20121,#10000,4,15,4,15)
+hasLocation(#20120,#20121)
+#20122=*
+tokeninfo(#20122,8,#20001,56,",")
+#20123=@"loc,{#10000},4,16,4,16"
+locations_default(#20123,#10000,4,16,4,16)
+hasLocation(#20122,#20123)
+#20124=*
+tokeninfo(#20124,8,#20001,57,"...")
+#20125=@"loc,{#10000},4,18,4,20"
+locations_default(#20125,#10000,4,18,4,20)
+hasLocation(#20124,#20125)
+#20126=*
+tokeninfo(#20126,8,#20001,58,",")
+#20127=@"loc,{#10000},4,21,4,21"
+locations_default(#20127,#10000,4,21,4,21)
+hasLocation(#20126,#20127)
+#20128=*
+tokeninfo(#20128,8,#20001,59,"}")
+#20129=@"loc,{#10000},4,23,4,23"
+locations_default(#20129,#10000,4,23,4,23)
+hasLocation(#20128,#20129)
+#20130=*
+tokeninfo(#20130,8,#20001,60,";")
+#20131=@"loc,{#10000},4,24,4,24"
+locations_default(#20131,#10000,4,24,4,24)
+hasLocation(#20130,#20131)
+#20132=*
+tokeninfo(#20132,0,#20001,61,"")
+#20133=@"loc,{#10000},4,25,4,24"
+locations_default(#20133,#10000,4,25,4,24)
+hasLocation(#20132,#20133)
 toplevels(#20001,0)
-#20050=@"loc,{#10000},1,1,2,29"
+#20134=@"loc,{#10000},1,1,4,24"
-locations_default(#20050,#10000,1,1,2,29)
+locations_default(#20134,#10000,1,1,4,24)
-hasLocation(#20001,#20050)
+hasLocation(#20001,#20134)
-#20051=*
+#20135=*
-entry_cfg_node(#20051,#20001)
+entry_cfg_node(#20135,#20001)
-#20052=@"loc,{#10000},1,1,1,0"
+#20136=@"loc,{#10000},1,1,1,0"
-locations_default(#20052,#10000,1,1,1,0)
+locations_default(#20136,#10000,1,1,1,0)
-hasLocation(#20051,#20052)
+hasLocation(#20135,#20136)
-#20053=*
+#20137=*
-exit_cfg_node(#20053,#20001)
+exit_cfg_node(#20137,#20001)
-hasLocation(#20053,#20049)
+hasLocation(#20137,#20133)
-successor(#20051,#20053)
+successor(#20135,#20137)
-numlines(#10000,2,2,0)
+numlines(#10000,4,4,0)
 filetype(#10000,"javascript")

javascript/ql/src/semmle/javascript/NodeJS.qll

@@ -162,12 +162,11 @@ private predicate moduleInFile(Module m, File f) { m.getFile() = f }
  * ```
  */
 class Require extends CallExpr, Import {
+  cached
   Require() {
-    exists(RequireVariable req |
+    any(RequireVariable req).getAnAccess() = getCallee() and
-      this.getCallee() = req.getAnAccess() and
     // `mjs` files explicitly disallow `require`
     not getFile().getExtension() = "mjs"
-    )
   }
 
   override PathExpr getImportedPath() { result = getArgument(0) }

python/ql/src/Filters/NotGenerated.ql

@@ -1,7 +1,7 @@
 /**
  * @name Filter: non-generated files
  * @description Only keep results that aren't (or don't appear to be) generated.
- * @kind file-classifier
+ * @kind problem
  * @id py/not-generated-file-filter
  */
 import python

python/ql/src/Filters/NotTest.ql

@@ -1,7 +1,7 @@
 /**
  * @name Filter: non-test files
  * @description Only keep results that aren't in tests
- * @kind file-classifier
+ * @kind problem
  * @id py/not-test-file-filter
  */
 import python