diff --git a/change-notes/1.21/analysis-cpp.md b/change-notes/1.21/analysis-cpp.md index 39667c5e70a..efe1c3e6c40 100644 --- a/change-notes/1.21/analysis-cpp.md +++ b/change-notes/1.21/analysis-cpp.md @@ -6,37 +6,36 @@ | **Query** | **Tags** | **Purpose** | |-----------------------------|-----------|--------------------------------------------------------------------| -| `()`-declared function called with too few arguments (`cpp/too-few-arguments`) | Correctness | Find all cases where the number of arguments is less than the number of parameters of the function, provided the function is also properly declared/defined elsewhere. | -| `()`-declared function called with mismatched arguments (`cpp/mismatched-function-arguments`) | Correctness | Find all cases where the types of arguments do not match the types of parameters of the function, provided the function is also properly declared/defined elsewhere. | -| Call to alloca in a loop (`cpp/alloca-in-loop`) | reliability, correctness, external/cwe/cwe-770 | Finds calls to `alloca` in loops, which can lead to stack overflow if the number of iterations is large. Newly displayed on LGTM. | -| Use of dangerous function (`dangerous-function-overflow`) | reliability, security, external/cwe/cwe-242 | Finds calls to `gets`, which does not guard against buffer overflow. These results were previously detected by the `cpp/potentially-dangerous-function` query. | +| Call to alloca in a loop (`cpp/alloca-in-loop`) | reliability, correctness, external/cwe/cwe-770 | Finds calls to `alloca` in loops, which can lead to stack overflow if the number of iterations is large. Newly displayed [on LGTM](https://lgtm.com/rules/1508831665988/). | +| Call to function with fewer arguments than declared parameters (`cpp/too-few-arguments`) | correctness, maintainability, security | Finds all cases where the number of arguments is fewer than the number of parameters of the function, provided the function is also properly declared/defined elsewhere. Results are displayed by default [on LGTM](https://lgtm.com/rules/1508860726279/). | +| Call to a function with one or more incompatible arguments (`cpp/mistyped-function-arguments`) | correctness, maintainability | Finds all cases where the types of arguments do not match the types of parameters of the function, provided the function is also properly declared/defined elsewhere. Results are not displayed by default [on LGTM](https://lgtm.com/rules/1508849286093/). | +| Use of dangerous function (`cpp/dangerous-function-overflow`) | reliability, security, external/cwe/cwe-242 | Finds calls to `gets`, which does not guard against buffer overflow. These results were previously detected by the `cpp/potentially-dangerous-function` query. Results for both queries are displayed by default on LGTM. | ## Changes to existing queries | **Query** | **Expected impact** | **Change** | |----------------------------|------------------------|------------------------------------------------------------------| | Buffer not sufficient for string (`cpp/overflow-calculated`) | Fewer results | This query no longer reports results that would be found by the 'No space for zero terminator' (`cpp/no-space-for-terminator`) query. | -| No space for zero terminator (`cpp/no-space-for-terminator`) | More correct results | This query now detects calls to `std::malloc`. | -| Commented-out code (`cpp/commented-out-code`) | More correct results | Commented out preprocessor code is now detected by this query. | +| Call to function with extraneous arguments (`cpp/futile-params`) | Improved coverage | Query has been generalized to find all cases where the number of arguments exceeds the number of parameters of the function, provided the function is also properly declared/defined elsewhere. | +| Commented-out code (`cpp/commented-out-code`) | More correct results | Commented-out preprocessor code is now detected by this query. | +| Comparison result is always the same | Fewer false positive results | The range analysis library is now more conservative about floating point values being possibly `NaN`. | +| Constructor with default arguments will be used as a copy constructor (`cpp/constructor-used-as-copy-constructor`) | Lowered severity and precision | The severity and precision of this query have been reduced to "warning" and "low", respectively. This coding pattern is used intentionally and safely in a number of real-world projects. Results are no longer displayed on LGTM unless you choose to display them. | | Dead code due to goto or break statement (`cpp/dead-code-goto`) | Fewer false positive results | Functions containing preprocessor logic are now excluded from this analysis. | -| Mismatching new/free or malloc/delete (`cpp/new-free-mismatch`) | Fewer false positive results | Fixed an issue where functions were being identified as allocation functions inappropriately. Also affects `cpp/new-array-delete-mismatch` and `cpp/new-delete-array-mismatch`. | -| Overflow in uncontrolled allocation size (`cpp/uncontrolled-allocation-size`) | More correct results | This query has been reworked so that it can find a wider variety of results. | -| Memory may not be freed (`cpp/memory-may-not-be-freed`) | More correct results | Support added for more Microsoft-specific allocation functions, including `LocalAlloc`, `GlobalAlloc`, `HeapAlloc` and `CoTaskMemAlloc`. | | Memory is never freed (`cpp/memory-never-freed`) | More correct results | Support added for more Microsoft-specific allocation functions, including `LocalAlloc`, `GlobalAlloc`, `HeapAlloc` and `CoTaskMemAlloc`. | +| Memory may not be freed (`cpp/memory-may-not-be-freed`) | More correct results | Support added for more Microsoft-specific allocation functions, including `LocalAlloc`, `GlobalAlloc`, `HeapAlloc` and `CoTaskMemAlloc`. | +| Mismatching new/free or malloc/delete (`cpp/new-free-mismatch`) | Fewer false positive results | Fixed an issue where functions were being identified as allocation functions inappropriately. This correction also affects `cpp/new-array-delete-mismatch` and `cpp/new-delete-array-mismatch`. | +| No space for zero terminator (`cpp/no-space-for-terminator`) | More correct results | This query now detects calls to `std::malloc`. | +| Overflow in uncontrolled allocation size (`cpp/uncontrolled-allocation-size`) | More correct results | This query has been reworked so that it can find a wider variety of results. | | Resource not released in destructor (`cpp/resource-not-released-in-destructor`) | Fewer false positive results | Resource allocation and deallocation functions are now determined more accurately. | -| Comparison result is always the same | Fewer false positive results | The range analysis library is now more conservative about floating point values being possibly `NaN` | | Use of potentially dangerous function | More correct results | Calls to `localtime`, `ctime` and `asctime` are now detected by this query. | -| Wrong type of arguments to formatting function (`cpp/wrong-type-format-argument`) | More correct results and fewer false positive results | This query now more accurately identifies wide and non-wide string/character format arguments on different platforms. Platform detection has also been made more accurate for the purposes of this query. | -| Wrong type of arguments to formatting function (`cpp/wrong-type-format-argument`) | Fewer false positive results | Non-standard uses of %L are now understood. | -| `()`-declared function called with too many arguments (`cpp/futile-params`) | Improved coverage | Query has been generalized to find all cases where the number of arguments exceedes the number of parameters of the function, provided the function is also properly declared/defined elsewhere. | -| Use of potentially dangerous function (`cpp/potentially-dangerous-function`) | Fewer results | Results relating to the standard library `gets` function have been moved into a new query (`dangerous-function-overflow`). | -| Constructor with default arguments will be used as a copy constructor (`cpp/constructor-used-as-copy-constructor`) | Lowered severity and precision | The severity and precision of this query have been reduced to "warning" and "low", respectively, due to this coding pattern being used intentionally and safely in a number of real-world projects. | +| Wrong type of arguments to formatting function (`cpp/wrong-type-format-argument`) | More correct results and fewer false positive results | This query now understands non-standard uses of `%L`. In addition, it more accurately identifies wide and non-wide string/character format arguments on different platforms. | +| Use of potentially dangerous function (`cpp/potentially-dangerous-function`) | Fewer results | Results relating to the standard library `gets` function have been moved into a new query (`cpp/dangerous-function-overflow`). | ## Changes to QL libraries - The predicate `Declaration.hasGlobalName` now only holds for declarations that are not nested in a class. For example, it no longer holds for a member function `MyClass::myFunction` or a constructor `MyClass::MyClass`, whereas previously it would classify those two declarations as global names. -- In class `Declaration`, predicates `getQualifiedName/0` and `hasQualifiedName/1` are no longer recommended for finding functions by name. Instead, use `hasGlobalName/1` and the new `hasQualifiedName/2` and `hasQualifiedName/3` predicates. This improves performance and makes it more reliable to identify names involving templates and inline namespaces. -- Additional support for definition by reference has been added to the `semmle.code.cpp.dataflow.TaintTracking` library. - - The taint tracking library now includes taint-specific edges for functions modeled in `semmle.code.cpp.models.interfaces.DataFlow`. - - The taint tracking library adds flow through library functions that are modeled in `semmle.code.cpp.models.interfaces.Taint`. Queries can add subclasses of `TaintFunction` to specify additional flow. +- In class `Declaration`, predicates `getQualifiedName/0` and `hasQualifiedName/1` are no longer recommended for finding functions by name. Instead, use `hasGlobalName/1` and the new `hasQualifiedName/2` and `hasQualifiedName/3` predicates. This improves performance and identifies names involving templates and inline namespaces more reliably. +- Additional support for definition by reference has been added to the `semmle.code.cpp.dataflow.TaintTracking` library, including: + - Taint-specific edges for functions modeled in `semmle.code.cpp.models.interfaces.DataFlow`. + - Flow through library functions that are modeled in `semmle.code.cpp.models.interfaces.Taint`. Queries can add subclasses of `TaintFunction` to specify additional flow. - There is a new `FoldExpr` class, representing C++17 fold expressions. - The member predicates `DeclarationEntry.getUnspecifiedType`, `Expr.getUnspecifiedType`, and `Variable.getUnspecifiedType` have been added. These should be preferred over the existing `getUnderlyingType` predicates. \ No newline at end of file diff --git a/change-notes/1.21/analysis-csharp.md b/change-notes/1.21/analysis-csharp.md index d38316872ac..f46218c230d 100644 --- a/change-notes/1.21/analysis-csharp.md +++ b/change-notes/1.21/analysis-csharp.md @@ -1,12 +1,16 @@ # Improvements to C# analysis +## General improvements + +C# analysis now supports the extraction and analysis of many C# 8 features. For details see [Changes to code extraction](#changes-to-code-extraction) and [Changes to QL libraries](#changes-to-ql-libraries) below. + ## Changes to existing queries | **Query** | **Expected impact** | **Change** | |------------------------------|------------------------|-----------------------------------| -| Class defines a field that uses an ICryptoTransform class in a way that would be unsafe for concurrent threads (`cs/thread-unsafe-icryptotransform-field-in-class`) | Fewer false positive results | The criteria for a result has changed to include nested properties, nested fields and collections. The format of the alert message has changed to highlight the static field. | -| Constant condition (`cs/constant-condition`) | Fewer false positive results | Results have been removed where the `null` value is in a conditional expression on the left hand side of a null-coalescing expression. For example, in `(a ? b : null) ?? c`, `null` is not considered to be a constant condition. | -| Useless upcast (`cs/useless-upcast`) | Fewer false positive results | Results have been removed where the upcast is used to disambiguate the target of a constructor call. | +| Class defines a field that uses an ICryptoTransform class in a way that would be unsafe for concurrent threads (`cs/thread-unsafe-icryptotransform-field-in-class`) | Fewer false positive results | The criteria for a result has changed to include nested properties, nested fields, and collections. The format of the alert message has changed to highlight the static field. | +| Constant condition (`cs/constant-condition`) | Fewer false positive results | The query now ignores code where the `null` value is in a conditional expression on the left hand side of a null-coalescing expression. For example, in `(a ? b : null) ?? c`, `null` is not considered to be a constant condition. | +| Useless upcast (`cs/useless-upcast`) | Fewer false positive results | The query now ignores code where the upcast is used to disambiguate the target of a constructor call. | ## Changes to code extraction @@ -17,12 +21,12 @@ - `static` modifiers on local functions - Null-coalescing assignment expressions -* The `unmanaged` type parameter constraint is now extracted. +* The `unmanaged` type parameter constraint is also now extracted. ## Changes to QL libraries -* The class `Attribute` has two new predicates: `getConstructorArgument()` and `getNamedArgument()`. The first predicate returns arguments to the underlying constructor call and the latter returns named arguments for initializing fields and properties. -* The class `TypeParameterConstraints` has a new predicate `hasUnmanagedTypeConstraint()`, indicating that the type parameter has the `unmanaged` constraint. +* The class `Attribute` has two new predicates: `getConstructorArgument()` and `getNamedArgument()`. The first predicate returns arguments to the underlying constructor call and the second returns named arguments for initializing fields and properties. +* The class `TypeParameterConstraints` has a new predicate `hasUnmanagedTypeConstraint()`. This shows whether the type parameter has the `unmanaged` constraint. * The following QL classes have been added to model C# 8 features: - Class `AssignCoalesceExpr` models null-coalescing assignment, for example `x ??= y` - Class `IndexExpr` models from-end index expressions, for example `^1` @@ -37,5 +41,3 @@ - Class `Switch` models both `SwitchExpr` and `SwitchStmt` - Class `Case` models both `CaseStmt` and `SwitchCaseExpr` - Class `UsingStmt` models both `UsingBlockStmt` and `UsingDeclStmt` - -## Changes to autobuilder diff --git a/change-notes/1.21/analysis-java.md b/change-notes/1.21/analysis-java.md index 2b71148b3a4..b3ccee73c78 100644 --- a/change-notes/1.21/analysis-java.md +++ b/change-notes/1.21/analysis-java.md @@ -1,10 +1,5 @@ # Improvements to Java analysis -## New queries - -| **Query** | **Tags** | **Purpose** | -|-----------------------------|-----------|--------------------------------------------------------------------| - ## Changes to existing queries | **Query** | **Expected impact** | **Change** | @@ -19,14 +14,13 @@ `checkArgument` and `checkState` methods in `com.google.common.base.Preconditions`, the `isTrue` and `validState` methods in `org.apache.commons.lang3.Validate`, as well as any similar custom - methods. This means that more guards are recognized yielding precision - improvements in a number of queries including `java/index-out-of-bounds`, + methods. This means that more guards are recognized which improves the precision of a number of queries including `java/index-out-of-bounds`, `java/dereferenced-value-may-be-null`, and `java/useless-null-check`. * The default sanitizer in taint tracking has been made more precise. The - sanitizer works by looking for guards that inspect tainted strings, and it - used to work at the level of individual variables. This has been changed to - use the `Guards` library, such that only guarded variable accesses are - sanitized. This may give additional results in the security queries. -* Spring framework support is enhanced by taking into account additional + sanitizer works by looking for guards that inspect tainted strings. It + previously worked at the level of individual variables. Now it + uses the `Guards` library, such that only guarded variable accesses are + sanitized. This may give additional results for security queries. +* Spring framework support now takes into account additional annotations that indicate remote user input. This affects all security - queries, which may yield additional results. + queries, which may give additional results. diff --git a/change-notes/1.21/analysis-python.md b/change-notes/1.21/analysis-python.md index 5e26854ec19..487535a46ec 100644 --- a/change-notes/1.21/analysis-python.md +++ b/change-notes/1.21/analysis-python.md @@ -31,22 +31,18 @@ The increased precision in tracking of values through `*` arguments may remove f Overall the number of true positive results should increase and the number false negative results should decline. We welcome feedback on the new implementation, particularly any surprising changes in results. - ## New queries - | **Query** | **Tags** | **Purpose** | - |-----------|----------|-------------| - | Accepting unknown SSH host keys when using Paramiko (`py/paramiko-missing-host-key-validation`) | security, external/cwe/cwe-295 | Finds instances where Paramiko is configured to accept unknown host keys. Results are shown on LGTM by default. | - | Use of 'return' or 'yield' outside a function (`py/return-or-yield-outside-function`) | reliability, correctness | Finds instances where `return`, `yield`, and `yield from` are used outside a function. Results are not shown on LGTM by default. | -## Changes to existing queries - - | **Query** | **Expected impact** | **Change** | - |-----------|---------------------|------------| +| **Query** | **Tags** | **Purpose** | +|-----------|----------|-------------| +| Accepting unknown SSH host keys when using Paramiko (`py/paramiko-missing-host-key-validation`) | security, external/cwe/cwe-295 | Finds instances where Paramiko is configured to accept unknown host keys. Results are shown [on LGTM](https://lgtm.com/rules/1508297729270/) by default. | +| Use of 'return' or 'yield' outside a function (`py/return-or-yield-outside-function`) | reliability, correctness | Finds instances where `return`, `yield`, and `yield from` are used outside a function. Results are not shown on LGTM by default. | ## Changes to code extraction -* *Series of bullet points* +* String literals as expressions within literal string interpolation (f-strings) are now handled correctly. -## Changes to QL libraries +* The Python extractor now handles invalid input more robustly. In particular, it exits gracefully when: -* *Series of bullet points* + * A non-existent file or directory is specified using the `--path` option, or as a file name. + * An invalid number is specified for the `--max-procs` option. diff --git a/change-notes/support/language-support.rst b/change-notes/support/language-support.rst index 451138c10a5..d4fcfd5ae71 100644 --- a/change-notes/support/language-support.rst +++ b/change-notes/support/language-support.rst @@ -13,7 +13,9 @@ Note that where there are several versions or dialects of a language, the suppor .. container:: footnote-group - .. [1] The best results are achieved with COBOL code that stays close to the ANSI 85 standard. - .. [2] Builds that execute on Java 6 to 11 can be analyzed. The analysis understands Java 11 language features. - .. [3] JSX and Flow code, YAML, JSON, and HTML files may also be analyzed with JavaScript files. - .. [4] TypeScript analysis is performed by running the JavaScript extractor with TypeScript enabled. This is the default for LGTM. + .. [1] Support for the Arm Compiler (armcc) is preliminary. + .. [2] In addition, support is included for the preview features of C# 8.0 and .NET Core 3.0. + .. [3] The best results are achieved with COBOL code that stays close to the ANSI 85 standard. + .. [4] Builds that execute on Java 6 to 12 can be analyzed. The analysis understands Java 12 language features. + .. [5] JSX and Flow code, YAML, JSON, HTML, and XML files may also be analyzed with JavaScript files. + .. [6] TypeScript analysis is performed by running the JavaScript extractor with TypeScript enabled. This is the default for LGTM. diff --git a/change-notes/support/versions-compilers.csv b/change-notes/support/versions-compilers.csv index 348eb19ee48..df55c38f732 100644 --- a/change-notes/support/versions-compilers.csv +++ b/change-notes/support/versions-compilers.csv @@ -1,16 +1,18 @@ Language,Variants,Compilers,Extensions -C/C++,"C89, C99, C11, C++98, C++03, C++11, C++14, C++17","Clang extensions (up to Clang 6.0) +C/C++,"C89, C99, C11, C++98, C++03, C++11, C++14, C++17","Clang extensions (up to Clang 8.0) -GNU extensions (up to GCC 7.3), +GNU extensions (up to GCC 8.3), -Microsoft extensions (up to VS 2017)","``.cpp``, ``.c++``, ``.cxx``, ``.hpp``, ``.hh``, ``.h++``, ``.hxx``, ``.c``, ``.cc``, ``.h``" -C#,C# up to 7.2 together with .NET versions up to 4.7.1,"Microsoft Visual Studio up to 2017, +Microsoft extensions (up to VS 2019), -.NET Core up to 2.1","``.sln``, ``.csproj``, ``.cs``, ``.cshtml``, ``.xaml``" -COBOL,ANSI 85 or newer [1]_.,Not applicable,"``.cbl``, ``.CBL``, ``.cpy``, ``.CPY``, ``.copy``, ``.COPY``" -Java,"Java 6 to 11 [2]_.","javac (OpenJDK and Oracle JDK) +Arm Compiler 5.0 [1]_.","``.cpp``, ``.c++``, ``.cxx``, ``.hpp``, ``.hh``, ``.h++``, ``.hxx``, ``.c``, ``.cc``, ``.h``" +C#,C# up to 7.3. with .NET up to 4.8 [2]_.,"Microsoft Visual Studio up to 2019, + +.NET Core up to 2.2","``.sln``, ``.csproj``, ``.cs``, ``.cshtml``, ``.xaml``" +COBOL,ANSI 85 or newer [3]_.,Not applicable,"``.cbl``, ``.CBL``, ``.cpy``, ``.CPY``, ``.copy``, ``.COPY``" +Java,"Java 6 to 12 [4]_.","javac (OpenJDK and Oracle JDK) Eclipse compiler for Java (ECJ) batch compiler",``.java`` -JavaScript,ECMAScript 2019 or lower,Not applicable,"``.js``, ``.jsx``, ``.mjs``, ``.es``, ``.es6``, ``.htm``, ``.html``, ``.xhm``, ``.xhtml``, ``.vue``, ``.json``, ``.yaml``, ``.yml``, ``.raml`` [3]_." +JavaScript,ECMAScript 2019 or lower,Not applicable,"``.js``, ``.jsx``, ``.mjs``, ``.es``, ``.es6``, ``.htm``, ``.html``, ``.xhm``, ``.xhtml``, ``.vue``, ``.json``, ``.yaml``, ``.yml``, ``.raml``, ``.xml`` [5]_." Python,"2.7, 3.5, 3.6, 3.7",Not applicable,``.py`` -TypeScript [4]_.,"2.6-3.5",Standard TypeScript compiler,"``.ts``, ``.tsx``" +TypeScript [6]_.,"2.6-3.5",Standard TypeScript compiler,"``.ts``, ``.tsx``"