Merge pull request #16926 from github/post-release-prep/codeql-cli-2.18.0

Post-release preparation for codeql-cli-2.18.0

.github/workflows/compile-queries.yml

@@ -29,6 +29,8 @@ jobs:
           key: all-queries
       - name: check formatting
         run: find shared */ql -type f \( -name "*.qll" -o -name "*.ql" \) -print0 | xargs -0 -n 3000 -P 10 codeql query format -q --check-only
+      - name: Omit DatabaseQualityDiagnostics.ql from compile checking # Remove me once CodeQL 2.18.0 is released!
+        run: mv java/ql/src/Telemetry/DatabaseQualityDiagnostics.ql{,.hidden}
       - name: compile queries - check-only
         # run with --check-only if running in a PR (github.sha != main)
         if : ${{ github.event_name == 'pull_request' }}
@@ -39,3 +41,6 @@ jobs:
         if : ${{ github.event_name != 'pull_request' }}
         shell: bash
         run: codeql query compile -q -j0 */ql/{src,examples} --keep-going --warnings=error --compilation-cache "${{ steps.query-cache.outputs.cache-dir }}" --compilation-cache-size=500
+      - name: Restore DatabaseQualityDiagnostics.ql after compile checking # Remove me once CodeQL 2.18.0 is released
+        run: mv java/ql/src/Telemetry/DatabaseQualityDiagnostics.ql{.hidden,}
+

cpp/ql/lib/CHANGELOG.md

@@ -1,3 +1,15 @@
+## 1.2.0
+
+### New Features
+
+* The syntax for models-as-data rows has been extended to make it easier to select sources, sinks, and summaries that involve templated functions and classes. Additionally, the syntax has also been extended to make it easier to specify models with arbitrary levels of indirection. See `dataflow/ExternalFlow.qll` for the updated documentation and specification for the model format.
+* It is now possible to extend the classes `AllocationFunction` and `DeallocationFunction` via data extensions. Extensions of these classes should be added to the `lib/ext/allocation` and `lib/ext/deallocation` directories respectively.
+
+### Minor Analysis Improvements
+
+* The queries "Potential double free" (`cpp/double-free`) and "Potential use after free" (`cpp/use-after-free`) now produce fewer false positives.
+* The "Guards" library (`semmle.code.cpp.controlflow.Guards`) now also infers guards from calls to the builtin operation `__builtin_expect`. As a result, some queries may produce fewer false positives.
+
 ## 1.1.1
 
 No user-facing changes.

cpp/ql/lib/change-notes/2024-06-10-builtin-expect.md

@@ -1,4 +0,0 @@
----
-category: minorAnalysis
----
-* The "Guards" library (`semmle.code.cpp.controlflow.Guards`) now also infers guards from calls to the builtin operation `__builtin_expect`. As a result, some queries may produce fewer false positives.

cpp/ql/lib/change-notes/2024-06-13-double-free.md

@@ -1,4 +0,0 @@
----
-category: minorAnalysis
----
-* The queries "Potential double free" (`cpp/double-free`) and "Potential use after free" (`cpp/use-after-free`) now produce fewer false positives.

cpp/ql/lib/change-notes/2024-06-20-extensible-allocation-deallocation.md

@@ -1,4 +0,0 @@
----
-category: feature
----
-* It is now possible to extend the classes `AllocationFunction` and `DeallocationFunction` via data extensions. Extensions of these classes should be added to the `lib/ext/allocation` and `lib/ext/deallocation` directories respectively.

cpp/ql/lib/change-notes/2024-07-03-extended-mad-syntax.md

@@ -1,4 +0,0 @@
----
-category: feature
----
-* The syntax for models-as-data rows has been extended to make it easier to select sources, sinks, and summaries that involve templated functions and classes. Additionally, the syntax has also been extended to make it easier to specify models with arbitrary levels of indirection. See `dataflow/ExternalFlow.qll` for the updated documentation and specification for the model format.

cpp/ql/lib/change-notes/released/1.2.0.md

@@ -0,0 +1,11 @@
+## 1.2.0
+
+### New Features
+
+* The syntax for models-as-data rows has been extended to make it easier to select sources, sinks, and summaries that involve templated functions and classes. Additionally, the syntax has also been extended to make it easier to specify models with arbitrary levels of indirection. See `dataflow/ExternalFlow.qll` for the updated documentation and specification for the model format.
+* It is now possible to extend the classes `AllocationFunction` and `DeallocationFunction` via data extensions. Extensions of these classes should be added to the `lib/ext/allocation` and `lib/ext/deallocation` directories respectively.
+
+### Minor Analysis Improvements
+
+* The queries "Potential double free" (`cpp/double-free`) and "Potential use after free" (`cpp/use-after-free`) now produce fewer false positives.
+* The "Guards" library (`semmle.code.cpp.controlflow.Guards`) now also infers guards from calls to the builtin operation `__builtin_expect`. As a result, some queries may produce fewer false positives.

cpp/ql/lib/codeql-pack.release.yml

@@ -1,2 +1,2 @@
 ---
-lastReleaseVersion: 1.1.1
+lastReleaseVersion: 1.2.0

cpp/ql/lib/qlpack.yml

@@ -1,5 +1,5 @@
 name: codeql/cpp-all
-version: 1.1.2-dev
+version: 1.2.1-dev
 groups: cpp
 dbscheme: semmlecode.cpp.dbscheme
 extractor: cpp

cpp/ql/src/CHANGELOG.md

@@ -1,3 +1,7 @@
+## 1.0.3
+
+No user-facing changes.
+
 ## 1.0.2
 
 No user-facing changes.

cpp/ql/src/change-notes/released/1.0.3.md

@@ -0,0 +1,3 @@
+## 1.0.3
+
+No user-facing changes.

cpp/ql/src/codeql-pack.release.yml

@@ -1,2 +1,2 @@
 ---
-lastReleaseVersion: 1.0.2
+lastReleaseVersion: 1.0.3

cpp/ql/src/qlpack.yml

@@ -1,5 +1,5 @@
 name: codeql/cpp-queries
-version: 1.0.3-dev
+version: 1.0.4-dev
 groups:
   - cpp
   - queries

csharp/ql/campaigns/Solorigate/lib/CHANGELOG.md

@@ -1,3 +1,7 @@
+## 1.7.20
+
+No user-facing changes.
+
 ## 1.7.19
 
 No user-facing changes.

csharp/ql/campaigns/Solorigate/lib/change-notes/released/1.7.20.md

@@ -0,0 +1,3 @@
+## 1.7.20
+
+No user-facing changes.

csharp/ql/campaigns/Solorigate/lib/codeql-pack.release.yml

@@ -1,2 +1,2 @@
 ---
-lastReleaseVersion: 1.7.19
+lastReleaseVersion: 1.7.20

csharp/ql/campaigns/Solorigate/lib/qlpack.yml

@@ -1,5 +1,5 @@
 name: codeql/csharp-solorigate-all
-version: 1.7.20-dev
+version: 1.7.21-dev
 groups:
   - csharp
   - solorigate

csharp/ql/campaigns/Solorigate/src/CHANGELOG.md

@@ -1,3 +1,7 @@
+## 1.7.20
+
+No user-facing changes.
+
 ## 1.7.19
 
 No user-facing changes.

csharp/ql/campaigns/Solorigate/src/change-notes/released/1.7.20.md

@@ -0,0 +1,3 @@
+## 1.7.20
+
+No user-facing changes.

csharp/ql/campaigns/Solorigate/src/codeql-pack.release.yml

@@ -1,2 +1,2 @@
 ---
-lastReleaseVersion: 1.7.19
+lastReleaseVersion: 1.7.20

csharp/ql/campaigns/Solorigate/src/qlpack.yml

@@ -1,5 +1,5 @@
 name: codeql/csharp-solorigate-queries
-version: 1.7.20-dev
+version: 1.7.21-dev
 groups:
   - csharp
   - solorigate

csharp/ql/lib/CHANGELOG.md

@@ -1,3 +1,7 @@
+## 1.0.3
+
+No user-facing changes.
+
 ## 1.0.2
 
 No user-facing changes.

csharp/ql/lib/change-notes/released/1.0.3.md

@@ -0,0 +1,3 @@
+## 1.0.3
+
+No user-facing changes.

csharp/ql/lib/codeql-pack.release.yml

@@ -1,2 +1,2 @@
 ---
-lastReleaseVersion: 1.0.2
+lastReleaseVersion: 1.0.3

csharp/ql/lib/qlpack.yml

@@ -1,5 +1,5 @@
 name: codeql/csharp-all
-version: 1.0.3-dev
+version: 1.0.4-dev
 groups: csharp
 dbscheme: semmlecode.csharp.dbscheme
 extractor: csharp

csharp/ql/src/CHANGELOG.md

@@ -1,3 +1,7 @@
+## 1.0.3
+
+No user-facing changes.
+
 ## 1.0.2
 
 No user-facing changes.

csharp/ql/src/change-notes/released/1.0.3.md

@@ -0,0 +1,3 @@
+## 1.0.3
+
+No user-facing changes.

csharp/ql/src/codeql-pack.release.yml

@@ -1,2 +1,2 @@
 ---
-lastReleaseVersion: 1.0.2
+lastReleaseVersion: 1.0.3

csharp/ql/src/qlpack.yml

@@ -1,5 +1,5 @@
 name: codeql/csharp-queries
-version: 1.0.3-dev
+version: 1.0.4-dev
 groups:
   - csharp
   - queries

go/ql/consistency-queries/CHANGELOG.md

@@ -1,3 +1,7 @@
+## 1.0.3
+
+No user-facing changes.
+
 ## 1.0.2
 
 No user-facing changes.

go/ql/consistency-queries/change-notes/released/1.0.3.md

@@ -0,0 +1,3 @@
+## 1.0.3
+
+No user-facing changes.

go/ql/consistency-queries/codeql-pack.release.yml

@@ -1,2 +1,2 @@
 ---
-lastReleaseVersion: 1.0.2
+lastReleaseVersion: 1.0.3

go/ql/consistency-queries/qlpack.yml

@@ -1,5 +1,5 @@
 name: codeql-go-consistency-queries
-version: 1.0.3-dev
+version: 1.0.4-dev
 groups:
   - go
   - queries

go/ql/lib/CHANGELOG.md

@@ -1,3 +1,14 @@
+## 1.1.2
+
+### Minor Analysis Improvements
+
+* DataFlow queries which previously used `RemoteFlowSource` to define their sources have been modified to instead use `ThreatModelFlowSource`. This means these queries will now respect threat model configurations. The default threat model configuration is equivalent to `RemoteFlowSource`, so there should be no change in results for users using the default.
+* Added the `ThreatModelFlowSource` class to `FlowSources.qll`. The `ThreatModelFlowSource` class can be used to include sources which match the current *threat model* configuration. This is the first step in supporting threat modeling for Go.
+
+### Bug Fixes
+
+* Fixed dataflow via global variables other than via a direct write: for example, via a side-effect on a global, such as `io.copy(SomeGlobal, ...)` or via assignment to a field or array or slice cell of a global. This means that any data-flow query may return more results where global variables are involved.
+
 ## 1.1.1
 
 No user-facing changes.

go/ql/lib/change-notes/2024-06-06-add-go-threatmodelflowsource.md

@@ -1,4 +0,0 @@
----
-category: minorAnalysis
----
-* Added the `ThreatModelFlowSource` class to `FlowSources.qll`. The `ThreatModelFlowSource` class can be used to include sources which match the current *threat model* configuration. This is the first step in supporting threat modeling for Go.

go/ql/lib/change-notes/2024-06-08-refactor-go-queries-to-use-threatmodelflowsource.md

@@ -1,4 +0,0 @@
----
-category: minorAnalysis
----
-* DataFlow queries which previously used `RemoteFlowSource` to define their sources have been modified to instead use `ThreatModelFlowSource`. This means these queries will now respect threat model configurations. The default threat model configuration is equivalent to `RemoteFlowSource`, so there should be no change in results for users using the default.

go/ql/lib/change-notes/2024-06-17-go-global-variable-writes.md

@@ -1,4 +0,0 @@
----
-category: fix
----
-* Fixed dataflow via global variables other than via a direct write: for example, via a side-effect on a global, such as `io.copy(SomeGlobal, ...)` or via assignment to a field or array or slice cell of a global. This means that any data-flow query may return more results where global variables are involved.

go/ql/lib/change-notes/released/1.1.2.md

@@ -0,0 +1,10 @@
+## 1.1.2
+
+### Minor Analysis Improvements
+
+* DataFlow queries which previously used `RemoteFlowSource` to define their sources have been modified to instead use `ThreatModelFlowSource`. This means these queries will now respect threat model configurations. The default threat model configuration is equivalent to `RemoteFlowSource`, so there should be no change in results for users using the default.
+* Added the `ThreatModelFlowSource` class to `FlowSources.qll`. The `ThreatModelFlowSource` class can be used to include sources which match the current *threat model* configuration. This is the first step in supporting threat modeling for Go.
+
+### Bug Fixes
+
+* Fixed dataflow via global variables other than via a direct write: for example, via a side-effect on a global, such as `io.copy(SomeGlobal, ...)` or via assignment to a field or array or slice cell of a global. This means that any data-flow query may return more results where global variables are involved.

go/ql/lib/codeql-pack.release.yml

@@ -1,2 +1,2 @@
 ---
-lastReleaseVersion: 1.1.1
+lastReleaseVersion: 1.1.2

go/ql/lib/qlpack.yml

@@ -1,5 +1,5 @@
 name: codeql/go-all
-version: 1.1.2-dev
+version: 1.1.3-dev
 groups: go
 dbscheme: go.dbscheme
 extractor: go

go/ql/src/CHANGELOG.md

@@ -1,3 +1,7 @@
+## 1.0.3
+
+No user-facing changes.
+
 ## 1.0.2
 
 No user-facing changes.

go/ql/src/change-notes/released/1.0.3.md

@@ -0,0 +1,3 @@
+## 1.0.3
+
+No user-facing changes.

go/ql/src/codeql-pack.release.yml

@@ -1,2 +1,2 @@
 ---
-lastReleaseVersion: 1.0.2
+lastReleaseVersion: 1.0.3

go/ql/src/qlpack.yml

@@ -1,5 +1,5 @@
 name: codeql/go-queries
-version: 1.0.3-dev
+version: 1.0.4-dev
 groups:
   - go
   - queries

java/ql/automodel/src/CHANGELOG.md

@@ -1,3 +1,7 @@
+## 1.0.3
+
+No user-facing changes.
+
 ## 1.0.2
 
 No user-facing changes.

java/ql/automodel/src/change-notes/released/1.0.3.md

@@ -0,0 +1,3 @@
+## 1.0.3
+
+No user-facing changes.

java/ql/automodel/src/codeql-pack.release.yml

@@ -1,2 +1,2 @@
 ---
-lastReleaseVersion: 1.0.2
+lastReleaseVersion: 1.0.3

java/ql/automodel/src/qlpack.yml

@@ -1,5 +1,5 @@
 name: codeql/java-automodel-queries
-version: 1.0.3-dev
+version: 1.0.4-dev
 groups:
     - java
     - automodel

java/ql/integration-tests/all-platforms/java/buildless-erroneous/DatabaseQualityDiagnostics.expected

@@ -0,0 +1,6 @@
+diagnosticAttributes
+| Scanning Java code completed successfully, but the scan encountered issues. This may be caused by problems identifying dependencies or use of generated source code, among other reasons -- see other CodeQL diagnostics reported on the CodeQL status page for more details of possible causes. Addressing these warnings is advisable to avoid false-positive or missing results. If they cannot be addressed, consider scanning Java using either the `autobuild` or `manual` [build modes](https://docs.github.com/en/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/codeql-code-scanning-for-compiled-languages#comparison-of-the-build-modes). | visibilityCliSummaryTable | true |
+| Scanning Java code completed successfully, but the scan encountered issues. This may be caused by problems identifying dependencies or use of generated source code, among other reasons -- see other CodeQL diagnostics reported on the CodeQL status page for more details of possible causes. Addressing these warnings is advisable to avoid false-positive or missing results. If they cannot be addressed, consider scanning Java using either the `autobuild` or `manual` [build modes](https://docs.github.com/en/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/codeql-code-scanning-for-compiled-languages#comparison-of-the-build-modes). | visibilityStatusPage | true |
+| Scanning Java code completed successfully, but the scan encountered issues. This may be caused by problems identifying dependencies or use of generated source code, among other reasons -- see other CodeQL diagnostics reported on the CodeQL status page for more details of possible causes. Addressing these warnings is advisable to avoid false-positive or missing results. If they cannot be addressed, consider scanning Java using either the `autobuild` or `manual` [build modes](https://docs.github.com/en/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/codeql-code-scanning-for-compiled-languages#comparison-of-the-build-modes). | visibilityTelemetry | true |
+#select
+| Scanning Java code completed successfully, but the scan encountered issues. This may be caused by problems identifying dependencies or use of generated source code, among other reasons -- see other CodeQL diagnostics reported on the CodeQL status page for more details of possible causes. Addressing these warnings is advisable to avoid false-positive or missing results. If they cannot be addressed, consider scanning Java using either the `autobuild` or `manual` [build modes](https://docs.github.com/en/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/codeql-code-scanning-for-compiled-languages#comparison-of-the-build-modes). | Scanning Java code completed successfully, but the scan encountered issues. This may be caused by problems identifying dependencies or use of generated source code, among other reasons -- see other CodeQL diagnostics reported on the CodeQL status page for more details of possible causes. Addressing these warnings is advisable to avoid false-positive or missing results. If they cannot be addressed, consider scanning Java using either the `autobuild` or `manual` [build modes](https://docs.github.com/en/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/codeql-code-scanning-for-compiled-languages#comparison-of-the-build-modes). | 1 |

java/ql/integration-tests/all-platforms/java/buildless-erroneous/DatabaseQualityDiagnostics.qlref

@@ -0,0 +1 @@
+Telemetry/DatabaseQualityDiagnostics.ql

java/ql/lib/CHANGELOG.md

@@ -1,3 +1,25 @@
+## 1.1.2
+
+### Minor Analysis Improvements
+
+* Added models for the following packages:
+
+  * io.undertow.server.handlers.resource
+  * jakarta.faces.context
+  * javax.faces.context
+  * javax.servlet
+  * org.jboss.vfs
+  * org.springframework.core.io
+* A bug has been fixed in the heuristic identification of uncertain control
+  flow, which is used to filter data flow in order to improve performance and
+  reduce false positives. This fix means that slightly more code is identified
+  and hence pruned from data flow.
+* Excluded reverse DNS from the loopback address as a source of untrusted data.
+
+### Bug Fixes
+
+* Support for `codeql test run` for Kotlin sources has been fixed.
+
 ## 1.1.1
 
 No user-facing changes.

java/ql/lib/change-notes/2024-06-13-kotlin-qltest-support.md

@@ -1,5 +0,0 @@
----
-category: fix
----
-
-* Support for `codeql test run` for Kotlin sources has been fixed.

java/ql/lib/change-notes/2024-06-14-exclude-loopback-from-reverse-dns.md

@@ -1,4 +0,0 @@
----
-category: minorAnalysis
----
-* Excluded reverse DNS from the loopback address as a source of untrusted data.

java/ql/lib/change-notes/2024-06-17-ffbl-implicit-this.md

@@ -1,7 +0,0 @@
----
-category: minorAnalysis
----
-* A bug has been fixed in the heuristic identification of uncertain control
-  flow, which is used to filter data flow in order to improve performance and
-  reduce false positives. This fix means that slightly more code is identified
-  and hence pruned from data flow.

java/ql/lib/change-notes/2024-06-25-java-tools-status.md

@@ -0,0 +1,4 @@
+---
+category: minorAnalysis
+---
+* Java build-mode `none` analyses now only report a warning on the CodeQL status page when there are significant analysis problems-- defined as 5% of expressions lacking a type, or 5% of call targets being unknown. Other messages reported on the status page are downgraded from warnings to notes and so are less prominent, but are still available for review.

java/ql/lib/change-notes/2024-06-28-resource-models.md

@@ -1,11 +0,0 @@
----
-category: minorAnalysis
----
-* Added models for the following packages:
-
-  * io.undertow.server.handlers.resource
-  * jakarta.faces.context
-  * javax.faces.context
-  * javax.servlet
-  * org.jboss.vfs
-  * org.springframework.core.io

java/ql/lib/change-notes/released/1.1.2.md

@@ -0,0 +1,21 @@
+## 1.1.2
+
+### Minor Analysis Improvements
+
+* Added models for the following packages:
+
+  * io.undertow.server.handlers.resource
+  * jakarta.faces.context
+  * javax.faces.context
+  * javax.servlet
+  * org.jboss.vfs
+  * org.springframework.core.io
+* A bug has been fixed in the heuristic identification of uncertain control
+  flow, which is used to filter data flow in order to improve performance and
+  reduce false positives. This fix means that slightly more code is identified
+  and hence pruned from data flow.
+* Excluded reverse DNS from the loopback address as a source of untrusted data.
+
+### Bug Fixes
+
+* Support for `codeql test run` for Kotlin sources has been fixed.

java/ql/lib/codeql-pack.release.yml

@@ -1,2 +1,2 @@
 ---
-lastReleaseVersion: 1.1.1
+lastReleaseVersion: 1.1.2

java/ql/lib/qlpack.yml

@@ -1,5 +1,5 @@
 name: codeql/java-all
-version: 1.1.2-dev
+version: 1.1.3-dev
 groups: java
 dbscheme: config/semmlecode.dbscheme
 extractor: java

java/ql/src/CHANGELOG.md

@@ -1,3 +1,14 @@
+## 1.1.0
+
+### Major Analysis Improvements
+
+* The query `java/weak-cryptographic-algorithm` no longer alerts about `RSA/ECB` algorithm strings.
+
+### Minor Analysis Improvements
+
+* The query `java/tainted-permissions-check` now uses threat models. This means that `local` sources are no longer included by default for this query, but can be added by enabling the `local` threat model.
+* Added more `org.apache.commons.io.FileUtils`-related sinks to the path injection query.
+
 ## 1.0.2
 
 No user-facing changes.

java/ql/src/Telemetry/DatabaseQuality.qll

@@ -0,0 +1,65 @@
+/**
+ * Provides database quality statistics that are reported by java/telemetry/extractor-information
+ * and perhaps warned about by java/diagnostics/database-quality.
+ */
+
+import java
+
+signature module StatsSig {
+  int getNumberOfOk();
+
+  int getNumberOfNotOk();
+
+  string getOkText();
+
+  string getNotOkText();
+}
+
+module ReportStats<StatsSig Stats> {
+  predicate numberOfOk(string key, int value) {
+    value = Stats::getNumberOfOk() and
+    key = "Number of " + Stats::getOkText()
+  }
+
+  predicate numberOfNotOk(string key, int value) {
+    value = Stats::getNumberOfNotOk() and
+    key = "Number of " + Stats::getNotOkText()
+  }
+
+  predicate percentageOfOk(string key, float value) {
+    value = Stats::getNumberOfOk() * 100.0 / (Stats::getNumberOfOk() + Stats::getNumberOfNotOk()) and
+    key = "Percentage of " + Stats::getOkText()
+  }
+}
+
+module CallTargetStats implements StatsSig {
+  int getNumberOfOk() { result = count(Call c | exists(c.getCallee())) }
+
+  int getNumberOfNotOk() { result = count(Call c | not exists(c.getCallee())) }
+
+  string getOkText() { result = "calls with call target" }
+
+  string getNotOkText() { result = "calls with missing call target" }
+}
+
+private class SourceExpr extends Expr {
+  SourceExpr() { this.getFile().isSourceFile() }
+}
+
+private predicate hasGoodType(Expr e) {
+  exists(e.getType()) and not e.getType() instanceof ErrorType
+}
+
+module ExprTypeStats implements StatsSig {
+  int getNumberOfOk() { result = count(SourceExpr e | hasGoodType(e)) }
+
+  int getNumberOfNotOk() { result = count(SourceExpr e | not hasGoodType(e)) }
+
+  string getOkText() { result = "expressions with known type" }
+
+  string getNotOkText() { result = "expressions with unknown type" }
+}
+
+module CallTargetStatsReport = ReportStats<CallTargetStats>;
+
+module ExprTypeStatsReport = ReportStats<ExprTypeStats>;

java/ql/src/Telemetry/DatabaseQualityDiagnostics.ql

@@ -0,0 +1,44 @@
+/**
+ * @name Low Java analysis quality
+ * @description Low Java analysis quality
+ * @kind diagnostic
+ * @id java/diagnostic/database-quality
+ */
+
+import java
+import DatabaseQuality
+
+private newtype TDbQualityDiagnostic =
+  TTheDbQualityDiagnostic() {
+    exists(float percentageGood |
+      CallTargetStatsReport::percentageOfOk(_, percentageGood)
+      or
+      ExprTypeStatsReport::percentageOfOk(_, percentageGood)
+    |
+      percentageGood < 95
+    )
+  }
+
+class DbQualityDiagnostic extends TDbQualityDiagnostic {
+  string toString() {
+    result =
+      "Scanning Java code completed successfully, but the scan encountered issues. " +
+        "This may be caused by problems identifying dependencies or use of generated source code, among other reasons -- "
+        +
+        "see other CodeQL diagnostics reported on the CodeQL status page for more details of possible causes. "
+        +
+        "Addressing these warnings is advisable to avoid false-positive or missing results. If they cannot be addressed, consider scanning Java "
+        +
+        "using either the `autobuild` or `manual` [build modes](https://docs.github.com/en/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/codeql-code-scanning-for-compiled-languages#comparison-of-the-build-modes)."
+  }
+}
+
+query predicate diagnosticAttributes(DbQualityDiagnostic e, string key, string value) {
+  e = e and // Quieten warning about unconstrained 'e'
+  key = ["visibilityCliSummaryTable", "visibilityTelemetry", "visibilityStatusPage"] and
+  value = "true"
+}
+
+from DbQualityDiagnostic d
+select d, d.toString(), 1
+/* Warning severity */

java/ql/src/Telemetry/ExtractorInformation.ql

@@ -8,6 +8,7 @@
 
 import java
 import semmle.code.java.Diagnostics
+import DatabaseQuality
 
 extensible predicate extractorInformationSkipKey(string key);
 
@@ -85,65 +86,6 @@ predicate extractorTotalDiagnostics(string key, int value) {
   )
 }
 
-signature module StatsSig {
-  int getNumberOfOk();
-
-  int getNumberOfNotOk();
-
-  string getOkText();
-
-  string getNotOkText();
-}
-
-module ReportStats<StatsSig Stats> {
-  predicate numberOfOk(string key, int value) {
-    value = Stats::getNumberOfOk() and
-    key = "Number of " + Stats::getOkText()
-  }
-
-  predicate numberOfNotOk(string key, int value) {
-    value = Stats::getNumberOfNotOk() and
-    key = "Number of " + Stats::getNotOkText()
-  }
-
-  predicate percentageOfOk(string key, float value) {
-    value = Stats::getNumberOfOk() * 100.0 / (Stats::getNumberOfOk() + Stats::getNumberOfNotOk()) and
-    key = "Percentage of " + Stats::getOkText()
-  }
-}
-
-module CallTargetStats implements StatsSig {
-  int getNumberOfOk() { result = count(Call c | exists(c.getCallee())) }
-
-  int getNumberOfNotOk() { result = count(Call c | not exists(c.getCallee())) }
-
-  string getOkText() { result = "calls with call target" }
-
-  string getNotOkText() { result = "calls with missing call target" }
-}
-
-private class SourceExpr extends Expr {
-  SourceExpr() { this.getFile().isSourceFile() }
-}
-
-private predicate hasGoodType(Expr e) {
-  exists(e.getType()) and not e.getType() instanceof ErrorType
-}
-
-module ExprTypeStats implements StatsSig {
-  int getNumberOfOk() { result = count(SourceExpr e | hasGoodType(e)) }
-
-  int getNumberOfNotOk() { result = count(SourceExpr e | not hasGoodType(e)) }
-
-  string getOkText() { result = "expressions with known type" }
-
-  string getNotOkText() { result = "expressions with unknown type" }
-}
-
-module CallTargetStatsReport = ReportStats<CallTargetStats>;
-
-module ExprTypeStatsReport = ReportStats<ExprTypeStats>;
-
 from string key, int value
 where
   not exists(string pattern | extractorInformationSkipKey(pattern) and key.matches(pattern)) and

java/ql/src/change-notes/2024-05-13-rsa-ecb-secure.md

@@ -1,4 +0,0 @@
----
-category: majorAnalysis
----
-* The query `java/weak-cryptographic-algorithm` no longer alerts about `RSA/ECB` algorithm strings.

java/ql/src/change-notes/2024-06-10-path-injection-fileutils-sinks.md

@@ -1,4 +0,0 @@
----
-category: minorAnalysis
----
-* Added more `org.apache.commons.io.FileUtils`-related sinks to the path injection query.

java/ql/src/change-notes/2024-06-17-tainted-permissions-check.md

@@ -1,4 +0,0 @@
----
-category: minorAnalysis
----
-* The query `java/tainted-permissions-check` now uses threat models. This means that `local` sources are no longer included by default for this query, but can be added by enabling the `local` threat model.

java/ql/src/change-notes/released/1.1.0.md

@@ -0,0 +1,10 @@
+## 1.1.0
+
+### Major Analysis Improvements
+
+* The query `java/weak-cryptographic-algorithm` no longer alerts about `RSA/ECB` algorithm strings.
+
+### Minor Analysis Improvements
+
+* The query `java/tainted-permissions-check` now uses threat models. This means that `local` sources are no longer included by default for this query, but can be added by enabling the `local` threat model.
+* Added more `org.apache.commons.io.FileUtils`-related sinks to the path injection query.

java/ql/src/codeql-pack.release.yml

@@ -1,2 +1,2 @@
 ---
-lastReleaseVersion: 1.0.2
+lastReleaseVersion: 1.1.0

java/ql/src/qlpack.yml

@@ -1,5 +1,5 @@
 name: codeql/java-queries
-version: 1.0.3-dev
+version: 1.1.1-dev
 groups:
   - java
   - queries

javascript/ql/lib/CHANGELOG.md

@@ -1,3 +1,14 @@
+## 1.1.0
+
+### Major Analysis Improvements
+
+* Added support for TypeScript 5.5.
+
+### Minor Analysis Improvements
+
+* Enabled type-tracking to follow content through array methods
+* Improved modeling of `Array.prototype.splice` for when it is called with more than two arguments
+
 ## 1.0.2
 
 No user-facing changes.

javascript/ql/lib/change-notes/2024-04-26-typescript-5-5.md

@@ -1,4 +0,0 @@
----
-category: majorAnalysis
----
-* Added support for TypeScript 5.5.

javascript/ql/lib/change-notes/released/1.1.0.md

@@ -1,5 +1,10 @@
----
-category: minorAnalysis
----
+## 1.1.0
+
+### Major Analysis Improvements
+
+* Added support for TypeScript 5.5.
+
+### Minor Analysis Improvements
+
 * Enabled type-tracking to follow content through array methods
 * Improved modeling of `Array.prototype.splice` for when it is called with more than two arguments

javascript/ql/lib/codeql-pack.release.yml

@@ -1,2 +1,2 @@
 ---
-lastReleaseVersion: 1.0.2
+lastReleaseVersion: 1.1.0

javascript/ql/lib/qlpack.yml

@@ -1,5 +1,5 @@
 name: codeql/javascript-all
-version: 1.0.3-dev
+version: 1.1.1-dev
 groups: javascript
 dbscheme: semmlecode.javascript.dbscheme
 extractor: javascript

javascript/ql/src/CHANGELOG.md

@@ -1,3 +1,9 @@
+## 1.0.3
+
+### Minor Analysis Improvements
+
+* Added a new experimental query, `js/cors-misconfiguration`, which detects misconfigured CORS HTTP headers in the `cors` and `apollo` libraries. 
+
 ## 1.0.2
 
 No user-facing changes.

javascript/ql/src/change-notes/released/1.0.3.md

@@ -1,4 +1,5 @@
----
-category: minorAnalysis
----
-* Added a new experimental query, `js/cors-misconfiguration`, which detects misconfigured CORS HTTP headers in the `cors` and `apollo` libraries. 
+## 1.0.3
+
+### Minor Analysis Improvements
+
+* Added a new experimental query, `js/cors-misconfiguration`, which detects misconfigured CORS HTTP headers in the `cors` and `apollo` libraries. 

javascript/ql/src/codeql-pack.release.yml

@@ -1,2 +1,2 @@
 ---
-lastReleaseVersion: 1.0.2
+lastReleaseVersion: 1.0.3

javascript/ql/src/qlpack.yml

@@ -1,5 +1,5 @@
 name: codeql/javascript-queries
-version: 1.0.3-dev
+version: 1.0.4-dev
 groups:
   - javascript
   - queries

misc/suite-helpers/CHANGELOG.md

@@ -1,3 +1,7 @@
+## 1.0.3
+
+No user-facing changes.
+
 ## 1.0.2
 
 No user-facing changes.

misc/suite-helpers/change-notes/released/1.0.3.md

@@ -0,0 +1,3 @@
+## 1.0.3
+
+No user-facing changes.

misc/suite-helpers/codeql-pack.release.yml

@@ -1,2 +1,2 @@
 ---
-lastReleaseVersion: 1.0.2
+lastReleaseVersion: 1.0.3

misc/suite-helpers/qlpack.yml

@@ -1,4 +1,4 @@
 name: codeql/suite-helpers
-version: 1.0.3-dev
+version: 1.0.4-dev
 groups: shared
 warnOnImplicitThis: true

python/ql/lib/CHANGELOG.md

@@ -1,3 +1,9 @@
+## 1.0.3
+
+### Minor Analysis Improvements
+
+* A number of Python queries now support sinks defined using data extensions. The format of data extensions for Python has been documented.
+
 ## 1.0.2
 
 No user-facing changes.

python/ql/lib/change-notes/released/1.0.3.md

@@ -1,4 +1,5 @@
----
-category: minorAnalysis
----
-* A number of Python queries now support sinks defined using data extensions. The format of data extensions for Python has been documented.
+## 1.0.3
+
+### Minor Analysis Improvements
+
+* A number of Python queries now support sinks defined using data extensions. The format of data extensions for Python has been documented.

python/ql/lib/codeql-pack.release.yml

@@ -1,2 +1,2 @@
 ---
-lastReleaseVersion: 1.0.2
+lastReleaseVersion: 1.0.3

python/ql/lib/qlpack.yml

@@ -1,5 +1,5 @@
 name: codeql/python-all
-version: 1.0.3-dev
+version: 1.0.4-dev
 groups: python
 dbscheme: semmlecode.python.dbscheme
 extractor: python

python/ql/src/CHANGELOG.md

@@ -1,3 +1,10 @@
+## 1.0.3
+
+### Minor Analysis Improvements
+
+* Adding Python support for Hardcoded Credentials as Models as Data
+* Additional sanitizers have been added to the `py/full-ssrf` and `py/partial-ssrf` queries for methods that verify a string contains only a certain set of characters, such as `.isalnum()` as well as regular expression tests.
+
 ## 1.0.2
 
 No user-facing changes.

python/ql/src/change-notes/2024-06-28-cred-hardcoded.md

@@ -1,4 +0,0 @@
----
-category: minorAnalysis
----
-* Adding Python support for Hardcoded Credentials as Models as Data

python/ql/src/change-notes/released/1.0.3.md

@@ -1,4 +1,6 @@
----
-category: minorAnalysis
----
-* Additional sanitizers have been added to the `py/full-ssrf` and `py/partial-ssrf` queries for methods that verify a string contains only a certain set of characters, such as `.isalnum()` as well as regular expression tests.
+## 1.0.3
+
+### Minor Analysis Improvements
+
+* Adding Python support for Hardcoded Credentials as Models as Data
+* Additional sanitizers have been added to the `py/full-ssrf` and `py/partial-ssrf` queries for methods that verify a string contains only a certain set of characters, such as `.isalnum()` as well as regular expression tests.

python/ql/src/codeql-pack.release.yml

@@ -1,2 +1,2 @@
 ---
-lastReleaseVersion: 1.0.2
+lastReleaseVersion: 1.0.3

python/ql/src/qlpack.yml

@@ -1,5 +1,5 @@
 name: codeql/python-queries
-version: 1.0.3-dev
+version: 1.0.4-dev
 groups:
   - python
   - queries

ruby/ql/lib/CHANGELOG.md

@@ -1,3 +1,10 @@
+## 1.0.3
+
+### Minor Analysis Improvements
+
+* Element references with blocks, such as `foo[:bar] { |x| puts x}`, are now parsed correctly.
+* The `CleartextSources.qll` library, used by `rb/clear-text-logging-sensitive-data` and `rb/clear-text-logging-sensitive-data`, has been updated to consider heuristics for additional categories of sensitive data. 
+
 ## 1.0.2
 
 No user-facing changes.

ruby/ql/lib/change-notes/2024-06-25-element-reference-block.md

@@ -1,4 +0,0 @@
----
-category: minorAnalysis
----
-* Element references with blocks, such as `foo[:bar] { |x| puts x}`, are now parsed correctly.

ruby/ql/lib/change-notes/released/1.0.3.md

@@ -1,4 +1,6 @@
----
-category: minorAnalysis
----
-* The `CleartextSources.qll` library, used by `rb/clear-text-logging-sensitive-data` and `rb/clear-text-logging-sensitive-data`, has been updated to consider heuristics for additional categories of sensitive data. 
+## 1.0.3
+
+### Minor Analysis Improvements
+
+* Element references with blocks, such as `foo[:bar] { |x| puts x}`, are now parsed correctly.
+* The `CleartextSources.qll` library, used by `rb/clear-text-logging-sensitive-data` and `rb/clear-text-logging-sensitive-data`, has been updated to consider heuristics for additional categories of sensitive data. 

ruby/ql/lib/codeql-pack.release.yml

@@ -1,2 +1,2 @@
 ---
-lastReleaseVersion: 1.0.2
+lastReleaseVersion: 1.0.3

ruby/ql/lib/qlpack.yml

@@ -1,5 +1,5 @@
 name: codeql/ruby-all
-version: 1.0.3-dev
+version: 1.0.4-dev
 groups: ruby
 extractor: ruby
 dbscheme: ruby.dbscheme

ruby/ql/src/CHANGELOG.md

@@ -1,3 +1,7 @@
+## 1.0.3
+
+No user-facing changes.
+
 ## 1.0.2
 
 No user-facing changes.

ruby/ql/src/change-notes/released/1.0.3.md

@@ -0,0 +1,3 @@
+## 1.0.3
+
+No user-facing changes.

ruby/ql/src/codeql-pack.release.yml

@@ -1,2 +1,2 @@
 ---
-lastReleaseVersion: 1.0.2
+lastReleaseVersion: 1.0.3

ruby/ql/src/qlpack.yml

@@ -1,5 +1,5 @@
 name: codeql/ruby-queries
-version: 1.0.3-dev
+version: 1.0.4-dev
 groups:
   - ruby
   - queries