Merge pull request #13272 from github/post-release-prep/codeql-cli-2.13.3

Post-release preparation for codeql-cli-2.13.3

.github/workflows/swift.yml

@@ -16,6 +16,7 @@ on:
     branches:
       - main
       - rc/*
+      - codeql-cli-*
   push:
     paths:
       - "swift/**"
@@ -30,6 +31,7 @@ on:
     branches:
       - main
       - rc/*
+      - codeql-cli-*
 
 jobs:
   # not using a matrix as you cannot depend on a specific job in a matrix, and we want to start linux checks

cpp/ql/lib/CHANGELOG.md

@@ -1,3 +1,19 @@
+## 0.7.2
+
+### New Features
+
+* Added an AST-based interface (`semmle.code.cpp.rangeanalysis.new.RangeAnalysis`) for the relative range analysis library.
+* A new predicate `BarrierGuard::getAnIndirectBarrierNode` has been added to the new dataflow library (`semmle.code.cpp.dataflow.new.DataFlow`) to mark indirect expressions as barrier nodes using the `BarrierGuard` API.
+
+### Major Analysis Improvements
+
+* In the intermediate representation, handling of control flow after non-returning calls has been improved. This should remove false positives in queries that use the intermedite representation or libraries based on it, including the new data flow library.
+
+### Minor Analysis Improvements
+
+* The `StdNamespace` class now also includes all inline namespaces that are children of `std` namespace.
+* The new dataflow (`semmle.code.cpp.dataflow.new.DataFlow`) and taint-tracking libraries (`semmle.code.cpp.dataflow.new.TaintTracking`) now support tracking flow through static local variables.
+
 ## 0.7.1
 
 No user-facing changes.

cpp/ql/lib/change-notes/2023-04-28-indirect-barrier-node.md

@@ -1,4 +0,0 @@
----
-category: feature
----
-* A new predicate `BarrierGuard::getAnIndirectBarrierNode` has been added to the new dataflow library (`semmle.code.cpp.dataflow.new.DataFlow`) to mark indirect expressions as barrier nodes using the `BarrierGuard` API.

cpp/ql/lib/change-notes/2023-04-28-static-local-dataflow.md

@@ -1,4 +0,0 @@
----
-category: minorAnalysis
----
-* The new dataflow (`semmle.code.cpp.dataflow.new.DataFlow`) and taint-tracking libraries (`semmle.code.cpp.dataflow.new.TaintTracking`) now support tracking flow through static local variables.

cpp/ql/lib/change-notes/2023-05-02-ir-noreturn-calls.md

@@ -1,4 +0,0 @@
----
-category: majorAnalysis
----
-* In the intermediate representation, handling of control flow after non-returning calls has been improved. This should remove false positives in queries that use the intermedite representation or libraries based on it, including the new data flow library.

cpp/ql/lib/change-notes/2023-05-02-range-analysis-wrapper.md

@@ -1,4 +0,0 @@
----
-category: feature
----
-* Added an AST-based interface (`semmle.code.cpp.rangeanalysis.new.RangeAnalysis`) for the relative range analysis library.

cpp/ql/lib/change-notes/2023-05-22-inline-in-std-namespace.md

@@ -1,4 +0,0 @@
----
-category: minorAnalysis
----
-* The `StdNamespace` class now also includes all inline namespaces that are children of `std` namespace.

cpp/ql/lib/change-notes/released/0.7.2.md

@@ -0,0 +1,15 @@
+## 0.7.2
+
+### New Features
+
+* Added an AST-based interface (`semmle.code.cpp.rangeanalysis.new.RangeAnalysis`) for the relative range analysis library.
+* A new predicate `BarrierGuard::getAnIndirectBarrierNode` has been added to the new dataflow library (`semmle.code.cpp.dataflow.new.DataFlow`) to mark indirect expressions as barrier nodes using the `BarrierGuard` API.
+
+### Major Analysis Improvements
+
+* In the intermediate representation, handling of control flow after non-returning calls has been improved. This should remove false positives in queries that use the intermedite representation or libraries based on it, including the new data flow library.
+
+### Minor Analysis Improvements
+
+* The `StdNamespace` class now also includes all inline namespaces that are children of `std` namespace.
+* The new dataflow (`semmle.code.cpp.dataflow.new.DataFlow`) and taint-tracking libraries (`semmle.code.cpp.dataflow.new.TaintTracking`) now support tracking flow through static local variables.

cpp/ql/lib/codeql-pack.release.yml

@@ -1,2 +1,2 @@
 ---
-lastReleaseVersion: 0.7.1
+lastReleaseVersion: 0.7.2

cpp/ql/lib/qlpack.yml

@@ -1,5 +1,5 @@
 name: codeql/cpp-all
-version: 0.7.2-dev
+version: 0.7.3-dev
 groups: cpp
 dbscheme: semmlecode.cpp.dbscheme
 extractor: cpp

cpp/ql/src/CHANGELOG.md

@@ -1,3 +1,7 @@
+## 0.6.2
+
+No user-facing changes.
+
 ## 0.6.1
 
 ### New Queries

cpp/ql/src/change-notes/released/0.6.2.md

@@ -0,0 +1,3 @@
+## 0.6.2
+
+No user-facing changes.

cpp/ql/src/codeql-pack.release.yml

@@ -1,2 +1,2 @@
 ---
-lastReleaseVersion: 0.6.1
+lastReleaseVersion: 0.6.2

cpp/ql/src/qlpack.yml

@@ -1,5 +1,5 @@
 name: codeql/cpp-queries
-version: 0.6.2-dev
+version: 0.6.3-dev
 groups: 
   - cpp
   - queries

csharp/ql/campaigns/Solorigate/lib/CHANGELOG.md

@@ -1,3 +1,7 @@
+## 1.5.2
+
+No user-facing changes.
+
 ## 1.5.1
 
 No user-facing changes.

csharp/ql/campaigns/Solorigate/lib/change-notes/released/1.5.2.md

@@ -0,0 +1,3 @@
+## 1.5.2
+
+No user-facing changes.

csharp/ql/campaigns/Solorigate/lib/codeql-pack.release.yml

@@ -1,2 +1,2 @@
 ---
-lastReleaseVersion: 1.5.1
+lastReleaseVersion: 1.5.2

csharp/ql/campaigns/Solorigate/lib/qlpack.yml

@@ -1,5 +1,5 @@
 name: codeql/csharp-solorigate-all
-version: 1.5.2-dev
+version: 1.5.3-dev
 groups:
     - csharp
     - solorigate

csharp/ql/campaigns/Solorigate/src/CHANGELOG.md

@@ -1,3 +1,7 @@
+## 1.5.2
+
+No user-facing changes.
+
 ## 1.5.1
 
 No user-facing changes.

csharp/ql/campaigns/Solorigate/src/change-notes/released/1.5.2.md

@@ -0,0 +1,3 @@
+## 1.5.2
+
+No user-facing changes.

csharp/ql/campaigns/Solorigate/src/codeql-pack.release.yml

@@ -1,2 +1,2 @@
 ---
-lastReleaseVersion: 1.5.1
+lastReleaseVersion: 1.5.2

csharp/ql/campaigns/Solorigate/src/qlpack.yml

@@ -1,5 +1,5 @@
 name: codeql/csharp-solorigate-queries
-version: 1.5.2-dev
+version: 1.5.3-dev
 groups:
     - csharp
     - solorigate

csharp/ql/lib/CHANGELOG.md

@@ -1,3 +1,10 @@
+## 0.6.2
+
+### Minor Analysis Improvements
+
+* The `cs/log-forging`, `cs/cleartext-storage`, and `cs/exposure-of-sensitive-information` queries now correctly handle unsanitized arguments to `ILogger` extension methods.
+* Updated the `neutralModel` extensible predicate to include a `kind` column.
+
 ## 0.6.1
 
 No user-facing changes.

csharp/ql/lib/change-notes/2023-04-26-neutral-model-kinds.md

@@ -1,4 +0,0 @@
----
-category: minorAnalysis
----
-* Updated the `neutralModel` extensible predicate to include a `kind` column.

csharp/ql/lib/change-notes/released/0.6.2.md

@@ -1,4 +1,6 @@
----
+## 0.6.2
-category: minorAnalysis
+
----
+### Minor Analysis Improvements
+
 * The `cs/log-forging`, `cs/cleartext-storage`, and `cs/exposure-of-sensitive-information` queries now correctly handle unsanitized arguments to `ILogger` extension methods.
+* Updated the `neutralModel` extensible predicate to include a `kind` column.

csharp/ql/lib/codeql-pack.release.yml

@@ -1,2 +1,2 @@
 ---
-lastReleaseVersion: 0.6.1
+lastReleaseVersion: 0.6.2

csharp/ql/lib/qlpack.yml

@@ -1,5 +1,5 @@
 name: codeql/csharp-all
-version: 0.6.2-dev
+version: 0.6.3-dev
 groups: csharp
 dbscheme: semmlecode.csharp.dbscheme
 extractor: csharp

csharp/ql/src/CHANGELOG.md

@@ -1,3 +1,7 @@
+## 0.6.2
+
+No user-facing changes.
+
 ## 0.6.1
 
 ### Minor Analysis Improvements

csharp/ql/src/change-notes/released/0.6.2.md

@@ -0,0 +1,3 @@
+## 0.6.2
+
+No user-facing changes.

csharp/ql/src/codeql-pack.release.yml

@@ -1,2 +1,2 @@
 ---
-lastReleaseVersion: 0.6.1
+lastReleaseVersion: 0.6.2

csharp/ql/src/qlpack.yml

@@ -1,5 +1,5 @@
 name: codeql/csharp-queries
-version: 0.6.2-dev
+version: 0.6.3-dev
 groups: 
   - csharp
   - queries

go/ql/lib/CHANGELOG.md

@@ -1,3 +1,9 @@
+## 0.5.2
+
+### Minor Analysis Improvements
+
+* Fixed data flow through variadic function parameters. The arguments corresponding to a variadic parameter are no longer returned by `CallNode.getArgument(int i)` and `CallNode.getAnArgument()`, and hence aren't `ArgumentNode`s. They now have one result, which is an `ImplicitVarargsSlice` node. For example, a call `f(a, b, c)` to a function `f(T...)` is treated like `f([]T{a, b, c})`. The old behaviour is preserved by `CallNode.getSyntacticArgument(int i)` and `CallNode.getASyntacticArgument()`. `CallExpr.getArgument(int i)` and `CallExpr.getAnArgument()` are unchanged, and will still have three results in the example given.
+
 ## 0.5.1
 
 ### Minor Analysis Improvements

go/ql/lib/change-notes/released/0.5.2.md

@@ -1,4 +1,5 @@
----
+## 0.5.2
-category: minorAnalysis
+
----
+### Minor Analysis Improvements
-* Fixed data flow through variadic function parameters. The arguments corresponding to a variadic parameter are no longer returned by `CallNode.getArgument(int i)` and `CallNode.getAnArgument()`, and hence aren't `ArgumentNode`s. They now have one result, which is an `ImplicitVarargsSlice` node. For example, a call `f(a, b, c)` to a function `f(T...)` is treated like `f([]T{a, b, c})`. The old behaviour is preserved by `CallNode.getSyntacticArgument(int i)` and `CallNode.getASyntacticArgument()`. `CallExpr.getArgument(int i)` and `CallExpr.getAnArgument()` are unchanged, and will still have three results in the example given.
+
+* Fixed data flow through variadic function parameters. The arguments corresponding to a variadic parameter are no longer returned by `CallNode.getArgument(int i)` and `CallNode.getAnArgument()`, and hence aren't `ArgumentNode`s. They now have one result, which is an `ImplicitVarargsSlice` node. For example, a call `f(a, b, c)` to a function `f(T...)` is treated like `f([]T{a, b, c})`. The old behaviour is preserved by `CallNode.getSyntacticArgument(int i)` and `CallNode.getASyntacticArgument()`. `CallExpr.getArgument(int i)` and `CallExpr.getAnArgument()` are unchanged, and will still have three results in the example given.

go/ql/lib/codeql-pack.release.yml

@@ -1,2 +1,2 @@
 ---
-lastReleaseVersion: 0.5.1
+lastReleaseVersion: 0.5.2

go/ql/lib/qlpack.yml

@@ -1,5 +1,5 @@
 name: codeql/go-all
-version: 0.5.2-dev
+version: 0.5.3-dev
 groups: go
 dbscheme: go.dbscheme
 extractor: go

go/ql/src/CHANGELOG.md

@@ -1,3 +1,7 @@
+## 0.5.2
+
+No user-facing changes.
+
 ## 0.5.1
 
 No user-facing changes.

go/ql/src/RedundantCode/DeadStoreOfField.ql

@@ -36,7 +36,10 @@ predicate escapes(DataFlow::Node nd) {
   exists(SendStmt s | nd.asExpr() = s.getValue())
   or
   // if `nd` is passed to a function, then it escapes
-  nd instanceof DataFlow::ArgumentNode
+  nd = any(DataFlow::CallNode c).getASyntacticArgument()
+  or
+  // if `nd` is the receiver of a function, then it escapes
+  nd = any(DataFlow::MethodCallNode c).getReceiver()
   or
   // if `nd` has its address taken, then it escapes
   exists(AddressExpr ae | nd.asExpr() = ae.getOperand())

go/ql/src/change-notes/released/0.5.2.md

@@ -0,0 +1,3 @@
+## 0.5.2
+
+No user-facing changes.

go/ql/src/codeql-pack.release.yml

@@ -1,2 +1,2 @@
 ---
-lastReleaseVersion: 0.5.1
+lastReleaseVersion: 0.5.2

go/ql/src/qlpack.yml

@@ -1,5 +1,5 @@
 name: codeql/go-queries
-version: 0.5.2-dev
+version: 0.5.3-dev
 groups:
   - go
   - queries

java/ql/lib/CHANGELOG.md

@@ -1,3 +1,44 @@
+## 0.6.2
+
+### Minor Analysis Improvements
+
+* Added SQL injection sinks for Spring JDBC's `NamedParameterJdbcOperations`.
+* Added models for the following packages:
+
+  * org.apache.hadoop.fs
+* Added the `ArithmeticCommon.qll` library to provide predicates for reasoning about arithmetic operations.
+* Added the `ArithmeticTaintedLocalQuery.qll` library to provide the `ArithmeticTaintedLocalOverflowFlow` and `ArithmeticTaintedLocalUnderflowFlow` taint-tracking modules to reason about arithmetic with unvalidated user input.
+* Added the `ArithmeticTaintedQuery.qll` library to provide the `RemoteUserInputOverflow` and `RemoteUserInputUnderflow` taint-tracking modules to reason about arithmetic with unvalidated user input.
+* Added the `ArithmeticUncontrolledQuery.qll` library to provide the `ArithmeticUncontrolledOverflowFlow`  and `ArithmeticUncontrolledUnderflowFlow` taint-tracking modules to reason about arithmetic with uncontrolled user input.
+* Added the `ArithmeticWithExtremeValuesQuery.qll` library to provide the `MaxValueFlow` and `MinValueFlow` dataflow modules to reason about arithmetic with extreme values.
+* Added the `BrokenCryptoAlgorithmQuery.qll` library to provide the `InsecureCryptoFlow` taint-tracking module to reason about broken cryptographic algorithm vulnerabilities.
+* Added the `ExecTaintedLocalQuery.qll` library to provide the `LocalUserInputToArgumentToExecFlow` taint-tracking module to reason about command injection vulnerabilities caused by local data flow.
+* Added the `ExternallyControlledFormatStringLocalQuery.qll` library to provide the `ExternallyControlledFormatStringLocalFlow` taint-tracking module to reason about format string vulnerabilities caused by local data flow.
+* Added the `ImproperValidationOfArrayConstructionCodeSpecifiedQuery.qll` library to provide the `BoundedFlowSourceFlow` dataflow module to reason about improper validation of code-specified sizes used for array construction.
+* Added the `ImproperValidationOfArrayConstructionLocalQuery.qll` library to provide the `ImproperValidationOfArrayConstructionLocalFlow` taint-tracking module to reason about improper validation of local user-provided sizes used for array construction caused by local data flow.
+* Added the `ImproperValidationOfArrayConstructionQuery.qll` library to provide the `ImproperValidationOfArrayConstructionFlow` taint-tracking module to reason about improper validation of user-provided size used for array construction.
+* Added the `ImproperValidationOfArrayIndexCodeSpecifiedQuery.qll` library to provide the `BoundedFlowSourceFlow` data flow module to reason about about improper validation of code-specified array index.
+* Added the `ImproperValidationOfArrayIndexLocalQuery.qll` library to provide the `ImproperValidationOfArrayIndexLocalFlow` taint-tracking module to reason about improper validation of a local user-provided array index.
+* Added the `ImproperValidationOfArrayIndexQuery.qll` library to provide the `ImproperValidationOfArrayIndexFlow` taint-tracking module to reason about improper validation of user-provided array index.
+* Added the `InsecureCookieQuery.qll` library to provide the `SecureCookieFlow` taint-tracking module to reason about insecure cookie vulnerabilities.
+* Added the `MaybeBrokenCryptoAlgorithmQuery.qll` library to provide the `InsecureCryptoFlow` taint-tracking module to reason about broken cryptographic algorithm vulnerabilities.
+* Added the `NumericCastTaintedQuery.qll` library to provide the `NumericCastTaintedFlow` taint-tracking module to reason about numeric cast vulnerabilities.
+* Added the `ResponseSplittingLocalQuery.qll` library to provide the `ResponseSplittingLocalFlow` taint-tracking module to reason about response splitting vulnerabilities caused by local data flow.
+* Added the `SqlConcatenatedQuery.qll` library to provide the `UncontrolledStringBuilderSourceFlow` taint-tracking module to reason about SQL injection vulnerabilities caused by concatenating untrusted strings.
+* Added the `SqlTaintedLocalQuery.qll` library to provide the `LocalUserInputToArgumentToSqlFlow` taint-tracking module to reason about SQL injection vulnerabilities caused by local data flow.
+* Added the `StackTraceExposureQuery.qll` library to provide the `printsStackExternally`, `stringifiedStackFlowsExternally`, and `getMessageFlowsExternally` predicates to reason about stack trace exposure vulnerabilities.
+* Added the `TaintedPermissionQuery.qll` library to provide the `TaintedPermissionFlow` taint-tracking module to reason about tainted permission vulnerabilities.
+* Added the `TempDirLocalInformationDisclosureQuery.qll` library to provide the `TempDirSystemGetPropertyToCreate` taint-tracking module to reason about local information disclosure vulnerabilities caused by local data flow.
+* Added the `UnsafeHostnameVerificationQuery.qll` library to provide the `TrustAllHostnameVerifierFlow` taint-tracking module to reason about insecure hostname verification vulnerabilities.
+* Added the `UrlRedirectLocalQuery.qll` library to provide the `UrlRedirectLocalFlow` taint-tracking module to reason about URL redirection vulnerabilities caused by local data flow.
+* Added the `UrlRedirectQuery.qll` library to provide the `UrlRedirectFlow` taint-tracking module to reason about URL redirection vulnerabilities.
+* Added the `XPathInjectionQuery.qll` library to provide the `XPathInjectionFlow` taint-tracking module to reason about XPath injection vulnerabilities.
+* Added the `XssLocalQuery.qll` library to provide the `XssLocalFlow` taint-tracking module to reason about XSS vulnerabilities caused by local data flow.
+* Moved the `url-open-stream` sink models to experimental and removed `url-open-stream` as a sink option from the [Customizing Library Models for Java](https://github.com/github/codeql/blob/733a00039efdb39c3dd76ddffad5e6d6c85e6774/docs/codeql/codeql-language-guides/customizing-library-models-for-java.rst#customizing-library-models-for-java) documentation.
+* Added models for the Apache Commons Net library.
+* Updated the `neutralModel` extensible predicate to include a `kind` column.
+* Added models for the `io.jsonwebtoken` library.
+
 ## 0.6.1
 
 ### Deprecated APIs

java/ql/lib/change-notes/2023-04-20-create-model-for-io-jsonwebtoken.md

@@ -1,5 +0,0 @@
----
-category: minorAnalysis
----
-* Added models for the `io.jsonwebtoken` library.
-

java/ql/lib/change-notes/2023-04-26-neutral-model-kinds.md

@@ -1,4 +0,0 @@
----
-category: minorAnalysis
----
-* Updated the `neutralModel` extensible predicate to include a `kind` column.

java/ql/lib/change-notes/2023-05-02-apache-commons-net-models.md

@@ -1,4 +0,0 @@
----
-category: minorAnalysis
----
-* Added models for the Apache Commons Net library.

java/ql/lib/change-notes/2023-05-03-url-open-stream-as-experimental.md

@@ -1,4 +0,0 @@
----
-category: minorAnalysis
----
-* Moved the `url-open-stream` sink models to experimental and removed `url-open-stream` as a sink option from the [Customizing Library Models for Java](https://github.com/github/codeql/blob/733a00039efdb39c3dd76ddffad5e6d6c85e6774/docs/codeql/codeql-language-guides/customizing-library-models-for-java.rst#customizing-library-models-for-java) documentation.

java/ql/lib/change-notes/2023-05-11-new-models.md

@@ -1,6 +0,0 @@
----
-category: minorAnalysis
----
-* Added models for the following packages:
-
-  * org.apache.hadoop.fs

java/ql/lib/change-notes/2023-05-12-spring-jdbc-sql-sinks.md

@@ -1,4 +0,0 @@
----
-category: minorAnalysis
----
-* Added SQL injection sinks for Spring JDBC's `NamedParameterJdbcOperations`.

java/ql/lib/change-notes/released/0.6.2.md

@@ -1,6 +1,11 @@
----
+## 0.6.2
-category: minorAnalysis
+
----
+### Minor Analysis Improvements
+
+* Added SQL injection sinks for Spring JDBC's `NamedParameterJdbcOperations`.
+* Added models for the following packages:
+
+  * org.apache.hadoop.fs
 * Added the `ArithmeticCommon.qll` library to provide predicates for reasoning about arithmetic operations.
 * Added the `ArithmeticTaintedLocalQuery.qll` library to provide the `ArithmeticTaintedLocalOverflowFlow` and `ArithmeticTaintedLocalUnderflowFlow` taint-tracking modules to reason about arithmetic with unvalidated user input.
 * Added the `ArithmeticTaintedQuery.qll` library to provide the `RemoteUserInputOverflow` and `RemoteUserInputUnderflow` taint-tracking modules to reason about arithmetic with unvalidated user input.
@@ -28,4 +33,8 @@ category: minorAnalysis
 * Added the `UrlRedirectLocalQuery.qll` library to provide the `UrlRedirectLocalFlow` taint-tracking module to reason about URL redirection vulnerabilities caused by local data flow.
 * Added the `UrlRedirectQuery.qll` library to provide the `UrlRedirectFlow` taint-tracking module to reason about URL redirection vulnerabilities.
 * Added the `XPathInjectionQuery.qll` library to provide the `XPathInjectionFlow` taint-tracking module to reason about XPath injection vulnerabilities.
-* Added the `XssLocalQuery.qll` library to provide the `XssLocalFlow` taint-tracking module to reason about XSS vulnerabilities caused by local data flow.
+* Added the `XssLocalQuery.qll` library to provide the `XssLocalFlow` taint-tracking module to reason about XSS vulnerabilities caused by local data flow.
+* Moved the `url-open-stream` sink models to experimental and removed `url-open-stream` as a sink option from the [Customizing Library Models for Java](https://github.com/github/codeql/blob/733a00039efdb39c3dd76ddffad5e6d6c85e6774/docs/codeql/codeql-language-guides/customizing-library-models-for-java.rst#customizing-library-models-for-java) documentation.
+* Added models for the Apache Commons Net library.
+* Updated the `neutralModel` extensible predicate to include a `kind` column.
+* Added models for the `io.jsonwebtoken` library.

java/ql/lib/codeql-pack.release.yml

@@ -1,2 +1,2 @@
 ---
-lastReleaseVersion: 0.6.1
+lastReleaseVersion: 0.6.2

java/ql/lib/qlpack.yml

@@ -1,5 +1,5 @@
 name: codeql/java-all
-version: 0.6.2-dev
+version: 0.6.3-dev
 groups: java
 dbscheme: config/semmlecode.dbscheme
 extractor: java

java/ql/src/CHANGELOG.md

@@ -1,3 +1,11 @@
+## 0.6.2
+
+### Minor Analysis Improvements
+
+* The query `java/groovy-injection` now recognizes `groovy.text.TemplateEngine.createTemplate` as a sink.
+* The queries `java/xxe` and `java/xxe-local` now recognize the second argument of calls to `XPath.evaluate` as a sink.
+* Experimental sinks for the query "Resolving XML external entity in user-controlled data" (`java/xxe`) have been promoted to the main query pack. These sinks were originally [submitted as part of an experimental query by @haby0](https://github.com/github/codeql/pull/6564).
+
 ## 0.6.1
 
 No user-facing changes.

java/ql/src/change-notes/2023-05-15-xpath-xxe-sink.md

@@ -1,4 +0,0 @@
----
-category: minorAnalysis
----
-* The queries `java/xxe` and `java/xxe-local` now recognize the second argument of calls to `XPath.evaluate` as a sink.

java/ql/src/change-notes/2023-05-19-groovy-injection-sink.md

@@ -1,4 +0,0 @@
----
-category: minorAnalysis
----
-* The query `java/groovy-injection` now recognizes `groovy.text.TemplateEngine.createTemplate` as a sink.

java/ql/src/change-notes/released/0.6.2.md

@@ -1,4 +1,7 @@
----
+## 0.6.2
-category: minorAnalysis
+
----
+### Minor Analysis Improvements
+
+* The query `java/groovy-injection` now recognizes `groovy.text.TemplateEngine.createTemplate` as a sink.
+* The queries `java/xxe` and `java/xxe-local` now recognize the second argument of calls to `XPath.evaluate` as a sink.
 * Experimental sinks for the query "Resolving XML external entity in user-controlled data" (`java/xxe`) have been promoted to the main query pack. These sinks were originally [submitted as part of an experimental query by @haby0](https://github.com/github/codeql/pull/6564).

java/ql/src/codeql-pack.release.yml

@@ -1,2 +1,2 @@
 ---
-lastReleaseVersion: 0.6.1
+lastReleaseVersion: 0.6.2

java/ql/src/qlpack.yml

@@ -1,5 +1,5 @@
 name: codeql/java-queries
-version: 0.6.2-dev
+version: 0.6.3-dev
 groups: 
   - java
   - queries

javascript/ql/lib/CHANGELOG.md

@@ -1,3 +1,9 @@
+## 0.6.2
+
+### Minor Analysis Improvements
+
+* Improved the queries for injection vulnerabilities in GitHub Actions workflows (`js/actions/command-injection` and `js/actions/pull-request-target`) and the associated library `semmle.javascript.Actions`. These now support steps defined in composite actions, in addition to steps defined in Actions workflow files. It supports more potentially untrusted input values. Additionally to the shell injections it now also detects injections in `actions/github-script`. It also detects simple injections from user controlled `${{ env.name }}`. Additionally to the `yml` extension now it also supports workflows with the `yaml` extension.
+
 ## 0.6.1
 
 ### Major Analysis Improvements

javascript/ql/lib/change-notes/released/0.6.2.md

@@ -1,4 +1,5 @@
----
+## 0.6.2
-category: minorAnalysis
+
----
+### Minor Analysis Improvements
-* Improved the queries for injection vulnerabilities in GitHub Actions workflows (`js/actions/command-injection` and `js/actions/pull-request-target`) and the associated library `semmle.javascript.Actions`. These now support steps defined in composite actions, in addition to steps defined in Actions workflow files. It supports more potentially untrusted input values. Additionally to the shell injections it now also detects injections in `actions/github-script`. It also detects simple injections from user controlled `${{ env.name }}`. Additionally to the `yml` extension now it also supports workflows with the `yaml` extension.
+
+* Improved the queries for injection vulnerabilities in GitHub Actions workflows (`js/actions/command-injection` and `js/actions/pull-request-target`) and the associated library `semmle.javascript.Actions`. These now support steps defined in composite actions, in addition to steps defined in Actions workflow files. It supports more potentially untrusted input values. Additionally to the shell injections it now also detects injections in `actions/github-script`. It also detects simple injections from user controlled `${{ env.name }}`. Additionally to the `yml` extension now it also supports workflows with the `yaml` extension.

javascript/ql/lib/codeql-pack.release.yml

@@ -1,2 +1,2 @@
 ---
-lastReleaseVersion: 0.6.1
+lastReleaseVersion: 0.6.2

javascript/ql/lib/qlpack.yml

@@ -1,5 +1,5 @@
 name: codeql/javascript-all
-version: 0.6.2-dev
+version: 0.6.3-dev
 groups: javascript
 dbscheme: semmlecode.javascript.dbscheme
 extractor: javascript

javascript/ql/src/CHANGELOG.md

@@ -1,3 +1,23 @@
+## 0.6.2
+
+### Major Analysis Improvements
+
+* Added taint sources from the `@actions/core` and `@actions/github` packages.
+* Added command-injection sinks from the `@actions/exec` package.
+
+### Minor Analysis Improvements
+
+* The `js/indirect-command-line-injection` query no longer flags command arguments that cannot be interpreted as a shell string.
+* The `js/unsafe-deserialization` query no longer flags deserialization through the `js-yaml` library, except
+  when it is used with an unsafe schema.
+* The Forge module in `CryptoLibraries.qll` now correctly classifies SHA-512/224,
+  SHA-512/256, and SHA-512/384 hashes used in message digests as NonKeyCiphers.
+
+### Bug Fixes
+
+* Fixed a spurious diagnostic warning about comments in JSON files being illegal.
+  Comments in JSON files are in fact fully supported, and the diagnostic message was misleading.
+
 ## 0.6.1
 
 ### Minor Analysis Improvements

javascript/ql/src/change-notes/2023-04-13-Forge-truncated-sha512-hash.md

@@ -1,5 +0,0 @@
----
-category: minorAnalysis
----
-* The Forge module in `CryptoLibraries.qll` now correctly classifies SHA-512/224,
-  SHA-512/256, and SHA-512/384 hashes used in message digests as NonKeyCiphers.

javascript/ql/src/change-notes/2023-04-26-unsafe-yaml-deserialization.md

@@ -1,5 +0,0 @@
----
-category: minorAnalysis
----
-* The `js/unsafe-deserialization` query no longer flags deserialization through the `js-yaml` library, except
-  when it is used with an unsafe schema.

javascript/ql/src/change-notes/2023-04-28-json-with-comments.md

@@ -1,5 +0,0 @@
----
-category: fix
----
-* Fixed a spurious diagnostic warning about comments in JSON files being illegal.
-  Comments in JSON files are in fact fully supported, and the diagnostic message was misleading.

javascript/ql/src/change-notes/2023-05-02-github-actions-sources.md

@@ -1,5 +0,0 @@
----
-category: majorAnalysis
----
-* Added taint sources from the `@actions/core` and `@actions/github` packages.
-* Added command-injection sinks from the `@actions/exec` package.

javascript/ql/src/change-notes/2023-05-17-indirect-shell.md

@@ -1,4 +0,0 @@
----
-category: minorAnalysis
----
-* The `js/indirect-command-line-injection` query no longer flags command arguments that cannot be interpreted as a shell string.

javascript/ql/src/change-notes/released/0.6.2.md

@@ -0,0 +1,19 @@
+## 0.6.2
+
+### Major Analysis Improvements
+
+* Added taint sources from the `@actions/core` and `@actions/github` packages.
+* Added command-injection sinks from the `@actions/exec` package.
+
+### Minor Analysis Improvements
+
+* The `js/indirect-command-line-injection` query no longer flags command arguments that cannot be interpreted as a shell string.
+* The `js/unsafe-deserialization` query no longer flags deserialization through the `js-yaml` library, except
+  when it is used with an unsafe schema.
+* The Forge module in `CryptoLibraries.qll` now correctly classifies SHA-512/224,
+  SHA-512/256, and SHA-512/384 hashes used in message digests as NonKeyCiphers.
+
+### Bug Fixes
+
+* Fixed a spurious diagnostic warning about comments in JSON files being illegal.
+  Comments in JSON files are in fact fully supported, and the diagnostic message was misleading.

javascript/ql/src/codeql-pack.release.yml

@@ -1,2 +1,2 @@
 ---
-lastReleaseVersion: 0.6.1
+lastReleaseVersion: 0.6.2

javascript/ql/src/qlpack.yml

@@ -1,5 +1,5 @@
 name: codeql/javascript-queries
-version: 0.6.2-dev
+version: 0.6.3-dev
 groups: 
   - javascript
   - queries

misc/codegen/generators/qlgen.py

@@ -102,7 +102,7 @@ def _get_doc(cls: schema.Class, prop: schema.Property, plural=None):
     return f"{prop_name} of this {class_name}"
 
 
-def get_ql_property(cls: schema.Class, prop: schema.Property, prev_child: str = "") -> ql.Property:
+def get_ql_property(cls: schema.Class, prop: schema.Property, lookup: typing.Dict[str, schema.Class], prev_child: str = "") -> ql.Property:
     args = dict(
         type=prop.type if not prop.is_predicate else "predicate",
         qltest_skip="qltest_skip" in prop.pragmas,
@@ -110,7 +110,8 @@ def get_ql_property(cls: schema.Class, prop: schema.Property, prev_child: str =
         is_optional=prop.is_optional,
         is_predicate=prop.is_predicate,
         is_unordered=prop.is_unordered,
-        description=prop.description
+        description=prop.description,
+        type_is_hideable=lookup[prop.type].hideable if prop.type in lookup else False,
     )
     if prop.is_single:
         args.update(
@@ -147,12 +148,12 @@ def get_ql_property(cls: schema.Class, prop: schema.Property, prev_child: str =
     return ql.Property(**args)
 
 
-def get_ql_class(cls: schema.Class) -> ql.Class:
+def get_ql_class(cls: schema.Class, lookup: typing.Dict[str, schema.Class]) -> ql.Class:
     pragmas = {k: True for k in cls.pragmas if k.startswith("ql")}
     prev_child = ""
     properties = []
     for p in cls.properties:
-        prop = get_ql_property(cls, p, prev_child)
+        prop = get_ql_property(cls, p, lookup, prev_child)
         if prop.is_child:
             prev_child = prop.singular
         properties.append(prop)
@@ -164,6 +165,7 @@ def get_ql_class(cls: schema.Class) -> ql.Class:
         dir=pathlib.Path(cls.group or ""),
         ipa=bool(cls.ipa),
         doc=cls.doc,
+        hideable=cls.hideable,
         **pragmas,
     )
 
@@ -254,7 +256,7 @@ def _get_all_properties_to_be_tested(cls: schema.Class, lookup: typing.Dict[str,
     for c, p in _get_all_properties(cls, lookup):
         if not ("qltest_skip" in c.pragmas or "qltest_skip" in p.pragmas):
             # TODO here operations are duplicated, but should be better if we split ql and qltest generation
-            p = get_ql_property(c, p)
+            p = get_ql_property(c, p, lookup)
             yield ql.PropertyForTest(p.getter, is_total=p.is_single or p.is_predicate,
                                      type=p.type if not p.is_predicate else None, is_indexed=p.is_indexed)
             if p.is_repeated and not p.is_optional:
@@ -329,7 +331,7 @@ def generate(opts, renderer):
 
     data = schemaloader.load_file(input)
 
-    classes = {name: get_ql_class(cls) for name, cls in data.classes.items()}
+    classes = {name: get_ql_class(cls, data.classes) for name, cls in data.classes.items()}
     if not classes:
         raise NoClasses
     root = next(iter(classes.values()))

misc/codegen/lib/ql.py

@@ -42,6 +42,7 @@ class Property:
     description: List[str] = field(default_factory=list)
     doc: Optional[str] = None
     doc_plural: Optional[str] = None
+    type_is_hideable: bool = False
 
     def __post_init__(self):
         if self.tableparams:
@@ -113,6 +114,7 @@ class Class:
     ql_internal: bool = False
     ipa: bool = False
     doc: List[str] = field(default_factory=list)
+    hideable: bool = False
 
     def __post_init__(self):
         self.bases = [Base(str(b), str(prev)) for b, prev in zip(self.bases, itertools.chain([""], self.bases))]

misc/codegen/lib/schema.py

@@ -91,6 +91,7 @@ class Class:
     """^^^ filled with `True` for non-final classes with only synthesized final descendants """
     doc: List[str] = field(default_factory=list)
     default_doc_name: Optional[str] = None
+    hideable: bool = False
 
     @property
     def final(self):

misc/codegen/lib/schemadefs.py

@@ -145,6 +145,7 @@ _Pragma("qltest_collapse_hierarchy")
 _Pragma("qltest_uncollapse_hierarchy")
 
 ql.default_doc_name = lambda doc: _annotate(doc_name=doc)
+ql.hideable = _annotate(hideable=True)
 _Pragma("ql_internal")
 
 _Pragma("cpp_skip")

misc/codegen/loaders/schemaloader.py

@@ -37,6 +37,7 @@ def _get_class(cls: type) -> schema.Class:
                         derived={d.__name__ for d in cls.__subclasses__()},
                         # getattr to inherit from bases
                         group=getattr(cls, "_group", ""),
+                        hideable=getattr(cls, "_hideable", False),
                         # in the following we don't use `getattr` to avoid inheriting
                         pragmas=cls.__dict__.get("_pragmas", []),
                         ipa=cls.__dict__.get("_ipa", None),
@@ -94,6 +95,18 @@ def _fill_ipa_information(classes: typing.Dict[str, schema.Class]):
             cls.ipa = True
 
 
+def _fill_hideable_information(classes: typing.Dict[str, schema.Class]):
+    """ Update the class map propagating the `hideable` attribute upwards in the hierarchy """
+    todo = [cls for cls in classes.values() if cls.hideable]
+    while todo:
+        cls = todo.pop()
+        for base in cls.bases:
+            supercls = classes[base]
+            if not supercls.hideable:
+                supercls.hideable = True
+                todo.append(supercls)
+
+
 def load(m: types.ModuleType) -> schema.Schema:
     includes = set()
     classes = {}
@@ -122,6 +135,7 @@ def load(m: types.ModuleType) -> schema.Schema:
             cls.is_null_class = True
 
     _fill_ipa_information(classes)
+    _fill_hideable_information(classes)
 
     return schema.Schema(includes=includes, classes=_toposort_classes_by_group(classes), null=null)
 

misc/codegen/templates/ql_class.mustache

@@ -60,7 +60,7 @@ module Generated {
     {{/final}}
     {{#properties}}
 
-    {{#type_is_class}}
+    {{#type_is_hideable}}
     /**
      * {{>ql_property_doc}}     *
      * This includes nodes from the "hidden" AST. It can be overridden in subclasses to change the
@@ -85,11 +85,11 @@ module Generated {
      */
     final {{type}} {{getter}}({{#is_indexed}}int index{{/is_indexed}}) {
       exists({{type}} immediate | immediate = this.get{{#is_unordered}}An{{/is_unordered}}Immediate{{singular}}({{#is_indexed}}index{{/is_indexed}}) and
-        if exists(this.getResolveStep()) then result = immediate else result = immediate.resolve())
+        {{#hideable}}if exists(this.getResolveStep()) then result = immediate else {{/hideable}}result = immediate.resolve())
     }
 
-    {{/type_is_class}}
+    {{/type_is_hideable}}
-    {{^type_is_class}}
+    {{^type_is_hideable}}
     /**
      * {{>ql_property_doc}}     *
     {{#has_description}}
@@ -100,14 +100,14 @@ module Generated {
      */
     {{type}} {{getter}}({{#is_indexed}}int index{{/is_indexed}}) {
       {{^ipa}}
-      {{^is_predicate}}result = {{/is_predicate}}Synth::convert{{name}}ToRaw(this){{^root}}.(Raw::{{name}}){{/root}}.{{getter}}({{#is_indexed}}index{{/is_indexed}})
+      {{^is_predicate}}result = {{/is_predicate}}{{#type_is_class}}Synth::convert{{type}}FromRaw({{/type_is_class}}Synth::convert{{name}}ToRaw(this){{^root}}.(Raw::{{name}}){{/root}}.{{getter}}({{#is_indexed}}index{{/is_indexed}}){{#type_is_class}}){{/type_is_class}}
       {{/ipa}}
       {{#ipa}}
       none()
       {{/ipa}}
     }
 
-    {{/type_is_class}}
+    {{/type_is_hideable}}
     {{#is_optional}}
     /**
      * Holds if `{{getter}}({{#is_repeated}}index{{/is_repeated}})` exists.

misc/codegen/templates/ql_parent.mustache

@@ -28,7 +28,7 @@ private module Impl {
             {{! for single and optional properties it adds 1 (regardless of whether the optional property exists) }}
             {{! for repeated it adds 1 + the maximum index (which works for repeated optional as well) }}
                 and
-                n{{singular}} = n{{prev_child}} + 1{{#is_repeated}}+ max(int i | i = -1 or exists(e.getImmediate{{singular}}(i)) | i){{/is_repeated}}
+                n{{singular}} = n{{prev_child}} + 1{{#is_repeated}}+ max(int i | i = -1 or exists(e.get{{#type_is_hideable}}Immediate{{/type_is_hideable}}{{singular}}(i)) | i){{/is_repeated}}
             {{/is_child}}
         {{/properties}} and (
         none()
@@ -40,10 +40,10 @@ private module Impl {
             {{#is_child}}
                 or
                 {{#is_repeated}}
-                    result = e.getImmediate{{singular}}(index - n{{prev_child}}) and partialPredicateCall = "{{singular}}(" + (index - n{{prev_child}}).toString() + ")"
+                    result = e.get{{#type_is_hideable}}Immediate{{/type_is_hideable}}{{singular}}(index - n{{prev_child}}) and partialPredicateCall = "{{singular}}(" + (index - n{{prev_child}}).toString() + ")"
                 {{/is_repeated}}
                 {{^is_repeated}}
-                    index = n{{prev_child}} and result = e.getImmediate{{singular}}() and partialPredicateCall = "{{singular}}()"
+                    index = n{{prev_child}} and result = e.get{{#type_is_hideable}}Immediate{{/type_is_hideable}}{{singular}}() and partialPredicateCall = "{{singular}}()"
                 {{/is_repeated}}
             {{/is_child}}
         {{/properties}}
@@ -71,21 +71,21 @@ none()
 * if `e` has conversions, `getImmediateParent(e)` will give the innermost conversion in the hidden AST.
 */
 Element getImmediateParent(Element e) {
-// `unique` is used here to tell the optimizer that there is in fact only one result
+    // `unique` is used here to tell the optimizer that there is in fact only one result
-// this is tested by the `library-tests/parent/no_double_parents.ql` test
+    // this is tested by the `library-tests/parent/no_double_parents.ql` test
-result = unique(Element x | e = Impl::getImmediateChild(x, _, _) | x)
+    result = unique(Element x | e = Impl::getImmediateChild(x, _, _) | x)
 }
 
 /**
 * Gets the immediate child indexed at `index`. Indexes are not guaranteed to be contiguous, but are guaranteed to be distinct. `accessor` is bound the member predicate call resulting in the given child.
 */
 Element getImmediateChildAndAccessor(Element e, int index, string accessor) {
-exists(string partialAccessor | result = Impl::getImmediateChild(e, index, partialAccessor) and accessor = "get" + partialAccessor)
+    exists(string partialAccessor | result = Impl::getImmediateChild(e, index, partialAccessor) and accessor = "get" + partialAccessor)
 }
 
 /**
 * Gets the child indexed at `index`. Indexes are not guaranteed to be contiguous, but are guaranteed to be distinct. `accessor` is bound the member predicate call resulting in the given child.
 */
 Element getChildAndAccessor(Element e, int index, string accessor) {
-exists(string partialAccessor | result = Impl::getImmediateChild(e, index, partialAccessor).resolve() and accessor = "get" + partialAccessor)
+    exists(string partialAccessor | result = Impl::getImmediateChild(e, index, partialAccessor).resolve() and accessor = "get" + partialAccessor)
 }

misc/codegen/test/test_qlgen.py

@@ -139,15 +139,16 @@ def a_ql_class(**kwargs):
     return ql.Class(**kwargs, import_prefix=gen_import)
 
 
-def a_ql_stub(**kwargs):
+def a_ql_stub(*, name, import_prefix="", **kwargs):
-    return ql.Stub(**kwargs, import_prefix=gen_import)
+    return ql.Stub(name=name, **kwargs, import_prefix=gen_import,
+                   base_import=f"{gen_import_prefix}{import_prefix}{name}")
 
 
 def test_one_empty_class(generate_classes):
     assert generate_classes([
         schema.Class("A")
     ]) == {
-        "A.qll": (a_ql_stub(name="A", base_import=gen_import_prefix + "A"),
+        "A.qll": (a_ql_stub(name="A"),
                   a_ql_class(name="A", final=True)),
     }
 
@@ -159,15 +160,11 @@ def test_hierarchy(generate_classes):
         schema.Class("B", bases=["A"], derived={"D"}),
         schema.Class("A", derived={"B", "C"}),
     ]) == {
-        "A.qll": (a_ql_stub(name="A", base_import=gen_import_prefix + "A"),
+        "A.qll": (a_ql_stub(name="A"), a_ql_class(name="A")),
-                  a_ql_class(name="A")),
+        "B.qll": (a_ql_stub(name="B"), a_ql_class(name="B", bases=["A"], imports=[stub_import_prefix + "A"])),
-        "B.qll": (a_ql_stub(name="B", base_import=gen_import_prefix + "B"),
+        "C.qll": (a_ql_stub(name="C"), a_ql_class(name="C", bases=["A"], imports=[stub_import_prefix + "A"])),
-                  a_ql_class(name="B", bases=["A"], imports=[stub_import_prefix + "A"])),
+        "D.qll": (a_ql_stub(name="D"), a_ql_class(name="D", final=True, bases=["B", "C"],
-        "C.qll": (a_ql_stub(name="C", base_import=gen_import_prefix + "C"),
+                                                  imports=[stub_import_prefix + cls for cls in "BC"])),
-                  a_ql_class(name="C", bases=["A"], imports=[stub_import_prefix + "A"])),
-        "D.qll": (a_ql_stub(name="D", base_import=gen_import_prefix + "D"),
-                  a_ql_class(name="D", final=True, bases=["B", "C"],
-                             imports=[stub_import_prefix + cls for cls in "BC"])),
     }
 
 
@@ -213,7 +210,7 @@ def test_single_property(generate_classes):
         schema.Class("MyObject", properties=[
             schema.SingleProperty("foo", "bar")]),
     ]) == {
-        "MyObject.qll": (a_ql_stub(name="MyObject", base_import=gen_import_prefix + "MyObject"),
+        "MyObject.qll": (a_ql_stub(name="MyObject"),
                          a_ql_class(name="MyObject", final=True,
                                     properties=[
                                         ql.Property(singular="Foo", type="bar", tablename="my_objects",
@@ -236,9 +233,8 @@ def test_children(generate_classes):
             schema.RepeatedOptionalProperty("child_4", "int", is_child=True),
         ]),
     ]) == {
-        "FakeRoot.qll": (a_ql_stub(name="FakeRoot", base_import=gen_import_prefix + "FakeRoot"),
+        "FakeRoot.qll": (a_ql_stub(name="FakeRoot"), a_ql_class(name="FakeRoot", final=True)),
-                         a_ql_class(name="FakeRoot", final=True)),
+        "MyObject.qll": (a_ql_stub(name="MyObject"),
-        "MyObject.qll": (a_ql_stub(name="MyObject", base_import=gen_import_prefix + "MyObject"),
                          a_ql_class(name="MyObject", final=True,
                                     properties=[
                                         ql.Property(singular="A", type="int", tablename="my_objects",
@@ -286,7 +282,7 @@ def test_single_properties(generate_classes):
             schema.SingleProperty("three", "z"),
         ]),
     ]) == {
-        "MyObject.qll": (a_ql_stub(name="MyObject", base_import=gen_import_prefix + "MyObject"),
+        "MyObject.qll": (a_ql_stub(name="MyObject"),
                          a_ql_class(name="MyObject", final=True,
                                     properties=[
                                         ql.Property(singular="One", type="x", tablename="my_objects",
@@ -309,9 +305,8 @@ def test_optional_property(generate_classes, is_child, prev_child):
         schema.Class("MyObject", properties=[
             schema.OptionalProperty("foo", "bar", is_child=is_child)]),
     ]) == {
-        "FakeRoot.qll": (a_ql_stub(name="FakeRoot", base_import=gen_import_prefix + "FakeRoot"),
+        "FakeRoot.qll": (a_ql_stub(name="FakeRoot"), a_ql_class(name="FakeRoot", final=True)),
-                         a_ql_class(name="FakeRoot", final=True)),
+        "MyObject.qll": (a_ql_stub(name="MyObject"),
-        "MyObject.qll": (a_ql_stub(name="MyObject", base_import=gen_import_prefix + "MyObject"),
                          a_ql_class(name="MyObject", final=True, properties=[
                                     ql.Property(singular="Foo", type="bar", tablename="my_object_foos",
                                                 tableparams=["this", "result"],
@@ -327,9 +322,8 @@ def test_repeated_property(generate_classes, is_child, prev_child):
         schema.Class("MyObject", properties=[
             schema.RepeatedProperty("foo", "bar", is_child=is_child)]),
     ]) == {
-        "FakeRoot.qll": (a_ql_stub(name="FakeRoot", base_import=gen_import_prefix + "FakeRoot"),
+        "FakeRoot.qll": (a_ql_stub(name="FakeRoot"), a_ql_class(name="FakeRoot", final=True)),
-                         a_ql_class(name="FakeRoot", final=True)),
+        "MyObject.qll": (a_ql_stub(name="MyObject"),
-        "MyObject.qll": (a_ql_stub(name="MyObject", base_import=gen_import_prefix + "MyObject"),
                          a_ql_class(name="MyObject", final=True, properties=[
                                     ql.Property(singular="Foo", plural="Foos", type="bar", tablename="my_object_foos",
                                                 tableparams=["this", "index", "result"], prev_child=prev_child,
@@ -344,9 +338,8 @@ def test_repeated_unordered_property(generate_classes):
         schema.Class("MyObject", properties=[
             schema.RepeatedUnorderedProperty("foo", "bar")]),
     ]) == {
-        "FakeRoot.qll": (a_ql_stub(name="FakeRoot", base_import=gen_import_prefix + "FakeRoot"),
+        "FakeRoot.qll": (a_ql_stub(name="FakeRoot"), a_ql_class(name="FakeRoot", final=True)),
-                         a_ql_class(name="FakeRoot", final=True)),
+        "MyObject.qll": (a_ql_stub(name="MyObject"),
-        "MyObject.qll": (a_ql_stub(name="MyObject", base_import=gen_import_prefix + "MyObject"),
                          a_ql_class(name="MyObject", final=True, properties=[
                                     ql.Property(singular="Foo", plural="Foos", type="bar", tablename="my_object_foos",
                                                 tableparams=["this", "result"], is_unordered=True,
@@ -363,9 +356,8 @@ def test_repeated_optional_property(generate_classes, is_child, prev_child):
             schema.RepeatedOptionalProperty("foo", "bar", is_child=is_child)]),
     ]) == {
 
-        "FakeRoot.qll": (a_ql_stub(name="FakeRoot", base_import=gen_import_prefix + "FakeRoot"),
+        "FakeRoot.qll": (a_ql_stub(name="FakeRoot"), a_ql_class(name="FakeRoot", final=True)),
-                         a_ql_class(name="FakeRoot", final=True)),
+        "MyObject.qll": (a_ql_stub(name="MyObject"),
-        "MyObject.qll": (a_ql_stub(name="MyObject", base_import=gen_import_prefix + "MyObject"),
                          a_ql_class(name="MyObject", final=True, properties=[
                                     ql.Property(singular="Foo", plural="Foos", type="bar", tablename="my_object_foos",
                                                 tableparams=["this", "index", "result"], is_optional=True,
@@ -380,7 +372,7 @@ def test_predicate_property(generate_classes):
         schema.Class("MyObject", properties=[
             schema.PredicateProperty("is_foo")]),
     ]) == {
-        "MyObject.qll": (a_ql_stub(name="MyObject", base_import=gen_import_prefix + "MyObject"),
+        "MyObject.qll": (a_ql_stub(name="MyObject"),
                          a_ql_class(name="MyObject", final=True, properties=[
                                     ql.Property(singular="isFoo", type="predicate", tablename="my_object_is_foo",
                                                 tableparams=["this"], is_predicate=True, doc="this my object is foo"),
@@ -395,7 +387,7 @@ def test_single_class_property(generate_classes, is_child, prev_child):
         schema.Class("MyObject", properties=[
             schema.SingleProperty("foo", "Bar", is_child=is_child)]),
     ]) == {
-        "MyObject.qll": (a_ql_stub(name="MyObject", base_import=gen_import_prefix + "MyObject"),
+        "MyObject.qll": (a_ql_stub(name="MyObject"),
                          a_ql_class(
             name="MyObject", final=True, imports=[stub_import_prefix + "Bar"], properties=[
                 ql.Property(singular="Foo", type="Bar", tablename="my_objects",
@@ -404,8 +396,7 @@ def test_single_class_property(generate_classes, is_child, prev_child):
                             prev_child=prev_child, doc="foo of this my object"),
             ],
         )),
-        "Bar.qll": (a_ql_stub(name="Bar", base_import=gen_import_prefix + "Bar"),
+        "Bar.qll": (a_ql_stub(name="Bar"), a_ql_class(name="Bar", final=True)),
-                    a_ql_class(name="Bar", final=True)),
     }
 
 
@@ -414,8 +405,7 @@ def test_class_with_doc(generate_classes):
     assert generate_classes([
         schema.Class("A", doc=doc),
     ]) == {
-        "A.qll": (a_ql_stub(name="A", base_import=gen_import_prefix + "A"),
+        "A.qll": (a_ql_stub(name="A"), a_ql_class(name="A", final=True, doc=doc)),
-                  a_ql_class(name="A", final=True, doc=doc)),
     }
 
 
@@ -425,9 +415,8 @@ def test_class_dir(generate_classes):
         schema.Class("A", derived={"B"}, group=dir),
         schema.Class("B", bases=["A"]),
     ]) == {
-        f"{dir}/A.qll": (a_ql_stub(name="A", base_import=gen_import_prefix + "another.rel.path.A"),
+        f"{dir}/A.qll": (a_ql_stub(name="A", import_prefix="another.rel.path."), a_ql_class(name="A", dir=pathlib.Path(dir))),
-                         a_ql_class(name="A", dir=pathlib.Path(dir))),
+        "B.qll": (a_ql_stub(name="B"),
-        "B.qll": (a_ql_stub(name="B", base_import=gen_import_prefix + "B"),
                   a_ql_class(name="B", final=True, bases=["A"],
                              imports=[stub_import_prefix + "another.rel.path.A"])),
     }
@@ -586,11 +575,11 @@ def test_test_partial_properties(opts, generate_tests):
                                                                                type="bool")),
         "B/B_getZ.ql": a_ql_property_tester(class_name="B",
                                             property=ql.PropertyForTest(getter="getZ", is_total=False,
-                                                                        is_indexed=True,
+                                                                               is_indexed=True,
-                                                                        type="int")),
+                                                                               type="int")),
         "B/B_getAW.ql": a_ql_property_tester(class_name="B",
                                              property=ql.PropertyForTest(getter="getAW", is_total=False,
-                                                                         type="string")),
+                                                                                type="string")),
     }
 
 
@@ -611,7 +600,7 @@ def test_test_properties_deduplicated(opts, generate_tests):
         ]),
         "Final/Final_getY.ql": a_ql_property_tester(class_name="Final",
                                                     property=ql.PropertyForTest(getter="getY", is_total=False,
-                                                                                is_indexed=True,
+                                                                                       is_indexed=True,
                                                                                        type="bool")),
     }
 
@@ -706,7 +695,7 @@ def test_property_description(generate_classes):
             schema.SingleProperty("foo", "bar", description=description),
         ]),
     ]) == {
-        "MyObject.qll": (a_ql_stub(name="MyObject", base_import=gen_import_prefix + "MyObject"),
+        "MyObject.qll": (a_ql_stub(name="MyObject"),
                          a_ql_class(name="MyObject", final=True,
                                     properties=[
                                         ql.Property(singular="Foo", type="bar", tablename="my_objects",
@@ -722,7 +711,7 @@ def test_property_doc_override(generate_classes):
         schema.Class("MyObject", properties=[
             schema.SingleProperty("foo", "bar", doc="baz")]),
     ]) == {
-        "MyObject.qll": (a_ql_stub(name="MyObject", base_import=gen_import_prefix + "MyObject"),
+        "MyObject.qll": (a_ql_stub(name="MyObject"),
                          a_ql_class(name="MyObject", final=True,
                                     properties=[
                                         ql.Property(singular="Foo", type="bar", tablename="my_objects",
@@ -737,7 +726,7 @@ def test_repeated_property_doc_override(generate_classes):
             schema.RepeatedProperty("x", "int", doc="children of this"),
             schema.RepeatedOptionalProperty("y", "int", doc="child of this")]),
     ]) == {
-        "MyObject.qll": (a_ql_stub(name="MyObject", base_import=gen_import_prefix + "MyObject"),
+        "MyObject.qll": (a_ql_stub(name="MyObject"),
                          a_ql_class(name="MyObject", final=True,
                                     properties=[
                                         ql.Property(singular="X", plural="Xes", type="int",
@@ -759,7 +748,7 @@ def test_property_doc_abbreviations(generate_classes, abbr, expected):
         schema.Class("Object", properties=[
             schema.SingleProperty(f"foo_{abbr}_bar", "baz")]),
     ]) == {
-        "Object.qll": (a_ql_stub(name="Object", base_import=gen_import_prefix + "Object"),
+        "Object.qll": (a_ql_stub(name="Object"),
                        a_ql_class(name="Object", final=True,
                                   properties=[
                                       ql.Property(singular=f"Foo{abbr.capitalize()}Bar", type="baz",
@@ -776,7 +765,7 @@ def test_property_doc_abbreviations_ignored_if_within_word(generate_classes, abb
         schema.Class("Object", properties=[
             schema.SingleProperty(f"foo_{abbr}acadabra_bar", "baz")]),
     ]) == {
-        "Object.qll": (a_ql_stub(name="Object", base_import=gen_import_prefix + "Object"),
+        "Object.qll": (a_ql_stub(name="Object"),
                        a_ql_class(name="Object", final=True,
                                   properties=[
                                       ql.Property(singular=f"Foo{abbr.capitalize()}acadabraBar", type="baz",
@@ -792,7 +781,7 @@ def test_repeated_property_doc_override_with_format(generate_classes):
             schema.RepeatedProperty("x", "int", doc="special {children} of this"),
             schema.RepeatedOptionalProperty("y", "int", doc="special {child} of this")]),
     ]) == {
-        "MyObject.qll": (a_ql_stub(name="MyObject", base_import=gen_import_prefix + "MyObject"),
+        "MyObject.qll": (a_ql_stub(name="MyObject"),
                          a_ql_class(name="MyObject", final=True,
                                     properties=[
                                         ql.Property(singular="X", plural="Xes", type="int",
@@ -815,7 +804,7 @@ def test_repeated_property_doc_override_with_multiple_formats(generate_classes):
             schema.RepeatedProperty("x", "int", doc="{cat} or {dog}"),
             schema.RepeatedOptionalProperty("y", "int", doc="{cats} or {dogs}")]),
     ]) == {
-        "MyObject.qll": (a_ql_stub(name="MyObject", base_import=gen_import_prefix + "MyObject"),
+        "MyObject.qll": (a_ql_stub(name="MyObject"),
                          a_ql_class(name="MyObject", final=True,
                                     properties=[
                                         ql.Property(singular="X", plural="Xes", type="int",
@@ -835,7 +824,7 @@ def test_property_doc_override_with_format(generate_classes):
         schema.Class("MyObject", properties=[
             schema.SingleProperty("foo", "bar", doc="special {baz} of this")]),
     ]) == {
-        "MyObject.qll": (a_ql_stub(name="MyObject", base_import=gen_import_prefix + "MyObject"),
+        "MyObject.qll": (a_ql_stub(name="MyObject"),
                          a_ql_class(name="MyObject", final=True,
                                     properties=[
                                         ql.Property(singular="Foo", type="bar", tablename="my_objects",
@@ -850,7 +839,7 @@ def test_property_on_class_with_default_doc_name(generate_classes):
             schema.SingleProperty("foo", "bar")],
             default_doc_name="baz"),
     ]) == {
-        "MyObject.qll": (a_ql_stub(name="MyObject", base_import=gen_import_prefix + "MyObject"),
+        "MyObject.qll": (a_ql_stub(name="MyObject"),
                          a_ql_class(name="MyObject", final=True,
                                     properties=[
                                         ql.Property(singular="Foo", type="bar", tablename="my_objects",
@@ -863,7 +852,7 @@ def test_stub_on_class_with_ipa_from_class(generate_classes):
     assert generate_classes([
         schema.Class("MyObject", ipa=schema.IpaInfo(from_class="A")),
     ]) == {
-        "MyObject.qll": (a_ql_stub(name="MyObject", base_import=gen_import_prefix + "MyObject", ipa_accessors=[
+        "MyObject.qll": (a_ql_stub(name="MyObject", ipa_accessors=[
             ql.IpaUnderlyingAccessor(argument="Entity", type="Raw::A", constructorparams=["result"]),
         ]),
             a_ql_class(name="MyObject", final=True, ipa=True)),
@@ -874,7 +863,7 @@ def test_stub_on_class_with_ipa_on_arguments(generate_classes):
     assert generate_classes([
         schema.Class("MyObject", ipa=schema.IpaInfo(on_arguments={"base": "A", "index": "int", "label": "string"})),
     ]) == {
-        "MyObject.qll": (a_ql_stub(name="MyObject", base_import=gen_import_prefix + "MyObject", ipa_accessors=[
+        "MyObject.qll": (a_ql_stub(name="MyObject", ipa_accessors=[
             ql.IpaUnderlyingAccessor(argument="Base", type="Raw::A", constructorparams=["result", "_", "_"]),
             ql.IpaUnderlyingAccessor(argument="Index", type="int", constructorparams=["_", "result", "_"]),
             ql.IpaUnderlyingAccessor(argument="Label", type="string", constructorparams=["_", "_", "result"]),
@@ -883,5 +872,30 @@ def test_stub_on_class_with_ipa_on_arguments(generate_classes):
     }
 
 
+def test_hideable_class(generate_classes):
+    assert generate_classes([
+        schema.Class("MyObject", hideable=True),
+    ]) == {
+        "MyObject.qll": (a_ql_stub(name="MyObject"), a_ql_class(name="MyObject", final=True, hideable=True)),
+    }
+
+
+def test_hideable_property(generate_classes):
+    assert generate_classes([
+        schema.Class("MyObject", hideable=True),
+        schema.Class("Other", properties=[
+            schema.SingleProperty("x", "MyObject"),
+        ]),
+    ]) == {
+        "MyObject.qll": (a_ql_stub(name="MyObject"), a_ql_class(name="MyObject", final=True, hideable=True)),
+        "Other.qll": (a_ql_stub(name="Other"),
+                      a_ql_class(name="Other", imports=[stub_import_prefix + "MyObject"],
+                                 final=True, properties=[
+                          ql.Property(singular="X", type="MyObject", tablename="others", type_is_hideable=True,
+                                      tableparams=["this", "result"], doc="x of this other"),
+                      ])),
+    }
+
+
 if __name__ == '__main__':
     sys.exit(pytest.main([__file__] + sys.argv[1:]))

misc/codegen/test/test_schemaloader.py

@@ -688,5 +688,33 @@ def test_uppercase_acronyms_are_rejected():
                 pass
 
 
+def test_hideable():
+    @load
+    class data:
+        class Root:
+            pass
+
+        @defs.ql.hideable
+        class A(Root):
+            pass
+
+        class IndirectlyHideable(Root):
+            pass
+
+        class B(A, IndirectlyHideable):
+            pass
+
+        class NonHideable(Root):
+            pass
+
+    assert data.classes == {
+        "Root": schema.Class("Root", derived={"A", "IndirectlyHideable", "NonHideable"}, hideable=True),
+        "A": schema.Class("A", bases=["Root"], derived={"B"}, hideable=True),
+        "IndirectlyHideable": schema.Class("IndirectlyHideable", bases=["Root"], derived={"B"}, hideable=True),
+        "B": schema.Class("B", bases=["A", "IndirectlyHideable"], hideable=True),
+        "NonHideable": schema.Class("NonHideable", bases=["Root"], hideable=False),
+    }
+
+
 if __name__ == '__main__':
     sys.exit(pytest.main([__file__] + sys.argv[1:]))

misc/suite-helpers/CHANGELOG.md

@@ -1,3 +1,7 @@
+## 0.5.2
+
+No user-facing changes.
+
 ## 0.5.1
 
 No user-facing changes.

misc/suite-helpers/change-notes/released/0.5.2.md

@@ -0,0 +1,3 @@
+## 0.5.2
+
+No user-facing changes.

misc/suite-helpers/codeql-pack.release.yml

@@ -1,2 +1,2 @@
 ---
-lastReleaseVersion: 0.5.1
+lastReleaseVersion: 0.5.2

misc/suite-helpers/qlpack.yml

@@ -1,3 +1,3 @@
 name: codeql/suite-helpers
-version: 0.5.2-dev
+version: 0.5.3-dev
 groups: shared

python/ql/lib/CHANGELOG.md

@@ -1,3 +1,10 @@
+## 0.9.2
+
+### Minor Analysis Improvements
+
+* Type tracking is now aware of reads of captured variables (variables defined in an outer scope). This leads to a richer API graph, and may lead to more results in some queries.
+* Added more content-flow/field-flow for dictionaries, by adding support for reads through `mydict.get("key")` and `mydict.setdefault("key", value)`, and store steps through `dict["key"] = value` and `mydict.setdefault("key", value)`.
+
 ## 0.9.1
 
 ### Minor Analysis Improvements

python/ql/lib/change-notes/2023-03-16-typetracking-read-captured-variables.md

@@ -1,4 +0,0 @@
----
-category: minorAnalysis
----
-* Type tracking is now aware of reads of captured variables (variables defined in an outer scope). This leads to a richer API graph, and may lead to more results in some queries.

python/ql/lib/change-notes/released/0.9.2.md

@@ -1,4 +1,6 @@
----
+## 0.9.2
-category: minorAnalysis
+
----
+### Minor Analysis Improvements
+
+* Type tracking is now aware of reads of captured variables (variables defined in an outer scope). This leads to a richer API graph, and may lead to more results in some queries.
 * Added more content-flow/field-flow for dictionaries, by adding support for reads through `mydict.get("key")` and `mydict.setdefault("key", value)`, and store steps through `dict["key"] = value` and `mydict.setdefault("key", value)`.

python/ql/lib/codeql-pack.release.yml

@@ -1,2 +1,2 @@
 ---
-lastReleaseVersion: 0.9.1
+lastReleaseVersion: 0.9.2

python/ql/lib/qlpack.yml

@@ -1,5 +1,5 @@
 name: codeql/python-all
-version: 0.9.2-dev
+version: 0.9.3-dev
 groups: python
 dbscheme: semmlecode.python.dbscheme
 extractor: python

python/ql/src/CHANGELOG.md

@@ -1,3 +1,7 @@
+## 0.7.2
+
+No user-facing changes.
+
 ## 0.7.1
 
 No user-facing changes.

python/ql/src/change-notes/released/0.7.2.md

@@ -0,0 +1,3 @@
+## 0.7.2
+
+No user-facing changes.

python/ql/src/codeql-pack.release.yml

@@ -1,2 +1,2 @@
 ---
-lastReleaseVersion: 0.7.1
+lastReleaseVersion: 0.7.2

python/ql/src/qlpack.yml

@@ -1,5 +1,5 @@
 name: codeql/python-queries
-version: 0.7.2-dev
+version: 0.7.3-dev
 groups: 
   - python
   - queries

ruby/ql/lib/CHANGELOG.md

@@ -1,3 +1,9 @@
+## 0.6.2
+
+### Minor Analysis Improvements
+
+* Support for the `sqlite3` gem has been added. Method calls that execute queries against an SQLite3 database that may be vulnerable to injection attacks will now be recognized.
+
 ## 0.6.1
 
 No user-facing changes.

ruby/ql/lib/change-notes/released/0.6.2.md

@@ -1,4 +1,5 @@
----
+## 0.6.2
-category: minorAnalysis
+
----
+### Minor Analysis Improvements
+
 * Support for the `sqlite3` gem has been added. Method calls that execute queries against an SQLite3 database that may be vulnerable to injection attacks will now be recognized.

ruby/ql/lib/codeql-pack.release.yml

@@ -1,2 +1,2 @@
 ---
-lastReleaseVersion: 0.6.1
+lastReleaseVersion: 0.6.2

ruby/ql/lib/qlpack.yml

@@ -1,5 +1,5 @@
 name: codeql/ruby-all
-version: 0.6.2-dev
+version: 0.6.3-dev
 groups: ruby
 extractor: ruby
 dbscheme: ruby.dbscheme

ruby/ql/src/CHANGELOG.md

@@ -1,3 +1,7 @@
+## 0.6.2
+
+No user-facing changes.
+
 ## 0.6.1
 
 No user-facing changes.

ruby/ql/src/change-notes/released/0.6.2.md

@@ -0,0 +1,3 @@
+## 0.6.2
+
+No user-facing changes.

ruby/ql/src/codeql-pack.release.yml

@@ -1,2 +1,2 @@
 ---
-lastReleaseVersion: 0.6.1
+lastReleaseVersion: 0.6.2

ruby/ql/src/qlpack.yml

@@ -1,5 +1,5 @@
 name: codeql/ruby-queries
-version: 0.6.2-dev
+version: 0.6.3-dev
 groups: 
   - ruby
   - queries

shared/regex/CHANGELOG.md

@@ -1,3 +1,7 @@
+## 0.0.13
+
+No user-facing changes.
+
 ## 0.0.12
 
 No user-facing changes.