Merge pull request #15834 from github/post-release-prep/codeql-cli-2.16.4

Post-release preparation for codeql-cli-2.16.4

cpp/ql/lib/CHANGELOG.md

@@ -1,3 +1,9 @@
+## 0.12.7
+
+### Minor Analysis Improvements
+
+* Added destructors for named objects to the intermediate representation.
+
 ## 0.12.6
 
 ### New Features

cpp/ql/lib/change-notes/released/0.12.7.md

@@ -1,4 +1,5 @@
----
-category: minorAnalysis
----
-* Added destructors for named objects to the intermediate representation.
+## 0.12.7
+
+### Minor Analysis Improvements
+
+* Added destructors for named objects to the intermediate representation.

cpp/ql/lib/codeql-pack.release.yml

@@ -1,2 +1,2 @@
 ---
-lastReleaseVersion: 0.12.6
+lastReleaseVersion: 0.12.7

cpp/ql/lib/qlpack.yml

@@ -1,5 +1,5 @@
 name: codeql/cpp-all
-version: 0.12.7-dev
+version: 0.12.8-dev
 groups: cpp
 dbscheme: semmlecode.cpp.dbscheme
 extractor: cpp

cpp/ql/src/CHANGELOG.md

@@ -1,3 +1,10 @@
+## 0.9.6
+
+### Minor Analysis Improvements
+
+* The "non-constant format string" query (`cpp/non-constant-format`) has been converted to a `path-problem` query.
+* The new C/C++ dataflow and taint-tracking libraries (`semmle.code.cpp.dataflow.new.DataFlow` and `semmle.code.cpp.dataflow.new.TaintTracking`) now implicitly assume that dataflow and taint modelled via `DataFlowFunction` and `TaintFunction` always fully overwrite their buffers and thus act as flow barriers. As a result, many dataflow and taint-tracking queries now produce fewer false positives. To remove this assumption and go back to the previous behavior for a given model, one can override the new `isPartialWrite` predicate.
+
 ## 0.9.5
 
 ### Minor Analysis Improvements

cpp/ql/src/change-notes/2024-02-29-non-constant-format-path-query.md

@@ -1,4 +0,0 @@
----
-category: minorAnalysis
----
-* The "non-constant format string" query (`cpp/non-constant-format`) has been converted to a `path-problem` query.

cpp/ql/src/change-notes/released/0.9.6.md

@@ -1,4 +1,6 @@
----
-category: minorAnalysis
----
+## 0.9.6
+
+### Minor Analysis Improvements
+
+* The "non-constant format string" query (`cpp/non-constant-format`) has been converted to a `path-problem` query.
 * The new C/C++ dataflow and taint-tracking libraries (`semmle.code.cpp.dataflow.new.DataFlow` and `semmle.code.cpp.dataflow.new.TaintTracking`) now implicitly assume that dataflow and taint modelled via `DataFlowFunction` and `TaintFunction` always fully overwrite their buffers and thus act as flow barriers. As a result, many dataflow and taint-tracking queries now produce fewer false positives. To remove this assumption and go back to the previous behavior for a given model, one can override the new `isPartialWrite` predicate.

cpp/ql/src/codeql-pack.release.yml

@@ -1,2 +1,2 @@
 ---
-lastReleaseVersion: 0.9.5
+lastReleaseVersion: 0.9.6

cpp/ql/src/qlpack.yml

@@ -1,5 +1,5 @@
 name: codeql/cpp-queries
-version: 0.9.6-dev
+version: 0.9.7-dev
 groups:
   - cpp
   - queries

csharp/ql/campaigns/Solorigate/lib/CHANGELOG.md

@@ -1,3 +1,7 @@
+## 1.7.10
+
+No user-facing changes.
+
 ## 1.7.9
 
 No user-facing changes.

csharp/ql/campaigns/Solorigate/lib/change-notes/released/1.7.10.md

@@ -0,0 +1,3 @@
+## 1.7.10
+
+No user-facing changes.

csharp/ql/campaigns/Solorigate/lib/codeql-pack.release.yml

@@ -1,2 +1,2 @@
 ---
-lastReleaseVersion: 1.7.9
+lastReleaseVersion: 1.7.10

csharp/ql/campaigns/Solorigate/lib/qlpack.yml

@@ -1,5 +1,5 @@
 name: codeql/csharp-solorigate-all
-version: 1.7.10-dev
+version: 1.7.11-dev
 groups:
   - csharp
   - solorigate

csharp/ql/campaigns/Solorigate/src/CHANGELOG.md

@@ -1,3 +1,7 @@
+## 1.7.10
+
+No user-facing changes.
+
 ## 1.7.9
 
 No user-facing changes.

csharp/ql/campaigns/Solorigate/src/change-notes/released/1.7.10.md

@@ -0,0 +1,3 @@
+## 1.7.10
+
+No user-facing changes.

csharp/ql/campaigns/Solorigate/src/codeql-pack.release.yml

@@ -1,2 +1,2 @@
 ---
-lastReleaseVersion: 1.7.9
+lastReleaseVersion: 1.7.10

csharp/ql/campaigns/Solorigate/src/qlpack.yml

@@ -1,5 +1,5 @@
 name: codeql/csharp-solorigate-queries
-version: 1.7.10-dev
+version: 1.7.11-dev
 groups:
   - csharp
   - solorigate

csharp/ql/lib/CHANGELOG.md

@@ -1,3 +1,17 @@
+## 0.8.10
+
+### Major Analysis Improvements
+
+* Improved support for flow through captured variables that properly adheres to inter-procedural control flow.
+* We no longer make use of CodeQL database stats, which may affect join-orders in custom queries. It is therefore recommended to test performance of custom queries after upgrading to this version.
+
+### Minor Analysis Improvements
+
+* C# 12: Add QL library support (`ExperimentalAttribute`) for the experimental attribute.
+* C# 12: Add extractor and QL library support for `ref readonly` parameters.
+* C#: The table `expr_compiler_generated` has been deleted and its content has been added to `compiler_generated`.
+* Data flow via get only properties like `public object Obj { get; }` is now captured by the data flow library.
+
 ## 0.8.9
 
 ### Minor Analysis Improvements

csharp/ql/lib/change-notes/2024-02-21-getonly-properties.md

@@ -1,4 +0,0 @@
----
-category: minorAnalysis
----
-* Data flow via get only properties like `public object Obj { get; }` is now captured by the data flow library.

csharp/ql/lib/change-notes/2024-02-22-no-db-stats.md

@@ -1,4 +0,0 @@
----
-category: majorAnalysis
----
-* We no longer make use of CodeQL database stats, which may affect join-orders in custom queries. It is therefore recommended to test performance of custom queries after upgrading to this version.

csharp/ql/lib/change-notes/2024-02-23-compiler-generated.md

@@ -1,4 +0,0 @@
----
-category: minorAnalysis
----
-* C#: The table `expr_compiler_generated` has been deleted and its content has been added to `compiler_generated`.

csharp/ql/lib/change-notes/2024-02-26-variable-capture-flow.md

@@ -1,4 +0,0 @@
----
-category: majorAnalysis
----
-* Improved support for flow through captured variables that properly adheres to inter-procedural control flow.

csharp/ql/lib/change-notes/2024-02-28-experimental-attribute.md

@@ -1,4 +0,0 @@
----
-category: minorAnalysis
----
-* C# 12: Add QL library support (`ExperimentalAttribute`) for the experimental attribute.

csharp/ql/lib/change-notes/2024-02-28-refreadonly-parameter.md

@@ -1,4 +0,0 @@
----
-category: minorAnalysis
----
-* C# 12: Add extractor and QL library support for `ref readonly` parameters.

csharp/ql/lib/change-notes/released/0.8.10.md

@@ -0,0 +1,13 @@
+## 0.8.10
+
+### Major Analysis Improvements
+
+* Improved support for flow through captured variables that properly adheres to inter-procedural control flow.
+* We no longer make use of CodeQL database stats, which may affect join-orders in custom queries. It is therefore recommended to test performance of custom queries after upgrading to this version.
+
+### Minor Analysis Improvements
+
+* C# 12: Add QL library support (`ExperimentalAttribute`) for the experimental attribute.
+* C# 12: Add extractor and QL library support for `ref readonly` parameters.
+* C#: The table `expr_compiler_generated` has been deleted and its content has been added to `compiler_generated`.
+* Data flow via get only properties like `public object Obj { get; }` is now captured by the data flow library.

csharp/ql/lib/codeql-pack.release.yml

@@ -1,2 +1,2 @@
 ---
-lastReleaseVersion: 0.8.9
+lastReleaseVersion: 0.8.10

csharp/ql/lib/qlpack.yml

@@ -1,5 +1,5 @@
 name: codeql/csharp-all
-version: 0.8.10-dev
+version: 0.8.11-dev
 groups: csharp
 dbscheme: semmlecode.csharp.dbscheme
 extractor: csharp

csharp/ql/src/CHANGELOG.md

@@ -1,3 +1,9 @@
+## 0.8.10
+
+### Minor Analysis Improvements
+
+* Most data flow queries that track flow from *remote* flow sources now use the current *threat model* configuration instead. This doesn't lead to any changes in the produced alerts (as the default configuration is *remote* flow sources) unless the threat model configuration is changed. The changed queries are `cs/code-injection`, `cs/command-line-injection`, `cs/user-controlled-bypass`, `cs/count-untrusted-data-external-api`, `cs/untrusted-data-to-external-api`, `cs/ldap-injection`, `cs/log-forging`, `cs/xml/missing-validation`, `cs/redos`, `cs/regex-injection`, `cs/resource-injection`, `cs/sql-injection`, `cs/path-injection`, `cs/unsafe-deserialization-untrusted-input`, `cs/web/unvalidated-url-redirection`, `cs/xml/insecure-dtd-handling`, `cs/xml/xpath-injection`, `cs/web/xss`, and `cs/uncontrolled-format-string`.
+
 ## 0.8.9
 
 ### Minor Analysis Improvements

csharp/ql/src/change-notes/released/0.8.10.md

@@ -1,4 +1,5 @@
----
-category: minorAnalysis
----
-* Most data flow queries that track flow from *remote* flow sources now use the current *threat model* configuration instead. This doesn't lead to any changes in the produced alerts (as the default configuration is *remote* flow sources) unless the threat model configuration is changed. The changed queries are `cs/code-injection`, `cs/command-line-injection`, `cs/user-controlled-bypass`, `cs/count-untrusted-data-external-api`, `cs/untrusted-data-to-external-api`, `cs/ldap-injection`, `cs/log-forging`, `cs/xml/missing-validation`, `cs/redos`, `cs/regex-injection`, `cs/resource-injection`, `cs/sql-injection`, `cs/path-injection`, `cs/unsafe-deserialization-untrusted-input`, `cs/web/unvalidated-url-redirection`, `cs/xml/insecure-dtd-handling`, `cs/xml/xpath-injection`, `cs/web/xss`, and `cs/uncontrolled-format-string`.
+## 0.8.10
+
+### Minor Analysis Improvements
+
+* Most data flow queries that track flow from *remote* flow sources now use the current *threat model* configuration instead. This doesn't lead to any changes in the produced alerts (as the default configuration is *remote* flow sources) unless the threat model configuration is changed. The changed queries are `cs/code-injection`, `cs/command-line-injection`, `cs/user-controlled-bypass`, `cs/count-untrusted-data-external-api`, `cs/untrusted-data-to-external-api`, `cs/ldap-injection`, `cs/log-forging`, `cs/xml/missing-validation`, `cs/redos`, `cs/regex-injection`, `cs/resource-injection`, `cs/sql-injection`, `cs/path-injection`, `cs/unsafe-deserialization-untrusted-input`, `cs/web/unvalidated-url-redirection`, `cs/xml/insecure-dtd-handling`, `cs/xml/xpath-injection`, `cs/web/xss`, and `cs/uncontrolled-format-string`.

csharp/ql/src/codeql-pack.release.yml

@@ -1,2 +1,2 @@
 ---
-lastReleaseVersion: 0.8.9
+lastReleaseVersion: 0.8.10

csharp/ql/src/qlpack.yml

@@ -1,5 +1,5 @@
 name: codeql/csharp-queries
-version: 0.8.10-dev
+version: 0.8.11-dev
 groups:
   - csharp
   - queries

go/extractor/cli/go-autobuilder/go-autobuilder.go

@@ -3,7 +3,6 @@ package main
 import (
 	"fmt"
 	"log"
-	"net/url"
 	"os"
 	"os/exec"
 	"path/filepath"
@@ -56,63 +55,6 @@ Build behavior:
 	fmt.Fprintf(os.Stderr, "Usage:\n\n  %s\n", os.Args[0])
 }
 
-// Returns the import path of the package being built, or "" if it cannot be determined.
-func getImportPath() (importpath string) {
-	importpath = os.Getenv("LGTM_INDEX_IMPORT_PATH")
-	if importpath == "" {
-		repourl := os.Getenv("SEMMLE_REPO_URL")
-		if repourl == "" {
-			githubrepo := os.Getenv("GITHUB_REPOSITORY")
-			if githubrepo == "" {
-				log.Printf("Unable to determine import path, as neither LGTM_INDEX_IMPORT_PATH nor GITHUB_REPOSITORY is set\n")
-				return ""
-			} else {
-				importpath = "github.com/" + githubrepo
-			}
-		} else {
-			importpath = getImportPathFromRepoURL(repourl)
-			if importpath == "" {
-				log.Printf("Failed to determine import path from SEMMLE_REPO_URL '%s'\n", repourl)
-				return
-			}
-		}
-	}
-	log.Printf("Import path is '%s'\n", importpath)
-	return
-}
-
-// Returns the import path of the package being built from `repourl`, or "" if it cannot be
-// determined.
-func getImportPathFromRepoURL(repourl string) string {
-	// check for scp-like URL as in "git@github.com:github/codeql-go.git"
-	shorturl := regexp.MustCompile(`^([^@]+@)?([^:]+):([^/].*?)(\.git)?$`)
-	m := shorturl.FindStringSubmatch(repourl)
-	if m != nil {
-		return m[2] + "/" + m[3]
-	}
-
-	// otherwise parse as proper URL
-	u, err := url.Parse(repourl)
-	if err != nil {
-		log.Fatalf("Malformed repository URL '%s'\n", repourl)
-	}
-
-	if u.Scheme == "file" {
-		// we can't determine import paths from file paths
-		return ""
-	}
-
-	if u.Hostname() == "" || u.Path == "" {
-		return ""
-	}
-
-	host := u.Hostname()
-	path := u.Path
-	// strip off leading slashes and trailing `.git` if present
-	path = regexp.MustCompile(`^/+|\.git$`).ReplaceAllString(path, "")
-	return host + "/" + path
-}
-
 func restoreRepoLayout(fromDir string, dirEntries []string, scratchDirName string, toDir string) {
 	for _, dirEntry := range dirEntries {
 		if dirEntry != scratchDirName {
@@ -568,7 +510,7 @@ func installDependenciesAndBuild() {
 	if len(workspaces) == 1 {
 		workspace := workspaces[0]
 
-		importpath := getImportPath()
+		importpath := util.GetImportPath()
 		needGopath := getNeedGopath(workspace, importpath)
 
 		inLGTM := os.Getenv("LGTM_SRC") != "" || os.Getenv("LGTM_INDEX_NEED_GOPATH") != ""

go/extractor/project/project.go

@@ -439,8 +439,9 @@ func getBuildRoots(emitDiagnostics bool) (goWorkspaces []GoWorkspace, totalModul
 			for _, component := range components {
 				path = filepath.Join(path, component)
 
-				// Try to initialize a `go.mod` file automatically for the stray source files.
-				if !slices.Contains(goModDirs, path) {
+				// Try to initialize a `go.mod` file automatically for the stray source files if
+				// doing so would not place it in a parent directory of an existing `go.mod` file.
+				if !startsWithAnyOf(path, goModDirs) {
 					goWorkspaces = append(goWorkspaces, GoWorkspace{
 						BaseDir: path,
 						DepMode: GoGetNoModules,
@@ -477,6 +478,16 @@ func getBuildRoots(emitDiagnostics bool) (goWorkspaces []GoWorkspace, totalModul
 	return
 }
 
+// Determines whether `str` starts with any of `prefixes`.
+func startsWithAnyOf(str string, prefixes []string) bool {
+	for _, prefix := range prefixes {
+		if relPath, err := filepath.Rel(str, prefix); err == nil && !strings.HasPrefix(relPath, "..") {
+			return true
+		}
+	}
+	return false
+}
+
 // Finds Go workspaces in the current working directory.
 func GetWorkspaceInfo(emitDiagnostics bool) []GoWorkspace {
 	bazelPaths := slices.Concat(

go/extractor/project/project_test.go

@@ -0,0 +1,27 @@
+package project
+
+import (
+	"path/filepath"
+	"testing"
+)
+
+func testStartsWithAnyOf(t *testing.T, path string, prefix string, expectation bool) {
+	result := startsWithAnyOf(path, []string{prefix})
+	if result != expectation {
+		t.Errorf("Expected startsWithAnyOf(%s, %s) to be %t, but it is %t.", path, prefix, expectation, result)
+	}
+}
+
+func TestStartsWithAnyOf(t *testing.T) {
+	testStartsWithAnyOf(t, ".", ".", true)
+	testStartsWithAnyOf(t, ".", "dir", true)
+	testStartsWithAnyOf(t, ".", filepath.Join("foo", "bar"), true)
+	testStartsWithAnyOf(t, "dir", "dir", true)
+	testStartsWithAnyOf(t, "foo", filepath.Join("foo", "bar"), true)
+	testStartsWithAnyOf(t, filepath.Join("foo", "bar"), filepath.Join("foo", "bar"), true)
+	testStartsWithAnyOf(t, filepath.Join("foo", "bar"), filepath.Join("foo", "bar", "baz"), true)
+
+	testStartsWithAnyOf(t, filepath.Join("foo", "bar"), "foo", false)
+	testStartsWithAnyOf(t, filepath.Join("foo", "bar"), "bar", false)
+	testStartsWithAnyOf(t, filepath.Join("foo", "bar"), filepath.Join("foo", "baz"), false)
+}

go/extractor/toolchain/toolchain.go

@@ -5,8 +5,10 @@ import (
 	"log"
 	"os"
 	"os/exec"
+	"path/filepath"
 	"strings"
 
+	"github.com/github/codeql-go/extractor/util"
 	"golang.org/x/mod/semver"
 )
 
@@ -81,7 +83,20 @@ func TidyModule(path string) *exec.Cmd {
 
 // Run `go mod init` in the directory given by `path`.
 func InitModule(path string) *exec.Cmd {
-	modInit := exec.Command("go", "mod", "init", "codeql/auto-project")
+	moduleName := "codeql/auto-project"
+
+	if importpath := util.GetImportPath(); importpath != "" {
+		// This should be something like `github.com/user/repo`
+		moduleName = importpath
+
+		// If we are not initialising the new module in the root directory of the workspace,
+		// append the relative path to the module name.
+		if relPath, err := filepath.Rel(".", path); err != nil && relPath != "." {
+			moduleName = moduleName + "/" + relPath
+		}
+	}
+
+	modInit := exec.Command("go", "mod", "init", moduleName)
 	modInit.Dir = path
 	return modInit
 }

go/extractor/util/util.go

@@ -6,9 +6,11 @@ import (
 	"io"
 	"io/fs"
 	"log"
+	"net/url"
 	"os"
 	"os/exec"
 	"path/filepath"
+	"regexp"
 	"runtime"
 	"slices"
 	"strings"
@@ -350,3 +352,60 @@ func GetParentDirs(paths []string) []string {
 	}
 	return dirs
 }
+
+// Returns the import path of the package being built, or "" if it cannot be determined.
+func GetImportPath() (importpath string) {
+	importpath = os.Getenv("LGTM_INDEX_IMPORT_PATH")
+	if importpath == "" {
+		repourl := os.Getenv("SEMMLE_REPO_URL")
+		if repourl == "" {
+			githubrepo := os.Getenv("GITHUB_REPOSITORY")
+			if githubrepo == "" {
+				log.Printf("Unable to determine import path, as neither LGTM_INDEX_IMPORT_PATH nor GITHUB_REPOSITORY is set\n")
+				return ""
+			} else {
+				importpath = "github.com/" + githubrepo
+			}
+		} else {
+			importpath = getImportPathFromRepoURL(repourl)
+			if importpath == "" {
+				log.Printf("Failed to determine import path from SEMMLE_REPO_URL '%s'\n", repourl)
+				return
+			}
+		}
+	}
+	log.Printf("Import path is '%s'\n", importpath)
+	return
+}
+
+// Returns the import path of the package being built from `repourl`, or "" if it cannot be
+// determined.
+func getImportPathFromRepoURL(repourl string) string {
+	// check for scp-like URL as in "git@github.com:github/codeql-go.git"
+	shorturl := regexp.MustCompile(`^([^@]+@)?([^:]+):([^/].*?)(\.git)?$`)
+	m := shorturl.FindStringSubmatch(repourl)
+	if m != nil {
+		return m[2] + "/" + m[3]
+	}
+
+	// otherwise parse as proper URL
+	u, err := url.Parse(repourl)
+	if err != nil {
+		log.Fatalf("Malformed repository URL '%s'\n", repourl)
+	}
+
+	if u.Scheme == "file" {
+		// we can't determine import paths from file paths
+		return ""
+	}
+
+	if u.Hostname() == "" || u.Path == "" {
+		return ""
+	}
+
+	host := u.Hostname()
+	path := u.Path
+	// strip off leading slashes and trailing `.git` if present
+	path = regexp.MustCompile(`^/+|\.git$`).ReplaceAllString(path, "")
+	return host + "/" + path
+}

go/extractor/util/util_test.go

@@ -1,4 +1,4 @@
-package main
+package util
 
 import "testing"
 

go/ql/consistency-queries/CHANGELOG.md

@@ -1,3 +1,7 @@
+## 0.0.9
+
+No user-facing changes.
+
 ## 0.0.8
 
 No user-facing changes.

go/ql/consistency-queries/change-notes/released/0.0.9.md

@@ -0,0 +1,3 @@
+## 0.0.9
+
+No user-facing changes.

go/ql/consistency-queries/codeql-pack.release.yml

@@ -1,2 +1,2 @@
 ---
-lastReleaseVersion: 0.0.8
+lastReleaseVersion: 0.0.9

go/ql/consistency-queries/qlpack.yml

@@ -1,5 +1,5 @@
 name: codeql-go-consistency-queries
-version: 0.0.9-dev
+version: 0.0.10-dev
 groups:
   - go
   - queries

go/ql/integration-tests/all-platforms/go/mixed-layout/diagnostics.expected

@@ -0,0 +1,28 @@
+{
+  "markdownMessage": "1 `go.work` file was found:\n\n`workspace/go.work`",
+  "severity": "note",
+  "source": {
+    "extractorName": "go",
+    "id": "go/autobuilder/go-work-found",
+    "name": "`go.work` file found"
+  },
+  "visibility": {
+    "cliSummaryTable": false,
+    "statusPage": false,
+    "telemetry": true
+  }
+}
+{
+  "markdownMessage": "Go files were found outside of the Go modules corresponding to these `go.mod` files.\n\n`workspace/subdir/go.mod`, `module/go.mod`",
+  "severity": "note",
+  "source": {
+    "extractorName": "go",
+    "id": "go/autobuilder/go-files-outside-go-modules",
+    "name": "Go files were found outside Go modules"
+  },
+  "visibility": {
+    "cliSummaryTable": false,
+    "statusPage": false,
+    "telemetry": true
+  }
+}

go/ql/integration-tests/all-platforms/go/mixed-layout/force_sequential_test_execution

@@ -0,0 +1,2 @@
+# go get has been observed to sometimes fail when multiple tests try to simultaneously fetch the same package.
+goget

go/ql/integration-tests/all-platforms/go/mixed-layout/src/module/go.mod

@@ -0,0 +1,5 @@
+go 1.14
+
+require golang.org/x/net v0.0.0-20200505041828-1ed23360d12c
+
+module module

go/ql/integration-tests/all-platforms/go/mixed-layout/src/module/go.sum

@@ -0,0 +1,7 @@
+golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
+golang.org/x/net v0.0.0-20200505041828-1ed23360d12c h1:zJ0mtu4jCalhKg6Oaukv6iIkb+cOvDrajDH9DH46Q4M=
+golang.org/x/net v0.0.0-20200505041828-1ed23360d12c/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=
+golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
+golang.org/x/sys v0.0.0-20200323222414-85ca7c5b95cd h1:xhmwyvizuTgC2qz7ZlMluP20uW+C3Rm0FD/WLDX8884=
+golang.org/x/sys v0.0.0-20200323222414-85ca7c5b95cd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=

go/ql/integration-tests/all-platforms/go/mixed-layout/src/module/test.go

@@ -0,0 +1,13 @@
+package subdir
+
+import (
+	"fmt"
+
+	"golang.org/x/net/ipv4"
+)
+
+func test() {
+
+	header := ipv4.Header{}
+	fmt.Print(header.String())
+}

go/ql/integration-tests/all-platforms/go/mixed-layout/src/stray-files/test.go

@@ -0,0 +1,13 @@
+package subdir
+
+import (
+	"fmt"
+
+	"golang.org/x/net/ipv4"
+)
+
+func test() {
+
+	header := ipv4.Header{}
+	fmt.Print(header.String())
+}

go/ql/integration-tests/all-platforms/go/mixed-layout/src/workspace/go.work

@@ -0,0 +1,3 @@
+go 1.22.0
+
+use ./subdir

go/ql/integration-tests/all-platforms/go/mixed-layout/src/workspace/subdir/go.mod

@@ -0,0 +1,5 @@
+go 1.22.0
+
+require golang.org/x/net v0.0.0-20200505041828-1ed23360d12c
+
+module subdir

go/ql/integration-tests/all-platforms/go/mixed-layout/src/workspace/subdir/go.sum

@@ -0,0 +1,7 @@
+golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
+golang.org/x/net v0.0.0-20200505041828-1ed23360d12c h1:zJ0mtu4jCalhKg6Oaukv6iIkb+cOvDrajDH9DH46Q4M=
+golang.org/x/net v0.0.0-20200505041828-1ed23360d12c/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=
+golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
+golang.org/x/sys v0.0.0-20200323222414-85ca7c5b95cd h1:xhmwyvizuTgC2qz7ZlMluP20uW+C3Rm0FD/WLDX8884=
+golang.org/x/sys v0.0.0-20200323222414-85ca7c5b95cd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=

go/ql/integration-tests/all-platforms/go/mixed-layout/src/workspace/subdir/test.go

@@ -0,0 +1,13 @@
+package subdir
+
+import (
+	"fmt"
+
+	"golang.org/x/net/ipv4"
+)
+
+func test() {
+
+	header := ipv4.Header{}
+	fmt.Print(header.String())
+}

go/ql/integration-tests/all-platforms/go/mixed-layout/test.expected

@@ -0,0 +1,8 @@
+extractedFiles
+| src/module/go.mod:0:0:0:0 | src/module/go.mod |
+| src/module/test.go:0:0:0:0 | src/module/test.go |
+| src/stray-files/go.mod:0:0:0:0 | src/stray-files/go.mod |
+| src/stray-files/test.go:0:0:0:0 | src/stray-files/test.go |
+| src/workspace/subdir/go.mod:0:0:0:0 | src/workspace/subdir/go.mod |
+| src/workspace/subdir/test.go:0:0:0:0 | src/workspace/subdir/test.go |
+#select

go/ql/integration-tests/all-platforms/go/mixed-layout/test.py

@@ -0,0 +1,18 @@
+import os
+import subprocess
+
+from create_database_utils import *
+from diagnostics_test_utils import *
+
+# Set up a GOPATH relative to this test's root directory;
+# we set os.environ instead of using extra_env because we
+# need it to be set for the call to "go clean -modcache" later
+goPath = os.path.join(os.path.abspath(os.getcwd()), ".go")
+os.environ['GOPATH'] = goPath
+run_codeql_database_create([], lang="go", source="src")
+
+check_diagnostics()
+
+# Clean up the temporary GOPATH to prevent Bazel failures next
+# time the tests are run; see https://github.com/golang/go/issues/27161
+subprocess.call(["go", "clean", "-modcache"])

go/ql/integration-tests/all-platforms/go/mixed-layout/test.ql

@@ -0,0 +1,8 @@
+import go
+import semmle.go.DiagnosticsReporting
+
+query predicate extractedFiles(File f) { any() }
+
+from string msg, int sev
+where reportableDiagnostics(_, msg, sev)
+select msg, sev

go/ql/integration-tests/all-platforms/go/single-go-mod-and-go-files-not-under-it/test.expected

@@ -1,6 +1,4 @@
 extractedFiles
-| src/go.mod:0:0:0:0 | src/go.mod |
-| src/main.go:0:0:0:0 | src/main.go |
 | src/subdir/go.mod:0:0:0:0 | src/subdir/go.mod |
 | src/subdir/subsubdir/add.go:0:0:0:0 | src/subdir/subsubdir/add.go |
 | src/subdir/test.go:0:0:0:0 | src/subdir/test.go |

go/ql/lib/CHANGELOG.md

@@ -1,3 +1,14 @@
+## 0.7.10
+
+### Major Analysis Improvements
+
+* We have significantly improved the Go autobuilder to understand a greater range of project layouts, which allows Go source files to be analysed that could previously not be processed.
+* Go 1.22 has been included in the range of supported Go versions.
+
+### Bug Fixes
+
+* Fixed dataflow out of a `map` using a `range` statement.
+
 ## 0.7.9
 
 No user-facing changes.

go/ql/lib/change-notes/2024-02-14-range-map-read.md

@@ -1,4 +0,0 @@
----
-category: fix
----
-* Fixed dataflow out of a `map` using a `range` statement.

go/ql/lib/change-notes/released/0.7.10.md

@@ -1,5 +1,10 @@
----
-category: majorAnalysis
----
+## 0.7.10
+
+### Major Analysis Improvements
+
 * We have significantly improved the Go autobuilder to understand a greater range of project layouts, which allows Go source files to be analysed that could previously not be processed.
 * Go 1.22 has been included in the range of supported Go versions.
+
+### Bug Fixes
+
+* Fixed dataflow out of a `map` using a `range` statement.

go/ql/lib/codeql-pack.release.yml

@@ -1,2 +1,2 @@
 ---
-lastReleaseVersion: 0.7.9
+lastReleaseVersion: 0.7.10

go/ql/lib/qlpack.yml

@@ -1,5 +1,5 @@
 name: codeql/go-all
-version: 0.7.10-dev
+version: 0.7.11-dev
 groups: go
 dbscheme: go.dbscheme
 extractor: go

go/ql/src/CHANGELOG.md

@@ -1,3 +1,7 @@
+## 0.7.10
+
+No user-facing changes.
+
 ## 0.7.9
 
 ### New Queries

go/ql/src/change-notes/released/0.7.10.md

@@ -0,0 +1,3 @@
+## 0.7.10
+
+No user-facing changes.

go/ql/src/codeql-pack.release.yml

@@ -1,2 +1,2 @@
 ---
-lastReleaseVersion: 0.7.9
+lastReleaseVersion: 0.7.10

go/ql/src/qlpack.yml

@@ -1,5 +1,5 @@
 name: codeql/go-queries
-version: 0.7.10-dev
+version: 0.7.11-dev
 groups:
   - go
   - queries

java/ql/automodel/src/CHANGELOG.md

@@ -1,3 +1,7 @@
+## 0.0.17
+
+No user-facing changes.
+
 ## 0.0.16
 
 No user-facing changes.

java/ql/automodel/src/change-notes/released/0.0.17.md

@@ -0,0 +1,3 @@
+## 0.0.17
+
+No user-facing changes.

java/ql/automodel/src/codeql-pack.release.yml

@@ -1,2 +1,2 @@
 ---
-lastReleaseVersion: 0.0.16
+lastReleaseVersion: 0.0.17

java/ql/automodel/src/qlpack.yml

@@ -1,5 +1,5 @@
 name: codeql/java-automodel-queries
-version: 0.0.17-dev
+version: 0.0.18-dev
 groups:
     - java
     - automodel

java/ql/lib/CHANGELOG.md

@@ -1,3 +1,14 @@
+## 0.8.10
+
+### Minor Analysis Improvements
+
+* Java expressions with erroneous types (e.g. the result of a call whose callee couldn't be resolved during extraction) are now given a CodeQL `ErrorType` more often.
+
+### Bug Fixes
+
+* Fixed the Java autobuilder overriding the version of Maven used by a project when the Maven wrapper `mvnw` is in use and the `maven-wrapper.jar` file is not present in the repository.
+* Some flow steps related to `android.text.Editable.toString` that were accidentally disabled have been re-enabled.
+
 ## 0.8.9
 
 ### Deprecated APIs

java/ql/lib/change-notes/2024-02-23-widget-flowsteps.md

@@ -1,4 +0,0 @@
----
-category: fix
----
-* Some flow steps related to `android.text.Editable.toString` that were accidentally disabled have been re-enabled.

java/ql/lib/change-notes/2024-02-27-error-types.md

@@ -1,4 +0,0 @@
----
-category: minorAnalysis
----
-* Java expressions with erroneous types (e.g. the result of a call whose callee couldn't be resolved during extraction) are now given a CodeQL `ErrorType` more often.

java/ql/lib/change-notes/2024-02-27-mvnw-versions.md

@@ -1,4 +0,0 @@
----
-category: fix
----
-* Fixed the Java autobuilder overriding the version of Maven used by a project when the Maven wrapper `mvnw` is in use and the `maven-wrapper.jar` file is not present in the repository.

java/ql/lib/change-notes/released/0.8.10.md

@@ -0,0 +1,10 @@
+## 0.8.10
+
+### Minor Analysis Improvements
+
+* Java expressions with erroneous types (e.g. the result of a call whose callee couldn't be resolved during extraction) are now given a CodeQL `ErrorType` more often.
+
+### Bug Fixes
+
+* Fixed the Java autobuilder overriding the version of Maven used by a project when the Maven wrapper `mvnw` is in use and the `maven-wrapper.jar` file is not present in the repository.
+* Some flow steps related to `android.text.Editable.toString` that were accidentally disabled have been re-enabled.

java/ql/lib/codeql-pack.release.yml

@@ -1,2 +1,2 @@
 ---
-lastReleaseVersion: 0.8.9
+lastReleaseVersion: 0.8.10

java/ql/lib/qlpack.yml

@@ -1,5 +1,5 @@
 name: codeql/java-all
-version: 0.8.10-dev
+version: 0.8.11-dev
 groups: java
 dbscheme: config/semmlecode.dbscheme
 extractor: java

java/ql/src/CHANGELOG.md

@@ -1,3 +1,13 @@
+## 0.8.10
+
+### New Queries
+
+* Added a new query `java/android/insecure-local-key-gen` for finding instances of keys generated for biometric authentication in an insecure way.
+
+### Minor Analysis Improvements
+
+* To reduce the number of false positives in the query "Insertion of sensitive information into log files" (`java/sensitive-log`), variables with names that contain "null" (case-insensitively) are no longer considered sources of sensitive information.
+
 ## 0.8.9
 
 ### New Queries

java/ql/src/change-notes/2024-02-12-android-insecure-keys.md

@@ -1,4 +0,0 @@
----
-category: newQuery
----
-* Added a new query `java/android/insecure-local-key-gen` for finding instances of keys generated for biometric authentication in an insecure way.

java/ql/src/change-notes/released/0.8.10.md

@@ -1,4 +1,9 @@
----
-category: minorAnalysis
----
+## 0.8.10
+
+### New Queries
+
+* Added a new query `java/android/insecure-local-key-gen` for finding instances of keys generated for biometric authentication in an insecure way.
+
+### Minor Analysis Improvements
+
 * To reduce the number of false positives in the query "Insertion of sensitive information into log files" (`java/sensitive-log`), variables with names that contain "null" (case-insensitively) are no longer considered sources of sensitive information.

java/ql/src/codeql-pack.release.yml

@@ -1,2 +1,2 @@
 ---
-lastReleaseVersion: 0.8.9
+lastReleaseVersion: 0.8.10

java/ql/src/qlpack.yml

@@ -1,5 +1,5 @@
 name: codeql/java-queries
-version: 0.8.10-dev
+version: 0.8.11-dev
 groups:
   - java
   - queries

javascript/ql/lib/CHANGELOG.md

@@ -1,3 +1,7 @@
+## 0.8.10
+
+No user-facing changes.
+
 ## 0.8.9
 
 ### Minor Analysis Improvements

javascript/ql/lib/change-notes/released/0.8.10.md

@@ -0,0 +1,3 @@
+## 0.8.10
+
+No user-facing changes.

javascript/ql/lib/codeql-pack.release.yml

@@ -1,2 +1,2 @@
 ---
-lastReleaseVersion: 0.8.9
+lastReleaseVersion: 0.8.10

javascript/ql/lib/qlpack.yml

@@ -1,5 +1,5 @@
 name: codeql/javascript-all
-version: 0.8.10-dev
+version: 0.8.11-dev
 groups: javascript
 dbscheme: semmlecode.javascript.dbscheme
 extractor: javascript

javascript/ql/src/CHANGELOG.md

@@ -1,3 +1,7 @@
+## 0.8.10
+
+No user-facing changes.
+
 ## 0.8.9
 
 ### Bug Fixes

javascript/ql/src/change-notes/released/0.8.10.md

@@ -0,0 +1,3 @@
+## 0.8.10
+
+No user-facing changes.

javascript/ql/src/codeql-pack.release.yml

@@ -1,2 +1,2 @@
 ---
-lastReleaseVersion: 0.8.9
+lastReleaseVersion: 0.8.10

javascript/ql/src/qlpack.yml

@@ -1,5 +1,5 @@
 name: codeql/javascript-queries
-version: 0.8.10-dev
+version: 0.8.11-dev
 groups:
   - javascript
   - queries

misc/suite-helpers/CHANGELOG.md

@@ -1,3 +1,7 @@
+## 0.7.10
+
+No user-facing changes.
+
 ## 0.7.9
 
 No user-facing changes.

misc/suite-helpers/change-notes/released/0.7.10.md

@@ -0,0 +1,3 @@
+## 0.7.10
+
+No user-facing changes.

misc/suite-helpers/codeql-pack.release.yml

@@ -1,2 +1,2 @@
 ---
-lastReleaseVersion: 0.7.9
+lastReleaseVersion: 0.7.10

misc/suite-helpers/qlpack.yml

@@ -1,4 +1,4 @@
 name: codeql/suite-helpers
-version: 0.7.10-dev
+version: 0.7.11-dev
 groups: shared
 warnOnImplicitThis: true

python/ql/lib/CHANGELOG.md

@@ -1,3 +1,10 @@
+## 0.11.10
+
+### Minor Analysis Improvements
+
+* Fixed missing flow for dictionary updates (`d[<key>] = ...`) when `<key>` is a string constant not used in dictionary literals or as name of keyword-argument.
+* Fixed flow for iterable unpacking (`a,b = my_tuple`) when it occurs on top-level (module) scope.
+
 ## 0.11.9
 
 ### Minor Analysis Improvements

python/ql/lib/change-notes/2024-02-28-iterable-unpacking-module-scope.md

@@ -1,4 +0,0 @@
----
-category: minorAnalysis
----
-* Fixed flow for iterable unpacking (`a,b = my_tuple`) when it occurs on top-level (module) scope.

python/ql/lib/change-notes/released/0.11.10.md

@@ -1,4 +1,6 @@
----
-category: minorAnalysis
----
+## 0.11.10
+
+### Minor Analysis Improvements
+
 * Fixed missing flow for dictionary updates (`d[<key>] = ...`) when `<key>` is a string constant not used in dictionary literals or as name of keyword-argument.
+* Fixed flow for iterable unpacking (`a,b = my_tuple`) when it occurs on top-level (module) scope.

python/ql/lib/codeql-pack.release.yml

@@ -1,2 +1,2 @@
 ---
-lastReleaseVersion: 0.11.9
+lastReleaseVersion: 0.11.10

python/ql/lib/qlpack.yml

@@ -1,5 +1,5 @@
 name: codeql/python-all
-version: 0.11.10-dev
+version: 0.11.11-dev
 groups: python
 dbscheme: semmlecode.python.dbscheme
 extractor: python

python/ql/src/CHANGELOG.md

@@ -1,3 +1,9 @@
+## 0.9.10
+
+### New Queries
+
+* The query `py/nosql-injection` for finding NoSQL injection vulnerabilities is now part of the default security suite.
+
 ## 0.9.9
 
 No user-facing changes.

python/ql/src/change-notes/released/0.9.10.md

@@ -1,4 +1,5 @@
----
-category: newQuery
----
+## 0.9.10
+
+### New Queries
+
 * The query `py/nosql-injection` for finding NoSQL injection vulnerabilities is now part of the default security suite.

python/ql/src/codeql-pack.release.yml

@@ -1,2 +1,2 @@
 ---
-lastReleaseVersion: 0.9.9
+lastReleaseVersion: 0.9.10