138 Commits

Author SHA1 Message Date
Nick Rolfe
5aaed8941a Go: pass source root from autobuilder to extractor
This ensures the extractor can resolve the relative paths for files
changed in the overlay.
2025-11-07 16:52:16 +00:00
Nick Rolfe
aff874e835 Go: merge with incoming path transformer when setting GOPATH 2025-11-07 16:52:12 +00:00
Nick Rolfe
05e5502680 Go: recognize CODEQL_PATH_TRANSFORMER env var 2025-11-07 16:52:07 +00:00
Michael B. Gale
8c13faf3d8 Go: Set log level based on CODEQL_VERBOSITY 2025-09-05 14:18:31 +01:00
Michael B. Gale
e805d1ee90 Merge remote-tracking branch 'origin/main' into mbg/go/private-registries 2025-04-25 12:55:36 +01:00
Paolo Tranquilli
69b87a63b8 Go: remove invalid toolchain version diagnostics
This diagnostic was introduced by https://github.com/github/codeql/pull/15979.
However in the meantime the Go team [has backtracked](https://github.com/golang/go/issues/62278#issuecomment-2062002018)
on their decision, which leads to confusing alerts for user (e.g. https://github.com/github/codeql-action/issues/2868).
Even using Go toolchains from 1.21 to 1.22 we weren't immediately able
to reproduce the problem that this diagnostics was meant to guard
against. Therefore it was deemed simpler to just remove it.

_En passant_ the `Makefile` now accepts `rtjo` not being set.
2025-04-24 14:41:05 +02:00
Michael B. Gale
ded27bcee4 Go: Replace exec.Command("go" with toolchain.GoCommand( 2025-04-08 11:19:42 +01:00
Chris Smowton
209f9ec93d Amend comments per review 2024-09-23 15:20:18 +01:00
Chris Smowton
f5ff822681 Convert extract-tests option to an official extractor option 2024-09-20 10:03:54 +01:00
Chris Smowton
76e6942594 Go: support extracting test code
This implements support for test extraction by two mechanisms:

* In autobuild mode, setting `CODEQL_EXTRACTOR_GO_EXTRACT_TESTS` to `true`.
* In manual build mode, tracing a `go test` command (`go test -c` is to be recommended for efficiency).

Go deals with test compilation by creating several extra packages on top of those expected from inspection of the source code (see docs of `packages.Load` for more detail): packages whose IDs include a suffix like `mydomain.com/mypackage [mydomain.com/mypackage.test]`, and packages containing generated test driver code like `mydomain.com/mypackage.test`. There are also additional packages like `mydomain.com/mypackage_tests` which are explicitly present in source code, but not compiled by a normal `go build`.

So far as I can tell, the purpose of the two variants of the package is to resolve dependency cycles (because the tests variant of the package can have more dependencies than the non-tests variant, and non-test code can compile against non-test package variants). Since the test package variants seems to be a superset of the non-tests variant, I employ the simple heuristic of ignoring the variant of each package with the shortest ID. I haven't seen a case where there are three or more variants of a package, so I expect this to always identify the tests variant as the preferred one. If several variants were extracted, and we were to attempt to match Golang's linkage strategy among the different variants, we would need to extend trap-file name and most top-level symbol trap IDs with the package variant they come from; I hope this won't prove necessary.

"Real" `_tests` packages, and wholly synthetic driver code packages, are extracted just like normal.
2024-09-20 09:56:28 +01:00
Chris Smowton
15b5bcc67c Output to stdout, not stderr 2024-08-20 17:01:54 +01:00
Chris Smowton
21366dd502 Go / configure-baseline: account for multiple vendor directories and the CODEQL_EXTRACTOR_GO_EXTRACT_VENDOR_DIRS setting
Our existing configure-baseline scripts would give the wrong result if a `vendor` directory wasn't at the root of the repository, or if the `CODEQL_EXTRACTOR_GO_EXTRACT_VENDOR_DIRS` variable was set to `true` indicating the user wants their vendored code scanned.

Here I replace the shell scripts that implemented the very simplest behaviour with a small Go program.
2024-08-20 15:56:22 +01:00
Michael B. Gale
e7a60b72f1 Go: Check dependencies per workspace 2024-06-07 16:22:41 +00:00
Michael B. Gale
6c0c3365cf Go: Fix vendor issues and go.sum files in separate loop 2024-06-07 16:22:39 +00:00
Michael B. Gale
d344f720aa Go: Add methods to GoModule for the tidy and vendor commands
These ensure that the module path is used automatically
2024-06-07 16:22:37 +00:00
Michael B. Gale
054efa648c Go: Move version constants to shared location 2024-05-15 10:03:35 +01:00
Michael B. Gale
a6d2aa3913 Go: Use SemVer type in project package 2024-05-15 10:03:32 +01:00
Michael B. Gale
010df54657 Go: Use SemVer type in toolchain package 2024-05-15 10:03:01 +01:00
Michael B. Gale
65c654e946 Go: Do not use a named return variable in buildWithoutCustomCommands 2024-05-13 15:33:52 +01:00
Michael B. Gale
4d5f05a8dd Go: Adjust log messages depending on whether build scripts were found or not 2024-05-13 12:42:00 +01:00
Michael B. Gale
1b9e8ae86f Go: Modify Autobuild to return an array of scripts that were run 2024-05-13 12:38:51 +01:00
Michael B. Gale
bbc359ebde Go: Improve messages in buildWithoutCustomCommands 2024-05-13 12:31:04 +01:00
Michael B. Gale
724d026238 Go: Move go list functions to toolchain package 2024-05-08 13:29:28 +01:00
Paolo Tranquilli
00baccbc15 Go: autoformat 2024-05-02 17:08:23 +02:00
Paolo Tranquilli
76067cb12d Go: skip X:nocoverageredesign printing by autobuilder built with bazel 2024-05-02 17:02:48 +02:00
Paolo Tranquilli
7d9a68bf17 Go: wrap gazelle to regenerate from scratch and add header 2024-04-26 11:19:22 +02:00
Paolo Tranquilli
4ca8faa9c9 Go: introduce universal binaries on macOS 2024-04-25 09:27:59 +02:00
Paolo Tranquilli
393f6b7666 Go: add gazelle-generated BUILD files 2024-04-25 08:53:26 +02:00
Michael B. Gale
73f71d8ace Merge pull request #15979 from github/mbg/go/deal-with-incorrect-versions 2024-03-28 14:16:14 +00:00
Michael B. Gale
be027e217e Go: Emit diagnostic for invalid toolchain versions 2024-03-19 16:26:06 +00:00
Michael B. Gale
2aa093c95c Go: Move getImportPath to shared util package 2024-03-05 12:48:19 +00:00
Michael B. Gale
52b273ae23 Go: Include workspace count in success message 2024-03-04 12:14:24 +00:00
Michael B. Gale
8055c5d9e3 Go: Avoid extra loop to track unsuccessfulProjects 2024-03-04 12:14:23 +00:00
Michael B. Gale
4d28c0d2a9 Go: Call go mod vendor to synchronise vendor directory when it exists 2024-02-15 16:19:07 +00:00
Michael B. Gale
6267506a77 Go: Postpone go.mod creation until necessary 2024-02-14 19:12:36 +00:00
Michael B. Gale
e2c673417f Go: Only call EmitNewerGoVersionNeeded at most once 2024-02-14 19:12:35 +00:00
Michael B. Gale
251888a0bd Go: Tell extractor to extract subdirectories as well 2024-02-14 19:12:33 +00:00
Michael B. Gale
843f7694fd Go: Only relocate project to temp dir if there is only one workspace 2024-02-14 19:12:32 +00:00
Michael B. Gale
f084829154 Go: Only fail autobuilder if all projects cannot be extracted 2024-02-14 19:12:31 +00:00
Michael B. Gale
fbd7946cfd Go: Fall back to ./... if there are no modules
Fixes issues for `dep` and `glide`
2024-02-14 19:12:31 +00:00
Michael B. Gale
51eb487022 Go: Handle filepath.Rel failure 2024-02-14 19:12:30 +00:00
Michael B. Gale
c96735e17a Go: Remove auto-generated go.mod files when done 2024-02-14 19:12:30 +00:00
Michael B. Gale
aa5e14f59f Go: Replace BuildInfo with GoWorkspace 2024-02-14 19:12:28 +00:00
Michael B. Gale
1bf747ef3a Go: Create go.mod file if necessary in project discovery 2024-02-14 19:12:27 +00:00
Michael B. Gale
254634075f Go: Add shared TidyModule function 2024-02-14 19:12:26 +00:00
Michael B. Gale
fc75e44238 Go: Allow GetBuildInfo to return multiple BuildInfo objects 2024-02-14 19:12:26 +00:00
Michael B. Gale
237bf5653a Go: Move getEnvGoSemVer to toolchain.go 2024-02-14 19:12:25 +00:00
Michael B. Gale
bd36847ca2 Go: Emit relative path import diagnostic if prompted by go mod tidy
The corresponding integration test now successfully extracts the project
2024-02-14 19:12:25 +00:00
Michael B. Gale
df212807a0 Go: Try to initialise go.mod file for legacy projects 2024-02-14 19:12:24 +00:00
Michael B. Gale
90eba711a3 Go: Include versions in newer Go version log message 2024-01-31 17:45:58 +00:00