Commit Graph

83 Commits

Author SHA1 Message Date
Max Schaefer
65c449cff0 Merge pull request #325 from max-schaefer/revert-237
Revert "Revert "autobuilder: Add support for GITHUB_REPOSITORY environment variable""
2020-09-08 08:04:58 +01:00
Max Schaefer
52a659183d Merge pull request #314 from smowton/smowton/admin/bump-golang-tools
Bump to latest version of golang.org/x/tools
2020-09-07 16:02:55 +01:00
Max Schaefer
655e229d1e Revert "Revert "autobuilder: Add support for GITHUB_REPOSITORY environment variable""
This reverts commit ccfccb4828.
2020-09-07 15:14:52 +01:00
Chris Smowton
47958e6de8 Go.mod comments: trim newlines
These weren't previously reported as part of the comment text, but are as of the latest version of golang.org/x/tools
2020-09-03 15:54:56 +01:00
Chris Smowton
e386346a25 Extractor: tolerate ast.File structures without a package declaration
In earlier versions of golang/x/tools these would be omitted entirely; now they can result in ast.File structures whose ast.Package field is zero (NoPos), and in my experience these contain no information in their other fields either.
2020-09-03 14:32:23 +01:00
Chris Smowton
246e8b1b27 Make failure to restore a file to its original location non-fatal 2020-09-02 11:44:43 +01:00
Chris Smowton
8de188a6ca Restore repo layout post-autobuild 2020-09-02 11:44:43 +01:00
Max Schaefer
b4550f244b Merge pull request #313 from github/rc/1.25
Merge rc/1.25 into main
2020-08-27 14:27:26 +01:00
Chris Smowton
4d084372b5 Fix autobuilder Go version comparison
The semver package requires versions of the form v1.2.3, and unhelpfully evaluates any malformed versions as equal.
2020-08-27 11:02:23 +01:00
Chris Smowton
c6dbb9fcb2 Tidy up -mod argument stringification 2020-08-27 10:46:36 +01:00
Chris Smowton
b13b54f7d7 Don't try to use -mod=... when go.mod doesn't exist
Also don't pass a blank argument to `go` when using an old version.
2020-08-26 13:56:36 +01:00
Chris Smowton
9ad2d6c119 Factor default and custom install paths
These now follow the same route:

* Run a default or custom build script
* If needed, check if vendor/ is usable
* If it isn't, or if their build failed, install dependencies using go get etc

This commit shouldn't cause any behavioural change.
2020-08-26 12:02:54 +01:00
Chris Smowton
859b427881 Check if the vendor/ directory is usable, even after a successful build 2020-08-26 11:53:50 +01:00
Sauyon Lee
8f6b25e0ac autobuilder: Use -mod=mod for vendor directories wihtout modules.txt 2020-08-26 11:25:30 +01:00
Sauyon Lee
70d425d317 autobuilder: move vendor check before dependency installation check
This means dependency installation is still attempted when a vendor
directory is inconsistent.
2020-08-26 11:25:30 +01:00
Sauyon Lee
852ae9397b autobuilder: Test for vendor inconsistency 2020-08-26 11:25:30 +01:00
Sauyon Lee
28c69743a4 Add workaround for go 1.14 explicit vendoring requirement
This only applies for module files for which no Go version has
been specified; Go will assume these should be parsed with the
latest Go version, which will cause them to fail if the vendor
directory has been generated with an old version of Go, as
the vendor/modules.txt will not meet the new requirements for
consistency.
2020-08-26 11:25:30 +01:00
Max Schaefer
bdfd1d131f Teach extractor to record the presence of an ellipsis in a call expression. 2020-08-10 07:30:23 +01:00
Sauyon Lee
8e6c1835dd autobuilder: Don't try to determine import paths for file URLs
Also improve logging
2020-08-05 23:21:34 -07:00
Chris Smowton
3ab948f81c Extractor: assign unique indices to comment-groups in go.mod files
The schema requires that (parent, index) is a key.
2020-07-13 11:28:28 +01:00
Sauyon Lee
32510eb2d0 Merge pull request #255 from max-schaefer/alias-types
Improve modelling of alias declarations
2020-07-10 21:07:48 -07:00
Max Schaefer
1a8688a8f4 Extract enough information to distinguish type definitions from alias declarations. 2020-07-10 14:12:51 +01:00
Max Schaefer
4257a68c27 Include newlines in messages printed by go-gen-dbscheme. 2020-07-10 14:08:37 +01:00
Chris Smowton
d05657ddff Make the gofmt CI test actaully fatal
Turns out gofmt doesn't actually return 1 when it finds problems, only when it finds source files which don't compile (all of which are now excluded).

This also fixes existing overlooked inconsistencies as a result of this mistake.
2020-07-10 11:02:50 +01:00
Chris Smowton
6bf3802b3f Make CommentGroups AST-children of Files
Previously they were roots, with children hanging off them. Now they are children of Files, and both CommentGroups and Comments can be discovered using AstNode.getAChild.

The PrintAst pass is also adapted to account for their new position.
2020-07-08 17:49:47 +01:00
Max Schaefer
f74a94e382 Merge pull request #170 from sauyon/tracing
Extract more dependency ASTs
2020-07-01 14:25:52 +01:00
Sauyon Lee
6e5e9ce5de Improve comments for extractor utility functions 2020-06-30 11:44:10 -07:00
Max Schaefer
ccfccb4828 Revert "autobuilder: Add support for GITHUB_REPOSITORY environment variable" 2020-06-30 15:13:26 +01:00
Chris Smowton
595866a6d8 Extractor: give the go.mod comment groups a source location (#232)
The comment group is now omitted entirely if empty, and otherwise delimits the range of the comments ascribed to this group.
2020-06-30 14:59:13 +01:00
Sauyon Lee
380060c7e4 extractor: Refactor regexp compilation for the relative directory check 2020-06-24 23:29:55 -07:00
Sauyon Lee
fa391b1516 extractor: Factor out common bits for running go list 2020-06-24 23:29:54 -07:00
Sauyon Lee
ebdd724b75 Simplify logic for deciding whether to extract a package 2020-06-24 23:29:53 -07:00
Sauyon Lee
e25b882e42 Clarify some comments
As suggested in code review

Co-authored-by: Max Schaefer <54907921+max-schaefer@users.noreply.github.com>
2020-06-24 23:29:52 -07:00
Sauyon Lee
9bd1f87d66 Address review comments 2020-06-24 23:29:51 -07:00
Sauyon Lee
7863bb656e Use the -mod argument from the build when calling go list 2020-06-24 23:29:49 -07:00
Sauyon Lee
296d2d5fd3 extractor: modify FileExists to check that the path isn't a directory 2020-06-24 23:29:48 -07:00
Sauyon Lee
3513c352e6 extractor: Factor out FileExists utility function 2020-06-24 23:29:48 -07:00
Sauyon Lee
f197975c6e Extract packages more intelligently
We now extract packages that have the same module root as the specified packages, as determined by
the `go list` command.
2020-06-24 23:29:47 -07:00
Sauyon Lee
ba0f922a28 autobuilder: Add support for GITHUB_REPOSITORY environment variable
This is for use within GitHub actions
2020-06-09 11:52:23 -07:00
Sauyon Lee
aef7524f35 Add a go-gen-dbscheme for generating dbschemes 2020-05-22 08:04:40 -07:00
Max Schaefer
97b1d3a57c Fix extraction of error positions for paths containing colon. 2020-04-20 08:04:16 +01:00
Max Schaefer
a837d5f4ce Add support for extracting Go frontend errors. 2020-04-09 16:39:47 +01:00
Sauyon Lee
a663eaae23 Merge pull request #89 from max-schaefer/extractor-profiling
Add environment variables for dumping CPU and memory profiles.
2020-04-06 03:33:08 -07:00
Max Schaefer
510b6070c9 Introduce official environment variable for goroutine limiting.
We've had to tell people how to do this, so we should have a name for it that doesn't refer to a defunct company.
2020-04-02 10:45:52 +01:00
Sauyon Lee
eba8dd0a36 Merge pull request #82 from github/max-schaefer-patch-1
Improve autobuilder logging
2020-04-01 09:07:55 -07:00
Max Schaefer
611751a9c3 Improve autobuilder logging 2020-04-01 15:31:50 +01:00
Max Schaefer
efc9ecefc8 Introduce CODEQL_GO_EXTRACTOR_BUILD_COMMAND as an alias for LGTM_INDEX_BUILD_COMMAND.
We've occasionally had to tell people to set this variable manually, so we might as well have an alias that doesn't refer to a soon-to-be obsolete product.
2020-04-01 09:35:57 +01:00
Max Schaefer
590f01d0c2 Add environment variables for dumping CPU and memory profiles.
These are intentionally undocumented for now.
2020-03-31 07:50:06 +01:00
Sauyon Lee
b64a43f578 Merge pull request #57 from max-schaefer/trap.gz
Gzip TRAP files
2020-03-12 06:24:32 -07:00
Max Schaefer
6b0ba750e6 Put gzip writer on top of bufio writer. 2020-03-12 08:40:22 +00:00