Commit Graph

66 Commits

Author SHA1 Message Date
Sauyon Lee
28c69743a4 Add workaround for go 1.14 explicit vendoring requirement
This only applies for module files for which no Go version has
been specified; Go will assume these should be parsed with the
latest Go version, which will cause them to fail if the vendor
directory has been generated with an old version of Go, as
the vendor/modules.txt will not meet the new requirements for
consistency.
2020-08-26 11:25:30 +01:00
Sauyon Lee
8e6c1835dd autobuilder: Don't try to determine import paths for file URLs
Also improve logging
2020-08-05 23:21:34 -07:00
Chris Smowton
3ab948f81c Extractor: assign unique indices to comment-groups in go.mod files
The schema requires that (parent, index) is a key.
2020-07-13 11:28:28 +01:00
Sauyon Lee
32510eb2d0 Merge pull request #255 from max-schaefer/alias-types
Improve modelling of alias declarations
2020-07-10 21:07:48 -07:00
Max Schaefer
1a8688a8f4 Extract enough information to distinguish type definitions from alias declarations. 2020-07-10 14:12:51 +01:00
Max Schaefer
4257a68c27 Include newlines in messages printed by go-gen-dbscheme. 2020-07-10 14:08:37 +01:00
Chris Smowton
d05657ddff Make the gofmt CI test actaully fatal
Turns out gofmt doesn't actually return 1 when it finds problems, only when it finds source files which don't compile (all of which are now excluded).

This also fixes existing overlooked inconsistencies as a result of this mistake.
2020-07-10 11:02:50 +01:00
Chris Smowton
6bf3802b3f Make CommentGroups AST-children of Files
Previously they were roots, with children hanging off them. Now they are children of Files, and both CommentGroups and Comments can be discovered using AstNode.getAChild.

The PrintAst pass is also adapted to account for their new position.
2020-07-08 17:49:47 +01:00
Max Schaefer
f74a94e382 Merge pull request #170 from sauyon/tracing
Extract more dependency ASTs
2020-07-01 14:25:52 +01:00
Sauyon Lee
6e5e9ce5de Improve comments for extractor utility functions 2020-06-30 11:44:10 -07:00
Max Schaefer
ccfccb4828 Revert "autobuilder: Add support for GITHUB_REPOSITORY environment variable" 2020-06-30 15:13:26 +01:00
Chris Smowton
595866a6d8 Extractor: give the go.mod comment groups a source location (#232)
The comment group is now omitted entirely if empty, and otherwise delimits the range of the comments ascribed to this group.
2020-06-30 14:59:13 +01:00
Sauyon Lee
380060c7e4 extractor: Refactor regexp compilation for the relative directory check 2020-06-24 23:29:55 -07:00
Sauyon Lee
fa391b1516 extractor: Factor out common bits for running go list 2020-06-24 23:29:54 -07:00
Sauyon Lee
ebdd724b75 Simplify logic for deciding whether to extract a package 2020-06-24 23:29:53 -07:00
Sauyon Lee
e25b882e42 Clarify some comments
As suggested in code review

Co-authored-by: Max Schaefer <54907921+max-schaefer@users.noreply.github.com>
2020-06-24 23:29:52 -07:00
Sauyon Lee
9bd1f87d66 Address review comments 2020-06-24 23:29:51 -07:00
Sauyon Lee
7863bb656e Use the -mod argument from the build when calling go list 2020-06-24 23:29:49 -07:00
Sauyon Lee
296d2d5fd3 extractor: modify FileExists to check that the path isn't a directory 2020-06-24 23:29:48 -07:00
Sauyon Lee
3513c352e6 extractor: Factor out FileExists utility function 2020-06-24 23:29:48 -07:00
Sauyon Lee
f197975c6e Extract packages more intelligently
We now extract packages that have the same module root as the specified packages, as determined by
the `go list` command.
2020-06-24 23:29:47 -07:00
Sauyon Lee
ba0f922a28 autobuilder: Add support for GITHUB_REPOSITORY environment variable
This is for use within GitHub actions
2020-06-09 11:52:23 -07:00
Sauyon Lee
aef7524f35 Add a go-gen-dbscheme for generating dbschemes 2020-05-22 08:04:40 -07:00
Max Schaefer
97b1d3a57c Fix extraction of error positions for paths containing colon. 2020-04-20 08:04:16 +01:00
Max Schaefer
a837d5f4ce Add support for extracting Go frontend errors. 2020-04-09 16:39:47 +01:00
Sauyon Lee
a663eaae23 Merge pull request #89 from max-schaefer/extractor-profiling
Add environment variables for dumping CPU and memory profiles.
2020-04-06 03:33:08 -07:00
Max Schaefer
510b6070c9 Introduce official environment variable for goroutine limiting.
We've had to tell people how to do this, so we should have a name for it that doesn't refer to a defunct company.
2020-04-02 10:45:52 +01:00
Sauyon Lee
eba8dd0a36 Merge pull request #82 from github/max-schaefer-patch-1
Improve autobuilder logging
2020-04-01 09:07:55 -07:00
Max Schaefer
611751a9c3 Improve autobuilder logging 2020-04-01 15:31:50 +01:00
Max Schaefer
efc9ecefc8 Introduce CODEQL_GO_EXTRACTOR_BUILD_COMMAND as an alias for LGTM_INDEX_BUILD_COMMAND.
We've occasionally had to tell people to set this variable manually, so we might as well have an alias that doesn't refer to a soon-to-be obsolete product.
2020-04-01 09:35:57 +01:00
Max Schaefer
590f01d0c2 Add environment variables for dumping CPU and memory profiles.
These are intentionally undocumented for now.
2020-03-31 07:50:06 +01:00
Sauyon Lee
b64a43f578 Merge pull request #57 from max-schaefer/trap.gz
Gzip TRAP files
2020-03-12 06:24:32 -07:00
Max Schaefer
6b0ba750e6 Put gzip writer on top of bufio writer. 2020-03-12 08:40:22 +00:00
Max Schaefer
d7d5447689 Merge pull request #46 from sauyon/force-extract-methods
Extract methods when they don't exist
2020-03-12 08:16:44 +00:00
Max Schaefer
8901ba62e0 Gzip TRAP files. 2020-03-11 15:14:37 +00:00
Max Schaefer
8136ebbb91 Merge pull request #54 from sauyon/vendor-support
extractor: Use -mod=vendor when a vendor directory exists
2020-03-11 11:36:49 +00:00
Sauyon Lee
57b874e047 extractor: Only skip dependency installation when vendor folder is detected 2020-03-11 02:59:33 -07:00
Max Schaefer
a8c1731f9d Merge pull request #50 from sauyon/uintptr
Make uintptrtype a subclass of unsignedintegertype
2020-03-11 09:57:00 +00:00
Sauyon Lee
ecd4c42428 extractor: Factor out method extraction
This fixes a subtle bug where the underlying interface type was used
as the receiver when constructing method labels, causing some database
inconsistencies.
2020-03-10 22:01:16 -07:00
Sauyon Lee
0daf8c1fa3 extractor: Extract methods when their labels don't exist 2020-03-10 20:36:49 -07:00
Sauyon Lee
ccae530508 extractor: minor refactoring to use variables 2020-03-10 20:36:13 -07:00
Sauyon Lee
0aa46becf9 extractor: Use -mod=vendor when a vendor directory exists 2020-03-10 16:44:03 -07:00
Sauyon Lee
79ab831776 extractor: Close files even when writes fail 2020-03-10 00:52:33 -07:00
Sauyon Lee
2428efcb6d Make @uintptrtype a @unsignedintegertype 2020-03-09 04:40:02 -07:00
Sauyon Lee
dd3f98c549 extractor: Don't log directory being walked for go.mod files 2020-03-06 06:51:21 -08:00
Sauyon Lee
c027bbaadf Use buffered writers 2020-03-05 21:12:15 -08:00
Sauyon Lee
f2358a0a86 Find all go.mod files before extraction 2020-03-04 16:08:55 -08:00
Sauyon Lee
cca762dbc0 gomodextractor.go: Emit pseudo comment groups to table 2020-03-04 06:26:44 -08:00
Sauyon Lee
db4efd6124 Add support for extracting go.mod files 2020-03-04 06:26:42 -08:00
Sauyon Lee
89caafb0e1 labels.go: Make label generation slightly more uniform 2020-03-04 06:21:20 -08:00