Sauyon Lee
28c69743a4
Add workaround for go 1.14 explicit vendoring requirement
...
This only applies for module files for which no Go version has
been specified; Go will assume these should be parsed with the
latest Go version, which will cause them to fail if the vendor
directory has been generated with an old version of Go, as
the vendor/modules.txt will not meet the new requirements for
consistency.
2020-08-26 11:25:30 +01:00
Sauyon Lee
8e6c1835dd
autobuilder: Don't try to determine import paths for file URLs
...
Also improve logging
2020-08-05 23:21:34 -07:00
Chris Smowton
3ab948f81c
Extractor: assign unique indices to comment-groups in go.mod files
...
The schema requires that (parent, index) is a key.
2020-07-13 11:28:28 +01:00
Sauyon Lee
32510eb2d0
Merge pull request #255 from max-schaefer/alias-types
...
Improve modelling of alias declarations
2020-07-10 21:07:48 -07:00
Max Schaefer
1a8688a8f4
Extract enough information to distinguish type definitions from alias declarations.
2020-07-10 14:12:51 +01:00
Max Schaefer
4257a68c27
Include newlines in messages printed by go-gen-dbscheme.
2020-07-10 14:08:37 +01:00
Chris Smowton
d05657ddff
Make the gofmt CI test actaully fatal
...
Turns out gofmt doesn't actually return 1 when it finds problems, only when it finds source files which don't compile (all of which are now excluded).
This also fixes existing overlooked inconsistencies as a result of this mistake.
2020-07-10 11:02:50 +01:00
Chris Smowton
6bf3802b3f
Make CommentGroups AST-children of Files
...
Previously they were roots, with children hanging off them. Now they are children of Files, and both CommentGroups and Comments can be discovered using AstNode.getAChild.
The PrintAst pass is also adapted to account for their new position.
2020-07-08 17:49:47 +01:00
Max Schaefer
f74a94e382
Merge pull request #170 from sauyon/tracing
...
Extract more dependency ASTs
2020-07-01 14:25:52 +01:00
Sauyon Lee
6e5e9ce5de
Improve comments for extractor utility functions
2020-06-30 11:44:10 -07:00
Max Schaefer
ccfccb4828
Revert "autobuilder: Add support for GITHUB_REPOSITORY environment variable"
2020-06-30 15:13:26 +01:00
Chris Smowton
595866a6d8
Extractor: give the go.mod comment groups a source location ( #232 )
...
The comment group is now omitted entirely if empty, and otherwise delimits the range of the comments ascribed to this group.
2020-06-30 14:59:13 +01:00
Sauyon Lee
380060c7e4
extractor: Refactor regexp compilation for the relative directory check
2020-06-24 23:29:55 -07:00
Sauyon Lee
fa391b1516
extractor: Factor out common bits for running go list
2020-06-24 23:29:54 -07:00
Sauyon Lee
ebdd724b75
Simplify logic for deciding whether to extract a package
2020-06-24 23:29:53 -07:00
Sauyon Lee
e25b882e42
Clarify some comments
...
As suggested in code review
Co-authored-by: Max Schaefer <54907921+max-schaefer@users.noreply.github.com >
2020-06-24 23:29:52 -07:00
Sauyon Lee
9bd1f87d66
Address review comments
2020-06-24 23:29:51 -07:00
Sauyon Lee
7863bb656e
Use the -mod argument from the build when calling go list
2020-06-24 23:29:49 -07:00
Sauyon Lee
296d2d5fd3
extractor: modify FileExists to check that the path isn't a directory
2020-06-24 23:29:48 -07:00
Sauyon Lee
3513c352e6
extractor: Factor out FileExists utility function
2020-06-24 23:29:48 -07:00
Sauyon Lee
f197975c6e
Extract packages more intelligently
...
We now extract packages that have the same module root as the specified packages, as determined by
the `go list` command.
2020-06-24 23:29:47 -07:00
Sauyon Lee
ba0f922a28
autobuilder: Add support for GITHUB_REPOSITORY environment variable
...
This is for use within GitHub actions
2020-06-09 11:52:23 -07:00
Sauyon Lee
aef7524f35
Add a go-gen-dbscheme for generating dbschemes
2020-05-22 08:04:40 -07:00
Max Schaefer
97b1d3a57c
Fix extraction of error positions for paths containing colon.
2020-04-20 08:04:16 +01:00
Max Schaefer
a837d5f4ce
Add support for extracting Go frontend errors.
2020-04-09 16:39:47 +01:00
Sauyon Lee
a663eaae23
Merge pull request #89 from max-schaefer/extractor-profiling
...
Add environment variables for dumping CPU and memory profiles.
2020-04-06 03:33:08 -07:00
Max Schaefer
510b6070c9
Introduce official environment variable for goroutine limiting.
...
We've had to tell people how to do this, so we should have a name for it that doesn't refer to a defunct company.
2020-04-02 10:45:52 +01:00
Sauyon Lee
eba8dd0a36
Merge pull request #82 from github/max-schaefer-patch-1
...
Improve autobuilder logging
2020-04-01 09:07:55 -07:00
Max Schaefer
611751a9c3
Improve autobuilder logging
2020-04-01 15:31:50 +01:00
Max Schaefer
efc9ecefc8
Introduce CODEQL_GO_EXTRACTOR_BUILD_COMMAND as an alias for LGTM_INDEX_BUILD_COMMAND.
...
We've occasionally had to tell people to set this variable manually, so we might as well have an alias that doesn't refer to a soon-to-be obsolete product.
2020-04-01 09:35:57 +01:00
Max Schaefer
590f01d0c2
Add environment variables for dumping CPU and memory profiles.
...
These are intentionally undocumented for now.
2020-03-31 07:50:06 +01:00
Sauyon Lee
b64a43f578
Merge pull request #57 from max-schaefer/trap.gz
...
Gzip TRAP files
2020-03-12 06:24:32 -07:00
Max Schaefer
6b0ba750e6
Put gzip writer on top of bufio writer.
2020-03-12 08:40:22 +00:00
Max Schaefer
d7d5447689
Merge pull request #46 from sauyon/force-extract-methods
...
Extract methods when they don't exist
2020-03-12 08:16:44 +00:00
Max Schaefer
8901ba62e0
Gzip TRAP files.
2020-03-11 15:14:37 +00:00
Max Schaefer
8136ebbb91
Merge pull request #54 from sauyon/vendor-support
...
extractor: Use -mod=vendor when a vendor directory exists
2020-03-11 11:36:49 +00:00
Sauyon Lee
57b874e047
extractor: Only skip dependency installation when vendor folder is detected
2020-03-11 02:59:33 -07:00
Max Schaefer
a8c1731f9d
Merge pull request #50 from sauyon/uintptr
...
Make uintptrtype a subclass of unsignedintegertype
2020-03-11 09:57:00 +00:00
Sauyon Lee
ecd4c42428
extractor: Factor out method extraction
...
This fixes a subtle bug where the underlying interface type was used
as the receiver when constructing method labels, causing some database
inconsistencies.
2020-03-10 22:01:16 -07:00
Sauyon Lee
0daf8c1fa3
extractor: Extract methods when their labels don't exist
2020-03-10 20:36:49 -07:00
Sauyon Lee
ccae530508
extractor: minor refactoring to use variables
2020-03-10 20:36:13 -07:00
Sauyon Lee
0aa46becf9
extractor: Use -mod=vendor when a vendor directory exists
2020-03-10 16:44:03 -07:00
Sauyon Lee
79ab831776
extractor: Close files even when writes fail
2020-03-10 00:52:33 -07:00
Sauyon Lee
2428efcb6d
Make @uintptrtype a @unsignedintegertype
2020-03-09 04:40:02 -07:00
Sauyon Lee
dd3f98c549
extractor: Don't log directory being walked for go.mod files
2020-03-06 06:51:21 -08:00
Sauyon Lee
c027bbaadf
Use buffered writers
2020-03-05 21:12:15 -08:00
Sauyon Lee
f2358a0a86
Find all go.mod files before extraction
2020-03-04 16:08:55 -08:00
Sauyon Lee
cca762dbc0
gomodextractor.go: Emit pseudo comment groups to table
2020-03-04 06:26:44 -08:00
Sauyon Lee
db4efd6124
Add support for extracting go.mod files
2020-03-04 06:26:42 -08:00
Sauyon Lee
89caafb0e1
labels.go: Make label generation slightly more uniform
2020-03-04 06:21:20 -08:00