Arthur Baars
65c1f2c359
Merge pull request #20 from github/aibaars/extract-extra
...
Extract 'extra' nodes and their subtrees
2020-11-03 13:45:33 +01:00
Arthur Baars
25205a09a3
Update tree-sitter-ruby
2020-11-03 10:06:59 +01:00
Arthur Baars
dc3459de8e
Extract 'extra' nodes and their subtrees
2020-11-03 10:03:11 +01:00
Nick Rolfe
27c3c88b3c
Add buffered writing and gzip compression for trap files
2020-11-02 16:14:19 +00:00
Arthur Baars
0ccd97639b
Address comments
2020-11-02 13:30:46 +01:00
Arthur Baars
63ca8212f6
Limit string sizes to 1MB
2020-10-31 11:36:01 +01:00
Arthur Baars
f265ccef59
TrapWriter: add global ID caching and populate folders
2020-10-31 11:35:57 +01:00
Arthur Baars
0de8b0c069
Add TrapWriter::comment
2020-10-31 11:35:22 +01:00
Arthur Baars
748dee64ae
Escape label keys
2020-10-31 11:35:22 +01:00
Arthur Baars
57842e8a87
Add TrapWriter
2020-10-31 11:35:16 +01:00
Nick Rolfe
075c72e6ef
Iterate through path components to 'normalize' paths on windows
2020-10-30 15:26:46 +00:00
Nick Rolfe
e73500ef7c
Cope with empty filenames/extensions
2020-10-30 14:38:24 +00:00
Nick Rolfe
35cb379db7
Fix name of table for locations
2020-10-30 13:24:16 +00:00
Nick Rolfe
a54f923a73
Normalize the absolute path in the files table
2020-10-30 13:22:58 +00:00
Nick Rolfe
4b8bbd101c
Give locations full ids matching the common spec
2020-10-30 13:06:21 +00:00
Nick Rolfe
79d15051be
Fix full ids for files to match common spec
2020-10-30 12:45:23 +00:00
Nick Rolfe
7f03206b52
Use a key id for file entities
2020-10-30 11:29:04 +00:00
Nick Rolfe
826b4571a0
Canonicalize source file paths in main
2020-10-30 11:21:51 +00:00
Nick Rolfe
f198dc530f
Use fromSource = 1
...
Co-authored-by: Arthur Baars <aibaars@github.com >
2020-10-30 10:25:09 +00:00
Nick Rolfe
2232700428
Correct comment
...
Co-authored-by: Arthur Baars <aibaars@github.com >
2020-10-30 10:24:24 +00:00
Nick Rolfe
4d5d80c749
Fix location handling to match common db schema requirements
2020-10-29 19:44:16 +00:00
Nick Rolfe
547d12ca58
Add more info to error message
2020-10-29 15:13:04 +00:00
Nick Rolfe
11c9c18de4
Don't generate an index for optional fields that occur at most once
2020-10-29 13:04:26 +00:00
Arthur Baars
3350d9d3d4
TRAP locations: always fix-up empty ranges
2020-10-29 10:45:07 +01:00
Arthur Baars
4c04b8bb15
Add comment
2020-10-28 17:40:01 +01:00
Arthur Baars
d2f42552f6
Adjust source locations
...
Tree-sitter row and column numbers are 0-based while CodeQL expects 1-based.
In addition tree-sitter location ranges end-points are exclusive while
CodeQL's ranges are inclusive.
2020-10-28 17:30:03 +01:00
Nick Rolfe
f4b9c0c71a
Merge remote-tracking branch 'origin/main' into ql_gen
2020-10-28 11:41:18 +00:00
Nick Rolfe
11152583d5
Add get_name() method to simplify logic in field handling
2020-10-28 11:30:50 +00:00
Arthur Baars
fe1d8ec15f
Extractor: fix child index values
2020-10-27 22:32:53 +01:00
Douglas Creager
5f985be2d9
Use tree-sitter-ruby crate instead of vendoring it
2020-10-27 13:54:56 -04:00
Arthur Baars
7555141246
Extractor: include contents node-types.json as constant
2020-10-27 16:34:17 +01:00
Arthur Baars
74dd4dcc2c
Build parser.c and scanner.cc separately
2020-10-27 16:34:17 +01:00
Arthur Baars
1fd6fdd652
Address review comment from earlier pull-request
2020-10-27 13:43:59 +01:00
Arthur Baars
0439d4f674
Refactor dbscheme generator to use intermediate representation
...
* merge extractor/node_types.rs into node-types/lib.rs
* use intermediate representation in dbscheme generator
* move dbscheme naming and escaping functions to node-types so they can be shared
2020-10-27 13:27:45 +01:00
Arthur Baars
4c1682ef2e
Merge pull request #5 from github/aibaars/logger
...
Add logging based on the tracing library
2020-10-27 13:24:34 +01:00
Nick Rolfe
c02b735eec
Handle Windows path prefixes
2020-10-27 12:09:46 +00:00
Arthur Baars
52035ef672
Add tracing logger
2020-10-27 11:29:21 +01:00
Arthur Baars
9c534209f7
Add tracing:0.1
2020-10-27 11:26:35 +01:00
Arthur Baars
0f576fe29a
Address review comments
2020-10-26 19:10:44 +01:00
Arthur Baars
1d36b5085a
Do not recurse into 'extra' nodes for now
2020-10-26 18:39:10 +01:00
Arthur Baars
fd39524c5e
Improve error messages
...
Include file path and line number and emit better descriptions
2020-10-26 18:37:29 +01:00
Arthur Baars
47ccc33ab3
Initial version of extractor based on tree-sitter grammar
2020-10-24 13:22:39 +02:00
Arthur Baars
d00c956028
Build with clang for non-windows platforms
2020-10-24 13:22:39 +02:00
Nick Rolfe
6c697bf9b5
Split into generator and extractor packages
2020-10-15 13:20:11 +01:00