Commit Graph

44 Commits

Author SHA1 Message Date
Arthur Baars
65c1f2c359 Merge pull request #20 from github/aibaars/extract-extra
Extract 'extra' nodes and their subtrees
2020-11-03 13:45:33 +01:00
Arthur Baars
25205a09a3 Update tree-sitter-ruby 2020-11-03 10:06:59 +01:00
Arthur Baars
dc3459de8e Extract 'extra' nodes and their subtrees 2020-11-03 10:03:11 +01:00
Nick Rolfe
27c3c88b3c Add buffered writing and gzip compression for trap files 2020-11-02 16:14:19 +00:00
Arthur Baars
0ccd97639b Address comments 2020-11-02 13:30:46 +01:00
Arthur Baars
63ca8212f6 Limit string sizes to 1MB 2020-10-31 11:36:01 +01:00
Arthur Baars
f265ccef59 TrapWriter: add global ID caching and populate folders 2020-10-31 11:35:57 +01:00
Arthur Baars
0de8b0c069 Add TrapWriter::comment 2020-10-31 11:35:22 +01:00
Arthur Baars
748dee64ae Escape label keys 2020-10-31 11:35:22 +01:00
Arthur Baars
57842e8a87 Add TrapWriter 2020-10-31 11:35:16 +01:00
Nick Rolfe
075c72e6ef Iterate through path components to 'normalize' paths on windows 2020-10-30 15:26:46 +00:00
Nick Rolfe
e73500ef7c Cope with empty filenames/extensions 2020-10-30 14:38:24 +00:00
Nick Rolfe
35cb379db7 Fix name of table for locations 2020-10-30 13:24:16 +00:00
Nick Rolfe
a54f923a73 Normalize the absolute path in the files table 2020-10-30 13:22:58 +00:00
Nick Rolfe
4b8bbd101c Give locations full ids matching the common spec 2020-10-30 13:06:21 +00:00
Nick Rolfe
79d15051be Fix full ids for files to match common spec 2020-10-30 12:45:23 +00:00
Nick Rolfe
7f03206b52 Use a key id for file entities 2020-10-30 11:29:04 +00:00
Nick Rolfe
826b4571a0 Canonicalize source file paths in main 2020-10-30 11:21:51 +00:00
Nick Rolfe
f198dc530f Use fromSource = 1
Co-authored-by: Arthur Baars <aibaars@github.com>
2020-10-30 10:25:09 +00:00
Nick Rolfe
2232700428 Correct comment
Co-authored-by: Arthur Baars <aibaars@github.com>
2020-10-30 10:24:24 +00:00
Nick Rolfe
4d5d80c749 Fix location handling to match common db schema requirements 2020-10-29 19:44:16 +00:00
Nick Rolfe
547d12ca58 Add more info to error message 2020-10-29 15:13:04 +00:00
Nick Rolfe
11c9c18de4 Don't generate an index for optional fields that occur at most once 2020-10-29 13:04:26 +00:00
Arthur Baars
3350d9d3d4 TRAP locations: always fix-up empty ranges 2020-10-29 10:45:07 +01:00
Arthur Baars
4c04b8bb15 Add comment 2020-10-28 17:40:01 +01:00
Arthur Baars
d2f42552f6 Adjust source locations
Tree-sitter row and column numbers are 0-based while CodeQL expects 1-based.
In addition tree-sitter location ranges end-points are exclusive while
CodeQL's ranges are inclusive.
2020-10-28 17:30:03 +01:00
Nick Rolfe
f4b9c0c71a Merge remote-tracking branch 'origin/main' into ql_gen 2020-10-28 11:41:18 +00:00
Nick Rolfe
11152583d5 Add get_name() method to simplify logic in field handling 2020-10-28 11:30:50 +00:00
Arthur Baars
fe1d8ec15f Extractor: fix child index values 2020-10-27 22:32:53 +01:00
Douglas Creager
5f985be2d9 Use tree-sitter-ruby crate instead of vendoring it 2020-10-27 13:54:56 -04:00
Arthur Baars
7555141246 Extractor: include contents node-types.json as constant 2020-10-27 16:34:17 +01:00
Arthur Baars
74dd4dcc2c Build parser.c and scanner.cc separately 2020-10-27 16:34:17 +01:00
Arthur Baars
1fd6fdd652 Address review comment from earlier pull-request 2020-10-27 13:43:59 +01:00
Arthur Baars
0439d4f674 Refactor dbscheme generator to use intermediate representation
* merge extractor/node_types.rs into node-types/lib.rs
* use intermediate representation in dbscheme generator
* move dbscheme naming and escaping functions to node-types so they can be shared
2020-10-27 13:27:45 +01:00
Arthur Baars
4c1682ef2e Merge pull request #5 from github/aibaars/logger
Add logging based on the tracing library
2020-10-27 13:24:34 +01:00
Nick Rolfe
c02b735eec Handle Windows path prefixes 2020-10-27 12:09:46 +00:00
Arthur Baars
52035ef672 Add tracing logger 2020-10-27 11:29:21 +01:00
Arthur Baars
9c534209f7 Add tracing:0.1 2020-10-27 11:26:35 +01:00
Arthur Baars
0f576fe29a Address review comments 2020-10-26 19:10:44 +01:00
Arthur Baars
1d36b5085a Do not recurse into 'extra' nodes for now 2020-10-26 18:39:10 +01:00
Arthur Baars
fd39524c5e Improve error messages
Include file path and line number and emit better descriptions
2020-10-26 18:37:29 +01:00
Arthur Baars
47ccc33ab3 Initial version of extractor based on tree-sitter grammar 2020-10-24 13:22:39 +02:00
Arthur Baars
d00c956028 Build with clang for non-windows platforms 2020-10-24 13:22:39 +02:00
Nick Rolfe
6c697bf9b5 Split into generator and extractor packages 2020-10-15 13:20:11 +01:00