73 Commits

Author	SHA1	Message	Date
Alex Ford	11949c6b77	Merge pull request #176 from github/diagnostics-entries ... Start writing diagnostics to the DB, and some basic summary/diagnostics queries	2021-05-13 14:31:01 +01:00
Alex Ford	277a6a020a	diagnostics: use debug rather than hidden terminology, and leave gaps for other severities	2021-05-13 13:44:10 +01:00
Alex Ford	11376bc411	note that severity 3 corresponds to an error diagnostic level	2021-05-12 16:39:51 +01:00
Alex Ford	9663b74e12	use severity level 3 to indicate an extraction error for a file	2021-05-11 19:23:05 +01:00
Alex Ford	8ab95324eb	dedupe some error reporting code	2021-05-11 14:09:10 +01:00
Alex Ford	0f3168f293	record more parse errors	2021-05-10 21:23:24 +01:00
Alex Ford	269ae8331b	record 'unknown table type' extraction errors	2021-05-07 17:56:50 +01:00
Alex Ford	272aec27f2	clean up the parse_error writing code	2021-05-07 00:15:09 +01:00
Alex Ford	3a1dff1c95	start writing diagnostics entries for parse errors	2021-05-06 23:09:43 +01:00
Nick Rolfe	c37f390efc	Reserve more capacity for escaped key	2021-05-05 13:21:16 +01:00
Nick Rolfe	99ae17de03	Avoid copying key when it doesn't need escaping	2021-05-05 12:54:23 +01:00
Nick Rolfe	d2d5f31599	Escape keys for files and folders	2021-05-04 16:52:35 +01:00
Nick Rolfe	9b96bc32cc	Add ast_node_parent relation	2021-03-08 18:25:37 +00:00
Arthur Baars	846173732b	Add newline at the end of each code region in an ERB file	2021-02-05 09:49:25 +01:00
Arthur Baars	9d974bd56d	Extract ERB files	2021-02-05 09:48:54 +01:00
Nick Rolfe	1d3f06aca1	Simplify propagation of errors	2021-01-07 11:11:15 +00:00
Nick Rolfe	92c78e2b2d	Simplify num_codeql_threads function slightly	2021-01-07 11:10:43 +00:00
Nick Rolfe	bb2bdc01b5	Have the extract function create the TS parser object	2021-01-07 10:56:23 +00:00
Nick Rolfe	bf4eac5113	Parallelize extraction ... Use the Rayon library to do parallel iteration over the file list. The number of threads used respects the CODEQL_THREADS environment variable.	2021-01-06 18:22:27 +00:00
Nick Rolfe	282d20d766	Remove redundant field on ChildNode struct	2020-12-16 20:57:06 +00:00
Nick Rolfe	f5282edfc1	Simplifications based on PR feedback	2020-12-16 17:54:40 +00:00
Nick Rolfe	3f5eab04b5	Create disjoint db types for different operators	2020-12-15 15:22:33 +00:00
Arthur Baars	735aec9d34	Ensure top-level nodes have distinct parent_index values	2020-11-25 13:48:25 +01:00
Arthur Baars	89953fd87c	Add parent_index field to @astnode	2020-11-25 13:48:25 +01:00
Arthur Baars	b72db8b6f1	Add parent field to AstNode	2020-11-25 13:48:25 +01:00
Nick Rolfe	1a9663ff7d	Replace single-branch match with if let	2020-11-16 18:43:54 +00:00
Nick Rolfe	68c97a2d13	Use .. to ignore fields ... Co-authored-by: Arthur Baars <aibaars@github.com>	2020-11-16 18:41:18 +00:00
Nick Rolfe	bbe7c70d34	more refactoring of names	2020-11-16 17:54:16 +00:00
Nick Rolfe	83a0e5fea6	Refactor to move naming decisions to shared library	2020-11-16 17:54:14 +00:00
Arthur Baars	c565f323f6	Don't register extra tokens as children of the parent node	2020-11-05 12:53:58 +01:00
Arthur Baars	86aa05e3cb	Address comments	2020-11-04 14:49:47 +01:00
Arthur Baars	c3e8d85f0b	Tolerate tokens containing invalid UTF-8	2020-11-04 14:46:31 +01:00
Arthur Baars	053c9f60a4	Store tokens in a separate table	2020-11-04 13:35:24 +01:00
Arthur Baars	65c1f2c359	Merge pull request #20 from github/aibaars/extract-extra ... Extract 'extra' nodes and their subtrees	2020-11-03 13:45:33 +01:00
Arthur Baars	dc3459de8e	Extract 'extra' nodes and their subtrees	2020-11-03 10:03:11 +01:00
Nick Rolfe	27c3c88b3c	Add buffered writing and gzip compression for trap files	2020-11-02 16:14:19 +00:00
Arthur Baars	0ccd97639b	Address comments	2020-11-02 13:30:46 +01:00
Arthur Baars	63ca8212f6	Limit string sizes to 1MB	2020-10-31 11:36:01 +01:00
Arthur Baars	f265ccef59	TrapWriter: add global ID caching and populate folders	2020-10-31 11:35:57 +01:00
Arthur Baars	0de8b0c069	Add TrapWriter::comment	2020-10-31 11:35:22 +01:00
Arthur Baars	748dee64ae	Escape label keys	2020-10-31 11:35:22 +01:00
Arthur Baars	57842e8a87	Add TrapWriter	2020-10-31 11:35:16 +01:00
Nick Rolfe	075c72e6ef	Iterate through path components to 'normalize' paths on windows	2020-10-30 15:26:46 +00:00
Nick Rolfe	e73500ef7c	Cope with empty filenames/extensions	2020-10-30 14:38:24 +00:00
Nick Rolfe	35cb379db7	Fix name of table for locations	2020-10-30 13:24:16 +00:00
Nick Rolfe	a54f923a73	Normalize the absolute path in the files table	2020-10-30 13:22:58 +00:00
Nick Rolfe	4b8bbd101c	Give locations full ids matching the common spec	2020-10-30 13:06:21 +00:00
Nick Rolfe	79d15051be	Fix full ids for files to match common spec	2020-10-30 12:45:23 +00:00
Nick Rolfe	7f03206b52	Use a key id for file entities	2020-10-30 11:29:04 +00:00
Nick Rolfe	826b4571a0	Canonicalize source file paths in main	2020-10-30 11:21:51 +00:00