Commit Graph

162 Commits

Author SHA1 Message Date
Kristen Newbury
6b248d2474 Add missing columns to column order list
prev missing cols: source_location and sink_location
missing from prev patch
but were missing in problem only results case anyways
2023-06-02 16:22:18 -04:00
Kristen Newbury
a3aed444c1 Add explicit column ordering to to_csv writes
in interfaces:
sarif-pad-aggregates
sarif-extract-scans
2023-05-29 08:58:29 -04:00
Kristen Newbury
9407e5b00f Add ability to read automationDetails.id if present 2023-05-17 15:23:19 -04:00
Kristen Newbury
e2501b94a9 Fix fulldescription missing patch
previous patch added nonunique placeholder
but must be unique
2023-05-15 13:10:07 -04:00
Kristen Newbury
eb50bdf834 Merge branch 'main' 2023-05-15 13:09:21 -04:00
Kristen Newbury
1e1305bb25 Fix CLI sarif consumption for properties:
fullDescription
primaryLocationStartColumnFingerprint
2023-03-02 12:43:42 -05:00
Kristen Newbury
953d47edd3 Fix extract scans interface CLI default 2023-03-02 11:43:25 -05:00
Kristen Newbury
1be65372e8 Fix CLI sarif consumption for property - description 2023-03-02 11:16:27 -05:00
Peter Martin
6be328f785 Merge pull request #5 from bgroh2/fix-dir-names
Use repositoryUri instead of org/repo
2023-01-20 09:19:01 -05:00
Benjamin Groh
e8123903f6 Use repositoryUri instead of org/repo 2023-01-18 16:40:39 -05:00
Kristen Newbury
04e3dedb77 Merge pull request #2 from dbeer/exceptions
Fix exception reraising
2023-01-12 12:23:00 -05:00
Kristen Newbury
7dad175d4d Fix tool to default CLI not LGTM sarif input
update readme minor improvement
2023-01-12 12:03:51 -05:00
Kristen Newbury
1a915e4de8 Update how project_id is generated
previously relied on assumption:
naming like: <org>/<project> in
repositoryUri
now just uses full repositoryUri
2023-01-05 16:37:55 -05:00
Kristen Newbury
fc2c6bac99 Add capability to read sourceLanguage if exists in CLI sarif
otherwise dummy val
previously assumed never present in CLI sarif
2023-01-05 12:50:54 -05:00
Daniel Beer
6b475becd9 Fix exception reraising 2022-12-30 12:40:07 -05:00
Kristen Newbury
d602efd3f0 Bugfix signature subset superset mismatch
when the template signature portion contains
codeflows it was previously possible that a valid sarif
problem portion that contains extra fields
would be misdiagnosed as not parsable
2022-12-15 19:13:15 -05:00
Kristen Newbury
dae6c50d5b Bugfix CLI signature merge mistake 2022-12-15 19:13:12 -05:00
Kristen Newbury
dc4fd09e63 Update README missing minor syntax 2022-12-15 19:13:10 -05:00
Kristen Newbury
202f7f53a5 Update README for CLI usage instructions 2022-12-15 19:13:07 -05:00
Kristen Newbury
04a5aae14d Add CLI support
enabled by -f flag with CLI value
tested on sarif from CodeQL CLIs:
2.6.3, 2.9.4, 2.11.4
MUST contain versionControlProvenance property however
2022-12-15 19:12:58 -05:00
Kristen Newbury
009cf12d2c Fix load error csv output error 2022-12-12 17:15:49 -05:00
Kristen Newbury
02d8f4cfa2 Doc sarif-pad-aggregate in README 2022-12-05 11:27:59 -05:00
Kristen Newbury
1d1734eabe Add query_tags column to the results table 2022-12-05 11:27:27 -05:00
Kristen Newbury
ff17cbad2c Add missing sig fix for populating problem.severity 2022-12-05 10:14:26 -05:00
Kristen Newbury
fb0e1b9c1c Change sarif sig severity to problem.severity
and rm redundant table col for kind
2022-12-02 16:00:40 -05:00
Kristen Newbury
2bda917a4e Improve error handling on signature mismatch cases
and cleanup old todos that have been addressed
2022-11-23 14:06:23 -05:00
Kristen Newbury
01b248a2a9 Update readme 2022-11-16 20:58:27 -05:00
Kristen Newbury
15aa9573e2 Adjust extra properties status from error to warning 2022-11-15 13:35:52 -05:00
Kristen Newbury
e3f2323396 Minimize requirements file 2022-11-15 11:46:56 -05:00
Kristen Newbury
678219beb7 Add csv status aggregate tool 2022-11-15 10:18:12 -05:00
Kristen Newbury
d9bdcc8724 Fix runner defaults and setup more options
sarif-extract-scans-runner now takes specific outer
output dir
bin/sarif-aggregate-scans now takes specific directory
to summarize from
2022-11-14 14:30:55 -05:00
Kristen Newbury
066fcb8248 Add error handling csv writer
writer generates status csv per sarif
2022-11-14 13:02:36 -05:00
Kristen Newbury
ae4f71e804 Fix regex for repo url parsing 2022-11-10 15:56:49 -05:00
Kristen Newbury
a9d84ce26c Make sarif-aggregate-scans executable 2022-11-10 10:51:30 -05:00
Kristen Newbury
1caf03f5f0 Rework project name format and project id format 2022-11-07 13:56:50 -05:00
Kristen Newbury
4121072088 Rework project and scan id generation
goal:
deterministic across multiple instances of scan on same sarif file
no collisions between sarif files from different scan instances (regardless of for same project or not)

assumption sarif file naming will follow: <project>/<unique_filename_per_analysis> format
2022-10-26 12:00:38 -04:00
Kristen Newbury
c51dbba577 Add fake date ranges to scan default values 2022-10-26 11:28:06 -04:00
Kristen Newbury
d9116eba6a Move flakegen scan id to outermost bin tool runner 2022-10-25 10:40:25 -04:00
Kristen Newbury
4285b7a834 Add unique flakegen scan id 2022-10-21 12:16:44 -04:00
Kristen Newbury
3b3999cfd7 Add kind, precision, severity to scan table for path-problem 2022-10-13 16:44:20 -04:00
Kristen Newbury
3385d9a10a Add kind, precision, severity to scan table 2022-10-13 13:54:32 -04:00
Michael Hohn
203343df07 Add sarif-pad-aggregate to fill scan values
Fills the scans table's db_create_start/stop and scan_start/stop_date
columns with realistic random values.
2022-08-31 21:19:02 -07:00
Michael Hohn
2b42a7d306 scan table change: the results.query_id is the @id from the CodeQL query
Before, the query_id was
	==> results.csv <==
	query_id STRING,         -- git commit id of the ql query set

now, it's
	query_id STRING,         -- @id from the CodeQL query
2022-08-11 16:56:20 -07:00
Michael Hohn
8ad69a503b Reduce zero results from error to warning 2022-08-11 16:26:07 -07:00
Michael Hohn
235acf6b93 Quote all non-numeric CSV output 2022-08-10 17:44:29 -07:00
Michael Hohn
03a9ef0477 Rewrite sarif-combine-tables.py as full tool, bin/sarif-aggregate-scans 2022-08-10 17:34:35 -07:00
Michael Hohn
38af30ead9 Switch numpy.datetime64() to numpy.dtype('M') to get working equality comparison 2022-08-10 17:33:44 -07:00
Michael Hohn
1754c6c9ca Export codeflows column types for scan-related pandas tables 2022-08-08 16:49:13 -07:00
Michael Hohn
505ee8ea66 Export column types for scan-related pandas tables 2022-08-08 16:48:17 -07:00
Michael Hohn
7e996e746c Rewrite sarif-runner as full tool, sarif-extract-scans-runner 2022-08-08 14:47:25 -07:00