hohn/sarif-cli
1
0
Fork 0
mirror of https://github.com/hohn/sarif-cli.git synced 2025-12-16 17:23:03 +01:00
Code Issues Packages Projects Releases Wiki Activity
200 Commits 3 Branches 1 Tag
335017ba6843ea2c5838e598536f0bf000bd626e
Commit Graph

99 Commits

Author SHA1 Message Date
michael hohn
335017ba68 try a reduced signature for better compatibility with sarif variations 
This doesn't help; sarif is just too dynamic.  Try a pull parser instead.
 2025-10-19 23:36:38 -07:00
michael hohn
c15dc6d4bc Fix subtle type problem: M8 is required for early steps, datetime64[ns] later 2025-10-19 13:35:02 -07:00
michael hohn
bed9d3e659 Fix new float content handling: 
raise Exception("Unknown element type", t, elem)
Exception: ('Unknown element type', <class 'float'>, 99.96780515670306)

from

{
  "rule": {
    "id": "cpp/telemetry/database-quality",
    "index": 53
  },
  "ruleId": "cpp/telemetry/database-quality",
  "ruleIndex": 53,
  "value": 99.96780515670306,
  "message": {
    "text": "Percentage of functions without errors"
  }
}
 2025-08-23 20:31:54 -07:00
Michael Hohn
ee11214aee Add support for external timestamps 
This allows external files containing

    timestamps = {
        "db_create_start"      : pd.Timestamp(0.0, unit='s'),
        "db_create_stop"       : pd.Timestamp(0.0, unit='s'),
        "scan_start_date"      : pd.Timestamp(0.0, unit='s'),
        "scan_stop_date"       : pd.Timestamp(0.0, unit='s'),
    }

to be used to provide those values, instead of the above defaults.

This patch changes the top-level scripts
        bin/sarif-extract-scans
        bin/sarif-extract-scans-runner
and provides
        scripts/test-timestamps.sh
for verification.

The following keys are also accepted:
    {
      "db_create_start": ...,
      "db_create_stop": ...,
      "scan_start": ...
      "scan_stop": ...
    }
 2023-08-18 17:06:58 -07:00
Michael Hohn
3854036fa7 Don't trail the project name with a dash if the subset name is blank 2023-08-03 17:06:54 -07:00
Michael Hohn
bd9460dd61 Remove old comment 2023-07-27 09:34:17 -07:00
Michael Hohn
a900848268 Remove ipython debug call 2023-07-27 09:05:27 -07:00
Michael Hohn
3dfb297612 Make project_name unique by adding automationDetails to it 2023-07-20 22:39:10 -07:00
Michael Hohn
ebeaced0f4 Remove automationDetails from CSV output 
This reverses commit 68b43e05 to keep the CSV compatible with prior output
 2023-07-17 10:30:35 -07:00
Michael Hohn
c299321ab8 Remove repls; add scripts/test-vcp.sh 2023-07-13 16:03:01 -07:00
Michael Hohn
68b43e0514 wip: debug and get automationDetails into CSV output 2023-07-12 17:04:23 -07:00
Michael Hohn
606912c8c3 Merge remote-tracking branch 'refs/remotes/origin/master' 2023-07-10 09:03:45 -07:00
Kristen Newbury
6b248d2474 Add missing columns to column order list 
prev missing cols: source_location and sink_location
missing from prev patch
but were missing in problem only results case anyways
 2023-06-02 16:22:18 -04:00
Kristen Newbury
a3aed444c1 Add explicit column ordering to to_csv writes 
in interfaces:
sarif-pad-aggregates
sarif-extract-scans
 2023-05-29 08:58:29 -04:00
Kristen Newbury
9407e5b00f Add ability to read automationDetails.id if present 2023-05-17 15:23:19 -04:00
Kristen Newbury
e2501b94a9 Fix fulldescription missing patch 
previous patch added nonunique placeholder
but must be unique
 2023-05-15 13:10:07 -04:00
Kristen Newbury
1e1305bb25 Fix CLI sarif consumption for properties: 
fullDescription
primaryLocationStartColumnFingerprint
 2023-03-02 12:43:42 -05:00
Kristen Newbury
1be65372e8 Fix CLI sarif consumption for property - description 2023-03-02 11:16:27 -05:00
Michael Hohn
e62c351029 Merge remote-tracking branch 'kristen/main' 2023-01-23 13:14:14 -08:00
Kristen Newbury
1a915e4de8 Update how project_id is generated 
previously relied on assumption:
naming like: <org>/<project> in
repositoryUri
now just uses full repositoryUri
 2023-01-05 16:37:55 -05:00
Kristen Newbury
fc2c6bac99 Add capability to read sourceLanguage if exists in CLI sarif 
otherwise dummy val
previously assumed never present in CLI sarif
 2023-01-05 12:50:54 -05:00
Kristen Newbury
d602efd3f0 Bugfix signature subset superset mismatch 
when the template signature portion contains
codeflows it was previously possible that a valid sarif
problem portion that contains extra fields
would be misdiagnosed as not parsable
 2022-12-15 19:13:15 -05:00
Kristen Newbury
dae6c50d5b Bugfix CLI signature merge mistake 2022-12-15 19:13:12 -05:00
Kristen Newbury
04a5aae14d Add CLI support 
enabled by -f flag with CLI value
tested on sarif from CodeQL CLIs:
2.6.3, 2.9.4, 2.11.4
MUST contain versionControlProvenance property however
 2022-12-15 19:12:58 -05:00
Kristen Newbury
bbeba14dec Bugfix CLI signature merge mistake 2022-12-13 20:13:13 -05:00
Kristen Newbury
69f5ef09a4 Merge branch 'addCLISignature' 2022-12-13 12:22:43 -05:00
Kristen Newbury
2ba9593d70 Add CLI support 
enabled by -f flag with CLI value
tested on sarif from CodeQL CLIs:
2.6.3, 2.9.4, 2.11.4
MUST contain versionControlProvenance property however
 2022-12-13 12:14:32 -05:00
Kristen Newbury
009cf12d2c Fix load error csv output error 2022-12-12 17:15:49 -05:00
Kristen Newbury
1d1734eabe Add query_tags column to the results table 2022-12-05 11:27:27 -05:00
Kristen Newbury
ff17cbad2c Add missing sig fix for populating problem.severity 2022-12-05 10:14:26 -05:00
Kristen Newbury
fb0e1b9c1c Change sarif sig severity to problem.severity 
and rm redundant table col for kind
 2022-12-02 16:00:40 -05:00
Kristen Newbury
2bda917a4e Improve error handling on signature mismatch cases 
and cleanup old todos that have been addressed
 2022-11-23 14:06:23 -05:00
Kristen Newbury
15aa9573e2 Adjust extra properties status from error to warning 2022-11-15 13:35:52 -05:00
Kristen Newbury
678219beb7 Add csv status aggregate tool 2022-11-15 10:18:12 -05:00
Kristen Newbury
066fcb8248 Add error handling csv writer 
writer generates status csv per sarif
 2022-11-14 13:02:36 -05:00
Kristen Newbury
ae4f71e804 Fix regex for repo url parsing 2022-11-10 15:56:49 -05:00
Kristen Newbury
1caf03f5f0 Rework project name format and project id format 2022-11-07 13:56:50 -05:00
Kristen Newbury
c51dbba577 Add fake date ranges to scan default values 2022-10-26 11:28:06 -04:00
Kristen Newbury
3b3999cfd7 Add kind, precision, severity to scan table for path-problem 2022-10-13 16:44:20 -04:00
Kristen Newbury
3385d9a10a Add kind, precision, severity to scan table 2022-10-13 13:54:32 -04:00
Michael Hohn
2b42a7d306 scan table change: the results.query_id is the @id from the CodeQL query 
Before, the query_id was
	==> results.csv <==
	query_id STRING,         -- git commit id of the ql query set

now, it's
	query_id STRING,         -- @id from the CodeQL query
 2022-08-11 16:56:20 -07:00
Michael Hohn
8ad69a503b Reduce zero results from error to warning 2022-08-11 16:26:07 -07:00
Michael Hohn
38af30ead9 Switch numpy.datetime64() to numpy.dtype('M') to get working equality comparison 2022-08-10 17:33:44 -07:00
Michael Hohn
1754c6c9ca Export codeflows column types for scan-related pandas tables 2022-08-08 16:49:13 -07:00
Michael Hohn
505ee8ea66 Export column types for scan-related pandas tables 2022-08-08 16:48:17 -07:00
Michael Hohn
560b9ecf35 Enforce types when forming the scan tables (internal and output formatting) 
Force all column types to ensure appropriate formatting for writing.  In
particular, no character data in place of integers, no floats, no
objects in place of strings.

Table formation for the functions
- st.joins_for_results
- st.joins_for_scans
- st.joins_for_projects
enforces types.
 2022-08-07 19:04:13 -07:00
Michael Hohn
0e7a941be3 Include all typegraph samples, from raw to refined 2022-07-14 18:29:21 -07:00
Michael Hohn
741be0cfe1 Include project table in output of sarif-extract-scans; add commit_id to scans table 2022-06-02 16:45:04 -07:00
Michael Hohn
fd55969b76 fix: special concatenation case for empty tables 2022-06-01 17:44:50 -07:00
Michael Hohn
32413984e2 fix: only concatenate non-empty tables to suppress float conversion 2022-06-01 17:34:56 -07:00