sarif-cli/notes/tables.org

• Overview
• Currently Exported Tables
• New tables to be exported

Overview

The current => project.csv <= table is broken. It's a combination of project meta info (to be moved to a separate project table) and the entry point to a single project's sarif results

Currently Exported Tables

Tables exported by sarif-extract-multi, commit d5390bb87, [Mar-23-2022]

  ==> artifacts.csv <==
  artifacts_id
  index
  uri
  uriBaseId

  ==> codeflows.csv <==
  codeflow_id
  codeflow_index
  threadflow_index
  location_index
  endColumn
  endLine
  startColumn
  startLine
  artifact_index
  uri
  uriBaseId
  message

  ==> kind_pathproblem.csv <==
  results_array_id
  results_array_index
  codeFlows_id
  ruleId
  ruleIndex
  location_array_index
  location_id
  location_endColumn
  location_endLine
  location_startColumn
  location_startLine
  location_index
  location_uri
  location_uriBaseId
  location_message
  relatedLocation_array_index
  relatedLocation_id
  relatedLocation_endColumn
  relatedLocation_endLine
  relatedLocation_startColumn
  relatedLocation_startLine
  relatedLocation_index
  relatedLocation_uri
  relatedLocation_uriBaseId
  relatedLocation_message
  message_text
  primaryLocationLineHash
  primaryLocationStartColumnFingerprint
  rule_id
  rule_index

  ==> kind_problem.csv <==
  results_array_id
  results_array_index
  ruleId
  ruleIndex
  location_array_index
  location_id
  location_endColumn
  location_endLine
  location_startColumn
  location_startLine
  location_index
  location_uri
  location_uriBaseId
  location_message
  relatedLocation_array_index
  relatedLocation_id
  relatedLocation_endColumn
  relatedLocation_endLine
  relatedLocation_startColumn
  relatedLocation_startLine
  relatedLocation_index
  relatedLocation_uri
  relatedLocation_uriBaseId
  relatedLocation_message
  message_text
  primaryLocationLineHash
  primaryLocationStartColumnFingerprint
  rule_id
  rule_index

  ==> project.csv <==
  creation_date
  primary_language
  project_name
  query_commit_id
  sarif_file_name
  scan_id
  scan_start_date
  scan_stop_date
  tool_name
  tool_version
  $schema
  sarif_version
  run_index
  artifacts
  columnKind
  results
  semmle.formatSpecifier
  semmle.sourceLanguage
  driver_name
  organization
  rules
  driver_version
  repositoryUri
  revisionId

  ==> relatedLocations.csv <==
  struct_id
  uri
  startLine
  startColumn
  endLine
  endColumn
  message

  ==> rules.csv <==
  rules_array_id
  rules_array_index
  id
  name
  enabled
  level
  fullDescription
  shortDescription
  kind
  precision
  security-severity
  severity
  sub-severity
  tag_index
  tag_text

New tables to be exported

Possible splits from project.csv

The scan results are the root of the sarif tree, so this is a required base table.

  ==> project-scan-result.csv <==
  $schema
  sarif_version
  run_index
  artifacts
  columnKind
  results
  semmle.formatSpecifier
  semmle.sourceLanguage
  driver_name
  organization
  rules
  driver_version
  repositoryUri
  revisionId

The rest of the [Mar-23-2022] projects.csv table is ad-hoc and included in the other tables below; it can be discarded.

  ==> project-meta.csv <==
  creation_date
  primary_language
  project_name
  query_commit_id
  sarif_file_name
  scan_id
  scan_start_date
  scan_stop_date
  tool_name
  tool_version

New tables intended for reporting infrastructure:

  ==> project.csv <==
  id
  project_name
  creation_date
  repository_url -- new
  primary_language -- from github api
  languages_analyzed

  ==> scans.csv <==
  id
  commit_id
  project_id
  db_create_start
  db_create_stop
  scan_start_date
  scan_stop_date
  tool_name
  tool_version
  tool_query_commit_id
  sarif_content
  sarif_file_name

  ==> results.csv <==
  id INT,                  -- primary key
  scan_id INT,             -- scans.id
  query_id STRING,         -- git commit id of the ql query set
  location STRING,
  message STRING,
  message_object OBJ,
  -- for kind_path_problem, use distinct source / sink
  -- for kind_problem, use the same location for both
  result_type STRING,      -- kind_problem | kind_path_problem
  -- link to codeflows (kind_pathproblem.csv only, NULL otherwise)
  codeFlow_id INT,
  --
  source_startLine int,
  source_startCol int,
  source_endLine int,
  source_endCol int,
  --
  sink_startLine int,
  sink_startCol int,
  sink_endLine int,
  sink_endCol int,
  --
  source_object STRING, -- higher-level info: 'args', 'request', etc.
  sink_object string, -- higher level: 'execute', 'sql statement', etc.