mirror of
https://github.com/hohn/codeql-cli-end-to-end.git
synced 2025-12-16 13:13:03 +01:00
The importance of versioning
This commit is contained in:
Michael Hohn
committed by
=Michael Hohn
=Michael Hohn
parent
94fd0a3876
commit
fa76851f13
206
doc/readme.in
206
doc/readme.in
@@ -1,4 +1,4 @@
|
|||||||
# -*- mode: org; org-confirm-babel-evaluate: nil; coding: utf-8 -*-
|
-*- mode: org; org-confirm-babel-evaluate: nil; coding: utf-8 -*-
|
||||||
#+OPTIONS: H:3 num:t \n:nil @:t ::t |:t ^:{} f:t *:t TeX:t LaTeX:t skip:nil p:nil
|
#+OPTIONS: H:3 num:t \n:nil @:t ::t |:t ^:{} f:t *:t TeX:t LaTeX:t skip:nil p:nil
|
||||||
#+OPTIONS: org-confirm-babel-evaluate:nil
|
#+OPTIONS: org-confirm-babel-evaluate:nil
|
||||||
|
|
||||||
@@ -359,12 +359,200 @@
|
|||||||
|
|
||||||
#+INCLUDE: "../custom-suite-1.qls" src yaml
|
#+INCLUDE: "../custom-suite-1.qls" src yaml
|
||||||
|
|
||||||
**** TODO Include versioning:
|
*** The importance of versioning
|
||||||
***** TODO codeql cli
|
**** TODO CodeQL cli version
|
||||||
***** TODO query set version
|
XX: for the sarif-cli
|
||||||
Checks:
|
The CLI versions used against development of the CLI support were: 2.6.3,
|
||||||
**** For building DBs: Common case: 15 minutes for || cpp compilation, can
|
2.9.4, and 2.11.4.
|
||||||
be 2 h with codeql.
|
|
||||||
|
**** Database version
|
||||||
|
An attempt to run an analysis with an older version of the cli against a
|
||||||
|
database created with a newer cli version will likely abort with an error.
|
||||||
|
|
||||||
|
In terms of commands, the codeql versions used for
|
||||||
|
#+BEGIN_SRC sh
|
||||||
|
codeql database create ...
|
||||||
|
#+END_SRC
|
||||||
|
and
|
||||||
|
#+BEGIN_SRC sh
|
||||||
|
codeql database analyze ..
|
||||||
|
#+END_SRC
|
||||||
|
should be the same.
|
||||||
|
|
||||||
|
If you just have a collection of databases, you can check what version of
|
||||||
|
the cli produced it.
|
||||||
|
The database directory contains the codeql version used in a yaml file,
|
||||||
|
a human-readable check:
|
||||||
|
#+BEGIN_SRC sh :exports both :results output
|
||||||
|
cd ~/local/codeql-cli-end-to-end/codeql-workshop-vulnerable-linux-driver
|
||||||
|
grep -A 2 creationMetadata vulnerable-linux-driver-db/codeql-database.yml
|
||||||
|
#+END_SRC
|
||||||
|
|
||||||
|
#+RESULTS:
|
||||||
|
: creationMetadata:
|
||||||
|
: cliVersion: 2.13.0
|
||||||
|
: creationTime: 2023-04-24T21:39:15.963711665Z
|
||||||
|
|
||||||
|
**** Query set version
|
||||||
|
- For suites in our own source code
|
||||||
|
|
||||||
|
Your query sets /may/ have release versions or tags. But they almost
|
||||||
|
certainly have git commit ids that can be used, like the following:
|
||||||
|
#+BEGIN_SRC sh :exports both :results output
|
||||||
|
cd ~/local/codeql-cli-end-to-end/codeql-workshop-vulnerable-linux-driver
|
||||||
|
git rev-parse --short HEAD
|
||||||
|
#+END_SRC
|
||||||
|
|
||||||
|
#+RESULTS:
|
||||||
|
: d548189
|
||||||
|
|
||||||
|
If you use packs, you can fix the ids of dependencies in the =qlpack.yml=
|
||||||
|
file. In our example, this is done in several places. The =common=
|
||||||
|
version:
|
||||||
|
#+BEGIN_SRC sh :exports both :results output
|
||||||
|
cd ~/local/codeql-cli-end-to-end/codeql-workshop-vulnerable-linux-driver
|
||||||
|
cat common/qlpack.yml
|
||||||
|
#+END_SRC
|
||||||
|
|
||||||
|
#+RESULTS:
|
||||||
|
: ---
|
||||||
|
: library: true
|
||||||
|
: name: common
|
||||||
|
: version: 0.0.1
|
||||||
|
: dependencies:
|
||||||
|
: codeql/cpp-all: 0.7.0
|
||||||
|
|
||||||
|
The dependencies are transitive; both =queries= and =solutions= depend on
|
||||||
|
=common=, so packs fixed by common also fix packs used by the others.
|
||||||
|
And =common= is fixed by our =git= id, so we're done.
|
||||||
|
|
||||||
|
- Some optional details
|
||||||
|
|
||||||
|
We have specified these packs:
|
||||||
|
#+BEGIN_SRC sh :exports both :results output
|
||||||
|
cd ~/local/codeql-cli-end-to-end/codeql-workshop-vulnerable-linux-driver
|
||||||
|
grep codeql/cpp-all */qlpack.yml
|
||||||
|
#+END_SRC
|
||||||
|
|
||||||
|
#+RESULTS:
|
||||||
|
: common/qlpack.yml: codeql/cpp-all: 0.7.0
|
||||||
|
: queries/qlpack.yml: codeql/cpp-all: ^0.7.0
|
||||||
|
|
||||||
|
The caret notation =^= means "at least". So at least version 0.7.0.
|
||||||
|
|
||||||
|
After we install packs via
|
||||||
|
#+begin_src sh
|
||||||
|
codeql pack install --no-strict-mode ...
|
||||||
|
#+end_src
|
||||||
|
some lock files are generated, and those fix versions further down the
|
||||||
|
dependency chain:
|
||||||
|
#+begin_src sh :exports both :results output
|
||||||
|
cd ~/local/codeql-cli-end-to-end/codeql-workshop-vulnerable-linux-driver
|
||||||
|
cat common/codeql-pack.lock.yml
|
||||||
|
#+end_src
|
||||||
|
|
||||||
|
#+results:
|
||||||
|
#+begin_example
|
||||||
|
---
|
||||||
|
lockversion: 1.0.0
|
||||||
|
dependencies:
|
||||||
|
codeql/cpp-all:
|
||||||
|
version: 0.7.0
|
||||||
|
codeql/ssa:
|
||||||
|
version: 0.0.15
|
||||||
|
codeql/tutorial:
|
||||||
|
version: 0.0.8
|
||||||
|
codeql/util:
|
||||||
|
version: 0.0.8
|
||||||
|
compiled: false
|
||||||
|
#+end_example
|
||||||
|
|
||||||
|
- Note that a query suite is always in a codeql pack, so the pack id is also
|
||||||
|
the suite id.
|
||||||
|
|
||||||
|
For example, above we copied a suite and resolved it:
|
||||||
|
#+begin_src sh :exports both :results output
|
||||||
|
cd ~/local/codeql-cli-end-to-end
|
||||||
|
cp custom-suite-1.qls codeql-workshop-vulnerable-linux-driver/solutions/
|
||||||
|
codeql resolve queries \
|
||||||
|
codeql-workshop-vulnerable-linux-driver/solutions/custom-suite-1.qls
|
||||||
|
#+end_src
|
||||||
|
#+results:
|
||||||
|
: /users/hohn/local/codeql-cli-end-to-end/codeql-workshop-vulnerable-linux-driver/solutions/useafterfree.ql
|
||||||
|
|
||||||
|
To assign a version number, we can use the revision id:
|
||||||
|
#+begin_src sh :exports both :results output
|
||||||
|
cd ~/local/codeql-cli-end-to-end
|
||||||
|
git rev-parse --short head
|
||||||
|
#+end_src
|
||||||
|
|
||||||
|
#+RESULTS:
|
||||||
|
: 94fd0a3
|
||||||
|
|
||||||
|
- For manually selected library suites
|
||||||
|
|
||||||
|
For a library suite, we can use the pack id. For example, we can
|
||||||
|
list the packs
|
||||||
|
#+BEGIN_SRC sh :exports both :results output
|
||||||
|
export PATH=$HOME/local/codeql-cli-end-to-end/codeql:"$PATH"
|
||||||
|
codeql resolve qlpacks | grep cpp
|
||||||
|
#+END_SRC
|
||||||
|
|
||||||
|
#+RESULTS:
|
||||||
|
: codeql/cpp-all (/Users/hohn/local/codeql-cli-end-to-end/codeql/qlpacks/codeql/cpp-all/0.7.3)
|
||||||
|
: codeql/cpp-examples (/Users/hohn/local/codeql-cli-end-to-end/codeql/qlpacks/codeql/cpp-examples/0.0.0)
|
||||||
|
: codeql/cpp-queries (/Users/hohn/local/codeql-cli-end-to-end/codeql/qlpacks/codeql/cpp-queries/0.6.3)
|
||||||
|
|
||||||
|
Following the last one, we can find some query suites manually.
|
||||||
|
The pack is already known; 0.6.3.
|
||||||
|
#+BEGIN_SRC sh :exports both :results output
|
||||||
|
find ~/local/codeql-cli-end-to-end/codeql/qlpacks/codeql/cpp-queries/0.6.3 \
|
||||||
|
-name "*.qls"
|
||||||
|
#+END_SRC
|
||||||
|
|
||||||
|
#+RESULTS:
|
||||||
|
: /Users/hohn/local/codeql-cli-end-to-end/codeql/qlpacks/codeql/cpp-queries/0.6.3/codeql-suites/cpp-security-extended.qls
|
||||||
|
: /Users/hohn/local/codeql-cli-end-to-end/codeql/qlpacks/codeql/cpp-queries/0.6.3/codeql-suites/cpp-security-and-quality.qls
|
||||||
|
: /Users/hohn/local/codeql-cli-end-to-end/codeql/qlpacks/codeql/cpp-queries/0.6.3/codeql-suites/cpp-security-experimental.qls
|
||||||
|
: /Users/hohn/local/codeql-cli-end-to-end/codeql/qlpacks/codeql/cpp-queries/0.6.3/codeql-suites/cpp-code-scanning.qls
|
||||||
|
: /Users/hohn/local/codeql-cli-end-to-end/codeql/qlpacks/codeql/cpp-queries/0.6.3/codeql-suites/cpp-lgtm-full.qls
|
||||||
|
: /Users/hohn/local/codeql-cli-end-to-end/codeql/qlpacks/codeql/cpp-queries/0.6.3/codeql-suites/cpp-lgtm.qls
|
||||||
|
|
||||||
|
- For predefined suites from =codeql resolve queries=
|
||||||
|
|
||||||
|
A full list of suites is produced via =codeql resolve queries=, here is a
|
||||||
|
filtered version.
|
||||||
|
#+BEGIN_SRC sh :exports both :results output
|
||||||
|
export PATH=$HOME/local/codeql-cli-end-to-end/codeql:"$PATH"
|
||||||
|
codeql resolve queries 2>&1 | grep cpp
|
||||||
|
#+END_SRC
|
||||||
|
|
||||||
|
#+RESULTS:
|
||||||
|
: cpp-code-scanning.qls - Standard Code Scanning queries for C and C++
|
||||||
|
: cpp-lgtm-full.qls - Standard LGTM queries for C/C++, including ones not displayed by default
|
||||||
|
: cpp-lgtm.qls - Standard LGTM queries for C/C++
|
||||||
|
: cpp-security-and-quality.qls - Security-and-quality queries for C and C++
|
||||||
|
: cpp-security-experimental.qls - Extended and experimental security queries for C and C++
|
||||||
|
: cpp-security-extended.qls - Security-extended queries for C and C++
|
||||||
|
|
||||||
|
The following just counts the list but notice the header output has version
|
||||||
|
info reported on =stderr=:
|
||||||
|
#+BEGIN_SRC sh :exports both :results output
|
||||||
|
export PATH=$HOME/local/codeql-cli-end-to-end/codeql:"$PATH"
|
||||||
|
( codeql resolve queries cpp-code-scanning.qls | wc ) 2>&1
|
||||||
|
#+END_SRC
|
||||||
|
|
||||||
|
#+RESULTS:
|
||||||
|
: Recording pack reference codeql/cpp-queries at /Users/hohn/local/codeql-cli-end-to-end/codeql/qlpacks/codeql/cpp-queries/0.6.3.
|
||||||
|
: Recording pack reference codeql/suite-helpers at /Users/hohn/local/codeql-cli-end-to-end/codeql/qlpacks/codeql/cpp-queries/0.6.3/.codeql/libraries/codeql/suite-helpers/0.5.3.
|
||||||
|
: 47 65 5813
|
||||||
|
|
||||||
|
So we can use the codeql/cpp-queries version, 0.6.3, if we run the
|
||||||
|
=cpp-code-scanning.qls= query suite.
|
||||||
|
|
||||||
|
The difference in the last two approaches is the way the suite is chosen. The
|
||||||
|
version number will be the same.
|
||||||
|
|
||||||
** Review results
|
** Review results
|
||||||
*** SARIF Documentation
|
*** SARIF Documentation
|
||||||
The standard is defined at
|
The standard is defined at
|
||||||
@@ -667,7 +855,6 @@ git checkout 203343df
|
|||||||
./codeql/codeql --version
|
./codeql/codeql --version
|
||||||
#+END_SRC
|
#+END_SRC
|
||||||
|
|
||||||
#+RESULTS:
|
|
||||||
: CodeQL command-line toolchain release 2.13.4.
|
: CodeQL command-line toolchain release 2.13.4.
|
||||||
: Copyright (C) 2019-2023 GitHub, Inc.
|
: Copyright (C) 2019-2023 GitHub, Inc.
|
||||||
: Unpacked in: /Users/hohn/local/codeql-cli-end-to-end/codeql
|
: Unpacked in: /Users/hohn/local/codeql-cli-end-to-end/codeql
|
||||||
@@ -698,3 +885,6 @@ git checkout 203343df
|
|||||||
*** Expand query
|
*** Expand query
|
||||||
** Compare results.
|
** Compare results.
|
||||||
*** sarif-cli using compiler-style dump
|
*** sarif-cli using compiler-style dump
|
||||||
|
** Miscellany
|
||||||
|
- Scale factor for building DBs: Common case: 15 minutes for a parallel cpp
|
||||||
|
compilation can be a 2 hour database build for codeql.
|
||||||
|
|||||||
Reference in New Issue
Block a user