|
|
|
|
@@ -15,7 +15,7 @@ There are two types of CodeQL packs: query packs and library packs.
|
|
|
|
|
You can use the package management commands in the CodeQL CLI to create CodeQL packs, add dependencies to packs, and install or update dependencies. For more information, see ":ref:`Creating and working with CodeQL packs <creating-and-working-with-codeql-packs>`." You can also publish and download CodeQL packs using the CodeQL CLI. For more information, see ":doc:`Publishing and using CodeQL packs <publishing-and-using-codeql-packs>`."
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
The standard CodeQL packages for all supported languages are published in the `GitHub package registry <https://github.com/orgs/codeql/packages>`__.`
|
|
|
|
|
The standard CodeQL packages for all supported languages are published in the `GitHub Container registry <https://github.com/orgs/codeql/packages>`__.`
|
|
|
|
|
The `CodeQL repository <https://github.com/github/codeql>`__ contains sources for the standard CodeQL packs for all supported languages.
|
|
|
|
|
|
|
|
|
|
.. _codeql-pack-structure:
|
|
|
|
|
@@ -59,7 +59,7 @@ The following properties are supported in ``qlpack.yml`` files.
|
|
|
|
|
|
|
|
|
|
* - Property
|
|
|
|
|
- Example
|
|
|
|
|
- Required
|
|
|
|
|
- Required by
|
|
|
|
|
- Purpose
|
|
|
|
|
* - ``name``
|
|
|
|
|
- .. code-block:: yaml
|
|
|
|
|
@@ -82,14 +82,14 @@ The following properties are supported in ``qlpack.yml`` files.
|
|
|
|
|
codeql/cpp-all: ^0.0.2
|
|
|
|
|
|
|
|
|
|
- Packs that define CodeQL package dependencies on other packs
|
|
|
|
|
- A map from pack references to the semantic version range that is compatible with this pack. Supported from CLI version v2.6.0 and onwards.
|
|
|
|
|
- A map from pack references to the semantic version range that is compatible with this pack. Supported for CLI versions v2.6.0 and later.
|
|
|
|
|
* - ``defaultSuiteFile``
|
|
|
|
|
- .. code-block:: yaml
|
|
|
|
|
|
|
|
|
|
defaultSuiteFile: cpp-code-scanning.qls
|
|
|
|
|
|
|
|
|
|
- Packs that export a set of default queries to run
|
|
|
|
|
- The path to a query suite file containing all of the queries that are run by default when this pack is passed to the ``codeql database analyze`` command. Supported from CLI version v2.6.0 and onwards.
|
|
|
|
|
- The path to a query suite file relative to the package root, containing all of the queries that are run by default when this pack is passed to the ``codeql database analyze`` command. Supported from CLI version v2.6.0 and onwards. Only one of ``defaultSuiteFile`` or ``defaultSuite`` can be defined.
|
|
|
|
|
* - ``defaultSuite``
|
|
|
|
|
- .. code-block:: yaml
|
|
|
|
|
|
|
|
|
|
@@ -99,7 +99,7 @@ The following properties are supported in ``qlpack.yml`` files.
|
|
|
|
|
precision: medium
|
|
|
|
|
|
|
|
|
|
- Packs that export a set of default queries to run
|
|
|
|
|
- An inlined query suite containing all of the queries that are run by default when this pack is passed to the ``codeql database analyze`` command. Supported from CLI version v2.6.0 and onwards.
|
|
|
|
|
- An inlined query suite containing all of the queries that are run by default when this pack is passed to the ``codeql database analyze`` command. Supported from CLI version v2.6.0 and onwards. Only one of ``defaultSuiteFile`` or ``defaultSuite`` can be defined.
|
|
|
|
|
* - ``library``
|
|
|
|
|
- .. code-block:: yaml
|
|
|
|
|
|
|
|
|
|
@@ -113,7 +113,7 @@ The following properties are supported in ``qlpack.yml`` files.
|
|
|
|
|
suites: octo-org-query-suites
|
|
|
|
|
|
|
|
|
|
- Optional
|
|
|
|
|
- The path to a directory in the pack that contains the query suites you want to make known to the CLI, defined relative to the pack directory. CodeQL pack users can run "well-known" suites stored in this directory by specifying the pack name, without providing their full path. This is not supported for CodeQL packs downloaded from a package registry. For more information about query suites, see ":doc:`Creating CodeQL query suites <creating-codeql-query-suites>`."
|
|
|
|
|
- The path to a directory in the pack that contains the query suites you want to make known to the CLI, defined relative to the pack directory. CodeQL pack users can run "well-known" suites stored in this directory by specifying the pack name, without providing their full path. This is not supported for CodeQL packs downloaded from the Container registry. For more information about query suites, see ":doc:`Creating CodeQL query suites <creating-codeql-query-suites>`."
|
|
|
|
|
* - ``extractor``
|
|
|
|
|
- .. code-block:: yaml
|
|
|
|
|
|
|
|
|
|
@@ -141,7 +141,7 @@ The following properties are supported in ``qlpack.yml`` files.
|
|
|
|
|
upgrades: .
|
|
|
|
|
|
|
|
|
|
- Core language packs only
|
|
|
|
|
- The path to a directory within the pack that contains upgrade scripts, defined relative to the pack directory. The ``database upgrade`` action uses these scripts to upgrade databases that were created by an older version of an extractor or downgrade databases that were created by a newer version of the extractor so they're compatible with the current extractor (see `Upgrade scripts for a language <#upgrade-scripts-for-a-language>`__ below.)
|
|
|
|
|
- The path to a directory within the pack that contains database upgrade scripts, defined relative to the pack directory. Database upgrades are used internally to ensure a databasae created with a different version of the CodeQL CLI is compatible with the current version.
|
|
|
|
|
* - ``authors``
|
|
|
|
|
- .. code-block:: yaml
|
|
|
|
|
|
|
|
|
|
@@ -169,7 +169,7 @@ The following properties are supported in ``qlpack.yml`` files.
|
|
|
|
|
libraryPathDependencies: codeql/javascript-all
|
|
|
|
|
|
|
|
|
|
- Optional, deprecated
|
|
|
|
|
- Use the ``dependencies`` property instead. The names of any CodeQL packs that this CodeQL pack depends on, as a sequence. This gives the pack access to any libraries, database schema, and query suites defined in the dependency.
|
|
|
|
|
- Use the ``dependencies`` property instead. The names of any CodeQL packs that this CodeQL pack depends on, as an array. This gives the pack access to any libraries, database schema, and query suites defined in the dependency.
|
|
|
|
|
|
|
|
|
|
.. _about-codeql-pack-lock:
|
|
|
|
|
|
|
|
|
|
@@ -199,6 +199,9 @@ The ``codeql-pack.lock.yml`` file will contain something like the following:
|
|
|
|
|
my-user/transitive-dependency:
|
|
|
|
|
version: 1.2.4
|
|
|
|
|
|
|
|
|
|
..
|
|
|
|
|
TODO: Add a link to the CodeQL CLI documentation for query resolution, specifically in regards to resolving from source
|
|
|
|
|
|
|
|
|
|
The ``codeql/cpp-all`` dependency is locked to version 0.1.4. The ``my-user/my-lib`` dependency is locked to version 0.2.4. The ``my-user/transitive-dependency``, which is a transitive dependency and is not specified in the ``qlpack.yml`` file, is locked to version 1.2.4. The ``other-dependency/from-source`` is absent from the lock file since it is resolved from source. This dependency must be available in the same CodeQL workspace as the pack.
|
|
|
|
|
|
|
|
|
|
.. _custom-codeql-packs:
|
|
|
|
|
@@ -208,13 +211,27 @@ Examples of custom CodeQL packs
|
|
|
|
|
|
|
|
|
|
When you write custom queries or tests, you should save them in custom CodeQL packs. For simplicity, try to organize each pack logically. For more information, see `CodeQL pack structure <#codeql-pack-structure>`__. Save files for queries and tests in separate packs and, where possible, organize custom packs into specific folders for each target language. This is particuarly useful if you intend to publish your CodeQL packs so they can be shared with others or used in GitHub `Code scanning <https://docs.github.com/en/code-security/secure-coding/automatically-scanning-your-code-for-vulnerabilities-and-errors/about-code-scanning>`__.
|
|
|
|
|
|
|
|
|
|
CodeQL packs for custom libraries
|
|
|
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
|
|
|
|
|
|
A custom CodeQL pack containing custom C++ libraries, with no queries or tests, may have a ``qlpack.yml`` file containing:
|
|
|
|
|
|
|
|
|
|
.. code-block:: yaml
|
|
|
|
|
|
|
|
|
|
name: my-github-user/my-custom-libraries
|
|
|
|
|
version: 1.2.3
|
|
|
|
|
library: true
|
|
|
|
|
dependencies:
|
|
|
|
|
codeql/cpp-all: ^0.1.2
|
|
|
|
|
|
|
|
|
|
where ``codeql/cpp-all`` is the name of the CodeQL pack for C/C++ analysis included in the CodeQL repository. The version range ``^0.1.2`` indicates that this pack is compatible with all versions of ``codeql/cpp-all`` that are greater than or equal to ``0.1.2`` and less than ``0.2.0``. Any CodeQL library file (a file with a ``.qll`` extension) defined in this pack will be available to queries defined in any query pack that includes this pack in its dependencies block.
|
|
|
|
|
|
|
|
|
|
The ``library`` property indicates that this pack is a library pack and does not contain any queries.
|
|
|
|
|
|
|
|
|
|
CodeQL packs for custom queries
|
|
|
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
|
|
|
|
|
|
A custom CodeQL pack for queries must include a ``qlpack.yml`` file at the pack root, containing ``name``, ``version``, and ``dependencies`` properties. If the pack contains query suites, you can use the ``suites`` property to define their location. Query suites defined here are called "well-known" suites, and can be used on the command line by referring to their name only, rather than their full path. For more information about query suites, see ":doc:`Creating CodeQL query suites <creating-codeql-query-suites>`."
|
|
|
|
|
|
|
|
|
|
For example, a ``qlpack.yml`` file for a CodeQL pack featuring custom C++ queries
|
|
|
|
|
and libraries may contain:
|
|
|
|
|
A custom CodeQL pack containing custom C++ queries and libraries may have a ``qlpack.yml`` file containing:
|
|
|
|
|
|
|
|
|
|
.. code-block:: yaml
|
|
|
|
|
|
|
|
|
|
@@ -222,12 +239,15 @@ and libraries may contain:
|
|
|
|
|
version: 1.2.3
|
|
|
|
|
dependencies:
|
|
|
|
|
codeql/cpp-all: ^0.1.2
|
|
|
|
|
my-github-user/my-custom-libraries: ^1.2.3
|
|
|
|
|
suites: my-custom-suites
|
|
|
|
|
|
|
|
|
|
where ``codeql/cpp-all`` is the name of the CodeQL pack for C/C++ analysis included in the CodeQL repository. The version range ``^0.1.2`` indicates that this pack is compatible with all versions of ``codeql/cpp-all`` that are greater than or equal to ``0.1.2`` and less than ``0.2.0``.
|
|
|
|
|
where ``codeql/cpp-all`` is the name of the CodeQL pack for C/C++ analysis included in the CodeQL repository. The version range ``^0.1.2`` indicates that this pack is compatible with all versions of ``codeql/cpp-all`` that are greater than or equal to ``0.1.2`` and less than ``0.2.0``. ``my-github-user/my-custom-libraries`` is the name of a CodeQL pack containing custom CodeQL libraries for C++. Any CodeQL library file (a file with a ``.qll`` extension) defined in this pack will be available to queries in the ``my-github-user/my-custom-queries`` pack.
|
|
|
|
|
|
|
|
|
|
CodeQL packs for custom test files
|
|
|
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
|
The ``suites`` property indicates a directory where "well-known" query suites can be found. These suites can be used on the command line by referring to their name only, rather than their full path. For more information about query suites, see ":doc:`Creating CodeQL query suites <creating-codeql-query-suites>`."
|
|
|
|
|
|
|
|
|
|
CodeQL packs for custom tests
|
|
|
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
|
|
|
|
|
|
For custom CodeQL packs containing test files, you also need to include an
|
|
|
|
|
``extractor`` property so that the ``test run`` command knows how to create test
|
|
|
|
|
@@ -246,23 +266,18 @@ Examples of CodeQL packs in the CodeQL repository
|
|
|
|
|
Each of the languages in the CodeQL repository has four main CodeQL packs:
|
|
|
|
|
|
|
|
|
|
- Core library pack for the language, with the :ref:`database schema <codeql-database-schema>`
|
|
|
|
|
used by the language, and CodeQL libraries, and queries at ``ql/<language>/ql/lib``
|
|
|
|
|
used by the language, and CodeQL libraries, and queries at ``<language>/ql/lib``
|
|
|
|
|
- Core query pack for the language that includes the default queries for the language, along
|
|
|
|
|
with their query suites at ``ql/<language>/ql/src``
|
|
|
|
|
- Tests for the core language libraries and queries at ``ql/<language>/ql/test``
|
|
|
|
|
- Example queries for the language at ``ql/<language>/ql/examples``
|
|
|
|
|
with their query suites at ``<language>/ql/src``
|
|
|
|
|
- Tests for the core language libraries and queries at ``<language>/ql/test``
|
|
|
|
|
- Example queries for the language at ``<language>/ql/examples``
|
|
|
|
|
|
|
|
|
|
Core library pack
|
|
|
|
|
~~~~~~~~~~~~~~~~~
|
|
|
|
|
|
|
|
|
|
The ``qlpack.yml`` file for a core library pack uses the following properties:
|
|
|
|
|
``name``, ``version``, ``dbscheme``.
|
|
|
|
|
The ``dbscheme`` property should only be defined in the core QL
|
|
|
|
|
pack for a language.
|
|
|
|
|
|
|
|
|
|
For example, the ``qlpack.yml`` file for `C/C++ analysis libraries
|
|
|
|
|
Here is an example ``qlpack.yml`` file for `C/C++ analysis libraries
|
|
|
|
|
<https://github.com/github/codeql/blob/main/cpp/ql/lib/qlpack.yml>`__
|
|
|
|
|
contains:
|
|
|
|
|
core language pack:
|
|
|
|
|
|
|
|
|
|
.. code-block:: yaml
|
|
|
|
|
|
|
|
|
|
@@ -272,15 +287,18 @@ contains:
|
|
|
|
|
library: true
|
|
|
|
|
upgrades: upgrades
|
|
|
|
|
|
|
|
|
|
Some extra notes on the following properties:
|
|
|
|
|
|
|
|
|
|
- ``library``: Indicates that this is a library pack with no executable queries. It is only meant to be used as a dependency for other packs.
|
|
|
|
|
- ``dbscheme`` and ``upgrades``: Properties internal to the CodeQL CLI and should only be defined in the core QL pack for a language.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Core query pack
|
|
|
|
|
~~~~~~~~~~~~~~~
|
|
|
|
|
|
|
|
|
|
The ``qlpack.yml`` file for a core query pack uses the following properties:
|
|
|
|
|
``name``, ``version``, ``suites``, ``defaultSuiteFile``, ``dependencies`` .
|
|
|
|
|
|
|
|
|
|
For example, the ``qlpack.yml`` file for `C/C++ analysis queries
|
|
|
|
|
<https://github.com/github/codeql/blob/main/cpp/ql/lib/qlpack.yml>`__
|
|
|
|
|
contains:
|
|
|
|
|
Here is an example ``qlpack.yml`` file for `C/C++ analysis queries
|
|
|
|
|
<https://github.com/github/codeql/blob/main/cpp/ql/src/qlpack.yml>`__
|
|
|
|
|
core query pack:
|
|
|
|
|
|
|
|
|
|
.. code-block:: yaml
|
|
|
|
|
|
|
|
|
|
@@ -292,16 +310,18 @@ contains:
|
|
|
|
|
suites: codeql-suites
|
|
|
|
|
defaultSuiteFile: codeql-suites/cpp-code-scanning.qls
|
|
|
|
|
|
|
|
|
|
This query pack depends on ``codeql/cpp-all`` and ``codeql/suite-helpers``. Since these dependencies are resolved from source, it does not matter what version of the CodeQL pack they are compatible with.
|
|
|
|
|
Some extra notes on the following properties:
|
|
|
|
|
|
|
|
|
|
- ``dependencies``: This query pack depends on ``codeql/cpp-all`` and ``codeql/suite-helpers``. Since these dependencies are resolved from source, it does not matter what version of the CodeQL pack they are compatible with.
|
|
|
|
|
- ``suites``: Indicates the directory containing "well-known" query suites.
|
|
|
|
|
- ``defaultSuiteFile``: The name of the default query suite file that is used when no query suite is specified.
|
|
|
|
|
|
|
|
|
|
Tests for the core CodeQL pack
|
|
|
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
|
|
|
|
|
|
The ``qlpack.yml`` file for the tests for the core CodeQL packs use the following properties: ``name``, ``dependencies``, ``extractor``, ``tests``.
|
|
|
|
|
|
|
|
|
|
For example, the ``qlpack.yml`` file for `C/C++ analysis tests
|
|
|
|
|
<https://github.com/github/codeql/blob/main/cpp/ql/test/qlpack.yml>`__
|
|
|
|
|
contains:
|
|
|
|
|
Here is an example ``qlpack.yml`` file for `C/C++ analysis tests
|
|
|
|
|
<https://github.com/github/codeql/blob/main/cpp/ql/src/qlpack.yml>`__
|
|
|
|
|
core test pack:
|
|
|
|
|
|
|
|
|
|
.. code-block:: yaml
|
|
|
|
|
|
|
|
|
|
@@ -312,4 +332,9 @@ contains:
|
|
|
|
|
extractor: cpp
|
|
|
|
|
tests: .
|
|
|
|
|
|
|
|
|
|
The ``dependencies`` property always specifies the core CodeQL pack. The ``extractor`` property specifies that all the tests will use the same C++ extractor to create the database for the tests. The ``tests`` property specifies the location of the tests. In this case, the tests are in the root folder (and all sub-folders) of the pack. Note that there is no ``version`` property for the tests pack since this pack is not meant to be published.
|
|
|
|
|
Some extra notes on the following properties:
|
|
|
|
|
|
|
|
|
|
- ``dependencies``: This pack depends on the core CodeQL query and library packs for C++.
|
|
|
|
|
- ``extractor``: This specifies that all the tests will use the same C++ extractor to create the database for the tests.
|
|
|
|
|
- ``tests``: This specifies the location of the tests. In this case, the tests are in the root folder (and all sub-folders) of the pack.
|
|
|
|
|
- ``version``: There is no ``version`` property for the tests pack. This prevents test packs from accidentally being published.
|
|
|
|
|
|
Andrew Eisenberg