mirror of
https://github.com/github/codeql.git
synced 2026-02-01 07:42:57 +01:00
Merge and update about-ql-packs with about-codeql-packs
This is the first of a series of commits around updating packaging docs. `about-ql-packs.rst` is outdated. All relevant information has been moved to about-codeql-packs.rst`.
This commit is contained in:
Andrew Eisenberg
@@ -3,8 +3,6 @@
|
||||
About CodeQL packs
|
||||
==================
|
||||
|
||||
.. include:: ../reusables/beta-note-package-management.rst
|
||||
|
||||
CodeQL packs are used to create, share, depend on, and run CodeQL queries and libraries. You can publish your own CodeQL packs and download packs created by others. CodeQL packs contain queries, library files, query suites, and metadata.
|
||||
|
||||
There are two types of CodeQL packs: query packs and library packs.
|
||||
@@ -14,11 +12,18 @@ There are two types of CodeQL packs: query packs and library packs.
|
||||
|
||||
You can use the package management commands in the CodeQL CLI to create CodeQL packs, add dependencies to packs, and install or update dependencies. For more information, see ":ref:`Creating and working with CodeQL packs <creating-and-working-with-codeql-packs>`." You can also publish and download CodeQL packs using the CodeQL CLI. For more information, see ":doc:`Publishing and using CodeQL packs <publishing-and-using-codeql-packs>`."
|
||||
|
||||
|
||||
The standard CodeQL packages for all supported languages are published in the `GitHub package registry <https://github.com/orgs/codeql/packages>`__.`
|
||||
The `CodeQL repository <https://github.com/github/codeql>`__ contains sources for the standard CodeQL packs for all supported languages.
|
||||
|
||||
.. _codeql-pack-structure:
|
||||
|
||||
CodeQL pack structure
|
||||
---------------------
|
||||
|
||||
A CodeQL pack must contain a file called ``qlpack.yml`` in its root directory. In the ``qlpack.yml`` file, the ``name:`` field must have a value that follows the format of ``<scope>/<pack>``, where ``<scope>`` is the GitHub organization or user account that the pack will be published to and ``<pack>`` is the name of the pack. The other
|
||||
files and directories within the pack should be logically organized. For example, typically:
|
||||
A CodeQL pack must contain a file called ``qlpack.yml`` in its root directory. In the ``qlpack.yml`` file, the ``name:`` field must have a value that follows the format of ``<scope>/<pack>``, where ``<scope>`` is the GitHub organization or user account that the pack will be published to and ``<pack>`` is the name of the pack. Additionally, query packs contain a ``codeql-pack.lock.yml`` file that contains the resolved dependencies of the pack. This file is generated during a call to the ``codeql pack install`` command, is not meant to be edited by hand, and should be added to your version control system.
|
||||
|
||||
The other files and directories within the pack should be logically organized. For example, typically:
|
||||
|
||||
- Queries are organized into directories for specific categories.
|
||||
- Queries for specific products, libraries, and frameworks are organized into
|
||||
@@ -27,21 +32,17 @@ files and directories within the pack should be logically organized. For example
|
||||
About ``qlpack.yml`` files
|
||||
--------------------------
|
||||
|
||||
When executing query-related commands, CodeQL first looks in siblings of the installation directory (and their subdirectories) for ``qlpack.yml`` files.
|
||||
Then it checks the package cache for CodeQL packs which have been downloaded. This means that when you are developing queries locally, the local packages
|
||||
When executing query-related commands, CodeQL first looks in siblings of the installation directory (and their subdirectories) for ``qlpack.yml`` files.
|
||||
Then it checks the package cache for CodeQL packs which have been downloaded. This means that when you are developing queries locally, the local packages
|
||||
in the installation directory override packages of the same name in the package cache, so that you can test your local changes.
|
||||
|
||||
The metadata in each `qlpack.yml`` file tells
|
||||
CodeQL how to compile any queries in the pack, what libraries the pack depends on, and where to
|
||||
find query suite definitions.
|
||||
|
||||
The contents of the CodeQL pack (queries or libraries used in CodeQL analysis) is
|
||||
included in the same directory as ``qlpack.yml``, or its subdirectories.
|
||||
The contents of the CodeQL pack (queries or libraries used in CodeQL analysis) is included in the same directory as ``qlpack.yml``, or its subdirectories.
|
||||
|
||||
The location of ``qlpack.yml`` defines the library path for the content
|
||||
of the CodeQL pack. That is, for all ``.ql`` and ``.qll`` files in the pack,
|
||||
CodeQL will resolve all import statements relative to the ``qlpack.yml`` at the
|
||||
pack's root.
|
||||
The location of ``qlpack.yml`` defines the library path for the content of the CodeQL pack. That is, for all ``.ql`` and ``.qll`` files in the pack, CodeQL will resolve all import statements relative to the ``qlpack.yml`` at the pack's root.
|
||||
|
||||
.. _codeqlpack-yml-properties:
|
||||
|
||||
@@ -59,19 +60,41 @@ The following properties are supported in ``qlpack.yml`` files.
|
||||
- Required
|
||||
- Purpose
|
||||
* - ``name``
|
||||
- ``octo-org/security-queries``
|
||||
- ``name: octo-org/security-queries``
|
||||
- All packs
|
||||
- The scope, where the CodeQL pack is published, and the name of the pack defined using alphanumeric characters and hyphens. It must be unique as CodeQL cannot differentiate between CodeQL packs with identical names. Name components cannot start or end with a hyphen. Additionally, a period is not allowed in pack names at all. Use the pack name to specify queries to run using ``database analyze`` and to define dependencies between QL packs (see examples below).
|
||||
- The scope, where the CodeQL pack is published, and the name of the pack defined using alphanumeric characters and hyphens. It must be unique as CodeQL cannot differentiate between CodeQL packs with identical names. Use the pack name to specify queries to run using ``database analyze`` and to define dependencies between QL packs (see examples below).
|
||||
* - ``version``
|
||||
- ``0.0.0``
|
||||
- ``version: 0.0.0``
|
||||
- All packs
|
||||
- A version range for this CodeQL pack. This must be a valid semantic version that meets the `SemVer v2.0.0 specification <https://semver.org/spec/v2.0.0.html>`__.
|
||||
- A semantic version for this CodeQL pack that must adhere to the `SemVer v2.0.0 specification <https://semver.org/spec/v2.0.0.html>`__.
|
||||
* - ``dependencies``
|
||||
- ``codeql/javascript-all: ^1.2.3``
|
||||
- Optional
|
||||
- The names and version ranges of any CodeQL packs that this pack depends on, as a mapping. This gives the pack access to any libraries, database schema, and query suites defined in the dependency. For more information, see `SemVer ranges <https://docs.npmjs.com/cli/v6/using-npm/semver#ranges>`__ in the NPM documentation.
|
||||
- .. code-block:: yaml
|
||||
|
||||
dependencies:
|
||||
codeql/cpp-all: ^0.0.2
|
||||
|
||||
- Packs that define CodeQL package dependencies on other packs
|
||||
- A map from pack references to the semantic version range that is compatible with this pack. Supported from CLI version v2.6.0 and onwards.
|
||||
* - ``defaultSuiteFile``
|
||||
- ``defaultSuiteFile: cpp-code-scanning.qls``
|
||||
- Packs that export a set of default queries to run
|
||||
- The path to a query suite file containing all of the queries that are run by default when this pack is passed to the ``codeql database analyze`` command. Supported from CLI version v2.6.0 and onwards.
|
||||
* - ``defaultSuite``
|
||||
- .. code-block:: yaml
|
||||
|
||||
defaultSuite:
|
||||
queries: .
|
||||
exclude:
|
||||
precision: medium
|
||||
|
||||
- Packs that export a set of default queries to run
|
||||
- An inlined query suite containing all of the queries that are run by default when this pack is passed to the ``codeql database analyze`` command. Supported from CLI version v2.6.0 and onwards.
|
||||
* - ``library``
|
||||
- ``library: true``
|
||||
- Library packs
|
||||
- A boolean value that indicates whether this pack is a library pack. Library packs do not contain queries and are not compiled. Query packs can ignore this field or explicitly set it to ``false``.
|
||||
* - ``suites``
|
||||
- ``octo-org-query-suites``
|
||||
- ``suites: octo-org-query-suites``
|
||||
- Optional
|
||||
- The path to a directory in the pack that contains the query suites you want to make known to the CLI, defined relative to the pack directory. QL pack users can run "well-known" suites stored in this directory by specifying the pack name, without providing their full path. This is not supported for CodeQL packs downloaded from a package registry. For more information about query suites, see ":doc:`Creating CodeQL query suites <creating-codeql-query-suites>`."
|
||||
* - ``extractor``
|
||||
@@ -82,14 +105,14 @@ The following properties are supported in ``qlpack.yml`` files.
|
||||
- ``.``
|
||||
- Optional for test packs
|
||||
- The path to a directory within the pack that contains tests, defined relative to the pack directory. Use ``.`` to specify the whole pack. Any queries in this directory are run as tests when ``test run`` is run with the ``--strict-test-discovery`` option. These queries are ignored by query suite definitions that use ``queries`` or ``qlpack`` instructions to ask for all queries in a particular pack.
|
||||
* - ``dbscheme``
|
||||
* - ``dbscheme`` TODO: Internal only. Delete?
|
||||
- ``semmlecode.python.dbscheme``
|
||||
- Core language packs only
|
||||
- The path to the :ref:`database schema <codeql-database-schema>` for all libraries and queries written for this CodeQL language (see example below).
|
||||
* - ``upgrades``
|
||||
* - ``upgrades`` TODO: Internal only. Delete?
|
||||
- ``.``
|
||||
- Core language packs only
|
||||
- The path to a directory within the pack that contains upgrade scripts, defined relative to the pack directory. The ``database upgrade`` action uses these scripts to update databases that were created by an older version of an extractor so they're compatible with the current extractor (see `Upgrade scripts for a language <#upgrade-scripts-for-a-language>`__ below.)
|
||||
- The path to a directory within the pack that contains upgrade scripts, defined relative to the pack directory. The ``database upgrade`` action uses these scripts to upgrade databases that were created by an older version of an extractor or downgrade databases that were created by a newer version of the extractor so they're compatible with the current extractor (see `Upgrade scripts for a language <#upgrade-scripts-for-a-language>`__ below.)
|
||||
* - ``authors``
|
||||
- ``example@github.com``
|
||||
- All packs
|
||||
@@ -97,8 +120,151 @@ The following properties are supported in ``qlpack.yml`` files.
|
||||
* - ``licenses``
|
||||
- ``(LGPL-2.1 AND MIT)``
|
||||
- All packs
|
||||
- Metadata that will be displayed on the packaging search page in the packages section of the account that the CodeQL pack is published to. For a list of allowed licenses, see `SPDX License List <https://spdx.org/licenses/>`__ in the SPDX Specification.
|
||||
- Metadata that will be displayed on the packaging search page in the packages section of the account that the CodeQL pack is published to. For a list of allowed licenses, see `SPDX License List <https://spdx.org/licenses/>`__ in the SPDX Specification.
|
||||
* - ``description``
|
||||
- ``Human-readable description of the contents of the CodeQL pack.``
|
||||
- All packs
|
||||
- Metadata that will be displayed on the packaging search page in the packages section of the account that the CodeQL pack is published to.
|
||||
|
||||
.. _about-codeql-pack-lock:
|
||||
|
||||
About ``codeql-pack.lock.yml`` files
|
||||
------------------------------------
|
||||
|
||||
``codeql-pack.lock.yml`` files store the versions of the resolved transitive dependencies of a CodeQL pack. This file is created by the ``codeql pack install`` if it does not already exist and should be added to your version control system. The ``dependencies`` section of the ``qlpack.yml`` file contains version ranges that are compatible with the pack. The ``codeql-pack.lock.yml`` file locks the versions to precise dependencies. This ensures that anyone running ``codeql pack install`` on this the pack will always retrieve the same versions of dependencies even if newer compatible versions exist. Only query packs can have ``codeql-pack.lock.yml`` files. This is because library packs are not compiled independently of query packs. Without being compiled, there is no need to specify precise versions of their dependencies.
|
||||
|
||||
For example, if a ``qlpack.yml`` file contains the following dependencies:
|
||||
|
||||
.. code-block:: yaml
|
||||
|
||||
dependencies:
|
||||
codeql/cpp-all: ^0.1.2
|
||||
my-user/my-lib: ^0.2.3
|
||||
other-dependency/from-source: "*"
|
||||
|
||||
The ``codeql-pack.lock.yml`` file will contain something like the following:
|
||||
|
||||
.. code-block:: yaml
|
||||
|
||||
dependencies:
|
||||
codeql/cpp-all:
|
||||
version: 0.1.4
|
||||
my-user/my-lib:
|
||||
version: 0.2.4
|
||||
my-user/transitive-dependency:
|
||||
version: 1.2.4
|
||||
|
||||
The ``codeql/cpp-all`` dependency is locked to version 0.1.4. The ``my-user/my-lib`` dependency is locked to version 0.2.4. The ``my-user/transitive-dependency``, which is a transitive dependency and is not specified in the ``qlpack.yml`` file, is locked to version 1.2.4. The ``other-dependency/from-source`` is absent from the lock file since it is resolved from source. This dependency must be available in the same CodeQL workspace as the pack.
|
||||
|
||||
.. _custom-codeql-packs:
|
||||
|
||||
Examples of custom CodeQL packs
|
||||
-------------------------------
|
||||
|
||||
When you write custom queries or tests, you should save them in custom CodeQL packs. For simplicity, try to organize each pack logically. For more information, see `CodeQL pack structure <#codeql-pack-structure>`__. Save files for queries and tests in separate packs and, where possible, organize custom packs into specific folders for each target language. This is particuarly useful if you intend to publish your CodeQL packs so they can be shared with others or used in GitHub `Code scanning <https://docs.github.com/en/code-security/secure-coding/automatically-scanning-your-code-for-vulnerabilities-and-errors/about-code-scanning>`__.
|
||||
|
||||
CodeQL packs for custom queries
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
A custom QL pack for queries must include a ``qlpack.yml`` file at the pack root, containing ``name``, ``version``, and ``dependencies`` properties. If the pack contains query suites, you can use the ``suites`` property to define their location. Query suites defined here are called "well-known" suites, and can be used on the command line by referring to their name only, rather than their full path. For more information about query suites, see ":doc:`Creating CodeQL query suites <creating-codeql-query-suites>`."
|
||||
|
||||
For example, a ``qlpack.yml`` file for a QL pack featuring custom C++ queries
|
||||
and libraries may contain:
|
||||
|
||||
.. code-block:: yaml
|
||||
|
||||
name: my-github-user/my-custom-queries
|
||||
version: 1.2.3
|
||||
dependencies:
|
||||
codeql/cpp-all: ^0.1.2
|
||||
suites: my-custom-suites
|
||||
|
||||
where ``codeql/cpp-all`` is the name of the QL pack for C/C++ analysis included in the CodeQL repository. The version range ``^0.1.2`` indicates that this pack is compatible with all versions of ``codeql/cpp-all`` that are greater than or equal to ``0.1.2`` and less than ``0.2.0``.
|
||||
|
||||
CodeQL packs for custom test files
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
For custom CodeQL packs containing test files, you also need to include an
|
||||
``extractor`` property so that the ``test run`` command knows how to create test
|
||||
databases. You may also wish to specify the ``tests`` property.
|
||||
|
||||
.. include:: ../reusables/test-qlpack.rst
|
||||
|
||||
For more information about running tests, see ":doc:`Testing custom queries
|
||||
<testing-custom-queries>`."
|
||||
|
||||
.. _standard-ql-packs:
|
||||
|
||||
Examples of CodeQL packs in the CodeQL repository
|
||||
---------------------------------------------~~~~
|
||||
|
||||
Each of the languages in the CodeQL repository has four main CodeQL packs:
|
||||
|
||||
- Core library pack for the language, with the :ref:`database schema <codeql-database-schema>`
|
||||
used by the language, and CodeQL libraries, and queries at ``ql/<language>/ql/lib``
|
||||
- Core query pack for the language that includes the default queries for the language, along
|
||||
with their query suites at ``ql/<language>/ql/src``
|
||||
- Tests for the core language libraries and queries at ``ql/<language>/ql/test``
|
||||
- Example queries for the language at ``ql/<language>/ql/examples``
|
||||
|
||||
Core library pack
|
||||
~~~~~~~~~~~~~~~~~
|
||||
|
||||
The ``qlpack.yml`` file for a core library pack uses the following properties:
|
||||
``name``, ``version``, ``dbscheme``.
|
||||
The ``dbscheme`` property should only be defined in the core QL
|
||||
pack for a language.
|
||||
|
||||
For example, the ``qlpack.yml`` file for `C/C++ analysis libraries
|
||||
<https://github.com/github/codeql/blob/main/cpp/ql/lib/qlpack.yml>`__
|
||||
contains:
|
||||
|
||||
.. code-block:: yaml
|
||||
|
||||
name: codeql/cpp-all
|
||||
version: x.y.z-dev
|
||||
dbscheme: semmlecode.cpp.dbscheme
|
||||
library: true
|
||||
upgrades: upgrades
|
||||
|
||||
Core query pack
|
||||
~~~~~~~~~~~~~~~
|
||||
|
||||
The ``qlpack.yml`` file for a core query pack uses the following properties:
|
||||
``name``, ``version``, ``suites``, ``defaultSuiteFile``, ``dependencies`` .
|
||||
|
||||
For example, the ``qlpack.yml`` file for `C/C++ analysis queries
|
||||
<https://github.com/github/codeql/blob/main/cpp/ql/lib/qlpack.yml>`__
|
||||
contains:
|
||||
|
||||
.. code-block:: yaml
|
||||
|
||||
name: codeql/cpp-queries
|
||||
version: x.y.z-dev
|
||||
dependencies:
|
||||
codeql/cpp-all: "*"
|
||||
codeql/suite-helpers: "*"
|
||||
suites: codeql-suites
|
||||
defaultSuiteFile: codeql-suites/cpp-code-scanning.qls
|
||||
|
||||
This query pack depends on ``codeql/cpp-all`` and ``codeql/suite-helpers``. Since these dependencies are resolved from source, it does not matter what version of the CodeQL pack they are compatible with.
|
||||
|
||||
Tests for the core QL pack
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
The ``qlpack.yml`` file for the tests for the core QL packs use the following properties: ``name``, ``dependencies``, ``extractor``, ``tests``.
|
||||
|
||||
For example, the ``qlpack.yml`` file for `C/C++ analysis tests
|
||||
<https://github.com/github/codeql/blob/main/cpp/ql/test/qlpack.yml>`__
|
||||
contains:
|
||||
|
||||
.. code-block:: yaml
|
||||
|
||||
name: codeql/cpp-tests
|
||||
dependencies:
|
||||
codeql/cpp-all: "*"
|
||||
codeql/cpp-queries: "*"
|
||||
extractor: cpp
|
||||
tests: .
|
||||
|
||||
The ``dependencies`` property always specifies the core QL pack. The ``extractor`` property specifies that all the tests will use the same C++ extractor to create the database for the tests. The ``tests`` property specifies the location of the tests. In this case, the tests are in the root folder (and all sub-folders) of the pack. Note that there is no ``version`` property for the tests pack since this pack is not meant to be published.
|
||||
|
||||
@@ -1,245 +0,0 @@
|
||||
.. _about-ql-packs:
|
||||
|
||||
About QL packs
|
||||
==============
|
||||
|
||||
QL packs are used to organize the files used in CodeQL analysis. They
|
||||
contain queries, library files, query suites, and important metadata.
|
||||
|
||||
The `CodeQL repository <https://github.com/github/codeql>`__ contains standard QL packs for all supported languages.
|
||||
You can also make custom QL packs to contain your own queries and libraries.
|
||||
|
||||
QL pack structure
|
||||
-----------------
|
||||
|
||||
A QL pack must contain a file called ``qlpack.yml`` in its root directory. The other
|
||||
files and directories within the pack should be logically organized. For example, typically:
|
||||
|
||||
- Queries are organized into directories for specific categories.
|
||||
- Queries for specific products, libraries, and frameworks are organized into
|
||||
their own top-level directories.
|
||||
- There is a top-level directory named ``<owner>/<language>`` for query library
|
||||
(``.qll``) files. Within this directory, ``.qll`` files should be organized into
|
||||
subdirectories for specific categories.
|
||||
|
||||
About ``qlpack.yml`` files
|
||||
--------------------------
|
||||
|
||||
When executing commands, CodeQL scans siblings of the installation directory (and
|
||||
their subdirectories) for ``qlpack.yml`` files. The metadata in the file tells
|
||||
CodeQL how to compile queries, what libraries the pack depends on, and where to
|
||||
find query suite definitions.
|
||||
|
||||
The content of the QL pack (queries and libraries used in CodeQL analysis) is
|
||||
included in the same directory as ``qlpack.yml``, or its subdirectories.
|
||||
|
||||
The location of ``qlpack.yml`` defines the library path for the content
|
||||
of the QL pack. That is, for all ``.ql`` and ``.qll`` files in the QL pack,
|
||||
CodeQL will resolve all import statements relative to the ``qlpack.yml`` at the
|
||||
pack's root.
|
||||
|
||||
For example, in a QL pack with the following contents, you can import ``CustomSinks.qll``
|
||||
from any location in the pack by declaring ``import mycompany.java.CustomSinks``.
|
||||
|
||||
.. code-block:: none
|
||||
|
||||
qlpack.yml
|
||||
mycompany/
|
||||
java/
|
||||
security/
|
||||
CustomSinks.qll
|
||||
Security/
|
||||
CustomQuery.ql
|
||||
|
||||
For more information, see ":ref:`Importing modules <importing-modules>`"
|
||||
in the QL language reference.
|
||||
|
||||
.. _qlpack-yml-properties:
|
||||
|
||||
``qlpack.yml`` properties
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
The following properties are supported in ``qlpack.yml`` files.
|
||||
|
||||
.. list-table::
|
||||
:header-rows: 1
|
||||
:widths: auto
|
||||
|
||||
* - Property
|
||||
- Example
|
||||
- Required
|
||||
- Purpose
|
||||
* - ``name``
|
||||
- ``org-queries``
|
||||
- All packs
|
||||
- The name of the QL pack defined using alphanumeric characters, hyphens, and periods. It must be unique as CodeQL cannot differentiate between QL packs with identical names. If you intend to distribute the pack, prefix the name with your (or your organization's) name followed by a hyphen. Use the pack name to specify queries to run using ``database analyze`` and to define dependencies between QL packs (see examples below).
|
||||
* - ``version``
|
||||
- ``0.0.0``
|
||||
- All packs
|
||||
- A version number for this QL pack. This must be a valid semantic version that meets the `SemVer v2.0.0 specification <https://semver.org/spec/v2.0.0.html>`__.
|
||||
* - ``libraryPathDependencies``
|
||||
- ``codeql/javascript-all``
|
||||
- Optional
|
||||
- The names of any QL packs that this QL pack depends on, as a sequence. This gives the pack access to any libraries, database schema, and query suites defined in the dependency.
|
||||
* - ``suites``
|
||||
- ``suites``
|
||||
- Optional
|
||||
- The path to a directory in the pack that contains the query suites you want to make known to the CLI, defined relative to the pack directory. QL pack users can run "well-known" suites stored in this directory by specifying the pack name, without providing their full path. For more information about query suites, see ":doc:`Creating CodeQL query suites <creating-codeql-query-suites>`."
|
||||
* - ``extractor``
|
||||
- ``javascript``
|
||||
- All test packs
|
||||
- The CodeQL language extractor to use when the CLI creates a database from test files in the pack. For more information about testing queries, see ":doc:`Testing custom queries <testing-custom-queries>`."
|
||||
* - ``tests``
|
||||
- ``.``
|
||||
- Optional for test packs
|
||||
- Supported from release 2.1.0 onwards. The path to a directory within the pack that contains tests, defined relative to the pack directory. Use ``.`` to specify the whole pack. Any queries in this directory are run as tests when ``test run`` is run with the ``--strict-test-discovery`` option. These queries are ignored by query suite definitions that use ``queries`` or ``qlpack`` instructions to ask for all queries in a particular pack.
|
||||
* - ``dbscheme``
|
||||
- ``semmlecode.python.dbscheme``
|
||||
- Core language pack only
|
||||
- The path to the :ref:`database schema <codeql-database-schema>` for all libraries and queries written for this CodeQL language (see example below).
|
||||
* - ``upgrades``
|
||||
- ``.``
|
||||
- Packs with upgrades
|
||||
- The path to a directory within the pack that contains upgrade scripts, defined relative to the pack directory. The ``database upgrade`` action uses these scripts to update databases that were created by an older version of an extractor so they're compatible with the current extractor (see `Upgrade scripts for a language <#upgrade-scripts-for-a-language>`__ below.)
|
||||
* - ``dependencies``
|
||||
- .. code-block:: yaml
|
||||
|
||||
dependencies:
|
||||
codeql/cpp-all: ^0.0.2
|
||||
|
||||
- Packs that define CodeQL package dependencies on other packs
|
||||
- A map from pack references to the semantic version range that is compatible with this pack. Supported from CLI version v2.6.0 and onwards.
|
||||
* - ``defaultSuiteFile``
|
||||
- ``defaultSuiteFile: cpp-code-scanning.qls``
|
||||
- Packs that export a set of default queries to run
|
||||
- The path to a query suite file containing all of the queries that are run by default when this pack is passed to the ``codeql database analyze`` command. Supported from CLI version v2.6.0 and onwards.
|
||||
|
||||
.. _custom-ql-packs:
|
||||
|
||||
Examples of custom QL packs
|
||||
---------------------------
|
||||
|
||||
When you write custom queries or tests, you should save them in
|
||||
custom QL packs. For simplicity, try to organize each pack logically. For more
|
||||
information, see `QL pack structure <#ql-pack-structure>`__. Save files for queries
|
||||
and tests in separate packs and, where possible, organize custom packs into specific
|
||||
folders for each target language.
|
||||
|
||||
QL packs for custom queries
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
A custom QL pack for queries must include a ``qlpack.yml`` file at
|
||||
the pack root, containing ``name``, ``version``,
|
||||
and ``libraryPathDependencies`` properties. If the pack contains query suites, you can
|
||||
use the ``suites`` property to define their location. Query suites defined
|
||||
here are called "well-known" suites, and can be used on the command line by referring to
|
||||
their name only, rather than their full path.
|
||||
For more information about query suites, see ":doc:`Creating CodeQL query suites <creating-codeql-query-suites>`."
|
||||
|
||||
For example, a ``qlpack.yml`` file for a QL pack featuring custom C++ queries
|
||||
and libraries may contain:
|
||||
|
||||
.. code-block:: yaml
|
||||
|
||||
name: my-custom-queries
|
||||
version: 0.0.0
|
||||
libraryPathDependencies: codeql/cpp-all
|
||||
suites: my-custom-suites
|
||||
|
||||
where ``codeql/cpp-all`` is the name of the QL pack for C/C++ analysis included in
|
||||
the CodeQL repository.
|
||||
|
||||
.. pull-quote::
|
||||
|
||||
Note
|
||||
|
||||
When you create a custom QL pack, it's usually a good idea to add it to the search path in your CodeQL configuration.
|
||||
This will ensure that any libraries the pack contains are available to the CodeQL CLI.
|
||||
For more information, see ":ref:`Specifying command options in a CodeQL configuration file <specifying-command-options-in-a-codeql-configuration-file>`."
|
||||
|
||||
QL packs for custom test files
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
For custom QL packs containing test files, you also need to include an
|
||||
``extractor`` property so that the ``test run`` command knows how to create test
|
||||
databases. You may also wish to specify the ``tests`` property.
|
||||
|
||||
.. include:: ../reusables/test-qlpack.rst
|
||||
|
||||
For more information about running tests, see ":doc:`Testing custom queries
|
||||
<testing-custom-queries>`."
|
||||
|
||||
.. _standard-ql-packs:
|
||||
|
||||
Examples of QL packs in the CodeQL repository
|
||||
---------------------------------------------
|
||||
|
||||
Each of the languages in the CodeQL repository has four main QL packs:
|
||||
|
||||
- Core library pack for the language, with the :ref:`database schema <codeql-database-schema>`
|
||||
used by the language, and CodeQL libraries, and queries at ``ql/<language>/ql/lib``
|
||||
- Core query pack for the language that includes the default queries for the language, along
|
||||
with their query suites at ``ql/<language>/ql/src``
|
||||
- Tests for the core language libraries and queries at ``ql/<language>/ql/test``
|
||||
- Upgrade scripts for the language at ``ql/<language>/upgrades``
|
||||
|
||||
Core library pack
|
||||
~~~~~~~~~~~~~~~~~
|
||||
|
||||
The ``qlpack.yml`` file for a core library pack uses the following properties:
|
||||
``name``, ``version``, ``dbscheme``.
|
||||
The ``dbscheme`` property should only be defined in the core QL
|
||||
pack for a language.
|
||||
|
||||
For example, the ``qlpack.yml`` file for `C/C++ analysis libraries
|
||||
<https://github.com/github/codeql/blob/main/cpp/ql/lib/qlpack.yml>`__
|
||||
contains:
|
||||
|
||||
.. code-block:: yaml
|
||||
|
||||
name: codeql/cpp-all
|
||||
version: 0.0.0
|
||||
dbscheme: semmlecode.cpp.dbscheme
|
||||
upgrades: upgrades
|
||||
|
||||
Core query pack
|
||||
~~~~~~~~~~~~~~~
|
||||
|
||||
The ``qlpack.yml`` file for a core query pack uses the following properties:
|
||||
``name``, ``version``, ``suites``, ``defaultSuiteFile``, ``dependencies`` .
|
||||
|
||||
For example, the ``qlpack.yml`` file for `C/C++ analysis queries
|
||||
<https://github.com/github/codeql/blob/main/cpp/ql/lib/qlpack.yml>`__
|
||||
contains:
|
||||
|
||||
.. code-block:: yaml
|
||||
|
||||
name: codeql/cpp-queries
|
||||
version: 0.0.0
|
||||
suites: codeql-suites
|
||||
defaultSuiteFile: codeql-suites/cpp-code-scanning.qls
|
||||
dependencies:
|
||||
codeql/cpp-all: "*"
|
||||
codeql/suite-helpers: "*"
|
||||
|
||||
Tests for the core QL pack
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
The ``qlpack.yml`` file for the tests for the core QL packs use the following
|
||||
properties: ``name``, ``version``, and ``dependencies``.
|
||||
The ``dependencies`` always specifies the core QL pack.
|
||||
|
||||
For example, the ``qlpack.yml`` file for `C/C++ analysis tests
|
||||
<https://github.com/github/codeql/blob/main/cpp/ql/test/qlpack.yml>`__
|
||||
contains:
|
||||
|
||||
.. code-block:: yaml
|
||||
|
||||
name: codeql/cpp-tests
|
||||
version: 0.0.0
|
||||
dependencies:
|
||||
codeql/cpp-all: "*"
|
||||
codeql/cpp-queries: "*"
|
||||
|
||||
|
||||
.. _upgrade-ql-packs:
|
||||
Reference in New Issue
Block a user