Add docs for codeql workspaces

See https://github.com/github/codeql-core/issues/2687
This commit is contained in:
Andrew Eisenberg
2022-08-24 14:11:24 -07:00
parent 7f8fcef62c
commit 53d8bf27ff
4 changed files with 82 additions and 3 deletions

View File

@@ -199,10 +199,8 @@ The ``codeql-pack.lock.yml`` file will contain something like the following:
my-user/transitive-dependency:
version: 1.2.4
..
TODO: Add a link to the CodeQL CLI documentation for query resolution, specifically in regards to resolving from source
The ``codeql/cpp-all`` dependency is locked to version 0.1.4. The ``my-user/my-lib`` dependency is locked to version 0.2.4. The ``my-user/transitive-dependency``, which is a transitive dependency and is not specified in the ``qlpack.yml`` file, is locked to version 1.2.4. The ``other-dependency/from-source`` is absent from the lock file since it is resolved from source. This dependency must be available in the same CodeQL workspace as the pack.
The ``codeql/cpp-all`` dependency is locked to version 0.1.4. The ``my-user/my-lib`` dependency is locked to version 0.2.4. The ``my-user/transitive-dependency``, which is a transitive dependency and is not specified in the ``qlpack.yml`` file, is locked to version 1.2.4. The ``other-dependency/from-source`` is absent from the lock file since it is resolved from source. This dependency must be available in the same CodeQL workspace as the pack. For more information about CodeQL workspaces and resolving dependencies from source see ":doc:`About CodeQL Workspaces <about-codeql-workspaces>`."
.. _custom-codeql-packs:

View File

@@ -0,0 +1,77 @@
.. _about-codeql-workspaces:
About CodeQL Workspaces
=======================
.. include:: ../reusables/beta-note-package-management.rst
CodeQL workspaces are used to group multiple CodeQL packs together. CodeQL packs in the same workspace are automatically available as source dependencies for each other when running any CodeQL command that resolves queries. This makes it easier to develope and maintain multiple, related CodeQL packs. A typical use case for a CodeQL workspace is for developing one or more CodeQL library packs and one or more query packs that depends on it in them in the same location.
The main benefit of a CodeQl workspace is that it is easier to develop and maintain multiple CodeQL packs. When a CodeQL workspace is used, all CodeQL packs in the workspace are available as source dependencies for each other when running any CodeQL command that resolves queries. This makes it easier to develope and maintain multiple, related CodeQL packs.
In most cases, the CodeQL workspace and all CodeQL packs contained in it should be stored in the same git repository so the development environment is more easily sharable.
The ``codeql-workspae.yml`` file
--------------------------------
A CodeQL workspace is defined by a ``codeql-workspace.yml`` yaml file. This file contains a ``provide`` block, and optionally an ``ignore`` block. The ``provide`` block contains a list of glob patterns that define the CodeQL packs that are available in the workspace. The ``ignore`` block contains a list of glob patterns that define CodeQL packs that are not available in the workspace. Each entry in the ``provide`` or ``ignore`` section must map to a path to a ``qlpack.yml`` file. All glob patterns are relative to the directory containing the workspace file. See `@actions/glob <https://github.com/actions/toolkit/tree/main/packages/glob#patterns>` for a list of patterns accepted in this file.
For example, the following ``codeql-workspace.yml`` file defines a workspace that contains all CodeQl packs recursively found in the ``codeql-packs`` directory, except for the packs in the ``experimental`` directory:
.. code-block:: yaml
provide:
- "*/codeql-packs/**/qlpack.yml"
ignore:
- "*/codeql-packs/**/experimental/**/qlpack.yml"
To verify that you have the correct ``codeql-workspace.yml`` file, run ``codeql pack ls`` command in the same directory as your workspace. The result of the command is a list of all CodeQL packs in the workspace.
CodeQL workspaces and query resolution
--------------------------------------
All CodeQL packs in a workspace are available as source dependencies for each other when running any CodeQL command that resolves queries or packs. For example, when ``codeql pack install`` is run in a pack directory in a workspace, any dependency found in the workspace will not be downloaded to the package cache, nor will it be added to the resulting ``codeql-pack.lock.yml`` file. See `:ref:Adding and Installing Dependencies <adding-and-installing-dependencies>` for more information.
Similarly, publishing a CodeQL query pack to the GitHub container registry using ``codeql pack publish`` will always use dependencies found in the workspace instead of using dependencies found in the local package cache.
This ensures that any local change to a query library in a dependency in the same workspace will be automatically reflected in the published query pack.
Example
~~~~~~~
Consider the following ``codeql-workspace.yml`` file:
.. code-block:: yaml
provide:
- "**/qlpack.yml"
And the following CodeQL library pack ``qlpack.yml`` file in the workspace:
.. code-block:: yaml
name: my-company/my-library
library: true
version: 1.0.0
And the following CodeQL query pack ``qlpack.yml`` file in the workspace:
.. code-block:: yaml
name: my-company/my-queries
version: 1.0.0
dependencies:
my-company/my-library: "*"
codeql/cpp-all: ~0.2.0
Notice that ``"*"`` is specified as the version constraint for the library pack. Because the library pack is a source dependency, the version constraint is not needed since the library pack's content is always resolved from inside of the workspace. Any version constraint will be ignored in this case, but it is recommended to use ``"*"`` for source dependencies to avoid confusion.
When ``codeql pack install`` is executed from the query pack directory, an appropriate version of ``codeql/cpp-all`` will be downloaded to the local package cache. Also, a ``codeql-pack.lock.yml`` file will be created that contains the resolved version of ``codeql/cpp-all``, but no entry for ``my-company/my-library`` since it is resolved from source. The ``codeql-pack.lock.yml`` file will look something like this:
.. code-block:: yaml
dependencies:
codeql/cpp-all:
version: 0.2.2
When ``codeql pack publish`` is executed from the query pack directory, the ``codeql/cpp-all`` dependency from the package cache and the ``my-company/my-library`` from the workspace will be bundled with ``my-company/my-queries`` and published to the GitHub container registry.

View File

@@ -14,6 +14,7 @@ Learn more about the files you can use when running CodeQL processes and the res
sarif-output
exit-codes
extractor-options
about-codeql-workspaces
- :doc:`About CodeQL packs <about-codeql-packs>`: CodeQL packs are created with the CodeQL CLI and are used to create, depend on, publish, and run CodeQL queries, libraries, and query suites.
- :doc:`Query reference files <query-reference-files>`: A query reference file is text file that defines the location of one query to test.
@@ -21,3 +22,4 @@ Learn more about the files you can use when running CodeQL processes and the res
- :doc:`Exit codes <exit-codes>`: The CodeQL CLI reports the status of each command it runs as an exit code.
This exit code provides information for subsequent commands or for other tools that rely on the CodeQL CLI.
- :doc:`Extractor options <extractor-options>`: You can customize the behavior of extractors by setting options through the CodeQL CLI.
- :doc:`About CodeQL workspaces <about-codeql-workspaces>`: CodeQL workspaces are used to group multiple CodeQL packs together.

View File

@@ -45,6 +45,8 @@ If you already have a ``qlpack.yml`` file, you can edit it manually to convert i
For more information about the properties, see ":ref:`About CodeQL packs <about-codeql-packs>`."
.. _adding-and-installing-dependencies:
Adding and installing dependencies to a CodeQL pack
---------------------------------------------------
You can add dependencies on CodeQL packs using the command ``codeql pack add``. You must specify the scope, name, and (optionally) a compatible version range.