mirror of
https://github.com/github/codeql.git
synced 2026-04-26 09:15:12 +02:00
Merge pull request #6153 from github/docs-content-4456-multiple-dbs
Add information on multi-db options to the CodeQL CLI docs
This commit is contained in:
Felicity Chapman
@@ -56,6 +56,12 @@ You must specify:
|
||||
|
||||
You can also specify:
|
||||
|
||||
- ``--sarif-category``: an identifying category for the results. Used when
|
||||
you want to upload more than one set of results for a commit.
|
||||
For example, when you use ``github upload-results`` to send results for more than one
|
||||
language to the GitHub code scanning API. For more information about this use case,
|
||||
see `Configuring CodeQL CLI in your CI system <https://docs.github.com/en/code-security/secure-coding/using-codeql-code-scanning-with-your-existing-ci-system/configuring-codeql-cli-in-your-ci-system>`__ in the GitHub documentation.
|
||||
|
||||
- .. include:: ../reusables/threads-query-execution.rst
|
||||
|
||||
|
||||
@@ -85,8 +91,8 @@ repositories.
|
||||
Running a single query
|
||||
~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
To run a single query over a JavaScript codebase, you could use the following
|
||||
command from the directory containing your database::
|
||||
To run a single query over a CodeQL database for a JavaScript codebase,
|
||||
you could use the following command from the directory containing your database::
|
||||
|
||||
codeql database analyze <javascript-database> ../ql/javascript/ql/src/Declarations/UnusedVariable.ql --format=csv --output=js-analysis/js-results.csv
|
||||
|
||||
@@ -105,13 +111,23 @@ see ":doc:`Using custom queries with the CodeQL CLI <using-custom-queries-with-t
|
||||
Running GitHub code scanning suites
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
The CodeQL repository also includes query suites, which can be run over your
|
||||
code as part of a broader code review. CodeQL query suites are ``.qls`` files
|
||||
that use directives to select queries to run based on certain metadata
|
||||
properties.
|
||||
To run the GitHub code scanning suite of queries over a CodeQL database for a C/C++ codebase,
|
||||
you could use the following command from the directory containing your database::
|
||||
|
||||
The CodeQL repository includes query suites that are used by the CodeQL action on
|
||||
`GitHub.com <https://github.com>`__. The query suites are located at the following paths in
|
||||
codeql database analyze <cpp-database> cpp-code-scanning.qls --format=sarifv2.1.0 --output=cpp-results.sarif
|
||||
|
||||
The analysis generates a file in the v2.1.0 SARIF format that is supported by all versions of GitHub.
|
||||
This file can be uploaded to GitHub using ``github upload-results`` or the code scanning API.
|
||||
For more information, see `Analyzing a CodeQL database <https://docs.github.com/en/code-security/secure-coding/configuring-codeql-cli-in-your-ci-system#analyzing-a-codeql-database>`__
|
||||
or `Code scanning API <https://docs.github.com/en/rest/reference/code-scanning>`__ in the GitHub documentation.
|
||||
|
||||
CodeQL query suites are ``.qls`` files that use directives to select queries to run
|
||||
based on certain metadata properties. The standard QL packs have metadata that specify
|
||||
the location of the code scanning suites, so the CodeQL CLI knows where to find these
|
||||
suite files automatically, and you don't have to specify the full path on the command line.
|
||||
For more information, see ":ref:`About QL packs <standard-ql-packs>`."
|
||||
|
||||
The standard query suites are stored at the following paths in
|
||||
the CodeQL repository::
|
||||
|
||||
ql/<language>/ql/src/codeql-suites/<language>-code-scanning.qls
|
||||
@@ -120,23 +136,6 @@ and at the following path in the CodeQL for Go repository::
|
||||
|
||||
ql/src/codeql-suites/go-code-scanning.qls
|
||||
|
||||
These locations are specified in the metadata included in the standard QL packs.
|
||||
This means that the CodeQL CLI knows where to find the suite files automatically, and
|
||||
you don't have to specify the full path on the command line when running an
|
||||
analysis. For more information, see ":ref:`About QL packs <standard-ql-packs>`."
|
||||
|
||||
.. pull-quote::
|
||||
|
||||
Important
|
||||
|
||||
If you plan to upload the results to GitHub, you must generate SARIF results.
|
||||
For more information, see `Analyzing a CodeQL database <https://docs.github.com/en/code-security/secure-coding/running-codeql-cli-in-your-ci-system#analyzing-a-codeql-database>`__ in the GitHub documentation.
|
||||
|
||||
For example, to run the code scanning query suite on a C++ codebase and generate
|
||||
results in the v2.1 SARIF format supported by all versions of GitHub, you would run::
|
||||
|
||||
codeql database analyze <cpp-database> cpp-code-scanning.qls --format=sarifv2.1.0 --output=cpp-analysis/cpp-results.sarif
|
||||
|
||||
The repository also includes the query suites used by `LGTM.com <https://lgtm.com>`__.
|
||||
These are stored alongside the code scanning suites with names of the form: ``<language>-lgtm.qls``.
|
||||
|
||||
|
||||
@@ -17,6 +17,12 @@ Before you generate a CodeQL database, you need to:
|
||||
- Check out the version of your codebase you want to analyze. The directory
|
||||
should be ready to build, with all dependencies already installed.
|
||||
|
||||
For information about using the CodeQL CLI in a third-party CI system to create results
|
||||
to display in GitHub as code scanning alerts, see `Configuring CodeQL CLI in your CI system <https://docs.github.com/en/code-security/secure-coding/using-codeql-code-scanning-with-your-existing-ci-system/configuring-codeql-cli-in-your-ci-system>`__
|
||||
in the GitHub documentation. For information about enabling CodeQL code scanning using GitHub Actions,
|
||||
see `Setting up code scanning for a repository <https://docs.github.com/en/code-security/secure-coding/automatically-scanning-your-code-for-vulnerabilities-and-errors/setting-up-code-scanning-for-a-repository>`__
|
||||
in the GitHub documentation.
|
||||
|
||||
Running ``codeql database create``
|
||||
----------------------------------
|
||||
|
||||
@@ -33,21 +39,30 @@ You must specify:
|
||||
be created when you execute the command---you cannot specify an existing
|
||||
directory.
|
||||
- ``--language``: the identifier for the language to create a database for.
|
||||
When used with ``--db-cluster``, the option accepts a comma-separated list,
|
||||
or can be specified more than once.
|
||||
CodeQL supports creating databases for the following languages:
|
||||
|
||||
.. include:: ../reusables/extractors.rst
|
||||
|
||||
Other options may be specified depending on the location of your source file and
|
||||
the language you want to analyze:
|
||||
You can specify additional options depending on the location of your source file,
|
||||
if the code needs to be compiled, and if you want to create CodeQL databases for
|
||||
more than one language:
|
||||
|
||||
- ``--source-root``: the root folder for the primary source files used in
|
||||
database creation. By default, the command assumes that the current
|
||||
directory is the source root---use this option to specify a different location.
|
||||
- ``--command``: for compiled languages only, the build commands that invoke the
|
||||
compiler. Do not specify ``--command`` options for Python and
|
||||
JavaScript. Commands will be run from the current folder, or ``--source-root``
|
||||
- ``--db-cluster``: use for multi-language codebases when you want to create
|
||||
databases for more than one language.
|
||||
- ``--command``: used when you create a database for one or more compiled languages,
|
||||
omit if the only languages requested are Python and JavaScript.
|
||||
This specifies the build commands needed to invoke the compiler.
|
||||
Commands are run from the current folder, or ``--source-root``
|
||||
if specified. If you don't include a ``--command``, CodeQL will attempt to
|
||||
detect the build system automatically, using a built-in autobuilder.
|
||||
- ``--no-run-unnecessary-builds``: used with ``--db-cluster`` to suppress the build
|
||||
command for languages where the CodeQL CLI does not need to monitor the build
|
||||
(for example, Python and JavaScript/TypeScript).
|
||||
|
||||
For full details of all the options you can use when creating databases,
|
||||
see the `database create reference documentation <../manual/database-create>`__.
|
||||
@@ -62,31 +77,13 @@ it failed. For compiled languages, the console will display the output of the
|
||||
build system.
|
||||
|
||||
When the database is successfully created, you'll find a new directory at the
|
||||
path specified in the command. This directory contains a number of
|
||||
path specified in the command. If you used the ``--db-cluster`` option to create
|
||||
more than one database, a subdirectory is created for each language.
|
||||
Each CodeQL database directory contains a number of
|
||||
subdirectories, including the relational data (required for analysis) and a
|
||||
source archive---a copy of the source files made at the time the database was
|
||||
created---which is used for displaying analysis results.
|
||||
|
||||
Obtaining databases from LGTM.com
|
||||
---------------------------------
|
||||
|
||||
`LGTM.com <https://lgtm.com>`__ analyzes thousands of open-source projects using
|
||||
CodeQL. For each project on LGTM.com, you can download an archived CodeQL
|
||||
database corresponding to the most recently analyzed revision of the code. These
|
||||
databases can also be analyzed using the CodeQL CLI.
|
||||
|
||||
.. include:: ../reusables/download-lgtm-database.rst
|
||||
|
||||
Before running an analysis, unzip the databases and try :doc:`upgrading <upgrading-codeql-databases>` the
|
||||
unzipped databases to ensure they are compatible with your local copy of the
|
||||
CodeQL queries and libraries.
|
||||
|
||||
.. pull-quote::
|
||||
|
||||
Note
|
||||
|
||||
.. include:: ../reusables/index-files-note.rst
|
||||
|
||||
Creating databases for non-compiled languages
|
||||
---------------------------------------------
|
||||
|
||||
@@ -99,9 +96,11 @@ are available.
|
||||
|
||||
.. pull-quote:: Important
|
||||
|
||||
When running ``database create`` for JavaScript, TypeScript, and Python, you must not
|
||||
specify a ``--command`` option. If you do, you will override the normal
|
||||
extractor invocation, which will create an empty database.
|
||||
When you run ``database create`` for JavaScript, TypeScript, and Python, you should not
|
||||
specify a ``--command`` option. Otherwise this overrides the normal
|
||||
extractor invocation, which will create an empty database. If you create
|
||||
databases for multiple languages and one of them is a compiled language,
|
||||
use the ``--no-run-unnecessary-builds`` option to skip the command for the languages that don't need to be compiled.
|
||||
|
||||
JavaScript and TypeScript
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
@@ -229,6 +228,27 @@ commands that you can specify for compiled languages.
|
||||
This command runs a custom script that contains all of the commands required
|
||||
to build the project.
|
||||
|
||||
Obtaining databases from LGTM.com
|
||||
---------------------------------
|
||||
|
||||
`LGTM.com <https://lgtm.com>`__ analyzes thousands of open-source projects using
|
||||
CodeQL. For each project on LGTM.com, you can download an archived CodeQL
|
||||
database corresponding to the most recently analyzed revision of the code. These
|
||||
databases can also be analyzed using the CodeQL CLI or used with the CodeQL
|
||||
extension for Visual Studio Code.
|
||||
|
||||
.. include:: ../reusables/download-lgtm-database.rst
|
||||
|
||||
Before running an analysis, unzip the databases and try :doc:`upgrading <upgrading-codeql-databases>` the
|
||||
unzipped databases to ensure they are compatible with your local copy of the
|
||||
CodeQL queries and libraries.
|
||||
|
||||
.. pull-quote::
|
||||
|
||||
Note
|
||||
|
||||
.. include:: ../reusables/index-files-note.rst
|
||||
|
||||
|
||||
Further reading
|
||||
---------------
|
||||
|
||||
@@ -22,6 +22,11 @@ follow the steps below. For macOS version 10.15 ("Catalina"), steps 1 and 4 are
|
||||
slightly different---for further details, see the sections labeled **Information
|
||||
for macOS "Catalina" users**.
|
||||
|
||||
For information about installing the CodeQL CLI in a CI system to create results
|
||||
to display in GitHub as code scanning alerts, see
|
||||
`Installing CodeQL CLI in your CI system <https://docs.github.com/en/code-security/secure-coding/using-codeql-code-scanning-with-your-existing-ci-system/installing-codeql-cli-in-your-ci-system>`__
|
||||
in the GitHub documentation.
|
||||
|
||||
1. Download the CodeQL CLI zip package
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
|
||||
@@ -13,7 +13,7 @@ CodeQL includes queries to find the most relevant and interesting problems for e
|
||||
- **Alert queries**: queries that highlight issues in specific locations in your code.
|
||||
- **Path queries**: queries that describe the flow of information between a source and a sink in your code.
|
||||
|
||||
You can add custom queries to :doc:`QL packs <../codeql-cli/about-ql-packs>` to analyze your projects with "`Code scanning <https://docs.github.com/github/finding-security-vulnerabilities-and-errors-in-your-code>`__", use them to analyze a database with the ":ref:`CodeQL CLI <codeql-cli>`," or you can contribute to the standard CodeQL queries in our `open source repository on GitHub <https://github.com/github/codeql>`__.
|
||||
You can add custom queries to :doc:`QL packs <../codeql-cli/about-ql-packs>` to analyze your projects with "`Code scanning <https://docs.github.com/en/code-security/secure-coding/automatically-scanning-your-code-for-vulnerabilities-and-errors/about-code-scanning>`__", use them to analyze a database with the ":ref:`CodeQL CLI <codeql-cli>`," or you can contribute to the standard CodeQL queries in our `open source repository on GitHub <https://github.com/github/codeql>`__.
|
||||
|
||||
This topic is a basic introduction to query files. You can find more information on writing queries for specific programming languages in the ":ref:`CodeQL language guides <codeql-language-guides>`," and detailed technical information about QL in the ":ref:`QL language reference <ql-language-reference>`."
|
||||
For more information on how to format your code when contributing queries to the GitHub repository, see the `CodeQL style guide <https://github.com/github/codeql/blob/main/docs/ql-style-guide.md>`__.
|
||||
|
||||
Reference in New Issue
Block a user