mirror of
https://github.com/github/codeql.git
synced 2026-05-04 21:25:44 +02:00
Merge branch 'main' into markupsafe-modeling
This commit is contained in:
Rasmus Wriedt Larsen
@@ -56,6 +56,12 @@ You must specify:
|
||||
|
||||
You can also specify:
|
||||
|
||||
- ``--sarif-category``: an identifying category for the results. Used when
|
||||
you want to upload more than one set of results for a commit.
|
||||
For example, when you use ``github upload-results`` to send results for more than one
|
||||
language to the GitHub code scanning API. For more information about this use case,
|
||||
see `Configuring CodeQL CLI in your CI system <https://docs.github.com/en/code-security/secure-coding/using-codeql-code-scanning-with-your-existing-ci-system/configuring-codeql-cli-in-your-ci-system>`__ in the GitHub documentation.
|
||||
|
||||
- .. include:: ../reusables/threads-query-execution.rst
|
||||
|
||||
|
||||
@@ -85,8 +91,8 @@ repositories.
|
||||
Running a single query
|
||||
~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
To run a single query over a JavaScript codebase, you could use the following
|
||||
command from the directory containing your database::
|
||||
To run a single query over a CodeQL database for a JavaScript codebase,
|
||||
you could use the following command from the directory containing your database::
|
||||
|
||||
codeql database analyze <javascript-database> ../ql/javascript/ql/src/Declarations/UnusedVariable.ql --format=csv --output=js-analysis/js-results.csv
|
||||
|
||||
@@ -105,13 +111,23 @@ see ":doc:`Using custom queries with the CodeQL CLI <using-custom-queries-with-t
|
||||
Running GitHub code scanning suites
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
The CodeQL repository also includes query suites, which can be run over your
|
||||
code as part of a broader code review. CodeQL query suites are ``.qls`` files
|
||||
that use directives to select queries to run based on certain metadata
|
||||
properties.
|
||||
To run the GitHub code scanning suite of queries over a CodeQL database for a C/C++ codebase,
|
||||
you could use the following command from the directory containing your database::
|
||||
|
||||
The CodeQL repository includes query suites that are used by the CodeQL action on
|
||||
`GitHub.com <https://github.com>`__. The query suites are located at the following paths in
|
||||
codeql database analyze <cpp-database> cpp-code-scanning.qls --format=sarifv2.1.0 --output=cpp-results.sarif
|
||||
|
||||
The analysis generates a file in the v2.1.0 SARIF format that is supported by all versions of GitHub.
|
||||
This file can be uploaded to GitHub using ``github upload-results`` or the code scanning API.
|
||||
For more information, see `Analyzing a CodeQL database <https://docs.github.com/en/code-security/secure-coding/configuring-codeql-cli-in-your-ci-system#analyzing-a-codeql-database>`__
|
||||
or `Code scanning API <https://docs.github.com/en/rest/reference/code-scanning>`__ in the GitHub documentation.
|
||||
|
||||
CodeQL query suites are ``.qls`` files that use directives to select queries to run
|
||||
based on certain metadata properties. The standard QL packs have metadata that specify
|
||||
the location of the code scanning suites, so the CodeQL CLI knows where to find these
|
||||
suite files automatically, and you don't have to specify the full path on the command line.
|
||||
For more information, see ":ref:`About QL packs <standard-ql-packs>`."
|
||||
|
||||
The standard query suites are stored at the following paths in
|
||||
the CodeQL repository::
|
||||
|
||||
ql/<language>/ql/src/codeql-suites/<language>-code-scanning.qls
|
||||
@@ -120,23 +136,6 @@ and at the following path in the CodeQL for Go repository::
|
||||
|
||||
ql/src/codeql-suites/go-code-scanning.qls
|
||||
|
||||
These locations are specified in the metadata included in the standard QL packs.
|
||||
This means that the CodeQL CLI knows where to find the suite files automatically, and
|
||||
you don't have to specify the full path on the command line when running an
|
||||
analysis. For more information, see ":ref:`About QL packs <standard-ql-packs>`."
|
||||
|
||||
.. pull-quote::
|
||||
|
||||
Important
|
||||
|
||||
If you plan to upload the results to GitHub, you must generate SARIF results.
|
||||
For more information, see `Analyzing a CodeQL database <https://docs.github.com/en/code-security/secure-coding/running-codeql-cli-in-your-ci-system#analyzing-a-codeql-database>`__ in the GitHub documentation.
|
||||
|
||||
For example, to run the code scanning query suite on a C++ codebase and generate
|
||||
results in the v2.1 SARIF format supported by all versions of GitHub, you would run::
|
||||
|
||||
codeql database analyze <cpp-database> cpp-code-scanning.qls --format=sarifv2.1.0 --output=cpp-analysis/cpp-results.sarif
|
||||
|
||||
The repository also includes the query suites used by `LGTM.com <https://lgtm.com>`__.
|
||||
These are stored alongside the code scanning suites with names of the form: ``<language>-lgtm.qls``.
|
||||
|
||||
|
||||
@@ -17,6 +17,12 @@ Before you generate a CodeQL database, you need to:
|
||||
- Check out the version of your codebase you want to analyze. The directory
|
||||
should be ready to build, with all dependencies already installed.
|
||||
|
||||
For information about using the CodeQL CLI in a third-party CI system to create results
|
||||
to display in GitHub as code scanning alerts, see `Configuring CodeQL CLI in your CI system <https://docs.github.com/en/code-security/secure-coding/using-codeql-code-scanning-with-your-existing-ci-system/configuring-codeql-cli-in-your-ci-system>`__
|
||||
in the GitHub documentation. For information about enabling CodeQL code scanning using GitHub Actions,
|
||||
see `Setting up code scanning for a repository <https://docs.github.com/en/code-security/secure-coding/automatically-scanning-your-code-for-vulnerabilities-and-errors/setting-up-code-scanning-for-a-repository>`__
|
||||
in the GitHub documentation.
|
||||
|
||||
Running ``codeql database create``
|
||||
----------------------------------
|
||||
|
||||
@@ -33,21 +39,30 @@ You must specify:
|
||||
be created when you execute the command---you cannot specify an existing
|
||||
directory.
|
||||
- ``--language``: the identifier for the language to create a database for.
|
||||
When used with ``--db-cluster``, the option accepts a comma-separated list,
|
||||
or can be specified more than once.
|
||||
CodeQL supports creating databases for the following languages:
|
||||
|
||||
.. include:: ../reusables/extractors.rst
|
||||
|
||||
Other options may be specified depending on the location of your source file and
|
||||
the language you want to analyze:
|
||||
You can specify additional options depending on the location of your source file,
|
||||
if the code needs to be compiled, and if you want to create CodeQL databases for
|
||||
more than one language:
|
||||
|
||||
- ``--source-root``: the root folder for the primary source files used in
|
||||
database creation. By default, the command assumes that the current
|
||||
directory is the source root---use this option to specify a different location.
|
||||
- ``--command``: for compiled languages only, the build commands that invoke the
|
||||
compiler. Do not specify ``--command`` options for Python and
|
||||
JavaScript. Commands will be run from the current folder, or ``--source-root``
|
||||
- ``--db-cluster``: use for multi-language codebases when you want to create
|
||||
databases for more than one language.
|
||||
- ``--command``: used when you create a database for one or more compiled languages,
|
||||
omit if the only languages requested are Python and JavaScript.
|
||||
This specifies the build commands needed to invoke the compiler.
|
||||
Commands are run from the current folder, or ``--source-root``
|
||||
if specified. If you don't include a ``--command``, CodeQL will attempt to
|
||||
detect the build system automatically, using a built-in autobuilder.
|
||||
- ``--no-run-unnecessary-builds``: used with ``--db-cluster`` to suppress the build
|
||||
command for languages where the CodeQL CLI does not need to monitor the build
|
||||
(for example, Python and JavaScript/TypeScript).
|
||||
|
||||
For full details of all the options you can use when creating databases,
|
||||
see the `database create reference documentation <../manual/database-create>`__.
|
||||
@@ -62,31 +77,13 @@ it failed. For compiled languages, the console will display the output of the
|
||||
build system.
|
||||
|
||||
When the database is successfully created, you'll find a new directory at the
|
||||
path specified in the command. This directory contains a number of
|
||||
path specified in the command. If you used the ``--db-cluster`` option to create
|
||||
more than one database, a subdirectory is created for each language.
|
||||
Each CodeQL database directory contains a number of
|
||||
subdirectories, including the relational data (required for analysis) and a
|
||||
source archive---a copy of the source files made at the time the database was
|
||||
created---which is used for displaying analysis results.
|
||||
|
||||
Obtaining databases from LGTM.com
|
||||
---------------------------------
|
||||
|
||||
`LGTM.com <https://lgtm.com>`__ analyzes thousands of open-source projects using
|
||||
CodeQL. For each project on LGTM.com, you can download an archived CodeQL
|
||||
database corresponding to the most recently analyzed revision of the code. These
|
||||
databases can also be analyzed using the CodeQL CLI.
|
||||
|
||||
.. include:: ../reusables/download-lgtm-database.rst
|
||||
|
||||
Before running an analysis, unzip the databases and try :doc:`upgrading <upgrading-codeql-databases>` the
|
||||
unzipped databases to ensure they are compatible with your local copy of the
|
||||
CodeQL queries and libraries.
|
||||
|
||||
.. pull-quote::
|
||||
|
||||
Note
|
||||
|
||||
.. include:: ../reusables/index-files-note.rst
|
||||
|
||||
Creating databases for non-compiled languages
|
||||
---------------------------------------------
|
||||
|
||||
@@ -99,9 +96,11 @@ are available.
|
||||
|
||||
.. pull-quote:: Important
|
||||
|
||||
When running ``database create`` for JavaScript, TypeScript, and Python, you must not
|
||||
specify a ``--command`` option. If you do, you will override the normal
|
||||
extractor invocation, which will create an empty database.
|
||||
When you run ``database create`` for JavaScript, TypeScript, and Python, you should not
|
||||
specify a ``--command`` option. Otherwise this overrides the normal
|
||||
extractor invocation, which will create an empty database. If you create
|
||||
databases for multiple languages and one of them is a compiled language,
|
||||
use the ``--no-run-unnecessary-builds`` option to skip the command for the languages that don't need to be compiled.
|
||||
|
||||
JavaScript and TypeScript
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
@@ -229,6 +228,27 @@ commands that you can specify for compiled languages.
|
||||
This command runs a custom script that contains all of the commands required
|
||||
to build the project.
|
||||
|
||||
Obtaining databases from LGTM.com
|
||||
---------------------------------
|
||||
|
||||
`LGTM.com <https://lgtm.com>`__ analyzes thousands of open-source projects using
|
||||
CodeQL. For each project on LGTM.com, you can download an archived CodeQL
|
||||
database corresponding to the most recently analyzed revision of the code. These
|
||||
databases can also be analyzed using the CodeQL CLI or used with the CodeQL
|
||||
extension for Visual Studio Code.
|
||||
|
||||
.. include:: ../reusables/download-lgtm-database.rst
|
||||
|
||||
Before running an analysis, unzip the databases and try :doc:`upgrading <upgrading-codeql-databases>` the
|
||||
unzipped databases to ensure they are compatible with your local copy of the
|
||||
CodeQL queries and libraries.
|
||||
|
||||
.. pull-quote::
|
||||
|
||||
Note
|
||||
|
||||
.. include:: ../reusables/index-files-note.rst
|
||||
|
||||
|
||||
Further reading
|
||||
---------------
|
||||
|
||||
@@ -22,6 +22,11 @@ follow the steps below. For macOS version 10.15 ("Catalina"), steps 1 and 4 are
|
||||
slightly different---for further details, see the sections labeled **Information
|
||||
for macOS "Catalina" users**.
|
||||
|
||||
For information about installing the CodeQL CLI in a CI system to create results
|
||||
to display in GitHub as code scanning alerts, see
|
||||
`Installing CodeQL CLI in your CI system <https://docs.github.com/en/code-security/secure-coding/using-codeql-code-scanning-with-your-existing-ci-system/installing-codeql-cli-in-your-ci-system>`__
|
||||
in the GitHub documentation.
|
||||
|
||||
1. Download the CodeQL CLI zip package
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
|
||||
@@ -30,7 +30,7 @@ Choosing a version of the CodeQL CLI
|
||||
|
||||
The CodeQL extension uses the CodeQL CLI to run commands. If you already have the CLI installed and added to your ``PATH``, the extension uses that version. This might be the case if you create your own CodeQL databases instead of downloading them from LGTM.com. Otherwise, the extension automatically manages access to the executable of the CLI for you. For more information about creating databases, see ":ref:`Creating CodeQL databases <creating-codeql-databases>`" in the CLI help.
|
||||
|
||||
To override the default behavior and use a different CLI, you can specify the CodeQL CLI **Executable Path**. Note that this is only available as a user setting, not as a workspace setting.
|
||||
To override the default behavior and use a different CLI, you can specify the CodeQL CLI **Executable Path**.
|
||||
|
||||
Changing the labels of query history items
|
||||
--------------------------------------------
|
||||
|
||||
@@ -1774,6 +1774,8 @@ The following built-in predicates are members of type ``int``:
|
||||
+-------------------------+-------------+----------------+----------------------------------------------------------------------------------------------------------------+
|
||||
| ``toString`` | string | | The result is the decimal representation of the number as a string. |
|
||||
+-------------------------+-------------+----------------+----------------------------------------------------------------------------------------------------------------+
|
||||
| ``toUnicode`` | string | | The result is the unicode character for the receiver seen as a unicode code point. |
|
||||
+-------------------------+-------------+----------------+----------------------------------------------------------------------------------------------------------------+
|
||||
|
||||
The leftmost bit after ``bitShiftRightSigned`` depends on sign extension, whereas after ``bitShiftRight`` it is zero.
|
||||
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
C and C++ built-in support
|
||||
================================
|
||||
|
||||
.. csv-table::
|
||||
.. csv-table::
|
||||
:header-rows: 1
|
||||
:class: fullWidthTable
|
||||
:widths: auto
|
||||
@@ -14,7 +14,7 @@ C and C++ built-in support
|
||||
C# built-in support
|
||||
================================
|
||||
|
||||
.. csv-table::
|
||||
.. csv-table::
|
||||
:header-rows: 1
|
||||
:class: fullWidthTable
|
||||
:widths: auto
|
||||
@@ -84,7 +84,7 @@ Go built-in support
|
||||
Java built-in support
|
||||
==================================
|
||||
|
||||
.. csv-table::
|
||||
.. csv-table::
|
||||
:header-rows: 1
|
||||
:class: fullWidthTable
|
||||
:widths: auto
|
||||
@@ -109,7 +109,7 @@ Java built-in support
|
||||
JavaScript and TypeScript built-in support
|
||||
=======================================================
|
||||
|
||||
.. csv-table::
|
||||
.. csv-table::
|
||||
:header-rows: 1
|
||||
:class: fullWidthTable
|
||||
:widths: auto
|
||||
@@ -156,6 +156,7 @@ Python built-in support
|
||||
Django, Web framework
|
||||
Flask, Web framework
|
||||
Tornado, Web framework
|
||||
Twisted, Web framework
|
||||
PyYAML, Serialization
|
||||
dill, Serialization
|
||||
simplejson, Serialization
|
||||
@@ -163,13 +164,18 @@ Python built-in support
|
||||
fabric, Utility library
|
||||
idna, Utility library
|
||||
invoke, Utility library
|
||||
jmespath, Utility library
|
||||
multidict, Utility library
|
||||
yarl, Utility library
|
||||
aioch, Database
|
||||
clickhouse-driver, Database
|
||||
mysql-connector-python, Database
|
||||
MySQLdb, Database
|
||||
mysql-connector, Database
|
||||
MySQL-python, Database
|
||||
psycopg2, Database
|
||||
sqlite3, Database
|
||||
cryptography, Cryptography library
|
||||
pycryptodome, Cryptography library
|
||||
pycryptodomex, Cryptography library
|
||||
rsa, Cryptography library
|
||||
MarkupSafe, Escaping Library
|
||||
|
||||
@@ -4,31 +4,32 @@
|
||||
:stub-columns: 1
|
||||
|
||||
Language,Variants,Compilers,Extensions
|
||||
C/C++,"C89, C99, C11, C18, C++98, C++03, C++11, C++14, C++17","Clang (and clang-cl [1]_) extensions (up to Clang 9.0),
|
||||
C/C++,"C89, C99, C11, C18, C++98, C++03, C++11, C++14, C++17, C++20 [1]_","Clang (and clang-cl [2]_) extensions (up to Clang 12.0),
|
||||
|
||||
GNU extensions (up to GCC 9.2),
|
||||
GNU extensions (up to GCC 11.1),
|
||||
|
||||
Microsoft extensions (up to VS 2019),
|
||||
|
||||
Arm Compiler 5 [2]_","``.cpp``, ``.c++``, ``.cxx``, ``.hpp``, ``.hh``, ``.h++``, ``.hxx``, ``.c``, ``.cc``, ``.h``"
|
||||
Arm Compiler 5 [3]_","``.cpp``, ``.c++``, ``.cxx``, ``.hpp``, ``.hh``, ``.h++``, ``.hxx``, ``.c``, ``.cc``, ``.h``"
|
||||
C#,C# up to 9.0,"Microsoft Visual Studio up to 2019 with .NET up to 4.8,
|
||||
|
||||
.NET Core up to 3.1
|
||||
|
||||
.NET 5","``.sln``, ``.csproj``, ``.cs``, ``.cshtml``, ``.xaml``"
|
||||
Go (aka Golang), "Go up to 1.16", "Go 1.11 or more recent", ``.go``
|
||||
Java,"Java 7 to 16 [3]_","javac (OpenJDK and Oracle JDK),
|
||||
Java,"Java 7 to 16 [4]_","javac (OpenJDK and Oracle JDK),
|
||||
|
||||
Eclipse compiler for Java (ECJ) [4]_",``.java``
|
||||
JavaScript,ECMAScript 2021 or lower,Not applicable,"``.js``, ``.jsx``, ``.mjs``, ``.es``, ``.es6``, ``.htm``, ``.html``, ``.xhm``, ``.xhtml``, ``.vue``, ``.json``, ``.yaml``, ``.yml``, ``.raml``, ``.xml`` [5]_"
|
||||
Eclipse compiler for Java (ECJ) [5]_",``.java``
|
||||
JavaScript,ECMAScript 2021 or lower,Not applicable,"``.js``, ``.jsx``, ``.mjs``, ``.es``, ``.es6``, ``.htm``, ``.html``, ``.xhm``, ``.xhtml``, ``.vue``, ``.json``, ``.yaml``, ``.yml``, ``.raml``, ``.xml`` [6]_"
|
||||
Python,"2.7, 3.5, 3.6, 3.7, 3.8, 3.9",Not applicable,``.py``
|
||||
TypeScript [6]_,"2.6-4.2",Standard TypeScript compiler,"``.ts``, ``.tsx``"
|
||||
TypeScript [7]_,"2.6-4.2",Standard TypeScript compiler,"``.ts``, ``.tsx``"
|
||||
|
||||
.. container:: footnote-group
|
||||
|
||||
.. [1] Support for the clang-cl compiler is preliminary.
|
||||
.. [2] Support for the Arm Compiler (armcc) is preliminary.
|
||||
.. [3] Builds that execute on Java 7 to 16 can be analyzed. The analysis understands Java 15 standard language features.
|
||||
.. [4] ECJ is supported when the build invokes it via the Maven Compiler plugin or the Takari Lifecycle plugin.
|
||||
.. [5] JSX and Flow code, YAML, JSON, HTML, and XML files may also be analyzed with JavaScript files.
|
||||
.. [6] TypeScript analysis is performed by running the JavaScript extractor with TypeScript enabled. This is the default for LGTM.
|
||||
.. [1] C++20 support is currently in beta. Supported for GCC on Linux only. Modules are *not* supported.
|
||||
.. [2] Support for the clang-cl compiler is preliminary.
|
||||
.. [3] Support for the Arm Compiler (armcc) is preliminary.
|
||||
.. [4] Builds that execute on Java 7 to 16 can be analyzed. The analysis understands Java 15 standard language features.
|
||||
.. [5] ECJ is supported when the build invokes it via the Maven Compiler plugin or the Takari Lifecycle plugin.
|
||||
.. [6] JSX and Flow code, YAML, JSON, HTML, and XML files may also be analyzed with JavaScript files.
|
||||
.. [7] TypeScript analysis is performed by running the JavaScript extractor with TypeScript enabled. This is the default for LGTM.
|
||||
|
||||
@@ -13,7 +13,7 @@ CodeQL includes queries to find the most relevant and interesting problems for e
|
||||
- **Alert queries**: queries that highlight issues in specific locations in your code.
|
||||
- **Path queries**: queries that describe the flow of information between a source and a sink in your code.
|
||||
|
||||
You can add custom queries to :doc:`QL packs <../codeql-cli/about-ql-packs>` to analyze your projects with "`Code scanning <https://docs.github.com/github/finding-security-vulnerabilities-and-errors-in-your-code>`__", use them to analyze a database with the ":ref:`CodeQL CLI <codeql-cli>`," or you can contribute to the standard CodeQL queries in our `open source repository on GitHub <https://github.com/github/codeql>`__.
|
||||
You can add custom queries to :doc:`QL packs <../codeql-cli/about-ql-packs>` to analyze your projects with "`Code scanning <https://docs.github.com/en/code-security/secure-coding/automatically-scanning-your-code-for-vulnerabilities-and-errors/about-code-scanning>`__", use them to analyze a database with the ":ref:`CodeQL CLI <codeql-cli>`," or you can contribute to the standard CodeQL queries in our `open source repository on GitHub <https://github.com/github/codeql>`__.
|
||||
|
||||
This topic is a basic introduction to query files. You can find more information on writing queries for specific programming languages in the ":ref:`CodeQL language guides <codeql-language-guides>`," and detailed technical information about QL in the ":ref:`QL language reference <ql-language-reference>`."
|
||||
For more information on how to format your code when contributing queries to the GitHub repository, see the `CodeQL style guide <https://github.com/github/codeql/blob/main/docs/ql-style-guide.md>`__.
|
||||
|
||||
Reference in New Issue
Block a user