Merge branch 'main' of github.com:github/codeql into 'main'

Conflicts:
	docs/codeql/query-help/codeql-cwe-coverage.rst
This commit is contained in:
Arthur Baars
2021-10-22 21:51:25 +02:00
51 changed files with 2761 additions and 150 deletions

View File

@@ -7,7 +7,7 @@ QL packs are used to organize the files used in CodeQL analysis. They
contain queries, library files, query suites, and important metadata.
The `CodeQL repository <https://github.com/github/codeql>`__ contains QL packs for
C/C++, C#, Java, JavaScript, and Python. The `CodeQL for Go
C/C++, C#, Java, JavaScript, Python, and Ruby. The `CodeQL for Go
<https://github.com/github/codeql-go/>`__ repository contains a QL pack for Go
analysis. You can also make custom QL packs to contain your own queries and
libraries.

View File

@@ -88,15 +88,15 @@ Creating databases for non-compiled languages
---------------------------------------------
The CodeQL CLI includes extractors to create databases for non-compiled
languages---specifically, JavaScript (and TypeScript) and Python. These
extractors are automatically invoked when you specify JavaScript or Python as
languages---specifically, JavaScript (and TypeScript), Python, and Ruby. These
extractors are automatically invoked when you specify JavaScript, Python, or Ruby as
the ``--language`` option when executing ``database create``. When creating
databases for these languages you must ensure that all additional dependencies
are available.
.. pull-quote:: Important
When you run ``database create`` for JavaScript, TypeScript, and Python, you should not
When you run ``database create`` for JavaScript, TypeScript, Python, and Ruby, you should not
specify a ``--command`` option. Otherwise this overrides the normal
extractor invocation, which will create an empty database. If you create
databases for multiple languages and one of them is a compiled language,
@@ -116,6 +116,8 @@ Here, we have specified a ``--source-root`` path, which is the location where
database creation is executed, but is not necessarily the checkout root of the
codebase.
By default, files in ``node_modules`` and ``bower_components`` directories are not extracted.
Python
~~~~~~
@@ -127,14 +129,25 @@ When creating databases for Python you must ensure:
packages that the codebase depends on.
- You have installed the `virtualenv <https://pypi.org/project/virtualenv/>`__ pip module.
In the command line you must specify ``--language=python``. For example
In the command line you must specify ``--language=python``. For example::
::
codeql database create --language=python <output-folder>/python-database
executes the ``database create`` subcommand from the code's checkout root,
This executes the ``database create`` subcommand from the code's checkout root,
generating a new Python database at ``<output-folder>/python-database``.
Ruby
~~~~
Creating databases for Ruby requires no additional dependencies.
In the command line you must specify ``--language=ruby``. For example::
codeql database create --language=ruby --source-root <folder-to-extract> <output-folder>/ruby-database
Here, we have specified a ``--source-root`` path, which is the location where
database creation is executed, but is not necessarily the checkout root of the
codebase.
Creating databases for compiled languages
-----------------------------------------

View File

@@ -100,7 +100,7 @@ further options on the command line.
The `CodeQL repository <https://github.com/github/codeql>`__ contains
the queries and libraries required for CodeQL analysis of C/C++, C#, Java,
JavaScript/TypeScript, and Python.
JavaScript/TypeScript, Python, and Ruby.
Clone a copy of this repository into ``codeql-home``.
By default, the root of the cloned repository will be called ``codeql``.

View File

@@ -78,7 +78,7 @@ Using the starter workspace
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The starter workspace is a Git repository. It contains:
* The `repository of CodeQL libraries and queries <https://github.com/github/codeql>`__ for C/C++, C#, Java, JavaScript, and Python. This is included as a submodule, so it can be updated without affecting your custom queries.
* The `repository of CodeQL libraries and queries <https://github.com/github/codeql>`__ for C/C++, C#, Java, JavaScript, Python, and Ruby. This is included as a submodule, so it can be updated without affecting your custom queries.
* The `repository of CodeQL libraries and queries <https://github.com/github/codeql-go>`__ for Go. This is also included as a submodule.
* A series of folders named ``codeql-custom-queries-<language>``. These are ready for you to start developing your own custom queries for each language, using the standard libraries. There are some example queries to get you started.

View File

@@ -14,3 +14,5 @@ Experiment and learn how to write effective and efficient queries for CodeQL dat
- :doc:`Basic query for Ruby code <basic-query-for-ruby-code>`: Learn to write and run a simple CodeQL query using LGTM.
- :doc:`CodeQL library for Ruby <codeql-library-for-ruby>`: When you're analyzing a Ruby program, you can make use of the large collection of classes in the CodeQL library for Ruby.
.. include:: ../reusables/ruby-beta-note.rst

View File

@@ -1333,7 +1333,7 @@ The values of a set literal expression are all the values of all the contained e
Set literals are supported from release 2.1.0 of the CodeQL CLI, and release 1.24 of LGTM Enterprise.
Since release 2.6.3 of the CodeQL CLI, and release 1.28 of LGTM Enterprise, a trailing comma is allowed in a set literal.
Since release 2.7.0 of the CodeQL CLI, and release 1.28 of LGTM Enterprise, a trailing comma is allowed in a set literal.
Disambiguation of expressions
-----------------------------

View File

@@ -34,3 +34,5 @@ Note that the CWE coverage includes both "`supported queries <https://github.com
javascript-cwe
python-cwe
ruby-cwe
.. include:: ../reusables/ruby-beta-note.rst

View File

@@ -11,7 +11,6 @@ View the query help for the queries included in the ``code-scanning``, ``securit
- :doc:`CodeQL query help for Python <python>`
- :doc:`CodeQL query help for Ruby <ruby>`
.. pull-quote:: Information
Each query help article includes:
@@ -24,6 +23,8 @@ View the query help for the queries included in the ``code-scanning``, ``securit
For a full list of the CWEs covered by these queries, see ":doc:`CodeQL CWE coverage <codeql-cwe-coverage>`."
.. include:: ../reusables/ruby-beta-note.rst
.. toctree::
:hidden:
:titlesonly:

View File

@@ -15,4 +15,6 @@
* - JavaScript/TypeScript
- ``javascript``
* - Python
- ``python``
- ``python``
* - Ruby
- ``ruby``

View File

@@ -0,0 +1,4 @@
.. pull-quote:: Note
CodeQL analysis for Ruby is currently in beta. During the beta, analysis of Ruby code,
and the accompanying documentation, will not be as comprehensive as for other languages.

View File

@@ -22,7 +22,8 @@
Eclipse compiler for Java (ECJ) [5]_",``.java``
JavaScript,ECMAScript 2021 or lower,Not applicable,"``.js``, ``.jsx``, ``.mjs``, ``.es``, ``.es6``, ``.htm``, ``.html``, ``.xhm``, ``.xhtml``, ``.vue``, ``.json``, ``.yaml``, ``.yml``, ``.raml``, ``.xml`` [6]_"
Python,"2.7, 3.5, 3.6, 3.7, 3.8, 3.9",Not applicable,``.py``
TypeScript [7]_,"2.6-4.4",Standard TypeScript compiler,"``.ts``, ``.tsx``"
Ruby [7]_,"up to 3.02",Not applicable,"``.rb``, ``.erb``, ``.gemspec``, ``Gemfile``"
TypeScript [8]_,"2.6-4.4",Standard TypeScript compiler,"``.ts``, ``.tsx``"
.. container:: footnote-group
@@ -32,4 +33,5 @@
.. [4] Builds that execute on Java 7 to 16 can be analyzed. The analysis understands Java 16 standard language features.
.. [5] ECJ is supported when the build invokes it via the Maven Compiler plugin or the Takari Lifecycle plugin.
.. [6] JSX and Flow code, YAML, JSON, HTML, and XML files may also be analyzed with JavaScript files.
.. [7] TypeScript analysis is performed by running the JavaScript extractor with TypeScript enabled. This is the default for LGTM.
.. [7] Requires glibc 2.17.
.. [8] TypeScript analysis is performed by running the JavaScript extractor with TypeScript enabled. This is the default for LGTM.

View File

@@ -116,7 +116,7 @@ Declaring sources and sinks
You must provide information about the ``source`` and ``sink`` in your path query. These are objects that correspond to the nodes of the paths that you are exploring.
The name and the type of the ``source`` and the ``sink`` must be declared in the ``from`` statement of the query, and the types must be compatible with the nodes of the graph computed by the ``edges`` predicate.
If you are querying C/C++, C#, Java, or JavaScript code (and you have used ``import DataFlow::PathGraph`` in your query), the definitions of the ``source`` and ``sink`` are accessed via the ``Configuration`` class in the data flow library. You should declare all three of these objects in the ``from`` statement.
If you are querying C/C++, C#, Java, JavaScript, Python, or Ruby code (and you have used ``import DataFlow::PathGraph`` in your query), the definitions of the ``source`` and ``sink`` are accessed via the ``Configuration`` class in the data flow library. You should declare all three of these objects in the ``from`` statement.
For example:
.. code-block:: ql