mirror of
https://github.com/github/codeql.git
synced 2026-04-29 10:45:15 +02:00
Merge branch 'main' of github.com:github/codeql into python/promote-xpath-injection
This commit is contained in:
Rasmus Lerchedahl Petersen
@@ -216,7 +216,7 @@ commands that you can specify for compiled languages.
|
||||
|
||||
codeql database create csharp-database --language=csharp --command='dotnet build /p:UseSharedCompilation=false /t:rebuild'
|
||||
|
||||
- Go project built using the ``COEQL_EXTRACTOR_GO_BUILD_TRACING=on`` environment variable::
|
||||
- Go project built using the ``CODEQL_EXTRACTOR_GO_BUILD_TRACING=on`` environment variable::
|
||||
|
||||
CODEQL_EXTRACTOR_GO_BUILD_TRACING=on codeql database create go-database --language=go
|
||||
|
||||
|
||||
@@ -10,32 +10,34 @@ setting extractor configuration options through the CodeQL CLI.
|
||||
About extractor options
|
||||
-----------------------
|
||||
|
||||
Each extractor defines its own set of configuration options. To find out which options are available for a particular extractor, you can run ``codeql resolve languages`` or ``codeql resolve extractor`` with the ``--format=betterjson`` option. The ``betterjson`` output format provides the root paths of extractors and additional information. The output of ``codeql resolve extractor --format=betterjson`` will often be formatted like the following example::
|
||||
Each extractor defines its own set of configuration options. To find out which options are available for a particular extractor, you can run ``codeql resolve languages`` or ``codeql resolve extractor`` with the ``--format=betterjson`` option. The ``betterjson`` output format provides the root paths of extractors and additional information. The output of ``codeql resolve extractor --format=betterjson`` will often be formatted like the following example:
|
||||
|
||||
.. code-block:: json
|
||||
|
||||
{
|
||||
"extractor_root" : "/home/user/codeql/java",
|
||||
"extractor_options" : {
|
||||
"option1" : {
|
||||
"title" : "Java extractor option 1",
|
||||
"description" : "An example string option for the Java extractor.",
|
||||
"type" : "string",
|
||||
"pattern" : "[a-z]+"
|
||||
},
|
||||
"group1" : {
|
||||
"title" : "Java extractor group 1",
|
||||
"description" : "An example option group for the Java extractor.",
|
||||
"type" : "object",
|
||||
"properties" : {
|
||||
"option2" : {
|
||||
"title" : "Java extractor option 2",
|
||||
"description" : "An example array option for the Java extractor",
|
||||
"type" : "array",
|
||||
"pattern" : "[1-9][0-9]*"
|
||||
"extractor_root" : "/home/user/codeql/java",
|
||||
"extractor_options" : {
|
||||
"option1" : {
|
||||
"title" : "Java extractor option 1",
|
||||
"description" : "An example string option for the Java extractor.",
|
||||
"type" : "string",
|
||||
"pattern" : "[a-z]+"
|
||||
},
|
||||
"group1" : {
|
||||
"title" : "Java extractor group 1",
|
||||
"description" : "An example option group for the Java extractor.",
|
||||
"type" : "object",
|
||||
"properties" : {
|
||||
"option2" : {
|
||||
"title" : "Java extractor option 2",
|
||||
"description" : "An example array option for the Java extractor",
|
||||
"type" : "array",
|
||||
"pattern" : "[1-9][0-9]*"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
The extractor option names and descriptions are listed under ``extractor_options``. Each option may contain the following fields:
|
||||
|
||||
@@ -93,27 +95,31 @@ You can also set extractor options through a file. The CodeQL CLI subcommands th
|
||||
|
||||
Each option file contains a tree structure of nested maps. At the root is an extractor map key, and beneath it are map keys that correspond to extractor names. Starting at the third level, there are extractor options and option groups.
|
||||
|
||||
In JSON::
|
||||
In JSON:
|
||||
|
||||
.. code-block:: json
|
||||
|
||||
{
|
||||
"extractor" : {
|
||||
“java”: {
|
||||
"option1" : “abc”,
|
||||
"group1" : {
|
||||
"option2" : [ 102 ]
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
"java": {
|
||||
"option1" : "abc",
|
||||
"group1" : {
|
||||
"option2" : [ 102 ]
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
In YAML::
|
||||
In YAML:
|
||||
|
||||
.. code-block:: yaml
|
||||
|
||||
extractor:
|
||||
java:
|
||||
option1: “abc”
|
||||
group1:
|
||||
option2: [ 102 ]
|
||||
java:
|
||||
option1: "abc"
|
||||
group1:
|
||||
option2: [ 102 ]
|
||||
|
||||
The value for a ``string`` extractor option must be a string or a number (which will be converted to a string before further processing).
|
||||
|
||||
@@ -123,14 +129,16 @@ The value for an option group (of type ``object``) must be a map, which may cont
|
||||
|
||||
Each extractor option value must match the regular expression pattern of the extractor option (if it exists), and it must not contain newline characters.
|
||||
|
||||
Assigning an extractor option that does not exist is an error. You can make the CodeQL CLI ignore unknown extractor options by using a special ``__allow_unknown_properties`` Boolean field. For example, the following option file asks the CodeQL CLI to ignore all unknown extractor options and option groups under ``group1``::
|
||||
Assigning an extractor option that does not exist is an error. You can make the CodeQL CLI ignore unknown extractor options by using a special ``__allow_unknown_properties`` Boolean field. For example, the following option file asks the CodeQL CLI to ignore all unknown extractor options and option groups under ``group1``:
|
||||
|
||||
.. code-block:: yaml
|
||||
|
||||
extractor:
|
||||
java:
|
||||
option1: “abc”
|
||||
group1:
|
||||
__allow_unknown_properties: true
|
||||
option2: [ 102 ]
|
||||
java:
|
||||
option1: "abc"
|
||||
group1:
|
||||
__allow_unknown_properties: true
|
||||
option2: [ 102 ]
|
||||
|
||||
You can specify ``--extractor-options-file`` multiple times. The extractor option assignments are processed in the following order:
|
||||
|
||||
|
||||
@@ -23,7 +23,12 @@ and 4 are slightly different---for further details, see the sections labeled
|
||||
**Information for macOS "Catalina" (or newer) users**. If you are using macOS
|
||||
on Apple Silicon (e.g. Apple M1), ensure that the `Xcode command-line developer
|
||||
tools <https://developer.apple.com/downloads/index.action>`__ and `Rosetta 2
|
||||
<https://support.apple.com/en-us/HT211861>`__ are installed.
|
||||
<https://support.apple.com/en-us/HT211861>`__ are installed.
|
||||
|
||||
.. pull-quote:: Note
|
||||
|
||||
The CodeQL CLI is currently not compatible with non-glibc Linux
|
||||
distributions such as (muslc-based) Alpine Linux.
|
||||
|
||||
For information about installing the CodeQL CLI in a CI system to create results
|
||||
to display in GitHub as code scanning alerts, see
|
||||
@@ -200,7 +205,8 @@ clone and rename the repository in a single step by running
|
||||
|
||||
The CodeQL libraries and queries for Go analysis live in the `CodeQL for Go
|
||||
repository <https://github.com/github/codeql-go/>`__. Clone a copy of this
|
||||
repository into ``codeql-home``.
|
||||
repository into ``codeql-home``, and run ``codeql-go/scripts/install-deps.sh``
|
||||
to install its dependencies.
|
||||
|
||||
The cloned repositories should have a sibling relationship.
|
||||
For example, if the root of the cloned CodeQL repository is
|
||||
|
||||
@@ -154,7 +154,7 @@ For example, ``isThree(x)`` might be a call to a predicate that holds if the arg
|
||||
|
||||
A call to a predicate can also contain a closure operator, namely ``*`` or ``+``. For example,
|
||||
``a.isChildOf+(b)`` is a call to the :ref:`transitive closure <transitive-closures>` of
|
||||
``isChildOf()``, so it holds if ``a`` is a descendent of ``b``.
|
||||
``isChildOf()``, so it holds if ``a`` is a descendant of ``b``.
|
||||
|
||||
The predicate reference must resolve to exactly one predicate. For more information about how a predicate
|
||||
reference is resolved, see ":ref:`name-resolution`."
|
||||
|
||||
@@ -998,10 +998,14 @@ There are several kinds of expressions:
|
||||
| literal
|
||||
| variable
|
||||
| super_expr
|
||||
| callwithresult
|
||||
| postfix_cast
|
||||
| callwithresults
|
||||
| aggregation
|
||||
| expression_pragma
|
||||
| any
|
||||
| range
|
||||
| setliteral
|
||||
|
||||
|
||||
Parenthesized expressions
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
@@ -1303,6 +1307,26 @@ The values of an ``any`` expression are those values of the expression for which
|
||||
|
||||
The abbreviated cases for an ``any`` expression are interpreted in the same way as for an aggregation.
|
||||
|
||||
Expression Pragma
|
||||
~~~~~~~~~~~~~~~~~
|
||||
|
||||
Expression pragmas can be used to guide optimization.
|
||||
|
||||
::
|
||||
expression_pragma ::= "pragma" "[" expression_pragma_type "]" "(" expr ")"
|
||||
|
||||
expression_pragma_type ::= "only_bind_out" | "only_bind_into"
|
||||
|
||||
The values of an expression pragma are the values of the contained expression.
|
||||
|
||||
The type `only_bind_out` hints that uses of the result of the expression pragma should not be used to guide the evaluation of the result of the contained expression.
|
||||
When checking to see that all values are bound the compiler does not assume that if the result of the expression pragma is bound then the result of the contained
|
||||
expression is bound.
|
||||
|
||||
The type `only_bind_into` hints that uses of the contained expression should not be used to guide the evaluation of the result of the expression pragma.
|
||||
When checking to see that all values are bound the compiler does not assume that if the result of the contained expression is bound then the result of the
|
||||
expression pragma is bound.
|
||||
|
||||
Ranges
|
||||
~~~~~~
|
||||
|
||||
@@ -1506,9 +1530,10 @@ A range check has the following syntax:
|
||||
|
||||
::
|
||||
|
||||
inrange ::= expr "in" range
|
||||
inrange ::= expr "in" (range | setliteral)
|
||||
|
||||
The formula is equivalent to ``expr "=" range``.
|
||||
|
||||
The formula is equivalent to ``expr "=" range`` or ``expr "=" setliteral``.
|
||||
|
||||
Calls
|
||||
~~~~~
|
||||
@@ -2107,7 +2132,7 @@ The complete grammar for QL is as follows:
|
||||
|
||||
instanceof ::= expr "instanceof" type
|
||||
|
||||
inrange ::= expr "in" range
|
||||
inrange ::= expr "in" (range | setliteral)
|
||||
|
||||
call ::= predicateRef (closure)? "(" (exprs)? ")"
|
||||
| primary "." predicateName (closure)? "(" (exprs)? ")"
|
||||
@@ -2128,6 +2153,7 @@ The complete grammar for QL is as follows:
|
||||
| postfix_cast
|
||||
| callwithresults
|
||||
| aggregation
|
||||
| expression_pragma
|
||||
| any
|
||||
| range
|
||||
| setliteral
|
||||
@@ -2159,6 +2185,10 @@ The complete grammar for QL is as follows:
|
||||
| aggid ("[" expr "]")? "(" as_exprs ("order" "by" aggorderbys)? ")"
|
||||
| "unique" "(" var_decls "|" (formula)? ("|" as_exprs)? ")"
|
||||
|
||||
expression_pragma ::= "pragma" "[" expression_pragma_type "]" "(" expr ")"
|
||||
|
||||
expression_pragma_type ::= "only_bind_out" | "only_bind_into"
|
||||
|
||||
aggid ::= "avg" | "concat" | "count" | "max" | "min" | "rank" | "strictconcat" | "strictcount" | "strictsum" | "sum"
|
||||
|
||||
aggorderbys ::= aggorderby ("," aggorderby)*
|
||||
|
||||
@@ -171,6 +171,8 @@ Python built-in support
|
||||
Twisted, Web framework
|
||||
Flask-Admin, Web framework
|
||||
starlette, Asynchronous Server Gateway Interface (ASGI)
|
||||
python-ldap, Lightweight Directory Access Protocol (LDAP)
|
||||
ldap3, Lightweight Directory Access Protocol (LDAP)
|
||||
requests, HTTP client
|
||||
dill, Serialization
|
||||
PyYAML, Serialization
|
||||
|
||||
@@ -11,26 +11,26 @@
|
||||
Microsoft extensions (up to VS 2019),
|
||||
|
||||
Arm Compiler 5 [3]_","``.cpp``, ``.c++``, ``.cxx``, ``.hpp``, ``.hh``, ``.h++``, ``.hxx``, ``.c``, ``.cc``, ``.h``"
|
||||
C#,C# up to 9.0,"Microsoft Visual Studio up to 2019 with .NET up to 4.8,
|
||||
C#,C# up to 10.0,"Microsoft Visual Studio up to 2019 with .NET up to 4.8,
|
||||
|
||||
.NET Core up to 3.1
|
||||
|
||||
.NET 5","``.sln``, ``.csproj``, ``.cs``, ``.cshtml``, ``.xaml``"
|
||||
.NET 5, .NET 6","``.sln``, ``.csproj``, ``.cs``, ``.cshtml``, ``.xaml``"
|
||||
Go (aka Golang), "Go up to 1.17", "Go 1.11 or more recent", ``.go``
|
||||
Java,"Java 7 to 16 [4]_","javac (OpenJDK and Oracle JDK),
|
||||
Java,"Java 7 to 17 [4]_","javac (OpenJDK and Oracle JDK),
|
||||
|
||||
Eclipse compiler for Java (ECJ) [5]_",``.java``
|
||||
JavaScript,ECMAScript 2021 or lower,Not applicable,"``.js``, ``.jsx``, ``.mjs``, ``.es``, ``.es6``, ``.htm``, ``.html``, ``.xhtm``, ``.xhtml``, ``.vue``, ``.hbs``, ``.ejs``, ``.njk``, ``.json``, ``.yaml``, ``.yml``, ``.raml``, ``.xml`` [6]_"
|
||||
Python,"2.7, 3.5, 3.6, 3.7, 3.8, 3.9",Not applicable,``.py``
|
||||
Python,"2.7, 3.5, 3.6, 3.7, 3.8, 3.9, 3.10",Not applicable,``.py``
|
||||
Ruby [7]_,"up to 3.0.2",Not applicable,"``.rb``, ``.erb``, ``.gemspec``, ``Gemfile``"
|
||||
TypeScript [8]_,"2.6-4.5",Standard TypeScript compiler,"``.ts``, ``.tsx``"
|
||||
TypeScript [8]_,"2.6-4.6",Standard TypeScript compiler,"``.ts``, ``.tsx``"
|
||||
|
||||
.. container:: footnote-group
|
||||
|
||||
.. [1] C++20 support is currently in beta. Supported for GCC on Linux only. Modules are *not* supported.
|
||||
.. [2] Support for the clang-cl compiler is preliminary.
|
||||
.. [3] Support for the Arm Compiler (armcc) is preliminary.
|
||||
.. [4] Builds that execute on Java 7 to 16 can be analyzed. The analysis understands Java 16 standard language features.
|
||||
.. [4] Builds that execute on Java 7 to 17 can be analyzed. The analysis understands Java 17 standard language features.
|
||||
.. [5] ECJ is supported when the build invokes it via the Maven Compiler plugin or the Takari Lifecycle plugin.
|
||||
.. [6] JSX and Flow code, YAML, JSON, HTML, and XML files may also be analyzed with JavaScript files.
|
||||
.. [7] Requires glibc 2.17.
|
||||
|
||||
@@ -78,6 +78,8 @@ To avoid these problems, a data-flow ``Configuration`` comes with a mechanism fo
|
||||
*/
|
||||
final predicate hasPartialFlow(PartialPathNode source, PartialPathNode node, int dist) {
|
||||
|
||||
There is also a ``Configuration.hasPartialFlowRev`` for exploring flow backwards from a sink.
|
||||
|
||||
As noted in the documentation for ``hasPartialFlow`` (for example, in the
|
||||
`CodeQL for Java documentation <https://codeql.github.com/codeql-standard-libraries/java/semmle/code/java/dataflow/internal/DataFlowImpl2.qll/predicate.DataFlowImpl2$Configuration$hasPartialFlow.3.html>`__) you must first enable this by adding an override of ``explorationLimit``. For example:
|
||||
|
||||
@@ -87,6 +89,8 @@ As noted in the documentation for ``hasPartialFlow`` (for example, in the
|
||||
|
||||
This defines the exploration radius within which ``hasPartialFlow`` returns results.
|
||||
|
||||
To get good performance when using ``hasPartialFlow`` it is important to ensure the ``isSink`` predicate of the configuration has no results. Likewise, when using ``hasPartialFlowRev`` the ``isSource`` predicate of the configuration should have no results.
|
||||
|
||||
It is also useful to focus on a single source at a time as the starting point for the flow exploration. This is most easily done by adding a temporary restriction in the ``isSource`` predicate.
|
||||
|
||||
To do quick evaluations of partial flow it is often easiest to add a predicate to the query that is solely intended for quick evaluation (right-click the predicate name and choose "CodeQL: Quick Evaluation"). A good starting point is something like:
|
||||
@@ -113,4 +117,4 @@ Further reading
|
||||
----------------
|
||||
|
||||
- :ref:`About data flow analysis <about-data-flow-analysis>`
|
||||
- :ref:`Creating path queries <creating-path-queries>`
|
||||
- :ref:`Creating path queries <creating-path-queries>`
|
||||
|
||||
@@ -52,11 +52,6 @@ The following properties are supported by all query files:
|
||||
| ``@security-severity``| ``<score>`` | Defines the level of severity, between 0.0 and 10.0, for queries with ``@tags security``. For more information about calculating ``@security-severity``, see the `GitHub changelog <https://github.blog/changelog/2021-07-19-codeql-code-scanning-new-severity-levels-for-security-alerts/>`__. |
|
||||
+-----------------------+---------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|
||||
|
||||
Additional properties for filter queries
|
||||
----------------------------------------
|
||||
|
||||
Filter queries are used to define additional constraints to limit the results that are returned by other queries. A filter query must have the same ``@kind`` property as the query whose results it is filtering. No additional metadata properties are required.
|
||||
|
||||
Example
|
||||
-------
|
||||
|
||||
|
||||
Reference in New Issue
Block a user