update lots of links

This commit is contained in:
james
2020-08-11 15:40:42 +01:00
parent d214cecf4d
commit 0ff10c319d
28 changed files with 100 additions and 87 deletions

View File

@@ -138,7 +138,7 @@ To decide who should inherit the king's fortune, the villagers carefully read th
*"The heir to the throne is the closest living relative of the king. Any person with a criminal record will not be considered. If there are multiple candidates, the oldest person is the heir."*
As your final challenge, define a predicate ``hasCriminalRecord`` so that ``hasCriminalRecord(p)`` holds if ``p`` is any of the criminals you unmasked earlier (in the :doc:`Find the thief <find-the-thief>` and :doc:`Catch the fire starter <catch-the-fire-starter>` tutorials).
As your final challenge, define a predicate ``hasCriminalRecord`` so that ``hasCriminalRecord(p)`` holds if ``p`` is any of the criminals you unmasked earlier (in the ":doc:`Find the thief <find-the-thief>`" and ":doc:`Catch the fire starter <catch-the-fire-starter>`" tutorials).
`See the answer in the query console on LGTM.com <https://lgtm.com/query/1820692755164273290/>`__

View File

@@ -6,7 +6,7 @@ You can use data flow analysis to track the flow of potentially malicious or ins
About data flow
---------------
Data flow analysis computes the possible values that a variable can hold at various points in a program, determining how those values propagate through the program, and where they are used. In CodeQL, you can model both local data flow and global data flow. For a more general introduction to modeling data flow, see :doc:`About data flow analysis <../intro-to-data-flow>`.
Data flow analysis computes the possible values that a variable can hold at various points in a program, determining how those values propagate through the program, and where they are used. In CodeQL, you can model both local data flow and global data flow. For a more general introduction to modeling data flow, see ":doc:`About data flow analysis <../intro-to-data-flow>`."
Local data flow
---------------
@@ -390,7 +390,8 @@ Exercise 4
Further reading
---------------
- `Exploring data flow with path queries <https://help.semmle.com/codeql/codeql-for-vscode/procedures/exploring-paths.html>`__
- "`Exploring data flow with path queries <https://help.semmle.com/codeql/codeql-for-vscode/procedures/exploring-paths.html>`__"
.. include:: ../../reusables/cpp-further-reading.rst
.. include:: ../../reusables/codeql-ref-tools-further-reading.rst

View File

@@ -6,7 +6,7 @@ You can improve the results generated by a CodeQL query by adding conditions to
Overview
--------
This topic describes how a C++ query was developed. The example introduces recursive predicates and demonstrates the typical workflow used to refine a query. For a full overview of the topics available for learning to write queries for C/C++ code, see :doc:`CodeQL for C and C++ <ql-for-cpp>`.
This topic describes how a C++ query was developed. The example introduces recursive predicates and demonstrates the typical workflow used to refine a query. For a full overview of the topics available for learning to write queries for C/C++ code, see ":doc:`CodeQL for C and C++ <ql-for-cpp>`."
Finding every private field and checking for initialization
-----------------------------------------------------------
@@ -102,7 +102,7 @@ You may also wish to consider methods called by constructors that assign to the
int m_value;
};
This case can be excluded by creating a recursive predicate. The recursive predicate is given a function and a field, then checks whether the function assigns to the field. The predicate runs itself on all the functions called by the function that it has been given. By passing the constructor to this predicate, we can check for assignments of a field in all functions called by the constructor, and then do the same for all functions called by those functions all the way down the tree of function calls. For more information, see `Recursion <https://help.semmle.com/QL/ql-handbook/recursion.html>`__ in the QL language reference.
This case can be excluded by creating a recursive predicate. The recursive predicate is given a function and a field, then checks whether the function assigns to the field. The predicate runs itself on all the functions called by the function that it has been given. By passing the constructor to this predicate, we can check for assignments of a field in all functions called by the constructor, and then do the same for all functions called by those functions all the way down the tree of function calls. For more information, see "`Recursion <https://help.semmle.com/QL/ql-handbook/recursion.html>`__" in the QL language reference.
.. code-block:: ql
@@ -126,7 +126,7 @@ This case can be excluded by creating a recursive predicate. The recursive predi
Refinement 4—simplifying the query
----------------------------------
Finally we can simplify the query by using the transitive closure operator. In this final version of the query, ``c.calls*(fun)`` resolves to the set of all functions that are ``c`` itself, are called by ``c``, are called by a function that is called by ``c``, and so on. This eliminates the need to make a new predicate all together. For more information, see `Transitive closures <https://help.semmle.com/QL/ql-handbook/recursion.html#transitive-closures>`__ in the QL language reference.
Finally we can simplify the query by using the transitive closure operator. In this final version of the query, ``c.calls*(fun)`` resolves to the set of all functions that are ``c`` itself, are called by ``c``, are called by a function that is called by ``c``, and so on. This eliminates the need to make a new predicate all together. For more information, see "`Transitive closures <https://help.semmle.com/QL/ql-handbook/recursion.html#transitive-closures>`__" in the QL language reference.
.. code-block:: ql

View File

@@ -8,7 +8,7 @@ About this article
This article describes how data flow analysis is implemented in the CodeQL libraries for C# and includes examples to help you write your own data flow queries.
The following sections describe how to use the libraries for local data flow, global data flow, and taint tracking.
For a more general introduction to modeling data flow, see :doc:`About data flow analysis <../intro-to-data-flow>`.
For a more general introduction to modeling data flow, see ":doc:`About data flow analysis <../intro-to-data-flow>`."
Local data flow
---------------
@@ -553,7 +553,8 @@ This can be adapted from the ``SystemUriFlow`` class:
Further reading
---------------
- `Exploring data flow with path queries <https://help.semmle.com/codeql/codeql-for-vscode/procedures/exploring-paths.html>`__
- "`Exploring data flow with path queries <https://help.semmle.com/codeql/codeql-for-vscode/procedures/exploring-paths.html>`__"
.. include:: ../../reusables/csharp-further-reading.rst
.. include:: ../../reusables/codeql-ref-tools-further-reading.rst

View File

@@ -14,7 +14,7 @@ There is an extensive core library for analyzing CodeQL databases extracted from
Since this is required for all C# queries, it's omitted from code snippets below.
The core library contains all the program elements, including `files <#files>`__, `types <#types>`__, methods, `variables <#variables>`__, `statements <#statements>`__, and `expressions <#expressions>`__. This is sufficient for most queries, however additional libraries can be imported for bespoke functionality such as control flow and data flow. For information about these additional libraries, see :doc:`CodeQL for C# <ql-for-csharp>`.
The core library contains all the program elements, including `files <#files>`__, `types <#types>`__, methods, `variables <#variables>`__, `statements <#statements>`__, and `expressions <#expressions>`__. This is sufficient for most queries, however additional libraries can be imported for bespoke functionality such as control flow and data flow. For information about these additional libraries, see ":doc:`CodeQL for C# <ql-for-csharp>`."
Class hierarchies
~~~~~~~~~~~~~~~~~
@@ -1079,7 +1079,7 @@ Exercise 9
from Variable v, StringLiteral value
where v.getName().regexpMatch("[pP]ass(word|wd|)")
and value = v.getAnAssignedValue()
and value.getValue() != ""
and value.getValue() != "
select v, "Hard-coded password '" + value.getValue() + "'."
Exercise 10

View File

@@ -502,7 +502,7 @@ taint, you can define a subclass of ``TaintTracking::Configuration``, which work
data-flow configurations.
A detailed exposition of global data flow and taint tracking is out of scope for this brief
introduction. For a general overview of data flow and taint tracking, see `About data flow analysis <https://help.semmle.com/QL/learn-ql/intro-to-data-flow.html>`__.
introduction. For a general overview of data flow and taint tracking, see "`About data flow analysis <https://help.semmle.com/QL/learn-ql/intro-to-data-flow.html>`__."
Advanced libraries
------------------

View File

@@ -13,7 +13,7 @@ CodeQL is based on a powerful query language called QL. The following topics hel
Important
If you've previously used QL, you may notice slight changes in terms we use to describe some important concepts. For more information, see our note about :doc:`Recent terminology changes <terminology-note>`.
If you've previously used QL, you may notice slight changes in terms we use to describe some important concepts. For more information, see our note about ":doc:`Recent terminology changes <terminology-note>`."
.. toctree::
:maxdepth: 1

View File

@@ -14,17 +14,17 @@ The following sections provide a brief introduction to data flow analysis with C
See the following tutorials for more information about analyzing data flow in specific languages:
- :doc:`Analyzing data flow in C/C++ <cpp/dataflow>`
- :doc:`Analyzing data flow in C# <csharp/dataflow>`
- :doc:`Analyzing data flow in Java <java/dataflow>`
- :doc:`Analyzing data flow in JavaScript/TypeScript <javascript/dataflow>`
- :doc:`Analyzing data flow and tracking tainted data in Python <python/taint-tracking>`
- ":doc:`Analyzing data flow in C/C++ <cpp/dataflow>`"
- ":doc:`Analyzing data flow in C# <csharp/dataflow>`"
- ":doc:`Analyzing data flow in Java <java/dataflow>`"
- ":doc:`Analyzing data flow in JavaScript/TypeScript <javascript/dataflow>`"
- ":doc:`Analyzing data flow and tracking tainted data in Python <python/taint-tracking>`"
.. pull-quote::
Note
Data flow analysis is used extensively in path queries. To learn more about path queries, see :doc:`Creating path queries <writing-queries/path-queries>`.
Data flow analysis is used extensively in path queries. To learn more about path queries, see ":doc:`Creating path queries <writing-queries/path-queries>`."
.. _data-flow-graph:
@@ -82,4 +82,5 @@ These flow steps are modeled in the taint-tracking library using predicates that
Further reading
***************
- `Exploring data flow with path queries <https://help.semmle.com/codeql/codeql-for-vscode/procedures/exploring-paths.html>`__
- "`Exploring data flow with path queries <https://help.semmle.com/codeql/codeql-for-vscode/procedures/exploring-paths.html>`__"

View File

@@ -157,6 +157,6 @@ To import the CodeQL library for a specific programming language, type ``import
Further reading
---------------
- To find out more about how to write your own queries, try working through the :doc:`QL tutorials <beginner/ql-tutorials>`.
- For an overview of the other available resources, see :doc:`Learning CodeQL <../index>`.
- For a more technical description of the underlying language, see the `QL language reference <https://help.semmle.com/QL/ql-handbook>`__.
- To find out more about how to write your own queries, try working through the ":doc:`QL tutorials <beginner/ql-tutorials>`."
- For an overview of the other available resources, see ":doc:`Learning CodeQL <../index>`."
- For a more technical description of the underlying language, see the "`QL language reference <https://help.semmle.com/QL/ql-handbook>`__."

View File

@@ -180,7 +180,7 @@ Finally, we use these classes to find calls to deprecated methods, excluding cal
In our example, this query flags the call to ``A.m`` in ``A.r``, but not the one in ``A.n``.
For more information about the class ``Call``, see :doc:`Navigating the call graph <call-graph>`.
For more information about the class ``Call``, see ":doc:`Navigating the call graph <call-graph>`."
Improvements
~~~~~~~~~~~~

View File

@@ -9,7 +9,7 @@ About this article
This article describes how data flow analysis is implemented in the CodeQL libraries for Java and includes examples to help you write your own data flow queries.
The following sections describe how to use the libraries for local data flow, global data flow, and taint tracking.
For a more general introduction to modeling data flow, see :doc:`About data flow analysis <../intro-to-data-flow>`.
For a more general introduction to modeling data flow, see ":doc:`About data flow analysis <../intro-to-data-flow>`."
Local data flow
---------------
@@ -358,7 +358,8 @@ Exercise 4
Further reading
---------------
- `Exploring data flow with path queries <https://help.semmle.com/codeql/codeql-for-vscode/procedures/exploring-paths.html>`__
- "`Exploring data flow with path queries <https://help.semmle.com/codeql/codeql-for-vscode/procedures/exploring-paths.html>`__"
.. include:: ../../reusables/java-further-reading.rst
.. include:: ../../reusables/codeql-ref-tools-further-reading.rst

View File

@@ -26,7 +26,7 @@ If ``l`` is bigger than 2\ :sup:`31`\ - 1 (the largest positive value of type ``
All primitive numeric types have a maximum value, beyond which they will wrap around to their lowest possible value (called an "overflow"). For ``int``, this maximum value is 2\ :sup:`31`\ - 1. Type ``long`` can accommodate larger values up to a maximum of 2\ :sup:`63`\ - 1. In this example, this means that ``l`` can take on a value that is higher than the maximum for type ``int``; ``i`` will never be able to reach this value, instead overflowing and returning to a low value.
We're going to develop a query that finds code that looks like it might exhibit this kind of behavior. We'll be using several of the standard library classes for representing statements and functions. For a full list, see :doc:`Abstract syntax tree classes for working with Java programs <ast-class-reference>`.
We're going to develop a query that finds code that looks like it might exhibit this kind of behavior. We'll be using several of the standard library classes for representing statements and functions. For a full list, see ":doc:`Abstract syntax tree classes for working with Java programs <ast-class-reference>`."
Initial query
-------------

View File

@@ -210,7 +210,7 @@ Class ``Variable`` represents a variable `in the Java sense <https://docs.oracle
Abstract syntax tree
--------------------
Classes in this category represent abstract syntax tree (AST) nodes, that is, statements (class ``Stmt``) and expressions (class ``Expr``). For a full list of expression and statement types available in the standard QL library, see :doc:`Abstract syntax tree classes for working with Java programs <ast-class-reference>`.
Classes in this category represent abstract syntax tree (AST) nodes, that is, statements (class ``Stmt``) and expressions (class ``Expr``). For a full list of expression and statement types available in the standard QL library, see ":doc:`Abstract syntax tree classes for working with Java programs <ast-class-reference>`."
Both ``Expr`` and ``Stmt`` provide member predicates for exploring the abstract syntax tree of a program:
@@ -379,7 +379,7 @@ Conversely, ``Callable.getAReference`` returns a ``Call`` that refers to it. So
where not exists(c.getAReference())
select c
`See this in the query console on LGTM.com <https://lgtm.com/query/7261739919657747703/>`__. The LGTM.com demo projects all appear to have many methods that are not called directly, but this is unlikely to be the whole story. To explore this area further, see :doc:`Navigating the call graph <call-graph>`.
`See this in the query console on LGTM.com <https://lgtm.com/query/7261739919657747703/>`__. The LGTM.com demo projects all appear to have many methods that are not called directly, but this is unlikely to be the whole story. To explore this area further, see ":doc:`Navigating the call graph <call-graph>`."
For more information about callables and calls, see the :doc:`article on the call graph <call-graph>`.

View File

@@ -34,12 +34,12 @@ This query reports flow paths which:
- Step through variables, function calls, properties, strings, arrays, promises, exceptions, and steps added by `isAdditionalTaintStep <https://help.semmle.com/qldoc/javascript/semmle/javascript/dataflow/TaintTracking.qll/predicate.TaintTracking$TaintTracking$Configuration$isAdditionalTaintStep.2.html>`__.
- End at a node matched by `isSink <https://help.semmle.com/qldoc/javascript/semmle/javascript/dataflow/Configuration.qll/predicate.Configuration$Configuration$isSink.1.html>`__.
See also: `Global data flow <https://help.semmle.com/QL/learn-ql/javascript/dataflow.html#global-data-flow>`__ and :doc:`Creating path queries <../writing-queries/path-queries>`.
See also: "`Global data flow <https://help.semmle.com/QL/learn-ql/javascript/dataflow.html#global-data-flow>`__" and ":doc:`Creating path queries <../writing-queries/path-queries>`."
DataFlow module
---------------
Use data flow nodes to match program elements independently of syntax. See also: :doc:`Analyzing data flow in JavaScript and TypeScript <dataflow>`.
Use data flow nodes to match program elements independently of syntax. See also: ":doc:`Analyzing data flow in JavaScript and TypeScript <dataflow>`."
Predicates in the ``DataFlow::`` module:
@@ -142,7 +142,7 @@ Files
AST nodes
---------
See also: :doc:`Abstract syntax tree classes for working with JavaScript and TypeScript programs <ast-class-reference>`.
See also: ":doc:`Abstract syntax tree classes for working with JavaScript and TypeScript programs <ast-class-reference>`."
Conversion between DataFlow and AST nodes:
@@ -163,7 +163,7 @@ String matching
Type tracking
-------------
See also: :doc:`Using type tracking for API modeling <type-tracking>`.
See also: ":doc:`Using type tracking for API modeling <type-tracking>`."
Use the following template to define forward type tracking predicates:
@@ -220,7 +220,8 @@ Troubleshooting
Further reading
---------------
- `Exploring data flow with path queries <https://help.semmle.com/codeql/codeql-for-vscode/procedures/exploring-paths.html>`__
- "`Exploring data flow with path queries <https://help.semmle.com/codeql/codeql-for-vscode/procedures/exploring-paths.html>`__"
.. include:: ../../reusables/javascript-further-reading.rst
.. include:: ../../reusables/codeql-ref-tools-further-reading.rst

View File

@@ -9,7 +9,7 @@ The various sections in this article describe how to utilize the libraries for l
As our running example, we will develop a query that identifies command-line arguments that are passed as a file path to the standard Node.js ``readFile`` function.
While this is not a problematic pattern as such, it is typical of the kind of reasoning that is frequently used in security queries.
For a more general introduction to modeling data flow, see :doc:`About data flow analysis <../intro-to-data-flow>`.
For a more general introduction to modeling data flow, see ":doc:`About data flow analysis <../intro-to-data-flow>`."
Data flow nodes
---------------
@@ -554,7 +554,8 @@ Exercise 4
Further reading
---------------
- `Exploring data flow with path queries <https://help.semmle.com/codeql/codeql-for-vscode/procedures/exploring-paths.html>`__
- "`Exploring data flow with path queries <https://help.semmle.com/codeql/codeql-for-vscode/procedures/exploring-paths.html>`__"
.. include:: ../../reusables/java-further-reading.rst
.. include:: ../../reusables/codeql-ref-tools-further-reading.rst

View File

@@ -7,7 +7,7 @@ Overview
--------
You can use basic inter-procedural data-flow analysis and taint tracking as described in
:doc:`Analyzing data flow in JavaScript and TypeScript <dataflow>` to check whether there is a path in
":doc:`Analyzing data flow in JavaScript and TypeScript <dataflow>`" to check whether there is a path in
the data-flow graph from some source node to a sink node that does not pass through any sanitizer
nodes. Another way of thinking about this is that it statically models the flow of data through the
program, and associates a flag with every data value telling us whether it might have come from a
@@ -398,7 +398,8 @@ string may be an absolute path and whether it may contain ``..`` components.
Further reading
---------------
- `Exploring data flow with path queries <https://help.semmle.com/codeql/codeql-for-vscode/procedures/exploring-paths.html>`__
- "`Exploring data flow with path queries <https://help.semmle.com/codeql/codeql-for-vscode/procedures/exploring-paths.html>`__"
.. include:: ../../reusables/javascript-further-reading.rst
.. include:: ../../reusables/codeql-ref-tools-further-reading.rst

View File

@@ -136,7 +136,7 @@ The CodeQL class `ClassOrInterface <https://help.semmle.com/qldoc/javascript/sem
Note that the superclass of a class is an expression, not a type annotation. If the superclass has type arguments, it will be an expression of kind `ExpressionWithTypeArguments <https://help.semmle.com/qldoc/javascript/semmle/javascript/TypeScript.qll/type.TypeScript$ExpressionWithTypeArguments.html>`__.
Also see the documentation for classes in the `CodeQL libraries for JavaScript <introduce-libraries-js#classes>`__.
Also see the documentation for classes in the "`CodeQL libraries for JavaScript <introduce-libraries-js#classes>`__."
To select the type references to a class or an interface, use ``getTypeName()``.

View File

@@ -490,10 +490,10 @@ Prefer type tracking when:
Prefer data-flow configurations when:
- Tracking user-controlled data -- use `taint tracking <https://help.semmle.com/QL/learn-ql/javascript/dataflow.html#using-global-taint-tracking>`__.
- Differentiating between different kinds of user-controlled data -- see :doc:`Using flow labels for precise data flow analysis <flow-labels>`.
- Differentiating between different kinds of user-controlled data -- see ":doc:`Using flow labels for precise data flow analysis <flow-labels>`."
- Tracking transformations of a value through generic utility functions.
- Tracking values through string manipulation.
- Generating a path from source to sink -- see :doc:`Creating path queries <../writing-queries/path-queries>`.
- Generating a path from source to sink -- see ":doc:`Creating path queries <../writing-queries/path-queries>`."
Lastly, depending on the code base being analyzed, some alternatives to consider are:

View File

@@ -112,7 +112,7 @@ Example finding mutually exclusive blocks within the same function
)
select b1, b2
`See this in the query console on LGTM.com <https://lgtm.com/query/671000028/>`__. This typically gives a very large number of results, because it is a common occurrence in normal control flow. It is, however, an example of the sort of control-flow analysis that is possible. Control-flow analyses such as this are an important aid to data flow analysis. For more information, see :doc:`Analyzing data flow and tracking tainted data in Python <taint-tracking>`.
`See this in the query console on LGTM.com <https://lgtm.com/query/671000028/>`__. This typically gives a very large number of results, because it is a common occurrence in normal control flow. It is, however, an example of the sort of control-flow analysis that is possible. Control-flow analyses such as this are an important aid to data flow analysis. For more information, see ":doc:`Analyzing data flow and tracking tainted data in Python <taint-tracking>`."
Further reading
---------------

View File

@@ -259,7 +259,8 @@ which defines the simplest possible taint kind class, ``HardcodedValue``, and cu
Further reading
---------------
- `Exploring data flow with path queries <https://help.semmle.com/codeql/codeql-for-vscode/procedures/exploring-paths.html>`__
- "`Exploring data flow with path queries <https://help.semmle.com/codeql/codeql-for-vscode/procedures/exploring-paths.html>`__"
.. include:: ../../reusables/python-further-reading.rst
.. include:: ../../reusables/codeql-ref-tools-further-reading.rst

View File

@@ -25,7 +25,7 @@ When you have selected a presentation, use |arrow-r| and |arrow-l| to navigate b
Press **p** to view the additional notes on slides that have an information icon |info| in the top right corner, and press **f** to enter full-screen mode.
The presentations contain a number of query examples.
We recommend that you download `CodeQL for Visual Studio Code <https://help.semmle.com/codeql/codeql-for-vscode/procedures/setting-up.html>`__ and add the example database for each presentation so that you can find the bugs mentioned in the slides.
We recommend that you download `CodeQL for Visual Studio Code <https://help.semmle.com/codeql/codeql-for-vscode/procedures/setting-up.html>`__ and add the example database for each presentation so that you can find the bugs mentioned in the slides.
.. pull-quote::

View File

@@ -11,7 +11,7 @@ Before reading the tips below, it is worth reiterating a few important points ab
- CodeQL `predicates <https://help.semmle.com/QL/ql-handbook/predicates.html>`__ and `classes <https://help.semmle.com/QL/ql-handbook/types.html#classes>`__ are evaluated to database `tables <https://en.wikipedia.org/wiki/Table_(database)>`__. Large predicates generate large tables with many rows, and are therefore expensive to compute.
- The QL language is implemented using standard database operations and `relational algebra <https://en.wikipedia.org/wiki/Relational_algebra>`__ (such as join, projection, and union). For further information about query languages and databases, see `About the QL language <https://help.semmle.com/QL/learn-ql/about-ql.html>`__.
- Queries are evaluated *bottom-up*, which means that a predicate is not evaluated until *all* of the predicates that it depends on are evaluated. For more information on query evaluation, see `Evaluation of QL programs <https://help.semmle.com/QL/ql-handbook/evaluation.html>`__.
- Queries are evaluated *bottom-up*, which means that a predicate is not evaluated until *all* of the predicates that it depends on are evaluated. For more information on query evaluation, see "`Evaluation of QL programs <https://help.semmle.com/QL/ql-handbook/evaluation.html>`__."
Performance tips
----------------
@@ -54,7 +54,7 @@ To avoid making this mistake, ``this`` should be restricted in the member predic
Use specific types
~~~~~~~~~~~~~~~~~~
`Types <https://help.semmle.com/QL/ql-handbook/types.html>`__ provide an upper bound on the size of a relation.
"`Types <https://help.semmle.com/QL/ql-handbook/types.html>`__" provide an upper bound on the size of a relation.
This helps the query optimizer be more effective, so it's generally good to use the most specific types possible. For example::
predicate foo(LoggingCall e)
@@ -90,7 +90,7 @@ Use ``getAQlClass()`` as a debugging tool, but don't include it in the final ver
Avoid complex recursion
~~~~~~~~~~~~~~~~~~~~~~~
`Recursion <https://help.semmle.com/QL/ql-handbook/recursion.html>`__ is about self-referencing definitions.
"`Recursion <https://help.semmle.com/QL/ql-handbook/recursion.html>`__" is about self-referencing definitions.
It can be extremely powerful as long as it is used appropriately.
On the whole, you should try to make recursive predicates as simple as possible.
That is, you should define a *base case* that allows the predicate to *bottom out*, along with a single *recursive call*::

View File

@@ -11,7 +11,7 @@ CodeQL includes queries to find the most relevant and interesting problems for e
- **Alert queries**: queries that highlight issues in specific locations in your code.
- **Path queries**: queries that describe the flow of information between a source and a sink in your code.
You can add custom queries to `custom query packs <https://lgtm.com/help/lgtm/about-queries#what-are-query-packs>`__ to analyze your projects in `LGTM <https://lgtm.com>`__, use them to analyze a database with the `CodeQL CLI <https://help.semmle.com/codeql/codeql-cli.html>`__, or you can contribute to the standard CodeQL queries in our `open source repository on GitHub <https://github.com/github/codeql>`__.
You can add custom queries to `custom query packs <https://lgtm.com/help/lgtm/about-queries#what-are-query-packs>`__ to analyze your projects in `LGTM <https://lgtm.com>`__, use them to analyze a database with the "`CodeQL CLI <https://help.semmle.com/codeql/codeql-cli.html>`__" ," or you can contribute to the standard CodeQL queries in our `open source repository on GitHub <https://github.com/github/codeql>`__.
This topic is a basic introduction to query files. You can find more information on writing queries for specific programming languages `here <https://help.semmle.com/QL/learn-ql/>`__, and detailed technical information about QL in the `QL language reference <https://help.semmle.com/QL/ql-handbook/index.html>`__.
For more information on how to format your code when contributing queries to the GitHub repository, see the `CodeQL style guide <https://github.com/github/codeql/blob/master/docs/ql-style-guide.md>`__.
@@ -35,23 +35,24 @@ Basic query structure
where /* ... logical formula ... */
select /* ... expressions ... */
The following sections describe the information that is typically included in a query file for alerts. Path queries are discussed in more detail in :doc:`Creating path queries <path-queries>`.
The following sections describe the information that is typically included in a query file for alerts. Path queries are discussed in more detail in ":doc:`Creating path queries <path-queries>`."
Query metadata
==============
Query metadata is used to identify your custom queries when they are added to the GitHub repository or used in your analysis. Metadata provides information about the query's purpose, and also specifies how to interpret and display the query results. For a full list of metadata properties, see :doc:`Metadata for CodeQL queries <query-metadata>`. The exact metadata requirement depends on how you are going to run your query:
Query metadata is used to identify your custom queries when they are added to the GitHub repository or used in your analysis. Metadata provides information about the query's purpose, and also specifies how to interpret and display the query results. For a full list of metadata properties, see ":doc:`Metadata for CodeQL queries <query-metadata>`"
." The exact metadata requirement depends on how you are going to run your query:
- If you are contributing a query to the GitHub repository, please read the `query metadata style guide <https://github.com/github/codeql/blob/master/docs/query-metadata-style-guide.md#metadata-area>`__.
- If you are contributing a query to the GitHub repository, please read the `query metadata style guide <https://github.com/github/codeql/blob/master/docs/query-metadata-style-guide.md>`__.
- If you are adding a custom query to a query pack for analysis using LGTM , see `Writing custom queries to include in LGTM analysis <https://lgtm.com/help/lgtm/writing-custom-queries>`__.
- If you are analyzing a database using the `CodeQL CLI <https://help.semmle.com/codeql/codeql-cli.html>`__, your query metadata must contain ``@kind``.
- If you are analyzing a database using the "`CodeQL CLI <https://help.semmle.com/codeql/codeql-cli.html>`__" ," your query metadata must contain ``@kind``.
- If you are running a query in the query console on LGTM or with the CodeQL extension for VS Code, metadata is not mandatory. However, if you want your results to be displayed as either an 'alert' or a 'path', you must specify the correct ``@kind`` property, as explained below. For more information, see `Using the query console <https://lgtm.com/help/lgtm/using-query-console>`__ on LGTM.com and `Analyzing your projects <https://help.semmle.com/codeql/codeql-for-vscode/procedures/using-extension.html>`__ in the CodeQL for VS Code help.
.. pull-quote::
Note
Queries that are contributed to the open source repository, added to a query pack in LGTM, or used to analyze a database with the `CodeQL CLI <https://help.semmle.com/codeql/codeql-cli.html>`__ must have a query type (``@kind``) specified. The ``@kind`` property indicates how to interpret and display the results of the query analysis:
Queries that are contributed to the open source repository, added to a query pack in LGTM, or used to analyze a database with the `CodeQL CLI <https://help.semmle.com/codeql/codeql-cli.html>`__ must have a query type (``@kind``) specified. The ``@kind`` property indicates how to interpret and display the results of the query analysis:
- Alert query metadata must contain ``@kind problem``.
- Path query metadata must contain ``@kind path-problem``.
@@ -73,7 +74,7 @@ When writing your own alert queries, you would typically import the standard lib
- JavaScript/TypeScript: ``javascript``
- Python: ``python``
There are also libraries containing commonly used predicates, types, and other modules associated with different analyses, including data flow, control flow, and taint-tracking. In order to calculate path graphs, path queries require you to import a data flow library into the query file. For more information, see :doc:`Creating path queries <path-queries>`.
There are also libraries containing commonly used predicates, types, and other modules associated with different analyses, including data flow, control flow, and taint-tracking. In order to calculate path graphs, path queries require you to import a data flow library into the query file. For more information, see ":doc:`Creating path queries <path-queries>`."
You can explore the contents of all the standard libraries in the `CodeQL library reference documentation <https://help.semmle.com/QL/ql-libraries.html>`__ or in the `GitHub repository <https://github.com/github/codeql>`__.
@@ -106,9 +107,9 @@ Select clauses for alert queries (``@kind problem``) consist of two 'columns', w
- ``element``: a code element that is identified by the query, which defines where the alert is displayed.
- ``string``: a message, which can also include links and placeholders, explaining why the alert was generated.
You can modify the alert message defined in the final column of the ``select`` statement to give more detail about the alert or path found by the query using links and placeholders. For further information, see :doc:`Defining the results of a query <select-statement>`.
You can modify the alert message defined in the final column of the ``select`` statement to give more detail about the alert or path found by the query using links and placeholders. For further information, see ":doc:`Defining the results of a query <select-statement>`."
Select clauses for path queries (``@kind path-problem``) are crafted to display both an alert and the source and sink of an associated path graph. For more information, see :doc:`Creating path queries <path-queries>`.
Select clauses for path queries (``@kind path-problem``) are crafted to display both an alert and the source and sink of an associated path graph. For more information, see ":doc:`Creating path queries <path-queries>`."
Viewing the standard CodeQL queries
***********************************
@@ -123,12 +124,13 @@ Contributing queries
Contributions to the standard queries and libraries are very welcome. For more information, see our `contributing guidelines <https://github.com/github/codeql/blob/master/CONTRIBUTING.md>`__.
If you are contributing a query to the open source GitHub repository, writing a custom query for LGTM, or using a custom query in an analysis with the CodeQL CLI, then you need to include extra metadata in your query to ensure that the query results are interpreted and displayed correctly. See the following topics for more information on query metadata:
- :doc:`Metadata for CodeQL queries <query-metadata>`
- ":doc:`Metadata for CodeQL queries <query-metadata>`"
- `Query metadata style guide on GitHub <https://github.com/github/codeql/blob/master/docs/query-metadata-style-guide.md>`__
Query contributions to the open source GitHub repository may also have an accompanying query help file to provide information about their purpose for other users. For more information on writing query help, see the `Query help style guide on GitHub <https://github.com/github/codeql/blob/master/docs/query-help-style-guide.md>`__ and the :doc:`Query help files <query-help>`.
Query contributions to the open source GitHub repository may also have an accompanying query help file to provide information about their purpose for other users. For more information on writing query help, see the `Query help style guide on GitHub <https://github.com/github/codeql/blob/master/docs/query-help-style-guide.md>`__ and the ":doc:`Query help files <query-help>`."
Query help files
****************
When you write a custom query, we also recommend that you write a query help file to explain the purpose of the query to other users. For more information, see the `Query help style guide <https://github.com/github/codeql/blob/master/docs/query-help-style-guide.md>`__ on GitHub, and the :doc:`Query help files <query-help>`.
When you write a custom query, we also recommend that you write a query help file to explain the purpose of the query to other users. For more information, see the `Query help style guide <https://github.com/github/codeql/blob/master/docs/query-help-style-guide.md>`__ on GitHub, and the ":doc:`Query help files <query-help>`."

View File

@@ -19,14 +19,15 @@ This topic provides information on how to structure a path query file so you can
The alerts generated by path queries are displayed by default in `LGTM <https://lgtm.com>`__ and included in the results generated using the `CodeQL CLI <https://help.semmle.com/codeql/codeql-cli.html>`__. You can also view the path explanations generated by your path query `directly in LGTM <https://lgtm.com/help/lgtm/exploring-data-flow-paths>`__ or in the CodeQL `extension for VS Code <https://help.semmle.com/codeql/codeql-for-vscode.html>`__.
To learn more about modeling data flow with CodeQL, see :doc:`Introduction to data flow <../intro-to-data-flow>`.
To learn more about modeling data flow with CodeQL, see ":doc:`About data flow analysis <../intro-to-data-flow>`."
For more language-specific information on analyzing data flow, see:
- :doc:`Analyzing data flow in C/C++ <../cpp/dataflow>`
- :doc:`Analyzing data flow in C# <../csharp/dataflow>`
- :doc:`Analyzing data flow in Java <../java/dataflow>`
- :doc:`Analyzing data flow in JavaScript/TypeScript <../javascript/dataflow>`
- :doc:`Analyzing data flow and tracking tainted data in Python <../python/taint-tracking>`
- ":doc:`Analyzing data flow in C/C++ <../cpp/dataflow>`"
- ":doc:`Analyzing data flow in C# <../csharp/dataflow>`"
- ":doc:`Analyzing data flow in Java <../java/dataflow>`"
- ":doc:`Analyzing data flow in JavaScript/TypeScript <../javascript/dataflow>`"
- ":doc:`Analyzing data flow and tracking tainted data in Python <../python/taint-tracking>`"
Path query examples
*******************
@@ -97,7 +98,7 @@ Path query metadata
*******************
Path query metadata must contain the property ``@kind path-problem``this ensures that query results are interpreted and displayed correctly.
The other metadata requirements depend on how you intend to run the query. For more information, see `Query metadata <https://help.semmle.com/QL/learn-ql/writing-queries/introduction-to-queries.html#query-metadata>`__.
The other metadata requirements depend on how you intend to run the query. For more information, see ":doc:`Metadata for CodeQL queries <query-metadata>`."
Generating path explanations
****************************
@@ -148,7 +149,8 @@ The configuration class is accessed by importing the data flow library. This cla
- ``isSource()`` defines where data may flow from.
- ``isSink()`` defines where data may flow to.
For further information on using the configuration class in your analysis see the sections on global data flow in :doc:`Analyzing data flow in C/C++ <../cpp/dataflow>` and :doc:`Analyzing data flow in C# <../csharp/dataflow>`.
For further information on using the configuration class in your analysis see the sections on global data flow in ":doc:`Analyzing data flow in C/C++ <../cpp/dataflow>`"
and ":doc:`Analyzing data flow in C# <../csharp/dataflow>`."
You can also create a configuration for different frameworks and environments by extending the ``Configuration`` class.
For further information, see `defining a class <https://help.semmle.com/QL/ql-handbook/types.html#defining-a-class>`__.
@@ -182,15 +184,16 @@ Select clauses for path queries consist of four 'columns', with the following st
select element, source, sink, string
The ``element`` and ``string`` columns represent the location of the alert and the alert message respectively, as explained in :doc:`Introduction to writing queries <introduction-to-queries>`. The second and third columns, ``source`` and ``sink``, are nodes on the path graph selected by the query.
Each result generated by your query is displayed at a single location in the same way as an alert query. Additionally, each result also has an associated path, which can be viewed in LGTM or in the CodeQL `extension for VS Code <https://help.semmle.com/codeql/codeql-for-vscode.html>`__.
The ``element`` and ``string`` columns represent the location of the alert and the alert message respectively, as explained in ":doc:`About CodeQL queries <introduction-to-queries>`." The second and third columns, ``source`` and ``sink``, are nodes on the path graph selected by the query.
Each result generated by your query is displayed at a single location in the same way as an alert query. Additionally, each result also has an associated path, which can be viewed in LGTM or in the `CodeQL extension for VS Code <https://help.semmle.com/codeql/codeql-for-vscode.html>`__.
The ``element`` that you select in the first column depends on the purpose of the query and the type of issue that it is designed to find. This is particularly important for security issues. For example, if you believe the ``source`` value to be globally invalid or malicious it may be best to display the alert at the ``source``. In contrast, you should consider displaying the alert at the ``sink`` if you believe it is the element that requires sanitization.
The alert message defined in the final column in the ``select`` statement can be developed to give more detail about the alert or path found by the query using links and placeholders. For more information, see :doc:`Defining the results of a query <select-statement>`.
The alert message defined in the final column in the ``select`` statement can be developed to give more detail about the alert or path found by the query using links and placeholders. For more information, see ":doc:`Defining the results of a query <select-statement>`."
Further reading
***************
- `Exploring data flow with path queries <https://help.semmle.com/codeql/codeql-for-vscode/procedures/exploring-paths.html>`__
- "`Exploring data flow with path queries <https://help.semmle.com/codeql/codeql-for-vscode/procedures/exploring-paths.html>`__"
- `CodeQL repository <https://github.com/github/codeql>`__

View File

@@ -49,11 +49,11 @@ Section-level elements are used to group the information in the help file into s
+====================+=========================================+========================+===============================================================================================================================================+
| ``example`` | None | Any block element | Demonstrate an example of code that violates the rule implemented by the query with guidance on how to fix it. Default heading. |
+--------------------+-----------------------------------------+------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------+
| ``fragment`` | None | Any block element | See :ref:`Query help inclusion <qhelp-inclusion>` below. No heading. |
| ``fragment`` | None | Any block element | See ":ref:`Query help inclusion <qhelp-inclusion>`" below. No heading. |
+--------------------+-----------------------------------------+------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------+
| ``hr`` | None | None | A horizontal rule. No heading. |
+--------------------+-----------------------------------------+------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------+
| ``include`` | ``src`` The query help file to include. | None | Include a query help file at the location of this element. See :ref:`Query help inclusion <qhelp-inclusion>` below. No heading. |
| ``include`` | ``src`` The query help file to include. | None | Include a query help file at the location of this element. See ":ref:`Query help inclusion <qhelp-inclusion>`" below. No heading. |
+--------------------+-----------------------------------------+------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------+
| ``overview`` | None | Any block element | Overview of the purpose of the query. Typically this is the first section in a query document. No heading. |
+--------------------+-----------------------------------------+------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------+
@@ -69,7 +69,7 @@ Section-level elements are used to group the information in the help file into s
Block elements
==============
The following elements are optional child elements of the ``section``, ``example``, ``fragment``, ``recommendation``, ``overview``, and ``semmleNotes`` elements.
The following elements are optional child elements of the ``section``, ``example``, ``fragment``, ``recommendation``, ``overview``, and ``semmleNotes`` elements.
.. table::
:widths: 7 20 10 25
@@ -143,7 +143,7 @@ Inline content is used to define the content for paragraphs, list items, table c
+------------+--------------------------------------+----------------+--------------------------------------------------------------------------------------------------+
| ``i`` | None | Inline content | Defines content that should be displayed as italics. |
+------------+--------------------------------------+----------------+--------------------------------------------------------------------------------------------------+
| ``img`` | | ``src``  | None | Display an image. See the description above in Block elements. |
| ``img`` | | ``src`` | None | Display an image. See the description above in Block elements. |
| | | ``alt`` | | |
| | | ``height`` | | |
| | | ``width`` | | |
@@ -152,7 +152,7 @@ Inline content is used to define the content for paragraphs, list items, table c
+------------+--------------------------------------+----------------+--------------------------------------------------------------------------------------------------+
| ``sub`` | None | Inline content | Defines content that should be rendered as subscript. |
+------------+--------------------------------------+----------------+--------------------------------------------------------------------------------------------------+
| ``sup`` | None | Inline content | Defines content that should be rendered as superscript. |
| ``sup`` | None | Inline content | Defines content that should be rendered as superscript. |
+------------+--------------------------------------+----------------+--------------------------------------------------------------------------------------------------+
| ``tt`` | None | Inline content | Defines content that should be displayed with a monospace font. |
+------------+--------------------------------------+----------------+--------------------------------------------------------------------------------------------------+
@@ -162,9 +162,9 @@ Inline content is used to define the content for paragraphs, list items, table c
Query help inclusion
====================
To reuse content between different help topics, you can store shared content in one query help file and then include it in a number of other query help files using the ``include`` element. The shared content can be stored either in the same directory as the including files, or in ``SEMMLE_DIST/docs/include``.
To reuse content between different help topics, you can store shared content in one query help file and then include it in a number of other query help files using the ``include`` element. The shared content can be stored either in the same directory as the including files, or in ``SEMMLE_DIST/docs/include``.
The ``include`` element can be used as a section or block element. The content of the query help file defined by the ``src`` attribute must contain elements that are appropriate to the location of the ``include`` element.
The ``include`` element can be used as a section or block element. The content of the query help file defined by the ``src`` attribute must contain elements that are appropriate to the location of the ``include`` element.
Section-level include elements
------------------------------
@@ -177,7 +177,7 @@ Section-level ``include`` elements can be located beneath the top-level ``qhelp`
<include src="XSS.qhelp" />
</qhelp>
In this example, the `XSS.qhelp <https://github.com/github/codeql/blob/master/csharp/ql/src/Security%20Features/CWE-079/XSS.qhelp>`__ file must conform to the standard for a full query help file as described above. That is, the ``qhelp`` element may only contain non-``fragment``, section-level elements.
In this example, the `XSS.qhelp <https://github.com/github/codeql/blob/master/csharp/ql/src/Security%20Features/CWE-079/XSS.qhelp>`__ file must conform to the standard for a full query help file as described above. That is, the ``qhelp`` element may only contain non-``fragment``, section-level elements.
Block-level include elements
----------------------------
@@ -193,7 +193,7 @@ Block-level ``include`` elements can be included beneath section-level elements.
...
</qhelp>
The included file, `ThreadUnsafeICryptoTransformOverview.qhelp <https://github.com/github/codeql/blob/master/csharp/ql/src/Likely%20Bugs/ThreadUnsafeICryptoTransformOverview.qhelp>`_, may only contain one or more ``fragment`` sections. For example:
The included file, `ThreadUnsafeICryptoTransformOverview.qhelp <https://github.com/github/codeql/blob/master/csharp/ql/src/Likely%20Bugs/ThreadUnsafeICryptoTransformOverview.qhelp>`_, may only contain one or more ``fragment`` sections. For example:
.. code-block:: xml

View File

@@ -8,13 +8,13 @@ About query metadata
Any query that is run as part of an analysis includes a number of properties, known as query metadata. Metadata is included at the top of each query file as the content of a `QLDoc <https://help.semmle.com/QL/ql-spec/qldoc.html>`__ comment.
This metadata tells LGTM and the CodeQL `extension for VS Code <https://help.semmle.com/codeql/codeql-for-vscode.html>`__ how to handle the query and display its results correctly.
It also gives other users information about what the query results mean. For further information on query metadata, see the `query metadata style guide <https://github.com/github/codeql/blob/master/docs/query-metadata-style-guide.md#metadata-area>`__ in our `open source repository <https://github.com/github/codeql>`__ on GitHub.
It also gives other users information about what the query results mean. For further information on query metadata, see the `query metadata style guide <https://github.com/github/codeql/blob/master/docs/query-metadata-style-guide.md>`__ in our `open source repository <https://github.com/github/codeql>`__ on GitHub.
.. pull-quote::
Note
The exact metadata requirement depends on how you are going to run your query. For more information, see the section on query metadata in :doc:`About CodeQL queries <introduction-to-queries>`.
The exact metadata requirement depends on how you are going to run your query. For more information, see the section on query metadata in ":doc:`About CodeQL queries <introduction-to-queries>`."
Metadata properties
-------------------
@@ -28,13 +28,13 @@ The following properties are supported by all query files:
+-----------------------+---------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| ``@id`` | ``<text>`` | A sequence of words composed of lowercase letters or digits, delimited by ``/`` or ``-``, identifying and classifying the query. Each query must have a **unique** ID. To ensure this, it may be helpful to use a fixed structure for each ID. For example, the standard LGTM queries have the following format: ``<language>/<brief-description>``. |
+-----------------------+---------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| ``@kind`` | | ``problem`` | Identifies the query is an alert (``@kind problem``) or a path (``@kind path-problem``). For further information on these query types, see :doc:`About CodeQL queries <introduction-to-queries>`. |
| ``@kind`` | | ``problem`` | Identifies the query is an alert (``@kind problem``) or a path (``@kind path-problem``). For further information on these query types, see ":doc:`About CodeQL queries <introduction-to-queries>`." |
| | | ``path-problem`` | |
+-----------------------+---------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| ``@name`` | ``<text>`` | A statement that defines the label of the query. The name is written in plain text, and uses single quotes (``'``) to enclose code elements. |
+-----------------------+---------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| ``@tags`` | | ``correctness`` | These tags group queries together in broad categories to make it easier to search for them and identify them. In addition to the common tags listed here, there are also a number of more specific categories. For more information, see the |
| | | ``maintainability`` | `Query metadata style guide <https://github.com/github/codeql/blob/master/docs/query-metadata-style-guide.md#query-tags-tags>`__. |
| | | ``maintainability`` | `Query metadata style guide <https://github.com/github/codeql/blob/master/docs/query-metadata-style-guide.md>`__. |
| | | ``readability`` | |
| | | ``security`` | |
+-----------------------+---------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

View File

@@ -15,7 +15,7 @@ This topic explains how to write your select statement to generate helpful analy
Overview
--------
Alert queries must have the property ``@kind problem`` defined in their metadata. For further information, see :doc:`Metadata for CodeQL queries <query-metadata>`.
Alert queries must have the property ``@kind problem`` defined in their metadata. For further information, see ":doc:`Metadata for CodeQL queries <query-metadata>`."
In their most basic form, the ``select`` statement must select two 'columns':
- **Element**—a code element that's identified by the query. This defines the location of the alert.
@@ -27,7 +27,7 @@ If you look at some of the LGTM queries, you'll see that they can select extra e
Note
An in-depth discussion of ``select`` statements for path queries is not included in this topic. However, you can develop the string column of the ``select`` statement in the same way as for alert queries. For more specific information about path queries, see :doc:`Creating path queries <path-queries>`.
An in-depth discussion of ``select`` statements for path queries is not included in this topic. However, you can develop the string column of the ``select`` statement in the same way as for alert queries. For more specific information about path queries, see ":doc:`Creating path queries <path-queries>`."
Developing a select statement
-----------------------------

View File

@@ -1,2 +1,2 @@
- `QL language reference <https://help.semmle.com/QL/ql-handbook>`__
- `CodeQL tools <https://help.semmle.com/codeql/codeql-tools.html>`__
- "`QL language reference <https://help.semmle.com/QL/ql-handbook>`__"
- "`CodeQL tools <https://help.semmle.com/codeql/codeql-tools.html>`__"