Docs: Update terminology

A more in-depth attempt at changing terminology for GHU.
I've only updated the non-language specific topics so far.
This commit is contained in:
Shati Patel
2019-10-23 14:54:02 +01:00
parent 41969a3d92
commit 60226801aa
29 changed files with 179 additions and 163 deletions

View File

@@ -1,11 +1,11 @@
About QL
========
This section is aimed at users with a background in general purpose programming as well as in databases. For a basic introduction and information on how to get started, see :doc:`Introduction to the QL language <introduction-to-ql>` and :doc:`Learning QL <../index>`.
This section is aimed at users with a background in general purpose programming as well as in databases. For a basic introduction and information on how to get started, see :doc:`Introduction to QL <introduction-to-ql>` and :doc:`Learning CodeQL <../index>`.
QL is a declarative, object-oriented query language that is optimized to enable efficient analysis of hierarchical data structures, in particular, databases representing software artifacts.
The queries and metrics used in LGTM are implemented using QL. This ensures that they can be extended or revised easily to keep up with changes in definitions of best coding practice. We continually improve existing queries as we work towards the ultimate goal of 100% precision.
The queries and metrics used in LGTM are implemented using CodeQL, which uses QL to analyze code. This ensures that they can be extended or revised easily to keep up with changes in definitions of best coding practice. We continually improve existing queries as we work towards the ultimate goal of 100% precision.
You can write queries to identify security vulnerabilities, find coding errors and bugs, or find code that breaks your team's guidelines for best practice. You can also create customized versions of the default queries to accommodate a new framework.

View File

@@ -4,7 +4,7 @@ Semantics of abstract classes
Concrete classes
----------------
Concrete QL classes, as described in the QL language handbook topic on `Classes <https://help.semmle.com/QL/ql-handbook/types.html#classes>`__, lend themselves well to top-down modeling. We start from general superclasses representing large sets of values, and carve out individual subclasses representing more restricted sets of values.
Concrete classes, as described in the QL language handbook topic on `Classes <https://help.semmle.com/QL/ql-handbook/types.html#classes>`__, lend themselves well to top-down modeling. We start from general superclasses representing large sets of values, and carve out individual subclasses representing more restricted sets of values.
A classic example where this approach is useful is when modeling ASTs (Abstract Syntax Trees): the node types of an AST form a natural inheritance hierarchy, where, for example, there is a class ``Expr`` representing all expression nodes, with many different subclasses for different categories of expressions. There might be a class ``ArithmeticExpr`` representing arithmetic expressions, which in turn could have subclasses ``AddExpr`` and ``SubExpr``.
@@ -57,7 +57,7 @@ Like a concrete class, an abstract class has one or more superclasses and a char
Example
~~~~~~~
The following example is taken from the standard QL library for Java:
The following example is taken from the CodeQL library for Java:
.. code-block:: ql

View File

@@ -8,7 +8,7 @@ Advanced QL
./*
Topics on advanced uses of QL. These topics assume that you are familiar with the QL language and the basics of query writing.
Topics on advanced uses of QL. These topics assume that you are familiar with QL and the basics of query writing.
- :doc:`Semantics of abstract classes <abstract-classes>`
- :doc:`Choosing appropriate ways to constrain types <constraining-types>`

View File

@@ -8,7 +8,7 @@ Type constraint methods
Note
The examples below use the Java QL library. All QL libraries support using these methods to constrain variables, the only difference is in the names of the classes used.
The examples below use the CodeQL library for Java. All CodeQL libraries support using these methods to constrain variables, the only difference is in the names of the classes used.
There are several ways of imposing type constraints on variables:

View File

@@ -1,14 +1,14 @@
Determining the most specific types of a variable
=================================================
It is sometimes useful to be able to determine what QL types an entity has -- especially when you are unfamiliar with the QL library used by a query. To help with this, QL provides a predicate called ``getAQlClass()``, which returns the most specific QL types of the entity that it is called on.
It is sometimes useful to be able to determine what types an entity has -- especially when you are unfamiliar with the library used by a query. To help with this, there is a predicate called ``getAQlClass()``, which returns the most specific QL types of the entity that it is called on.
This can be useful when you are not sure of the most precise class of a value. Discovering a more precise class can allow you to cast to it and use predicates that are not available on the more general class.
Example
-------
If you were working with a Java snapshot database, you might use ``getAQlClass()`` on every ``Expr`` in a callable called ``c``:
If you were working with a Java database, you might use ``getAQlClass()`` on every ``Expr`` in a callable called ``c``:
**Java example**
@@ -23,6 +23,6 @@ If you were working with a Java snapshot database, you might use ``getAQlClass()
and e.getEnclosingCallable() = c
select e, e.getAQlClass()
The result of this query is a list of the most specific types of every ``Expr`` in that function. You will see multiple results for some expressions because that expression is represented by more than one QL type.
The result of this query is a list of the most specific types of every ``Expr`` in that function. You will see multiple results for some expressions because that expression is represented by more than one type.
For example, ``StringLiteral``\ s like ``"Hello"`` in Java belong to both the ``StringLiteral`` QL class (a specialization of the ``Literal`` QL class) and the ``CompileTimeConstantExpr`` QL class. So any instances of ``StringLiteral``\ s in the results will produce more than one result, one for each of the QL classes to which they belong.
For example, ``StringLiteral``\ s like ``"Hello"`` in Java belong to both the ``StringLiteral`` class (a specialization of the ``Literal`` class) and the ``CompileTimeConstantExpr`` class. So any instances of ``StringLiteral``\ s in the results will produce more than one result, one for each of the classes to which they belong.

View File

@@ -6,7 +6,7 @@ The two expressions:
#. ``a() != b()``
#. ``not(a() = b())``
look equivalent - so much so that inexperienced (and even experienced) QL programmers have been known to rewrite one as the other. However, they are not equivalent due to the quantifiers involved.
look equivalent - so much so that inexperienced (and even experienced) programmers have been known to rewrite one as the other. However, they are not equivalent due to the quantifiers involved.
Thinking of ``a()`` and ``b()`` as sets of values, the first expression says that there is a pair of values (one from each side of the inequality) which are different.

View File

@@ -1,7 +1,7 @@
Monotonic aggregates in QL
==========================
In addition to standard QL aggregates, QL also supports *monotonic* aggregates. These are a slightly different way of computing aggregates which have some advantages, notably the ability to be used recursively, which normal aggregates do not have. You can enable them in a scope by adding the \ ``language[monotonicAggregates]`` pragma on a predicate, class, or module.
In addition to standard aggregates, QL also supports *monotonic* aggregates. These are a slightly different way of computing aggregates which have some advantages, notably the ability to be used recursively, which normal aggregates do not have. You can enable them in a scope by adding the \ ``language[monotonicAggregates]`` pragma on a predicate, class, or module.
Syntax
------

View File

@@ -77,4 +77,4 @@ What next?
- Help the villagers track down another criminal in the :doc:`next tutorial <fire-1>`.
- Find out more about the concepts you discovered in this tutorial in the `QL language handbook <https://help.semmle.com/QL/ql-handbook/index.html>`__.
- Explore the libraries that help you get data about code in :doc:`Learning QL <../../index>`.
- Explore the libraries that help you get data about code in :doc:`Learning CodeQL <../../index>`.

View File

@@ -40,4 +40,4 @@ What next?
- Find out who will be the new ruler of the village in the :doc:`next tutorial <heir>`.
- Learn more about predicates and classes in the `QL language handbook <https://help.semmle.com/QL/ql-handbook/index.html>`__.
- Explore the libraries that help you get data about code in :doc:`Learning QL <../../index>`.
- Explore the libraries that help you get data about code in :doc:`Learning CodeQL <../../index>`.

View File

@@ -161,4 +161,4 @@ What next?
- Learn more about recursion in the `QL language handbook <https://help.semmle.com/QL/ql-handbook/index.html>`__.
- Put your QL skills to the test and solve the :doc:`River crossing puzzle <../ql-etudes/river-crossing>`.
- Start using QL to analyze projects. See :doc:`Learning QL <../../index>` for a summary of the available languages and resources.
- Start using QL to analyze projects. See :doc:`Learning CodeQL <../../index>` for a summary of the available languages and resources.

View File

@@ -7,18 +7,21 @@ QL detective tutorials
./*
Welcome to QL! These tutorials are aimed at complete beginners. They teach you how to write QL queries and introduce you to key logic concepts along the way.
Welcome to the detective tutorials! These are aimed at complete beginners who would like to learn the basics of QL,
before analyzing code with CodeQL.
The tutorials teach you how to write queries and introduce you to key logic concepts along the way.
We recommend you first read the :doc:`Introduction to the QL language <../introduction-to-ql>` page for a basic description of QL.
We recommend you first read the :doc:`Introduction to QL <../introduction-to-ql>` page for a description of the language and
some simple examples.
Currently the following tutorials are available:
Currently the following detective tutorials are available:
- :doc:`Find the thief <find-thief-1>` - a three part mystery that introduces logical connectives, quantifiers and aggregates
- :doc:`Find the thief <find-thief-1>` - a three part mystery that introduces logical connectives, quantifiers, and aggregates
- :doc:`Catch the fire starter <fire-1>` - an intriguing search that introduces predicates and classes
- :doc:`Crown the rightful heir <heir>` - a detective puzzle that introduces recursion
Further resources
-----------------
- For a summary of available learning resources, see :doc:`Learning QL <../../index>`.
- For a summary of available learning resources, see :doc:`Learning CodeQL <../../index>`.
- For an overview of the important concepts in QL, see the `QL language handbook <https://help.semmle.com/QL/ql-handbook/index.html>`__.

View File

@@ -0,0 +1,33 @@
What's in a CodeQL database?
============================
A CodeQL database contains a variety of data related to a particular code base at a particular point in time. For details of how the database is generated see `Database generation <https://lgtm.com/help/lgtm/generate-database>`__.
The database contains a full, hierarchical representation of the program defined by the code base. The database schema varies according to the language analyzed. The schema provides an interface between the initial lexical analysis during the extraction process, and the actual complex analysis using CodeQL. When the source code languages being analyzed change (such as Java 7 evolving into Java 8), this interface between the analysis phases can also change.
For each language, a CodeQL library defines classes to provide a layer of abstraction over the database tables. This provides an object-oriented view of the data which makes it easier to write queries. This is easiest to explain using an example.
Example
-------
For a Java program, two key tables are:
- The ``expressions`` table containing a row for every single expression in the source code that was analyzed during the build process.
- The ``statements`` table containing a row for every single statement in the source code that was analyzed during the build process.
The CodeQL library defines classes to provide a layer of abstraction over each of these tables (and the related auxiliary tables): ``Expr`` and ``Stmt``.
Most classes in the CodeQL library are similar: they are abstractions over one or more database tables. Looking at one of the CodeQL libraries illustrates this:
.. code-block:: ql
class Expr extends StmtParent, @expr {
...
/** the location of this expression */
Location getLocation() { exprs(this,_,_,result) }
...
}
The ``Expr`` class, shown here, extends from the database type ``@expr``. Member predicates of the ``Expr`` class are implemented in terms of the database-provided ``exprs`` table.

View File

@@ -1,14 +1,13 @@
Learning QL
###########
Learning CodeQL
###############
CodeQL is the code analysis platform used by security researchers to automate `variant analysis <https://semmle.com/variant-analysis>`__.
You can use CodeQL queries to explore code and quickly find variants of security vulnerabilities and bugs.
These queries are easy to write and sharevisit the topics below and `our open source repository on GitHub <https://github.com/Semmle/ql>`__ to learn more.
You can also try out CodeQL in the `query console <https://lgtm.com/query>`__ on `LGTM.com <https://lgtm.com>`__.
Here, you can query open source projects directly, without having to download CodeQL databases and libraries.
`QL <https://semmle.com/ql>`__ is the query language used in Semmle's `variant analysis <https://semmle.com/variant-analysis>`__ engine.
You can use queries written in QL to explore code and quickly find variants of security vulnerabilities and bugs.
The QL language is also part of the technology behind `LGTM <https://lgtm.com>`__, Semmle's analysis platform that combines deep semantic code search with data science insights to help developers ship secure code.
QL queries are easy to write and sharevisit the topics below and `our open source repository on GitHub <https://github.com/Semmle/ql>`__ to learn more.
You can also try out QL in the `query console <https://lgtm.com/query>`__ on `LGTM.com <https://lgtm.com>`__.
Here, you can write QL code to query open source projects directly, without having to download snapshots and libraries.
CodeQL is based on a powerful query language called QL. The following topics help you understand QL in general, as well as how to use it when analyzing code with CodeQL.
.. _getting-started:
@@ -25,10 +24,10 @@ If you are new to QL, start by looking at the following topics:
beginner/ql-tutorials
ql-etudes/river-crossing
QL training and variant analysis examples
******************************************
CodeQL training and variant analysis examples
*********************************************
To start learning how to use QL in variant analysis for a specific language, see:
To start learning how to use CodeQL for variant analysis for code written in a specific language, see:
.. toctree::
:maxdepth: -1
@@ -37,8 +36,8 @@ To start learning how to use QL in variant analysis for a specific language, see
.. _writing-ql-queries:
Writing QL queries
******************
Writing CodeQL queries
**********************
To learn more about writing your own queries, see:
@@ -48,7 +47,7 @@ To learn more about writing your own queries, see:
writing-queries/writing-queries
For more information on writing QL to query code written in a specific language see:
For more information on using CodeQL to query code written in a specific language, see:
.. toctree::
:maxdepth: 2
@@ -77,10 +76,10 @@ For more technical information see:
Reference topics
****************
For a more comprehensive guide to QL see the following reference topics:
For a more comprehensive guide to the query language itself, see the following reference topics:
- `QL language handbook <https://help.semmle.com/QL/ql-handbook/index.html>`__—a description of important concepts in QL
- `QL language specification <https://help.semmle.com/QL/ql-spec/language.html>`__—a formal specification of the QL language.
- `QL language handbook <https://help.semmle.com/QL/ql-handbook/index.html>`__—a description of important concepts in QL.
- `QL language specification <https://help.semmle.com/QL/ql-spec/language.html>`__—a formal specification of QL.
Search
******

View File

@@ -1,15 +1,15 @@
Introduction to data flow analysis in QL
########################################
Introduction to data flow analysis with CodeQL
##############################################
Overview
********
Data flow analysis computes the possible values that a variable can hold at various points in a program, determining how those values propagate through the program and where they are used.
Many of Semmle's built-in security queries implement data flow analysis, which can highlight the fate of potentially malicious or insecure data that can cause vulnerabilities in your code base.
Many CodeQL security queries implement data flow analysis, which can highlight the fate of potentially malicious or insecure data that can cause vulnerabilities in your code base.
These queries help you understand if data is used in an insecure way, whether dangerous arguments are passed to functions, or whether sensitive data can leak.
As well as highlighting potential security issues, you can also use data flow analysis to understand other aspects of how a program behaves, by finding, for example, uses of unititialized variables and resource leaks.
As well as highlighting potential security issues, you can also use data flow analysis to understand other aspects of how a program behaves, by finding, for example, uses of uninitialized variables and resource leaks.
The following sections provide a brief introduction to data flow analysis in QL.
The following sections provide a brief introduction to data flow analysis with CodeQL.
See the following tutorials for more information about analyzing data flow in specific languages:
@@ -30,7 +30,7 @@ See the following tutorials for more information about analyzing data flow in sp
Data flow graph
***************
The QL data flow libraries implement data flow analysis on a program or function by modeling its data flow graph.
The CodeQL data flow libraries implement data flow analysis on a program or function by modeling its data flow graph.
Unlike the `abstract syntax tree <https://en.wikipedia.org/wiki/Abstract_syntax_tree>`__, the
data flow graph does not reflect the syntactic structure of the program, but models the way data flows through the program at runtime. Nodes in the abstract syntax tree
represent syntactic elements such as statements or expressions. Nodes in the data flow graph, on the other hand, represent semantic elements that carry values at runtime.
@@ -58,18 +58,18 @@ Computing an accurate and complete data flow graph presents several challenges:
- Aliasing between variables can result in a single write changing the value that multiple pointers point to.
- The data flow graph can be very large and slow to compute.
To overcome these potential problems, two kinds of data flow are modeled in the QL libraries:
To overcome these potential problems, two kinds of data flow are modeled in the CodeQL libraries:
- Local data flow, concerning the data flow within a single function. When reasoning about local, you only considers edges between data flow nodes belonging to the same function.It is generally sufficiently fast, efficient and precise for many queries, and it is usually possible to compute the local data flow for all functions in a snapshot.
- Local data flow, concerning the data flow within a single function. When reasoning about local, you only considers edges between data flow nodes belonging to the same function.It is generally sufficiently fast, efficient and precise for many queries, and it is usually possible to compute the local data flow for all functions in a CodeQL database.
- Global data flow, effectively considers the data flow within an entire program, by calculating data flow between functions and through object properties. Computing global data flow is typically more time and energy intensive than local data flow, therefore queries should be refined to look for more specific sources and sinks.
Many of the built-in queries included in the latest Semmle release contain examples of both local and global data flow analysis. See `the built-in queries <https://help.semmle.com/wiki/display/QL/Built-in+queries>`__ for details.
Many CodeQL queries contain examples of both local and global data flow analysis. See `the built-in queries <https://help.semmle.com/wiki/display/QL/Built-in+queries>`__ for details.
Normal data flow vs taint tracking
**********************************
In the QL standard libraries, we make a distinction between 'normal' data flow and taint tracking.
In the standard CodeQL libraries, we make a distinction between 'normal' data flow and taint tracking.
The normal data flow libraries are used to analyze the information flow in which data values are preserved at each step.
For example, if you are tracking an insecure object ``x`` (which might be some untrusted or potentially malicious data), a step in the program may 'change' its value. So, in a simple process such as ``y = x + 1``, a normal data flow analysis will highlight the use of ``x``, but not ``y``.
@@ -81,5 +81,5 @@ These flow steps are modeled in the taint-tracking library using predicates that
What next?
**********
- Search for ``DataFlow`` and ``TaintTracking`` in the `QL standard libraries <https://help.semmle.com/wiki/display/QL/QL+standard+libraries>`__ to learn more about the technical implementation of data flow analysis in QL for specific programming languages.
- Visit `Learning QL <https://help.semmle.com/QL/learn-ql/>`__ to find language-specific QL tutorials on data flow and other topics.
- Search for ``DataFlow`` and ``TaintTracking`` in the `standard CodeQL libraries <https://help.semmle.com/wiki/display/QL/QL+standard+libraries>`__ to learn more about the technical implementation of data flow analysis for specific programming languages.
- Visit `Learning CodeQL <https://help.semmle.com/QL/learn-ql/>`__ to find language-specific tutorials on data flow and other topics.

View File

@@ -1,18 +1,22 @@
Introduction to the QL language
===============================
Introduction to QL
==================
QL is a powerful query language that is used to analyze code. Queries written in QL can be used to find errors and uncover variants of important security vulnerabilities. Visit Semmle's `security research page <https://lgtm.com/security>`__ to read about examples of vulnerabilities that we have recently found in open source projects using QL queries.
QL is a powerful query language that underlies CodeQL, which is used to analyze code.
Queries written with CodeQL can find errors and uncover variants of important security vulnerabilities.
Visit Semmle's `security research page <https://lgtm.com/security>`__ to read about examples of vulnerabilities that we have recently found in open source projects.
Before diving into code analysis with CodeQL, it can be helpful to learn about the underlying language more generally.
QL is a logic programming language, so it is built up of logical formulas. QL uses common logical connectives (such as ``and``, ``or``, and ``not``), quantifiers (such as ``forall`` and ``exists``), and other important logical concepts such as predicates.
QL also supports recursion and aggregates. This allows you to write complex recursive queries using simple QL syntax and directly use aggregates such as ``count``, ``sum`` and ``average``.
QL also supports recursion and aggregates. This allows you to write complex recursive queries using simple QL syntax and directly use aggregates such as ``count``, ``sum``, and ``average``.
Basic syntax
------------
The basic syntax will look familiar to anyone who has used SQL, but it is used somewhat differently.
The basic syntax of QL will look familiar to anyone who has used SQL, but it is used somewhat differently.
A QL query is defined by a **select** clause, which specifies what the result of the query should be. You can try out the examples and exercises in this topic directly in LGTM. Open the `query console <https://lgtm.com/query>`__. Before you can run a query, you need to select a language and project to query (for these logic examples, any language and project will do).
A query is defined by a **select** clause, which specifies what the result of the query should be. You can try out the examples and exercises in this topic directly in LGTM. Open the `query console <https://lgtm.com/query>`__. Before you can run a query, you need to select a language and project to query (for these logic examples, any language and project will do).
Once you have selected a language, the query console is populated with the query:
@@ -110,9 +114,12 @@ To simplify the query, we can introduce a class ``SmallInt`` representing the in
`See this in the query console <https://lgtm.com/query/2101340747/>`__
Now that you've seen some general examples, let's use QL queries to analyze projects. In particular, LGTM generates a database representing the code and then QL is used to query this database. See `Database generation <https://lgtm.com/help/lgtm/generate-database>`__ for more details on how the database is built.
Now that you've seen some general examples, let's use the CodeQL libraries to analyze projects.
In particular, LGTM generates a database representing the code and then CodeQL is used to query this database. See `Database generation <https://lgtm.com/help/lgtm/generate-database>`__ for more details on how the database is built.
The previous exercises just used the primitive types built in to QL. Although we chose a project to query, they did not use the project-specific database. The following example queries *do* use these databases and give you an idea of what QL can be used for. There are more details about how to write QL `below <#learning-ql>`__, so don't worry if you don't fully understand these examples yet!
.. XX: Perhaps a link to the "CodeQL libraries for X"?
The previous exercises just used the primitive types built in to QL. Although we chose a project to query, they did not use the project-specific database. The following example queries *do* use these databases and give you an idea of what CodeQL can be used for. There are more details about how to use CodeQL `below <#learning-ql>`__, so don't worry if you don't fully understand these examples yet!
Python
~~~~~~
@@ -153,9 +160,9 @@ Java
`See this in the query console <https://lgtm.com/query/2098670762/>`__. The ``from`` clause defines a variable ``p`` representing a parameter. The ``where`` clause finds unused parameters by limiting the parameters ``p`` to those which are not accessed. Finally, the ``select`` clause lists these parameters.
Learning QL
-----------
Learning CodeQL
---------------
- To find out more about how to write your own QL queries, try working through the :doc:`QL detective tutorials <beginner/ql-tutorials>`.
- For an overview of the other available resources, see :doc:`Learning QL <../index>`.
- For a more technical description of QL, see :doc:`About QL <about-ql>`.
- To find out more about how to write your own queries, try working through the :doc:`QL detective tutorials <beginner/ql-tutorials>`.
- For an overview of the other available resources, see :doc:`Learning CodeQL <../index>`.
- For a more technical description of the underlying language, see :doc:`About QL <about-ql>`.

View File

@@ -1,6 +1,8 @@
Locations and strings for QL entities
=====================================
.. Not sure how much of this topic needs to change, and what the title should be
Providing locations
-------------------
@@ -55,18 +57,18 @@ By convention, the location of an entire file may also be denoted by a ``file://
Other types of URL
^^^^^^^^^^^^^^^^^^
The following, less-common types of URL are valid QL but are not supported by LGTM and will be omitted from any results:
The following, less-common types of URL are valid but are not supported by LGTM and will be omitted from any results:
- **HTTP URLs** are supported in some client applications. For an example, see the code snippet above.
- **Folder URLs** can be useful, for example to provide folder-level metrics. They may use a file URL, for example ``file:///opt/src:0:0:0:0``, but they may also start with a scheme of ``folder://``, and no trailing numbers, for example ``folder:///opt/src``.
- **Relative file URLs** are like normal file URLs, but start with the scheme ``relative://``. They are typically only meaningful in the context of a particular snapshot, and are taken to be implicitly prefixed by the snapshot's source location. Note that, in particular, the relative URL of a file will stay constant regardless of where the snapshot is analyzed. It is often most convenient to produce these URLs as input when importing external information; selecting one from a QL class would be unusual, and client applications may not handle it appropriately.
- **Relative file URLs** are like normal file URLs, but start with the scheme ``relative://``. They are typically only meaningful in the context of a particular database, and are taken to be implicitly prefixed by the database's source location. Note that, in particular, the relative URL of a file will stay constant regardless of where the database is analyzed. It is often most convenient to produce these URLs as input when importing external information; selecting one from a QL class would be unusual, and client applications may not handle it appropriately.
Providing location information
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
If no ``getURL()`` member predicate is defined, a QL class is checked for the presence of a member predicate called ``hasLocationInfo(..)``. This can be understood as a convenient way of providing file URLs (see above) without constructing the long URL string in QL. ``hasLocationInfo(..)`` should be a predicate, its first column must be ``string``-typed (it corresponds to the "path" portion of a file URL), and it must have an additional 3 or 4 ``int``-typed columns, which are interpreted like a trailing group of three or four numbers on a file URL.
For example, let us imagine that the locations for methods provided by the extractor extend from the first character of the method name to the closing curly brace of the method body, and we want to "fix" them in QL to ensure that only the method name is selected. The following code shows two ways of achieving this:
For example, let us imagine that the locations for methods provided by the extractor extend from the first character of the method name to the closing curly brace of the method body, and we want to "fix" them to ensure that only the method name is selected. The following code shows two ways of achieving this:
.. code-block:: ql
@@ -104,7 +106,7 @@ Using extracted location information
Finally, if the above two predicates fail, client applications will attempt to call a predicate called ``getLocation()`` with no parameters, and try to apply one of the above two predicates to the result. This allows certain locations to be put into the database, assigned identifiers, and picked up.
By convention, the return value of the ``getLocation()`` predicate should be a QL class called ``Location``, and it should define a version of ``hasLocationInfo(..)`` (or ``getURL()``, though the former is preferable). If the ``Location`` class does not provide either of these member predicates, then no location information will be available.
By convention, the return value of the ``getLocation()`` predicate should be a class called ``Location``, and it should define a version of ``hasLocationInfo(..)`` (or ``getURL()``, though the former is preferable). If the ``Location`` class does not provide either of these member predicates, then no location information will be available.
The ``toString()`` predicate
----------------------------

View File

@@ -231,5 +231,5 @@ For more information on writing QL, see:
- `QL language handbook <https://help.semmle.com/QL/ql-handbook/index.html>`__ - an introduction to the concepts of QL.
- :doc:`Learning QL <../../index>` - an overview of the resources for learning how to write your own QL queries.
- `Database generation <https://lgtm.com/help/lgtm/generate-database>`__ - an overview of the process that creates a snapshot from source code.
- :doc:`What's in a snapshot? <../snapshot>` - a description of the snapshot database.
- `Database generation <https://lgtm.com/help/lgtm/generate-database>`__ - an overview of the process that creates a database from source code.
- :doc:`What's in a CodeQL database? <../database>` - a description of the CodeQL database.

View File

@@ -1,7 +1,7 @@
River crossing puzzle
#####################
The aim of this tutorial is to write a QL query that finds a solution to the following classical logic puzzle:
The aim of this tutorial is to write a query that finds a solution to the following classical logic puzzle:
.. pull-quote::
@@ -243,7 +243,7 @@ You could tweak the predicate and the select clause to make the solution clearer
Alternative solutions
---------------------
Here are some more example QL queries that solve the river crossing puzzle:
Here are some more example queries that solve the river crossing puzzle:
#. This query uses a modified ``path`` variable to describe the resulting path in
more detail.

View File

@@ -1,33 +0,0 @@
What's in a snapshot database?
===============================
A snapshot database contains a variety of data related to a particular code base at a particular point in time. For details of how the database is generated see `Database generation <https://lgtm.com/help/lgtm/generate-database>`__.
The database contains a full, hierarchical representation of the program defined by the code base. The database schema varies according to the language analyzed. The schema provides an interface between the initial lexical analysis during the extraction process, and the actual complex analysis using QL. When the source code languages being analyzed change (such as Java 7 evolving into Java 8), this interface between the analysis phases can also change.
For each language, a QL library defines classes to provide a layer of abstraction over the database tables. This provides an object-oriented view of the data which makes it easier to write queries. This is easiest to explain using an example.
Example
-------
For a Java program, two key tables are:
- The ``expressions`` table containing a row for every single expression in the source code that was analyzed during the build process.
- The ``statements`` table containing a row for every single statement in the source code that was analyzed during the build process.
The QL library defines classes to provide a layer of abstraction over each of these tables (and the related auxiliary tables): ``Expr`` and ``Stmt``.
Most classes in the QL library are similar: they are abstractions over one or more database tables. Looking at one of the QL libraries illustrates this:
.. code-block:: ql
class Expr extends StmtParent, @expr {
...
/** the location of this expression */
Location getLocation() { exprs(this,_,_,result) }
...
}
The ``Expr`` class, shown here, extends from the database type ``@expr``. Member predicates of the ``Expr`` class are implemented in terms of the database-provided ``exprs`` table.

View File

@@ -4,6 +4,6 @@ Technical information
.. toctree::
:hidden:
snapshot
database
- :doc:`What's in a snapshot database <snapshot>`
- :doc:`What's in a CodeQL database? <database>`

View File

@@ -4,13 +4,15 @@ Introduction to query files
Overview
********
Queries are programs written in QL. The QL queries included in the Semmle tools are designed to highlight issues related to the security, correctness, maintainability, and readability of a code base. You can also write custom queries to find specific issues relevant to your own project. Three important types of query are:
Queries are programs written with CodeQL. They are designed to highlight issues related to the security, correctness, maintainability, and readability of a code base. You can also write custom queries to find specific issues relevant to your own project. Three important types of query are:
- **Alert queries**: queries that highlight issues in specific locations in your code.
- **Path queries**: queries that describe the flow of information between a source and a sink in your code.
- **Metric queries**: queries that compute statistics for your code.
You can add custom queries to `custom query packs <https://lgtm.com/help/lgtm/about-queries#what-are-query-packs>`__ to analyze your projects in `LGTM <https://lgtm.com>`__, use them to analyze a project using the `QL command-line tools <https://help.semmle.com/wiki/display/SD/QL+command-line+tools>`__, or you can contribute to Semmle's built-in queries in our `open source repository on GitHub <https://github.com/semmle/ql>`__.
You can add custom queries to `custom query packs <https://lgtm.com/help/lgtm/about-queries#what-are-query-packs>`__ to analyze your projects in `LGTM <https://lgtm.com>`__, use them to analyze a project using the `command-line tools <https://help.semmle.com/wiki/display/SD/QL+command-line+tools>`__, or you can contribute to the standard CodeQL queries in our `open source repository on GitHub <https://github.com/semmle/ql>`__.
.. TODO: Change "command-line tools" to a link to the CodeQL CLI? Similarly, change "QL for Eclipse".
.. pull-quote::
@@ -22,13 +24,13 @@ You can add custom queries to `custom query packs <https://lgtm.com/help/lgtm/ab
This topic is a basic introduction to structuring query files. You can find further information on writing queries for specific programming languages `here <https://help.semmle.com/QL/learn-ql/>`__, and detailed technical information about QL in the `QL language handbook <https://help.semmle.com/QL/ql-handbook/index.html>`__ and the `QL language specification <https://help.semmle.com/QL/ql-spec/language.html>`__.
For information on how to format QL code when contributing queries to the GitHub repository, see the `QL style guide <https://github.com/Semmle/ql/blob/master/docs/ql-style-guide.md>`__.
For information on how to format your code when contributing queries to the GitHub repository, see the `QL style guide <https://github.com/Semmle/ql/blob/master/docs/ql-style-guide.md>`__.
Basic query structure
*********************
`Queries <https://help.semmle.com/QL/ql-handbook/queries.html>`__ written in QL have the file extension ``.ql``, and contain a ``select`` clause. Many of the built-in Semmle queries include additional optional information, and have the following structure::
`Queries <https://help.semmle.com/QL/ql-handbook/queries.html>`__ written with CodeQL have the file extension ``.ql``, and contain a ``select`` clause. Many of the existing CodeQL queries include additional optional information, and have the following structure::
/**
*
@@ -36,9 +38,9 @@ Basic query structure
*
*/
import /* ... QL libraries or modules ... */
import /* ... CodeQL libraries or modules ... */
/* ... Optional, define QL classes and predicates ... */
/* ... Optional, define CodeQL classes and predicates ... */
from /* ... variable declarations ... */
where /* ... logical formula ... */
@@ -49,9 +51,9 @@ The following sections describe the information that is typically included in a
Query metadata
==============
Query metadata is used to identify your custom queries when they are added to the Semmle repository or used in your analysis. Metadata provides information about the query's purpose, and also specifies how to interpret and display the query results. For a full list of metadata properties, see the :doc:`query metadata reference <query-metadata>`. The exact metadata requirement depends on how you are going to run your query:
Query metadata is used to identify your custom queries when they are added to the GitHub repository or used in your analysis. Metadata provides information about the query's purpose, and also specifies how to interpret and display the query results. For a full list of metadata properties, see the :doc:`query metadata reference <query-metadata>`. The exact metadata requirement depends on how you are going to run your query:
- If you are contributing a query to the Semmle open source repository for inclusion in the standard queries, please read the `query metadata style guide <https://github.com/Semmle/ql/blob/master/docs/query-metadata-style-guide.md#metadata-area>`__.
- If you are contributing a query to the GitHub repository, please read the `query metadata style guide <https://github.com/Semmle/ql/blob/master/docs/query-metadata-style-guide.md#metadata-area>`__.
- If you are adding a custom query to a query pack for analysis using LGTM , see `Writing custom queries to include in LGTM analysis <https://lgtm.com/help/lgtm/writing-custom-queries>`__.
- If you are analyzing a project using the `QL command-line tools <https://help.semmle.com/wiki/display/SD/QL+command-line+tools>`__, see `Preparing custom queries <https://help.semmle.com/wiki/display/SD/Preparing+custom+queries>`__.
- If you are running a query in the query console on LGTM or in the Quick query window in QL for Eclipse, metadata is not mandatory. However, if you want your results to be displayed as either an 'alert' or a 'path', you must specify the correct `@kind` property, as explained below. See `Using the query console <https://lgtm.com/help/lgtm/using-query-console>`__ and `Running a quick query <https://help.semmle.com/ql-for-eclipse/Content/WebHelp/run-quick-query.html>`__ for further information.
@@ -60,7 +62,7 @@ Query metadata is used to identify your custom queries when they are added to th
Note
Queries that are contributed to the Semmle open source repository, added to a query pack in LGTM, or used to analyze a project with the QL command-line tools must have a query type (``@kind``) specified. The ``@kind`` property indicates how to interpret and display the results of the query analysis:
Queries that are contributed to the open source repository, added to a query pack in LGTM, or used to analyze a project with the QL command-line tools must have a query type (``@kind``) specified. The ``@kind`` property indicates how to interpret and display the results of the query analysis:
- Alert query metadata must contain ``@kind problem``.
- Path query metadata must contain ``@kind path-problem``.
@@ -71,8 +73,8 @@ Query metadata is used to identify your custom queries when they are added to th
Import statements
=================
Each query generally contains one or more ``import`` statements, which define the `QL libraries <https://help.semmle.com/QL/ql-handbook/modules.html#library-modules>`__ or `modules <https://help.semmle.com/QL/ql-handbook/modules.html>`__ to import into the query. QL libraries and modules provide a way of grouping together related `types <https://help.semmle.com/QL/ql-handbook/types.html>`__, `predicates <https://help.semmle.com/QL/ql-handbook/predicates.html>`__, and other modules. The contents of each library or module that you import can then be accessed by the query.
`Semmle's open source repository on GitHub <https://github.com/semmle/ql>`__ contains all of the standard QL libraries for each supported language.
Each query generally contains one or more ``import`` statements, which define the `libraries <https://help.semmle.com/QL/ql-handbook/modules.html#library-modules>`__ or `modules <https://help.semmle.com/QL/ql-handbook/modules.html>`__ to import into the query. Libraries and modules provide a way of grouping together related `types <https://help.semmle.com/QL/ql-handbook/types.html>`__, `predicates <https://help.semmle.com/QL/ql-handbook/predicates.html>`__, and other modules. The contents of each library or module that you import can then be accessed by the query.
Our `open source repository on GitHub <https://github.com/semmle/ql>`__ contains the standard CodeQL libraries for each supported language.
When writing your own alert queries, you would typically import the standard library for the language of the project that you are querying, using ``import`` followed by a language:
@@ -83,13 +85,13 @@ When writing your own alert queries, you would typically import the standard lib
- JavaScript/TypeScript: ``javascript``
- Python: ``python``
There are also QL libraries containing commonly used predicates, types, and other modules associated with different analyses, including data flow, control flow, and taint-tracking. In order to calculate path graphs, path queries require you to import a data flow library into the query file. See :doc:`Constructing path queries <path-queries>` for further information.
There are also CodeQL libraries containing commonly used predicates, types, and other modules associated with different analyses, including data flow, control flow, and taint-tracking. In order to calculate path graphs, path queries require you to import a data flow library into the query file. See :doc:`Constructing path queries <path-queries>` for further information.
You can explore the contents of all the standard QL libraries in the `QL library reference documentation <https://help.semmle.com/wiki/display/QL/QL+standard+libraries>`__, using `QL for Eclipse <https://help.semmle.com/ql-for-eclipse/Content/WebHelp/standard-queries.html>`__, or in the `GitHub repository <https://github.com/semmle/ql>`__.
You can explore the contents of all the standard CodeQL libraries in the `CodeQL library reference documentation <https://help.semmle.com/wiki/display/QL/QL+standard+libraries>`__, using `QL for Eclipse <https://help.semmle.com/ql-for-eclipse/Content/WebHelp/standard-queries.html>`__, or in the `GitHub repository <https://github.com/semmle/ql>`__.
Optional QL classes and predicates
----------------------------------
Optional CodeQL classes and predicates
--------------------------------------
You can customize your analysis by defining your own predicates and classes in the query. See `Defining a predicate <https://help.semmle.com/QL/ql-handbook/predicates.html#defining-a-predicate>`__ and `Defining a class <https://help.semmle.com/QL/ql-handbook/types.html#defining-a-class>`__ for further details.
@@ -97,13 +99,13 @@ From clause
===========
The ``from`` clause declares the variables that are used in the query. Each declaration must be of the form ``<type> <variable name>``.
For more information on the `types <https://help.semmle.com/QL/ql-handbook/types.html>`__ available in QL, and to learn how to define your own types using `classes <https://help.semmle.com/QL/ql-handbook/types.html#classes>`__, see the `QL language handbook <https://help.semmle.com/QL/ql-handbook/index.html>`__.
For more information on the available `types <https://help.semmle.com/QL/ql-handbook/types.html>`__, and to learn how to define your own types using `classes <https://help.semmle.com/QL/ql-handbook/types.html#classes>`__, see the `QL language handbook <https://help.semmle.com/QL/ql-handbook/index.html>`__.
Where clause
============
The ``where`` clause defines the logical conditions to apply to the variables declared in the ``from`` clause to generate your results. This clause uses `aggregations <https://help.semmle.com/QL/ql-handbook/expressions.html#aggregations>`__, `predicates <https://help.semmle.com/QL/ql-handbook/predicates.html>`__, and logical `formulas <https://help.semmle.com/QL/ql-handbook/formulas.html>`_ to limit the variables of interest to a smaller set, which meet the defined conditions.
The QL libraries group commonly used predicates for specific languages and frameworks. You can also define your own predicates in the body of the query file or in your own custom modules, as described above.
The CodeQL libraries group commonly used predicates for specific languages and frameworks. You can also define your own predicates in the body of the query file or in your own custom modules, as described above.
Select clause
=============
@@ -138,6 +140,6 @@ What next?
- See the queries used in real-life variant analysis on the `Semmle blog <https://blog.semmle.com/tags/variant-analysis>`__.
- To learn more about writing path queries, see :doc:`Constructing path queries <path-queries>`.
- Take a look at the `built-in queries <https://help.semmle.com/wiki/display/QL/Built-in+queries>`__ to see examples of the queries included in the Semmle tools.
- Explore the `query cookbooks <https://help.semmle.com/wiki/display/QL/QL+cookbooks>`__ to see how to access the basic language elements contained in the QL libraries.
- For a full list of resources to help you learn QL, including beginner tutorials and language-specific examples, visit `Learning QL <https://help.semmle.com/QL/learn-ql/>`__.
- Take a look at the `built-in queries <https://help.semmle.com/wiki/display/QL/Built-in+queries>`__ to see examples of the queries included in CodeQL.
- Explore the `query cookbooks <https://help.semmle.com/wiki/display/QL/QL+cookbooks>`__ to see how to access the basic language elements contained in the CodeQL libraries.
- For a full list of resources to help you learn CodeQL, including beginner tutorials and language-specific examples, visit `Learning CodeQL <https://help.semmle.com/QL/learn-ql/>`__.

View File

@@ -5,8 +5,8 @@ Overview
========
Security researchers are particularly interested in the way that information flows in a program. Many vulnerabilities are caused by seemingly benign data flowing to unexpected locations, and being used in a malicious way.
Path queries written in QL are particularly useful for analyzing data flow as they can be used to track the path taken by a variable from its possible starting points (``source``) to its possible end points (``sink``).
To model paths in QL, your query must provide information about the ``source`` and the ``sink``, as well as the data flow steps that link them.
Path queries written with CodeQL are particularly useful for analyzing data flow as they can be used to track the path taken by a variable from its possible starting points (``source``) to its possible end points (``sink``).
To model paths with CodeQL, your query must provide information about the ``source`` and the ``sink``, as well as the data flow steps that link them.
This topic provides information on how to structure a path query file so you can explore the paths associated with the results of data flow analysis.
@@ -17,8 +17,8 @@ This topic provides information on how to structure a path query file so you can
The alerts generated by path queries are displayed by default in `LGTM <https://lgtm.com>`__ and included in the results generated using the `QL command-line tools <https://help.semmle.com/wiki/display/SD/QL+command-line+tools>`__. You can also view the paths explanations generated by your path query `directly in LGTM <https://lgtm.com/help/lgtm/exploring-data-flow-paths>`__, or using the `Path explorer view <https://help.semmle.com/ql-for-eclipse/Content/WebHelp/path-explorer-view.html>`__ in `QL for Eclipse <https://help.semmle.com/ql-for-eclipse/Content/WebHelp/home-page.html>`__.
To learn more about modeling data flow in QL, see :doc:`Introduction to data flow <../intro-to-data-flow>`.
For more language-specific information on analyzing data flow with QL see:
To learn more about modeling data flow with CodeQL, see :doc:`Introduction to data flow <../intro-to-data-flow>`.
For more language-specific information on analyzing data flow see:
- :doc:`Analyzing data flow in C/C++ <../cpp/dataflow>`
- :doc:`Analyzing data flow in C# <../csharp/dataflow>`
@@ -29,7 +29,7 @@ For more language-specific information on analyzing data flow with QL see:
Path query examples
*******************
The easiest way to get started writing your own path query is to modify one of the existing queries. Visit the links below to see all of the built-in path queries:
The easiest way to get started writing your own path query is to modify one of the existing queries. Visit the links below to see all the built-in path queries:
- `C/C++ path queries <https://help.semmle.com/wiki/label/CCPPOBJ/path-problem>`__
- `C# path queries <https://help.semmle.com/wiki/label/CSHARP/path-problem>`__
@@ -43,7 +43,7 @@ Constructing a path query
=========================
Path queries require certain metadata, query predicates, and ``select`` statement structures.
Many of the built-in path queries included in the Semmle tools follow a simple structure, which depends on how the language you are analyzing is modeled in QL.
Many of the built-in path queries included in CodeQL follow a simple structure, which depends on how the language you are analyzing is modeled with CodeQL.
For C/C++, C#, Java, and JavaScript you should use the following template::
@@ -63,7 +63,7 @@ For C/C++, C#, Java, and JavaScript you should use the following template::
Where:
- ``DataFlow::Pathgraph`` is the path graph module you need to import from the standard QL libraries.
- ``DataFlow::Pathgraph`` is the path graph module you need to import from the standard CodeQL libraries.
- ``source`` and ``sink`` are nodes on the `path graph <https://en.wikipedia.org/wiki/Path_graph>`__, and ``DataFlow::PathNode`` is their type.
- ``Configuration`` is a class containing the predicates which define how data may flow between the ``source`` and the ``sink``.
@@ -85,7 +85,7 @@ For Python you should use a slightly different template::
Where:
- ``semmle.python.security.Paths`` is the path graph module imported from the standard QL libraries.
- ``semmle.python.security.Paths`` is the path graph module imported from the standard CodeQL libraries.
- ``source`` and ``sink`` are nodes on the path graph, ``TaintedPathSource source`` and ``TaintedPathSink`` are their respective types. Note, you do not need to declare a configuration class to define the data flow from the ``source`` to the ``sink`` in a Python path query.
@@ -103,7 +103,7 @@ Generating path explanations
In order to generate path explanations, your query needs to compute a `path graph <https://en.wikipedia.org/wiki/Path_graph>`__.
To do this you need to define a `query predicate <https://help.semmle.com/QL/ql-handbook/queries.html#query-predicates>`__ called ``edges`` in your query.
This predicate defines the edge relations of the graph you are computing, and it is used to compute the paths related to each result that your query generates.
You can import a predefined ``edges`` predicate from a path graph module in one of the standard QL data flow libraries. In addition to the path graph module, the data flow libraries contain the other ``classes``, ``predicates``, and ``modules`` that are commonly used in data flow analysis. The import statement to use depends on the language that you are analyzing.
You can import a predefined ``edges`` predicate from a path graph module in one of the standard CodeQL data flow libraries. In addition to the path graph module, the data flow libraries contain the other ``classes``, ``predicates``, and ``modules`` that are commonly used in data flow analysis. The import statement to use depends on the language that you are analyzing.
For C/C++, C#. Java, and JavaScript you would use::
@@ -115,7 +115,7 @@ For Python, the ``Paths`` module contains the ``edges`` predicate::
import semmle.python.security.Paths
You can also import libraries specifically designed to implement data flow analysis in various common frameworks and environments, and many additional libraries are included with the Semmle tools. To see examples of the different libraries used in data flow analysis, see the links to the built-in queries above or browse the `standard QL libraries <https://help.semmle.com/wiki/display/QL/QL+standard+libraries>`__.
You can also import libraries specifically designed to implement data flow analysis in various common frameworks and environments, and many additional libraries are included with CodeQL. To see examples of the different libraries used in data flow analysis, see the links to the built-in queries above or browse the `standard CodeQL libraries <https://help.semmle.com/wiki/display/QL/QL+standard+libraries>`__.
For all languages, you can also optionally define a ``nodes`` query predicate, which specifies the nodes of the path graph that you are interested in. If ``nodes`` is defined, only edges with endpoints defined by these nodes are selected. If ``nodes`` is not defined, you select all possible endpoints of ``edges``.
@@ -128,7 +128,7 @@ You can also define your own ``edges`` predicate in the body of your query. It s
/** Logical conditions which hold if `(a,b)` is an edge in the data flow graph */
}
For more examples of how to define an ``edges`` predicate, visit the `standard QL libraries <https://help.semmle.com/wiki/display/QL/QL+standard+libraries>`__ and search for ``edges``.
For more examples of how to define an ``edges`` predicate, visit the `standard CodeQL libraries <https://help.semmle.com/wiki/display/QL/QL+standard+libraries>`__ and search for ``edges``.
Declaring sources and sinks
***************************
@@ -136,7 +136,7 @@ Declaring sources and sinks
You must provide information about the ``source`` and ``sink`` in your path query. These are objects that correspond to the nodes of the paths that you are exploring.
The name and the type of the ``source`` and the ``sink`` must be declared in the ``from`` statement of the query, and the types must be compatible with the nodes of the graph computed by the ``edges`` predicate.
If you are querying C/C++, C#, Java or JavaScript code (and you have used ``import DataFlow::PathGraph`` in your query), the definitions of the ``source`` and ``sink`` are accessed via the ``Configuration`` class in the data flow library. You should declare all three of these objects in the ``from`` statement.
If you are querying C/C++, C#, Java, or JavaScript code (and you have used ``import DataFlow::PathGraph`` in your query), the definitions of the ``source`` and ``sink`` are accessed via the ``Configuration`` class in the data flow library. You should declare all three of these objects in the ``from`` statement.
For example::
from Configuration config, DataFlow::PathNode source, DataFlow::PathNode sink
@@ -191,5 +191,5 @@ What next?
**********
- Take a look at the path queries for `C/C++ <https://help.semmle.com/wiki/label/CCPPOBJ/path-problem>`__, `C# <https://help.semmle.com/wiki/label/CSHARP/path-problem>`__, `Java <https://help.semmle.com/wiki/label/java/path-problem>`__, `JavaScript <https://help.semmle.com/wiki/label/js/path-problem>`__, and `Python <https://help.semmle.com/wiki/label/python/path-problem>`__ to see examples of the queries included in the Semmle tools.
- Explore the `query cookbooks <https://help.semmle.com/wiki/display/QL/QL+cookbooks>`__ to see how to access the basic language elements contained in the QL libraries.
- For a full list of resources to help you learn QL, including beginner tutorials and language-specific examples, visit `Learning QL <https://help.semmle.com/QL/learn-ql/>`__.
- Explore the `query cookbooks <https://help.semmle.com/wiki/display/QL/QL+cookbooks>`__ to see how to access the basic language elements contained in the CodeQL libraries.
- For a full list of resources to help you learn CodeQL, including beginner tutorials and language-specific examples, visit `Learning CodeQL <https://help.semmle.com/QL/learn-ql/>`__.

View File

@@ -2,16 +2,16 @@ Query help reference
********************
This topic provides detailed information on the structure of query help files.
For more information about how to write useful query help in a style that is consistent with Semmle's built-in queries, see the `Query help style guide <https://github.com/Semmle/ql/blob/master/docs/query-help-style-guide.md>`__ on GitHub.
For more information about how to write useful query help in a style that is consistent with the standard CodeQL queries, see the `Query help style guide <https://github.com/Semmle/ql/blob/master/docs/query-help-style-guide.md>`__ on GitHub.
.. pull-quote::
Note
You can access the query help for Semmle's built-in queries by visiting the `Built-in query pages <https://help.semmle.com/wiki/display/QL/Built-in+queries>`__.
You can also access the raw query help files in the `Semmle/ql GitHub repository <https://github.com/semmle/ql>`__.
For example, the `JavaScript security queries <https://github.com/Semmle/ql/tree/master/javascript/ql/src/Security>`__ and `C/C++ critical queries <https://github.com/Semmle/ql/tree/master/cpp/ql/src/Critical>`__.
You can access the query help for CodeQL queries by visiting the `Built-in query pages <https://help.semmle.com/wiki/display/QL/Built-in+queries>`__.
You can also access the raw query help files in the `GitHub repository <https://github.com/semmle/ql>`__.
For example, see the `JavaScript security queries <https://github.com/Semmle/ql/tree/master/javascript/ql/src/Security>`__ and `C/C++ critical queries <https://github.com/Semmle/ql/tree/master/cpp/ql/src/Critical>`__.
For queries run by default on LGTM, there are several different ways to access the query help. For further information, see `Where do I see the query help for a query on LGTM? <https://lgtm.com/help/lgtm/query-help#where-query-help-in-lgtm>`__ in the LGTM user help.
@@ -207,5 +207,5 @@ The included file, `ThreadUnsafeICryptoTransformOverview.qhelp <https://github.
Further information
===================
- To learn more about contributing to the standard QL queries and libraries, see our `Contributing guidelines <https://github.com/Semmle/ql/blob/master/CONTRIBUTING.md>`__ on GitHub.
- To learn more about writing custom queries, and how to format your QL for clarity and consistency, see `Writing QL queries <https://help.semmle.com/QL/learn-ql/writing-queries/writing-queries.html>`__.
- To learn more about contributing to the standard CodeQL queries and libraries, see our `Contributing guidelines <https://github.com/Semmle/ql/blob/master/CONTRIBUTING.md>`__ on GitHub.
- To learn more about writing custom queries, and how to format your code for clarity and consistency, see `Writing CodeQL queries <https://help.semmle.com/QL/learn-ql/writing-queries/writing-queries.html>`__.

View File

@@ -27,7 +27,7 @@ If you look at some of the LGTM queries, you'll see that they can select extra e
Developing a select statement
-----------------------------
Here's a simple query that uses the standard QL ``CodeDuplication.qll`` library to identify similar files.
Here's a simple query that uses the standard CodeQL ``CodeDuplication.qll`` library to identify similar files.
Basic select statement
~~~~~~~~~~~~~~~~~~~~~~

View File

@@ -1,8 +1,7 @@
Writing QL queries
##################
Writing CodeQL queries
######################
If you are familiar with QL, you can modify the existing Semmle queries or write custom queries to analyze, improve, and secure your own projects. Get started by reading the information for query writers and viewing the examples provided below.
If you are familiar with CodeQL, you can modify the existing queries or write custom queries to analyze, improve, and secure your own projects. Get started by reading the information for query writers and viewing the examples provided below.
Information for query writers
*****************************
@@ -18,19 +17,21 @@ Information for query writers
../locations
Visit `Learning QL <https://help.semmle.com/QL/learn-ql/>`__ to find basic information about QL, as well as help and advice on writing QL for specific programming languages. To learn more about the structure of query files, the key information to include when writing your own QL queries, and how to format your QL for clarity and consistency, see the following topics:
Visit `Learning CodeQL <https://help.semmle.com/QL/learn-ql/>`__ to find basic information about CodeQL. This includes information about the underlying query language QL, as well as help and advice on writing queries for specific programming languages.
To learn more about the structure of query files, the key information to include when writing your own queries, and how to format them for clarity and consistency, see the following topics:
- :doc:`Introduction to query files <introduction-to-queries>`an introduction to the information contained in a basic query file.
- :doc:`Constructing path queries <path-queries>`a quick guide to structuring path queries to use in security research.
- :doc:`Introduction to data flow analysis in QL <../intro-to-data-flow>`a brief introduction to modeling data flow using QL.
- :doc:`Introduction to data flow analysis with CodeQL <../intro-to-data-flow>`a brief introduction to modeling data flow using CodeQL.
- :doc:`Defining 'select' statements <select-statement>`further detail on developing query alert messages to provide extra information in your query results.
- :doc:`Locations and strings for QL entities <../locations>`further detail on providing location information in query results.
- :doc:`Locations and strings for CodeQL entities <../locations>`further detail on providing location information in query results.
- `QL style guide on GitHub <https://github.com/Semmle/ql/blob/master/docs/ql-style-guide.md>`__a guide to formatting QL for consistency and clarity.
Viewing the built-in QL queries
Viewing existing CodeQL queries
*******************************
The easiest way to get started writing your own queries is to modify an existing query. To view examples of the queries included in the latest release of the Semmle tools, or to try out the QL query cookbooks, visit `Exploring QL queries <https://help.semmle.com/QL/ql-explore-queries.html>`__. You can also find all of the Semmle queries in our `open source repository on GitHub <https://github.com/semmle/ql>`__.
The easiest way to get started writing your own queries is to modify an existing query. To see these queries, or to try out the CodeQL query cookbooks, visit `Exploring CodeQL queries <https://help.semmle.com/QL/ql-explore-queries.html>`__.
SYou can also find all the CodeQL queries in our `open source repository on GitHub <https://github.com/semmle/ql>`__.
You can also find examples of queries developed to find security vulnerabilities and bugs in open-source software projects in the `Semmle demos GitHub repository <https://github.com/semmle/demos>`__ and the `Semmle blog <https://blog.semmle.com/tags/security>`__.
@@ -45,7 +46,9 @@ Contributing queries
query-help
Contributions to the standard queries and libraries are very welcomesee our `contributing guidelines <https://github.com/Semmle/ql/blob/master/CONTRIBUTING.md>`__ for further information.
If you are contributing a query to the open source GitHub repository, writing a custom query for LGTM, or using a custom query in an analysis with the QL command-line tools, then you need to include extra metadata in your query to ensure that the query results are interpreted and displayed correctly. See the following topics for more information on query metadata:
If you are contributing a query to the open source GitHub repository, writing a custom query for LGTM, or using a custom query in an analysis with our command-line tools, then you need to include extra metadata in your query to ensure that the query results are interpreted and displayed correctly. See the following topics for more information on query metadata:
.. TODO: Change "command-line tools" to a link to the CodeQL CLI?
- :doc:`Query metadata reference <query-metadata>`
- `Query metadata style guide on GitHub <https://github.com/Semmle/ql/blob/master/docs/query-metadata-style-guide.md>`__

View File

@@ -245,7 +245,7 @@ Compiler pragmas
**Available for**: |characteristic predicates|, |member predicates|, |non-member predicates|
The following compiler pragmas affect the compilation and optimization of QL queries. You
The following compiler pragmas affect the compilation and optimization of queries. You
should avoid using these annotations unless you experience significant performance issues.
Before adding pragmas to your code, contact Semmle to describe the performance problems.

View File

@@ -9,7 +9,7 @@ help you understand how the language works, and how to write more advanced QL co
Each section describes an important concept or syntactic construct of QL. For an overview, see
the table of contents below.
If you are just getting started with QL, see `Learning QL <https://help.semmle.com/QL/learn-ql/>`_
If you are just getting started with QL, see `Learning CodeQL <https://help.semmle.com/QL/learn-ql/>`_
for a list of the available resources.
.. index:: specification

View File

@@ -230,7 +230,7 @@ abstract class, you can add more subclasses to it.
**Example**
If you are writing a security query in QL, you may be interested in identifying
If you are writing a security query, you may be interested in identifying
all expressions that can be interpreted as SQL queries.
You can use the following abstract class to describe these expressions::
@@ -438,7 +438,7 @@ In the standard QL language libraries, this is usually done as follows:
and provide a definition for any abstract predicates from ``A``.
- Annotate the algebraic datatype with :ref:`private`, and leave the classes public.
For example, the following code snippet from the QL for C# data flow library defines classes
For example, the following code snippet from the CodeQL data-flow library for C# defines classes
for dealing with tainted or untainted values. In this case, it doesn't make sense for
``TaintType`` to extend a database type. It is part of the taint analysis, not the underlying
program, so it's helpful to extend a new type (namely ``TTaintType``)::
@@ -473,7 +473,7 @@ Database types
Database types are defined in the database schema. This means that they depend on the database
that you are querying, and vary according to the data you are analyzing.
For example, if you are querying a QL database for a Java project, the database types may
For example, if you are querying a CodeQL database for a Java project, the database types may
include ``@ifstmt``, representing an if statement in the Java code, and ``@variable``,
representing a variable.

View File

@@ -5,7 +5,7 @@ To try the examples in this presentation we recommend you download `QL for Eclip
**QL language resources**
- If you are new to QL, try the QL language tutorials at `Learning QL <https://help.semmle.com/QL/learn-ql/>`__.
- If you are new to QL, try the QL language tutorials at `Learning CodeQL <https://help.semmle.com/QL/learn-ql/>`__.
- To learn more about the main features of QL, try looking at the `QL language handbook <https://help.semmle.com/QL/ql-handbook/>`__.
- For further information about writing queries in QL, see `Writing QL queries <https://help.semmle.com/QL/learn-ql/writing-queries/writing-queries.html>`__.