hohn/codeql
1
0
Fork 0
mirror of https://github.com/github/codeql.git synced 2026-04-26 01:05:15 +02:00
Code Issues Packages Projects Releases Wiki Activity

docs: rename ql-documentation > language

Browse Source
This commit is contained in:
james
2019-08-09 12:30:31 +01:00
parent 605b56f8ca
commit b6d6774329
234 changed files with 0 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

233
docs/language/learn-ql/cpp/conversions-classes.rst Normal file
View File

@@ -0,0 +1,233 @@
Tutorial: Conversions and classes
=================================
Overview
--------
This topic contains worked examples of how to write queries using the standard QL library classes for C/C++ conversions and classes.
Conversions
-----------
Let us take a look at the QL ``Conversion`` class in the standard library:
- ``Expr``
- ``Conversion``
- ``Cast``
- ``CStyleCast``
- ``StaticCast``
- ``ConstCastReinterpretCast``
- ``DynamicCast``
- ``ArrayToPointerConversion``
- ``VirtualMemberToFunctionPointerConversion``
All conversions change the type of an expression. They may be implicit conversions (generated by the compiler) or explicit conversions (requested by the user).
Exploring the subexpressions of an assignment
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Let us consider the following C code:
.. code-block:: cpp
typedef signed int myInt;
int main(int argc, char *argv[])
{
unsigned int i;
i = (myInt)1;
return 0;
}
And this simple query:
.. code-block:: ql
import cpp
from AssignExpr a
select a, a.getLValue().getType(), a.getRValue().getType()
The query examines the code for assignments, and tells us the type of their left and right subexpressions. In the example C code above, there is just one assignment. Notably, this assignment has two conversions (of type ``CStyleCast``) on the right side:
#. Explicit cast of the integer ``1`` to a ``myInt``.
#. Implicit conversion generated by the compiler, in preparation for the assignment, converting that expression into an ``unsigned int``.
The query actually reports the result:
.. code-block:: cpp
... = ... | unsigned int | int
It is as though the conversions are not there! The reason for this is that ``Conversion`` expressions do not wrap the objects they convert; instead conversions are attached to expressions and can be accessed using ``Expr.getConversion()``. The whole assignment in our example is seen by the standard library classes like this:
.. |arrow| unicode:: U+21b3
| ``AssignExpr, i = (myInt)1``
| |arrow| ``VariableAccess, i``
| |arrow| ``Literal, 1``
| |arrow| ``CStyleCast, myInt (explicit)``
| |arrow| ``CStyleCast, unsigned int (implicit)``
Accessing parts of the assignment:
- Left side—access value using ``Assignment.getLValue()``.
- Right side—access value using ``Assignment.getRValue()``.
- Conversions of the ``Literal`` on the right side—access both using calls to ``Expr.getConversion()``. As a shortcut, you can use ``Expr.GetFullyConverted()`` to follow all the way to the resulting type, or ``Expr.GetExplicitlyConverted()`` to find the last explicit conversion from an expression.
Using these predicates we can refine our query so that it reports the results that we expected:
.. code-block:: ql
import cpp
from AssignExpr a
select a, a.getLValue().getExplicitlyConverted().getType(), a.getRValue().getExplicitlyConverted().getType()
The result is now:
.. code-block:: cpp
... = ... | unsigned int | myInt
We can refine the query further by adding ``Type.getUnderlyingType()`` to resolve the ``typedef``:
.. code-block:: ql
import cpp
from AssignExpr a
select a, a.getLValue().getExplicitlyConverted().getType().getUnderlyingType(), a.getRValue().getExplicitlyConverted().getType().getUnderlyingType()
The result is now:
.. code-block:: cpp
... = ... | unsigned int | signed int
If you simply wanted to get the values of all assignments in expressions, regardless of position, you could replace ``Assignment.getLValue()`` and ``Assignment.getRValue()`` with ``Operation.getAnOperand()``:
.. code-block:: ql
import cpp
from AssignExpr a
select a, a.getAnOperand().getExplicitlyConverted().getType()
Unlike the earlier versions of the query, this query would return each side of the expression as a separate result:
.. code-block:: cpp
... = ... | unsigned int
... = ... | myInt
.. pull-quote::
Note
In general, QL predicates named ``getAXxx`` exploit the ability to return multiple results (multiple instances of ``Xxx``) whereas plain ``getXxx`` predicates usually return at most one specific instance of ``Xxx``.
Classes
-------
Next we're going to look at C++ classes, using the following QL classes:
- ``Type``
- ``UserType``—includes classes, typedefs and enums
- ``Class``—a class or struct
- ``Struct``—a struct, which is treated as a subtype of Class in QL.
- ``TemplateClass``—a C++ class template
Finding derived classes
~~~~~~~~~~~~~~~~~~~~~~~
We want to create a query that checks for destructors that should be ``virtual``. Specifically, when a class and a class derived from it both have destructors, the base class destructor should generally be virtual. This ensures that the derived class destructor is always invoked. A ``Destructor`` in QL is a subtype of ``MemberFunction``:
- ``Function``
- ``MemberFunction``
- ``Constructor``
- ``Destructor``
Our starting point for the query is pairs of a base class and a derived class, connected using ``Class.getABaseClass()``:
.. code-block:: ql
import cpp
from Class base, Class derived
where derived.getABaseClass+() = base
select base, derived, "The second class is derived from the first."
➤ `See this in the query console <https://lgtm.com/query/1505902347211/>`__
Note that the transitive closure symbol ``+`` indicates that ``Class.getABaseClass()`` may be followed one or more times, rather than only accepting a direct base class.
A lot of the results are uninteresting template parameters. You can remove those results by updating the ``where`` clause as follows:
.. code-block:: ql
where derived.getABaseClass+() = base
and not exists(base.getATemplateArgument())
and not exists(derived.getATemplateArgument())
➤ `See this in the query console <https://lgtm.com/query/1505907047251/>`__
Finding derived classes with destructors
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Now we can extend the query to find derived classes with destructors, using the ``Class.getDestructor()`` predicate:
.. code-block:: ql
import cpp
from Class base, Class derived, Destructor d1, Destructor d2
where derived.getABaseClass+() = base
and not exists(base.getATemplateArgument())
and not exists(derived.getATemplateArgument())
and d1 = base.getDestructor()
and d2 = derived.getDestructor()
select base, derived, "The second class is derived from the first, and both have a destructor."
➤ `See this in the query console <https://lgtm.com/query/1505901767389/>`__
Notice that getting the destructor implicitly asserts that one exists. As a result, this version of the query returns fewer results than before.
Finding base classes where the destructor is not virtual
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Our last change is to use ``Function.isVirtual()`` to find cases where the base destructor is not virtual:
.. code-block:: ql
import cpp
from Class base, Destructor d1, Class derived, Destructor d2
where derived.getABaseClass+() = base
and d1.getDeclaringType() = base
and d2.getDeclaringType() = derived
and not d1.isVirtual()
select d1, "This destructor should probably be virtual."
➤ `See this in the query console <https://lgtm.com/query/1505908156827/>`__
That completes the query.
There is a similar built-in LGTM `query <https://lgtm.com/rules/2158670642/>`__ that finds classes in a C/C++ project with virtual functions but no virtual destructor. You can take a look at the QL code for this query by clicking **Open in query console** at the top of that page.
What next?
----------
- Explore other ways of querying classes using examples from the `C/C++ cookbook <https://help.semmle.com/wiki/label/CBCPP/class>`__.
- Take a look at the :doc:`Analyzing data flow in C/C++ <dataflow>` tutorial.
- Try the worked examples in the following topics: :doc:`Example: Checking that constructors initialize all private fields <private-field-initialization>` and :doc:`Example: Checking for allocations equal to 'strlen(string)' without space for a null terminator <zero-space-terminator>`.
- Find out more about QL in the `QL language handbook <https://help.semmle.com/QL/ql-handbook/index.html>`__ and `QL language specification <https://help.semmle.com/QL/QLLanguageSpecification.html>`__.
- Learn more about the query console in `Using the query console <https://lgtm.com/help/lgtm/using-query-console>`__.

347
docs/language/learn-ql/cpp/dataflow.rst Normal file
View File

@@ -0,0 +1,347 @@
Analyzing data flow in C/C++
============================
Overview
--------
This topic describes how data flow analysis is implemented in the QL for C/C++ library and includes examples to help you write your own data flow queries.
The following sections describe how to utilize the QL libraries for local data flow, global data flow and taint tracking.
For a more general introduction to modeling data flow in QL, see :doc:`Introduction to data flow analysis in QL <../intro-to-data-flow>`.
Local data flow
---------------
Local data flow is data flow within a single function. Local data flow is usually easier, faster, and more precise than global data flow, and is sufficient for many queries.
Using local data flow
~~~~~~~~~~~~~~~~~~~~~
The local data flow library is in the module ``DataFlow``, which defines the class ``Node`` denoting any element that data can flow through. ``Node``\ s are divided into expression nodes (``ExprNode``) and parameter nodes (``ParameterNode``). It is possible to map between data flow nodes and expressions/parameters using the member predicates ``asExpr`` and ``asParameter``:
.. code-block:: ql
class Node {
/** Gets the expression corresponding to this node, if any. */
Expr asExpr() { ... }
/** Gets the parameter corresponding to this node, if any. */
Parameter asParameter() { ... }
...
}
or using the predicates ``exprNode`` and ``parameterNode``:
.. code-block:: ql
/**
* Gets the node corresponding to expression `e`.
*/
ExprNode exprNode(Expr e) { ... }
/**
* Gets the node corresponding to the value of parameter `p` at function entry.
*/
ParameterNode parameterNode(Parameter p) { ... }
The predicate ``localFlowStep(Node nodeFrom, Node, nodeTo)`` holds if there is an immediate data flow edge from the node ``nodeFrom`` to the node ``nodeTo``. The predicate can be applied recursively (using the ``+`` and ``*`` operators), or through the predefined recursive predicate ``localFlow``, which is equivalent to ``localFlowStep*``.
For example, finding flow from a parameter ``source`` to an expression ``sink`` in zero or more local steps can be achieved as follows:
.. code-block:: ql
DataFlow::localFlow(DataFlow::parameterNode(source), DataFlow::exprNode(sink))
Using local taint tracking
~~~~~~~~~~~~~~~~~~~~~~~~~~
Local taint tracking extends local data flow by including non-value-preserving flow steps. For example:
.. code-block:: cpp
int i = tainted_user_input();
some_big_struct *array = malloc(i * sizeof(some_big_struct));
In this case, the argument to ``malloc`` is tainted.
The local taint tracking library is in the module ``TaintTracking``. Like local data flow, a predicate ``localTaintStep(DataFlow::Node nodeFrom, DataFlow::Node nodeTo)`` holds if there is an immediate taint propagation edge from the node ``nodeFrom`` to the node ``nodeTo``. The predicate can be applied recursively (using the ``+`` and ``*`` operators), or through the predefined recursive predicate ``localTaint``, which is equivalent to ``localTaintStep*``.
For example, finding taint propagation from a parameter ``source`` to an expression ``sink`` in zero or more local steps can be achieved as follows:
.. code-block:: ql
TaintTracking::localTaint(DataFlow::parameterNode(source), DataFlow::exprNode(sink))
Examples
~~~~~~~~
The following query finds the filename passed to ``fopen``.
.. code-block:: ql
import cpp
from Function fopen, FunctionCall fc
where fopen.hasQualifiedName("fopen")
and fc.getTarget() = fopen
select fc.getArgument(0)
Unfortunately, this will only give the expression in the argument, not the values which could be passed to it. So we use local data flow to find all expressions that flow into the argument:
.. code-block:: ql
import cpp
import semmle.code.cpp.dataflow.DataFlow
from Function fopen, FunctionCall fc, Expr src
where fopen.hasQualifiedName("fopen")
and fc.getTarget() = fopen
and DataFlow::localFlow(DataFlow::exprNode(src), DataFlow::exprNode(fc.getArgument(0)))
select src
Then we can vary the source, for example an access to a public parameter. The following query finds where a public parameter is used to open a file:
.. code-block:: ql
import cpp
import semmle.code.cpp.dataflow.DataFlow
from Function fopen, FunctionCall fc, Parameter p
where fopen.hasQualifiedName("fopen")
and fc.getTarget() = fopen
and DataFlow::localFlow(DataFlow::parameterNode(p), DataFlow::exprNode(fc.getArgument(0)))
select p
The following example finds calls to formatting functions where the format string is not hard-coded.
.. code-block:: ql
import semmle.code.cpp.dataflow.DataFlow
import semmle.code.cpp.commons.Printf
from FormattingFunction format, FunctionCall call, Expr formatString
where call.getTarget() = format
and call.getArgument(format.getFormatParameterIndex()) = formatString
and not exists(DataFlow::Node source, DataFlow::Node sink |
DataFlow::localFlow(source, sink) and
source.asExpr() instanceof StringLiteral and
sink.asExpr() = formatString
)
select call, "Argument to " + format.getQualifiedName() + " isn't hard-coded."
Exercises
~~~~~~~~~
Exercise 1: Write a query that finds all hard-coded strings used to create a ``host_ent`` via ``gethostbyname``, using local data flow. (`Answer <#exercise-1>`__)
Global data flow
----------------
Global data flow tracks data flow throughout the entire program, and is therefore more powerful than local data flow. However, global data flow is less precise than local data flow, and the analysis typically requires significantly more time and memory to perform.
Using global data flow
~~~~~~~~~~~~~~~~~~~~~~
The global data flow library is used by extending the class ``DataFlow::Configuration`` as follows:
.. code-block:: ql
import semmle.code.cpp.dataflow.DataFlow
class MyDataFlowConfiguration extends DataFlow::Configuration {
MyDataFlowConfiguration() { this = "MyDataFlowConfiguration" }
override predicate isSource(DataFlow::Node source) {
...
}
override predicate isSink(DataFlow::Node sink) {
...
}
}
The following predicates are defined in the configuration:
- ``isSource``—defines where data may flow from
- ``isSink``—defines where data may flow to
- ``isBarrier``—optional, restricts the data flow
- ``isAdditionalFlowStep``—optional, adds additional flow steps
The characteristic predicate ``MyDataFlowConfiguration()`` defines the name of the configuration, so ``"MyDataFlowConfiguration"`` should be replaced by the name of your class.
The data flow analysis is performed using the predicate ``hasFlow(DataFlow::Node source, DataFlow::Node sink)``:
.. code-block:: ql
from MyDataFlowConfiguration dataflow, DataFlow::Node source, DataFlow::Node sink
where dataflow.hasFlow(source, sink)
select source, "Data flow to $@.", sink, sink.toString()
Using global taint tracking
~~~~~~~~~~~~~~~~~~~~~~~~~~~
Global taint tracking is to global data flow as local taint tracking is to local data flow. That is, global taint tracking extends global data flow with additional non-value-preserving steps. The global taint tracking library is used by extending the class ``TaintTracking::Configuration`` as follows:
.. code-block:: ql
import semmle.code.cpp.dataflow.TaintTracking
class MyTaintTrackingConfiguration extends TaintTracking::Configuration {
MyTaintTrackingConfiguration() { this = "MyTaintTrackingConfiguration" }
override predicate isSource(DataFlow::Node source) {
...
}
override predicate isSink(DataFlow::Node sink) {
...
}
}
The following predicates are defined in the configuration:
- ``isSource``—defines where taint may flow from
- ``isSink``—defines where taint may flow to
- ``isSanitizer``—optional, restricts the taint flow
- ``isAdditionalTaintStep``—optional, adds additional taint steps
Similar to global data flow, the characteristic predicate ``MyTaintTrackingConfiguration()`` defines the unique name of the configuration, so ``"MyTaintTrackingConfiguration"`` should be replaced by the name of your class.
The taint tracking analysis is performed using the predicate ``hasFlow(DataFlow::Node source, DataFlow::Node sink)``.
Examples
~~~~~~~~
The following data flow configuration tracks data flow from environment variables to opening files in a Unix-like environment:
.. code-block:: ql
import semmle.code.cpp.dataflow.DataFlow
class EnvironmentToFileConfiguration extends DataFlow::Configuration {
EnvironmentToFileConfiguration() { this = "EnvironmentToFileConfiguration" }
override predicate isSource(DataFlow::Node source) {
exists (Function getenv |
source.asExpr().(FunctionCall).getTarget() = getenv and
getenv.hasQualifiedName("getenv")
)
}
override predicate isSink(DataFlow::Node sink) {
exists (FunctionCall fc |
sink.asExpr() = fc.getArgument(0) and
fc.getTarget().hasQualifiedName("fopen")
)
}
}
from Expr getenv, Expr fopen, EnvironmentToFileConfiguration config
where config.hasFlow(DataFlow::exprNode(getenv), DataFlow::exprNode(fopen))
select fopen, "This 'fopen' uses data from $@.",
getenv, "call to 'getenv'"
Exercises
~~~~~~~~~
Exercise 2: Write a query that finds all hard-coded strings used to create a ``host_ent`` via ``gethostbyname``, using global data flow. (`Answer <#exercise-2>`__)
Exercise 3: Write a class that represents flow sources from ``getenv``. (`Answer <#exercise-3>`__)
Exercise 4: Using the answers from 2 and 3, write a query which finds all global data flows from ``getenv`` to ``gethostbyname``. (`Answer <#exercise-4>`__)
What next?
----------
- Try the worked examples in the following topics: :doc:`Example: Checking that constructors initialize all private fields <private-field-initialization>` and :doc:`Example: Checking for allocations equal to 'strlen(string)' without space for a null terminator <zero-space-terminator>`.
- Find out more about QL in the `QL language handbook <https://help.semmle.com/QL/ql-handbook/index.html>`__ and `QL language specification <https://help.semmle.com/QL/ql-spec/language.html>`__.
- Learn more about the query console in `Using the query console <https://lgtm.com/help/lgtm/using-query-console>`__.
Answers
-------
Exercise 1
~~~~~~~~~~
.. code-block:: ql
import semmle.code.cpp.dataflow.DataFlow
from StringLiteral sl, FunctionCall fc
where fc.getTarget().hasName("gethostbyname")
and DataFlow::localFlow(DataFlow::exprNode(sl), DataFlow::exprNode(fc.getArgument(0)))
select sl, fc
Exercise 2
~~~~~~~~~~
.. code-block:: ql
import semmle.code.cpp.dataflow.DataFlow
class LiteralToGethostbynameConfiguration extends DataFlow::Configuration {
LiteralToGethostbynameConfiguration() {
this = "LiteralToGethostbynameConfiguration"
}
override predicate isSource(DataFlow::Node source) {
source.asExpr() instanceof StringLiteral
}
override predicate isSink(DataFlow::Node sink) {
exists (FunctionCall fc |
sink.asExpr() = fc.getArgument(0) and
fc.getTarget().hasName("gethostbyname"))
}
}
from StringLiteral sl, FunctionCall fc, LiteralToGethostbynameConfiguration cfg
where cfg.hasFlow(DataFlow::exprNode(sl), DataFlow::exprNode(fc.getArgument(0)))
select sl, fc
Exercise 3
~~~~~~~~~~
.. code-block:: ql
import cpp
class GetenvSource extends FunctionCall {
GetenvSource() {
this.getTarget().hasQualifiedName("getenv")
}
}
Exercise 4
~~~~~~~~~~
.. code-block:: ql
import semmle.code.cpp.dataflow.DataFlow
class GetenvSource extends DataFlow::Node {
GetenvSource() {
this.asExpr().(FunctionCall).getTarget().hasQualifiedName("getenv")
}
}
class GetenvToGethostbynameConfiguration extends DataFlow::Configuration {
GetenvToGethostbynameConfiguration() {
this = "GetenvToGethostbynameConfiguration"
}
override predicate isSource(DataFlow::Node source) {
source instanceof GetenvSource
}
override predicate isSink(DataFlow::Node sink) {
exists (FunctionCall fc |
sink.asExpr() = fc.getArgument(0) and
fc.getTarget().hasName("gethostbyname"))
}
}
from DataFlow::Node getenv, FunctionCall fc, GetenvToGethostbynameConfiguration cfg
where cfg.hasFlow(getenv, DataFlow::exprNode(fc.getArgument(0)))
select getenv.asExpr(), fc

141
docs/language/learn-ql/cpp/expressions-types.rst Normal file
View File

@@ -0,0 +1,141 @@
Tutorial: Expressions, types and statements
===========================================
Overview
--------
This topic contains worked examples of how to write queries using the standard QL library classes for C/C++ expressions, types, and statements.
Expressions and types
---------------------
Each part of an expression in C becomes an instance of the QL ``Expr`` class. For example, the C code ``x = x + 1`` becomes an ``AssignExpr``, an ``AddExpr``, two instances of ``VariableAccess`` and a ``Literal``. All of these QL classes extend ``Expr``.
Finding assignments to zero
~~~~~~~~~~~~~~~~~~~~~~~~~~~
In the following example we find instances of ``AssignExpr`` which assign the constant value zero:
.. code-block:: ql
import cpp
from AssignExpr e
where e.getRValue().getValue().toInt() = 0
select e, "Assigning the value 0 to something."
➤ `See this in the query console <https://lgtm.com/query/1505908086530/>`__
The ``where`` clause in this example gets the expression on the right side of the assignment, ``getRValue()``, and compares it with zero. Notice that there are no checks to make sure that the right side of the assignment is an integer or that it has a value (that is, it is compile-time constant, rather than a variable). For expressions where either of these assumptions is wrong, the associated QL predicate simply does not return anything and the ``where`` clause will not produce a result. You could think of it as if there is an implicit ``exists(e.getRValue().getValue().toInt())`` at the beginning of this line.
It is also worth noting that the query above would find this C code:
.. code-block:: cpp
yPtr = NULL;
This is because the snapshot contains a representation of the code base after the preprocessor transforms have run (for more information, see `Database generation <https://lgtm.com/help/lgtm/generate-database>`__). This means that any macro invocations, such as the ``NULL`` define used here, are expanded during the creation of the snapshot. If you want to write queries about macros then there are some special library classes that have been designed specifically for this purpose (for example, the ``Macro``, ``MacroInvocation`` classes and predicates like ``Element.isInMacroExpansion()``). In this case, it is good that macros are expanded, but we do not want to find assignments to pointers.
Finding assignments of 0 to an integer
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
We can make the query more specific by defining a condition for the left side of the expression. For example:
.. code-block:: ql
import cpp
from AssignExpr e
where e.getRValue().getValue().toInt() = 0
and e.getLValue().getType().getUnspecifiedType() instanceof IntegralType
select e, "Assigning the value 0 to an integer."
➤ `See this in the query console <https://lgtm.com/query/1505906986578/>`__
This checks that the left side of the assignment has a type that is some kind of integer. Note the call to ``Type.getUnspecifiedType()``. This resolves ``typedef`` types to their underlying types so that the query finds assignments like this one:
.. code-block:: cpp
typedef int myInt;
myInt i;
i = 0;
Statements
----------
We can refine the query further using statements. In this case we use the class ``ForStmt``:
- ``Stmt`` - C/C++ statements
- ``Loop``
- ``WhileStmt``
- ``ForStmt``
- ``DoStmt``
- ``ConditionalStmt``
- ``IfStmt``
- ``SwitchStmt``
- ``TryStmt``
- ``ExprStmt`` - expressions used as a statement; for example, an assignment
- ``Block`` - { } blocks containing more statements
Finding assignments of 0 in 'for' loop initialization
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
We can restrict the previous query so that it only considers assignments inside ``for`` statements by adding the ``ForStmt`` class to the query. Then we want to compare the expression to ``ForStmt.getInitialization()``:
.. code-block:: ql
import cpp
from AssignExpr e, ForStmt f
// the assignment is the for loop initialization
where e = f.getInitialization()
...
Unfortunately this would not quite work, because the loop initialization is actually a ``Stmt`` not an ``Expr``—the ``AssignExpr`` class is wrapped in an ``ExprStmt`` class. Instead, we need to find the closest enclosing ``Stmt`` around the expression using ``Expr.getEnclosingStmt()``:
.. code-block:: ql
import cpp
from AssignExpr e, ForStmt f
// the assignment is in the 'for' loop initialization statement
where e.getEnclosingStmt() = f.getInitialization()
and e.getRValue().getValue().toInt() = 0
and e.getLValue().getType().getUnspecifiedType() instanceof IntegralType
select e, "Assigning the value 0 to an integer, inside a for loop initialization."
➤ `See this in the query console <https://lgtm.com/query/1505909016965/>`__
Finding assignments of 0 within the loop body
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
We can find assignments inside the loop body using similar code with the predicate ``Loop.getStmt():``
.. code-block:: ql
import cpp
from AssignExpr e, ForStmt f
// the assignment is in the for loop body
where e.getEnclosingStmt().getParentStmt*() = f.getStmt()
and e.getRValue().getValue().toInt() = 0
and e.getLValue().getType().getUnderlyingType() instanceof IntegralType
select e, "Assigning the value 0 to an integer, inside a for loop body."
➤ `See this in the query console <https://lgtm.com/query/1505901437190/>`__
Note that we replaced ``e.getEnclosingStmt()`` with ``e.getEnclosingStmt().getParentStmt*()``, to find an assignment expression that is deeply nested inside the loop body. The transitive closure modifier ``*`` here indicates that ``Stmt.getParentStmt()`` may be followed zero or more times, rather than just once, giving us the statement, its parent statement, its parent's parent statement etc.
What next?
----------
- Explore other ways of finding types and statements using examples from the C/C++ cookbook for `types <https://help.semmle.com/wiki/label/CBCPP/type>`__ and `statements <https://help.semmle.com/wiki/label/CBCPP/statement>`__.
- Take a look at the :doc:`Conversions and classes <conversions-classes>` and :doc:`Analyzing data flow in C/C++ <dataflow>` tutorials.
- Find out more about QL in the `QL language handbook <https://help.semmle.com/QL/ql-handbook/index.html>`__ and `QL language specification <https://help.semmle.com/QL/QLLanguageSpecification.html>`__.
- Learn more about the query console in `Using the query console <https://lgtm.com/help/lgtm/using-query-console>`__.

96
docs/language/learn-ql/cpp/function-classes.rst Normal file
View File

@@ -0,0 +1,96 @@
Tutorial: Function classes
==========================
Overview
--------
The standard QL library for C and C++ represents functions using the ``Function`` class (see :doc:`Introducing the C/C++ libraries <introduce-libraries-cpp>`).
The example queries in this topic explore some of the most useful library predicates for querying functions.
Finding all static functions
----------------------------
Using the member predicate ``Function.isStatic()`` we can list all of the static functions in a snapshot:
.. code-block:: ql
import cpp
from Function f
where f.isStatic()
select f, "This is a static function."
This query is very general, so there are probably too many results to be interesting for most nontrivial projects.
Finding functions that are not called
-------------------------------------
It might be more interesting to find functions that are not called, using the standard QL ``FunctionCall`` class from the **abstract syntax tree** category (see :doc:`Introducing the C/C++ libraries <introduce-libraries-cpp>`). The ``FunctionCall`` class can be used to identify places where a function is actually used, and it is related to ``Function`` through the ``FunctionCall.getTarget()`` predicate.
.. code-block:: ql
import cpp
from Function f
where not exists(FunctionCall fc | fc.getTarget() = f)
select f, "This function is never called."
➤ `See this in the query console <https://lgtm.com/query/1505891246456/>`__
The new query finds functions that are not the target of any ``FunctionCall``—in other words, functions that are never called. You may be surprised by how many results the query finds. However, if you examine the results, you can see that many of the functions it finds are used indirectly. To create a query that finds only unused functions, we need to refine the query and exclude other ways of using a function.
Excluding functions that are referenced with a function pointer
---------------------------------------------------------------
You can modify the query to remove functions where a function pointer is used to reference the function:
.. code-block:: ql
import cpp
from Function f
where not exists(FunctionCall fc | fc.getTarget() = f)
and not exists(FunctionAccess fa | fa.getTarget() = f)
select f, "This function is never called, or referenced with a function pointer."
➤ `See this in the query console <https://lgtm.com/query/1505890446605/>`__
This query returns fewer results. However, if you examine the results then you can probably still find potential refinements.
For example, there is a more complicated LGTM `query <https://lgtm.com/rules/2152580467/>`__ that finds unused static functions. To see the QL code for this query, click **Open in query console** at the top of the page.
You can explore the definition of an element in the standard QL libraries and see what predicates are available. Use the keyboard **F3** button to open the definition of any element. Alternatively, hover over the element and click **Jump to definition** in the tooltip displayed. The library file is opened in a new tab with the definition highlighted.
Finding a specific function
---------------------------
This query uses ``Function`` and ``FunctionCall`` to find calls to the function ``sprintf`` that have a variable format string—which is potentially a security hazard.
.. code-block:: ql
import cpp
from FunctionCall fc
where fc.getTarget().getQualifiedName() = "sprintf"
and not fc.getArgument(1) instanceof StringLiteral
select fc, "sprintf called with variable format string."
➤ `See this in the query console <https://lgtm.com/query/1505889506751/>`__
This uses:
- ``Declaration.getQualifiedName()`` to identify calls to the specific function ``sprintf``.
- ``FunctionCall.getArgument(1)`` to fetch the format string argument.
Note that we could have used ``Declaration.getName()``, but ``Declaration.getQualifiedName()`` is a better choice because it includes the namespace. For example: ``getName()`` would return ``vector`` where ``getQualifiedName`` would return ``std::vector``.
The LGTM version of this query is considerably more complicated, but if you look carefully you will find that its structure is the same. See `Non-constant format string <https://lgtm.com/rules/2152810612/>`__ and click **Open in query console** at the top of the page.
What next?
----------
- Explore other ways of finding functions using examples from the `C/C++ cookbook <https://help.semmle.com/wiki/label/CBCPP/function>`__.
- Take a look at some of the other tutorials: :doc:`Expressions, types and statements <expressions-types>`, :doc:`Conversions and classes <conversions-classes>`, and :doc:`Analyzing data flow in C/C++ <dataflow>`.
- Find out more about QL in the `QL language handbook <https://help.semmle.com/QL/ql-handbook/index.html>`__ and `QL language specification <https://help.semmle.com/QL/QLLanguageSpecification.html>`__.
- Learn more about the query console in `Using the query console <https://lgtm.com/help/lgtm/using-query-console>`__.

121
docs/language/learn-ql/cpp/introduce-libraries-cpp.rst Normal file
View File

@@ -0,0 +1,121 @@
Introducing the C/C++ libraries
===============================
Overview
--------
There is an extensive QL library for analyzing C/C++ code. The QL classes in this library present the data from a snapshot database in an object-oriented form and provide abstractions and predicates to help you with common analysis tasks. The library is implemented as a set of QL modules, that is, files with the extension ``.qll``. The module ``cpp.qll`` imports all the core library modules, so you can include the complete library by beginning your query with:
.. code-block:: ql
import cpp
The rest of this topic briefly summarizes the most important QL classes and predicates provided by this library.
You can find related classes and features using the query console's auto-complete feature. You can also press **F3** to jump to the definition of any element (QL library files are opened in new tabs in the console).
Summary of the library classes
------------------------------
The most commonly used standard QL library classes are organized as follows:
Preprocessor logic
~~~~~~~~~~~~~~~~~~
- ``Include`` — ``#include`` directives (defined in the `Include.qll <https://help.semmle.com/qldoc/cpp/semmle/code/cpp/Include.qll/module.Include.html>`__ library)
- ``Macro``, ``MacroInvocation`` — ``#define`` directives and uses (defined in the `Macro.qll <https://help.semmle.com/qldoc/cpp/semmle/code/cpp/Macro.qll/module.Macro.html>`__ library)
- ``PreprocessorIf``, ``PreprocessorElse``, ``PreprocessorIfdef``, ``PreprocessorIfndef``, ``PreprocessorEndif`` — conditional processing directives (defined in the `Preprocessor.qll <https://help.semmle.com/qldoc/cpp/semmle/code/cpp/Preprocessor.qll/module.Preprocessor.html>`__ library)
Symbol table
~~~~~~~~~~~~
- ``Function`` — all functions, including member functions (defined in the `Function.qll <https://help.semmle.com/qldoc/cpp/semmle/code/cpp/Function.qll/module.Function.html>`__ library)
- ``MemberFunction``
- ``Variable`` (defined in the `Variable.qll <https://help.semmle.com/qldoc/cpp/semmle/code/cpp/Variable.qll/module.Variable.html>`__ library)
- ``LocalScopeVariable``— local variables and parameters
- ``Parameter``
- ``MemberVariable`` — member variables of classes
- ``GlobalOrNamespaceVariable`` — global variables
- ``Type`` — built-in and user-defined types (defined in the `Type.qll <https://help.semmle.com/qldoc/cpp/semmle/code/cpp/Type.qll/module.Type.html>`__ library)
- ``Class`` — classes and structs
- ``Enum`` — enums, including scoped enums
- ``TypedefType`` — typedefs
- ``ArrayType`` — arrays
- ``PointerType`` — pointers
- ``SpecifiedType`` — qualifiers such as ``const``, ``volatile``, or ``restrict``
- ``Namespace`` — namespaces (defined in the `Namespace.qll <https://help.semmle.com/qldoc/cpp/semmle/code/cpp/Namespace.qll/module.Namespace.html>`__ library)
- ``GlobalNamespace`` — the global namespace
- ``Initializer`` — initializers for variables (defined in the `Initializer.qll <https://help.semmle.com/qldoc/cpp/semmle/code/cpp/Initializer.qll/module.Initializer.html>`__ library)
Abstract syntax tree
~~~~~~~~~~~~~~~~~~~~
- `Expr <https://help.semmle.com/qldoc/cpp/semmle/code/cpp/exprs/Expr.qll/type.Expr$Expr.html>`__ — all expressions that occur in the code
- `Assignment <https://help.semmle.com/qldoc/cpp/semmle/code/cpp/exprs/Assignment.qll/type.Assignment$Assignment.html>`__ — assignment operators, for example: ``=``, ``+=``, ``-=``, ``*=``, ``/=``, ``%=``, ``&=``, ``|=``, ``^=``, ``<<=``, ``>>=``
- `UnaryOperation <https://help.semmle.com/qldoc/cpp/semmle/code/cpp/exprs/Expr.qll/type.Expr$UnaryOperation.html>`__ — unary operators, for example: ``+``, ``-``, ``~``, ``++``, ``--``, ``~``, ``!``, ``&``, ``*``
- `BinaryOperation <https://help.semmle.com/qldoc/cpp/semmle/code/cpp/exprs/Expr.qll/type.Expr$BinaryOperation.html>`__ — binary operators, for example: ``+``, ``-``, ``*``, ``/``, ``%``, ``&``, ``|``, ``^``, ``<<``, ``>>``, ``&&``, ``||``, ``==``, ``!=``, ``<=``, ``<``, ``>``, ``>=``
- `ConditionalExpr <https://help.semmle.com/qldoc/cpp/semmle/code/cpp/exprs/LogicalOperation.qll/type.LogicalOperation$ConditionalExpr.html>`__ — the ternary conditional operator, ``? :``
- `ParenthesisExpr <https://help.semmle.com/qldoc/cpp/semmle/code/cpp/exprs/Expr.qll/type.Expr$ParenthesisExpr.html>`__
- `Literal <https://help.semmle.com/qldoc/cpp/semmle/code/cpp/exprs/Literal.qll/type.Literal$Literal.html>`__ — string, numeric and character literals
- `Conversion <https://help.semmle.com/qldoc/cpp/semmle/code/cpp/exprs/Cast.qll/type.Cast$Conversion.html>`__ — casts and compiler generated conversions
- `Call <https://help.semmle.com/qldoc/cpp/semmle/code/cpp/exprs/Call.qll/type.Call$Call.html>`__ — function calls, function pointer calls...
- ``FunctionCall``
- `Access <https://help.semmle.com/qldoc/cpp/semmle/code/cpp/exprs/Access.qll/type.Access$Access.html>`__
- ``VariableAccess``
- ``FunctionAccess``
- `ArrayExpr <https://help.semmle.com/qldoc/cpp/semmle/code/cpp/exprs/Access.qll/type.Access$ArrayExpr.html>`__
- `SizeofOperator <https://help.semmle.com/qldoc/cpp/semmle/code/cpp/exprs/Cast.qll/type.Cast$SizeofOperator.html>`__
- `ThrowExpr <https://help.semmle.com/qldoc/cpp/semmle/code/cpp/exprs/Call.qll/type.Call$ThrowExpr.html>`__
- `ThisExpr <https://help.semmle.com/qldoc/cpp/semmle/code/cpp/exprs/Expr.qll/type.Expr$ThisExpr.html>`__
- `NewExpr <https://help.semmle.com/qldoc/cpp/semmle/code/cpp/exprs/Expr.qll/type.Expr$NewExpr.html>`__, `NewArrayExpr <https://help.semmle.com/qldoc/cpp/semmle/code/cpp/exprs/Expr.qll/type.Expr$NewArrayExpr.html>`__, `DeleteExpr <https://help.semmle.com/qldoc/cpp/semmle/code/cpp/exprs/Expr.qll/type.Expr$DeleteExpr.html>`__, `DeleteArrayExpr <https://help.semmle.com/qldoc/cpp/semmle/code/cpp/exprs/Expr.qll/type.Expr$DeleteArrayExpr.html>`__
- `Stmt <https://help.semmle.com/qldoc/cpp/semmle/code/cpp/stmts/Stmt.qll/type.Stmt$Stmt.html>`__ — C/C++ statements
- `IfStmt <https://help.semmle.com/qldoc/cpp/semmle/code/cpp/stmts/Stmt.qll/type.Stmt$IfStmt.html>`__, `WhileStmt <https://help.semmle.com/qldoc/cpp/semmle/code/cpp/stmts/Stmt.qll/type.Stmt$WhileStmt.html>`__, `ForStmt <https://help.semmle.com/qldoc/cpp/semmle/code/cpp/stmts/Stmt.qll/type.Stmt$ForStmt.html>`__, `DoStmt <https://help.semmle.com/qldoc/cpp/semmle/code/cpp/stmts/Stmt.qll/type.Stmt$DoStmt.html>`__, `SwitchStmt <https://help.semmle.com/qldoc/cpp/semmle/code/cpp/stmts/Stmt.qll/type.Stmt$SwitchStmt.html>`__, `TryStmt <https://help.semmle.com/qldoc/cpp/semmle/code/cpp/stmts/Stmt.qll/type.Stmt$TryStmt.html>`__
- `ExprStmt <https://help.semmle.com/qldoc/cpp/semmle/code/cpp/stmts/Stmt.qll/type.Stmt$ExprStmt.html>`__ — expressions used as a statement, for example, an assignment
- `Block <https://help.semmle.com/qldoc/cpp/semmle/code/cpp/stmts/Block.qll/type.Block$Block.html>`__ — ``{ }`` blocks containing more statements
- `DeclarationEntry <https://help.semmle.com/qldoc/cpp/semmle/code/cpp/Declaration.qll/type.Declaration$DeclarationEntry.html>`__ — sites where items from the symbol table are declared and/or defined in code
- ``FunctionDeclarationEntry``
- ``VariableDeclarationEntry``
- ``TypeDeclarationEntry``
Control flow graph
~~~~~~~~~~~~~~~~~~
- `ControlFlowNode <https://help.semmle.com/qldoc/cpp/semmle/code/cpp/controlflow/ControlFlowGraph.qll/type.ControlFlowGraph$ControlFlowNode.html>`__ — statements, expressions and functions; control flow can be explored via the predicate ``ControlFlowNode.getASuccessor()``
- `BasicBlock <https://help.semmle.com/qldoc/cpp/semmle/code/cpp/controlflow/BasicBlocks.qll/type.BasicBlocks$BasicBlock.html>`__ — a `basic block <http://en.wikipedia.org/wiki/Basic_block>`__
External data
~~~~~~~~~~~~~
- `Diagnostic <https://help.semmle.com/qldoc/cpp/semmle/code/cpp/Diagnostics.qll/type.Diagnostics$Diagnostic.html>`__ — messages (such as errors or warnings) that were produced by the compiler
- `XMLParent <https://help.semmle.com/qldoc/cpp/semmle/code/cpp/XML.qll/type.XML$XMLParent.html>`__ — XML data that was added to the snapshot
- ``XMLFile``
- ``XMLElement``
- `ExternalData <https://help.semmle.com/qldoc/cpp/external/ExternalArtifact.qll/type.ExternalArtifact$ExternalData.html>`__ — any CSV data that has been imported into the snapshot
What next?
----------
- Experiment with the worked examples in the QL for C/C++ topics: :doc:`Function classes <function-classes>`, :doc:`Expressions, types and statements <expressions-types>`, :doc:`Conversions and classes <conversions-classes>`, and :doc:`Analyzing data flow in C/C++ <dataflow>`.
- Find out more about QL in the `QL language handbook <https://help.semmle.com/QL/ql-handbook/index.html>`__ and `QL language specification <https://help.semmle.com/QL/QLLanguageSpecification.html>`__.
- Learn more about the query console in `Using the query console <https://lgtm.com/help/lgtm/using-query-console>`__.

152
docs/language/learn-ql/cpp/private-field-initialization.rst Normal file
View File

@@ -0,0 +1,152 @@
Example: Checking that constructors initialize all private fields
=================================================================
Overview
--------
This topic describes how a C++ query was developed. The example introduces recursive predicates and demonstrates the typical workflow used to refine a query. For a full overview of the topics available for learning to write QL queries for C/C++ code, see :doc:`QL for C/C++ <ql-for-cpp>`.
Problem—finding every private field and checking for initialization
-------------------------------------------------------------------
Writing a query to check if a constructor initializes all private fields seems like a simple problem, but there are several edge cases to account for.
Basic query
-----------
We can start by looking at every private field in a class and checking that every constructor in that class initializes them. Once you are familiar with the library for C++ this is not too hard to do.
.. code-block:: ql
import cpp
from Constructor c, Field f
where f.getDeclaringType() = c.getDeclaringType() and f.isPrivate()
and not exists(Assignment a | a = f.getAnAssignment() and a.getEnclosingFunction() = c)
select c, "Constructor does not initialize fields $@.", f, f.getName()
#. ``f.getDeclaringType() = c.getDeclaringType()`` asserts that the field and constructor are both part of the same class.
#. ``f.isPrivate()`` checks if the field is private.
#. ``not exists(Assignment a | a = f.getAnAssignment() and a.getEnclosingFunction() = c)`` checks that there is no assignment to the field in the constructor.
This QL code looks fairly complete, but when you test it on a project, there are several results that contain examples that we have overlooked.
Refinement 1—excluding fields initialized by lists
--------------------------------------------------
You may see that the results contain fields that are initialized by constructor initialization lists, instead of by assignment statements. For example, the following class:
.. code-block:: cpp
class BoxedInt {
public:
BoxedInt(int value) : m_value(value) {}
private:
int m_value;
};
These can be excluded by adding an extra condition to check for this special constructor-only form of assignment.
.. code-block:: ql
import cpp
from Constructor c, Field f
where f.getDeclaringType() = c.getDeclaringType() and f.isPrivate()
and not exists(Assignment a | a = f.getAnAssignment() and a.getEnclosingFunction() = c)
// check for constructor initialization lists as well
and not exists(ConstructorFieldInit i | i.getTarget() = f and i.getEnclosingFunction() = c)
select c, "Constructor does not initialize fields $@.", f, f.getName()
Refinement 2—excluding fields initialized by external libraries
---------------------------------------------------------------
When you test the revised query, you may discover that fields from classes in external libraries are over-reported. This is often because a header file declares a constructor that is defined in a source file that is not analyzed (external libraries are often excluded from analysis). When the source code is analyzed, the snapshot is populated with a ``Constructor`` entry with no body. This ``constructor`` therefore contains no assignments and consequently the query reports that any fields initialized by the constructor are "uninitialized." There is no particular reason to be suspicious of these cases, and we can exclude them from the results by defining a condition to exclude constructors that have no body:
.. code-block:: ql
import cpp
from Constructor c, Field f
where f.getDeclaringType() = c.getDeclaringType() and f.isPrivate()
and not exists(Assignment a | a = f.getAnAssignment() and a.getEnclosingFunction() = c)
// check for constructor initialization lists as well
and not exists(ConstructorFieldInit i | i.getTarget() = f and i.getEnclosingFunction() = c)
// ignore cases where the constructor source code is not available
and exists(c.getBlock())
select c, "Constructor does not initialize fields $@.", f, f.getName()
This is a reasonably precise query—most of the results that it reports are interesting. However, you could make further refinements.
Refinement 3—excluding fields initialized indirectly
----------------------------------------------------
You may also wish to consider methods called by constructors that assign to the fields, or even to the methods called by those methods. As a concrete example of this, consider the following class.
.. code-block:: cpp
class BoxedInt {
public:
BoxedInt(int value) {
setValue(value);
}
void setValue(int value) {
m_value = value;
}
private:
int m_value;
};
This case can be excluded by creating a recursive predicate. The recursive predicate is given a function and a field, then checks whether the function assigns to the field. The predicate runs itself on all the functions called by the function that it has been given. By passing the constructor to this predicate, we can check for assignments of a field in all functions called by the constructor, and then do the same for all functions called by those functions all the way down the tree of function calls (see `Recursion <https://help.semmle.com/QL/ql-handbook/recursion.html>`__ for more information).
.. code-block:: ql
import cpp
predicate getSubAssignment(Function c, Field f){
exists(Assignment a | a = f.getAnAssignment() and a.getEnclosingFunction() = c)
or exists(Function fun | c.calls(fun) and getSubAssignment(fun, f))
}
from Constructor c, Field f
where f.getDeclaringType() = c.getDeclaringType() and f.isPrivate()
// check for constructor initialization lists as well
and not exists(ConstructorFieldInit i | i.getTarget() = f and i.getEnclosingFunction() = c)
// check for initializations performed indirectly by methods called
// as a result of the constructor being called
and not getSubAssignment(c, f)
// ignore cases where the constructor source code is not available
and exists(c.getBlock())
select c, "Constructor does not initialize fields $@.", f, f.getName()
Refinement 4—simplifying the query
----------------------------------
Finally we can simplify the query by using the `transitive closure operator <https://help.semmle.com/QL/ql-handbook/recursion.html#transitive-closures>`__. In this final version of the query, ``c.calls*(fun)`` resolves to the set of all functions that are ``c`` itself, are called by ``c``, are called by a function that is called by ``c``, and so on. This eliminates the need to make a new predicate all together.
.. code-block:: ql
import cpp
from Constructor c, Field f
where f.getDeclaringType() = c.getDeclaringType() and f.isPrivate()
// check for constructor initialization lists as well
and not exists(ConstructorFieldInit i | i.getTarget() = f and i.getEnclosingFunction() = c)
// check for initializations performed indirectly by methods called
// as a result of the constructor being called
and not exists(Function fun, Assignment a |
c.calls*(fun) and a = f.getAnAssignment() and a.getEnclosingFunction() = fun)
// ignore cases where the constructor source code is not available
and exists(c.getBlock())
select c, "Constructor does not initialize fields $@.", f, f.getName()
➤ `See this in the query console <https://lgtm.com/query/1505896968215/>`__
What next?
----------
- Take a look at another example: :doc:`Checking for allocations equal to 'strlen(string)' without space for a null terminator <zero-space-terminator>`.
- Find out more about QL in the `QL language handbook <https://help.semmle.com/QL/ql-handbook/index.html>`__ and `QL language specification <https://help.semmle.com/QL/QLLanguageSpecification.html>`__.
- Learn more about the query console in `Using the query console <https://lgtm.com/help/lgtm/using-query-console>`__.

39
docs/language/learn-ql/cpp/ql-for-cpp.rst Normal file
View File

@@ -0,0 +1,39 @@
QL for C/C++
============
.. toctree::
:glob:
:hidden:
introduce-libraries-cpp
function-classes
expressions-types
conversions-classes
dataflow
private-field-initialization
zero-space-terminator
These topics provide an overview of the QL C/C++ standard libraries and show examples of how to write queries that use them.
- `Basic C/C++ QL query <https://lgtm.com/help/lgtm/console/ql-cpp-basic-example>`__ describes how to write and run queries using LGTM.
- :doc:`Introducing the QL libraries for C/C++ <introduce-libraries-cpp>` introduces the standard libraries used to write queries for C and C++ code.
- :doc:`Tutorial: Function classes <function-classes>` demonstrates how to write queries using the standard QL library classes for C/C++ functions.
- :doc:`Tutorial: Expressions, types and statements <expressions-types>` demonstrates how to write queries using the standard QL library classes for C/C++ expressions, types and statements.
- :doc:`Tutorial: Conversions and classes <conversions-classes>` demonstrates how to write queries using the standard QL library classes for C/C++ conversions and classes.
- :doc:`Tutorial: Analyzing data flow in C/C++ <dataflow>` demonstrates how to write queries using the standard QL for C/C++ data flow and taint tracking libraries.
- :doc:`Example: Checking that constructors initialize all private fields <private-field-initialization>` works through the development of a query. It introduces recursive predicates and shows the typical workflow used to refine a query.
- :doc:`Example: Checking for allocations equal to strlen(string) without space for a null terminator <zero-space-terminator>` shows how a query to detect this particular buffer issue was developed.
Other resources
---------------
- For examples of how to query common C/C++ elements, see the `C/C++ QL cookbook <https://help.semmle.com/wiki/display/CBCPP>`__.
- For the queries used in LGTM, display a `C/C++ query <https://lgtm.com/search?q=language%3Acpp&t=rules>`__ and click **Open in query console** to see the QL code used to find alerts.
- For more information about the C/C++ QL library see the `QL library for C/C++ <https://help.semmle.com/qldoc/cpp>`__.

227
docs/language/learn-ql/cpp/zero-space-terminator.rst Normal file
View File

@@ -0,0 +1,227 @@
Example: Checking for allocations equal to ``strlen(string)`` without space for a null terminator
=================================================================================================
Overview
--------
This topic describes how a C/C++ query for detecting a potential buffer overflow was developed. For a full overview of the topics available for learning to write QL queries for C/C++ code, see :doc:`QL for C/C++ <ql-for-cpp>`.
Problem—detecting memory allocation that omits space for a null termination character
-------------------------------------------------------------------------------------
The objective of this query is to detect C/C++ code which allocates an amount of memory equal to the length of a null terminated string, without adding +1 to make room for a null termination character. For example the following code demonstrates this mistake, and results in a buffer overflow:
.. code-block:: cpp
void processString(const char *input)
{
char *buffer = malloc(strlen(input));
strcpy(buffer, input);
...
}
Basic query
-----------
Before you can write a query you need to decide what entities to search for and then define how to identify them.
Defining the entities of interest
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
You could approach this problem either by searching for code similar to the call to ``malloc`` in line 3 or the call to ``strcpy`` in line 5 (see example above). For our basic query, we start with a simple assumption: any call to ``malloc`` with only a ``strlen`` to define the memory size is likely to cause an error when the memory is populated.
Calls to ``strlen`` can be identified using the library `StrlenCall <https://help.semmle.com/qldoc/cpp/semmle/code/cpp/commons/StringAnalysis.qll/type.StringAnalysis$StrlenCall.html>`__ class, but we need to define a new class to identify calls to ``malloc``. Both the library class and the new class need to extend the standard QL class ``FunctionCall``, with the added restriction of the function name that they apply to:
.. code-block:: ql
import cpp
class MallocCall extends FunctionCall
{
MallocCall() { this.getTarget().hasGlobalName("malloc") }
}
.. pull-quote::
Note
You could easily extend this class to include similar functions such as ``realloc``, or your own custom allocator. With a little effort they could even include C++ ``new`` expressions (to do this, ``MallocCall`` would need to extend a common superclass of both ``FunctionCall`` and ``NewExpr``, such as ``Expr``).
Finding the ``strlen(string)`` pattern
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Before we start to write our query, there's one remaining task. We need to modify our new ``MallocCall`` class, so it returns an expression for the size of the allocation. Currently this will be the first argument to the ``malloc`` call, ``FunctionCall.getArgument(0)``, but converting this into a QL predicate makes it more flexible for future refinements.
.. code-block:: ql
class MallocCall extends FunctionCall
{
MallocCall() { this.getTarget().hasGlobalName("malloc") }
Expr getAllocatedSize() {
result = this.getArgument(0)
}
}
Defining the basic query
~~~~~~~~~~~~~~~~~~~~~~~~
Now we can write a query using these classes:
.. code-block:: ql
import cpp
class MallocCall extends FunctionCall
{
MallocCall() { this.getTarget().hasGlobalName("malloc") }
Expr getAllocatedSize() {
result = this.getArgument(0)
}
}
from MallocCall malloc
where malloc.getAllocatedSize() instanceof StrlenCall
select malloc, "This allocation does not include space to null-terminate the string."
Note that there is no need to check whether anything is added to the ``strlen`` expression, as it would be in the corrected C code ``malloc(strlen(string) + 1)``. This is because the corrected code would in fact be an ``AddExpr`` containing a ``StrlenCall``, not an instance of ``StrlenCall`` itself. A side-effect of this approach is that we omit certain unlikely patterns such as ``malloc(strlen(string) + 0``). In practice we can always come back and extend our query to cover this pattern if it is a concern.
For some projects, this query may not return any results. Possibly the project you are querying does not have any problems of this kind, but it is also important to make sure the query itself is working properly. One solution is to set up a test project with examples of correct and incorrect code to run the query against (the C code at the very top of this page makes a good starting point). Another approach is to test each part of the query individually to make sure everything is working.
When you have defined the basic query then you can refine the query to include further coding patterns or to exclude false positives:
Improving the query using the 'SSA' library
-------------------------------------------
The ``SSA`` library represents variables in `static single assignment <http://en.wikipedia.org/wiki/Static_single_assignment_form>`__ (SSA) form. In this form, each variable is assigned exactly once and every variable is defined before it is used. The use of SSA variables simplifies queries considerably as much of the local data flow analysis has been done for us.
Including examples where the string size is stored before use
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The query above works for simple cases, but does not identify a common coding pattern where ``strlen(string)`` is stored in a variable before being passed to ``malloc``, as in the following example:
.. code-block:: cpp
int len = strlen(input);
buffer = malloc(len);
To identify this case we can use the standard QL library ``SSA.qll`` (imported as ``semmle.code.cpp.controlflow.SSA``).
This library helps us identify where values assigned to local variables may subsequently be used.
For example, consider the following code:
.. code-block:: cpp
void myFunction(bool condition)
{
const char* x = "alpha"; // definition #1 of x
printf("x = %s\n", x); // use #1 of x
if (condition)
{
x = "beta"; // definition #2 of x
} else {
x = "gamma"; // definition #3 of x
}
printf("x = %s\n", x); // use #2 of x
}
If we run the following query on the code, we get three results:
.. code-block:: ql
import cpp
import semmle.code.cpp.controlflow.SSA
from Variable var, Expr defExpr, Expr use
where exists(SsaDefinition ssaDef |
defExpr = ssaDef.getAnUltimateDefiningValue(var)
and use = ssaDef.getAUse(var))
select var, defExpr.getLocation().getStartLine() as dline, use.getLocation().getStartLine() as uline
**Results:**
+---------+-----------+-----------+
| ``var`` | ``dline`` | ``uline`` |
+=========+===========+===========+
| ``x`` | 3 | 5 |
+---------+-----------+-----------+
| ``x`` | 9 | 14 |
+---------+-----------+-----------+
| ``x`` | 11 | 14 |
+---------+-----------+-----------+
It is often useful to also display the defining expression ``defExpr``, if there is one. For example we might adjust the query above as follows:
.. code-block:: ql
import cpp
import semmle.code.cpp.controlflow.SSA
from Variable var, Expr defExpr, Expr use
where exists(SsaDefinition ssaDef |
defExpr = ssaDef.getAnUltimateDefiningValue(var)
and use = ssaDef.getAUse(var))
select var, defExpr.getLocation().getStartLine() as dline, use.getLocation().getStartLine() as uline, defExpr
Now we can see the assigned expression in our results:
+---------+-----------+-----------+-------------+
| ``var`` | ``dline`` | ``uline`` | ``defExpr`` |
+=========+===========+===========+=============+
| ``x`` | 3 | 5 | alpha |
+---------+-----------+-----------+-------------+
| ``x`` | 9 | 14 | beta |
+---------+-----------+-----------+-------------+
| ``x`` | 11 | 14 | gamma |
+---------+-----------+-----------+-------------+
Extending the query to include allocations passed via a variable
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Using our experiments above we can expand our simple implementation of ``MallocCall.getAllocatedSize()``. With the following refinement, if the argument is an access to a variable, ``getAllocatedSize()`` returns a value assigned to that variable instead of the variable access itself:
.. code-block:: ql
Expr getAllocatedSize() {
if this.getArgument(0) instanceof VariableAccess then
exists(LocalScopeVariable v, SsaDefinition ssaDef |
result = ssaDef.getAnUltimateDefiningValue(v)
and this.getArgument(0) = ssaDef.getAUse(v))
else
result = this.getArgument(0)
}
The completed query will now identify cases where the result of ``strlen`` is stored in a local variable before it is used in a call to ``malloc``. Here is the query in full:
.. code-block:: ql
import cpp
class MallocCall extends FunctionCall
{
MallocCall() { this.getTarget().hasGlobalName("malloc") }
Expr getAllocatedSize() {
if this.getArgument(0) instanceof VariableAccess then
exists(LocalScopeVariable v, SsaDefinition ssaDef |
result = ssaDef.getAnUltimateDefiningValue(v)
and this.getArgument(0) = ssaDef.getAUse(v))
else
result = this.getArgument(0)
}
}
from MallocCall malloc
where malloc.getAllocatedSize() instanceof StrlenCall
select malloc, "This allocation does not include space to null-terminate the string."
What next?
----------
- Find out more about QL in the `QL language handbook <https://help.semmle.com/QL/ql-handbook/index.html>`__ and `QL language specification <https://help.semmle.com/QL/QLLanguageSpecification.html>`__.
- Learn more about the query console in `Using the query console <https://lgtm.com/help/lgtm/using-query-console>`__.