hohn/codeql
1
0
Fork 0
mirror of https://github.com/github/codeql.git synced 2025-12-16 08:43:11 +01:00
Code Issues Packages Projects Releases Wiki Activity

Merge pull request #11471 from github/rc/3.8

Browse Source 
Merge rc/3.8 into main
This commit is contained in:
Arthur Baars
2022-11-29 12:57:34 +01:00
committed by GitHub
parent bc6f0c1622 5898615f5a
commit cf7ebe2fa8
48 changed files with 162 additions and 434 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
.github/ISSUE_TEMPLATE/ql---general.md vendored
View File

@@ -10,5 +10,5 @@ assignees: ''
**Description of the issue**
<!-- Please explain briefly what is the problem.
If it is about an LGTM project, please include its URL.-->
If it is about a GitHub project, please include its URL. -->

4
cpp/ql/lib/definitions.qll
View File

@@ -12,8 +12,8 @@ import IDEContextual
*
* In some cases it is preferable to modify locations (the
* `hasLocationInfo()` predicate) so that they are short, and
* non-overlapping with other locations that might be highlighted in
* the LGTM interface.
* non-overlapping with other locations that might be reported as
* code scanning alerts on GitHub.
*
* We need to give locations that may not be in the database, so
* we use `hasLocationInfo()` rather than `getLocation()`.

4
cpp/ql/src/Likely Bugs/RedundantNullCheckSimple.ql
View File

@@ -12,8 +12,8 @@
*/
/*
* Note: this query is not assigned a precision yet because we don't want it on
* LGTM until its performance is well understood.
* Note: this query is not assigned a precision yet because we don't want it
* to be included in query suites until its performance is well understood.
*/
import cpp

2
cpp/ql/src/Metrics/Dependencies/ExternalDependencies.qll
View File

@@ -52,7 +52,7 @@ class Library extends LibraryT {
// The versions reported for C/C++ dependencies are just the versions that
// happen to be installed on the system where the build takes place.
// Reporting those versions is likely to cause misunderstandings, both for
// people reading them and for the vulnerability checker of lgtm.
// people reading them and for vulnerability checkers.
result = "unknown"
}

6
docs/codeql/codeql-language-guides/analyzing-control-flow-in-python.rst
View File

@@ -47,7 +47,7 @@ Example finding unreachable AST nodes
where not exists(node.getAFlowNode())
select node
➤ `See this in the query console on LGTM.com <https://lgtm.com/query/669220024/>`__. The demo projects on LGTM.com all have some code that has no control flow node, and is therefore unreachable. However, since the ``Module`` class is also a subclass of the ``AstNode`` class, the query also finds any modules implemented in C or with no source code. Therefore, it is better to find all unreachable statements.
Many codebases have some code that has no control flow node, and is therefore unreachable. However, since the ``Module`` class is also a subclass of the ``AstNode`` class, the query also finds any modules implemented in C or with no source code. Therefore, it is better to find all unreachable statements.
Example finding unreachable statements
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
@@ -60,7 +60,7 @@ Example finding unreachable statements
where not exists(s.getAFlowNode())
select s
➤ `See this in the query console on LGTM.com <https://lgtm.com/query/670720181/>`__. This query gives fewer results, but most of the projects have some unreachable nodes. These are also highlighted by the standard "Unreachable code" query. For more information, see `Unreachable code <https://lgtm.com/rules/3980095>`__ on LGTM.com.
This query should give fewer results. You can also find unreachable code using the standard "Unreachable code" query. For more information, see `Unreachable code <https://codeql.github.com/codeql-query-help/python/py-unreachable-statement/>`__.
The ``BasicBlock`` class
------------------------
@@ -114,7 +114,7 @@ Example finding mutually exclusive blocks within the same function
)
select b1, b2
➤ `See this in the query console on LGTM.com <https://lgtm.com/query/671000028/>`__. This typically gives a very large number of results, because it is a common occurrence in normal control flow. It is, however, an example of the sort of control-flow analysis that is possible. Control-flow analyses such as this are an important aid to data flow analysis. For more information, see ":doc:`Analyzing data flow in Python <analyzing-data-flow-in-python>`."
This typically gives a very large number of results, because it is a common occurrence in normal control flow. It is, however, an example of the sort of control-flow analysis that is possible. Control-flow analyses such as this are an important aid to data flow analysis. For more information, see ":doc:`Analyzing data flow in Python <analyzing-data-flow-in-python>`."
Further reading
---------------

18
docs/codeql/codeql-language-guides/analyzing-data-flow-in-python.rst
View File

@@ -97,11 +97,9 @@ Python has builtin functionality for reading and writing files, such as the func
call = API::moduleImport("os").getMember("open").getACall()
select call.getArg(0)
➤ `See this in the query console on LGTM.com <https://lgtm.com/query/8635258505893505141/>`__. Two of the demo projects make use of this low-level API.
Notice the use of the ``API`` module for referring to library functions. For more information, see ":doc:`Using API graphs in Python <using-api-graphs-in-python>`."
Unfortunately this will only give the expression in the argument, not the values which could be passed to it. So we use local data flow to find all expressions that flow into the argument:
Unfortunately this query will only give the expression in the argument, not the values which could be passed to it. So we use local data flow to find all expressions that flow into the argument:
.. code-block:: ql
@@ -115,9 +113,7 @@ Unfortunately this will only give the expression in the argument, not the values
DataFlow::localFlow(expr, call.getArg(0))
select call, expr
➤ `See this in the query console on LGTM.com <https://lgtm.com/query/8213643003890447109/>`__. Many expressions flow to the same call.
We see that we get several data-flow nodes for an expression as it flows towards a call (notice repeated locations in the ``call`` column). We are mostly interested in the "first" of these, what might be called the local source for the file name. To restrict attention to such local sources, and to simultaneously make the analysis more performant, we have the QL class ``LocalSourceNode``. We could demand that ``expr`` is such a node:
Typically, you will see several data-flow nodes for an expression as it flows towards a call (notice repeated locations in the ``call`` column). We are mostly interested in the "first" of these, what might be called the local source for the file name. To restrict attention to such local sources, and to simultaneously make the analysis more performant, we have the QL class ``LocalSourceNode``. We could demand that ``expr`` is such a node:
.. code-block:: ql
@@ -160,9 +156,9 @@ As an alternative, we can ask more directly that ``expr`` is a local source of t
expr = call.getArg(0).getALocalSource()
select call, expr
➤ `See this in the query console on LGTM.com <https://lgtm.com/query/6602079735954016687/>`__. All these three queries give identical results. We now mostly have one expression per call.
These three queries all give identical results. We now mostly have one expression per call.
We still have some cases of more than one expression flowing to a call, but then they flow through different code paths (possibly due to control-flow splitting, as in the second case).
We still have some cases of more than one expression flowing to a call, but then they flow through different code paths (possibly due to control-flow splitting).
We might want to make the source more specific, for example a parameter to a function or method. This query finds instances where a parameter is used as the name when opening a file:
@@ -178,7 +174,7 @@ We might want to make the source more specific, for example a parameter to a fun
DataFlow::localFlow(p, call.getArg(0))
select call, p
➤ `See this in the query console on LGTM.com <https://lgtm.com/query/3998032643497238063/>`__. Very few results now; these could feasibly be inspected manually.
For most codebases, this will return only a few results and these could be inspected manually.
Using the exact name supplied via the parameter may be too strict. If we want to know if the parameter influences the file name, we can use taint tracking instead of data flow. This query finds calls to ``os.open`` where the filename is derived from a parameter:
@@ -194,7 +190,7 @@ Using the exact name supplied via the parameter may be too strict. If we want to
TaintTracking::localTaint(p, call.getArg(0))
select call, p
➤ `See this in the query console on LGTM.com <https://lgtm.com/query/2129957933670836953/>`__. Now we get more results and in more projects.
Typically, this finds more results.
Global data flow
----------------
@@ -369,8 +365,6 @@ This data flow configuration tracks data flow from environment variables to open
select fileOpen, "This call to 'os.open' uses data from $@.",
environment, "call to 'os.getenv'"
➤ `Running this in the query console on LGTM.com <https://lgtm.com/query/6582374907796191895/>`__ unsurprisingly yields no results in the demo projects.
Further reading
---------------

8
docs/codeql/codeql-language-guides/annotations-in-java.rst
View File

@@ -51,7 +51,7 @@ We could then write this query to find all ``@SuppressWarnings`` annotations att
anntp.hasQualifiedName("java.lang", "SuppressWarnings")
select ann, ann.getValue("value")
➤ `See the full query in the query console on LGTM.com <https://lgtm.com/query/1775658606775222283/>`__. Several of the LGTM.com demo projects use the ``@SuppressWarnings`` annotation. Looking at the ``value``\ s of the annotation element returned by the query, we can see that the *apache/activemq* project uses the ``"rawtypes"`` value described above.
If the codebase you are analyzing uses the ``@SuppressWarnings`` annotation, you can check the ``value``\ s of the annotation element returned by the query. They should use the ``"rawtypes"`` value described above.
As another example, this query finds all annotation types that only have a single annotation element, which has name ``value``:
@@ -66,8 +66,6 @@ As another example, this query finds all annotation types that only have a singl
)
select anntp
➤ `See the full query in the query console on LGTM.com <https://lgtm.com/query/2145264152490258283/>`__.
Example: Finding missing ``@Override`` annotations
--------------------------------------------------
@@ -124,7 +122,7 @@ This makes it very easy to write our query for finding methods that override ano
not overriding.getAnAnnotation() instanceof OverrideAnnotation
select overriding, "Method overrides another method, but does not have an @Override annotation."
➤ `See this in the query console on LGTM.com <https://lgtm.com/query/7419756266089837339/>`__. In practice, this query may yield many results from compiled library code, which aren't very interesting. It's therefore a good idea to add another conjunct ``overriding.fromSource()`` to restrict the result to only report methods for which source code is available.
In practice, this query may yield many results from compiled library code, which aren't very interesting. It's therefore a good idea to add another conjunct ``overriding.fromSource()`` to restrict the result to only report methods for which source code is available.
Example: Finding calls to deprecated methods
--------------------------------------------
@@ -237,7 +235,7 @@ Now we can extend our query to filter out calls in methods carrying a ``Suppress
and not call.getCaller().getAnAnnotation() instanceof SuppressDeprecationWarningAnnotation
select call, "This call invokes a deprecated method."
➤ `See this in the query console on LGTM.com <https://lgtm.com/query/8706367340403790260/>`__. It's fairly common for projects to contain calls to methods that appear to be deprecated.
It's fairly common for projects to contain calls to methods that appear to be deprecated.
Further reading
---------------

2
docs/codeql/codeql-language-guides/codeql-for-cpp.rst
View File

@@ -21,7 +21,7 @@ Experiment and learn how to write effective and efficient queries for CodeQL dat
hash-consing-and-value-numbering
- :doc:`Basic query for C and C++ code <basic-query-for-cpp-code>`: Learn to write and run a simple CodeQL query using LGTM.
- :doc:`Basic query for C and C++ code <basic-query-for-cpp-code>`: Learn to write and run a simple CodeQL query.
- :doc:`CodeQL library for C and C++ <codeql-library-for-cpp>`: When analyzing C or C++ code, you can use the large collection of classes in the CodeQL library for C and C++.

2
docs/codeql/codeql-language-guides/codeql-for-csharp.rst
View File

@@ -12,7 +12,7 @@ Experiment and learn how to write effective and efficient queries for CodeQL dat
codeql-library-for-csharp
analyzing-data-flow-in-csharp
- :doc:`Basic query for C# code <basic-query-for-csharp-code>`: Learn to write and run a simple CodeQL query using LGTM.
- :doc:`Basic query for C# code <basic-query-for-csharp-code>`: Learn to write and run a simple CodeQL query.
- :doc:`CodeQL library for C# <codeql-library-for-csharp>`: When you're analyzing a C# program, you can make use of the large collection of classes in the CodeQL library for C#.

2
docs/codeql/codeql-language-guides/codeql-for-go.rst
View File

@@ -13,7 +13,7 @@ Experiment and learn how to write effective and efficient queries for CodeQL dat
abstract-syntax-tree-classes-for-working-with-go-programs
modeling-data-flow-in-go-libraries
- :doc:`Basic query for Go code <basic-query-for-go-code>`: Learn to write and run a simple CodeQL query using LGTM.
- :doc:`Basic query for Go code <basic-query-for-go-code>`: Learn to write and run a simple CodeQL query.
- :doc:`CodeQL library for Go <codeql-library-for-go>`: When you're analyzing a Go program, you can make use of the large collection of classes in the CodeQL library for Go.

2
docs/codeql/codeql-language-guides/codeql-for-java.rst
View File

@@ -26,7 +26,7 @@ Experiment and learn how to write effective and efficient queries for CodeQL dat
working-with-source-locations
abstract-syntax-tree-classes-for-working-with-java-programs
- :doc:`Basic query for Java code <basic-query-for-java-code>`: Learn to write and run a simple CodeQL query using LGTM.
- :doc:`Basic query for Java code <basic-query-for-java-code>`: Learn to write and run a simple CodeQL query.
- :doc:`CodeQL library for Java <codeql-library-for-java>`: When analyzing Java code, you can use the large collection of classes in the CodeQL library for Java.

2
docs/codeql/codeql-language-guides/codeql-for-javascript.rst
View File

@@ -18,7 +18,7 @@ Experiment and learn how to write effective and efficient queries for CodeQL dat
abstract-syntax-tree-classes-for-working-with-javascript-and-typescript-programs
data-flow-cheat-sheet-for-javascript
- :doc:`Basic query for JavaScript code <basic-query-for-javascript-code>`: Learn to write and run a simple CodeQL query using LGTM.
- :doc:`Basic query for JavaScript code <basic-query-for-javascript-code>`: Learn to write and run a simple CodeQL query.
- :doc:`CodeQL library for JavaScript <codeql-library-for-javascript>`: When you're analyzing a JavaScript program, you can make use of the large collection of classes in the CodeQL library for JavaScript.

2
docs/codeql/codeql-language-guides/codeql-for-python.rst
View File

@@ -16,7 +16,7 @@ Experiment and learn how to write effective and efficient queries for CodeQL dat
expressions-and-statements-in-python
analyzing-control-flow-in-python
- :doc:`Basic query for Python code <basic-query-for-python-code>`: Learn to write and run a simple CodeQL query using LGTM.
- :doc:`Basic query for Python code <basic-query-for-python-code>`: Learn to write and run a simple CodeQL query.
- :doc:`CodeQL library for Python <codeql-library-for-python>`: When you need to analyze a Python program, you can make use of the large collection of classes in the CodeQL library for Python.

2
docs/codeql/codeql-language-guides/codeql-for-ruby.rst
View File

@@ -14,7 +14,7 @@ Experiment and learn how to write effective and efficient queries for CodeQL dat
analyzing-data-flow-in-ruby
using-api-graphs-in-ruby
- :doc:`Basic query for Ruby code <basic-query-for-ruby-code>`: Learn to write and run a simple CodeQL query using LGTM.
- :doc:`Basic query for Ruby code <basic-query-for-ruby-code>`: Learn to write and run a simple CodeQL query.
- :doc:`CodeQL library for Ruby <codeql-library-for-ruby>`: When you're analyzing a Ruby program, you can make use of the large collection of classes in the CodeQL library for Ruby.

32
docs/codeql/codeql-language-guides/codeql-library-for-java.rst
View File

@@ -70,7 +70,7 @@ For example, the following query finds all variables of type ``int`` in the prog
pt.hasName("int")
select v
➤ `See this in the query console on LGTM.com <https://lgtm.com/query/860076406167044435/>`__. You're likely to get many results when you run this query because most projects contain many variables of type ``int``.
You're likely to get many results when you run this query because most projects contain many variables of type ``int``.
Reference types are also categorized according to their declaration scope:
@@ -87,7 +87,7 @@ For instance, this query finds all top-level types whose name is not the same as
where tl.getName() != tl.getCompilationUnit().getName()
select tl
➤ `See this in the query console on LGTM.com <https://lgtm.com/query/4340983612585284460/>`__. This pattern is seen in many projects. When we ran it on the LGTM.com demo projects, most of the projects had at least one instance of this problem in the source code. There were many more instances in the files referenced by the source code.
You will typically see this pattern in the source code of a repository, with many more instances in the files referenced by the source code.
Several more specialized classes are available as well:
@@ -109,7 +109,7 @@ As an example, we can write a query that finds all nested classes that directly
where nc.getASupertype() instanceof TypeObject
select nc
➤ `See this in the query console on LGTM.com <https://lgtm.com/query/8482509736206423238/>`__. You're likely to get many results when you run this query because many projects include nested classes that extend ``Object`` directly.
You're likely to get many results when you run this query because many projects include nested classes that extend ``Object`` directly.
Generics
~~~~~~~~
@@ -143,8 +143,6 @@ For instance, we could use the following query to find all parameterized instanc
pt.getSourceDeclaration() = map
select pt
➤ `See this in the query console on LGTM.com <https://lgtm.com/query/7863873821043873550/>`__. None of the LGTM.com demo projects contain parameterized instances of ``java.util.Map`` in their source code, but they all have results in reference files.
In general, generic types may restrict which types a type parameter can be bound to. For instance, a type of maps from strings to numbers could be declared as follows:
.. code-block:: java
@@ -166,8 +164,6 @@ As an example, the following query finds all type variables with type bound ``Nu
tb.getType().hasQualifiedName("java.lang", "Number")
select tv
➤ `See this in the query console on LGTM.com <https://lgtm.com/query/6740696080876162817/>`__. When we ran it on the LGTM.com demo projects, the *neo4j/neo4j*, *hibernate/hibernate-orm* and *apache/hadoop* projects all contained examples of this pattern.
For dealing with legacy code that is unaware of generics, every generic type has a "raw" version without any type parameters. In the CodeQL libraries, raw types are represented using class ``RawType``, which has the expected subclasses ``RawClass`` and ``RawInterface``. Again, there is a predicate ``getSourceDeclaration`` for obtaining the corresponding generic type. As an example, we can find variables of (raw) type ``Map``:
.. code-block:: ql
@@ -179,8 +175,6 @@ For dealing with legacy code that is unaware of generics, every generic type has
rt.getSourceDeclaration().hasQualifiedName("java.util", "Map")
select v
➤ `See this in the query console on LGTM.com <https://lgtm.com/query/4032913402499547882/>`__. Many projects have variables of raw type ``Map``.
For example, in the following code snippet this query would find ``m1``, but not ``m2``:
.. code-block:: java
@@ -230,7 +224,7 @@ For example, the following query finds all expressions whose parents are ``retur
where e.getParent() instanceof ReturnStmt
select e
➤ `See this in the query console on LGTM.com <https://lgtm.com/query/1947757851560375919/>`__. Many projects have examples of ``return`` statements with child expressions.
Many projects have examples of ``return`` statements with child expressions.
Therefore, if the program contains a return statement ``return x + y;``, this query will return ``x + y``.
@@ -244,7 +238,7 @@ As another example, the following query finds statements whose parent is an ``if
where s.getParent() instanceof IfStmt
select s
➤ `See this in the query console on LGTM.com <https://lgtm.com/query/1989464153689219612/>`__. Many projects have examples of ``if`` statements with child statements.
Many projects have examples of ``if`` statements with child statements.
This query will find both ``then`` branches and ``else`` branches of all ``if`` statements in the program.
@@ -258,8 +252,6 @@ Finally, here is a query that finds method bodies:
where s.getParent() instanceof Method
select s
➤ `See this in the query console on LGTM.com <https://lgtm.com/query/1016821702972128245/>`__. Most projects have many method bodies.
As these examples show, the parent node of an expression is not always an expression: it may also be a statement, for example, an ``IfStmt``. Similarly, the parent node of a statement is not always a statement: it may also be a method or a constructor. To capture this, the QL Java library provides two abstract class ``ExprParent`` and ``StmtParent``, the former representing any node that may be the parent node of an expression, and the latter any node that may be the parent node of a statement.
For more information on working with AST classes, see the :doc:`article on overflow-prone comparisons in Java <overflow-prone-comparisons-in-java>`.
@@ -278,7 +270,7 @@ For annotations, class ``Annotatable`` is a superclass of all program elements t
from Constructor c
select c.getAnAnnotation()
➤ `See this in the query console on LGTM.com <https://lgtm.com/query/3206112561297137365/>`__. The LGTM.com demo projects all use annotations, you can see examples where they are used to suppress warnings and mark code as deprecated.
You may see examples where annotations are used to suppress warnings or to mark code as deprecated.
These annotations are represented by class ``Annotation``. An annotation is simply an expression whose type is an ``AnnotationType``. For example, you can amend this query so that it only reports deprecated constructors:
@@ -292,7 +284,7 @@ These annotations are represented by class ``Annotation``. An annotation is simp
anntp.hasQualifiedName("java.lang", "Deprecated")
select ann
➤ `See this in the query console on LGTM.com <https://lgtm.com/query/5393027107459215059/>`__. Only constructors with the ``@Deprecated`` annotation are reported this time.
Only constructors with the ``@Deprecated`` annotation are reported this time.
For more information on working with annotations, see the :doc:`article on annotations <annotations-in-java>`.
@@ -307,7 +299,7 @@ For Javadoc, class ``Element`` has a member predicate ``getDoc`` that returns a
jdoc = f.getDoc().getJavadoc()
select jdoc
➤ `See this in the query console on LGTM.com <https://lgtm.com/query/6022769142134600659/>`__. You can see this pattern in many projects.
You can see this pattern in many projects.
Class ``Javadoc`` represents an entire Javadoc comment as a tree of ``JavadocElement`` nodes, which can be traversed using member predicates ``getAChild`` and ``getParent``. For instance, you could edit the query so that it finds all ``@author`` tags in Javadoc comments on private fields:
@@ -321,8 +313,6 @@ Class ``Javadoc`` represents an entire Javadoc comment as a tree of ``JavadocEle
at.getParent+() = jdoc
select at
➤ `See this in the query console on LGTM.com <https://lgtm.com/query/2510220694395289111/>`__. None of the LGTM.com demo projects uses the ``@author`` tag on private fields.
.. pull-quote::
Note
@@ -349,7 +339,7 @@ For example, the following query finds methods with a `cyclomatic complexity <ht
mc.getCyclomaticComplexity() > 40
select m
➤ `See this in the query console on LGTM.com <https://lgtm.com/query/6566950741051181919/>`__. Most large projects include some methods with a very high cyclomatic complexity. These methods are likely to be difficult to understand and test.
Most large projects include some methods with a very high cyclomatic complexity. These methods are likely to be difficult to understand and test.
Call graph
----------
@@ -369,8 +359,6 @@ We can use predicate ``Call.getCallee`` to find out which method or constructor
m.hasName("println")
select c
➤ `See this in the query console on LGTM.com <https://lgtm.com/query/5861255162551917595/>`__. The LGTM.com demo projects all include many calls to methods of this name.
Conversely, ``Callable.getAReference`` returns a ``Call`` that refers to it. So we can find methods and constructors that are never called using this query:
.. code-block:: ql
@@ -381,7 +369,7 @@ Conversely, ``Callable.getAReference`` returns a ``Call`` that refers to it. So
where not exists(c.getAReference())
select c
➤ `See this in the query console on LGTM.com <https://lgtm.com/query/7261739919657747703/>`__. The LGTM.com demo projects all appear to have many methods that are not called directly, but this is unlikely to be the whole story. To explore this area further, see ":doc:`Navigating the call graph <navigating-the-call-graph>`."
Codebases often have many methods that are not called directly, but this is unlikely to be the whole story. To explore this area further, see ":doc:`Navigating the call graph <navigating-the-call-graph>`."
For more information about callables and calls, see the :doc:`article on the call graph <navigating-the-call-graph>`.

38
docs/codeql/codeql-language-guides/codeql-library-for-javascript.rst
View File

@@ -43,7 +43,7 @@ Textual level
At its most basic level, a JavaScript code base can simply be viewed as a collection of files organized into folders, where each file is composed of zero or more lines of text.
Note that the textual content of a program is not included in the CodeQL database unless you specifically request it during extraction. In particular, databases on LGTM (also known as "snapshots") do not normally include textual information.
Note that the textual content of a program is not included in the CodeQL database unless you specifically request it during extraction.
Files and folders
^^^^^^^^^^^^^^^^^
@@ -77,7 +77,7 @@ For example, the following query computes, for each folder, the number of JavaSc
from Folder d
select d.getRelativePath(), count(File f | f = d.getAFile() and f.getExtension() = "js")
➤ `See this in the query console on LGTM.com <https://lgtm.com/query/1506075865985/>`__. When you run the query on most projects, the results include folders that contain files with a ``js`` extension and folders that don't.
When you run the query on most projects, the results include folders that contain files with a ``js`` extension and folders that don't.
Locations
^^^^^^^^^
@@ -138,7 +138,7 @@ As an example of a query operating entirely on the lexical level, consider the f
where comma.getNextToken() instanceof CommaToken
select comma, "Omitted array elements are bad style."
➤ `See this in the query console on LGTM.com <https://lgtm.com/query/659662177/>`__. If the query returns no results, this pattern isn't used in the projects that you analyzed.
If the query returns no results, this pattern isn't used in the projects that you analyzed.
You can use predicate ``Locatable.getFirstToken()`` and ``Locatable.getLastToken()`` to access the first and last token (if any) belonging to an element with a source location.
@@ -179,8 +179,6 @@ As an example of a query using only lexical information, consider the following
from HtmlLineComment c
select c, "Do not use HTML comments."
➤ `See this in the query console on LGTM.com <https://lgtm.com/query/686330023/>`__. When we ran this query on the *mozilla/pdf.js* project in LGTM.com, we found three HTML comments.
Syntactic level
~~~~~~~~~~~~~~~
@@ -230,7 +228,7 @@ The `TopLevel <https://codeql.github.com/codeql-standard-libraries/javascript/se
Note
By default, LGTM filters out alerts in minified top-levels, since they are often hard to interpret. When writing your own queries in the LGTM query console, this filtering is *not* done automatically, so you may want to explicitly add a condition of the form ``and not e.getTopLevel().isMinified()`` or similar to your query to exclude results in minified code.
By default, GitHub code scanning filters out alerts in minified top-levels, since they are often hard to interpret. When you write your own queries in Visual Studio Code, this filtering is *not* done automatically, so you may want to explicitly add a condition of the form ``and not e.getTopLevel().isMinified()`` or similar to your query to exclude results in minified code.
Statements and expressions
^^^^^^^^^^^^^^^^^^^^^^^^^^
@@ -351,8 +349,6 @@ As an example of how to use expression AST nodes, here is a query that finds exp
where add = shift.getAnOperand()
select add, "This expression should be bracketed to clarify precedence rules."
➤ `See this in the query console on LGTM.com <https://lgtm.com/query/690010024/>`__. When we ran this query on the *meteor/meteor* project in LGTM.com, we found many results where precedence could be clarified using brackets.
Functions
^^^^^^^^^
@@ -373,8 +369,6 @@ As an example, here is a query that finds all expression closures:
where fe.getBody() instanceof Expr
select fe, "Use arrow expressions instead of expression closures."
➤ `See this in the query console on LGTM.com <https://lgtm.com/query/668510056/>`__. None of the LGTM.com demo projects uses expression closures, but you may find this query gets results on other projects.
As another example, this query finds functions that have two parameters that bind the same variable:
.. code-block:: ql
@@ -388,8 +382,6 @@ As another example, this query finds functions that have two parameters that bin
p.getAVariable() = q.getAVariable()
select fun, "This function has two parameters that bind the same variable."
➤ `See this in the query console on LGTM.com <https://lgtm.com/query/673860037/>`__. None of the LGTM.com demo projects has functions where two parameters bind the same variable.
Classes
^^^^^^^
@@ -444,7 +436,7 @@ Here is an example of a query to find declaration statements that declare the sa
not ds.getTopLevel().isMinified()
select ds, "Variable " + v.getName() + " is declared both $@ and $@.", d1, "here", d2, "here"
➤ `See this in the query console on LGTM.com <https://lgtm.com/query/668700496/>`__. This is not a common problem, so you may not find any results in your own projects. The *angular/angular.js* project on LGTM.com has one instance of this problem at the time of writing.
This is not a common problem, so you may not find any results in your own projects.
Notice the use of ``not ... isMinified()`` here and in the next few queries. This excludes any results found in minified code. If you delete ``and not ds.getTopLevel().isMinified()`` and re-run the query, two results in minified code in the *meteor/meteor* project are reported.
@@ -471,8 +463,6 @@ As an example of a query involving properties, consider the following query that
not oe.getTopLevel().isMinified()
select oe, "Property " + p1.getName() + " is defined both $@ and $@.", p1, "here", p2, "here"
➤ `See this in the query console on LGTM.com <https://lgtm.com/query/660700064/>`__. Many projects have a few instances of object expressions with two identically named properties.
Modules
^^^^^^^
@@ -537,7 +527,7 @@ As an example, consider the following query which finds distinct function declar
not g.getTopLevel().isMinified()
select f, g
➤ `See this in the query console on LGTM.com <https://lgtm.com/query/667290067/>`__. Some projects declare conflicting functions of the same name and rely on platform-specific behavior to disambiguate the two declarations.
Some projects declare conflicting functions of the same name and rely on platform-specific behavior to disambiguate the two declarations.
Control flow
~~~~~~~~~~~~
@@ -574,7 +564,7 @@ As an example of an analysis using basic blocks, ``BasicBlock.isLiveAtEntry(v, u
not f.getStartBB().isLiveAtEntry(gv, _)
select f, "This function uses " + gv + " like a local variable."
➤ `See this in the query console on LGTM.com <https://lgtm.com/query/686320048/>`__. Many projects have some variables which look as if they were intended to be local.
Many projects have some variables which look as if they were intended to be local.
Data flow
~~~~~~~~~
@@ -599,8 +589,6 @@ As an example, the following query finds definitions of local variables that are
not exists (VarUse use | def = use.getADef())
select def, "Dead store of local variable."
➤ `See this in the query console on LGTM.com <https://lgtm.com/query/2086440429/>`__. Many projects have some examples of useless assignments to local variables.
SSA
^^^
@@ -642,8 +630,6 @@ For example, here is a query that finds all invocations of a method called ``sen
send.getMethodName() = "send"
select send
➤ `See this in the query console on LGTM.com <https://lgtm.com/query/1506058347056/>`__. The query finds HTTP response sends in the `AMP HTML <https://lgtm.com/projects/g/ampproject/amphtml>`__ project.
Note that the data flow modeling in this library is intraprocedural, that is, flow across function calls and returns is *not* modeled. Likewise, flow through object properties and global variables is not modeled.
Type inference
@@ -707,8 +693,6 @@ As an example of a call-graph-based query, here is a query to find invocations f
not exists(invk.getACallee())
select invk, "Unable to find a callee for this invocation."
➤ `See this in the query console on LGTM.com <https://lgtm.com/query/3260345690335671362/>`__
Inter-procedural data flow
~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -843,7 +827,7 @@ As an example of the use of these classes, here is a query that counts for every
from NodeModule m
select m, count(m.getAnImportedModule())
➤ `See this in the query console on LGTM.com <https://lgtm.com/query/659662207/>`__. When you analyze a project, for each module you can see how many other modules it imports.
When you analyze a project, for each module you can see how many other modules it imports.
NPM
^^^
@@ -872,8 +856,6 @@ As an example of the use of these classes, here is a query that identifies unuse
not exists (Require req | req.getTopLevel() = pkg.getAModule() | name = req.getImportedPath().getValue())
select deps, "Unused dependency '" + name + "'."
➤ `See this in the query console on LGTM.com <https://lgtm.com/query/666680077/>`__. It is not uncommon for projects to have some unused dependencies.
React
^^^^^
@@ -899,8 +881,6 @@ For example, here is a query to find SQL queries that use string concatenation (
where ss instanceof AddExpr
select ss, "Use templating instead of string concatenation."
➤ `See this in the query console on LGTM.com <https://lgtm.com/query/1506076336224/>`__, showing two (benign) results on `strong-arc <https://lgtm.com/projects/g/strongloop/strong-arc/>`__.
Miscellaneous
~~~~~~~~~~~~~
@@ -965,8 +945,6 @@ As an example, here is a query that finds ``@param`` tags that do not specify th
not exists(t.getName())
select t, "@param tag is missing name."
➤ `See this in the query console on LGTM.com <https://lgtm.com/query/673060054/>`__. Of the LGTM.com demo projects analyzed, only *Semantic-Org/Semantic-UI* has an example where the ``@param`` tag omits the name.
For full details on these and other classes representing JSDoc comments and type expressions, see `the API documentation <https://codeql.github.com/codeql-standard-libraries/javascript/semmle/javascript/JSDoc.qll/module.JSDoc.html>`__.
JSX

10
docs/codeql/codeql-language-guides/codeql-library-for-python.rst
View File

@@ -53,7 +53,7 @@ All scopes are basically a list of statements, although ``Scope`` classes have a
where f.getScope() instanceof Function
select f
➤ `See this in the query console on LGTM.com <https://lgtm.com/query/665620040/>`__. Many projects have nested functions.
Many codebases use nested functions.
Statement
^^^^^^^^^
@@ -95,7 +95,7 @@ As an example, to find expressions of the form ``a+2`` where the left is a simpl
where bin.getLeft() instanceof Name and bin.getRight() instanceof Num
select bin
➤ `See this in the query console on LGTM.com <https://lgtm.com/query/669950026/>`__. Many projects include examples of this pattern.
Many codebases include examples of this pattern.
Variable
^^^^^^^^
@@ -126,7 +126,7 @@ For our first example, we can find all ``finally`` blocks by using the ``Try`` c
from Try t
select t.getFinalbody()
➤ `See this in the query console on LGTM.com <https://lgtm.com/query/659662193/>`__. Many projects include examples of this pattern.
Many codebases include examples of this pattern.
2. Finding ``except`` blocks that do nothing
''''''''''''''''''''''''''''''''''''''''''''
@@ -157,7 +157,7 @@ Both forms are equivalent. Using the positive expression, the whole query looks
where forall(Stmt s | s = ex.getAStmt() | s instanceof Pass)
select ex
➤ `See this in the query console on LGTM.com <https://lgtm.com/query/690010036/>`__. Many projects include pass-only ``except`` blocks.
Many codebases include pass-only ``except`` blocks.
Summary
^^^^^^^
@@ -278,8 +278,6 @@ Using this predicate we can select the longest ``BasicBlock`` by selecting the `
where bb_length(b) = max(bb_length(_))
select b
➤ `See this in the query console on LGTM.com <https://lgtm.com/query/666730036/>`__. When we ran it on the LGTM.com demo projects, the *openstack/nova* and *ytdl-org/youtube-dl* projects both contained source code results for this query.
.. pull-quote::
Note

10
docs/codeql/codeql-language-guides/conversions-and-classes-in-cpp.rst
View File

@@ -165,8 +165,6 @@ Our starting point for the query is pairs of a base class and a derived class, c
where derived.getABaseClass+() = base
select base, derived, "The second class is derived from the first."
➤ `See this in the query console on LGTM.com <https://lgtm.com/query/1505902347211/>`__
Note that the transitive closure symbol ``+`` indicates that ``Class.getABaseClass()`` may be followed one or more times, rather than only accepting a direct base class.
A lot of the results are uninteresting template parameters. You can remove those results by updating the ``where`` clause as follows:
@@ -177,8 +175,6 @@ A lot of the results are uninteresting template parameters. You can remove those
and not exists(base.getATemplateArgument())
and not exists(derived.getATemplateArgument())
➤ `See this in the query console on LGTM.com <https://lgtm.com/query/1505907047251/>`__
Finding derived classes with destructors
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -196,8 +192,6 @@ Now we can extend the query to find derived classes with destructors, using the
and d2 = derived.getDestructor()
select base, derived, "The second class is derived from the first, and both have a destructor."
➤ `See this in the query console on LGTM.com <https://lgtm.com/query/1505901767389/>`__
Notice that getting the destructor implicitly asserts that one exists. As a result, this version of the query returns fewer results than before.
Finding base classes where the destructor is not virtual
@@ -216,11 +210,9 @@ Our last change is to use ``Function.isVirtual()`` to find cases where the base
and not d1.isVirtual()
select d1, "This destructor should probably be virtual."
➤ `See this in the query console on LGTM.com <https://lgtm.com/query/1505908156827/>`__
That completes the query.
There is a similar built-in `query <https://lgtm.com/rules/2158670642/>`__ on LGTM.com that finds classes in a C/C++ project with virtual functions but no virtual destructor. You can take a look at the code for this query by clicking **Open in query console** at the top of that page.
There is a similar standard query `Non-virtual destructor in base class <https://codeql.github.com/codeql-query-help/cpp/cpp-virtual-destructor/>`__ that finds classes in a C/C++ project with virtual functions but no virtual destructor.
Further reading
---------------

10
docs/codeql/codeql-language-guides/expressions-and-statements-in-python.rst
View File

@@ -54,8 +54,6 @@ The ``global`` statement in Python declares a variable with a global (module-lev
where g.getScope() instanceof Module
select g
➤ `See this in the query console on LGTM.com <https://lgtm.com/query/686330052/>`__. None of the demo projects on LGTM.com has a global statement that matches this pattern.
The line: ``g.getScope() instanceof Module`` ensures that the ``Scope`` of ``Global g`` is a ``Module``, rather than a class or function.
Example finding 'if' statements with redundant branches
@@ -81,7 +79,7 @@ To find statements like this that could be simplified we can write a query.
and forall(Stmt p | p = l.getAnItem() | p instanceof Pass)
select i
➤ `See this in the query console on LGTM.com <https://lgtm.com/query/672230053/>`__. Many projects have some ``if`` statements that match this pattern.
Many codebases have some ``if`` statements that match this pattern.
The line: ``(l = i.getBody() or l = i.getOrelse())`` restricts the ``StmtList l`` to branches of the ``if`` statement.
@@ -150,8 +148,6 @@ We can check for these using a query.
and cmp.getOp(0) instanceof Is and cmp.getComparator(0) = literal
select cmp
➤ `See this in the query console on LGTM.com <https://lgtm.com/query/688180010/>`__. Two of the demo projects on LGTM.com use this pattern: *saltstack/salt* and *openstack/nova*.
The clause ``cmp.getOp(0) instanceof Is and cmp.getComparator(0) = literal`` checks that the first comparison operator is "is" and that the first comparator is a literal.
.. pull-quote::
@@ -180,7 +176,7 @@ If there are duplicate keys in a Python dictionary, then the second key will ove
and k1 != k2 and same_key(k1, k2)
select k1, "Duplicate key in dict literal"
➤ `See this in the query console on LGTM.com <https://lgtm.com/query/663330305/>`__. When we ran this query on LGTM.com, the source code of the *saltstack/salt* project contained an example of duplicate dictionary keys. The results were also highlighted as alerts by the standard "Duplicate key in dict literal" query. Two of the other demo projects on LGTM.com refer to duplicate dictionary keys in library files. For more information, see `Duplicate key in dict literal <https://lgtm.com/rules/3980087>`__ on LGTM.com.
When we ran this query on some test codebases, we found examples of duplicate dictionary keys. The results were also highlighted as alerts by the standard "Duplicate key in dict literal" query. For more information, see `Duplicate key in dict literal <https://codeql.github.com/codeql-query-help/python/py-duplicate-key-dict-literal/>`__.
The supporting predicate ``same_key`` checks that the keys have the same identifier. Separating this part of the logic into a supporting predicate, instead of directly including it in the query, makes it easier to understand the query as a whole. The casts defined in the predicate restrict the expression to the type specified and allow predicates to be called on the type that is cast-to. For example:
@@ -222,8 +218,6 @@ This basic query can be improved by checking that the one line of code is a Java
and attr.getObject() = self and self.getId() = "self"
select f, "This function is a Java-style getter."
➤ `See this in the query console on LGTM.com <https://lgtm.com/query/669220054/>`__. Of the demo projects on LGTM.com, only the *openstack/nova* project has examples of functions that appear to be Java-style getters.
.. code-block:: ql
ret = f.getStmt(0) and ret.getValue() = attr

10
docs/codeql/codeql-language-guides/expressions-types-and-statements-in-cpp.rst
View File

@@ -23,8 +23,6 @@ In the following example we find instances of ``AssignExpr`` which assign the co
where e.getRValue().getValue().toInt() = 0
select e, "Assigning the value 0 to something."
➤ `See this in the query console on LGTM.com <https://lgtm.com/query/1505908086530/>`__
The ``where`` clause in this example gets the expression on the right side of the assignment, ``getRValue()``, and compares it with zero. Notice that there are no checks to make sure that the right side of the assignment is an integer or that it has a value (that is, it is compile-time constant, rather than a variable). For expressions where either of these assumptions is wrong, the associated predicate simply does not return anything and the ``where`` clause will not produce a result. You could think of it as if there is an implicit ``exists(e.getRValue().getValue().toInt())`` at the beginning of this line.
It is also worth noting that the query above would find this C code:
@@ -33,7 +31,7 @@ It is also worth noting that the query above would find this C code:
yPtr = NULL;
This is because the database contains a representation of the code base after the preprocessor transforms have run. This means that any macro invocations, such as the ``NULL`` define used here, are expanded during the creation of the database. If you want to write queries about macros then there are some special library classes that have been designed specifically for this purpose (for example, the ``Macro``, ``MacroInvocation`` classes and predicates like ``Element.isInMacroExpansion()``). In this case, it is good that macros are expanded, but we do not want to find assignments to pointers. For more information, see `Database generation <https://lgtm.com/help/lgtm/generate-database>`__ on LGTM.com.
This is because the database contains a representation of the code base after the preprocessor transforms have run. This means that any macro invocations, such as the ``NULL`` define used here, are expanded during the creation of the database. If you want to write queries about macros then there are some special library classes that have been designed specifically for this purpose (for example, the ``Macro``, ``MacroInvocation`` classes and predicates like ``Element.isInMacroExpansion()``). In this case, it is good that macros are expanded, but we do not want to find assignments to pointers. For more information, see `Database creation <https://codeql.github.com/docs/codeql-overview/about-codeql/#database-creation>`__.
Finding assignments of 0 to an integer
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -49,8 +47,6 @@ We can make the query more specific by defining a condition for the left side of
and e.getLValue().getType().getUnspecifiedType() instanceof IntegralType
select e, "Assigning the value 0 to an integer."
➤ `See this in the query console on LGTM.com <https://lgtm.com/query/1505906986578/>`__
This checks that the left side of the assignment has a type that is some kind of integer. Note the call to ``Type.getUnspecifiedType()``. This resolves ``typedef`` types to their underlying types so that the query finds assignments like this one:
.. code-block:: cpp
@@ -109,8 +105,6 @@ Unfortunately this would not quite work, because the loop initialization is actu
and e.getLValue().getType().getUnspecifiedType() instanceof IntegralType
select e, "Assigning the value 0 to an integer, inside a for loop initialization."
➤ `See this in the query console on LGTM.com <https://lgtm.com/query/1505909016965/>`__
Finding assignments of 0 within the loop body
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -127,8 +121,6 @@ We can find assignments inside the loop body using similar code with the predica
and e.getLValue().getType().getUnderlyingType() instanceof IntegralType
select e, "Assigning the value 0 to an integer, inside a for loop body."
➤ `See this in the query console on LGTM.com <https://lgtm.com/query/1505901437190/>`__
Note that we replaced ``e.getEnclosingStmt()`` with ``e.getEnclosingStmt().getParentStmt*()``, to find an assignment expression that is deeply nested inside the loop body. The transitive closure modifier ``*`` here indicates that ``Stmt.getParentStmt()`` may be followed zero or more times, rather than just once, giving us the statement, its parent statement, its parent's parent statement etc.
Further reading

12
docs/codeql/codeql-language-guides/functions-in-cpp.rst
View File

@@ -40,8 +40,6 @@ It might be more interesting to find functions that are not called, using the st
where not exists(FunctionCall fc | fc.getTarget() = f)
select f, "This function is never called."
➤ `See this in the query console on LGTM.com <https://lgtm.com/query/1505891246456/>`__
The new query finds functions that are not the target of any ``FunctionCall``—in other words, functions that are never called. You may be surprised by how many results the query finds. However, if you examine the results, you can see that many of the functions it finds are used indirectly. To create a query that finds only unused functions, we need to refine the query and exclude other ways of using a function.
Excluding functions that are referenced with a function pointer
@@ -58,13 +56,11 @@ You can modify the query to remove functions where a function pointer is used to
and not exists(FunctionAccess fa | fa.getTarget() = f)
select f, "This function is never called, or referenced with a function pointer."
➤ `See this in the query console on LGTM.com <https://lgtm.com/query/1505890446605/>`__
This query returns fewer results. However, if you examine the results then you can probably still find potential refinements.
For example, there is a more complicated LGTM `query <https://lgtm.com/rules/2152580467/>`__ that finds unused static functions. To see the code for this query, click **Open in query console** at the top of the page.
For example, there is a more complicated standard query, `Unused static function <https://codeql.github.com/codeql-query-help/cpp/cpp-unused-static-function/>`__, that finds unused static functions.
You can explore the definition of an element in the standard libraries and see what predicates are available. Use the keyboard **F3** button to open the definition of any element. Alternatively, hover over the element and click **Jump to definition** in the tooltip displayed. The library file is opened in a new tab with the definition highlighted.
You can explore the definition of an element in the standard libraries and see what predicates are available. Right-click the element to display the context menu, and click **Go to Definition**. The library file is opened in a new tab with the definition of the element highlighted.
Finding a specific function
---------------------------
@@ -80,8 +76,6 @@ This query uses ``Function`` and ``FunctionCall`` to find calls to the function
and not fc.getArgument(1) instanceof StringLiteral
select fc, "sprintf called with variable format string."
➤ `See this in the query console on LGTM.com <https://lgtm.com/query/1505889506751/>`__
This uses:
- ``Declaration.getQualifiedName()`` to identify calls to the specific function ``sprintf``.
@@ -89,7 +83,7 @@ This uses:
Note that we could have used ``Declaration.getName()``, but ``Declaration.getQualifiedName()`` is a better choice because it includes the namespace. For example: ``getName()`` would return ``vector`` where ``getQualifiedName`` would return ``std::vector``.
The LGTM version of this query is considerably more complicated, but if you look carefully you will find that its structure is the same. See `Non-constant format string <https://lgtm.com/rules/2152810612/>`__ and click **Open in query console** at the top of the page.
The published version of this query is considerably more complicated, but if you look carefully you will find that its structure is the same. See `Non-constant format string <https://codeql.github.com/codeql-query-help/cpp/cpp-non-constant-format/>`__.
Further reading
---------------

8
docs/codeql/codeql-language-guides/functions-in-python.rst
View File

@@ -28,7 +28,7 @@ Using the member predicate ``Function.getName()``, we can list all of the getter
where f.getName().matches("get%")
select f, "This is a function called get..."
➤ `See this in the query console on LGTM.com <https://lgtm.com/query/669220031/>`__. This query typically finds a large number of results. Usually, many of these results are for functions (rather than methods) which we are not interested in.
This query typically finds a large number of results. Usually, many of these results are for functions (rather than methods) which we are not interested in.
Finding all methods called "get..."
-----------------------------------
@@ -43,7 +43,7 @@ You can modify the query above to return more interesting results. As we are onl
where f.getName().matches("get%") and f.isMethod()
select f, "This is a method called get..."
➤ `See this in the query console on LGTM.com <https://lgtm.com/query/690010035/>`__. This finds methods whose name starts with ``"get"``, but many of those are not the sort of simple getters we are interested in.
This finds methods whose name starts with ``"get"``, but many of those are not the sort of simple getters we are interested in.
Finding one line methods called "get..."
----------------------------------------
@@ -59,7 +59,7 @@ We can modify the query further to include only methods whose body consists of a
and count(f.getAStmt()) = 1
select f, "This function is (probably) a getter."
➤ `See this in the query console on LGTM.com <https://lgtm.com/query/667290044/>`__. This query returns fewer results, but if you examine the results you can see that there are still refinements to be made. This is refined further in ":doc:`Expressions and statements in Python <expressions-and-statements-in-python>`."
This query returns fewer results, but if you examine the results you can see that there are still refinements to be made. This is refined further in ":doc:`Expressions and statements in Python <expressions-and-statements-in-python>`."
Finding a call to a specific function
-------------------------------------
@@ -74,8 +74,6 @@ This query uses ``Call`` and ``Name`` to find calls to the function ``eval`` - w
where call.getFunc() = name and name.getId() = "eval"
select call, "call to 'eval'."
➤ `See this in the query console on LGTM.com <https://lgtm.com/query/6718356557331218618/>`__. Some of the demo projects on LGTM.com use this function.
The ``Call`` class represents calls in Python. The ``Call.getFunc()`` predicate gets the expression being called. ``Name.getId()`` gets the identifier (as a string) of the ``Name`` expression.
Due to the dynamic nature of Python, this query will select any call of the form ``eval(...)`` regardless of whether it is a call to the built-in function ``eval`` or not.
In a later tutorial we will see how to use the type-inference library to find calls to the built-in function ``eval`` regardless of name of the variable called.

4
docs/codeql/codeql-language-guides/javadoc.rst
View File

@@ -149,8 +149,6 @@ Now we can write a query for finding all callables ``c`` and ``@throws`` tags ``
not mayThrow(c, exn)
select tt, "Spurious @throws tag."
➤ `See this in the query console on LGTM.com <https://lgtm.com/query/1258570917227966396/>`__. This finds several results in the LGTM.com demo projects.
Improvements
~~~~~~~~~~~~
@@ -216,7 +214,7 @@ The first case can be covered by changing ``getDocumentedException`` to use the
(result.hasName(tt.getExceptionName()) and visibleIn(tt.getFile(), result))
}
➤ `See this in the query console on LGTM.com <https://lgtm.com/query/8016848987103345329/>`__. This finds many fewer, more interesting results in the LGTM.com demo projects.
This query should find many fewer, more interesting results.
Currently, ``visibleIn`` only considers single-type imports, but you could extend it with support for other kinds of imports.

40
docs/codeql/codeql-language-guides/overflow-prone-comparisons-in-java.rst
View File

@@ -44,7 +44,7 @@ We'll start by writing a query that finds less-than expressions (CodeQL class ``
expr.getRightOperand().getType().hasName("long")
select expr
➤ `See this in the query console on LGTM.com <https://lgtm.com/query/490866529746563234/>`__. This query usually finds results on most projects.
This query usually finds results on most codebases.
Notice that we use the predicate ``getType`` (available on all subclasses of ``Expr``) to determine the type of the operands. Types, in turn, define the ``hasName`` predicate, which allows us to identify the primitive types ``int`` and ``long``. As it stands, this query finds *all* less-than expressions comparing ``int`` and ``long``, but in fact we are only interested in comparisons that are part of a loop condition. Also, we want to filter out comparisons where either operand is constant, since these are less likely to be real bugs. The revised query looks like this:
@@ -59,7 +59,7 @@ Notice that we use the predicate ``getType`` (available on all subclasses of ``E
not expr.getAnOperand().isCompileTimeConstant()
select expr
➤ `See this in the query console on LGTM.com <https://lgtm.com/query/4315986481180063825/>`__. Notice that fewer results are found.
Notice that fewer results are found.
The class ``LoopStmt`` is a common superclass of all loops, including, in particular, ``for`` loops as in our example above. While different kinds of loops have different syntax, they all have a loop condition, which can be accessed through predicate ``getCondition``. We use the reflexive transitive closure operator ``*`` applied to the ``getAChildExpr`` predicate to express the requirement that ``expr`` should be nested inside the loop condition. In particular, it can be the loop condition itself.
@@ -113,17 +113,45 @@ Now we rewrite our query to make use of these new classes:
.. code-block:: ql
import Java
import java
// Insert the class definitions from above
// Return the width (in bits) of a given integral type
int width(PrimitiveType pt) {
(pt.hasName("byte") and result=8) or
(pt.hasName("short") and result=16) or
(pt.hasName("char") and result=16) or
(pt.hasName("int") and result=32) or
(pt.hasName("long") and result=64)
}
// Find any comparison where the width of the type on the smaller end of
// the comparison is less than the width of the type on the greater end
abstract class OverflowProneComparison extends ComparisonExpr {
Expr getLesserOperand() { none() }
Expr getGreaterOperand() { none() }
}
// Return `<=` and `<` comparisons
class LTOverflowProneComparison extends OverflowProneComparison {
LTOverflowProneComparison() {
(this instanceof LEExpr or this instanceof LTExpr) and
width(this.getLeftOperand().getType()) < width(this.getRightOperand().getType())
}
}
// Return `>=` and `>` comparisons
class GTOverflowProneComparison extends OverflowProneComparison {
GTOverflowProneComparison() {
(this instanceof GEExpr or this instanceof GTExpr) and
width(this.getRightOperand().getType()) < width(this.getLeftOperand().getType())
}
}
from OverflowProneComparison expr
where exists(LoopStmt l | l.getCondition().getAChildExpr*() = expr) and
not expr.getAnOperand().isCompileTimeConstant()
select expr
➤ `See the full query in the query console on LGTM.com <https://lgtm.com/query/506868054626167462/>`__.
Further reading
---------------

2
docs/codeql/codeql-language-guides/refining-a-query-to-account-for-edge-cases.rst
View File

@@ -146,8 +146,6 @@ Finally we can simplify the query by using the transitive closure operator. In t
and exists(c.getBlock())
select c, "Constructor does not initialize fields $@.", f, f.getName()
➤ `See this in the query console on LGTM.com <https://lgtm.com/query/1505896968215/>`__
Further reading
---------------

45
docs/codeql/codeql-language-guides/types-in-java.rst
View File

@@ -34,7 +34,7 @@ To determine ancestor types (including immediate super types, and also *their* s
where B.hasName("B")
select B.getASupertype+()
➤ `See this in the query console on LGTM.com <https://lgtm.com/query/1506430738755934285/>`__. If this query were run on the example snippet above, the query would return ``A``, ``I``, and ``java.lang.Object``.
If we ran this query on the example snippet above, the query would return ``A``, ``I``, and ``java.lang.Object``.
.. pull-quote::
@@ -80,7 +80,7 @@ This recipe is not too difficult to translate into a query:
target.getElementType().(RefType).getASupertype+() = source.getElementType()
select ce, "Potentially problematic array downcast."
➤ `See this in the query console on LGTM.com <https://lgtm.com/query/8378564667548381869/>`__. Many projects return results for this query.
Many projects return results for this query.
Note that by casting ``target.getElementType()`` to a ``RefType``, we eliminate all cases where the element type is a primitive type, that is, ``target`` is an array of primitive type: the problem we are looking for cannot arise in that case. Unlike in Java, a cast in QL never fails: if an expression cannot be cast to the desired type, it is simply excluded from the query results, which is exactly what we want.
@@ -143,7 +143,7 @@ Using these new classes we can extend our query to exclude calls to ``toArray``
not ce.getExpr().(CollectionToArrayCall).getActualReturnType() = target
select ce, "Potentially problematic array downcast."
➤ `See this in the query console on LGTM.com <https://lgtm.com/query/3150404889854131463/>`__. Notice that fewer results are found by this improved query.
Notice that fewer results are found by this improved query.
Example: Finding mismatched contains checks
-------------------------------------------
@@ -269,8 +269,6 @@ Now we are ready to write a first version of our query:
not haveCommonDescendant(collEltType, argType)
select juccc, "Element type " + collEltType + " is incompatible with argument type " + argType
➤ `See this in the query console on LGTM.com <https://lgtm.com/query/7947831380785106258/>`__.
Improvements
~~~~~~~~~~~~
@@ -286,7 +284,40 @@ Adding these three improvements, our final query becomes:
import java
// Insert the class definitions from above
class JavaUtilCollection extends GenericInterface {
JavaUtilCollection() {
this.hasQualifiedName("java.util", "Collection")
}
}
class JavaUtilCollectionContains extends Method {
JavaUtilCollectionContains() {
this.getDeclaringType() instanceof JavaUtilCollection and
this.hasStringSignature("contains(Object)")
}
}
class JavaUtilCollectionContainsCall extends MethodAccess {
JavaUtilCollectionContainsCall() {
exists(JavaUtilCollectionContains jucc |
this.getMethod().getSourceDeclaration().overrides*(jucc)
)
}
Type getArgumentType() {
result = this.getArgument(0).getType()
}
Type getCollectionElementType() {
exists(RefType D, ParameterizedInterface S |
D = this.getMethod().getDeclaringType() and
D.hasSupertype*(S) and S.getSourceDeclaration() instanceof JavaUtilCollection and
result = S.getTypeArgument(0)
)
}
}
predicate haveCommonDescendant(RefType tp1, RefType tp2) {
exists(RefType commondesc | commondesc.hasSupertype*(tp1) and commondesc.hasSupertype*(tp2))
}
from JavaUtilCollectionContainsCall juccc, Type collEltType, Type argType
where collEltType = juccc.getCollectionElementType() and argType = juccc.getArgumentType() and
@@ -296,8 +327,6 @@ Adding these three improvements, our final query becomes:
not argType.hasName("<nulltype>")
select juccc, "Element type " + collEltType + " is incompatible with argument type " + argType
➤ `See the full query in the query console on LGTM.com <https://lgtm.com/query/8846334903769538099/>`__.
Further reading
---------------

12
docs/codeql/codeql-language-guides/using-api-graphs-in-python.rst
View File

@@ -29,8 +29,6 @@ following snippet demonstrates.
select API::moduleImport("re")
➤ `See this in the query console on LGTM.com <https://lgtm.com/query/1876172022264324639/>`__.
This query selects the API graph node corresponding to the ``re`` module. This node represents the fact that the ``re`` module has been imported rather than a specific location in the program where the import happens. Therefore, there will be at most one result per project, and it will not have a useful location, so you'll have to click `Show 1 non-source result` in order to see it.
To find where the ``re`` module is referenced in the program, you can use the ``getAUse`` method. The following query selects all references to the ``re`` module in the current database.
@@ -42,8 +40,6 @@ To find where the ``re`` module is referenced in the program, you can use the ``
select API::moduleImport("re").getAUse()
➤ `See this in the query console on LGTM.com <https://lgtm.com/query/8072356519514905526/>`__.
Note that the ``getAUse`` method accounts for local flow, so that ``my_re_compile``
in the following snippet is
correctly recognized as a reference to the ``re.compile`` function.
@@ -77,8 +73,6 @@ the above ``re.compile`` example, you can now find references to ``re.compile``.
select API::moduleImport("re").getMember("compile").getAUse()
➤ `See this in the query console on LGTM.com <https://lgtm.com/query/7970570434725297676/>`__.
In addition to ``getMember``, you can use the ``getUnknownMember`` method to find references to API
components where the name is not known statically. You can use the ``getAMember`` method to
access all members, both known and unknown.
@@ -97,15 +91,11 @@ where the return value of ``re.compile`` is used:
select API::moduleImport("re").getMember("compile").getReturn().getAUse()
➤ `See this in the query console on LGTM.com <https://lgtm.com/query/4346050399960356921/>`__.
Note that this includes all uses of the result of ``re.compile``, including those reachable via
local flow. To get just the *calls* to ``re.compile``, you can use ``getAnImmediateUse`` instead of
``getAUse``. As this is a common occurrence, you can use ``getACall`` instead of
``getReturn`` followed by ``getAnImmediateUse``.
➤ `See this in the query console on LGTM.com <https://lgtm.com/query/8143347716552092926/>`__.
Note that the API graph does not distinguish between class instantiations and function calls. As far
as it's concerned, both are simply places where an API graph node is called.
@@ -134,8 +124,6 @@ all subclasses of ``View``, you must explicitly include the subclasses of ``Meth
select viewClass().getAUse()
➤ `See this in the query console on LGTM.com <https://lgtm.com/query/288293322319747121/>`__.
Note the use of the set literal ``["View", "MethodView"]`` to match both classes simultaneously.
Built-in functions and classes

4
docs/codeql/codeql-overview/supported-languages-and-frameworks.rst
View File

@@ -16,7 +16,7 @@ CodeQL library packs (`source <https://github.com/github/codeql/tree/codeql-cli/
and CodeQL bundle (`releases <https://github.com/github/codeql-action/releases>`__)
support the following languages and compilers.
.. include:: ../support/reusables/versions-compilers.rst
.. include:: ../reusables/supported-versions-compilers.rst
Frameworks and libraries
########################
@@ -31,4 +31,4 @@ The current versions of the CodeQL library and query packs (`source <https://git
For example, by extending the data flow libraries to include data sources
and sinks for additional libraries or frameworks.
.. include:: ../support/reusables/frameworks.rst
.. include:: ../reusables/supported-frameworks.rst

2
docs/codeql/codeql-overview/system-requirements.rst
View File

@@ -10,7 +10,7 @@ System requirements for running the latest version of CodeQL.
Supported platforms
#######################
.. include:: ../support/reusables/platforms.rst
.. include:: ../reusables/supported-platforms.rst
.. include:: ../reusables/kotlin-beta-note.rst

1
docs/codeql/ql-language-reference/annotations.rst
View File

@@ -433,7 +433,6 @@ The ``bindingset`` annotation takes a comma-separated list of variables.
(for characteristic predicates and member predicates) and ``result`` (for predicates that return a result).
For more information, see ":ref:`predicate-binding`."
- When you annotate a class, each variable must be ``this`` or a field in the class.
Binding sets for classes are supported from release 2.3.0 of the CodeQL CLI, and release 1.26 of LGTM Enterprise.
.. Links to use in substitutions

4
docs/codeql/ql-language-reference/expressions.rst
View File

@@ -117,8 +117,6 @@ The values of the contained expressions need to be of :ref:`compatible types <ty
Furthermore, at least one of the set elements has to be of a type that is a supertype of the types of all
the other contained expressions.
Set literals are supported from release 2.1.0 of the CodeQL CLI, and release 1.24 of LGTM Enterprise.
.. index:: super
.. _super:
@@ -341,8 +339,6 @@ The following aggregates are available in QL:
where x in [-5 .. 5] and x != 0
select unique(int y | y = x or y = x.abs() | y)
The ``unique`` aggregate is supported from release 2.1.0 of the CodeQL CLI, and release 1.24 of LGTM Enterprise.
Evaluation of aggregates
========================

4
docs/codeql/ql-language-reference/ql-language-specification.rst
View File

@@ -1356,9 +1356,7 @@ Set literals can be of any type, but the types within a set literal have to be c
The values of a set literal expression are all the values of all the contained element expressions.
Set literals are supported from release 2.1.0 of the CodeQL CLI, and release 1.24 of LGTM Enterprise.
Since release 2.7.1 of the CodeQL CLI, and release 1.30 of LGTM Enterprise, a trailing comma is allowed in a set literal.
Since release 2.7.1 of the CodeQL CLI, a trailing comma is allowed in a set literal.
Disambiguation of expressions
-----------------------------

2
docs/codeql/ql-training/template.rst
View File

@@ -41,7 +41,7 @@ For this example you should download:
.. note::
Some notes about the project, perhaps a link to the project page on LGTM.
Some notes about the project.
.. Agenda slide. Explaining what is to be covered in the presentation

0
docs/codeql/support/reusables/frameworks.rst → docs/codeql/reusables/supported-frameworks.rst
View File

0
docs/codeql/support/reusables/platforms.rst → docs/codeql/reusables/supported-platforms.rst
View File

2
docs/codeql/support/reusables/versions-compilers.rst → docs/codeql/reusables/supported-versions-compilers.rst
View File

@@ -37,4 +37,4 @@
.. [7] JSX and Flow code, YAML, JSON, HTML, and XML files may also be analyzed with JavaScript files.
.. [8] The extractor requires Python 3 to run. To analyze Python 2.7 you should install both versions of Python.
.. [9] Requires glibc 2.17.
.. [10] TypeScript analysis is performed by running the JavaScript extractor with TypeScript enabled. This is the default for LGTM.
.. [10] TypeScript analysis is performed by running the JavaScript extractor with TypeScript enabled. This is the default.

101
docs/codeql/support/conf.py
View File

@@ -1,101 +0,0 @@
# -*- coding: utf-8 -*-
#
# CodeQL analysis support for LGTM Enterprise docs build configuration file.
#
# This file is execfile()d with the current directory set to its
# containing dir.
#
# Note that not all possible configuration values are present in this
# autogenerated file.
#
# All configuration values have a default; values that are commented out
# serve to show the default.
# For details of all possible config values,
# see https://www.sphinx-doc.org/en/master/usage/configuration.html
##############################################################################
#
# Modified 22032021.
# The configuration values below are specific to the supported languages and frameworks project
# To amend html_theme_options, update version/release number, or add more sphinx extensions,
# refer to code/documentation/ql-documentation/global-sphinx-files/global-conf.py
##############################################################################
# -- Project-specific configuration -----------------------------------
# Set QL as the default language for highlighting code. Set to none to disable
# syntax highlighting. If omitted or left blank, it defaults to Python 3.
highlight_language = 'none'
# The name of the Pygments (syntax highlighting) style to use.
pygments_style = 'sphinx'
# The master toctree document.
master_doc = 'index'
# Project-specific information.
project = u'Supported languages and frameworks for LGTM Enterprise'
# The version info for this project, if different from version and release in main conf.py file.
# The short X.Y version.
# LGTM Enterprise release
release = u'1.30'
# CodeQL CLI version used by LGTM Enterprise release
version = u'2.7.6'
# -- Project-specifc options for HTML output ----------------------------------------------
# The name for this set of Sphinx documents. If None, it defaults to
# "<project> v<release> documentation".
html_title = 'Supported languages and frameworks'
# Output file base name for HTML help builder.
htmlhelp_basename = 'Supported languages and frameworks'
# Add any paths that contain templates here, relative to this directory.
templates_path = ['../_templates']
# Add any paths that contain custom static files (such as style sheets) here,
# relative to this directory. They are copied after the builtin static files,
# so a file named "default.css" will overwrite the builtin "default.css".
html_static_path = ['../_static']
html_theme_options = {'font_size': '16px',
'body_text': '#333',
'link': '#2F1695',
'link_hover': '#2F1695',
'show_powered_by': False,
'nosidebar':True,
'head_font_family': '-apple-system, BlinkMacSystemFont, "Segoe UI", Helvetica, Arial, sans-serif, "Apple Color Emoji", "Segoe UI Emoji"',
}
html_favicon = '../images/site/favicon.ico'
# -- Currently unused, but potentially useful, configs--------------------------------------
# Add any paths that contain custom themes here, relative to this directory.
#html_theme_path = []
# A shorter title for the navigation bar. Default is the same as html_title.
#html_short_title = None
# The name of an image file (relative to this directory) to place at the top
# of the sidebar.
#html_logo = None
# Custom sidebar templates, maps document names to template names.
#html_sidebars = {}
# Add any extra paths that contain custom files (such as robots.txt or
# .htaccess) here, relative to this directory. These files are copied
# directly to the root of the documentation.
#html_extra_path = []
# List of patterns, relative to source directory, that match files and
# directories to ignore when looking for source files.
exclude_patterns = ['read-me-project.rst', 'reusables/*']

19
docs/codeql/support/framework-support.rst
View File

@@ -1,19 +0,0 @@
Frameworks and libraries
########################
LGTM Enterprise |release| includes CodeQL CLI |version|. The CodeQL libraries and queries used by this version of LGTM Enterprise have been explicitly checked against the libraries and frameworks listed below.
.. pull-quote::
Note
For details of framework and library support in the most recent release of the CodeQL CLI, see `Supported languages and frameworks <https://codeql.github.com/docs/codeql-overview/supported-languages-and-frameworks/>`__ in the CodeQL CLI documentation.
.. pull-quote::
Tip
If you're interested in other libraries or frameworks, you can extend the analysis to cover them.
For example, by extending the data flow libraries to include data sources and sinks for additional libraries or frameworks.
.. include:: reusables/frameworks.rst

19
docs/codeql/support/index.rst
View File

@@ -1,19 +0,0 @@
Supported languages and frameworks
##################################
These pages describe the languages and frameworks supported in the latest enterprise release of CodeQL and LGTM. (CodeQL was previously known as QL.)
Users of `LGTM.com <https://lgtm.com/>`_ may find that additional features are supported because it's updated more frequently.
For details see:
.. toctree::
language-support.rst
framework-support.rst
For details of the CodeQL libraries, see `CodeQL standard libraries <https://codeql.github.com/codeql-standard-libraries/>`_.
.. toctree::
:hidden:
ql-training

16
docs/codeql/support/language-support.rst
View File

@@ -1,16 +0,0 @@
Languages and compilers
#######################
LGTM Enterprise |release| includes CodeQL CLI |version|. LGTM Enterprise supports analysis of the following languages compiled by the following compilers.
.. pull-quote::
Note
For details of language and compiler support in the most recent release of the CodeQL CLI, see `Supported languages and frameworks <https://codeql.github.com/docs/codeql-overview/supported-languages-and-frameworks/>`__ in the CodeQL CLI documentation.
Note that where there are several versions or dialects of a language, the supported variants are listed.
If your code requires a particular version of a compiler, check that this version is included below.
If you have any questions about language and compiler support, you can find help on the `GitHub Security Lab discussions board <https://github.com/github/securitylab/discussions>`__.
.. include:: reusables/versions-compilers.rst

63
docs/codeql/support/ql-training.rst
View File

@@ -1,63 +0,0 @@
CodeQL training and variant analysis examples
=============================================
CodeQL and variant analysis
---------------------------
Variant analysis is the process of using a known vulnerability as a seed to find similar problems in your code. Security engineers typically perform variant analysis to identify possible vulnerabilities and to ensure that these threats are properly fixed across multiple code bases.
CodeQL is the code analysis engine that underpins LGTM, the community driven security analysis platform. Together, CodeQL and LGTM provide continuous monitoring and scalable variant analysis for your projects, even if you don’t have your own team of dedicated security engineers. You can read more about using CodeQL and LGTM in variant analysis on the `Security Lab research page <https://securitylab.github.com/research>`__.
CodeQL is easy to learn, and exploring code using CodeQL is the most efficient way to perform variant analysis.
Learning CodeQL for variant analysis
------------------------------------
Start learning how to use CodeQL in variant analysis for a specific language by looking at the topics below. Each topic links to a short presentation on CodeQL, its libraries, or an example variant discovered using CodeQL.
.. |arrow-l| unicode:: U+2190
.. |arrow-r| unicode:: U+2192
.. |info| unicode:: U+24D8
When you have selected a presentation, use |arrow-r| and |arrow-l| to navigate between slides.
Press **p** to view the additional notes on slides that have an information icon |info| in the top right corner, and press **f** to enter full-screen mode.
The presentations contain a number of query examples.
We recommend that you download `CodeQL for Visual Studio Code <https://codeql.github.com/docs/codeql-for-visual-studio-code/>`__ and add the example database for each presentation so that you can find the bugs mentioned in the slides.
.. pull-quote::
Information
The presentations listed below are used in CodeQL and variant analysis training sessions run by GitHub engineers.
Therefore, be aware that the slides are designed to be presented by an instructor.
If you are using the slides without an instructor, please use the additional notes to help guide you through the examples.
CodeQL and variant analysis for C/C++
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
- `Introduction to variant analysis: CodeQL for C/C++ </QL/ql-training/cpp/intro-ql-cpp.html>`__–an introduction to variant analysis and CodeQL for C/C++ programmers.
- `Example: Bad overflow guard </QL/ql-training/cpp/bad-overflow-guard.html>`__–an example of iterative query development to find bad overflow guards in a C++ project.
- `Program representation: CodeQL for C/C++ </QL/ql-training/cpp/program-representation-cpp.html>`__–information on how CodeQL analysis represents C/C++ programs.
- `Introduction to local data flow </QL/ql-training/cpp/data-flow-cpp.html>`__–an introduction to analyzing local data flow in C/C++ using CodeQL, including an example demonstrating how to develop a query to find a real CVE.
- `Exercise: snprintf overflow </QL/ql-training/cpp/snprintf.html>`__–an example demonstrating how to develop a data flow query.
- `Introduction to global data flow </QL/ql-training/cpp/global-data-flow-cpp.html>`__–an introduction to analyzing global data flow in C/C++ using CodeQL.
- `Analyzing control flow: CodeQL for C/C++ </QL/ql-training/cpp/control-flow-cpp.html>`__–an introduction to analyzing control flow in C/C++ using CodeQL.
CodeQL and variant analysis for Java
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
- `Introduction to variant analysis: CodeQL for Java </QL/ql-training/java/intro-ql-java.html>`__–an introduction to variant analysis and CodeQL for Java programmers.
- `Example: Query injection </QL/ql-training/java/query-injection-java.html>`__–an example of iterative query development to find unsanitized SPARQL injections in a Java project.
- `Program representation: CodeQL for Java </QL/ql-training/java/program-representation-java.html>`__–information on how CodeQL analysis represents Java programs.
- `Introduction to local data flow </QL/ql-training/java/data-flow-java.html>`__–an introduction to analyzing local data flow in Java using CodeQL, including an example demonstrating how to develop a query to find a real CVE.
- `Exercise: Apache Struts </QL/ql-training/java/apache-struts-java.html>`__–an example demonstrating how to develop a data flow query.
- `Introduction to global data flow </QL/ql-training/java/global-data-flow-java.html>`__–an introduction to analyzing global data flow in Java using CodeQL.
Further reading
~~~~~~~~~~~~~~~
- `GitHub Security Lab <https://securitylab.github.com/research>`__

15
docs/codeql/support/read-me-project.rst
View File

@@ -1,15 +0,0 @@
Publishing this project for a new version
#########################################
To update this project for a new version:
1. Check with the language teams that all information in the ``ql/change-notes/support/`` directory is ready.
2. Open the ``global-conf.py`` file in the ``global-sphinx-files`` directory and change the following variables
to the correct value(s) if necessary:
* ``version =``
* ``release = ``
* If it's the first release of the year, ``copyright =``
3. Commit your changes. The output of the ``doc/sphinx`` PR check should be correct for the new version and ready to publish.

8
docs/codeql/writing-codeql-queries/metadata-for-codeql-queries.rst
View File

@@ -9,7 +9,7 @@ About query metadata
--------------------
Any query that is run as part of an analysis includes a number of properties, known as query metadata. Metadata is included at the top of each query file as the content of a QLDoc comment.
This metadata tells LGTM and the CodeQL :ref:`extension for VS Code <codeql-for-visual-studio-code>` how to handle the query and display its results correctly.
This metadata tells the CodeQL :ref:`extension for VS Code <codeql-for-visual-studio-code>` and the `Code scanning feature in GitHub <https://docs.github.com/en/code-security/code-scanning/automatically-scanning-your-code-for-vulnerabilities-and-errors/about-code-scanning-with-codeql>`__ how to handle the query and display its results correctly.
It also gives other users information about what the query results mean. For more information on query metadata, see the `query metadata style guide <https://github.com/github/codeql/blob/main/docs/query-metadata-style-guide.md>`__ in our `open source repository <https://github.com/github/codeql>`__ on GitHub.
.. pull-quote::
@@ -28,7 +28,7 @@ The following properties are supported by all query files:
+=======================+===========================+=======================================================================================================================================================================================================================================================================================================================================================================+
| ``@description`` | ``<text>`` | A sentence or short paragraph to describe the purpose of the query and *why* the result is useful or important. The description is written in plain text, and uses single quotes (``'``) to enclose code elements. |
+-----------------------+---------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| ``@id`` | ``<text>`` | A sequence of words composed of lowercase letters or digits, delimited by ``/`` or ``-``, identifying and classifying the query. Each query must have a **unique** ID. To ensure this, it may be helpful to use a fixed structure for each ID. For example, the standard LGTM queries have the following format: ``<language>/<brief-description>``. |
| ``@id`` | ``<text>`` | A sequence of words composed of lowercase letters or digits, delimited by ``/`` or ``-``, identifying and classifying the query. Each query must have a **unique** ID. To ensure this, it may be helpful to use a fixed structure for each ID. For example, the standard CodeQL queries have the following format: ``<language>/<brief-description>``. |
+-----------------------+---------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| ``@kind`` | | ``problem`` | Identifies the query is an alert (``@kind problem``) or a path (``@kind path-problem``). For more information on these query types, see ":doc:`About CodeQL queries <about-codeql-queries>`." |
| | | ``path-problem`` | |
@@ -40,12 +40,12 @@ The following properties are supported by all query files:
| | | ``readability`` | |
| | | ``security`` | |
+-----------------------+---------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| ``@precision`` | | ``low`` | Indicates the percentage of query results that are true positives (as opposed to false positive results). This, along with the ``@problem.severity`` property, determines whether the results are displayed by default on LGTM. |
| ``@precision`` | | ``low`` | Indicates the percentage of query results that are true positives (as opposed to false positive results). This, along with the ``@problem.severity`` property, determines how the results are displayed on GitHub. |
| | | ``medium`` | |
| | | ``high`` | |
| | | ``very-high`` | |
+-----------------------+---------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| ``@problem.severity`` | | ``error`` | Defines the level of severity of any alerts generated by a non-security query. This, along with the ``@precision`` property, determines whether the results are displayed by default on LGTM. |
| ``@problem.severity`` | | ``error`` | Defines the level of severity of any alerts generated by a non-security query. This, along with the ``@precision`` property, determines how the results are displayed on GitHub. |
| | | ``warning`` | |
| | | ``recommendation`` | |
+-----------------------+---------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

8
docs/codeql/writing-codeql-queries/providing-locations-in-codeql-queries.rst
View File

@@ -11,7 +11,7 @@ CodeQL includes mechanisms for extracting the location of elements in a codebase
About locations
---------------
When displaying information to the user, LGTM needs to be able to extract location information from the results of a query. In order to do this, all QL classes which can provide location information should do this by using one of the following mechanisms:
When displaying information to the user, applications need to be able to extract location information from the results of a query. In order to do this, all QL classes which can provide location information should do this by using one of the following mechanisms:
- `Providing URLs <#providing-urls>`__
- `Providing location information <#providing-location-information>`__
@@ -49,7 +49,7 @@ A custom URL can be provided by defining a QL predicate returning ``string`` wit
File URLs
^^^^^^^^^
LGTM supports the display of URLs which define a line and column in a source file.
The CodeQL extension for Visual Studio Code and the code scanning views in GitHub support the display of URLs which define a line and column in a source file.
The schema is ``file://``, which is followed by the absolute path to a file, followed by four numbers separated by colons. The numbers denote start line, start column, end line and end column. Both line and column numbers are **1-based**, for example:
@@ -57,12 +57,12 @@ The schema is ``file://``, which is followed by the absolute path to a file, fol
- ``file:///opt/src/my/file.java:1:1:2:1`` denotes the location that starts at the beginning of the file and extends to the first character of the second line (the range is inclusive).
- ``file:///opt/src/my/file.java:1:0:1:0`` is taken, by convention, to denote the entire first line of the file.
By convention, the location of an entire file may also be denoted by a ``file://`` URL without trailing numbers. Optionally, the location within a file can be denoted using three numbers to define the start line number, character offset and character length of the location respectively. Results of these types are not displayed in LGTM.
By convention, the location of an entire file may also be denoted by a ``file://`` URL without trailing numbers. Optionally, the location within a file can be denoted using three numbers to define the start line number, character offset and character length of the location respectively. Results of these types are not displayed as code scanning alerts.
Other types of URL
^^^^^^^^^^^^^^^^^^
The following, less-common types of URL are valid but are not supported by LGTM and will be omitted from any results:
The following, less-common types of URL are valid but are not interpreted as code scanning alerts and will be omitted from any results:
- **HTTP URLs** are supported in some client applications. For an example, see the code snippet above.
- **Folder URLs** can be useful, for example to provide folder-level metrics. They may use a file URL, for example ``file:///opt/src:0:0:0:0``, but they may also start with a scheme of ``folder://``, and no trailing numbers, for example ``folder:///opt/src``.

2
go/ql/src/Security/CWE-601/BadRedirectCheck.qhelp
View File

@@ -7,7 +7,7 @@ Redirect URLs should be checked to ensure that user input cannot cause a site to
to arbitrary domains. This is often done with a check that the redirect URL begins with a slash,
which most of the time is an absolute redirect on the same host. However, browsers interpret URLs
beginning with <code>//</code> or <code>/\</code> as absolute URLs. For example, a redirect to
<code>//lgtm.com</code> will redirect to <code>https://lgtm.com</code>. Thus, redirect checks must
<code>//example.com</code> will redirect to <code>https://example.com</code>. Thus, redirect checks must
also check the second character of redirect URLs.
</p>
</overview>

7
python/ql/src/Imports/SyntaxError.qhelp
View File

@@ -26,9 +26,10 @@ However, it is worth investigating why a module containing a syntax error
was able to persist and address that problem as well.
</p>
<p>If you suspect that the syntax error is caused by the analysis using the
wrong version of Python, consider specifying the version explicitly. For
LGTM.com, you can customize extraction using an <code>lgtm.yml</code> file as
described <a href="https://lgtm.com/help/lgtm/python-extraction">here</a>.
wrong version of Python, consider specifying the version explicitly. When
you run code scanning using the CodeQL action, you can configure the Python
version to use. For more information, see
<a href="https://docs.github.com/en/code-security/code-scanning/automatically-scanning-your-code-for-vulnerabilities-and-errors/configuring-code-scanning#analyzing-python-dependencies">Analyzing Python dependencies</a>.
</p>
</recommendation>
<references>

4
ruby/ql/lib/codeql/Locations.qll
View File

@@ -6,7 +6,7 @@ import files.FileSystem
* A location as given by a file, a start line, a start column,
* an end line, and an end column.
*
* For more information about locations see [LGTM locations](https://codeql.github.com/docs/writing-codeql-queries/providing-locations-in-codeql-queries/).
* For more information about locations see [Locations](https://codeql.github.com/docs/writing-codeql-queries/providing-locations-in-codeql-queries/).
*/
class Location extends @location {
/** Gets the file for this location. */
@@ -40,7 +40,7 @@ class Location extends @location {
* The location spans column `startcolumn` of line `startline` to
* column `endcolumn` of line `endline` in file `filepath`.
* For more information, see
* [LGTM locations](https://codeql.github.com/docs/writing-codeql-queries/providing-locations-in-codeql-queries/).
* [Providing locations in CodeQL queries](https://codeql.github.com/docs/writing-codeql-queries/providing-locations-in-codeql-queries/).
*/
predicate hasLocationInfo(
string filepath, int startline, int startcolumn, int endline, int endcolumn

2
ruby/ql/lib/codeql/ruby/printAst.qll
View File

@@ -70,7 +70,7 @@ class PrintAstNode extends TPrintNode {
* Holds if this node is at the specified location. The location spans column
* `startcolumn` of line `startline` to column `endcolumn` of line `endline`
* in file `filepath`. For more information, see
* [LGTM locations](https://codeql.github.com/docs/writing-codeql-queries/providing-locations-in-codeql-queries/).
* [Locations](https://codeql.github.com/docs/writing-codeql-queries/providing-locations-in-codeql-queries/).
*/
predicate hasLocationInfo(
string filepath, int startline, int startcolumn, int endline, int endcolumn