Commit Graph

19517 Commits

Author SHA1 Message Date
Taus Brock-Nannestad
f6e1ea5b2a Python: Fix missing global variable source nodes
In lieu of removing the offending flow (which would likely have
consequences for a lot of other tests), I opted to simply _include_
the relevant nodes directly.
2021-02-04 18:07:13 +01:00
Taus Brock-Nannestad
2524f23a46 Python: Add more test cases
There is now a bit of redundancy in the tests, but I thought it useful
to actually include some of the cases called out explicitly in the
documentation, so as to make it easy to see that the code actually
does what we expect (in these cases, anyway).
2021-02-04 18:05:33 +01:00
Taus Brock-Nannestad
aa7e9f0b56 Python: Add big explanatory comment about prefixes. 2021-02-04 18:03:34 +01:00
Taus Brock-Nannestad
305bfaba2d Python: Fix imports/2 2021-02-04 16:46:22 +01:00
Taus Brock-Nannestad
07ffa9f1ae Python: More documentation 2021-02-04 15:59:00 +01:00
Taus Brock-Nannestad
e54c925b70 Python: Greatly simplify imports/2 predicate 2021-02-04 15:58:15 +01:00
Taus Brock-Nannestad
d01d7eea82 Python: Add documentation from DataFlowUtil::importNode 2021-02-04 13:08:19 +01:00
Taus
4627799c93 Python: Fix more typos
Co-authored-by: Rasmus Wriedt Larsen <rasmuswriedtlarsen@gmail.com>
2021-02-04 12:41:17 +01:00
Taus
e5ec1e105c Python: Fix typos in test files
Co-authored-by: yoff <lerchedahl@gmail.com>
2021-02-04 12:18:07 +01:00
Taus Brock-Nannestad
5974af661e Python: Update test file
Makes the `a.b.c.d` test more sensible.

Also adds a test that shows a case where we're currently _not_ getting
the right flow.
2021-02-03 22:43:21 +01:00
Taus Brock-Nannestad
ba98b08001 Python: Further elaboration of use/3 2021-02-03 22:31:33 +01:00
Taus Brock-Nannestad
ebfb1faf77 Python: Autoformat 2021-02-03 22:26:46 +01:00
Taus
56515c5708 Python: Improve documentation for moduleImport
Co-authored-by: Rasmus Wriedt Larsen <rasmuswriedtlarsen@gmail.com>
2021-02-03 21:29:15 +01:00
Taus Brock-Nannestad
05f290f734 Python: Better explanation in use/3 2021-02-03 19:52:40 +01:00
Taus Brock-Nannestad
c5d6792c1e Python: Make toString abstract 2021-02-03 19:52:40 +01:00
Taus Brock-Nannestad
6ce160c51c Python: Use call instead of invocation 2021-02-03 19:52:40 +01:00
Taus Brock-Nannestad
3fafb47b16 Python: Fix global flow
A slightly odd fix, but still morally okay, I think. The main issue
here was that global variables have their first occurrence in an inner
scope inside a so-called "scope entry definition", that then
subsequently flows to the first use of this variable. This meant that
that first use was _not_ a `LocalSourceNode` (since _something_ flowed
into it), and this blocked `trackUseNode` from type-tracking to it (as
it expects all nodes to be `LocalSourceNode`s).

The answer, then, is to say that a `LocalSourceNode` is simply one
that doesn't have flow to it from _any `CfgNode`_ (through one or more
steps). This disregards the flow from the scope entry definition, as
that is flow from an `EssaNode`.

Additionally, it makes sense to exclude `ModuleVariableNode`s. These
should never be considered local sources, since they always have flow
from (at least) the place where the corresponding global variable is
introduced.
2021-02-03 16:41:22 +01:00
Taus Brock-Nannestad
e4c3544a3f Python: Add support for from foo.bar import baz
This turned out to be fairly simple. Given an import such as
```python
from foo.bar.baz import quux
```
we create an API-graph node for each valid dotted prefix of
`foo.bar.baz`, i.e. `foo`, `foo.bar`, and `foo.bar.baz`. For these, we
then insert nodes in the API graph, such that `foo` steps to `foo.bar`
along an edge labeled `bar`, etc.

Finally, we only allow undotted names to hang off of the API-graph
root. Thus, `foo` will have a `moduleImport` edge off of the root, and
a `getMember` edge for `bar` (which in turn has a `getMember` edge for
`baz`).

Relative imports are explicitly ignored.

Finally, this commit also adds inline tests for a variety of ways of
importing modules, including a copy of the "import-helper" tests (with
a few modifications to allow a single annotation per line, as these
get rather long quickly!).
2021-02-02 21:59:33 +01:00
Taus Brock-Nannestad
cd7b013a0c Python: Add missing documentation 2021-02-01 18:57:25 +01:00
Taus Brock-Nannestad
b8194bd1f8 Python: Add support for API graphs
Currently only supports the "use" side of things.

For the most part, this follows the corresponding implementation for
JavaScript. Major differences include:

- No `MkImportUse` nodes -- we just move directly from
  `MkModuleImport` to its uses.

- Paths are no longer labelled by s-expressions, but rather by a
string that mirrors how you would access it in QL. This makes it very
easy to see how to access an API component -- simply look at its
`toString`!

This PR also extends `LocalSourceNode` to support looking up attribute
references and invocations of such nodes. This was again based on the
JavaScript equivalent (though without specific classes for
`InvokeNode` and the like, it's a bit more awkward to use).
2021-02-01 14:38:59 +01:00
CodeQL CI
c9537f2639 Merge pull request #5029 from asgerf/js/silence-angular-template-fps
Approved by erik-krogh
2021-01-29 06:06:37 -08:00
Taus
cb195a0dc4 Merge pull request #4752 from yoff/python-dataflow-unpacking-assignment
Python: Dataflow, unpacking assignment
2021-01-29 14:15:28 +01:00
Taus
be5b7bb4c4 Merge pull request #5022 from yoff/python-split-lambdas
Python: Callable for lambdas
2021-01-29 14:12:26 +01:00
Geoffrey White
50f2557dd2 Merge pull request #5043 from MathiasVP/uniform-treatment-of-params-and-qualifiers-in-model-dataflow
C++: Uniform treatment of parameters and qualifiers in model dataflow
2021-01-29 09:48:07 +00:00
Geoffrey White
7d9ebaf9d8 Merge pull request #5040 from MathiasVP/strset-and-strtok-models
C++: Strset and strtok model implementations
2021-01-28 18:34:06 +00:00
Geoffrey White
768be9ec2c Merge pull request #5041 from ihsinme/ihsinme-patch-198
CPP: Improve cpp/memory-leak-on-failed-call-to-realloc
2021-01-28 18:29:24 +00:00
Mathias Vorreiter Pedersen
23eb4d2009 C++: Fix isParameterDeref typo. 2021-01-28 18:29:30 +01:00
Mathias Vorreiter Pedersen
75aa1e8a3b C++: Respond to review comments. 2021-01-28 16:39:11 +01:00
Geoffrey White
02d60a26eb Merge pull request #5037 from github/igfoo/decltype
C++: decltypes may have multiple expressions
2021-01-28 14:44:53 +00:00
Shati Patel
1c56c30eba Merge pull request #5028 from shati-patel/docs/update-footer
Docs: Update copyright date in footer
2021-01-28 13:11:43 +00:00
Tom Hvitved
59d87e2570 Merge pull request #4557 from hvitved/csharp/dataflow/parameters
C#: Simpler data-flow modelling of parameters
2021-01-28 14:02:42 +01:00
ihsinme
f94a7fc2f0 Update MemoryLeakOnFailedCallToRealloc.ql 2021-01-28 15:47:38 +03:00
Mathias Vorreiter Pedersen
5a420f2bae C++: Use the new predicates for uniform treatment of parameters and qualifiers in model dataflow. 2021-01-28 13:33:08 +01:00
Rasmus Wriedt Larsen
b6007cf324 Merge pull request #5023 from yoff/python-unify-synthetic-post-update-nodes
Python: Only generate one post-update node, even if there are multiple reasons for doing so.
2021-01-28 13:11:50 +01:00
ihsinme
2b4296feb1 Update MemoryLeakOnFailedCallToRealloc.ql 2021-01-28 13:38:26 +03:00
ihsinme
cf565970e3 Merge pull request #1 from github/main
update fork
2021-01-28 13:26:11 +03:00
yoff
1068edeb28 Merge pull request #5038 from RasmusWL/import-fix
Python: Fix too many results from DataFlow::importNode
2021-01-28 11:25:17 +01:00
Mathias Vorreiter Pedersen
2c70106d2d Merge pull request #5009 from ihsinme/ihsinme-patch-219
CPP: add query for CWE-788 Access of memory location after the end of a buffer using strncat.
2021-01-28 11:10:30 +01:00
Anders Schack-Mulligen
bbdd7c9b57 Merge pull request #4963 from joefarebrother/guava-collections
Java: Add flow steps for Guava collection utilities
2021-01-28 11:01:03 +01:00
Mathias Vorreiter Pedersen
7affbfc6cb C++: Add tests. 2021-01-28 10:57:39 +01:00
Mathias Vorreiter Pedersen
6255662114 C++: Add two new model implementation classes. 2021-01-28 10:57:30 +01:00
Tom Hvitved
e6f81bcf0b C#: Update expected test output 2021-01-28 10:34:50 +01:00
Tom Hvitved
6ee5cdf2b2 C#: Simpler data-flow modelling of parameters 2021-01-28 10:34:47 +01:00
Jonas Jensen
69ce24d4b8 Merge pull request #5035 from MathiasVP/implied-deref-flow
C++: Implied dataflow models
2021-01-28 09:35:58 +01:00
Tom Hvitved
65ea01e145 Merge pull request #4999 from hvitved/csharp/dataflow/phi-input
C#: Adjust flow into phi nodes
2021-01-28 09:07:01 +01:00
Rasmus Lerchedahl Petersen
0e0b18c214 Python: Adjust comment based on review. 2021-01-28 01:09:03 +01:00
Robert Marsh
0addce5be4 Merge pull request #5036 from MathiasVP/memcpy-models
C++: Model aliasing of memcpy-like functions and include more functions
2021-01-27 14:38:08 -08:00
Rasmus Lerchedahl Petersen
ae2c122159 Python: Small refactor
- align synthetic pre-update nodes with synthetic post -update nodes
- move the classes into the modules
- rename modules after the new main class (eliding "needs")
2021-01-27 23:15:50 +01:00
Mathias Vorreiter Pedersen
24f76f9a17 C++: Accept test changes. 2021-01-27 21:57:12 +01:00
Rasmus Lerchedahl Petersen
2120868939 Python: format 2021-01-27 19:48:01 +01:00