Commit Graph

17948 Commits

Author SHA1 Message Date
Geoffrey White
6c38f55e28 CPP: QLDoc protocols.qll. 2019-11-06 17:06:57 +00:00
semmle-qlci
717490b670 Merge pull request #2265 from tausbn/python-fix-unused-import-global-name-used-join-order
Approved by RasmusWL
2019-11-06 16:38:07 +00:00
Max Schaefer
54e40a8977 JavaScript: Move --html all extractor options into options file. 2019-11-06 16:30:01 +00:00
Geoffrey White
e886cf7297 CPP: 'i.e.' -> 'that is'. 2019-11-06 16:27:06 +00:00
Taus Brock-Nannestad
2b24eb2e70 Python: Fix bad join order for py/multiple-calls-to-init.
The `multiple_invocation_paths` predicate had a bad join order where
we (essentially) joined `i1` with `i2` and only then joined `i1` and `i2`
separately to reduce the number of tuples. The join coming from `i1 != i2` had
little impact, but `i1.getFunction() = multi` made a big difference (and
similarly for `i2`). I factored out the code so that these joins would be done
more eagerly. Thus, we went from

```
[2019-11-06 16:53:05] (38s) Starting to evaluate predicate MethodCallOrder::multiple_invocation_paths#ffff/4@2ce75a
[2019-11-06 16:53:35] (68s) Tuple counts for MethodCallOrder::multiple_invocation_paths#ffff:
                      134547    ~9%       {2} r1 = SCAN CallGraph::TInvocation#fff AS I OUTPUT I.<0>, I.<2>
                      235284431 ~3%       {4} r2 = JOIN r1 WITH CallGraph::TInvocation#fff AS R ON FIRST 1 OUTPUT r1.<0>, r1.<1>, R.<1>, R.<2>
                      235149884 ~3%       {4} r3 = SELECT r2 ON r2.<3> != r2.<1>
                      235149884 ~4%       {3} r4 = SCAN r3 OUTPUT r3.<1>, r3.<0>, r3.<3>
                      166753634 ~5%       {4} r5 = JOIN r4 WITH #CallGraph::FunctionInvocation::getACallee_dispred#ffPlus#swapped AS R ON FIRST 1 OUTPUT R.<1>, r4.<2>, r4.<1>, r4.<0>
                      129778    ~0%       {4} r6 = JOIN r5 WITH #CallGraph::FunctionInvocation::getACallee_dispred#ffPlus AS R ON FIRST 2 OUTPUT r5.<0>, r5.<3>, r5.<1>, r5.<2>
                                          return r6
[2019-11-06 16:53:35] (68s) Registering MethodCallOrder::multiple_invocation_paths#ffff + [] with content 1705dcbc08kd9aa40rp2g2e9civhv
[2019-11-06 16:53:35] (68s)  >>> Wrote relation MethodCallOrder::multiple_invocation_paths#ffff with 129778 rows and 4 columns.
```

to

```
[2019-11-06 17:22:22] (25s) Starting to evaluate predicate MethodCallOrder::multiple_invocation_paths_helper#ffff/4@586aec
[2019-11-06 17:22:22] (25s) Tuple counts for MethodCallOrder::multiple_invocation_paths_helper#ffff:
                      134547 ~0%     {2} r1 = SCAN CallGraph::TInvocation#fff AS I OUTPUT I.<2>, I.<0>
                      88111  ~4%     {3} r2 = JOIN r1 WITH #CallGraph::FunctionInvocation::getACallee_dispred#ffPlus#swapped AS R ON FIRST 1 OUTPUT R.<1>, r1.<1>, r1.<0>
                      761305 ~0%     {4} r3 = JOIN r2 WITH #CallGraph::FunctionInvocation::getACallee_dispred#ffPlus AS R ON FIRST 1 OUTPUT r2.<1>, r2.<2>, r2.<0>, R.<1>
                      673194 ~0%     {4} r4 = SELECT r3 ON r3.<3> != r3.<1>
                      673194 ~0%     {4} r5 = SCAN r4 OUTPUT r4.<2>, r4.<1>, r4.<3>, r4.<0>
                                     return r5
[2019-11-06 17:22:22] (25s) Registering MethodCallOrder::multiple_invocation_paths_helper#ffff + [] with content 20edaaecf25nldgp24d9c4et8m3kv
[2019-11-06 17:22:22] (25s)  >>> Wrote relation MethodCallOrder::multiple_invocation_paths_helper#ffff with 673194 rows and 4 columns.
[2019-11-06 17:22:22] (25s) Starting to evaluate predicate MethodCallOrder::multiple_invocation_paths_helper#ffff_2301#join_rhs/4@9e5441
[2019-11-06 17:22:22] (25s) Tuple counts for MethodCallOrder::multiple_invocation_paths_helper#ffff_2301#join_rhs:
                      673194 ~0%     {4} r1 = SCAN MethodCallOrder::multiple_invocation_paths_helper#ffff AS I OUTPUT I.<2>, I.<3>, I.<0>, I.<1>
                                     return r1
[2019-11-06 17:22:22] (25s) Registering MethodCallOrder::multiple_invocation_paths_helper#ffff_2301#join_rhs + [] with content 2069301e655fi9mcovngg9hetfqas
[2019-11-06 17:22:22] (25s)  >>> Wrote relation MethodCallOrder::multiple_invocation_paths_helper#ffff_2301#join_rhs with 673194 rows and 4 columns.
[2019-11-06 17:22:22] (25s) Starting to evaluate predicate MethodCallOrder::multiple_invocation_paths#ffff/4@2f7c34
[2019-11-06 17:22:22] (25s) Tuple counts for MethodCallOrder::multiple_invocation_paths#ffff:
                      134547 ~0%     {2} r1 = SCAN CallGraph::TInvocation#fff AS I OUTPUT I.<2>, I.<0>
                      129778 ~0%     {4} r2 = JOIN r1 WITH MethodCallOrder::multiple_invocation_paths_helper#ffff_2301#join_rhs AS R ON FIRST 2 OUTPUT R.<2>, R.<3>, r1.<0>, r1.<1>
                                     return r2
[2019-11-06 17:22:22] (25s) Registering MethodCallOrder::multiple_invocation_paths#ffff + [] with content 1705dcbc08kd9aa40rp2g2e9civhv
[2019-11-06 17:22:22] (25s)  >>> Wrote relation MethodCallOrder::multiple_invocation_paths#ffff with 129778 rows and 4 columns.
[2019-11-06 17:22:22] (25s) Starting to evaluate predicate MethodCallOrder::multiple_invocation_paths#ffff_0312#join_rhs/4@9f9146
[2019-11-06 17:22:22] (25s) Tuple counts for MethodCallOrder::multiple_invocation_paths#ffff_0312#join_rhs:
                      129778 ~0%     {4} r1 = SCAN MethodCallOrder::multiple_invocation_paths#ffff AS I OUTPUT I.<0>, I.<3>, I.<1>, I.<2>
                                     return r1
[2019-11-06 17:22:22] (25s) Registering MethodCallOrder::multiple_invocation_paths#ffff_0312#join_rhs + [] with content 17c3fe1fcbf6ghhdr7hiukqp41rst
[2019-11-06 17:22:22] (25s)  >>> Wrote relation MethodCallOrder::multiple_invocation_paths#ffff_0312#join_rhs with 129778 rows and 4 columns.
```

Execution time on `salt` went from 29.5s to somewhere below 299ms (the predicate
was not listed in the timing report).
2019-11-06 17:27:03 +01:00
Geoffrey White
81c58d5a64 CPP: Improve QLDoc comments. 2019-11-06 16:20:39 +00:00
Geoffrey White
0c3f4e530f CPP: Make some library predicates private. 2019-11-06 16:07:28 +00:00
Geoffrey White
399ac1f112 CPP: Rename 'getAssertedFalseCondition' to something less misleading. 2019-11-06 15:57:44 +00:00
Taus Brock-Nannestad
43148083eb Python: Fix bad join order for global_name_used.
As it turns out, there was a further bad join-order in the `global_name_used`
predicate. In this case, there was a common subexpression in the RA that was
being factored out and evaluated separately, producing a large number of tuples.
2019-11-06 16:37:01 +01:00
Nick Rolfe
5b00b21713 Merge pull request #2153 from matt-gretton-dann/cpp-447-support-non-type-template-parameters
RFC: C++ Support non type template parameter values
2019-11-06 15:11:34 +00:00
Rasmus Wriedt Larsen
47a292b241 Merge pull request #2264 from tausbn/python-fix-unused-import-join-order
Python: Fix bad join order in `py/unused-import`
2019-11-06 16:09:44 +01:00
Jonas Jensen
8ffd7c1055 Merge pull request #2222 from geoffw0/libraryperf
CPP: Improvements for ConditionallyInitializedVariable.ql
2019-11-06 15:54:16 +01:00
Jonas Jensen
217ecd3551 C++: Add <p> tags to split text into paragraphs
Without this, the rendered output is one big paragraph.
2019-11-06 15:50:08 +01:00
shati-patel
5d1d7ea075 Merge pull request #2257 from jf205/codeql-training-material
docs: update terminology in training material
2019-11-06 14:43:55 +00:00
Asger F
d9beb54dde Merge pull request #2102 from erik-krogh/deferredModel
JS: add Deferred model in js/use-of-returnless-function
2019-11-06 14:30:03 +00:00
Jonas Jensen
76a3db9eed Merge remote-tracking branch 'upstream/master' into ir-copy-unloaded-result 2019-11-06 15:21:22 +01:00
Taus Brock-Nannestad
b6f16dee81 Python: Fix bad join order in py/unused-import 2019-11-06 15:14:48 +01:00
semmle-qlci
f73caac88d Merge pull request #2254 from asger-semmle/for-of-propread
Approved by max-schaefer
2019-11-06 13:44:55 +00:00
Geoffrey White
f9feb05a72 CPP: Add a test of NtohlArrayNoBoundOpenSource.ql. 2019-11-06 13:36:31 +00:00
Jonas Jensen
aa841c306d C++: Use virtual dispatch in DefaultTaintTracking
This bit is only used by the compatibility code that sends flow into
parameters of functions without body.
2019-11-06 14:04:07 +01:00
Jonas Jensen
ec9ef33486 C++: IR data flow through inheritance conversions
This makes IR data flow behave more like AST data flow, and it makes IR
virtual dispatch work without further changes.
2019-11-06 14:04:07 +01:00
Jonas Jensen
49008c9ff5 C++: IR data flow local virtual dispatch
This is just good enough to cause no performance regressions and pass
the virtual-dispatch tests we have for `security.TaintTracking`. In
particular, it fixes the tests for `UncontrolledProcessOperation.ql`
when enabling `DefaultTaintTracking.qll`.
2019-11-06 14:04:02 +01:00
Max Schaefer
725059deea JavaScript: Remove --source-type module extractor options. 2019-11-06 13:01:59 +00:00
Max Schaefer
3ad5af7cef JavaScript: Move --extract-program-text extractor options into options files. 2019-11-06 13:01:55 +00:00
Max Schaefer
6b817203fd JavaScript: Move --tolerate-parse-errors extractor options into options file. 2019-11-06 13:01:28 +00:00
Max Schaefer
5681565d4a JavaScript: Move --html elements extractor options into options file. 2019-11-06 13:01:28 +00:00
Max Schaefer
8fdf6298b9 JavaScript: Remove --platform node extractor options. 2019-11-06 13:01:28 +00:00
Max Schaefer
4848372435 JavaScript: Replace --externs extractor flag with /** @externs */ comment. 2019-11-06 13:01:28 +00:00
Max Schaefer
1fa8c43a8c JavaScript: Remove a redundant extractor option. 2019-11-06 13:01:28 +00:00
Max Schaefer
79f1079460 JavaScript: Add options files with --experimental extractor options. 2019-11-06 13:01:23 +00:00
James Fletcher
2e7bd4db6f Update docs/language/ql-training/cpp/program-representation-cpp.rst
Co-Authored-By: shati-patel <42641846+shati-patel@users.noreply.github.com>
2019-11-06 12:59:45 +00:00
Max Schaefer
a4bf361f64 JavaScript: Remove remaining --experimental extractor options. 2019-11-06 12:54:44 +00:00
james
24615f2bdb docs: further review comments 2019-11-06 12:44:45 +00:00
Erik Krogh Kristensen
19554ff6e7 change "e.g." to "for example" in qldoc 2019-11-06 13:37:54 +01:00
Erik Krogh Kristensen
dc923ef694 remove change note
Co-Authored-By: Esben Sparre Andreasen <esbena@github.com>
2019-11-06 13:28:46 +01:00
Asger F
81723ab92a JS: Update GlobalAccessPaths test 2019-11-06 11:58:06 +00:00
Asger F
3ec95881b4 Update javascript/ql/src/semmle/javascript/GlobalAccessPaths.qll
Co-Authored-By: Max Schaefer <54907921+max-schaefer@users.noreply.github.com>
2019-11-06 11:58:06 +00:00
Asger F
7e80823cb6 JS: Fix deprecated API usage 2019-11-06 11:58:06 +00:00
Asger F
b373901e11 JS: Avoid leading dot in access paths 2019-11-06 11:58:06 +00:00
Asger F
c365833731 JS: Refactor the public access path API 2019-11-06 11:58:06 +00:00
Asger F
e90516d4d8 JS: Dont use getALocalSource in fromRhs 2019-11-06 11:58:06 +00:00
Asger F
bc35f24f31 JS: Generalize access paths to arbitrary root nodes 2019-11-06 11:58:06 +00:00
Asger F
7a7a8b2b09 JS: More steps in getImmediatePredecessor 2019-11-06 11:58:06 +00:00
Max Schaefer
2bcd418c23 Merge pull request #2261 from Semmle/esbena-patch-2
Doc: Add missing `t` in `support`.
2019-11-06 09:24:30 +00:00
Esben Sparre Andreasen
21d4e5f186 Doc: Add missing t in support. 2019-11-06 10:16:43 +01:00
semmle-qlci
04f0c22f24 Merge pull request #2203 from erik-krogh/ignorePureFunction
Approved by max-schaefer, mchammer01
2019-11-06 09:09:11 +00:00
Jonas Jensen
fca7d9a391 C#: Cache IRType like for C++ 2019-11-06 07:42:34 +01:00
Ziemowit Laski
1f82ea7750 [zlaski/pointer-overflow-check] Refine query to exclude macros (other than 'assert'). 2019-11-05 18:25:41 -08:00
Ziemowit Laski
0df3d2ce60 [zlaski/pointer-overflow-check] Improve test case. 2019-11-05 13:00:55 -08:00
james
8d02a740dd docs: address remaining comments 2019-11-05 20:03:47 +00:00