Commit Graph

8215 Commits

Author SHA1 Message Date
semmle-qlci
9986de87c4 Merge pull request #2284 from shati-patel/ql-codeql-1
Approved by jf205
2019-11-08 14:37:26 +00:00
semmle-qlci
d9c7549dbe Merge pull request #2279 from max-schaefer/js/touchstone-files
Approved by asger-semmle
2019-11-08 14:33:23 +00:00
shati-patel
fe654a9c99 update to match support page 2019-11-08 14:32:59 +00:00
shati-patel
3f51260fb4 Docs: Update sidebar 2019-11-08 14:04:44 +00:00
Esben Sparre Andreasen
9b346b1d52 Merge pull request #2260 from max-schaefer/js/_min
JavaScript: Classify files with names ending in `_min` as minified.
2019-11-08 13:52:33 +01:00
semmle-qlci
867ed16777 Merge pull request #2276 from asger-semmle/inclusion-test
Approved by max-schaefer
2019-11-08 10:57:11 +00:00
Max Schaefer
d7831d2680 JavaScript: Short-circuit bad-header check on empty files. 2019-11-08 10:30:53 +00:00
Felicity Chapman
8ed0d726ee Merge pull request #2280 from jf205/codeql-homepage-links
docs: update banner links
2019-11-08 10:06:27 +00:00
james
0554de06a1 docs: update banner links 2019-11-08 09:32:20 +00:00
Max Schaefer
e8510fe71a TypeScript: Skip Touchstone files. 2019-11-08 09:17:05 +00:00
Robert Marsh
ee185ea92e Merge pull request #2273 from geoffw0/ntohl
CPP: Add tests of NtohlArrayNoBoundOpenSource.ql.
2019-11-07 14:06:32 -08:00
Robert Marsh
c5396d9980 Merge pull request #2262 from jbj/ir-virtual-dispatch-local
C++: Rudimentary support for IR data flow virtual dispatch
2019-11-07 13:09:24 -08:00
Dave Bartolomeo
df1d64fbeb Merge pull request #2244 from jbj/IRType-cached
C++: Minimal caching of the IR type system
2019-11-07 12:44:16 -07:00
Dave Bartolomeo
f808dcefab Merge pull request #2277 from ian-semmle/cfg_diffs
C++: Remove tests for CFG differences
2019-11-07 12:41:40 -07:00
Dave Bartolomeo
64480c2ace Merge pull request #1999 from jbj/ir-copy-unloaded-result
C++: Make sure there's a Instruction for each Expr
2019-11-07 12:31:54 -07:00
Dave Bartolomeo
2c88848d2f Merge pull request #2272 from jbj/getIRTypeForPRValue-join-order
C++/C#: Fix getIRTypeForPRValue join order
2019-11-07 12:22:39 -07:00
Ian Lynagh
b5af4e5acd C++: Remove tests for CFG differences
Now that we have switched over, they are no longer interesting.
2019-11-07 16:32:18 +00:00
igfoo
c8c37c4976 Merge pull request #2271 from matt-gretton-dann/cpp-172-template-members
Template members
2019-11-07 16:30:08 +00:00
shati-patel
ec2008d57a Merge pull request #2275 from jf205/sd-4017
Learn CodeQL docs: add short note about new terminology
2019-11-07 16:12:12 +00:00
semmle-qlci
e65271dfad Merge pull request #2251 from asger-semmle/barrier-guard-improvements
Approved by esbena
2019-11-07 15:50:23 +00:00
semmle-qlci
f79c2a7630 Merge pull request #2224 from asger-semmle/access-paths-with-source-node-root
Approved by max-schaefer
2019-11-07 15:46:14 +00:00
James Fletcher
8178e3e671 Update docs/language/learn-ql/terminology-note.rst
Co-Authored-By: shati-patel <42641846+shati-patel@users.noreply.github.com>
2019-11-07 14:40:04 +00:00
James Fletcher
d31ec56ea6 Update docs/language/learn-ql/index.rst
Co-Authored-By: shati-patel <42641846+shati-patel@users.noreply.github.com>
2019-11-07 14:39:52 +00:00
Rasmus Wriedt Larsen
9ffb67a460 Merge pull request #2266 from tausbn/python-multiple-calls-to-init-join-order-fix
Python: Fix bad join order for `py/multiple-calls-to-init`.
2019-11-07 15:38:43 +01:00
Asger F
8544850945 JS: Generalize StringOps::Includes to ::InclusionTest 2019-11-07 14:35:17 +00:00
james
db2039d8a7 docs: learn codeql terminology note 2019-11-07 14:32:05 +00:00
semmle-qlci
2b120def01 Merge pull request #2211 from hvitved/csharp/unsafe-deserialization
Approved by jf205
2019-11-07 14:16:13 +00:00
Matthew Gretton-Dann
ddf1ef8a7d C++: Add new test case for template member change
We now output literals for accesses to members of template parameters:

So for `foo` in the following example:

```
template<typename T> void bar(T& t) {
  T.foo(1)
}
```
2019-11-07 14:08:25 +00:00
Matthew Gretton-Dann
c0884e9a88 C++: Update expected results. 2019-11-07 14:08:25 +00:00
semmle-qlci
3a7f9a588d Merge pull request #2267 from max-schaefer/js/qltest-extractor-options
Approved by asger-semmle
2019-11-07 11:36:45 +00:00
shati-patel
7394d5c726 Merge pull request #2242 from felicitymay/codeql/SD-4059-markdown
Docs: update terminology in markdown files (SD-4059)
2019-11-07 11:30:32 +00:00
Jonas Jensen
6385528d5f C++/C#: Fix getIRTypeForPRValue join order
This predicate was taking 39s on a snapshot of Facebook Fizz because it
had disjuncts like this:

    43685     ~0%     {1} r34 = JOIN Type::FunctionPointerIshType#f AS L WITH Type::Type::getUnspecifiedType_dispred#ff_10#join_rhs AS R ON FIRST 1 OUTPUT R.<1>
    43685     ~1%     {2} r35 = JOIN r34 WITH CppType::getTypeSize#ff AS R ON FIRST 1 OUTPUT R.<1>, r34.<0>
    170371500 ~2%     {2} r36 = JOIN r35 WITH IRType::IRSizedType#ff_10#join_rhs AS R ON FIRST 1 OUTPUT R.<1>, r35.<1>
    43685     ~6%     {2} r37 = JOIN r36 WITH IRType::IRFunctionAddressType#class#ff AS R ON FIRST 1 OUTPUT r36.<1>, r36.<0>

Instead of fixing the joins in `getIRTypeForPRValue` itself, I've
changed the `IRType::getByteSize` predicate such that the optimiser
knows how to join with it efficiently.

The disjunct shown above now looks like this instead:

    43685  ~0%     {1} r26 = JOIN Type::FunctionPointerIshType#f AS L WITH Type::Type::getUnspecifiedType_dispred#ff_10#join_rhs AS R ON FIRST 1 OUTPUT R.<1>
    43685  ~1%     {2} r27 = JOIN r26 WITH CppType::getTypeSize#ff AS R ON FIRST 1 OUTPUT R.<1>, r26.<0>
    43685  ~6%     {2} r28 = JOIN r27 WITH IRType::IRFunctionAddressType::getByteSize#ff_10#join_rhs AS R ON FIRST 1 OUTPUT r27.<1>, R.<1>
2019-11-07 11:48:16 +01:00
Max Schaefer
e314869e5c JavaScript: Classify files with names ending in _min as minified.
We already do the same for `-min` and `.min`. [Here](https://github.com/antoniogarrote/rdfstore-js/blob/master/dist/rdfstore_min.js) is a real-world example.
2019-11-07 10:33:47 +00:00
semmle-qlci
717490b670 Merge pull request #2265 from tausbn/python-fix-unused-import-global-name-used-join-order
Approved by RasmusWL
2019-11-06 16:38:07 +00:00
Max Schaefer
54e40a8977 JavaScript: Move --html all extractor options into options file. 2019-11-06 16:30:01 +00:00
Taus Brock-Nannestad
2b24eb2e70 Python: Fix bad join order for py/multiple-calls-to-init.
The `multiple_invocation_paths` predicate had a bad join order where
we (essentially) joined `i1` with `i2` and only then joined `i1` and `i2`
separately to reduce the number of tuples. The join coming from `i1 != i2` had
little impact, but `i1.getFunction() = multi` made a big difference (and
similarly for `i2`). I factored out the code so that these joins would be done
more eagerly. Thus, we went from

```
[2019-11-06 16:53:05] (38s) Starting to evaluate predicate MethodCallOrder::multiple_invocation_paths#ffff/4@2ce75a
[2019-11-06 16:53:35] (68s) Tuple counts for MethodCallOrder::multiple_invocation_paths#ffff:
                      134547    ~9%       {2} r1 = SCAN CallGraph::TInvocation#fff AS I OUTPUT I.<0>, I.<2>
                      235284431 ~3%       {4} r2 = JOIN r1 WITH CallGraph::TInvocation#fff AS R ON FIRST 1 OUTPUT r1.<0>, r1.<1>, R.<1>, R.<2>
                      235149884 ~3%       {4} r3 = SELECT r2 ON r2.<3> != r2.<1>
                      235149884 ~4%       {3} r4 = SCAN r3 OUTPUT r3.<1>, r3.<0>, r3.<3>
                      166753634 ~5%       {4} r5 = JOIN r4 WITH #CallGraph::FunctionInvocation::getACallee_dispred#ffPlus#swapped AS R ON FIRST 1 OUTPUT R.<1>, r4.<2>, r4.<1>, r4.<0>
                      129778    ~0%       {4} r6 = JOIN r5 WITH #CallGraph::FunctionInvocation::getACallee_dispred#ffPlus AS R ON FIRST 2 OUTPUT r5.<0>, r5.<3>, r5.<1>, r5.<2>
                                          return r6
[2019-11-06 16:53:35] (68s) Registering MethodCallOrder::multiple_invocation_paths#ffff + [] with content 1705dcbc08kd9aa40rp2g2e9civhv
[2019-11-06 16:53:35] (68s)  >>> Wrote relation MethodCallOrder::multiple_invocation_paths#ffff with 129778 rows and 4 columns.
```

to

```
[2019-11-06 17:22:22] (25s) Starting to evaluate predicate MethodCallOrder::multiple_invocation_paths_helper#ffff/4@586aec
[2019-11-06 17:22:22] (25s) Tuple counts for MethodCallOrder::multiple_invocation_paths_helper#ffff:
                      134547 ~0%     {2} r1 = SCAN CallGraph::TInvocation#fff AS I OUTPUT I.<2>, I.<0>
                      88111  ~4%     {3} r2 = JOIN r1 WITH #CallGraph::FunctionInvocation::getACallee_dispred#ffPlus#swapped AS R ON FIRST 1 OUTPUT R.<1>, r1.<1>, r1.<0>
                      761305 ~0%     {4} r3 = JOIN r2 WITH #CallGraph::FunctionInvocation::getACallee_dispred#ffPlus AS R ON FIRST 1 OUTPUT r2.<1>, r2.<2>, r2.<0>, R.<1>
                      673194 ~0%     {4} r4 = SELECT r3 ON r3.<3> != r3.<1>
                      673194 ~0%     {4} r5 = SCAN r4 OUTPUT r4.<2>, r4.<1>, r4.<3>, r4.<0>
                                     return r5
[2019-11-06 17:22:22] (25s) Registering MethodCallOrder::multiple_invocation_paths_helper#ffff + [] with content 20edaaecf25nldgp24d9c4et8m3kv
[2019-11-06 17:22:22] (25s)  >>> Wrote relation MethodCallOrder::multiple_invocation_paths_helper#ffff with 673194 rows and 4 columns.
[2019-11-06 17:22:22] (25s) Starting to evaluate predicate MethodCallOrder::multiple_invocation_paths_helper#ffff_2301#join_rhs/4@9e5441
[2019-11-06 17:22:22] (25s) Tuple counts for MethodCallOrder::multiple_invocation_paths_helper#ffff_2301#join_rhs:
                      673194 ~0%     {4} r1 = SCAN MethodCallOrder::multiple_invocation_paths_helper#ffff AS I OUTPUT I.<2>, I.<3>, I.<0>, I.<1>
                                     return r1
[2019-11-06 17:22:22] (25s) Registering MethodCallOrder::multiple_invocation_paths_helper#ffff_2301#join_rhs + [] with content 2069301e655fi9mcovngg9hetfqas
[2019-11-06 17:22:22] (25s)  >>> Wrote relation MethodCallOrder::multiple_invocation_paths_helper#ffff_2301#join_rhs with 673194 rows and 4 columns.
[2019-11-06 17:22:22] (25s) Starting to evaluate predicate MethodCallOrder::multiple_invocation_paths#ffff/4@2f7c34
[2019-11-06 17:22:22] (25s) Tuple counts for MethodCallOrder::multiple_invocation_paths#ffff:
                      134547 ~0%     {2} r1 = SCAN CallGraph::TInvocation#fff AS I OUTPUT I.<2>, I.<0>
                      129778 ~0%     {4} r2 = JOIN r1 WITH MethodCallOrder::multiple_invocation_paths_helper#ffff_2301#join_rhs AS R ON FIRST 2 OUTPUT R.<2>, R.<3>, r1.<0>, r1.<1>
                                     return r2
[2019-11-06 17:22:22] (25s) Registering MethodCallOrder::multiple_invocation_paths#ffff + [] with content 1705dcbc08kd9aa40rp2g2e9civhv
[2019-11-06 17:22:22] (25s)  >>> Wrote relation MethodCallOrder::multiple_invocation_paths#ffff with 129778 rows and 4 columns.
[2019-11-06 17:22:22] (25s) Starting to evaluate predicate MethodCallOrder::multiple_invocation_paths#ffff_0312#join_rhs/4@9f9146
[2019-11-06 17:22:22] (25s) Tuple counts for MethodCallOrder::multiple_invocation_paths#ffff_0312#join_rhs:
                      129778 ~0%     {4} r1 = SCAN MethodCallOrder::multiple_invocation_paths#ffff AS I OUTPUT I.<0>, I.<3>, I.<1>, I.<2>
                                     return r1
[2019-11-06 17:22:22] (25s) Registering MethodCallOrder::multiple_invocation_paths#ffff_0312#join_rhs + [] with content 17c3fe1fcbf6ghhdr7hiukqp41rst
[2019-11-06 17:22:22] (25s)  >>> Wrote relation MethodCallOrder::multiple_invocation_paths#ffff_0312#join_rhs with 129778 rows and 4 columns.
```

Execution time on `salt` went from 29.5s to somewhere below 299ms (the predicate
was not listed in the timing report).
2019-11-06 17:27:03 +01:00
Taus Brock-Nannestad
43148083eb Python: Fix bad join order for global_name_used.
As it turns out, there was a further bad join-order in the `global_name_used`
predicate. In this case, there was a common subexpression in the RA that was
being factored out and evaluated separately, producing a large number of tuples.
2019-11-06 16:37:01 +01:00
Nick Rolfe
5b00b21713 Merge pull request #2153 from matt-gretton-dann/cpp-447-support-non-type-template-parameters
RFC: C++ Support non type template parameter values
2019-11-06 15:11:34 +00:00
Rasmus Wriedt Larsen
47a292b241 Merge pull request #2264 from tausbn/python-fix-unused-import-join-order
Python: Fix bad join order in `py/unused-import`
2019-11-06 16:09:44 +01:00
Jonas Jensen
8ffd7c1055 Merge pull request #2222 from geoffw0/libraryperf
CPP: Improvements for ConditionallyInitializedVariable.ql
2019-11-06 15:54:16 +01:00
shati-patel
5d1d7ea075 Merge pull request #2257 from jf205/codeql-training-material
docs: update terminology in training material
2019-11-06 14:43:55 +00:00
Asger F
d9beb54dde Merge pull request #2102 from erik-krogh/deferredModel
JS: add Deferred model in js/use-of-returnless-function
2019-11-06 14:30:03 +00:00
Jonas Jensen
76a3db9eed Merge remote-tracking branch 'upstream/master' into ir-copy-unloaded-result 2019-11-06 15:21:22 +01:00
Taus Brock-Nannestad
b6f16dee81 Python: Fix bad join order in py/unused-import 2019-11-06 15:14:48 +01:00
semmle-qlci
f73caac88d Merge pull request #2254 from asger-semmle/for-of-propread
Approved by max-schaefer
2019-11-06 13:44:55 +00:00
Geoffrey White
f9feb05a72 CPP: Add a test of NtohlArrayNoBoundOpenSource.ql. 2019-11-06 13:36:31 +00:00
Jonas Jensen
aa841c306d C++: Use virtual dispatch in DefaultTaintTracking
This bit is only used by the compatibility code that sends flow into
parameters of functions without body.
2019-11-06 14:04:07 +01:00
Jonas Jensen
ec9ef33486 C++: IR data flow through inheritance conversions
This makes IR data flow behave more like AST data flow, and it makes IR
virtual dispatch work without further changes.
2019-11-06 14:04:07 +01:00
Jonas Jensen
49008c9ff5 C++: IR data flow local virtual dispatch
This is just good enough to cause no performance regressions and pass
the virtual-dispatch tests we have for `security.TaintTracking`. In
particular, it fixes the tests for `UncontrolledProcessOperation.ql`
when enabling `DefaultTaintTracking.qll`.
2019-11-06 14:04:02 +01:00
Max Schaefer
725059deea JavaScript: Remove --source-type module extractor options. 2019-11-06 13:01:59 +00:00