Commit Graph

41418 Commits

Author SHA1 Message Date
Max Schaefer
42bc5353e3 Refine our modelling of test functions and split it out into a separate library. 2020-04-06 14:46:13 +01:00
Geoffrey White
050e239507 C++: Change note. 2020-04-06 14:39:07 +01:00
Pavel Avgustinov
c45a5cf1f2 Update copyright date 2020-04-06 14:37:09 +01:00
Pavel Avgustinov
291150a755 Delete outdated COPYRIGHT file
This seems to be copied from Semmle/ql, and it claims the wrong license.
2020-04-06 14:35:44 +01:00
Geoffrey White
cbe133d0e6 C++: Deprecate freeCall in the legacy wrapper Alloc.qll. 2020-04-06 14:32:49 +01:00
Geoffrey White
e223557201 C++: Wean NewDelete.qll off the legacy wrapper Alloc.qll. 2020-04-06 14:32:15 +01:00
Calum Grant
0d86866ba3 Merge pull request #3160 from hvitved/csharp/null-maybe-fp
C#: Add false-positive test for NullMaybe.ql
2020-04-06 14:30:31 +01:00
Geoffrey White
8059d69bbd C++: Model calls to operator new / delete for NewFreeMismatch.ql. 2020-04-06 14:27:05 +01:00
Geoffrey White
3e9f9645ae C++: Exclude calls to operator new / delete from NewFreeMismatch.ql. 2020-04-06 14:08:00 +01:00
Geoffrey White
97cdcbee63 C++: Test for NewFreeMismatch.ql with operator new / delete. 2020-04-06 13:57:28 +01:00
Jonas Jensen
bf7614a4c9 C++: Move Expr location workaround to Expr.qll
This workaround from `DataFlowUtil.qll` should be useful for any query
that selects an `Expr`. In particular, it's useful for IR data flow.

This commit does not include test changes.
2020-04-06 14:13:22 +02:00
Calum Grant
6cce0de9b2 Merge pull request #3124 from hvitved/csharp/dataflow/sources-and-sinks
C#: Introduce `RemoteFlowSink` class
2020-04-06 12:36:14 +01:00
Asger Feldthaus
7da0345c6a JS: Autoformat 2020-04-06 12:30:04 +01:00
Asger Feldthaus
2c6beadf68 JS: Recognize more forms of scheme checks 2020-04-06 12:30:03 +01:00
Sauyon Lee
a663eaae23 Merge pull request #89 from max-schaefer/extractor-profiling
Add environment variables for dumping CPU and memory profiles.
2020-04-06 03:33:08 -07:00
Max Schaefer
1f7441e027 Make readsField and friends work modulo implicit dereferences, like writesField. 2020-04-06 11:06:46 +01:00
Jonas Jensen
d4338473b0 C++: Enforce unique enclosing callable
Every data-flow node should have a unique enclosing function (_callable_
in the terminology of the data-flow library), but this was not evident
for the optimizer, and it led to a bad join order in `pathStep`. This
commit fixes the join order for C++ AST data flow. All other copies of
data flow seem to be fine.

These are the tuple counts for OpenJDK before this commit:

    (231s) Tuple counts for DataFlowImplLocal::pathStep#fffff#cur_delta:
    5882       ~0%       {6} r1 = SCAN DataFlowImplLocal::PathNodeMid#class#ffffff#prev_delta AS I OUTPUT I.<2>, I.<0>, I.<1>, I.<3>, I.<4>, I.<5>
    1063406780 ~0%       {7} r2 = JOIN r1 WITH DataFlowImplCommon::CallContext::relevantFor_dispred#ff AS R ON FIRST 1 OUTPUT r1.<2>, R.<1>, r1.<1>, r1.<0>, r1.<3>, r1.<4>, r1.<5>
    5882       ~1%       {6} r3 = JOIN r2 WITH DataFlowUtil::Node::getFunction_dispred#ff AS R ON FIRST 2 OUTPUT r2.<0>, r2.<6>, r2.<2>, r2.<3>, r2.<4>, r2.<5>
    105        ~0%       {5} r4 = JOIN r3 WITH project#DataFlowImplLocal::LocalFlowBigStep::localFlowBigStep#ffffff_021#join_rhs AS R ON FIRST 2 OUTPUT r3.<2>, r3.<3>, r3.<4>, r3.<5>, R.<2>
    5882       ~1%       {6} r5 = JOIN r2 WITH DataFlowUtil::Node::getFunction_dispred#ff AS R ON FIRST 2 OUTPUT r2.<5>, r2.<2>, r2.<0>, r2.<3>, r2.<4>, r2.<6>
    5882       ~0%       {6} r6 = JOIN r5 WITH DataFlowImplLocal::TNil#ff_1#join_rhs AS R ON FIRST 1 OUTPUT r5.<2>, false, r5.<5>, r5.<1>, r5.<3>, r5.<4>
    0          ~0%       {5} r7 = JOIN r6 WITH DataFlowImplLocal::LocalFlowBigStep::localFlowBigStep#ffffff_02413#join_rhs AS R ON FIRST 3 OUTPUT R.<4>, r6.<3>, r6.<4>, r6.<5>, R.<3>
    0          ~0%       {5} r8 = JOIN r7 WITH DataFlowImplLocal::TNil#ff AS R ON FIRST 1 OUTPUT r7.<1>, r7.<2>, r7.<3>, R.<1>, r7.<4>
    105        ~0%       {5} r9 = r4 \/ r8

The problem is that `DataFlowUtil::Node::getFunction_dispred#ff`
(`getEnclosingCallable`) is joined too late.

After this commit, the tuple counts look like this:

    (13s) Tuple counts for DataFlowImplLocal::pathStep#fffff#cur_delta:
    5882    ~1%       {6} r1 = SCAN DataFlowImplLocal::PathNodeMid#class#ffffff#prev_delta AS I OUTPUT I.<1>, I.<0>, I.<2>, I.<3>, I.<4>, I.<5>
    5882    ~3%       {7} r2 = JOIN r1 WITH DataFlowUtil::Node::getEnclosingCallable_dispred#ff AS R ON FIRST 1 OUTPUT r1.<2>, R.<1>, r1.<1>, r1.<0>, r1.<3>, r1.<4>, r1.<5>
    5882    ~1%       {6} r3 = JOIN r2 WITH DataFlowImplCommon::CallContext::relevantFor_dispred#ff AS R ON FIRST 2 OUTPUT r2.<3>, r2.<6>, r2.<2>, r2.<0>, r2.<4>, r2.<5>
    105     ~0%       {5} r4 = JOIN r3 WITH project#DataFlowImplLocal::LocalFlowBigStep::localFlowBigStep#ffffff_021#join_rhs AS R ON FIRST 2 OUTPUT r3.<2>, r3.<3>, r3.<4>, r3.<5>, R.<2>
    5882    ~1%       {6} r5 = JOIN r2 WITH DataFlowImplCommon::CallContext::relevantFor_dispred#ff AS R ON FIRST 2 OUTPUT r2.<5>, r2.<2>, r2.<3>, r2.<0>, r2.<4>, r2.<6>
    5882    ~0%       {6} r6 = JOIN r5 WITH DataFlowImplLocal::TNil#ff_1#join_rhs AS R ON FIRST 1 OUTPUT r5.<2>, false, r5.<5>, r5.<1>, r5.<3>, r5.<4>
    0       ~0%       {5} r7 = JOIN r6 WITH DataFlowImplLocal::LocalFlowBigStep::localFlowBigStep#ffffff_02413#join_rhs AS R ON FIRST 3 OUTPUT R.<4>, r6.<3>, r6.<4>, r6.<5>, R.<3>
    0       ~0%       {5} r8 = JOIN r7 WITH DataFlowImplLocal::TNil#ff AS R ON FIRST 1 OUTPUT r7.<1>, r7.<2>, r7.<3>, R.<1>, r7.<4>
    105     ~0%       {5} r9 = r4 \/ r8

There is a slight slowdown coming from the introduction of a new
predicate `DataFlowImplLocal::pathStep#fffff#join_rhs`, which is used
only in the standard order:

    (12s) Tuple counts for DataFlowImplLocal::pathStep#fffff#join_rhs:
    282057  ~0%     {2} r1 = SCAN DataFlowImplCommon::CallContext::relevantFor_dispred#ff AS I OUTPUT I.<1>, I.<0>
    9159890 ~1%     {2} r2 = JOIN r1 WITH DataFlowUtil::Node::getEnclosingCallable_dispred#ff_10#join_rhs AS R ON FIRST 1 OUTPUT R.<1>, r1.<1>
                    return r2

The evaluation of `unique` is cheap but not free:

    DataFlowUtil::Node::getEnclosingCallable_dispred#ff .............. 3.9s
    DataFlowUtil::Node::getEnclosingCallable_dispred#ff_10#join_rhs .. 3.5s

The first of these two predicates evaluates `unique`, and the second
simply reorders columns. They take about the same time, which suggests
that `unique` is about as fast as it can be, given the number of tuples
it needs to push around. Note that the column reordering predicate is
only needed because of the standard order.
2020-04-06 12:04:39 +02:00
Mathias Vorreiter Pedersen
3aa293210d C++: Ensure that only non-conflated chi instructions are used everywhere 2020-04-06 12:02:56 +02:00
James Fletcher
5034d40e64 Merge pull request #3203 from jf205/sd-55
CodeQL support docs: combine table and footnotes into single snippet
2020-04-06 09:52:02 +01:00
james
d2b0599b63 docs: combine table and footnotes 2020-04-06 09:27:30 +01:00
Max Schaefer
76f2748cbc Teach SsaWithFields to properly handle implicit dereferences. 2020-04-06 09:23:07 +01:00
Max Schaefer
4a1071cac6 Merge pull request #77 from robertbrignull/code_scanning_suites
Add code-scanning suites
2020-04-06 09:14:58 +01:00
Jonas Jensen
46fc91315b Java/C++/C#: Revert the join order fix from #2872
This revert brings back the performance problems in
`DataFlowImplLocal.qll` so they can be fixed in a different way. The fix
in #2872 was asymptotically good but had undesired overhead because it
introduced another predicate in the SCC that existed purely for join
ordering.

I did the revert by inlining the helper predicate, eliminating the
`enclosing` variable, and re-ordering the resulting lines to what they
were before #2872.
2020-04-06 10:04:50 +02:00
Robert
1096e5d947 Merge pull request #3163 from robertbrignull/code_scanning_suites
Add code-scanning suites
2020-04-06 08:45:40 +01:00
Rasmus Wriedt Larsen
4ce3d5b748 Merge pull request #3040 from BekaValentine/python-objectapi-to-valueapi-iterreturnsnonself
Python: ObjectAPI to ValueAPI: IterReturnsNonSelf
2020-04-06 09:37:40 +02:00
Tom Hvitved
c8c706a0ba C#: Un-deprecate PublicCallableParameterFlowSource 2020-04-06 09:01:44 +02:00
Mathias Vorreiter Pedersen
317734f41e C++: Attach PostUpdateNodes to Chi nodes following aschackmull's suggestion 2020-04-05 22:35:26 +02:00
Jonas Jensen
530d4294b0 Merge remote-tracking branch 'upstream/master' into DefaultTaintTracking-Configuration 2020-04-05 07:27:07 +02:00
Jonas Jensen
58366b19e9 C++: Path explanations in the last two queries
For some reason I thought that these two queries were special because
they manipulate `SecurityOptions` to change the taint-tracking sources.
It turns out it was just the opposite: the queries used to be special
because they invalidated the cache for the `tainted` predicate, but that
predicate is no longer used, so these queries are no longer special.
2020-04-04 16:47:06 +02:00
Jonas Jensen
54a23a486a C++: Accept test changes for 108d5177b8 2020-04-04 16:46:59 +02:00
Jonas Jensen
d7332644f0 C++: Fix DefinitionByReferenceNode.toString
This predicate now has a value also for calls to function pointers.
2020-04-04 15:31:01 +02:00
Jonas Jensen
108d5177b8 C++: Fix two bugs found by @rdmarsh2
Co-Authored-By: Robert Marsh <rdmarsh2@gmail.com>
2020-04-04 15:24:44 +02:00
Rebecca Valentine
01aac8273c Python: ObjAPI to ValAPI: WrongNumArgsInCall: Update queries to use objectapi 2020-04-04 03:11:25 -07:00
Rebecca Valentine
8c1aeb24cb Python: ObjAPI to ValAPI: WrongNumArgsInCall: Updates query expected results 2020-04-04 00:57:09 -07:00
Rebecca Valentine
88be3359ac Python: ObjAPI to ValAPI: WrongNumForArgsInCall: Update affected queries to use objectapi 2020-04-04 00:15:10 -07:00
Rebecca Valentine
26bdb9ab04 Python: ObjAPI to ValAPI: WrongNumArgsInCall: Exceptions.qll: Removes extraneous spaces in comments 2020-04-03 20:33:17 -07:00
Rebecca Valentine
bbe7314c17 Python: ObjAPI to ValAPI: WrongNumArgsInCall: Autoformats ObjectAPI.qll 2020-04-03 20:30:24 -07:00
Rebecca Valentine
7615452b31 Python: ObjAPI to ValAPI: WrongNumArgsInCall: Autoformat CallArgs.qll 2020-04-03 20:28:19 -07:00
Rebecca Valentine
1e76720a92 Merge branch 'master' into python-objectapi-to-valueapi-wrongnumberargumentsincall 2020-04-03 20:25:53 -07:00
Rebecca Valentine
e0f26d4f7e Python: ObjectAPI to ValueAPI: WrongNumberArgumentsInCall: Updates expected results 2020-04-03 20:11:42 -07:00
Rebecca Valentine
f25428b7a9 Python: ObjectAPI to ValueAPI: WrongNumberArgumentsInCall: Exceptions.qll: Fixes incorrect implementation of localRaisedType 2020-04-03 20:08:00 -07:00
Rebecca Valentine
c6fbbb1cd1 Python: ObjectAPI to ValueAPI: WrongNumberArgumentsInCall: CallArgs.qll: Fixes too_*_args refs to getA*Call 2020-04-03 20:06:43 -07:00
Rebecca Valentine
2ad0f5af2d Python: ObjectAPI to ValueAPI: WrongNumberArgumentsInCall: ObjectAPI.qll: Adds getAFunctionCall and getAMethodCall predicates to FunctionValue 2020-04-03 20:04:32 -07:00
Rebecca Valentine
be86c9c066 Python: ObjectAPI to ValueAPI: IterReturnsNonSelf: ObjectAPI.qll: Explains why getAnInferredReturnType is weird for builtins 2020-04-03 15:16:16 -07:00
Rebecca Valentine
64b17888e5 Python: ObjectAPI to ValueAPI: IterReturnsNonSelf: ObjectAPI.qll: Reorganizes getAnInferredReturnType() 2020-04-03 15:14:25 -07:00
Robert Marsh
316d932829 Merge pull request #3198 from MathiasVP/valuenumbering-provider-new-file
C++/C#: Prevent accidental import of ValueNumberPropertyProvider
2020-04-03 13:31:11 -07:00
Grzegorz Golawski
1d8da905ac Make the test runnable via codeql test run 2020-04-03 21:44:13 +02:00
ggolawski
79d7ea36ff Update java/ql/src/experimental/Security/CWE/CWE-016/SpringBootActuators.qll
Co-Authored-By: Anders Schack-Mulligen <aschackmull@users.noreply.github.com>
2020-04-03 21:36:34 +02:00
Max Schaefer
5721d1fb3e Merge pull request #86 from sauyon/dependency-fix
Allow dependencies to match imports of subpackages
2020-04-03 20:35:07 +01:00
Max Schaefer
c240fdbd45 Merge pull request #87 from max-schaefer/more-qldoc
Add missing QlDoc.
2020-04-03 19:43:38 +01:00