Commit Graph

17948 Commits

Author SHA1 Message Date
Geoffrey White
ff39f714e8 C++: Autoformat. 2020-04-07 17:07:31 +01:00
Robert Marsh
0ccf39777c Merge pull request #3189 from jbj/DefaultTaintTracking-Configuration
C++: Path explanations in DefaultTaintTracking
2020-04-07 08:38:10 -07:00
Jonas Jensen
39911af56b C++: Avoid partial chi flow to struct/class
Flow through partial chi-instruction operands was introduced to make
definition-by-reference work, but its implementation also allowed all
other partial writes to propagate. In particular, tainting a field would
taint the whole struct, which in turn led to taint propagating across
unrelated fields of a struct.

The security test `CWE-134/semmle/argv/argvLocal.c` shows that we also
want to propagate taint from an array element to the whole array, and it
also seems right to propagate taint from a union member to the whole
union.
2020-04-07 16:24:24 +02:00
Mathias Vorreiter Pedersen
d56284fe8f C++: Move added flow from simpleLocalFlowStep to simpleInstructionLocalFlowStep and remove flow that could cause field conflation 2020-04-07 16:00:40 +02:00
Mathias Vorreiter Pedersen
8928091dfb Merge pull request #3181 from jbj/DefaultTaintTracking-qldoc
C++: QLDoc in DefaultTaintTracking
2020-04-07 14:58:21 +02:00
Jonas Jensen
057155f28f Merge remote-tracking branch 'upstream/master' into DefaultTaintTracking-Configuration 2020-04-07 14:39:30 +02:00
Rasmus Wriedt Larsen
7af5f038ab Python: Add missing override to ClassValue.hasAttribute
I was considering if this was actually something different than
Value.hasAttribute, and the names were just accidentially the same. But after
looking at the definition for Value, I'm happy about marking this as an
override (I did not test whether it was neede though):

```codeql
class Value extends TObject {
    ...

    /** Holds if this value has the attribute `name` */
    predicate hasAttribute(string name) { this.(ObjectInternal).hasAttribute(name) }
```
2020-04-07 14:02:53 +02:00
Jonas Jensen
9a1c2d83af Merge pull request #3205 from Semmle/lic/MIT
Relicense under MIT
2020-04-07 13:24:12 +02:00
Pavel Avgustinov
6fc814632c Relicense under MIT 2020-04-07 12:03:26 +01:00
Calum Grant
abf6be6030 C#: Avoid qualifying explicit interface implementations. 2020-04-07 11:17:35 +01:00
Calum Grant
9ec0c8f3ec C#: Qualify type parameters with the entity that declares them 2020-04-07 11:17:35 +01:00
Calum Grant
39eb1a3825 C#: Remove assembly qualifier from some trap-ids. 2020-04-07 11:17:34 +01:00
Mathias Vorreiter Pedersen
5719967a8e C++: Remove single-field case from PostUpdateNode and accept tests 2020-04-07 12:03:28 +02:00
Rasmus Wriedt Larsen
befe73df14 Python: supress non-useful results (w/ tests) for iter str/seq query
Fixes https://github.com/Semmle/ql/issues/3207
2020-04-07 11:46:36 +02:00
Rasmus Wriedt Larsen
94ccc16f85 Python: iter str/seq query gives non-useful results with tests 2020-04-07 11:46:35 +02:00
Rasmus Wriedt Larsen
46ecbef8f2 Python: Consistenly use TestScope instead of Test
It's not clear which one is the correct to use, but there were more uses of
TestScope than Test, so I'm assuming that is the right one ¯\_(ツ)_/¯
2020-04-07 11:42:13 +02:00
Rasmus Wriedt Larsen
2871eb307a Python: Fix (some) shortcomings of tests filter 2020-04-07 11:24:06 +02:00
Rasmus Wriedt Larsen
6ed7b3dd92 Python: Add tests to illustrate shortcomings of current tests filter 2020-04-07 11:22:16 +02:00
Rasmus Wriedt Larsen
75e6470009 Python: Fix FN in py/unused-import 2020-04-07 09:41:42 +02:00
Rasmus Wriedt Larsen
1077ce3a35 Python: Add FN for py/unused-import 2020-04-07 09:26:14 +02:00
Tom Hvitved
6685a5ed4d Merge pull request #3136 from calumgrant/cs/buildless-extraction
C#: Improvements to buildless extraction
2020-04-07 08:52:00 +02:00
Tom Hvitved
bacb11a563 Merge pull request #3150 from calumgrant/cs/enable-nullability
C#: Enable nullability for Autobuilder and Utils projects
2020-04-07 08:51:43 +02:00
Rebecca Valentine
c1ba1345df Python: ObjAPI to ValAPI: WrongNumArgsInCall: ObjectAPI.qll: Adds doc for getAnInferredType 2020-04-06 11:03:42 -07:00
Jonas Jensen
db0d8bbb40 Merge pull request #3208 from geoffw0/issue44
C++: Add test cases more similar to issues/44.
2020-04-06 19:40:24 +02:00
Rebecca Valentine
16eedf9ea5 Merge branch 'master' into python-objectapi-to-valueapi-wrongnumberargumentsincall 2020-04-06 10:35:49 -07:00
Jonas Jensen
0743c42807 Merge remote-tracking branch 'upstream/master' into dataflow-indirect-args
Accepted test results that were in semantic merge conflict between
these branches. The changed results are due to a bug that that's part of
https://github.com/github/codeql-c-analysis-team/issues/35.
2020-04-06 19:26:08 +02:00
Henning Makholm
d1ff3211ef Add extractor fields to test qlpack.yml files. 2020-04-06 19:21:41 +02:00
Henning Makholm
bf579dedd4 Add extractor field in base language QL packs 2020-04-06 18:48:01 +02:00
Geoffrey White
d5accc70e1 C++: Add a test similar to issues/44. 2020-04-06 16:47:24 +01:00
Jonas Jensen
e37aab5002 C++: Suppress FieldAddressInstruction taint
See code comment. This fixes false positives on openjdk/jdk.
2020-04-06 16:14:26 +02:00
Jonas Jensen
3b76509159 C++: Test DefaultTaintTracking field conflation 2020-04-06 16:13:41 +02:00
Geoffrey White
a71ae2b468 C++: Consistent treatment of placement new. 2020-04-06 14:54:15 +01:00
Geoffrey White
492c5f367f C++: Simplify NewDelete.qll. 2020-04-06 14:54:15 +01:00
semmle-qlci
e5d3286ee9 Merge pull request #3183 from asger-semmle/js/bad-url-scheme-check
Approved by esbena
2020-04-06 14:53:15 +01:00
Mathias Vorreiter Pedersen
c577541850 C++: Fix reverse read dataflow consistency failure and accept tests 2020-04-06 15:50:08 +02:00
Geoffrey White
050e239507 C++: Change note. 2020-04-06 14:39:07 +01:00
Geoffrey White
cbe133d0e6 C++: Deprecate freeCall in the legacy wrapper Alloc.qll. 2020-04-06 14:32:49 +01:00
Geoffrey White
e223557201 C++: Wean NewDelete.qll off the legacy wrapper Alloc.qll. 2020-04-06 14:32:15 +01:00
Calum Grant
0d86866ba3 Merge pull request #3160 from hvitved/csharp/null-maybe-fp
C#: Add false-positive test for NullMaybe.ql
2020-04-06 14:30:31 +01:00
Geoffrey White
8059d69bbd C++: Model calls to operator new / delete for NewFreeMismatch.ql. 2020-04-06 14:27:05 +01:00
Geoffrey White
3e9f9645ae C++: Exclude calls to operator new / delete from NewFreeMismatch.ql. 2020-04-06 14:08:00 +01:00
Geoffrey White
97cdcbee63 C++: Test for NewFreeMismatch.ql with operator new / delete. 2020-04-06 13:57:28 +01:00
Jonas Jensen
bf7614a4c9 C++: Move Expr location workaround to Expr.qll
This workaround from `DataFlowUtil.qll` should be useful for any query
that selects an `Expr`. In particular, it's useful for IR data flow.

This commit does not include test changes.
2020-04-06 14:13:22 +02:00
Calum Grant
6cce0de9b2 Merge pull request #3124 from hvitved/csharp/dataflow/sources-and-sinks
C#: Introduce `RemoteFlowSink` class
2020-04-06 12:36:14 +01:00
Asger Feldthaus
7da0345c6a JS: Autoformat 2020-04-06 12:30:04 +01:00
Asger Feldthaus
2c6beadf68 JS: Recognize more forms of scheme checks 2020-04-06 12:30:03 +01:00
Jonas Jensen
d4338473b0 C++: Enforce unique enclosing callable
Every data-flow node should have a unique enclosing function (_callable_
in the terminology of the data-flow library), but this was not evident
for the optimizer, and it led to a bad join order in `pathStep`. This
commit fixes the join order for C++ AST data flow. All other copies of
data flow seem to be fine.

These are the tuple counts for OpenJDK before this commit:

    (231s) Tuple counts for DataFlowImplLocal::pathStep#fffff#cur_delta:
    5882       ~0%       {6} r1 = SCAN DataFlowImplLocal::PathNodeMid#class#ffffff#prev_delta AS I OUTPUT I.<2>, I.<0>, I.<1>, I.<3>, I.<4>, I.<5>
    1063406780 ~0%       {7} r2 = JOIN r1 WITH DataFlowImplCommon::CallContext::relevantFor_dispred#ff AS R ON FIRST 1 OUTPUT r1.<2>, R.<1>, r1.<1>, r1.<0>, r1.<3>, r1.<4>, r1.<5>
    5882       ~1%       {6} r3 = JOIN r2 WITH DataFlowUtil::Node::getFunction_dispred#ff AS R ON FIRST 2 OUTPUT r2.<0>, r2.<6>, r2.<2>, r2.<3>, r2.<4>, r2.<5>
    105        ~0%       {5} r4 = JOIN r3 WITH project#DataFlowImplLocal::LocalFlowBigStep::localFlowBigStep#ffffff_021#join_rhs AS R ON FIRST 2 OUTPUT r3.<2>, r3.<3>, r3.<4>, r3.<5>, R.<2>
    5882       ~1%       {6} r5 = JOIN r2 WITH DataFlowUtil::Node::getFunction_dispred#ff AS R ON FIRST 2 OUTPUT r2.<5>, r2.<2>, r2.<0>, r2.<3>, r2.<4>, r2.<6>
    5882       ~0%       {6} r6 = JOIN r5 WITH DataFlowImplLocal::TNil#ff_1#join_rhs AS R ON FIRST 1 OUTPUT r5.<2>, false, r5.<5>, r5.<1>, r5.<3>, r5.<4>
    0          ~0%       {5} r7 = JOIN r6 WITH DataFlowImplLocal::LocalFlowBigStep::localFlowBigStep#ffffff_02413#join_rhs AS R ON FIRST 3 OUTPUT R.<4>, r6.<3>, r6.<4>, r6.<5>, R.<3>
    0          ~0%       {5} r8 = JOIN r7 WITH DataFlowImplLocal::TNil#ff AS R ON FIRST 1 OUTPUT r7.<1>, r7.<2>, r7.<3>, R.<1>, r7.<4>
    105        ~0%       {5} r9 = r4 \/ r8

The problem is that `DataFlowUtil::Node::getFunction_dispred#ff`
(`getEnclosingCallable`) is joined too late.

After this commit, the tuple counts look like this:

    (13s) Tuple counts for DataFlowImplLocal::pathStep#fffff#cur_delta:
    5882    ~1%       {6} r1 = SCAN DataFlowImplLocal::PathNodeMid#class#ffffff#prev_delta AS I OUTPUT I.<1>, I.<0>, I.<2>, I.<3>, I.<4>, I.<5>
    5882    ~3%       {7} r2 = JOIN r1 WITH DataFlowUtil::Node::getEnclosingCallable_dispred#ff AS R ON FIRST 1 OUTPUT r1.<2>, R.<1>, r1.<1>, r1.<0>, r1.<3>, r1.<4>, r1.<5>
    5882    ~1%       {6} r3 = JOIN r2 WITH DataFlowImplCommon::CallContext::relevantFor_dispred#ff AS R ON FIRST 2 OUTPUT r2.<3>, r2.<6>, r2.<2>, r2.<0>, r2.<4>, r2.<5>
    105     ~0%       {5} r4 = JOIN r3 WITH project#DataFlowImplLocal::LocalFlowBigStep::localFlowBigStep#ffffff_021#join_rhs AS R ON FIRST 2 OUTPUT r3.<2>, r3.<3>, r3.<4>, r3.<5>, R.<2>
    5882    ~1%       {6} r5 = JOIN r2 WITH DataFlowImplCommon::CallContext::relevantFor_dispred#ff AS R ON FIRST 2 OUTPUT r2.<5>, r2.<2>, r2.<3>, r2.<0>, r2.<4>, r2.<6>
    5882    ~0%       {6} r6 = JOIN r5 WITH DataFlowImplLocal::TNil#ff_1#join_rhs AS R ON FIRST 1 OUTPUT r5.<2>, false, r5.<5>, r5.<1>, r5.<3>, r5.<4>
    0       ~0%       {5} r7 = JOIN r6 WITH DataFlowImplLocal::LocalFlowBigStep::localFlowBigStep#ffffff_02413#join_rhs AS R ON FIRST 3 OUTPUT R.<4>, r6.<3>, r6.<4>, r6.<5>, R.<3>
    0       ~0%       {5} r8 = JOIN r7 WITH DataFlowImplLocal::TNil#ff AS R ON FIRST 1 OUTPUT r7.<1>, r7.<2>, r7.<3>, R.<1>, r7.<4>
    105     ~0%       {5} r9 = r4 \/ r8

There is a slight slowdown coming from the introduction of a new
predicate `DataFlowImplLocal::pathStep#fffff#join_rhs`, which is used
only in the standard order:

    (12s) Tuple counts for DataFlowImplLocal::pathStep#fffff#join_rhs:
    282057  ~0%     {2} r1 = SCAN DataFlowImplCommon::CallContext::relevantFor_dispred#ff AS I OUTPUT I.<1>, I.<0>
    9159890 ~1%     {2} r2 = JOIN r1 WITH DataFlowUtil::Node::getEnclosingCallable_dispred#ff_10#join_rhs AS R ON FIRST 1 OUTPUT R.<1>, r1.<1>
                    return r2

The evaluation of `unique` is cheap but not free:

    DataFlowUtil::Node::getEnclosingCallable_dispred#ff .............. 3.9s
    DataFlowUtil::Node::getEnclosingCallable_dispred#ff_10#join_rhs .. 3.5s

The first of these two predicates evaluates `unique`, and the second
simply reorders columns. They take about the same time, which suggests
that `unique` is about as fast as it can be, given the number of tuples
it needs to push around. Note that the column reordering predicate is
only needed because of the standard order.
2020-04-06 12:04:39 +02:00
Mathias Vorreiter Pedersen
3aa293210d C++: Ensure that only non-conflated chi instructions are used everywhere 2020-04-06 12:02:56 +02:00
James Fletcher
5034d40e64 Merge pull request #3203 from jf205/sd-55
CodeQL support docs: combine table and footnotes into single snippet
2020-04-06 09:52:02 +01:00
james
d2b0599b63 docs: combine table and footnotes 2020-04-06 09:27:30 +01:00