Commit Graph

3366 Commits

Author SHA1 Message Date
yo-h
43bcd5b26c Add guidelines for experimental CodeQL queries and libraries 2020-02-24 15:08:31 -05:00
Jonas Jensen
2d9df70abc Merge pull request #2887 from MathiasVP/fix-ir-gen-switch
C++: Fix IR generation for switch statements
2020-02-24 13:29:27 +01:00
Mathias Vorreiter Pedersen
ed430ce855 C++/C#: Bind parameter in new case. 2020-02-24 09:12:14 +01:00
Mathias Vorreiter Pedersen
af364e66fc C++/C#: Move sanity check inside InstructionSanity module and accept tests 2020-02-23 20:53:49 +01:00
Dave Bartolomeo
170331b105 C++: Better fix for void type on buffer access
Fixes issue https://github.com/github/codeql-c-analysis-team/issues/20

This change undoes the workaround in https://github.com/Semmle/ql/pull/2736, and replaces it with a fix for the underlying cause. The problem was that the IR construction code for side effects incorrectly assumed that `BufferAccessOpcode` included `SizedBufferAccessOpcode`. I think that was actually a perfectly reasonable assumption to make, so I changed the `Opcode` hierarchy to make it true.
2020-02-21 18:46:32 -07:00
Mathias Vorreiter Pedersen
da41cbca06 C#: Add similar fix to translation of switch statements in C# 2020-02-21 13:33:54 +01:00
Anders Schack-Mulligen
771cb754c2 Merge pull request #2822 from hvitved/dataflow/node-cand-simple-call-context
Data flow: Track simple call contexts in `nodeCand[Fwd]1`
2020-02-21 10:02:06 +01:00
Tom Hvitved
0cc3218115 Merge pull request #2872 from aschackmull/dataflow/pathstep-localflow-join
Java/C++/C#: Improve join-order in pathStep predicate
2020-02-21 09:39:17 +01:00
Mathias Vorreiter Pedersen
780010d8f9 C++/C#: Sync identical files 2020-02-20 22:15:06 +01:00
Tom Hvitved
a772b82fea Address review comments 2020-02-20 19:48:49 +01:00
Robert Marsh
bed6d2b225 Merge branch 'master' into rdmarsh/cpp/malloc-alias-locations 2020-02-19 16:44:13 -08:00
Anders Schack-Mulligen
91166431d2 Java/C++/C#: s/Callable/DataFlowCallable/ 2020-02-19 17:23:01 +01:00
Anders Schack-Mulligen
c6016bb08c Java/C++/C#: Improve join-order in pathStep predicate 2020-02-19 14:47:39 +01:00
Mathias Vorreiter Pedersen
59a19679ea C++/C#: Sync identical files after merge 2020-02-19 11:06:00 +01:00
Mathias Vorreiter Pedersen
bbcc1e1c37 Merge branch 'master' into sync-ir-valuenumbering-internals 2020-02-19 10:43:48 +01:00
Robert Marsh
adfe5f30a1 Merge branch 'master' into rdmarsh/cpp/ir-flow-through-outparams 2020-02-18 08:42:27 -08:00
Tom Hvitved
a695b567ec Data flow: Sync files 2020-02-17 19:39:52 +01:00
Tom Hvitved
3a4f52315c Data flow: Track simple call contexts in nodeCand[Fwd]1 2020-02-17 19:37:35 +01:00
semmle-qlci
ecad925101 Merge pull request #2631 from hvitved/dataflow/generalize-flow-summaries
Approved by aschackmull
2020-02-17 18:22:46 +00:00
Tom Hvitved
0e7838aca5 Data flow: Sync files 2020-02-17 15:08:26 +01:00
Tom Hvitved
c7aa31d90a Address review comments 2020-02-17 15:07:53 +01:00
Tom Hvitved
28307399f8 Data flow: Sync files 2020-02-17 10:45:35 +01:00
Tom Hvitved
bc6c4744b1 Data flow: Follow-up changes to FlowExploration module 2020-02-17 10:43:26 +01:00
Tom Hvitved
307ac7f0b3 Data flow: Remove UntypedAccessPath again 2020-02-17 10:32:27 +01:00
Mathias Vorreiter Pedersen
8b8a8cae5b C++/C#: Sync identical files 2020-02-14 16:11:57 +01:00
Mathias Vorreiter Pedersen
98db6d8fd7 C#: Fix format and sync files 2020-02-14 08:22:25 +01:00
Mathias Vorreiter Pedersen
d4c6f487bc C++/C#: Fix sync config file for value numbering sharing 2020-02-13 22:32:52 +01:00
Tom Hvitved
b5b0c2b8cf Data flow: Sync files 2020-02-13 16:34:06 +01:00
Tom Hvitved
6be2bf8374 Address review comments 2020-02-13 16:34:06 +01:00
Jonas Jensen
033a4c30ea C++: Fix perf of IR value numbering
On some snapshots, notably ffmpeg, the IR `ValueNumbering` recursion
would generate billions of tuples and eventually run out of space.

It turns out it was fairly common for an `Instruction` to get more than
one `ValueNumber` in the base cases for `VariableAddressInstruction` and
`InitializeParameterInstruction`, and it could also happen in an
instruction with more than one operand of the same `OperandTag`. When a
binary operation was applied to an instruction with `m` value numbers
and another instruction with `n` value numbers, the result would get
`m * n` value numbers. This led to doubly-exponential growth in the
number of value numbers in rare cases.

The underlying reason why a `VariableAddressInstruction` could get
multiple value numbers is that it was keyed on the associated
`IRVariable`, and the `IRVariable` is defined in part by the type of its
underlying `Variable` (or other AST element). If the extractor defines a
variable to have multiple types because of linker ambiguity, this leads
to the creation of multiple `IRVariable`s. That should ideally be solved
in `TIRVariable.qll`, but for now I've put a workaround in
`ValueNumberingInternal.qll` instead.

To remove the problem with instructions having multiple operands, the
construction in `Operand.qll` will now filter out any such operand. It
wasn't enough to apply that filter to the `raw` stage, so I've applied
it to all three stages.
2020-02-12 14:38:41 +01:00
Tom Hvitved
78380f5d59 Merge pull request #2658 from calumgrant/cs/serialization-check-bypass-type
C#: Fix cs/serialization-check-bypass
2020-02-12 10:26:01 +01:00
Robert Marsh
837fe84cec C++/C#: autoformat Opcode.qll 2020-02-11 12:18:45 -08:00
Robert Marsh
f467260815 C++: respond to PR comments. 2020-02-11 12:17:46 -08:00
Tom Hvitved
1948446ad3 Address review comments 2020-02-11 11:56:40 +01:00
Tom Hvitved
96e71c731d C#: Add DB stats for explicitly_sized_array_creation 2020-02-10 20:33:57 +01:00
Tom Hvitved
bbf082b285 C#: Extract stackalloc information 2020-02-10 20:32:52 +01:00
Calum Grant
803cb3f4d1 C#: Address review comment
- Flow from expressions with a value is excluded.
2020-02-10 16:02:29 +00:00
Tom Hvitved
9b699618a8 Data flow: Improve performance of flowFwdStore() 2020-02-09 19:34:04 +01:00
Robert Marsh
3266a89023 C++: document DynamicAllocationInstruction 2020-02-07 16:56:57 -08:00
Robert Marsh
3ea3519674 C++/C#: sync changes 2020-02-07 16:37:42 -08:00
Robert Marsh
d1d19a7446 Merge branch 'master' into rdmarsh/cpp/ir-flow-through-outparams
Update test expectations
2020-02-07 12:24:05 -08:00
Calum Grant
3c8aeb946a Merge pull request #2729 from hvitved/csharp/assignment-dataflow-node
C#: Add new class `AssignableDefinitionNode` to the data-flow library
2020-02-07 12:53:55 +00:00
Tom Hvitved
f30a42ce26 Data flow: Fix bad join-order in TPathNodeSink
Avoids a Cartesian product on nodes:

```
[2020-02-07 11:01:22] (432s) Tuple counts for dom#DataFlowImpl::TPathNodeSink#ff:
                      0          ~0%      {2} r1 = JOIN DataFlowImpl::Configuration::isSource_dispred#ff AS L WITH DataFlowImpl::Configuration::isSink_dispred#ff AS R ON FIRST 2 OUTPUT R.<1>, R.<0>
                      101611     ~0%      {2} r2 = SCAN DataFlowImpl::PathNodeMid#class#ffffff AS I OUTPUT I.<5>, I.<0>
                      3534537047 ~3%      {3} r3 = JOIN r2 WITH DataFlowImpl::Configuration::isSink_dispred#ff AS R ON FIRST 1 OUTPUT r2.<1>, R.<1>, R.<0>
                      251        ~41%     {3} r4 = JOIN r3 WITH project#DataFlowImpl::pathStep#fffff AS R ON FIRST 2 OUTPUT R.<2>, r3.<2>, r3.<1>
                      251        ~50%     {2} r5 = JOIN r4 WITH DataFlowImpl::TNil#ff_1#join_rhs AS R ON FIRST 1 OUTPUT r4.<2>, r4.<1>
                      251        ~50%     {2} r6 = r1 \/ r5
                      323        ~67%     {3} r7 = JOIN r6 WITH DataFlowImpl::flow#ff AS R ON FIRST 1 OUTPUT r6.<1>, r6.<0>, R.<1>
                      288        ~58%     {3} r8 = SELECT r7 ON r7.<2> >= r7.<0>
                      251        ~53%     {3} r9 = SELECT r8 ON r8.<2> <= r8.<0>
                      251        ~50%     {2} r10 = SCAN r9 OUTPUT r9.<1>, r9.<0>
```
2020-02-07 12:08:31 +01:00
Calum Grant
389e6266d9 Merge pull request #2773 from hvitved/csharp/useless-assignment-to-local-default
C#: Remove false positives for `cs/useless-assignment-to-local`
2020-02-07 10:37:19 +00:00
Robert Marsh
692207472a Merge branch 'master' into rdmarsh/cpp/ir-flow-through-outparams 2020-02-06 11:42:30 -08:00
Tom Hvitved
b3af3ad12f Data flow: Fix bad join order in getReturnPosition()
Joining on the enclosing callable before the kind is crucial, as witnessed by this pipeline:

```
[2020-02-06 17:58:21] (1086s) Starting to evaluate predicate DataFlowImplCommon::getReturnPosition#ff/2@83c546
[2020-02-06 18:53:16] (4382s) Tuple counts for DataFlowImplCommon::getReturnPosition#ff:
                      385478      ~1%     {3} r1 = SCAN DataFlowImplCommon::Cached::TReturnPosition0#fff@staged_ext AS I OUTPUT I.<2>, I.<0>, I.<1>
                      385478      ~2%     {3} r2 = JOIN r1 WITH DataFlowImplCommon::Cached::TReturnPosition0#fff_2#join_rhs AS R ON FIRST 1 OUTPUT r1.<2>, r1.<1>, r1.<0>
                      58638116860 ~0%     {3} r3 = JOIN r2 WITH DataFlowImplCommon::ReturnNodeExt::getKind_dispred#ff_10#join_rhs AS R ON FIRST 1 OUTPUT R.<1>, r2.<1>, r2.<2>
                      914049      ~0%     {2} r4 = JOIN r3 WITH DataFlowImplCommon::returnNodeGetEnclosingCallable#ff AS R ON FIRST 2 OUTPUT r3.<0>, r3.<2>
                                          return r4
```
2020-02-06 19:06:40 +01:00
Mathias Vorreiter Pedersen
2017ca8154 C++/C#: Sync identical files 2020-02-06 15:53:23 +01:00
Mathias Vorreiter Pedersen
8041804bac C#: Include TValueNumber in internal files 2020-02-06 15:39:27 +01:00
Robert Marsh
ffaaed0550 C++: separate IR ValueNumber newtype and interface 2020-02-06 15:35:20 +01:00
Mathias Vorreiter Pedersen
aaa6233a99 C++/C#: Sync identical files 2020-02-06 11:24:07 +01:00