Commit Graph

19777 Commits

Author SHA1 Message Date
Mathias Vorreiter Pedersen
bcd84efe8d C++: Add += and friends to adjustedSink 2020-02-10 15:50:52 +01:00
Jonas Jensen
cf1bc693b4 C++: Fix coversEntireVariable perf in AliasedSSA
This predicate got an unfortunate join order, leading to these tuple
counts on ElektraInitiative/libelektra:

    (290s) Tuple counts for AliasedSSA::VariableMemoryLocation::coversEntireVariable_dispred#f:
    57117     ~0%     {3} r1 = SCAN IRType::IRType::getByteSize_dispred#ff AS I OUTPUT 0, (I.<1> * 8), I.<0>
    421445272 ~0%     {3} r2 = JOIN r1 WITH AliasedSSA::VariableMemoryLocation#fffffff_5601#join_rhs AS R ON FIRST 2 OUTPUT R.<3>, r1.<2>, R.<2>
    103282    ~2%     {1} r3 = JOIN r2 WITH AliasConfiguration::Allocation::getIRType_dispred#ff AS R ON FIRST 2 OUTPUT r2.<2>
                      return r3

With this commit, we get these tuple counts instead:

    (0s) Tuple counts for AliasedSSA::VariableMemoryLocation::varIRTypeHasBitRange#bff:
    361874 ~0%     {3} r1 = SCAN AliasedSSA::VariableMemoryLocation#fffffff AS I OUTPUT I.<1>, 0, I.<0>
    361874 ~0%     {3} r2 = JOIN r1 WITH AliasConfiguration::Allocation::getIRType_dispred#ff AS R ON FIRST 1 OUTPUT R.<1>, 0, r1.<2>
    361874 ~1%     {3} r3 = JOIN r2 WITH IRType::IRType::getByteSize_dispred#ff AS R ON FIRST 1 OUTPUT r2.<2>, 0, (R.<1> * 8)
                   return r3

    (0s) Tuple counts for AliasedSSA::VariableMemoryLocation::coversEntireVariable_dispred#f:
    103282 ~2%     {1} r1 = JOIN AliasedSSA::VariableMemoryLocation#fffffff_056#join_rhs AS L WITH AliasedSSA::VariableMemoryLocation::varIRTypeHasBitRange#bff AS R ON FIRST 3 OUTPUT L.<0>
    103282 ~2%     {1} r2 = STREAM DEDUP r1
                   return r2
2020-02-10 15:18:34 +01:00
Rasmus Wriedt Larsen
2bffbf0734 Python: Add testcases for py/import-own-module
You can try out:

python2 -c "import pkg_ok; print(pkg_ok.foo1); print(pkg_ok.foo2); print(pkg_ok.foo3); print(pkg_ok.foo4); print(pkg_ok.foo5); print(pkg_ok.Foo3); print(pkg_ok.Foo5); print(pkg_ok.pkg_ok)"

python3 -c "import pkg_ok; print(pkg_ok.foo1); print(pkg_ok.foo2); print(pkg_ok.foo3); print(pkg_ok.foo4); print(pkg_ok.foo5); print(pkg_ok.Foo3); print(pkg_ok.Foo5); print(pkg_ok.pkg_ok)"
2020-02-10 15:16:47 +01:00
Jonas Jensen
47c12817ad C++: IR DataFlowUtil::modelFlow join order fix
We had these tuple counts on ElektraInitiative/libelektra (note that the
`modelFlow` predicate got inlined into
`simpleInstructionLocalFlowStep`):

    (652s) Tuple counts for DataFlowUtil::simpleInstructionLocalFlowStep#ff:
    ...
    19701      ~1%      {4} r27 = JOIN r26 WITH Instruction::SideEffectInstruction::getPrimaryInstruction_dispred#3#ff_10#join_rhs AS R ON FIRST 1 OUTPUT R.<1>, r26.<2>, r26.<1>, r26.<0>
    7908       ~0%      {3} r28 = JOIN r27 WITH SSAConstruction::Cached::getInstructionIndex#ff@staged_ext AS R ON FIRST 2 OUTPUT r27.<0>, r27.<2>, r27.<3>
    4023       ~0%      {3} r29 = JOIN r28 WITH Instruction::WriteSideEffectInstruction#class#ff AS R ON FIRST 1 OUTPUT r28.<1>, r28.<2>, r28.<0>
    ...
    1060807009 ~3%      {3} r34 = JOIN r33 WITH SSAConstruction::Cached::getInstructionIndex#ff_10#join_rhs AS R ON FIRST 1 OUTPUT R.<1>, r33.<1>, r33.<2>
    15670      ~5%      {2} r35 = JOIN r34 WITH Instruction::SideEffectInstruction::getPrimaryInstruction_dispred#3#ff AS R ON FIRST 2 OUTPUT r34.<0>, r34.<2>
    7973       ~0%      {2} r36 = JOIN r35 WITH Instruction::ReadSideEffectInstruction::getSideEffectOperand_dispred#ff AS R ON FIRST 1 OUTPUT R.<1>, r35.<1>
    ...

In this predicate there are two cases (`WriteSideEffectInstruction` and
`ReadSideEffectInstruction`) where we need to join on both the call and
the argument index of a side effect. It works well enough for the first
case, `WriteSideEffectInstruction`, where the call is joined on before
the index, but it explodes in the second case,
`ReadSideEffectInstruction`, where the index is joined first. To fix the
second case, and to guard against future optimizer accidents in the
first case, this commit changes both of those cases to use a new helper
predicate that makes it possible to join on both columns at once. The
resulting tuple counts are:

    (3s) Tuple counts for DataFlowUtil::simpleInstructionLocalFlowStep#ff:
    ...
    7908    ~0%      {3} r27 = JOIN r26 WITH DataFlowUtil::getSideEffectFor#fff AS R ON FIRST 2 OUTPUT R.<2>, r26.<2>, r26.<0>
    4023    ~0%      {3} r28 = JOIN r27 WITH Instruction::WriteSideEffectInstruction#class#ff AS R ON FIRST 1 OUTPUT r27.<1>, r27.<2>, r27.<0>
    ...
    15670   ~5%      {2} r33 = JOIN r32 WITH DataFlowUtil::getSideEffectFor#fff AS R ON FIRST 2 OUTPUT R.<2>, r32.<2>
    7973    ~0%      {2} r34 = JOIN r33 WITH Instruction::ReadSideEffectInstruction::getSideEffectOperand_dispred#ff AS R ON FIRST 1 OUTPUT R.<1>, r33.<1>
    ...

The bulge is now limited to a factor of two, and that's just because I
didn't write separate versions of `getSideEffectFor` for
`ReadSideEffectInstruction` and `WriteSideEffectInstruction`.
2020-02-10 15:11:30 +01:00
Erik Krogh Kristensen
67cd303a91 add change note 2020-02-10 13:51:48 +01:00
Mathias Vorreiter Pedersen
99a9d7f676 C++: Simplify 2020-02-10 13:01:40 +01:00
Erik Krogh Kristensen
8e316d2f05 add unary type-tracking predicates 2020-02-10 12:51:09 +01:00
Calum Grant
a95ef31984 C#: Analysis change notes 2020-02-10 11:36:30 +00:00
Calum Grant
5fef77bf44 C#: Handle expressions of typeImplicitStackAllocArrayCreationExpressionSyntax 2020-02-10 11:30:12 +00:00
Mathias Vorreiter Pedersen
6804018a64 C++: Accept output 2020-02-10 11:37:40 +01:00
Mathias Vorreiter Pedersen
522c629441 C++: Move fix to adjustedSink to avoid generating too many instructions 2020-02-10 11:37:26 +01:00
Pavel Avgustinov
09960e0a7d Merge pull request #2791 from aibaars/language-spec-fixes
Fix a few typos in the QL grammar
2020-02-10 10:33:00 +00:00
Erik Krogh Kristensen
0f511c92b4 Merge remote-tracking branch 'upstream/master' into FalsySanitizer 2020-02-10 09:54:58 +01:00
Mathias Vorreiter Pedersen
52bc25b608 C++: Accept output 2020-02-10 08:50:29 +01:00
SpaceWhite
949914e48a Add XPathInjection 2020-02-10 13:48:02 +09:00
Mathias Vorreiter Pedersen
bb30275e2e C++: Fix false negatives for postfix crement expressions 2020-02-09 21:35:07 +01:00
Tom Hvitved
9b699618a8 Data flow: Improve performance of flowFwdStore() 2020-02-09 19:34:04 +01:00
Mathias Vorreiter Pedersen
bcb4759b6a Merge pull request #2795 from agustingianni/master
cpp: Adds a member predicate to get a FunctionAccess from a Function
2020-02-08 10:39:36 +01:00
Robert Marsh
3266a89023 C++: document DynamicAllocationInstruction 2020-02-07 16:56:57 -08:00
Robert Marsh
3ea3519674 C++/C#: sync changes 2020-02-07 16:37:42 -08:00
Robert Marsh
7f69cdfb56 C++: Dynamic allocations in IR alias analysis 2020-02-07 16:37:36 -08:00
Robert Marsh
05c8610bbc C++: tests for alias analysis of malloc 2020-02-07 16:35:58 -08:00
Robert Marsh
68097116b0 C++:autoformat 2020-02-07 15:51:23 -08:00
semmle-qlci
37360e7d93 Merge pull request #2794 from esbena/js/move-EnumeratedPropName
Approved by asgerf
2020-02-07 21:31:37 +00:00
semmle-qlci
76ba48c6fb Merge pull request #2790 from esbena/js/model-send
Approved by asgerf
2020-02-07 21:30:54 +00:00
Robert Marsh
d1d19a7446 Merge branch 'master' into rdmarsh/cpp/ir-flow-through-outparams
Update test expectations
2020-02-07 12:24:05 -08:00
Asger Feldthaus
e4844bfad2 JS: Fix deprecated API usage 2020-02-07 17:17:48 +00:00
Asger Feldthaus
ad10414604 JS: Update expected output of existing test 2020-02-07 16:57:57 +00:00
Agustin Gianni
033eeab41e cpp: Adds a member predicate to get a FunctionAccess from a Function
- Adds a member predicate to get a FunctionAccess from a Function
- Adds QLDoc to getACallToThisFunction.
2020-02-07 16:45:14 +01:00
Erik Krogh Kristensen
06e13cb3a1 Merge branch 'master' of git.semmle.com:Semmle/ql into FalsySanitizer 2020-02-07 16:13:02 +01:00
Erik Krogh Kristensen
c6668da02e expand how indirectCommandArguments are found 2020-02-07 15:00:05 +01:00
Asger Feldthaus
254af4f3a8 JS: Rewrite LodashUnderscore::AnalyzedThisInBoundCallback 2020-02-07 13:58:07 +00:00
Erik Krogh Kristensen
dd9e3d2fec expose TaintTracking::arrayFunctionTaintStep and add a step for "concat" 2020-02-07 14:57:32 +01:00
Asger Feldthaus
fea5a4331d JS: Rewrite React::AnalyzedThisInBoundCallback 2020-02-07 13:55:42 +00:00
Asger Feldthaus
3b28bdbeed JS: Rewrite AnalyzedThisInArrayIterationFunction 2020-02-07 13:55:36 +00:00
Asger Feldthaus
f942e69482 JS: Improve flow through partial invokes 2020-02-07 13:54:14 +00:00
Anders Schack-Mulligen
85adc3be10 Java: Add String.join as default taint step. 2020-02-07 14:43:31 +01:00
Esben Sparre Andreasen
dcdaa96570 JS: remove unused imports 2020-02-07 14:10:50 +01:00
Esben Sparre Andreasen
cb30329b3d JS: make DynamicPropertyAccess.qll from PrototypePollutionUtility.ql 2020-02-07 13:57:52 +01:00
Calum Grant
3c8aeb946a Merge pull request #2729 from hvitved/csharp/assignment-dataflow-node
C#: Add new class `AssignableDefinitionNode` to the data-flow library
2020-02-07 12:53:55 +00:00
Anders Schack-Mulligen
ee3af0a247 Java: Add String.format as default taint step. 2020-02-07 13:43:35 +01:00
Erik Krogh Kristensen
1ece6b9afe update expected output of tests 2020-02-07 12:57:51 +01:00
semmle-qlci
125c6a071c Merge pull request #2787 from asger-semmle/js/lazy-cache-test-case
Approved by esbena
2020-02-07 11:53:04 +00:00
Arthur Baars
c91815f44d Remove trailing ; in QL language spec 2020-02-07 12:47:14 +01:00
Arthur Baars
c431d47481 Fix typos in QL language spec 2020-02-07 12:46:41 +01:00
Esben Sparre Andreasen
736ccb98c2 JS: model the send library for js/path-injection 2020-02-07 12:45:32 +01:00
Shati Patel
4cbf7d2a6d Merge pull request #2789 from jf205/codeql-178
QL language handbook: add link to information about module resolution in QL spec
2020-02-07 12:27:02 +01:00
Tom Hvitved
f30a42ce26 Data flow: Fix bad join-order in TPathNodeSink
Avoids a Cartesian product on nodes:

```
[2020-02-07 11:01:22] (432s) Tuple counts for dom#DataFlowImpl::TPathNodeSink#ff:
                      0          ~0%      {2} r1 = JOIN DataFlowImpl::Configuration::isSource_dispred#ff AS L WITH DataFlowImpl::Configuration::isSink_dispred#ff AS R ON FIRST 2 OUTPUT R.<1>, R.<0>
                      101611     ~0%      {2} r2 = SCAN DataFlowImpl::PathNodeMid#class#ffffff AS I OUTPUT I.<5>, I.<0>
                      3534537047 ~3%      {3} r3 = JOIN r2 WITH DataFlowImpl::Configuration::isSink_dispred#ff AS R ON FIRST 1 OUTPUT r2.<1>, R.<1>, R.<0>
                      251        ~41%     {3} r4 = JOIN r3 WITH project#DataFlowImpl::pathStep#fffff AS R ON FIRST 2 OUTPUT R.<2>, r3.<2>, r3.<1>
                      251        ~50%     {2} r5 = JOIN r4 WITH DataFlowImpl::TNil#ff_1#join_rhs AS R ON FIRST 1 OUTPUT r4.<2>, r4.<1>
                      251        ~50%     {2} r6 = r1 \/ r5
                      323        ~67%     {3} r7 = JOIN r6 WITH DataFlowImpl::flow#ff AS R ON FIRST 1 OUTPUT r6.<1>, r6.<0>, R.<1>
                      288        ~58%     {3} r8 = SELECT r7 ON r7.<2> >= r7.<0>
                      251        ~53%     {3} r9 = SELECT r8 ON r8.<2> <= r8.<0>
                      251        ~50%     {2} r10 = SCAN r9 OUTPUT r9.<1>, r9.<0>
```
2020-02-07 12:08:31 +01:00
Calum Grant
389e6266d9 Merge pull request #2773 from hvitved/csharp/useless-assignment-to-local-default
C#: Remove false positives for `cs/useless-assignment-to-local`
2020-02-07 10:37:19 +00:00
james
f2320bbe56 docs: add link to module resolution in ql spec 2020-02-07 10:26:31 +00:00