Commit Graph

33884 Commits

Author SHA1 Message Date
Jonas Jensen
4830e43b3e C++: Fix overlappingVariableMemoryLocations perf
The `overlappingVariableMemoryLocations` predicate was a helper
predicate introduced to fix a join-order issue in
`overlappingIRVariableMemoryLocations`. Unfortunately it caused a
performance issue of its own because it could grow too large. On the
small project (38MB zip) awslabs/s2n there were 181M rows in
`overlappingVariableMemoryLocations`, and it took 134s to evaluate.

The fix is to collapse the two predicates into one and fix join ordering
by including an extra column in the predicates being joined.

In addition, some parameters were reordered to avoid the overhead of
auto-generated `join_rhs` predicates.

Tuple counts of `overlappingVariableMemoryLocations` before:

    623285    ~176%     {2} r1 = JOIN AliasedSSA::isCoveredOffset#fff_120#join_rhs AS L WITH AliasedSSA::isCoveredOffset#fff_120#join_rhs AS R ON FIRST 2 OUTPUT L.<2>, R.<2>
    119138    ~3%       {2} r2 = SCAN AliasedSSA::VariableMemoryLocation::getVirtualVariable_dispred#ff AS I OUTPUT I.<1>, I.<0>
    172192346 ~0%       {2} r3 = JOIN r2 WITH AliasedSSA::hasUnknownOffset#ff_10#join_rhs AS R ON FIRST 1 OUTPUT R.<1>, r2.<1>
    172815631 ~0%       {2} r4 = r1 \/ r3
    172192346 ~0%       {2} r5 = JOIN r2 WITH AliasedSSA::hasUnknownOffset#ff_10#join_rhs AS R ON FIRST 1 OUTPUT r2.<1>, R.<1>
    345007977 ~87%      {2} r6 = r4 \/ r5
                        return r6

Tuple counts of `overlappingIRVariableMemoryLocations` after:

    117021 ~134%     {2} r1 = JOIN AliasedSSA::isCoveredOffset#ffff AS L WITH AliasedSSA::isCoveredOffset#ffff AS R ON FIRST 3 OUTPUT L.<3>, R.<3>
    201486 ~1%       {2} r2 = JOIN AliasedSSA::hasUnknownOffset#fff AS L WITH AliasedSSA::hasVariableAndVirtualVariable#fff AS R ON FIRST 2 OUTPUT L.<2>, R.<2>
    318507 ~26%      {2} r3 = r1 \/ r2
    201486 ~3%       {2} r4 = JOIN AliasedSSA::hasUnknownOffset#fff AS L WITH AliasedSSA::hasVariableAndVirtualVariable#fff AS R ON FIRST 2 OUTPUT R.<2>, L.<2>
    519993 ~92%      {2} r5 = r3 \/ r4
                     return r5
2019-12-27 16:06:24 +01:00
Calum Grant
68f42a6f47 C#: Analysis change notes 2019-12-27 12:07:26 +00:00
Calum Grant
3db900b183 C#: Remove false positive and update test output
C#: Mark results as GOOD
2019-12-27 12:07:19 +00:00
Calum Grant
fd0225ca59 C#: Add test 2019-12-27 11:44:39 +00:00
Jonas Jensen
618bf2e29e C++: IR data flow through total chi operands 2019-12-27 11:44:41 +01:00
Jonas Jensen
64c79bf9e1 C++: Deprecate UninitializedNode in IR data flow
It's not used outside of tests, and it's not useful. It will break the
tests when we start allowing flow through chi nodes.
2019-12-27 11:21:33 +01:00
Calum Grant
0f178be12e C#: Update change notes. 2019-12-23 15:29:20 +00:00
Calum Grant
a059c13f6c C#: Add test for tuple expressions. 2019-12-23 15:18:28 +00:00
Calum Grant
63afb30797 C#: Tests for tuple expressions. 2019-12-23 15:18:21 +00:00
Calum Grant
e83b159bf8 C#: Handle TupleType expression. 2019-12-23 15:18:21 +00:00
Calum Grant
ad764b1dc6 C#: Update .gitignore 2019-12-23 14:31:54 +00:00
Mathias Vorreiter Pedersen
c9fe3e4d2d C++: Updated upgrade script 2019-12-23 14:13:12 +01:00
Mathias Vorreiter Pedersen
bb282f403e Fix comments
Co-Authored-By: Jonas Jensen <jbj@github.com>
2019-12-23 12:37:18 +01:00
Mathias Vorreiter Pedersen
11a545e08e C++: Removed abstract classes from binary and assignment operations 2019-12-23 11:52:12 +01:00
Mathias Vorreiter Pedersen
6998336fb9 C++: Format .dbscheme file 2019-12-23 10:57:38 +01:00
Mathias Vorreiter Pedersen
46421efcef C++: Rename crement operations 2019-12-23 10:41:14 +01:00
Mathias Vorreiter Pedersen
1b29e6c082 Remove @prefix_crement_oper_expr and @postfix_crement_oper_expr clauses
Co-Authored-By: Jonas Jensen <jbj@github.com>
2019-12-23 10:28:35 +01:00
Jonas Jensen
7e84453ec9 Merge pull request #2542 from geoffw0/datetime
C++: Sort through the leap year and japanese era queries
2019-12-23 10:13:12 +01:00
semmle-qlci
f921cf7d01 Merge pull request #2512 from erik-krogh/moarExceptions
Approved by esbena, max-schaefer
2019-12-20 20:31:50 +00:00
Dave Bartolomeo
5b5d2f2b67 Merge pull request #2154 from rdmarsh2/rdmarsh/cpp/ir-callee-side-effects
C++: add InitializeIndirection for pointer params
2019-12-20 13:13:54 -07:00
Mathias Vorreiter Pedersen
cb22702908 C++: Added update script 2019-12-20 18:38:14 +01:00
Mathias Vorreiter Pedersen
006c8bb0cd C++: Remove abstract classes from unary operations 2019-12-20 18:38:09 +01:00
Rasmus Wriedt Larsen
92e272cc03 Python: Address comments for modernising Variables/ 2019-12-20 15:58:51 +01:00
Rasmus Wriedt Larsen
b8a9a353b8 Python: Autoformat Variables/* 2019-12-20 15:08:20 +01:00
Rasmus Wriedt Larsen
25ab0ed20f Python: Modernise Variables/MonkeyPatched.qll 2019-12-20 15:05:49 +01:00
Rasmus Wriedt Larsen
994ad197c4 Python: Add Module::builtinModule() 2019-12-20 15:05:49 +01:00
Rasmus Wriedt Larsen
58bb16e5dd Python: Modernise Variables/Undefined.qll 2019-12-20 15:05:49 +01:00
Rasmus Wriedt Larsen
34f9135492 Python: Modernise py/unused-parameter 2019-12-20 15:05:49 +01:00
Rasmus Wriedt Larsen
15bc4cd090 Python: Add override helpers to Value classes 2019-12-20 15:05:49 +01:00
Rasmus Wriedt Larsen
aba3ac7b66 Python: Modernise py/uninitialized-local-variable 2019-12-20 15:05:49 +01:00
Rasmus Wriedt Larsen
697a006ef2 Python: Modernise py/undefined-global-variable 2019-12-20 15:05:49 +01:00
Rasmus Wriedt Larsen
5faa7e7127 Python: Add ModuleValue::hasCompleteExportInfo 2019-12-20 15:05:49 +01:00
Rasmus Wriedt Larsen
3ffea599f1 Python: Rewrite casts for py/undefined-global-variable 2019-12-20 15:05:49 +01:00
Rasmus Wriedt Larsen
81e27aab8d Python: Modernise py/unused-loop-variable 2019-12-20 15:05:49 +01:00
Rasmus Wriedt Larsen
8f7ba0a06d Python: Modernise py/local-shadows-global 2019-12-20 15:05:49 +01:00
Rasmus Wriedt Larsen
9f4088413a Python: Modernise py/local-shadows-builtin
+ moved `scope instanceof Function` so it makes more sense :)
2019-12-20 15:05:49 +01:00
yo-h
cc7f98e0f6 Merge pull request #2555 from hvitved/csharp/xml-sync
C#: Sync `XML.qll` with other languages
2019-12-20 09:03:55 -05:00
Jonas Jensen
de55a6846f Merge pull request #2204 from alexet/cache-to-string
Cache the computation of core toString predicates for cpp c# and java.
2019-12-20 14:54:46 +01:00
Tom Hvitved
665d38647d Merge pull request #2557 from calumgrant/cs/extractor-label-catch
C# extractor: Catch exceptions when generating trap
2019-12-20 13:09:21 +01:00
Erik Krogh Kristensen
a0b5aa5ae4 more precise heuristic to identify allowed call targets 2019-12-20 10:51:39 +01:00
Jonas Jensen
18d4772508 Merge pull request #2463 from geoffw0/overflowcalc
CPP: Allocation and Deallocation libraries
2019-12-19 21:27:42 +01:00
Jonas Jensen
939979ddef Merge branch 'master' into overflowcalc 2019-12-19 14:12:00 +01:00
Jonas Jensen
a13748f484 Merge pull request #2259 from rdmarsh2/rdmarsh/cpp/default-taint-tracking-sources
C++: move sources into DefaultTaintTracking.qll
2019-12-19 14:09:41 +01:00
Jonas Jensen
4fffaabab9 Merge pull request #2551 from MathiasVP/argument-suppresion-c89-style
C++: Alert suppression through single-line /* */ style comments
2019-12-19 13:19:49 +01:00
Calum Grant
3c76346635 C#: WIP Adding exception handlers.
C#: Improve robustness by catching and logging exceptions when generating trap IDs.
2019-12-19 11:28:05 +00:00
Erik Krogh Kristensen
15d74b7d03 remove FP from js/regexpinjection where no regexp was constructed 2019-12-19 10:47:03 +01:00
Tom Hvitved
29cd6a9e30 Sync XML.qll 2019-12-19 10:29:30 +01:00
Tom Hvitved
1b6bd7a0fa C#: Update XML.qll for backwards compatibility 2019-12-19 10:27:59 +01:00
Tom Hvitved
82c368e13e C#: Sync XML.qll with other languages 2019-12-19 10:26:08 +01:00
James Fletcher
5a6a2e8a68 Merge pull request #2547 from shati-patel/ql/tutorial
QL tutorials: Update formatting and style
2019-12-19 09:06:08 +00:00