Commit Graph

19777 Commits

Author SHA1 Message Date
Jonas Jensen
8acbb3bfb9 C++: Further simplify a bit
This changes tuple counts!?
2020-01-08 11:36:50 -08:00
Jonas Jensen
5072201b7e C++: Fix join order 2020-01-08 11:36:40 -08:00
Jonas Jensen
838720bef0 C++: de-inline getDefinitionOrChiInstruction
Still has bad join order
2020-01-08 11:36:34 -08:00
Jonas Jensen
3d2cc7bbce C++: make hasPhiOperandDefinition feasible 2020-01-08 11:36:14 -08:00
Jonas Jensen
55f157e06d C++: Fix overlappingVariableMemoryLocations perf
The `overlappingVariableMemoryLocations` predicate was a helper
predicate introduced to fix a join-order issue in
`overlappingIRVariableMemoryLocations`. Unfortunately it caused a
performance issue of its own because it could grow too large. On the
small project (38MB zip) awslabs/s2n there were 181M rows in
`overlappingVariableMemoryLocations`, and it took 134s to evaluate.

The fix is to collapse the two predicates into one and fix join ordering
by including an extra column in the predicates being joined.

In addition, some parameters were reordered to avoid the overhead of
auto-generated `join_rhs` predicates.

Tuple counts of `overlappingVariableMemoryLocations` before:

    623285    ~176%     {2} r1 = JOIN AliasedSSA::isCoveredOffset#fff_120#join_rhs AS L WITH AliasedSSA::isCoveredOffset#fff_120#join_rhs AS R ON FIRST 2 OUTPUT L.<2>, R.<2>
    119138    ~3%       {2} r2 = SCAN AliasedSSA::VariableMemoryLocation::getVirtualVariable_dispred#ff AS I OUTPUT I.<1>, I.<0>
    172192346 ~0%       {2} r3 = JOIN r2 WITH AliasedSSA::hasUnknownOffset#ff_10#join_rhs AS R ON FIRST 1 OUTPUT R.<1>, r2.<1>
    172815631 ~0%       {2} r4 = r1 \/ r3
    172192346 ~0%       {2} r5 = JOIN r2 WITH AliasedSSA::hasUnknownOffset#ff_10#join_rhs AS R ON FIRST 1 OUTPUT r2.<1>, R.<1>
    345007977 ~87%      {2} r6 = r4 \/ r5
                        return r6

Tuple counts of `overlappingIRVariableMemoryLocations` after:

    117021 ~134%     {2} r1 = JOIN AliasedSSA::isCoveredOffset#ffff AS L WITH AliasedSSA::isCoveredOffset#ffff AS R ON FIRST 3 OUTPUT L.<3>, R.<3>
    201486 ~1%       {2} r2 = JOIN AliasedSSA::hasUnknownOffset#fff AS L WITH AliasedSSA::hasVariableAndVirtualVariable#fff AS R ON FIRST 2 OUTPUT L.<2>, R.<2>
    318507 ~26%      {2} r3 = r1 \/ r2
    201486 ~3%       {2} r4 = JOIN AliasedSSA::hasUnknownOffset#fff AS L WITH AliasedSSA::hasVariableAndVirtualVariable#fff AS R ON FIRST 2 OUTPUT R.<2>, L.<2>
    519993 ~92%      {2} r5 = r3 \/ r4
                     return r5
2020-01-08 11:07:20 -08:00
Robert Marsh
9b361f1701 Merge pull request #2601 from dbartol/dbartol/OpcodeProperties
C++: Consolidate opcode properties onto `Opcode` class
2020-01-08 11:05:41 -08:00
Geoffrey White
cf5dd85944 Merge pull request #2577 from MathiasVP/multiplication-overflow-not-possible-due-to-type-width
Multiplication overflow not possible due to type width
2020-01-08 17:18:33 +00:00
shati-patel
ad0ad3a3e4 Merge pull request #2612 from jf205/recent-changes
CodeQL docs: port recent fixes to rc/1.23
2020-01-08 16:36:27 +00:00
james
2407eb103a docs: fix list
(cherry picked from commit 618a3f91d8)
2020-01-08 16:16:39 +00:00
Rasmus Wriedt Larsen
cdcca630f3 docs: remove extra comma in dataflow articles
(cherry picked from commit e882060839)
2020-01-08 16:16:39 +00:00
Rasmus Wriedt Larsen
24e551905e docs: Fix Python taint tracking links
at some point we moved security/TaintTracking.qll to dataflow/TaintTracking.qll

(cherry picked from commit f44ce7d647)
2020-01-08 16:16:39 +00:00
james
97d3d1fca3 docs: fix ast node link
(cherry picked from commit cff5df0779)
2020-01-08 16:16:39 +00:00
shati-patel
3cfc7d2e54 Merge pull request #2611 from jf205/mergeback-123
Merge rc/1.23 into master
2020-01-08 16:12:47 +00:00
Dave Bartolomeo
6c8de44800 Merge pull request #2604 from geoffw0/returnthis
CPP: Exclude template classes from cpp/assignment-does-not-return-this
2020-01-08 09:12:22 -07:00
james
490e13060c Merge branch 'rc/1.23' into mergeback-123 2020-01-08 16:00:19 +00:00
shati-patel
e103527d32 Merge pull request #2610 from jf205/python-link
docs: fix link in Python topic
2020-01-08 15:48:37 +00:00
james
cff5df0779 docs: fix ast node link 2020-01-08 15:30:04 +00:00
Max Schaefer
308da0774d Merge pull request #2525 from asger-semmle/promise-missing-await
JS: New query: missing await
2020-01-08 15:29:45 +00:00
Max Schaefer
de15ecf47b Merge pull request #2593 from asger-semmle/regexp-always-matches
JS: Add RegExpAlwaysMatches query
2020-01-08 15:21:39 +00:00
James Fletcher
8e700081f1 Merge pull request #2609 from RasmusWL/doc-fix-python-taint-links
docs: Fix Python taint tracking links
2020-01-08 15:21:07 +00:00
Rasmus Wriedt Larsen
f44ce7d647 docs: Fix Python taint tracking links
at some point we moved security/TaintTracking.qll to dataflow/TaintTracking.qll
2020-01-08 16:10:27 +01:00
yo-h
1078424f79 Java: allow single-line /* ... */ comments for alert suppression 2020-01-08 09:19:25 -05:00
shati-patel
9b4f6af007 Merge pull request #2605 from RasmusWL/small-doc-fix
docs: remove extra comma in dataflow articles
2020-01-08 14:01:41 +00:00
Rasmus Wriedt Larsen
e882060839 docs: remove extra comma in dataflow articles 2020-01-08 14:53:31 +01:00
Geoffrey White
b6e1f35ff6 CPP: Generalize the fix to all template code. 2020-01-08 13:36:59 +00:00
Geoffrey White
8044fefb1f CPP: Change note. 2020-01-08 13:19:11 +00:00
Geoffrey White
527d29ba23 CPP: Exclude template classes from the query. 2020-01-08 13:16:38 +00:00
Geoffrey White
d527dbe47a CPP: Add test case. 2020-01-08 13:13:06 +00:00
Asger Feldthaus
775e63d9c0 JS: Fix qhelp validation error 2020-01-08 10:38:10 +00:00
Asger F
ef79023e52 Update javascript/ql/src/Expressions/MissingAwait.qhelp
Co-Authored-By: Max Schaefer <54907921+max-schaefer@users.noreply.github.com>
2020-01-08 10:23:30 +00:00
Tom Hvitved
85f6e5fe22 Merge pull request #2450 from calumgrant/cs/expr-nullability
C#: Expression nullability
2020-01-08 09:50:03 +01:00
Mathias Vorreiter Pedersen
100ace532f C++: Fixed handling of false negative. Query now supports global variables 2020-01-07 22:57:21 +01:00
Mathias Vorreiter Pedersen
db08076fed C++: Fixed false negative 2020-01-07 22:20:04 +01:00
Mathias Vorreiter Pedersen
229da0a9c0 C++: Add testcase demonstrating false negative 2020-01-07 22:12:34 +01:00
Dave Bartolomeo
690d23d15e C++: Fix formatting 2020-01-07 13:23:36 -07:00
Dave Bartolomeo
9df37399f8 C++: Consolidate opcode properties onto Opcode class
Previously, we had several predicates on `Instruction` and `Operand` whose values were determined solely by the opcode of the instruction. For large snapshots, this meant that we would populate large tables mapping each of the millions of `Instruction`s to the appropriate value, times three (once for each IR flavor).

This change moves all of these opcode properties onto `Opcode` itself, with inline wrapper predicates on `Instruction` and `Operand` where necessary. On smaller snapshots, like ChakraCore, performance is a wash, but this did speed up Wireshark by about 4%.

Even ignoring the modest performance benefit, having these properties defined on `Opcode` seems like a better organization than having them on `Instruction` and `Operand`.
2020-01-07 13:17:27 -07:00
Calum Grant
bc1b2c3ead C#: Address review comment 2020-01-07 18:39:52 +00:00
Calum Grant
d0d7ed620c C#: Update comments in test file to reflect fixed test output. 2020-01-07 18:39:52 +00:00
Calum Grant
359dea2c2b C#: Fixed test output. 2020-01-07 18:39:52 +00:00
Calum Grant
bcd8dca780 C#: When creating conditionally accessed expressions, use the typeinfo from the conditional expression to ensure correct flow state and type nullability. 2020-01-07 18:39:51 +00:00
Calum Grant
85c9459b35 C#: Add more tests showing incorrect extraction. 2020-01-07 18:39:51 +00:00
Calum Grant
10181e93e2 C#: Update QLtest output 2020-01-07 18:39:51 +00:00
Calum Grant
a1bedf2e06 C#: Use expression flow state to determine non-null guards 2020-01-07 18:39:51 +00:00
Calum Grant
2eb29224e8 C#: Update guards tests. 2020-01-07 18:39:51 +00:00
Calum Grant
ecb22f1379 C#: Fix typo in dbscheme. 2020-01-07 18:39:51 +00:00
Calum Grant
f67240a316 C#: Analysis change notes 2020-01-07 18:39:51 +00:00
Calum Grant
1036640e24 C#: Update db stats 2020-01-07 18:38:58 +00:00
Calum Grant
4e710e9685 C#: DB upgrade script 2020-01-07 18:38:58 +00:00
Calum Grant
8db46bc8ec C#: More tests for nullable flow state. 2020-01-07 18:38:58 +00:00
Calum Grant
6c9ebaba0b C#: Populate expression type nullability and nullable flow state. 2020-01-07 18:38:58 +00:00