Commit Graph

1425 Commits

Author SHA1 Message Date
Geoffrey White
803da339a1 CPP: Fix a mistake in Inet.qll. 2020-01-17 17:44:42 +00:00
Robert Marsh
9b361f1701 Merge pull request #2601 from dbartol/dbartol/OpcodeProperties
C++: Consolidate opcode properties onto `Opcode` class
2020-01-08 11:05:41 -08:00
Dave Bartolomeo
690d23d15e C++: Fix formatting 2020-01-07 13:23:36 -07:00
Dave Bartolomeo
9df37399f8 C++: Consolidate opcode properties onto Opcode class
Previously, we had several predicates on `Instruction` and `Operand` whose values were determined solely by the opcode of the instruction. For large snapshots, this meant that we would populate large tables mapping each of the millions of `Instruction`s to the appropriate value, times three (once for each IR flavor).

This change moves all of these opcode properties onto `Opcode` itself, with inline wrapper predicates on `Instruction` and `Operand` where necessary. On smaller snapshots, like ChakraCore, performance is a wash, but this did speed up Wireshark by about 4%.

Even ignoring the modest performance benefit, having these properties defined on `Opcode` seems like a better organization than having them on `Instruction` and `Operand`.
2020-01-07 13:17:27 -07:00
Jonas Jensen
4830e43b3e C++: Fix overlappingVariableMemoryLocations perf
The `overlappingVariableMemoryLocations` predicate was a helper
predicate introduced to fix a join-order issue in
`overlappingIRVariableMemoryLocations`. Unfortunately it caused a
performance issue of its own because it could grow too large. On the
small project (38MB zip) awslabs/s2n there were 181M rows in
`overlappingVariableMemoryLocations`, and it took 134s to evaluate.

The fix is to collapse the two predicates into one and fix join ordering
by including an extra column in the predicates being joined.

In addition, some parameters were reordered to avoid the overhead of
auto-generated `join_rhs` predicates.

Tuple counts of `overlappingVariableMemoryLocations` before:

    623285    ~176%     {2} r1 = JOIN AliasedSSA::isCoveredOffset#fff_120#join_rhs AS L WITH AliasedSSA::isCoveredOffset#fff_120#join_rhs AS R ON FIRST 2 OUTPUT L.<2>, R.<2>
    119138    ~3%       {2} r2 = SCAN AliasedSSA::VariableMemoryLocation::getVirtualVariable_dispred#ff AS I OUTPUT I.<1>, I.<0>
    172192346 ~0%       {2} r3 = JOIN r2 WITH AliasedSSA::hasUnknownOffset#ff_10#join_rhs AS R ON FIRST 1 OUTPUT R.<1>, r2.<1>
    172815631 ~0%       {2} r4 = r1 \/ r3
    172192346 ~0%       {2} r5 = JOIN r2 WITH AliasedSSA::hasUnknownOffset#ff_10#join_rhs AS R ON FIRST 1 OUTPUT r2.<1>, R.<1>
    345007977 ~87%      {2} r6 = r4 \/ r5
                        return r6

Tuple counts of `overlappingIRVariableMemoryLocations` after:

    117021 ~134%     {2} r1 = JOIN AliasedSSA::isCoveredOffset#ffff AS L WITH AliasedSSA::isCoveredOffset#ffff AS R ON FIRST 3 OUTPUT L.<3>, R.<3>
    201486 ~1%       {2} r2 = JOIN AliasedSSA::hasUnknownOffset#fff AS L WITH AliasedSSA::hasVariableAndVirtualVariable#fff AS R ON FIRST 2 OUTPUT L.<2>, R.<2>
    318507 ~26%      {2} r3 = r1 \/ r2
    201486 ~3%       {2} r4 = JOIN AliasedSSA::hasUnknownOffset#fff AS L WITH AliasedSSA::hasVariableAndVirtualVariable#fff AS R ON FIRST 2 OUTPUT R.<2>, L.<2>
    519993 ~92%      {2} r5 = r3 \/ r4
                     return r5
2019-12-27 16:06:24 +01:00
Jonas Jensen
7e84453ec9 Merge pull request #2542 from geoffw0/datetime
C++: Sort through the leap year and japanese era queries
2019-12-23 10:13:12 +01:00
Dave Bartolomeo
5b5d2f2b67 Merge pull request #2154 from rdmarsh2/rdmarsh/cpp/ir-callee-side-effects
C++: add InitializeIndirection for pointer params
2019-12-20 13:13:54 -07:00
yo-h
cc7f98e0f6 Merge pull request #2555 from hvitved/csharp/xml-sync
C#: Sync `XML.qll` with other languages
2019-12-20 09:03:55 -05:00
Jonas Jensen
de55a6846f Merge pull request #2204 from alexet/cache-to-string
Cache the computation of core toString predicates for cpp c# and java.
2019-12-20 14:54:46 +01:00
Jonas Jensen
939979ddef Merge branch 'master' into overflowcalc 2019-12-19 14:12:00 +01:00
Jonas Jensen
a13748f484 Merge pull request #2259 from rdmarsh2/rdmarsh/cpp/default-taint-tracking-sources
C++: move sources into DefaultTaintTracking.qll
2019-12-19 14:09:41 +01:00
Tom Hvitved
29cd6a9e30 Sync XML.qll 2019-12-19 10:29:30 +01:00
Robert Marsh
33067c8e31 Merge pull request #2519 from jbj/ir-backedge-notc
C++: Get rid of a fastTC and noopt in IR
2019-12-18 14:20:43 -08:00
Jonas Jensen
e7283afa3e Merge pull request #2531 from dbartol/dbartol/MissingToString
C++: Fix `toString()` predicates that don't hold
2019-12-18 19:09:48 +01:00
Robert Marsh
e209ed961a Merge branch 'master' into rdmarsh/cpp/ir-callee-side-effects 2019-12-17 15:11:02 -08:00
Robert Marsh
93ace5be35 C++: remove Chi node flow in DefaultTaintTracking 2019-12-17 14:23:11 -08:00
Dave Bartolomeo
240823019a Merge remote-tracking branch 'upstream/master' into dbartol/MissingToString 2019-12-17 11:50:36 -07:00
Robert Marsh
7468facb34 C++: autoformat more 2019-12-17 09:06:52 -08:00
Geoffrey White
eaf00c6683 C++: Fix override related warning. 2019-12-17 15:39:29 +00:00
Geoffrey White
9986206dc6 C++: Placement new does not necessarily require a delete. 2019-12-17 15:28:21 +00:00
Geoffrey White
a8c31c6590 C++: Add AllocationFunction/AllocationExpr.requiresDealloc(). 2019-12-17 11:07:08 +00:00
Geoffrey White
b2e2db1e5c CPP: Autoformat. 2019-12-17 10:39:52 +00:00
Max Schaefer
a17b615ae5 C++/Python: Deprecate XMLFile.getPath and XMLFile.getFolder.
Both can be expressed using predicates inherited from `File`.
2019-12-17 10:15:43 +00:00
Max Schaefer
47c1fc7358 C++/Python: Fix XMLFile.getPath and XMLFile.getFolder.
Previously, the former returned the file's stem (that is, basename without extension), and the latter never held.
2019-12-17 10:15:43 +00:00
Max Schaefer
bccdf59631 C++: Remove an unecessary unresolveElement.
As far as I can see, `@file` is not a `@usertype`, so this shouldn't be needed.
2019-12-17 10:15:43 +00:00
Max Schaefer
923e36ba4f C++/Java/JavaScript/Python: Make qldoc consistent. 2019-12-17 10:15:43 +00:00
Max Schaefer
a2fe678464 C++/Java/JavaScript/Python: Unify imports in XML.qll. 2019-12-17 10:15:43 +00:00
Geoffrey White
6d516e0192 C++: Make some implementations in DatTime.qll more robust (and cope with 'const'). 2019-12-17 10:14:08 +00:00
Jonas Jensen
9193a81e1a Merge pull request #2534 from geoffw0/xml
C++: Fix characteristic predicate of `XMLParent`.
2019-12-17 10:57:40 +01:00
Robert Marsh
fdaa06ce92 C++:autoformat 2019-12-16 12:54:31 -08:00
Robert Marsh
3c9d7953b6 Merge branch 'master' into rdmarsh/cpp/default-taint-tracking-sources 2019-12-16 11:49:22 -08:00
Dave Bartolomeo
60c40adf31 C++: Fix formatting 2019-12-16 10:00:54 -07:00
Geoffrey White
25dc2ad273 C++: Support '__builtin_alloca'. 2019-12-16 14:19:33 +00:00
Geoffrey White
acca39bfc7 C++: Repair following merge. 2019-12-16 14:12:32 +00:00
Geoffrey White
0da826f0c3 Merge branch 'master' into overflowcalc 2019-12-16 13:48:38 +00:00
Jonas Jensen
8b1892d900 Merge pull request #2533 from m00nbsd/master
Add NetBSD/Solaris allocation functions.
2019-12-16 14:42:12 +01:00
Geoffrey White
f990634481 C++: Fix the XMLParent Constructor. 2019-12-16 11:07:16 +00:00
Jonas Jensen
5cea452123 Merge pull request #2491 from rdmarsh2/rdmarsh/cpp/ir-taintedIncludingGlobalVars
C++: handle global vars in DefaultTaintTracking
2019-12-16 11:00:34 +01:00
m00nbsd
f7484171c9 Add NetBSD/Solaris allocation functions.
* kmem_{z}alloc is used by Solaris and NetBSD
 * pool_{cache_}get is used by NetBSD
2019-12-16 10:32:18 +01:00
Dave Bartolomeo
914288d3c5 C++: Fix toString() predicates that don't hold 2019-12-13 17:06:05 -07:00
Robert Marsh
57d891e3b4 C++: autoformat 2019-12-12 11:03:18 -08:00
james
f6029bd55c Merge branch 'rc/1.23' into mergeback-123-ql 2019-12-12 15:05:28 +00:00
Geoffrey White
73446ea610 Merge pull request #2511 from jbj/isInCycle-raw-only
C++: Compute isInCycle only for raw IR
2019-12-12 14:22:00 +00:00
Jonas Jensen
fd48dcf63e C++: Get rid of a fastTC and noopt in IR
The `getAChild*` fastTC was causing OOM on a `make allyesconfig` Linux
database with 8GB RAM, and I've observed it to be slow on other
databases too.
2019-12-12 09:31:39 +01:00
Jonas Jensen
66876d0f63 C++: Compute isInCycle only for raw IR
On wireshark/wireshark, `isInCycle` ran into a low-memory loop on the
`aliased_ssa` stage. It shouldn't be necessary to detect cycles after
the `raw` stage, so this commit moves cycle detection into the
`Construction` modules and makes it a no-op in `SSAConstruction.qll`.
2019-12-10 16:03:39 +01:00
Jonas Jensen
7c151644f5 C++: Fix getTempVariable join order in IR
This join order seems to have broken so it took forever on
wireshark/wireshark.
2019-12-10 13:43:36 +01:00
Robert Marsh
18e7eff43c C++: autoformat 2019-12-09 13:47:38 -08:00
Robert Marsh
b9f8c39fe2 C++: respond to PR comments 2019-12-09 10:55:56 -08:00
Robert Marsh
28fb3d606a C++: add fread and printf alias models 2019-12-05 12:14:33 -08:00
Robert Marsh
52a74718da C++: Move sources into DefaultTaintTracking 2019-12-05 12:14:25 -08:00