Commit Graph

19777 Commits

Author SHA1 Message Date
semmle-qlci
e195ac996e Merge pull request #868 from xiemaisi/js/discard-tokens-early
Approved by esben-semmle
2019-02-04 11:32:03 +00:00
Taus
32cc7c7cd9 Merge pull request #858 from markshannon/python-a-few-more-unknowns
Python: Slight improvement to reachability in points-to
2019-02-04 12:14:50 +01:00
Nick Rolfe
fa07364214 Vim syntax highlighting and file-type detection 2019-02-04 10:48:32 +00:00
Jonas Jensen
8368c37781 C++: Use shortestDistances HOP for IR BB indexes
This doesn't make it much faster, but it reduces the debug output
volume. It also simplifies the code.

I've found this change necessary when I compute the full IR on a
Wireshark snapshot in QL4E. Without it, Eclipse runs out of memory
because the console log is too large.
2019-02-04 11:40:11 +01:00
Tom Hvitved
910995af90 C#: Autoformat QL code 2019-02-04 10:32:30 +01:00
Max Schaefer
7be3336aaf JavaScript: Document per-function caches in CFG extractor. 2019-02-04 08:17:21 +00:00
Max Schaefer
6ad45b2451 JavaScript: Temporarily remove unstable test. 2019-02-04 08:10:39 +00:00
Jonas Jensen
60141bf317 C++: ignoreExprAndDescendants QL-796 workaround
The new predicate `isOrphan` gets inlined into
`ignoreExprAndDescendants`, whose performance improves from

    TranslatedElement::ignoreExprAndDescendants#f .. 23.4s (executed 9 times)

to

    TranslatedElement::ignoreExprAndDescendants#f ... 4.3s (executed 9 times)

This dramatic improvement is not only due to eliminating a type check in
the recursive case. Removing the type check from the other base cases
also enabled them to get better join orders.
2019-02-03 16:55:12 +01:00
Jonas Jensen
66e7c26d4e C++: Replace FastTC with iteration in ignoreExpr
Before, `ignoreExprAndDescendants` and its related predicates had this
timing on Wireshark.

    #TranslatedElement::getRealParent#ffPlus#swapped ......... 25.7s
    TranslatedElement::ignoreExprAndDescendants#f ............ 16.9s
    TranslatedElement::getRealParent#ff ...................... 7.2s
    TranslatedElement::ignoreExpr#f .......................... 4.8s
    TranslatedElement::ignoreExpr#f#antijoin_rhs ............. 3.2s
    TranslatedElement::getRealParent#ff_10#higher_order_body . 2.2s

After, it looks like this

    TranslatedElement::ignoreExprAndDescendants#f ............ 23.4s (executed 9 times)
    TranslatedElement::getRealParent#ff ...................... 6.3s
    TranslatedElement::ignoreExpr#f#antijoin_rhs ............. 4.8s
    TranslatedElement::ignoreExpr#f .......................... 3.7s
    TranslatedElement::getRealParent#ff_10#join_rhs .......... 2.5s
    project#TranslatedElement::getRealParent#ff .............. 1.3s
2019-02-03 16:55:12 +01:00
Patrik Schönfeldt
ac249cdbbe Fix reccomendation for LargeParameter (C++)
The previous reccomentation changed the behaviour of the code.
A user following the advice might have broken her/his code:
With call-by-value, the original parameter is not changed.
With a call-by-reference, however, it may be changed. To be sure,
nothing breaks by blindly following the advice, suggest to pass a
const reference.
2019-02-03 15:44:13 +01:00
Jonas Jensen
f8318ef96f C++: Move TIRVariable to its own file
The `SSAConstruction.getNewIRVariable` was very slow on Wireshark. This
was probably because it couldn't join on multiple columns
simultaneously. Instead of improving the join, I observed that the
`TIRVariable` type was the same between all three IR stages except for a
few occurrences of `FunctionIR` that could easily be changed to
`Function`. By sharing `TIRVariable` between all the stages, we avoid
recomputing it and translating it between every stage, turning the slow
`getNewIRVariable` predicate into a no-op.

This change means that later stages of the IR can't introduce new
variables, but that was already the case because
`config/identical-files.json` forced all three `IRVariable.qll` files to
be identical.
2019-02-03 13:36:30 +01:00
Jonas Jensen
3afefce8ef C++: Improve order of parameters in SSA def/use
This changes the order so the parameter that's sometimes projected away
is the last one, making the projection cheap.
2019-02-03 13:34:02 +01:00
Jonas Jensen
4ac22253eb C++: Speed up variableLiveOnEntryToBlock in IR
This predicate computed a local CP between all defs and uses of the same
virtual variable in a basic block. This wasn't a problem in
`unaliased_ssa`, but it became a huge problem in `aliased_ssa`, probably
because many variables can be modelled with a single virtual variable
there.

Before this commit, evaluation of `aliased_ssa`'s
`variableLiveOnEntryToBlock#ff#antijoin_rhs` on Wireshark took 80
_minutes_. After this commit, that predicate and its immediate
dependencies take around 5 _seconds_.
2019-02-03 13:25:18 +01:00
jf205
6243c722c6 Merge pull request #861 from jbj/revert-dbscheme-docs
C++: Revert doc-related changes to dbscheme
2019-02-01 10:59:18 +00:00
Max Schaefer
9bfde9553d Merge pull request #839 from asger-semmle/field-propwrite
JS: add PropWrites cases for instance fields initialization
2019-02-01 10:56:25 +00:00
semmle-qlci
222738072d Merge pull request #840 from esben-semmle/js/propagate-sound-avalue
Approved by xiemaisi
2019-02-01 09:23:43 +00:00
Jonas Jensen
e81d197ebd C++: Revert doc-related changes to dbscheme
These changes to the dbscheme were made in 7cc1442ecb and a98aae0a24
without a corresponding upgrade script in the internal repo.
2019-02-01 10:01:29 +01:00
Jonas Jensen
ee4526687d Merge pull request #859 from rdmarsh2/rdmarsh/cpp/ir-performance-1
C++: use field overrides in TranslatedElement and subclasses
2019-02-01 08:43:20 +01:00
Robert Marsh
e1d289ffb4 Merge pull request #759 from calumgrant/cs/interface-tostring
C#: Remove FPs from cs/call-to-object-tostring
2019-01-31 12:03:49 -08:00
Robert Marsh
f3b4cb4640 Merge pull request #848 from calumgrant/cs/lgtm-suppress-alerts
C#: Suppress cs/catch-of-all-exceptions
2019-01-31 12:03:26 -08:00
calum
5144f89a5b C#: Fix dataflow. 2019-01-31 19:53:46 +00:00
Robert Marsh
5327ca7f77 Merge pull request #812 from jbj/ir-backedge
C++: IR back-edge detection based on TranslatedStmt
2019-01-31 11:28:21 -08:00
Dave Bartolomeo
bbe8e7ebfc C++: fix typo, ThrowExpr -> ReThrowExpr
Co-Authored-By: rdmarsh2 <rdmarsh2@gmail.com>
2019-01-31 10:47:17 -08:00
Dave Bartolomeo
ab1f96fb2c Merge pull request #770 from jbj/cfg-static-init-pr
C++: Add addresses to `Expr.isConstant`
2019-01-31 10:24:48 -08:00
Dave Bartolomeo
b0b2fc80c1 Merge pull request #855 from jbj/ir-getRealParent
C++: Simplify TranslatedElement.getRealParent
2019-01-31 10:15:30 -08:00
Dave Bartolomeo
8896d3bf88 Merge pull request #856 from jbj/ir-getInstructionOperandDefinition
C++: Speed up `getInstructionOperandDefinition`
2019-01-31 10:11:59 -08:00
calum
92eb857dac C#: Add documentation. 2019-01-31 17:46:28 +00:00
calum
4274854f0f C#: Refactor TCallableFlowSinkDelegateArg 2019-01-31 16:03:23 +00:00
Robert Marsh
ffb46638b0 C++: use more field overrides in IR generation 2019-01-31 07:47:21 -08:00
Robert Marsh
fa56981bce C++: use field overrides in TranslatedExpr 2019-01-31 07:47:21 -08:00
calum
713c7a8c20 C#: Refactor TCallableFlowSourceDelegateArg 2019-01-31 15:42:09 +00:00
Mark Shannon
24d678b869 Python: Points-to; let values with no boolean constant value flow through pi-nodes. 2019-01-31 15:15:49 +00:00
Mark Shannon
f190f83406 Python: Give most non-self instance attributes the 'unknown' value; improves reachability analysis. 2019-01-31 15:05:13 +00:00
Jonas Jensen
be2a480394 Merge pull request #843 from geoffw0/strtoul
CPP: Improve ArithmeticTainted.ql
2019-01-31 07:04:17 -08:00
calum
18ff4583ef C#: Refactor TCallableFlowSourceArg to remove callable. 2019-01-31 12:05:26 +00:00
Jonas Jensen
b55573ebe3 C++: Accept test changes in ir_gvn.expected 2019-01-31 10:08:16 +01:00
Max Schaefer
b87abc9602 JavaScript: Extend suspiciousCredentials predicate to recognise authKey and similar. 2019-01-31 09:03:23 +00:00
Max Schaefer
87e62f0bd5 JavaScript: Teach PostMessageStar to reason about partially tainted objects. 2019-01-31 08:59:47 +00:00
Max Schaefer
aeb8cc62b2 JavaScript: Reclassify PostMessageStar as CWE-201. 2019-01-31 08:08:52 +00:00
Jonas Jensen
35d7fb5322 C++: Fix TranslatedElement.getInstruction perf
This relation was almost 40x the size it needed to be on Wireshark
because it lacked a restriction on the `tag` parameter. To implement
that restriction efficiently, I had to split the relation in two to
dictate the join order.

With the fix, `getInstruction` now computes the same as
`getInstructionTranslatedElementAndTag`, so the latter could be
simplified.

I made a corresponding change to `TranslatedElement.getTempVariable` for
the sake of consistency.
2019-01-31 08:45:02 +01:00
Jonas Jensen
947634f66f C++: Speed up getInstructionOperandDefinition
A part of `SSAConstruction.getInstructionOperandDefinition` was more
expensive than it had to be. On a ChakraCore snapshot, this changes the
tuple counts from

    3020569 ~2%       {3} r40 = JOIN OperandTag::TUnmodeledUseOperand#f WITH Instruction::Instruction::getFunction_dispred#ff CARTESIAN PRODUCT OUTPUT FIELDS {Instruction::Instruction::getFunction_dispred#ff.<0>,OperandTag::TUnmodeledUseOperand#f.<0>,Instruction::Instruction::getFunction_dispred#ff.<1>}
    62405   ~0%       {3} r41 = JOIN r40 WITH Instruction::UnmodeledUseInstruction#class#fffffff ON r40.<0>=Instruction::UnmodeledUseInstruction#class#fffffff.<0> OUTPUT FIELDS {r40.<2>,r40.<1>,r40.<0>}
    2868421 ~1%       {3} r42 = JOIN r41 WITH Instruction::Instruction::getFunction_dispred#ff_10#join_rhs ON r41.<0>=Instruction::Instruction::getFunction_dispred#ff_10#join_rhs.<0> OUTPUT FIELDS {Instruction::Instruction::getFunction_dispred#ff_10#join_rhs.<1>,r41.<1>,r41.<2>}
    62405   ~0%       {3} r43 = JOIN r42 WITH Instruction::UnmodeledDefinitionInstruction#class#fffffff ON r42.<0>=Instruction::UnmodeledDefinitionInstruction#class#fffffff.<0> OUTPUT FIELDS {r42.<2>,r42.<1>,r42.<0>}

to

    (0s) Starting to evaluate predicate SSAConstruction::Cached::getUnmodeledUseInstruction#ff
    (0s) Tuple counts:
    62405   ~0%     {2} r1 = JOIN Instruction::UnmodeledUseInstruction#class#fffffff WITH Instruction::Instruction::getFunction_dispred#ff ON Instruction::UnmodeledUseInstruction#class#fffffff.<0>=Instruction::Instruction::getFunction_dispred#ff.<0> OUTPUT FIELDS {Instruction::Instruction::getFunction_dispred#ff.<1>,Instruction::Instruction::getFunction_dispred#ff.<0>}
                                      return r1
    ...
    75716   ~0%       {3} r40 = JOIN OperandTag::TUnmodeledUseOperand#f WITH FunctionIR::FunctionIR::getUnmodeledDefinitionInstruction#ff CARTESIAN PRODUCT OUTPUT FIELDS {FunctionIR::FunctionIR::getUnmodeledDefinitionInstruction#ff.<0>,OperandTag::TUnmodeledUseOperand#f.<0>,FunctionIR::FunctionIR::getUnmodeledDefinitionInstruction#ff.<1>}
    62405   ~0%       {3} r41 = JOIN r40 WITH FunctionIR::FunctionIR::getUnmodeledUseInstruction#ff ON r40.<0>=FunctionIR::FunctionIR::getUnmodeledUseInstruction#ff.<0> OUTPUT FIELDS {FunctionIR::FunctionIR::getUnmodeledUseInstruction#ff.<1>,r40.<1>,r40.<2>}
2019-01-31 08:43:00 +01:00
Jonas Jensen
5b685383c8 C++: Simplify TranslatedElement.getRealParent
Now that we have `Expr.getParentWithConversions`, we can implement
`TranslatedElement.getRealParent` simpler. This implementation also
avoids recursion.
2019-01-31 08:41:29 +01:00
Geoffrey White
642faa6cf0 CPP: Change note. 2019-01-30 18:38:19 +00:00
Geoffrey White
07adf6f201 CPP: Handle array accesses. 2019-01-30 18:36:32 +00:00
Geoffrey White
4685f193f5 CPP: Widen varMaybeStackAllocated. 2019-01-30 18:36:32 +00:00
Geoffrey White
c87036f2fd CPP: Simplify. 2019-01-30 18:36:32 +00:00
Geoffrey White
276738a435 CPP: Auto-format the query. 2019-01-30 18:36:32 +00:00
Geoffrey White
25e11512db CPP: Add a test of ReturnStackAllocatedMemory. 2019-01-30 18:36:31 +00:00
Asger F
720f442ea5 JS: Rename to StaticClassMemberAsPropWrite 2019-01-30 15:49:21 +00:00
semmle-qlci
fc5b9dd55e Merge pull request #837 from asger-semmle/hardcoded-empty-string
Approved by esben-semmle
2019-01-30 13:40:39 +00:00