Commit Graph

18794 Commits

Author SHA1 Message Date
Mathias Vorreiter Pedersen
d2d8377e88 Update cpp/ql/src/semmle/code/cpp/ir/dataflow/internal/DataFlowUtil.qll
Co-authored-by: Jonas Jensen <jbj@github.com>
2020-12-22 16:34:53 +01:00
Mathias Vorreiter Pedersen
b95cf94824 Update cpp/ql/src/semmle/code/cpp/ir/dataflow/internal/DataFlowUtil.qll
Co-authored-by: Jonas Jensen <jbj@github.com>
2020-12-22 15:57:34 +01:00
Mathias Vorreiter Pedersen
ec35e0d518 C++: Respond to review comments. 2020-12-22 15:22:33 +01:00
Mathias Vorreiter Pedersen
77aa9615c0 C++: Accept test changes in paths. 2020-12-22 09:14:55 +01:00
Mathias Vorreiter Pedersen
fd8034cd8c C++: Add store step out of read side effects when we don't have a model for the callee. This brings back the lost result on boost. 2020-12-22 09:14:55 +01:00
Mathias Vorreiter Pedersen
a8b4fb6fd0 C++: Add qldoc (and remove predicate that wasn't needed). 2020-12-22 09:14:54 +01:00
Mathias Vorreiter Pedersen
72a80e3722 C++: Accept test changes. 2020-12-22 09:14:54 +01:00
Mathias Vorreiter Pedersen
145ab17f6b Fix a bad join-order caused by the optimizer not seeing that
`Node.getEnclosingCallable` is functional. This is fixed in the same way
as in Java's DataFlowUtil: We make create a non-virtual dispatching
`getEnclosingCallableImpl`, and implement `getEnclosingCallable` as a
wrapper that uses the `unique` aggregate to tell the compiler that there
is exactly 1 result.

Tuple numbers from an arbitrarily chosen iteration of
DataFlowImpl3::Stage4::fwdFlow0#fffff.

Before:
4569      ~5%        {5} r24 = SCAN DataFlowImpl3::Stage4::fwdFlow#fffff#prev_delta AS I OUTPUT I.<0>, I.<4> 'config', I.<1>, I.<2>, I.<3>
2876      ~1%        {5} r25 = JOIN r24 WITH DataFlowImpl3::LocalFlowBigStep::localFlowEntry#ff AS R ON FIRST 2 OUTPUT r24.<2> 'cc', r24.<0>, r24.<3> 'argAp', r24.<4>, r24.<1> 'config'
409404537 ~0%        {6} r26 = JOIN r25 WITH DataFlowImplCommon::getLocalCallContext#cpe#12#ff AS R ON FIRST 1 OUTPUT r25.<1>, R.<1>, r25.<0> 'cc', r25.<2> 'argAp', r25.<3>, r25.<4> 'config'
2876      ~0%        {6} r27 = JOIN r26 WITH DataFlowUtil::Node::getEnclosingCallable_dispred#ff AS R ON FIRST 2 OUTPUT r26.<0>, true, r26.<5> 'config', r26.<2> 'cc', r26.<3> 'argAp', r26.<4>
79821     ~1781%     {5} r28 = JOIN r27 WITH DataFlowImpl3::Stage4::localStep#ffbfff_0241#join_rhs AS R ON FIRST 3 OUTPUT r27.<3> 'cc', r27.<4> 'argAp', r27.<2> 'config', r27.<5> 'ap', R.<3> 'node'
2876      ~0%        {5} r29 = JOIN r26 WITH DataFlowUtil::Node::getEnclosingCallable_dispred#ff AS R ON FIRST 2 OUTPUT r26.<4>, r26.<0>, r26.<2> 'cc', r26.<3> 'argAp', r26.<5> 'config'
0         ~0%        {5} r30 = JOIN r29 WITH DataFlowImpl3::TNil#ff_1#join_rhs AS R ON FIRST 1 OUTPUT r29.<1>, false, r29.<4> 'config', r29.<2> 'cc', r29.<3> 'argAp'
0         ~0%        {5} r31 = JOIN r30 WITH DataFlowImpl3::Stage4::localStep#ffbfff_02413#join_rhs AS R ON FIRST 3 OUTPUT r30.<3> 'cc', r30.<4> 'argAp', r30.<2> 'config', R.<4> 'ap', R.<3> 'node'
4569      ~157%      {6} r32 = JOIN DataFlowImpl3::Stage4::fwdFlow0#fffff#join_rhs AS L WITH DataFlowImpl3::Stage4::fwdFlow#fffff#prev_delta AS R CARTESIAN PRODUCT OUTPUT R.<3>, L.<0> 'ap', L.<1> 'cc', L.<2> 'argAp', R.<0>, R.<4> 'config'
0         ~0%        {5} r33 = JOIN r32 WITH DataFlowImpl3::TNil#ff_1#join_rhs AS R ON FIRST 1 OUTPUT r32.<4>, r32.<5> 'config', r32.<1> 'ap', r32.<2> 'cc', r32.<3> 'argAp'
0         ~0%        {5} r34 = JOIN r33 WITH DataFlowImpl3::additionalJumpStep#fff_021#join_rhs AS R ON FIRST 2 OUTPUT R.<2> 'node', r33.<2> 'ap', r33.<3> 'cc', r33.<4> 'argAp', r33.<1> 'config'
0         ~0%        {5} r35 = JOIN r34 WITH DataFlowUtil::TIRDataFlowNode#f@staged_ext AS R ON FIRST 1 OUTPUT r34.<0> 'node', r34.<1> 'ap', r34.<2> 'cc', r34.<3> 'argAp', r34.<4> 'config'
0         ~0%        {6} r36 = JOIN r35 WITH project#DataFlowImpl3::Stage3::revFlow#fffff#12 AS R ON FIRST 1 OUTPUT r35.<1> 'ap', r35.<2> 'cc', r35.<3> 'argAp', r35.<4> 'config', r35.<0> 'node', R.<1>

After:
4569     ~0%        {5} r24 = SCAN DataFlowImpl3::Stage4::fwdFlow#fffff#prev_delta AS I OUTPUT I.<0>, I.<4> 'config', I.<1>, I.<2>, I.<3>
2876     ~0%        {5} r25 = JOIN r24 WITH DataFlowImpl3::LocalFlowBigStep::localFlowEntry#ff AS R ON FIRST 2 OUTPUT r24.<0>, r24.<2> 'cc', r24.<3> 'argAp', r24.<4>, r24.<1> 'config'
2876     ~0%        {6} r26 = JOIN r25 WITH DataFlowUtil::Node::getEnclosingCallable_dispred#ff AS R ON FIRST 1 OUTPUT r25.<1> 'cc', R.<1>, r25.<0>, r25.<2> 'argAp', r25.<3>, r25.<4> 'config'
2876     ~0%        {6} r27 = JOIN r26 WITH DataFlowImplCommon::getLocalCallContext#cpe#12#fb AS R ON FIRST 2 OUTPUT r26.<2>, true, r26.<5> 'config', r26.<0> 'cc', r26.<3> 'argAp', r26.<4>
79821    ~1862%     {5} r28 = JOIN r27 WITH DataFlowImpl3::Stage4::localStep#ffbfff_0241#join_rhs AS R ON FIRST 3 OUTPUT r27.<3> 'cc', r27.<4> 'argAp', r27.<2> 'config', r27.<5> 'ap', R.<3> 'node'
2876     ~0%        {5} r29 = JOIN r26 WITH DataFlowImplCommon::getLocalCallContext#cpe#12#fb AS R ON FIRST 2 OUTPUT r26.<4>, r26.<2>, r26.<0> 'cc', r26.<3> 'argAp', r26.<5> 'config'
0        ~0%        {5} r30 = JOIN r29 WITH DataFlowImpl3::TNil#ff_1#join_rhs AS R ON FIRST 1 OUTPUT r29.<1>, false, r29.<4> 'config', r29.<2> 'cc', r29.<3> 'argAp'
0        ~0%        {5} r31 = JOIN r30 WITH DataFlowImpl3::Stage4::localStep#ffbfff_02413#join_rhs AS R ON FIRST 3 OUTPUT r30.<3> 'cc', r30.<4> 'argAp', r30.<2> 'config', R.<4> 'ap', R.<3> 'node'
4569     ~148%      {6} r32 = JOIN DataFlowImpl3::Stage4::fwdFlow0#fffff#join_rhs AS L WITH DataFlowImpl3::Stage4::fwdFlow#fffff#prev_delta AS R CARTESIAN PRODUCT OUTPUT R.<3>, L.<0> 'ap', L.<1> 'cc', L.<2> 'argAp', R.<0>, R.<4> 'config'
0        ~0%        {5} r33 = JOIN r32 WITH DataFlowImpl3::TNil#ff_1#join_rhs AS R ON FIRST 1 OUTPUT r32.<4>, r32.<5> 'config', r32.<1> 'ap', r32.<2> 'cc', r32.<3> 'argAp'
0        ~0%        {5} r34 = JOIN r33 WITH DataFlowImpl3::additionalJumpStep#fff_021#join_rhs AS R ON FIRST 2 OUTPUT R.<2> 'node', r33.<2> 'ap', r33.<3> 'cc', r33.<4> 'argAp', r33.<1> 'config'
0        ~0%        {5} r35 = JOIN r34 WITH DataFlowUtil::TIRDataFlowNode#f@staged_ext AS R ON FIRST 1 OUTPUT r34.<0> 'node', r34.<1> 'ap', r34.<2> 'cc', r34.<3> 'argAp', r34.<4> 'config'
0        ~0%        {6} r36 = JOIN r35 WITH project#DataFlowImpl3::Stage3::revFlow#fffff#12 AS R ON FIRST 1 OUTPUT r35.<1> 'ap', r35.<2> 'cc', r35.<3> 'argAp', r35.<4> 'config', r35.<0> 'node', R.<1>
2020-12-22 09:14:54 +01:00
Mathias Vorreiter Pedersen
f5a2603cc1 C++: Add store steps that target the new partial definitions. 2020-12-22 09:14:54 +01:00
Mathias Vorreiter Pedersen
7a2b69feed C++: Add partial definition class backed by an IPA. 2020-12-22 09:14:54 +01:00
Mathias Vorreiter Pedersen
2930128421 C++: Implement read steps using ReadNodes. 2020-12-22 09:14:54 +01:00
Mathias Vorreiter Pedersen
91debe8669 C++: Add ReadNodes and implement local flow steps into them. 2020-12-22 09:14:54 +01:00
Mathias Vorreiter Pedersen
ba4da72b9e C++: Add examples that require longer access paths 2020-12-22 09:14:53 +01:00
Jonas Jensen
430194bb66 Merge pull request #4863 from MathiasVP/is-source-on-default-taint-tracking
C++: Overridable isSource on DefaultTaintTracking
2020-12-22 08:32:07 +01:00
Mathias Vorreiter Pedersen
4f07474b62 C++: Also allow custom sources in taintedWithoutGlobals 2020-12-21 19:55:47 +01:00
Mathias Vorreiter Pedersen
f4f96fe257 C++: Use isSource in queries. These were the only queries that restrict the source after dataflow terminates. 2020-12-21 16:35:35 +01:00
Mathias Vorreiter Pedersen
0e84c638b6 C++: Add isSource to AdjustedConfiguration 2020-12-21 16:34:22 +01:00
Tom Hvitved
0c78fb2933 Merge pull request #4855 from madneal/fix-for-csharp-docs
Fix for csharp docs
2020-12-21 14:11:36 +01:00
Shati Patel
0a0137bb5e Merge pull request #4859 from github/shati-patel-patch-1
Fix typo in docs title
2020-12-21 12:07:32 +00:00
Jonas Jensen
4308381057 Merge pull request #4846 from MathiasVP/default-taint-tracking-operand-instruction-interleaving
C++: Instruction -> Operand interleaving for DefaultTaintTracking
2020-12-21 12:44:06 +01:00
Shati Patel
66b85f1e5e Fix typo 2020-12-21 11:29:02 +00:00
Neal Caffery
ee0257836f removed, as it fixed by #4848 2020-12-21 19:05:37 +08:00
Mathias Vorreiter Pedersen
06366fa320 Merge pull request #4856 from jbj/gvn-wrapper-test
C++: Test the AST wrapper for IR GVN
2020-12-21 09:31:10 +01:00
Tom Hvitved
16aee6e71e Merge pull request #4842 from hvitved/csharp/format-method-no-insertion-param
C#: Recognize format methods without insertion parameters
2020-12-21 09:25:18 +01:00
Jonas Jensen
3236cbd83e C++: Test the AST wrapper for IR GVN
Out of our 3 GVN libraries, the one we actually use in production didn't
have tests -- except indirectly through `diff_ir_expr.ql`.
2020-12-21 08:21:02 +01:00
neal1991
b9d24b8255 fix for issue #4849 2020-12-21 08:54:15 +08:00
neal1991
eac83df40b fix for issue #4848 2020-12-21 08:52:42 +08:00
Rasmus Wriedt Larsen
49f902d28b Merge pull request #4757 from yoff/python-dataflow-synthetic-callables
Python: Enclosing callable for synthetic arguments
2020-12-18 16:06:26 +01:00
yoff
a08eb99778 Merge pull request #4779 from RasmusWL/django-class-based-handlers
Python: Add modeling of django class based view handlers
2020-12-18 15:58:51 +01:00
Anders Schack-Mulligen
5106d5df53 Merge pull request #4833 from luchua-bc/java-broken-crypto-algorithms
Java: Add missing broken crypto algorithms
2020-12-18 15:12:29 +01:00
Rasmus Wriedt Larsen
3e6296c7b8 Python: Fix grammar in QLDoc 2020-12-18 14:54:14 +01:00
Rasmus Wriedt Larsen
ed11e8f916 Python: Simplify predicate implementation
Co-authored-by: yoff <lerchedahl@gmail.com>
2020-12-18 14:52:20 +01:00
Mathias Vorreiter Pedersen
b5102043b1 Fix comments. 2020-12-18 14:19:02 +01:00
Chris Smowton
de4cdda839 Merge pull request #4841 from smowton/smowton/admin/mergeback-126-2020-12-16
Mergeback rc/1.26
2020-12-18 12:59:06 +00:00
Mathias Vorreiter Pedersen
f5e4725642 C++: Propagate flow from instruction's to non-exact operands for arrays and unions, and accept test changes. 2020-12-18 13:54:34 +01:00
Mathias Vorreiter Pedersen
2bf8e47932 Merge branch 'main' into default-taint-tracking-operand-instruction-interleaving 2020-12-18 11:59:10 +01:00
Mathias Vorreiter Pedersen
7ccd48e53c Make DefaultTaintTracking do operand->instruction->operand interleaving like DataFlowUtil. 2020-12-18 11:57:16 +01:00
Tom Hvitved
d53faa86dc C#: Restrict FormatInvalid.ql and UncontrolledFormatString.ql to calls with insertions 2020-12-18 10:53:11 +01:00
Rasmus Lerchedahl Petersen
0629d3e6e7 Python: Enclosing callable for synthetic arguments 2020-12-18 10:45:24 +01:00
Rasmus Lerchedahl Petersen
a16d58dfc0 Python: Add tests cases with synthetic arguments 2020-12-18 10:41:42 +01:00
Jonas Jensen
fd7dec7f20 Merge pull request #4824 from geoffw0/modelchanges5
C++: Add cases in the Allocation model.
2020-12-18 09:16:01 +01:00
yoff
39acc9a40b Merge pull request #4735 from RasmusWL/python-untrusted-flow
Python: Untrusted data used in external APIs
2020-12-18 00:15:08 +01:00
yoff
9dd6439e3c Merge pull request #4749 from RasmusWL/command-injection-tests
Python: Add some command injection tests
2020-12-17 23:36:06 +01:00
yoff
8a44405365 Merge pull request #4827 from RasmusWL/reword-qldoc-for-type-tracking-classes
Python: Reword QLDoc for class modeling with type-tracking
2020-12-17 23:28:19 +01:00
Tom Hvitved
7a132149a2 C#: Add change note 2020-12-17 15:39:01 +01:00
Tom Hvitved
fe559c190e C#: Recognize format methods without insertion parameters 2020-12-17 15:39:01 +01:00
Tamás Vajk
45893ab084 Merge pull request #4775 from tamasvajk/feature/cil-attribute-decoding2
C#: Improve CIL attribute decoding
2020-12-17 15:20:44 +01:00
Tamás Vajk
65c58edbed Merge pull request #4617 from tamasvajk/feature/csharp9-implicit-obj-creation
C#: Extract 'ImplicitObjectCreationExpressionSyntax'
2020-12-17 15:20:13 +01:00
Tamas Vajk
f12befdcd0 Add extra test for collection initialization 2020-12-17 13:49:02 +01:00
luchua-bc
bed8a68d28 Exclude broken algorithms from the list of secure algorithms 2020-12-17 00:41:23 +00:00