Commit Graph

242 Commits

Author SHA1 Message Date
Jeroen Ketema
2b37ebd7ed Merge remote-tracking branch 'upstream/main' into mathiasvp/replace-ast-with-ir-use-usedataflow 2022-11-11 17:24:34 +01:00
Anders Schack-Mulligen
b3b7711149 Dataflow: Sync. 2022-11-09 14:23:15 +01:00
Rasmus Wriedt Larsen
4895daba85 DataFlow: Add read/store stepIsLocal consistency checks 2022-11-08 13:32:49 +01:00
Jeroen Ketema
5732c3bca0 Merge branch 'main' into mathiasvp/replace-ast-with-ir-use-usedataflow 2022-11-07 15:03:26 +01:00
Anders Schack-Mulligen
a1dba82360 Dataflow: Sync. 2022-11-04 12:41:55 +01:00
Tom Hvitved
d3488da0c2 Data flow: Sync files 2022-11-03 15:52:30 +01:00
Jeroen Ketema
80ef3b39ff Merge branch 'main' into mathiasvp/replace-ast-with-ir-use-usedataflow 2022-10-31 18:26:34 +01:00
Mathias Vorreiter Pedersen
b85d3bc829 Merge branch 'main' into replace-ast-with-ir-use-usedataflow 2022-10-25 12:51:30 +02:00
Robert Marsh
88708d015c C++: additional comments for modulus analysis 2022-10-21 12:50:41 -04:00
Josh Soref
86ad9f5c92 spelling: whose
Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>
2022-10-14 15:08:44 -04:00
Josh Soref
c5acca4e24 spelling: sequence
Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>
2022-10-14 15:08:44 -04:00
Josh Soref
83cf8a85ce spelling: indirect
Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>
2022-10-14 15:08:43 -04:00
Josh Soref
56684ca937 spelling: configuration
Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>
2022-10-14 15:08:43 -04:00
Josh Soref
2a3e2d35e6 spelling: certain
Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>
2022-10-14 15:08:43 -04:00
Mathias Vorreiter Pedersen
d62ae96763 C++: Fix ql-for-ql alerts. 2022-10-14 10:14:53 +02:00
Anders Schack-Mulligen
036724ce8d Dataflow: Sync. 2022-10-13 11:03:30 +02:00
Nick Rolfe
cfb9277cd7 C++: use explicit this 2022-10-12 16:11:45 +01:00
Mathias Vorreiter Pedersen
f88aaf37a5 C++: Add 'UninitializedNode' to IR dataflow. 2022-10-11 16:08:06 +01:00
Mathias Vorreiter Pedersen
7ac9c1e832 Merge pull request #10713 from MathiasVP/fix-types-in-ir-dataflow
C++: Fix `getType` for experimental IR dataflow
2022-10-11 15:20:49 +01:00
Mathias Vorreiter Pedersen
95e798565b C++: Expand on the comment about missing types in the database. Also rename 'getType0' to 'getTypeImpl' to avoid confusion. 2022-10-11 12:57:51 +01:00
Mathias Vorreiter Pedersen
5cfc3fe8df C++: Use 'DataFlowType' instead of 'Type' for the 'getType' predicate in 'PostUpdateNode'. 2022-10-11 11:00:25 +01:00
Tom Hvitved
ffb2b1c15e Data flow: Sync files 2022-10-10 15:39:13 +02:00
Tom Hvitved
85344bfb13 Data flow: Improved fastTC bound in PathNodeImpl::getANonHiddenSuccessor
Before
```
[2022-10-10 14:34:54] Evaluated non-recursive predicate __DataFlowImplForRegExp#43df744e::PathNodeImpl::getASuccessorImpl#0#dispred#ff_DataFlowImplForRegExp__#higher_order_body@4bb14aoj in 262ms (size: 2418048).
Evaluated relational algebra for predicate __DataFlowImplForRegExp#43df744e::PathNodeImpl::getASuccessorImpl#0#dispred#ff_DataFlowImplForRegExp__#higher_order_body@4bb14aoj with tuple counts:
        4141389  ~75%    {1} r1 = SCAN _DataFlowImplForRegExp#43df744e::PathNodeImpl::getASuccessorImpl#0#dispred#ff_DataFlowImplForRegExp#__#shared OUTPUT In.1
                         return r1

[2022-10-10 14:34:57] Evaluated non-recursive predicate boundedFastTC:DataFlowImplForRegExp#43df744e::PathNodeImpl::getASuccessorIfHidden#0#dispred#ff:__DataFlowImplForRegExp#43df744e::PathNodeImpl::getASuccessorImpl#0#dispred#ff_DataFlowImplForRegExp__#higher_order_body@fb66bb06 in 2754ms (size: 7448123).
[2022-10-10 14:35:09] Evaluated non-recursive predicate DataFlowImplForRegExp#43df744e::PathNodeImpl::getANonHiddenSuccessor#0#dispred#ff@77ff066b in 10892ms (size: 2830055).
Evaluated relational algebra for predicate DataFlowImplForRegExp#43df744e::PathNodeImpl::getANonHiddenSuccessor#0#dispred#ff@77ff066b with tuple counts:
          4141389   ~0%    {3} r1 = SCAN _DataFlowImplForRegExp#43df744e::PathNodeImpl::getASuccessorImpl#0#dispred#ff_DataFlowImplForRegExp#__#shared OUTPUT In.0, In.1, In.1
          2192551   ~4%    {3} r2 = r1 AND NOT DataFlowImplForRegExp#43df744e::PathNodeImpl::isHidden#0#dispred#f(Lhs.2)
          2192551   ~4%    {2} r3 = SCAN r2 OUTPUT In.0, In.2

          4141389   ~0%    {2} r4 = SCAN _DataFlowImplForRegExp#43df744e::PathNodeImpl::getASuccessorImpl#0#dispred#ff_DataFlowImplForRegExp#__#shared OUTPUT In.1, In.0
        147138810   ~0%    {3} r5 = JOIN r4 WITH boundedFastTC:DataFlowImplForRegExp#43df744e::PathNodeImpl::getASuccessorIfHidden#0#dispred#ff:__DataFlowImplForRegExp#43df744e::PathNodeImpl::getASuccessorImpl#0#dispred#ff_DataFlowImplForRegExp__#higher_order_body ON FIRST 1 OUTPUT Lhs.1, Lhs.0, Rhs.1
           637649   ~3%    {3} r6 = r5 AND NOT DataFlowImplForRegExp#43df744e::PathNodeImpl::isHidden#0#dispred#f(Lhs.2)
           637649   ~2%    {2} r7 = SCAN r6 OUTPUT In.0, In.2

          2830200   ~0%    {2} r8 = r3 UNION r7
                           return r8
```

After
```
[2022-10-10 14:59:08] Evaluated non-recursive predicate boundedFastTC:DataFlowImplForRegExp#43df744e::PathNodeImpl::getASuccessorIfHidden#0#dispred#ff_10#higher_order_body:_DataFlowImplForRegExp#43df744e::PathNodeImpl::getASuccessorIfHidden#0#dispred#ff_DataFlowImplForReg__#higher_order_body@98a323ne in 384ms (size: 671076).
[2022-10-10 14:59:09] Evaluated non-recursive predicate DataFlowImplForRegExp#43df744e::PathNodeImpl::getANonHiddenSuccessor0#ff@69f158pf in 222ms (size: 2805795).
Evaluated relational algebra for predicate DataFlowImplForRegExp#43df744e::PathNodeImpl::getANonHiddenSuccessor0#ff@69f158pf with tuple counts:
        2155019   ~0%    {1} r1 = DataFlowImplForRegExp#43df744e::PathNodeImpl#class#f AND NOT DataFlowImplForRegExp#43df744e::PathNodeImpl::isHidden#0#dispred#f(Lhs.0)
        2155019   ~0%    {2} r2 = SCAN r1 OUTPUT In.0, In.0

         650776   ~0%    {2} r3 = boundedFastTC:DataFlowImplForRegExp#43df744e::PathNodeImpl::getASuccessorIfHidden#0#dispred#ff_10#higher_order_body:_DataFlowImplForRegExp#43df744e::PathNodeImpl::getASuccessorIfHidden#0#dispred#ff_DataFlowImplForReg__#higher_order_body AND NOT DataFlowImplForRegExp#43df744e::PathNodeImpl::isHidden#0#dispred#f(Lhs.0)
         650776   ~0%    {2} r4 = SCAN r3 OUTPUT In.1, In.0

        2805795   ~0%    {2} r5 = r2 UNION r4
                         return r5

[2022-10-10 14:59:09] Evaluated non-recursive predicate DataFlowImplForRegExp#43df744e::PathNodeImpl::getANonHiddenSuccessor#0#dispred#ff@5ae9fc5n in 445ms (size: 2830062).
Evaluated relational algebra for predicate DataFlowImplForRegExp#43df744e::PathNodeImpl::getANonHiddenSuccessor#0#dispred#ff@5ae9fc5n with tuple counts:
        4141389  ~5%    {2} r1 = DataFlowImplForRegExp#43df744e::PathNodeImpl::getASuccessorImpl#0#dispred#ff AND NOT DataFlowImplForRegExp#43df744e::PathNodeImpl::isHidden#0#dispred#f(Lhs.0)
        4141389  ~0%    {2} r2 = SCAN r1 OUTPUT In.1, In.0
        2830200  ~0%    {2} r3 = JOIN r2 WITH DataFlowImplForRegExp#43df744e::PathNodeImpl::getANonHiddenSuccessor0#ff ON FIRST 1 OUTPUT Lhs.1, Rhs.1
                        return r3
```
2022-10-10 15:36:58 +02:00
Tom Hvitved
296ec94a2a Data flow: Sync files 2022-10-09 19:48:45 +02:00
Mathias Vorreiter Pedersen
e147a6032e C++: Replace 'IRType' with 'Type' in dataflow. This means we're more compatible with the old AST library. 2022-10-06 17:26:56 +01:00
Mathias Vorreiter Pedersen
65a538ed41 C++: Exclude a few more operands from the dataflow graph. These aren't ever used for dataflow, and it should give us a slight speedup. 2022-10-06 17:22:09 +01:00
Mathias Vorreiter Pedersen
3fcb825e7f C++: Change a few indirectionIndex ranges from '[0 .. n - 1]' to '[1 .. n]'. This simplifies some arithmetic in a few predicates. 2022-10-06 17:21:09 +01:00
Mathias Vorreiter Pedersen
2a514d60d4 C++: Add 'isBarrierIn' to prevent path duplication. 2022-09-29 19:55:58 +01:00
Mathias Vorreiter Pedersen
769ff5c6f3 C++: Add 'isAdditionalFlowStep' predicates for both configurations in the product dataflow library and use them to fix missing results in the 'cpp/overrun-write' query. 2022-09-28 15:17:04 +01:00
Robert Marsh
82bbe67267 Merge pull request #10593 from MathiasVP/fix-fp-on-cwe-193
C++: Fix FPs on `cpp/invalid-pointer-deref`
2022-09-27 17:38:17 -04:00
Tom Hvitved
df2b586e7c Merge pull request #10577 from hvitved/dataflow/get-a-read-content-fan-in
Data flow: Fix bad join-order when getAReadContent has large fan-in
2022-09-27 20:04:58 +02:00
Mathias Vorreiter Pedersen
549eca1b17 C++: Fix 'implicit use of this'. 2022-09-27 16:29:30 +01:00
Mathias Vorreiter Pedersen
e4305948ef C++: Fix FP on CWE-193 by blocking flow through back-edges of phi nodes. 2022-09-27 16:28:03 +01:00
Tom Hvitved
335e1a8233 Address review comments 2022-09-27 13:36:52 +02:00
Tom Hvitved
45fc62f16b Data flow: Sync files 2022-09-26 20:39:48 +02:00
Tom Hvitved
1273db5a22 Data flow: Fix bad join-order when getAReadContent has large fan-in
Before (terminated before completion)
```
Evaluated relational algebra for predicate DataFlowImplForHttpClientLibraries#c536b619::store#5#fffff@e5ef07bh with tuple counts:
            151500     ~0%    {4} r1 = SCAN DataFlowImplCommon#4f8df883::Cached::store#4#ffff OUTPUT In.1, In.0, In.2, In.3
            150500     ~0%    {5} r2 = JOIN r1 WITH DataFlowImplCommon#4f8df883::Cached::MkTypedContent#fff_20#join_rhs ON FIRST 1 OUTPUT Lhs.1, Lhs.0, Lhs.2, Lhs.3, Rhs.1
            149500     ~0%    {5} r3 = JOIN r2 WITH num#DataFlowImplForHttpClientLibraries#c536b619::TNodeNormal#ff ON FIRST 1 OUTPUT Lhs.2, Lhs.1, Lhs.3, Lhs.4, Rhs.1
            148500     ~0%    {5} r4 = JOIN r3 WITH num#DataFlowImplForHttpClientLibraries#c536b619::TNodeNormal#ff ON FIRST 1 OUTPUT Lhs.3, Lhs.1, Lhs.2, Lhs.4, Rhs.1
        2003849000     ~0%    {5} r5 = JOIN r4 WITH DataFlowPublic#e1781e31::ContentSet::getAReadContent#0#dispred#ff_10#join_rhs ON FIRST 1 OUTPUT Rhs.1, Lhs.1, Lhs.2, Lhs.3, Lhs.4
         105066500  ~9036%    {5} r6 = JOIN r5 WITH project#DataFlowImplForHttpClientLibraries#c536b619::readSet#4#ffff ON FIRST 1 OUTPUT Lhs.3, Lhs.1, Lhs.4, Lhs.2, Rhs.1
                              return r6
```

After
```
Evaluated relational algebra for predicate DataFlowImplForHttpClientLibraries#c536b619::readProj#2#ff@302620cn with tuple counts:
        1461867  ~0%    {2} r1 = SCAN DataFlowPrivate#462ff392::Cached::TContent#f OUTPUT In.0, In.0
        3549054  ~1%    {2} r2 = JOIN r1 WITH DataFlowPublic#e1781e31::ContentSet::getAReadContent#0#dispred#ff_10#join_rhs ON FIRST 1 OUTPUT Rhs.1, Lhs.1
        5772824  ~5%    {2} r3 = JOIN r2 WITH project#DataFlowImplForHttpClientLibraries#c536b619::readSet#4#ffff ON FIRST 1 OUTPUT Lhs.1, Rhs.1
                        return r3

Evaluated relational algebra for predicate DataFlowImplForHttpClientLibraries#c536b619::store#5#fffff@016cd9o1 with tuple counts:
         267905  ~0%    {4} r1 = SCAN DataFlowImplCommon#4f8df883::Cached::store#4#ffff OUTPUT In.1, In.0, In.2, In.3
         267905  ~0%    {5} r2 = JOIN r1 WITH DataFlowImplCommon#4f8df883::Cached::MkTypedContent#fff_20#join_rhs ON FIRST 1 OUTPUT Lhs.1, Lhs.0, Lhs.2, Lhs.3, Rhs.1
         267905  ~0%    {5} r3 = JOIN r2 WITH num#DataFlowImplForHttpClientLibraries#c536b619::TNodeNormal#ff ON FIRST 1 OUTPUT Lhs.2, Lhs.1, Lhs.3, Lhs.4, Rhs.1
         267905  ~0%    {5} r4 = JOIN r3 WITH num#DataFlowImplForHttpClientLibraries#c536b619::TNodeNormal#ff ON FIRST 1 OUTPUT Lhs.3, Lhs.1, Lhs.2, Lhs.4, Rhs.1
        2109240  ~0%    {5} r5 = JOIN r4 WITH DataFlowImplForHttpClientLibraries#c536b619::readProj#2#ff ON FIRST 1 OUTPUT Lhs.3, Lhs.1, Lhs.4, Lhs.2, Rhs.1
                        return r5
```
2022-09-26 20:37:53 +02:00
Anders Schack-Mulligen
1687d08587 Dataflow: Sync. 2022-09-26 16:10:03 +02:00
Mathias Vorreiter Pedersen
73f279d6e7 Merge pull request #10555 from MathiasVP/testcase-for-php-cve
C++: Fix missing bounds in range analysis
2022-09-23 16:55:51 +01:00
Mathias Vorreiter Pedersen
f3212fe01c C++: Autoformat. 2022-09-23 13:00:22 +01:00
Mathias Vorreiter Pedersen
ac03242cfc C++: Add an SSAVariable for pointer-arithmetic expressions in guards. 2022-09-23 12:21:31 +01:00
Tom Hvitved
ad6b870f94 Data flow: Sync files 2022-09-22 15:01:33 +02:00
Tom Hvitved
db8b6ac69a Data flow: Sync files 2022-09-21 11:02:24 +02:00
Mathias Vorreiter Pedersen
74ccec75c8 C++: Sync identical files. 2022-09-20 13:37:54 +01:00
Mathias Vorreiter Pedersen
79654d978d C++: Sync identical files. 2022-09-20 12:57:21 +01:00
Mathias Vorreiter Pedersen
dc00643ad1 C++: More QLDoc. 2022-09-16 17:14:29 +01:00
Mathias Vorreiter Pedersen
b8a5aa5d85 C++: Fix a couple of range analysis issues:
1. The new query is expecting pointer arithmetic operations to generate
range-analysis bounds, but this wasn't true on main.
2. The bounds generated by `boundFlowCond` was incorrectly inferred as
non-strict when comparing a pointers (unlike when comparing values of
integral types). This gave FPs in the new query.

This also fixes a couple of missing results in existing queries that
use the new range-analysis library.
2022-09-15 17:46:52 +01:00
Mathias Vorreiter Pedersen
d981f898e4 C++: Add flow states to the product dataflow library. 2022-09-15 15:54:09 +01:00
Mathias Vorreiter Pedersen
6dcfe0348b C++: Copy over the required changes to non-experimental libraries. 2022-09-09 17:26:58 +01:00
Mathias Vorreiter Pedersen
6d313ace2d C++: Copy the new use-use flow code to experimental. 2022-09-09 14:20:10 +01:00
Robert Marsh
0feeafd0ac Merge pull request #10339 from MathiasVP/dont-use-get-unique-id-in-range-analysis
C++: Don't use `getUniqueId` in range analysis
2022-09-08 11:13:43 -04:00