Commit Graph

9682 Commits

Author SHA1 Message Date
Mathias Vorreiter Pedersen
08419b77af C++: Respond to PR reviews. 2023-03-15 14:07:04 +00:00
Mathias Vorreiter Pedersen
913ff201f1 Merge branch 'mathiasvp/replace-ast-with-ir-use-usedataflow' into local-flow-for-getAdditionalFlowIntoCallNodeTerm 2023-03-15 11:15:16 +00:00
Mathias Vorreiter Pedersen
11932a54e8 Merge branch 'main' into mathiasvp/replace-ast-with-ir-use-usedataflow 2023-03-15 11:13:49 +00:00
Mathias Vorreiter Pedersen
9bd3347a3c C++: Remove import. 2023-03-15 10:33:47 +00:00
Mathias Vorreiter Pedersen
623f6ff701 C++: Move things around so that 'getAdditionalFlowIntoCallNodeTerm' is in the same stage as 'DataFlowImplCommon'. 2023-03-15 10:01:47 +00:00
Mathias Vorreiter Pedersen
deb43c21e6 C++: Use local flow instead of GVN to find parameters that are used in switch statements. 2023-03-15 10:00:47 +00:00
Robert Marsh
623e39031c C++: fix ModulusAnalysis test 2023-03-14 16:46:09 -04:00
Robert Marsh
267c341965 C++: exclude ZeroBounds in relative stage 2023-03-14 10:41:50 -04:00
Tom Hvitved
bdd56f1b6e Data flow: Sync files 2023-03-14 10:01:56 +01:00
Robert Marsh
b4b7507fe4 C++: autoformat 2023-03-13 15:45:48 -04:00
Mathias Vorreiter Pedersen
136769d647 Merge pull request #12507 from MathiasVP/fix-as-expr-performance-2
C++: Map some indirect nodes to expressions in `localExprFlowStep`
2023-03-13 16:03:48 +00:00
Mathias Vorreiter Pedersen
a5051655a1 C++: Autoformat. 2023-03-13 15:41:17 +01:00
Mathias Vorreiter Pedersen
00450d10bb Merge branch 'main' into mathiasvp/replace-ast-with-ir-use-usedataflow 2023-03-13 14:50:17 +01:00
Mathias Vorreiter Pedersen
58c1518fbd C++: Fix QLDoc. 2023-03-13 14:45:00 +01:00
Mathias Vorreiter Pedersen
97462a3fa0 C++: Include more expressions in 'asExpr' in local expression flow. 2023-03-13 14:29:11 +01:00
Robert Marsh
64f23ebb4d Merge pull request #12436 from MathiasVP/ir-range-analysis-for-unary-minus
C++: IR-based range analysis for unary minus
2023-03-13 09:02:38 -04:00
Erik Krogh Kristensen
060c37b6a2 Merge pull request #12345 from erik-krogh/delOldDeps
delete old deprecations
2023-03-13 12:48:24 +01:00
Anders Schack-Mulligen
f53a05bf13 Merge pull request #12475 from aschackmull/dataflow/mergepathgraph
Dataflow: Add MergePathGraph module.
2023-03-13 11:26:24 +01:00
Anders Schack-Mulligen
c380ecbbbc Data flow: Add change notes. 2023-03-13 11:09:13 +01:00
erik-krogh
6c1ebd999e Merge branch 'main' into delOldDeps 2023-03-13 11:00:29 +01:00
Jeroen Ketema
bd47c4f9ec C++: Update .expected after frontend changes 2023-03-13 09:40:10 +01:00
Robert Marsh
d4e3f7f738 C++: fix missing bounds in new range analysis 2023-03-10 14:23:08 -05:00
Robert Marsh
e931d59bea Merge pull request #12479 from jketema/simple-range-analysis-wrapper
C++: Introduce wrapper for the semantic range analysis mimicking the simple one
2023-03-10 10:01:28 -05:00
Anders Schack-Mulligen
1e64748ffe Dataflow: Autoformat. 2023-03-10 15:12:19 +01:00
Anders Schack-Mulligen
289f921171 Dataflow: Sync. 2023-03-10 14:56:54 +01:00
Anders Schack-Mulligen
00f0879ff5 Dataflow: Sync. 2023-03-10 14:56:54 +01:00
Jeroen Ketema
faecf44870 C++: Add predicate implementations we get for free from simple range analysis 2023-03-10 14:31:15 +01:00
Jeroen Ketema
0e45c9982b C++: Introduce wrapper for the semantic range analysis mimicking the simple one 2023-03-10 13:30:24 +01:00
Anders Schack-Mulligen
159d8e978c Dataflow: one more autoformat post rebase 2023-03-10 10:04:35 +01:00
Anders Schack-Mulligen
ef97e539ec C/C++: Autoformat 2023-03-10 09:39:41 +01:00
Mathias Vorreiter Pedersen
d25a312557 Merge branch 'main' into mathiasvp/replace-ast-with-ir-use-usedataflow 2023-03-09 18:38:00 +00:00
Mathias Vorreiter Pedersen
d89b8ba446 Merge pull request #12469 from MathiasVP/speedup-CleartextSqliteDatabase
C++: Restrict sinks in `cpp/cleartext-storage-database`
2023-03-09 18:33:51 +00:00
Mathias Vorreiter Pedersen
84a61d1e02 C++: No need for 'matches'. 2023-03-09 15:36:26 +00:00
Mathias Vorreiter Pedersen
59402eb754 Merge pull request #12462 from MathiasVP/disable-std-order-in-fwd-flow-stage-1
DataFlow: Disable standard order in `Stage1::fwdFlow`
2023-03-09 15:30:05 +00:00
Mathias Vorreiter Pedersen
2931e5dea8 C++: Reduce duplication by blocking flow into sources (since we'll already be considering flow starting at those sources) and out of sinks (since we'll already be alerting on this sink if it's relevant). 2023-03-09 14:59:13 +00:00
Mathias Vorreiter Pedersen
03ba7ea851 C++: Move the weird global property 'not sqlite_encryption_used()' from the sink definition to the source definition. The dataflow library starts tracking flow from the sources, so it's better to to rule out the entire database in the source definition than in the sink definition. 2023-03-09 14:59:13 +00:00
Mathias Vorreiter Pedersen
7819a7d2bc C++: Severely restrict the set of sinks in 'cpp/cleartext-storage-database'. This reduces the number of sinks considered on the 'sysown/proxysql' from > 62000 sinks to ~1000 sinks. 2023-03-09 14:59:13 +00:00
Asger F
6e744093e2 Merge pull request #12398 from github/post-release-prep/codeql-cli-2.12.4
Post-release preparation for codeql-cli-2.12.4
2023-03-09 15:38:21 +01:00
Jeroen Ketema
de97ae38dc C++: C++: Use getAUse in getIRRepresentationOfIndirectOperand 2023-03-09 15:15:00 +01:00
Mathias Vorreiter Pedersen
1f77f77153 DataFlow: Sync identical files. 2023-03-09 10:41:15 +00:00
Mathias Vorreiter Pedersen
c7b41ca470 C++: Disable standard order for 'fwdFlow' in stage 1 of dataflow. 2023-03-09 10:41:06 +00:00
Jeroen Ketema
e5ce27fbf9 C++: Fix a number of instances where a dataflow node did not have a location 2023-03-09 10:32:30 +01:00
Jeroen Ketema
31fa230c0d C++: Remove toStringImpl predicate that is overridden in every subclass 2023-03-09 10:29:17 +01:00
Jeroen Ketema
4ee13a3234 C++: Remove getLocationImpl predicate that is always overridden
Also remove the predicate referred to in ints implementation, as it is no
longer used.
2023-03-09 10:27:15 +01:00
Jeroen Ketema
791f5913d2 C++: Fix multiple getLocation consistency error in use-use dataflow 2023-03-09 10:25:02 +01:00
Mathias Vorreiter Pedersen
f19f7967c2 C++: Fix join order.
Before (I stopped midway):

```
(72s) Tuple counts for _#Class#bacd9b46::Class::getADerivedClass#0#dispredPlus#ff_#Declaration#4bfb53be::AccessHolder::getE__#antijoin_rhs#1/3@fb0627h8 after 1m4s:
  ...

  20000     ~0%       {5} r28 = r26 UNION r27
  224367484 ~7%       {9} r29 = JOIN r28 WITH Class#bacd9b46::Class::accessOfBaseMember#2#dispred#ffff_1023#join_rhs ON FIRST 1 OUTPUT Rhs.3, "protected", Lhs.1 'arg0', Lhs.2 'arg1', Lhs.3 'arg2', Lhs.0, Lhs.4, Rhs.1, Rhs.2

  111914129 ~0%       {7} r30 = JOIN r29 WITH specifiers ON FIRST 2 OUTPUT Lhs.6, Lhs.2 'arg0', Lhs.3 'arg1', Lhs.4 'arg2', Lhs.5, Lhs.7, Lhs.8

  123503367 ~0%       {8} r31 = JOIN r30 WITH Declaration#4bfb53be::DirectAccessHolder::isFriendOfOrEqualTo#1#dispred#ff ON FIRST 1 OUTPUT Lhs.3 'arg2', Rhs.1, Lhs.1 'arg0', Lhs.2 'arg1', Lhs.4, Lhs.0, Lhs.5, Lhs.6
  331748250 ~0%       {10} r32 = JOIN r31 WITH Class#bacd9b46::Class::accessOfBaseMember#2#dispred#ffff ON FIRST 2 OUTPUT Lhs.2 'arg0', Lhs.3 'arg1', Lhs.0 'arg2', Lhs.4, Lhs.5, Lhs.6, Lhs.7, Lhs.1, Rhs.2, Rhs.3
  331748250 ~0%       {10} r33 = SELECT r32 ON In.8 = In.9
  331748250 ~2%       {9} r34 = SCAN r33 OUTPUT In.7, In.5, In.8, In.0 'arg0', In.1 'arg1', In.2 'arg2', In.3, In.4, In.6
  38000     ~4%       {10} r35 = JOIN r34 WITH Class#bacd9b46::Class::accessOfBaseMember#2#dispred#ffff ON FIRST 3 OUTPUT Rhs.3, Lhs.3 'arg0', Lhs.4 'arg1', Lhs.5 'arg2', Lhs.6, Lhs.7, Lhs.1, Lhs.8, Lhs.0, Lhs.2
  37500     ~0%       {11} r36 = JOIN r35 WITH specifiers ON FIRST 1 OUTPUT Lhs.1 'arg0', Lhs.2 'arg1', Lhs.3 'arg2', Lhs.4, Lhs.5, Lhs.6, Lhs.7, Lhs.8, Lhs.9, Lhs.0, Rhs.1
  28973     ~0%       {11} r37 = SELECT r36 ON In.10 >= "protected"
  28973     ~98%      {6} r38 = SCAN r37 OUTPUT In.8, "public", In.0 'arg0', In.1 'arg1', In.2 'arg2', In.6

  111913629 ~6%       {7} r39 = JOIN r29 WITH specifiers ON FIRST 2 OUTPUT Lhs.6, Lhs.4 'arg2', Lhs.2 'arg0', Lhs.3 'arg1', Lhs.5, Lhs.7, Lhs.8
  110582830 ~0%       {8} r40 = JOIN r39 WITH Declaration#4bfb53be::DirectAccessHolder::isFriendOfOrEqualTo#1#dispred#ff ON FIRST 2 OUTPUT Lhs.1 'arg2', Lhs.5, Lhs.6, Lhs.2 'arg0', Lhs.3 'arg1', Lhs.1 'arg2', Lhs.4, Lhs.0

  123503367 ~0%       {8} r41 = JOIN r30 WITH Declaration#4bfb53be::DirectAccessHolder::isFriendOfOrEqualTo#1#dispred#ff ON FIRST 1 OUTPUT Rhs.1, Lhs.3 'arg2', Lhs.1 'arg0', Lhs.2 'arg1', Lhs.4, Lhs.0, Lhs.5, Lhs.6
  0         ~0%       {8} r42 = JOIN r41 WITH #Class#bacd9b46::Class::getADerivedClass#0#dispredPlus#ff ON FIRST 2 OUTPUT Lhs.0, Lhs.6, Lhs.7, Lhs.2 'arg0', Lhs.3 'arg1', Lhs.1 'arg2', Lhs.4, Lhs.5

  110582830 ~0%       {8} r43 = r40 UNION r42
  15000     ~6%       {8} r44 = JOIN r43 WITH Class#bacd9b46::Class::accessOfBaseMember#2#dispred#ffff ON FIRST 3 OUTPUT Lhs.5 'arg2', Lhs.1, Lhs.3 'arg0', Lhs.4 'arg1', Lhs.6, Lhs.7, Lhs.2, Lhs.0
  ...
```

After:

```
Tuple counts for _#Class#bacd9b46::Class::getADerivedClass#0#dispredPlus#ff_#Declaration#4bfb53be::AccessHolder::getE__#antijoin_rhs#1/3@997a3ai9 after 744ms:
  ...

  78600   ~8%       {6} r29 = r26 UNION r28
  437816  ~0%       {9} r30 = JOIN r29 WITH Class#bacd9b46::Class::accessOfBaseMember#2#dispred#ffff ON FIRST 1 OUTPUT Lhs.1 'arg0', Lhs.2 'arg1', Lhs.0 'arg2', Lhs.3, Lhs.4, Lhs.5, Rhs.1, Rhs.2, Rhs.3
  430928  ~0%       {9} r31 = SELECT r30 ON In.7 = In.8
  430928  ~0%       {7} r32 = SCAN r31 OUTPUT In.5, In.6, In.0 'arg0', In.1 'arg1', In.2 'arg2', In.3, In.7
  1096333 ~0%       {7} r33 = JOIN r32 WITH Class#bacd9b46::Class::accessOfBaseMember#2#dispred#ffff ON FIRST 2 OUTPUT Lhs.1, Lhs.5, Rhs.2, Lhs.2 'arg0', Lhs.3 'arg1', Lhs.4 'arg2', Lhs.6
  777970  ~0%       {8} r34 = JOIN r33 WITH Class#bacd9b46::Class::accessOfBaseMember#2#dispred#ffff ON FIRST 3 OUTPUT Lhs.0, Lhs.1, Lhs.2, Rhs.3, Lhs.3 'arg0', Lhs.4 'arg1', Lhs.5 'arg2', Lhs.6

  334217  ~0%       {6} r35 = JOIN r14 WITH Declaration#4bfb53be::DirectAccessHolder::isFriendOfOrEqualTo#1#dispred#ff ON FIRST 1 OUTPUT Lhs.3 'arg2', Rhs.1, Lhs.1 'arg0', Lhs.2 'arg1', Lhs.4, Lhs.0
  235623  ~0%       {8} r36 = JOIN r35 WITH Class#bacd9b46::Class::accessOfBaseMember#2#dispred#ffff ON FIRST 2 OUTPUT Lhs.2 'arg0', Lhs.3 'arg1', Lhs.0 'arg2', Lhs.4, Lhs.5, Lhs.1, Rhs.2, Rhs.3
  235623  ~0%       {8} r37 = SELECT r36 ON In.6 = In.7
  235623  ~0%       {7} r38 = SCAN r37 OUTPUT In.5, In.6, In.0 'arg0', In.1 'arg1', In.2 'arg2', In.3, In.4
  437303  ~0%       {9} r39 = JOIN r38 WITH Class#bacd9b46::Class::accessOfBaseMember#2#dispred#ffff_0213#join_rhs ON FIRST 2 OUTPUT Rhs.3, Lhs.2 'arg0', Lhs.3 'arg1', Lhs.4 'arg2', Lhs.5, Lhs.6, Lhs.0, Lhs.1, Rhs.2
  437303  ~4%       {10} r40 = JOIN r39 WITH specifiers ON FIRST 1 OUTPUT Lhs.1 'arg0', Lhs.2 'arg1', Lhs.3 'arg2', Lhs.4, Lhs.5, Lhs.6, Lhs.7, Lhs.8, Lhs.0, Rhs.1
  352102  ~1%       {10} r41 = SELECT r40 ON In.9 >= "protected"
  352102  ~0%       {6} r42 = SCAN r41 OUTPUT In.7, In.3, In.0 'arg0', In.1 'arg1', In.2 'arg2', In.6
  775332  ~0%       {8} r43 = JOIN r42 WITH Class#bacd9b46::Class::accessOfBaseMember#2#dispred#ffff ON FIRST 2 OUTPUT Lhs.0, Lhs.1, Rhs.2, Rhs.3, Lhs.2 'arg0', Lhs.3 'arg1', Lhs.4 'arg2', Lhs.5

  1553302 ~51%      {8} r44 = r34 UNION r43
  1553302 ~152%     {7} r45 = JOIN r44 WITH Class#bacd9b46::Class::accessOfBaseMember#2#dispred#ffff ON FIRST 4 OUTPUT Lhs.7, "public", Lhs.4 'arg0', Lhs.5 'arg1', Lhs.6 'arg2', Lhs.2, Lhs.3
  ...
```
2023-03-09 09:23:56 +00:00
Jeroen Ketema
55da3257bf C++: Fix multiple toString consistency error in use-use dataflow 2023-03-09 10:08:25 +01:00
Jeroen Ketema
a2248cb5a0 C++: Use getAUse in getIRRepresentationOfOperand 2023-03-09 09:45:54 +01:00
Mathias Vorreiter Pedersen
540ce1f0db Contrary to what the QLDoc says, this predicate was way too large to be
evaluated on the 'quick-lint/quick-lint-js' project.

Before:
```
Most expensive predicates for completed query RuleOfTwo.ql:
        time  | evals |   max @ iter | predicate
        ------|-------|--------------|----------
        25m9s |       |              | Declaration#4bfb53be::DirectAccessHolder::thisCouldAccessMember#3#dispred#ffff@8a38e2tm
        17m1s |       |              | Declaration#4bfb53be::DirectAccessHolder::thisCouldAccessMember#3#dispred#fffb@0796c497
         3.5s |   130 | 116ms @ 3    | Declaration#4bfb53be::DirectAccessHolder::thisCanAccessClassTrans#fff@926a68j9
         3.3s |       |              | Declaration#4bfb53be::DirectAccessHolder::thisCouldAccessMember#3#dispred#fffb_1230#join_rhs@25e9ffj8
         1.7s |     3 |  1.7s @ 1    | Element#496c7fc2::ElementBase::toString#0#dispred#ff@fcd81c49
         1.3s |       |              | Declaration#4bfb53be::DirectAccessHolder::thisCouldAccessMember#3#dispred#fffb_0132#join_rhs@9c2065t1
         1.3s |       |              | Declaration#4bfb53be::DirectAccessHolder::thisCouldAccessMember#3#dispred#ffff_0132#join_rhs@672330eh
         1.1s |       |              | Declaration#4bfb53be::DirectAccessHolder::thisCanAccessClassTrans#fff_102#join_rhs@f7d5464o
        829ms |   336 |  85ms @ 6    | Enclosing#c50c5fbf::exprEnclosingElement#1#ff@e34d9wq1
        615ms |       |              | Expr#ef463c5d::Expr::getType#ff@e265e79q
```

After:
```
Most expensive predicates for completed query RuleOfTwo.ql:
        time  | evals |  max @ iter | predicate
        ------|-------|-------------|----------
        11.8s |       |             | _#Class#bacd9b46::Class::getADerivedClass#0#dispredPlus#ff_#Declaration#4bfb53be::AccessHolder::getE__#antijoin_rhs#1@fb0627h8
        4.8s |       |             | _#Class#bacd9b46::Class::getADerivedClass#0#dispredPlus#ff_#Declaration#4bfb53be::AccessHolder::getE__#antijoin_rhs#4@c43dbeia
        3.8s |       |             | _#Class#bacd9b46::Class::getADerivedClass#0#dispredPlus#ff_#Declaration#4bfb53be::AccessHolder::getE__#antijoin_rhs#3@313e5963
        3.4s |   130 | 93ms @ 3    | Declaration#4bfb53be::DirectAccessHolder::thisCanAccessClassTrans#fff@a0289bfg
        1.5s |     3 | 1.5s @ 1    | Element#496c7fc2::ElementBase::toString#0#dispred#ff@fcd81c49
        806ms |       |             | Declaration#4bfb53be::DirectAccessHolder::thisCanAccessClassTrans#fff_021#join_rhs@cc1b76s7
        721ms |   336 | 61ms @ 5    | Enclosing#c50c5fbf::exprEnclosingElement#1#ff@e34d9wq1
        489ms |       |             | Expr#ef463c5d::Expr::getType#ff@e265e79q
        337ms |   130 | 62ms @ 5    | Class#bacd9b46::Class::accessOfBaseMemberMulti#ffff@0165b0dr
        329ms |       |             | Variable#7a968d4e::ParameterDeclarationEntry::getAnonymousParameterDescription#0#dispred#ff@0f12bdvq
        211ms |       |             | exprs_10#join_rhs@5481143i
```
2023-03-08 17:44:19 +00:00
Robert Marsh
b941d54f1f C++ Move RangeAnalysis to work around shadowing 2023-03-08 11:32:37 -05:00