Commit Graph

51797 Commits

Author SHA1 Message Date
Mathias Vorreiter Pedersen
c7b41ca470 C++: Disable standard order for 'fwdFlow' in stage 1 of dataflow. 2023-03-09 10:41:06 +00:00
Owen Mansel-Chan
55003300fe Merge pull request #12341 from owen-mc/go-tools-status
Go: tools status page support
2023-03-09 09:59:01 +00:00
Jeroen Ketema
e5ce27fbf9 C++: Fix a number of instances where a dataflow node did not have a location 2023-03-09 10:32:30 +01:00
Jeroen Ketema
31fa230c0d C++: Remove toStringImpl predicate that is overridden in every subclass 2023-03-09 10:29:17 +01:00
Jeroen Ketema
4ee13a3234 C++: Remove getLocationImpl predicate that is always overridden
Also remove the predicate referred to in ints implementation, as it is no
longer used.
2023-03-09 10:27:15 +01:00
Jeroen Ketema
791f5913d2 C++: Fix multiple getLocation consistency error in use-use dataflow 2023-03-09 10:25:02 +01:00
Mathias Vorreiter Pedersen
f19f7967c2 C++: Fix join order.
Before (I stopped midway):

```
(72s) Tuple counts for _#Class#bacd9b46::Class::getADerivedClass#0#dispredPlus#ff_#Declaration#4bfb53be::AccessHolder::getE__#antijoin_rhs#1/3@fb0627h8 after 1m4s:
  ...

  20000     ~0%       {5} r28 = r26 UNION r27
  224367484 ~7%       {9} r29 = JOIN r28 WITH Class#bacd9b46::Class::accessOfBaseMember#2#dispred#ffff_1023#join_rhs ON FIRST 1 OUTPUT Rhs.3, "protected", Lhs.1 'arg0', Lhs.2 'arg1', Lhs.3 'arg2', Lhs.0, Lhs.4, Rhs.1, Rhs.2

  111914129 ~0%       {7} r30 = JOIN r29 WITH specifiers ON FIRST 2 OUTPUT Lhs.6, Lhs.2 'arg0', Lhs.3 'arg1', Lhs.4 'arg2', Lhs.5, Lhs.7, Lhs.8

  123503367 ~0%       {8} r31 = JOIN r30 WITH Declaration#4bfb53be::DirectAccessHolder::isFriendOfOrEqualTo#1#dispred#ff ON FIRST 1 OUTPUT Lhs.3 'arg2', Rhs.1, Lhs.1 'arg0', Lhs.2 'arg1', Lhs.4, Lhs.0, Lhs.5, Lhs.6
  331748250 ~0%       {10} r32 = JOIN r31 WITH Class#bacd9b46::Class::accessOfBaseMember#2#dispred#ffff ON FIRST 2 OUTPUT Lhs.2 'arg0', Lhs.3 'arg1', Lhs.0 'arg2', Lhs.4, Lhs.5, Lhs.6, Lhs.7, Lhs.1, Rhs.2, Rhs.3
  331748250 ~0%       {10} r33 = SELECT r32 ON In.8 = In.9
  331748250 ~2%       {9} r34 = SCAN r33 OUTPUT In.7, In.5, In.8, In.0 'arg0', In.1 'arg1', In.2 'arg2', In.3, In.4, In.6
  38000     ~4%       {10} r35 = JOIN r34 WITH Class#bacd9b46::Class::accessOfBaseMember#2#dispred#ffff ON FIRST 3 OUTPUT Rhs.3, Lhs.3 'arg0', Lhs.4 'arg1', Lhs.5 'arg2', Lhs.6, Lhs.7, Lhs.1, Lhs.8, Lhs.0, Lhs.2
  37500     ~0%       {11} r36 = JOIN r35 WITH specifiers ON FIRST 1 OUTPUT Lhs.1 'arg0', Lhs.2 'arg1', Lhs.3 'arg2', Lhs.4, Lhs.5, Lhs.6, Lhs.7, Lhs.8, Lhs.9, Lhs.0, Rhs.1
  28973     ~0%       {11} r37 = SELECT r36 ON In.10 >= "protected"
  28973     ~98%      {6} r38 = SCAN r37 OUTPUT In.8, "public", In.0 'arg0', In.1 'arg1', In.2 'arg2', In.6

  111913629 ~6%       {7} r39 = JOIN r29 WITH specifiers ON FIRST 2 OUTPUT Lhs.6, Lhs.4 'arg2', Lhs.2 'arg0', Lhs.3 'arg1', Lhs.5, Lhs.7, Lhs.8
  110582830 ~0%       {8} r40 = JOIN r39 WITH Declaration#4bfb53be::DirectAccessHolder::isFriendOfOrEqualTo#1#dispred#ff ON FIRST 2 OUTPUT Lhs.1 'arg2', Lhs.5, Lhs.6, Lhs.2 'arg0', Lhs.3 'arg1', Lhs.1 'arg2', Lhs.4, Lhs.0

  123503367 ~0%       {8} r41 = JOIN r30 WITH Declaration#4bfb53be::DirectAccessHolder::isFriendOfOrEqualTo#1#dispred#ff ON FIRST 1 OUTPUT Rhs.1, Lhs.3 'arg2', Lhs.1 'arg0', Lhs.2 'arg1', Lhs.4, Lhs.0, Lhs.5, Lhs.6
  0         ~0%       {8} r42 = JOIN r41 WITH #Class#bacd9b46::Class::getADerivedClass#0#dispredPlus#ff ON FIRST 2 OUTPUT Lhs.0, Lhs.6, Lhs.7, Lhs.2 'arg0', Lhs.3 'arg1', Lhs.1 'arg2', Lhs.4, Lhs.5

  110582830 ~0%       {8} r43 = r40 UNION r42
  15000     ~6%       {8} r44 = JOIN r43 WITH Class#bacd9b46::Class::accessOfBaseMember#2#dispred#ffff ON FIRST 3 OUTPUT Lhs.5 'arg2', Lhs.1, Lhs.3 'arg0', Lhs.4 'arg1', Lhs.6, Lhs.7, Lhs.2, Lhs.0
  ...
```

After:

```
Tuple counts for _#Class#bacd9b46::Class::getADerivedClass#0#dispredPlus#ff_#Declaration#4bfb53be::AccessHolder::getE__#antijoin_rhs#1/3@997a3ai9 after 744ms:
  ...

  78600   ~8%       {6} r29 = r26 UNION r28
  437816  ~0%       {9} r30 = JOIN r29 WITH Class#bacd9b46::Class::accessOfBaseMember#2#dispred#ffff ON FIRST 1 OUTPUT Lhs.1 'arg0', Lhs.2 'arg1', Lhs.0 'arg2', Lhs.3, Lhs.4, Lhs.5, Rhs.1, Rhs.2, Rhs.3
  430928  ~0%       {9} r31 = SELECT r30 ON In.7 = In.8
  430928  ~0%       {7} r32 = SCAN r31 OUTPUT In.5, In.6, In.0 'arg0', In.1 'arg1', In.2 'arg2', In.3, In.7
  1096333 ~0%       {7} r33 = JOIN r32 WITH Class#bacd9b46::Class::accessOfBaseMember#2#dispred#ffff ON FIRST 2 OUTPUT Lhs.1, Lhs.5, Rhs.2, Lhs.2 'arg0', Lhs.3 'arg1', Lhs.4 'arg2', Lhs.6
  777970  ~0%       {8} r34 = JOIN r33 WITH Class#bacd9b46::Class::accessOfBaseMember#2#dispred#ffff ON FIRST 3 OUTPUT Lhs.0, Lhs.1, Lhs.2, Rhs.3, Lhs.3 'arg0', Lhs.4 'arg1', Lhs.5 'arg2', Lhs.6

  334217  ~0%       {6} r35 = JOIN r14 WITH Declaration#4bfb53be::DirectAccessHolder::isFriendOfOrEqualTo#1#dispred#ff ON FIRST 1 OUTPUT Lhs.3 'arg2', Rhs.1, Lhs.1 'arg0', Lhs.2 'arg1', Lhs.4, Lhs.0
  235623  ~0%       {8} r36 = JOIN r35 WITH Class#bacd9b46::Class::accessOfBaseMember#2#dispred#ffff ON FIRST 2 OUTPUT Lhs.2 'arg0', Lhs.3 'arg1', Lhs.0 'arg2', Lhs.4, Lhs.5, Lhs.1, Rhs.2, Rhs.3
  235623  ~0%       {8} r37 = SELECT r36 ON In.6 = In.7
  235623  ~0%       {7} r38 = SCAN r37 OUTPUT In.5, In.6, In.0 'arg0', In.1 'arg1', In.2 'arg2', In.3, In.4
  437303  ~0%       {9} r39 = JOIN r38 WITH Class#bacd9b46::Class::accessOfBaseMember#2#dispred#ffff_0213#join_rhs ON FIRST 2 OUTPUT Rhs.3, Lhs.2 'arg0', Lhs.3 'arg1', Lhs.4 'arg2', Lhs.5, Lhs.6, Lhs.0, Lhs.1, Rhs.2
  437303  ~4%       {10} r40 = JOIN r39 WITH specifiers ON FIRST 1 OUTPUT Lhs.1 'arg0', Lhs.2 'arg1', Lhs.3 'arg2', Lhs.4, Lhs.5, Lhs.6, Lhs.7, Lhs.8, Lhs.0, Rhs.1
  352102  ~1%       {10} r41 = SELECT r40 ON In.9 >= "protected"
  352102  ~0%       {6} r42 = SCAN r41 OUTPUT In.7, In.3, In.0 'arg0', In.1 'arg1', In.2 'arg2', In.6
  775332  ~0%       {8} r43 = JOIN r42 WITH Class#bacd9b46::Class::accessOfBaseMember#2#dispred#ffff ON FIRST 2 OUTPUT Lhs.0, Lhs.1, Rhs.2, Rhs.3, Lhs.2 'arg0', Lhs.3 'arg1', Lhs.4 'arg2', Lhs.5

  1553302 ~51%      {8} r44 = r34 UNION r43
  1553302 ~152%     {7} r45 = JOIN r44 WITH Class#bacd9b46::Class::accessOfBaseMember#2#dispred#ffff ON FIRST 4 OUTPUT Lhs.7, "public", Lhs.4 'arg0', Lhs.5 'arg1', Lhs.6 'arg2', Lhs.2, Lhs.3
  ...
```
2023-03-09 09:23:56 +00:00
Rasmus Lerchedahl Petersen
32d95834d1 python: add test documenting effect of scopes 2023-03-09 10:19:44 +01:00
Jeroen Ketema
55da3257bf C++: Fix multiple toString consistency error in use-use dataflow 2023-03-09 10:08:25 +01:00
Jeroen Ketema
a2248cb5a0 C++: Use getAUse in getIRRepresentationOfOperand 2023-03-09 09:45:54 +01:00
Jeroen Ketema
fadd23a2a0 Merge pull request #12443 from jketema/even-more-configsig
C++: Update more queries with `DataFlow::ConfigSig`
2023-03-09 08:54:31 +01:00
dependabot[bot]
060cd9fada Bump serde from 1.0.152 to 1.0.154 in /ruby
Bumps [serde](https://github.com/serde-rs/serde) from 1.0.152 to 1.0.154.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.152...v1.0.154)

---
updated-dependencies:
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-03-09 04:06:43 +00:00
Ed Minnix
48ca1d0b72 Convert the taint tracking configurations to modules 2023-03-08 19:51:54 -05:00
Ed Minnix
ae0b4970ac Remove commented out code 2023-03-08 19:21:58 -05:00
Ed Minnix
da43a61506 Convert dataflow configuration to using new module-configuration 2023-03-08 19:19:00 -05:00
Chris Smowton
db5bd98781 Return on failure to create file 2023-03-08 22:48:57 +00:00
Owen Mansel-Chan
820de5d36f Remove fatal/panic exits from diagnostic code 2023-03-08 22:00:34 +00:00
Ed Minnix
eeb9a88c3a Renamed test file to follow camel casing convention 2023-03-08 13:56:25 -05:00
Ed Minnix
f680a2ecbf Update test java file to support InlineExpectationsTest 2023-03-08 13:50:02 -05:00
Jeroen Ketema
57c8f5111d C++: Address docs review comments 2023-03-08 19:38:25 +01:00
Ed Minnix
bfd430b446 Remove qlref tests 2023-03-08 13:21:31 -05:00
Ed Minnix
24c9a516c9 Add QLdoc to ArbitraryApkInstallationQuery.qll 2023-03-08 13:21:09 -05:00
Chris Smowton
cacae957b5 Merge pull request #12441 from smowton/smowton/fix/golang-incorrect-integer-conversion-sanitizer
Go: fix incorrect-integer-conversion sanitizer
2023-03-08 18:19:46 +00:00
Jeroen Ketema
3f905f2675 Update docs/codeql/codeql-language-guides/analyzing-data-flow-in-cpp-new.rst
Co-authored-by: Ben Ahmady <32935794+subatoi@users.noreply.github.com>
2023-03-08 19:10:11 +01:00
Michael B. Gale
695160d480 Remove check for stdout redirection 2023-03-08 18:09:09 +00:00
Jeroen Ketema
f6e05836f1 Update docs/codeql/codeql-language-guides/analyzing-data-flow-in-cpp-new.rst
Co-authored-by: Ben Ahmady <32935794+subatoi@users.noreply.github.com>
2023-03-08 19:08:35 +01:00
Jeroen Ketema
2ecc8a5abe Update docs/codeql/codeql-language-guides/analyzing-data-flow-in-cpp-new.rst
Co-authored-by: Ben Ahmady <32935794+subatoi@users.noreply.github.com>
2023-03-08 19:08:19 +01:00
Mathias Vorreiter Pedersen
540ce1f0db Contrary to what the QLDoc says, this predicate was way too large to be
evaluated on the 'quick-lint/quick-lint-js' project.

Before:
```
Most expensive predicates for completed query RuleOfTwo.ql:
        time  | evals |   max @ iter | predicate
        ------|-------|--------------|----------
        25m9s |       |              | Declaration#4bfb53be::DirectAccessHolder::thisCouldAccessMember#3#dispred#ffff@8a38e2tm
        17m1s |       |              | Declaration#4bfb53be::DirectAccessHolder::thisCouldAccessMember#3#dispred#fffb@0796c497
         3.5s |   130 | 116ms @ 3    | Declaration#4bfb53be::DirectAccessHolder::thisCanAccessClassTrans#fff@926a68j9
         3.3s |       |              | Declaration#4bfb53be::DirectAccessHolder::thisCouldAccessMember#3#dispred#fffb_1230#join_rhs@25e9ffj8
         1.7s |     3 |  1.7s @ 1    | Element#496c7fc2::ElementBase::toString#0#dispred#ff@fcd81c49
         1.3s |       |              | Declaration#4bfb53be::DirectAccessHolder::thisCouldAccessMember#3#dispred#fffb_0132#join_rhs@9c2065t1
         1.3s |       |              | Declaration#4bfb53be::DirectAccessHolder::thisCouldAccessMember#3#dispred#ffff_0132#join_rhs@672330eh
         1.1s |       |              | Declaration#4bfb53be::DirectAccessHolder::thisCanAccessClassTrans#fff_102#join_rhs@f7d5464o
        829ms |   336 |  85ms @ 6    | Enclosing#c50c5fbf::exprEnclosingElement#1#ff@e34d9wq1
        615ms |       |              | Expr#ef463c5d::Expr::getType#ff@e265e79q
```

After:
```
Most expensive predicates for completed query RuleOfTwo.ql:
        time  | evals |  max @ iter | predicate
        ------|-------|-------------|----------
        11.8s |       |             | _#Class#bacd9b46::Class::getADerivedClass#0#dispredPlus#ff_#Declaration#4bfb53be::AccessHolder::getE__#antijoin_rhs#1@fb0627h8
        4.8s |       |             | _#Class#bacd9b46::Class::getADerivedClass#0#dispredPlus#ff_#Declaration#4bfb53be::AccessHolder::getE__#antijoin_rhs#4@c43dbeia
        3.8s |       |             | _#Class#bacd9b46::Class::getADerivedClass#0#dispredPlus#ff_#Declaration#4bfb53be::AccessHolder::getE__#antijoin_rhs#3@313e5963
        3.4s |   130 | 93ms @ 3    | Declaration#4bfb53be::DirectAccessHolder::thisCanAccessClassTrans#fff@a0289bfg
        1.5s |     3 | 1.5s @ 1    | Element#496c7fc2::ElementBase::toString#0#dispred#ff@fcd81c49
        806ms |       |             | Declaration#4bfb53be::DirectAccessHolder::thisCanAccessClassTrans#fff_021#join_rhs@cc1b76s7
        721ms |   336 | 61ms @ 5    | Enclosing#c50c5fbf::exprEnclosingElement#1#ff@e34d9wq1
        489ms |       |             | Expr#ef463c5d::Expr::getType#ff@e265e79q
        337ms |   130 | 62ms @ 5    | Class#bacd9b46::Class::accessOfBaseMemberMulti#ffff@0165b0dr
        329ms |       |             | Variable#7a968d4e::ParameterDeclarationEntry::getAnonymousParameterDescription#0#dispred#ff@0f12bdvq
        211ms |       |             | exprs_10#join_rhs@5481143i
```
2023-03-08 17:44:19 +00:00
Ed Minnix
882e909862 Renamed ArbitraryAPKInstallation to ArbitraryApkInstallation 2023-03-08 12:16:46 -05:00
Ed Minnix
5fb5f1b23b Begin InlineExpectationsTest 2023-03-08 12:14:45 -05:00
Ed Minnix
3ea167cadf Split ArbitraryApkInstallation file into 3 files 2023-03-08 12:12:11 -05:00
Edward Minnix III
2d1088e923 Change severity level to error
Co-authored-by: Tony Torralba <atorralba@users.noreply.github.com>
2023-03-08 12:12:11 -05:00
Ed Minnix
0eaad4136e Add RemoteFlowSource as a valid source 2023-03-08 12:12:11 -05:00
Ed Minnix
4d51e4fed0 Change description wording 2023-03-08 12:12:11 -05:00
Ed Minnix
10cd6328dc Add missing QLDocs 2023-03-08 12:12:11 -05:00
Edward Minnix III
8ec5b5b7fa Apply suggestions from code review
Co-authored-by: Jami <57204504+jcogs33@users.noreply.github.com>
2023-03-08 12:12:10 -05:00
Ed Minnix
8fcf00b73d Test improvements 2023-03-08 12:12:10 -05:00
Ed Minnix
fa416564c7 Documentation and examples 2023-03-08 12:12:10 -05:00
Ed Minnix
d3d712fbff Remove Url#parse as a source 2023-03-08 12:12:10 -05:00
Ed Minnix
5f4e8e3e6a Add test cases relating to intents with the ACTION_INSTALL_PACKAGE action 2023-03-08 12:12:10 -05:00
Ed Minnix
f03e90f894 Remove http(s) literal sources 2023-03-08 12:12:10 -05:00
Edward Minnix III
839b88a4bc Formatting, capitalization, and typos
Co-authored-by: Tony Torralba <atorralba@users.noreply.github.com>
2023-03-08 12:12:10 -05:00
Ed Minnix
b606271a61 Additional documentation 2023-03-08 12:12:10 -05:00
Ed Minnix
01b20b3a26 Added external storage test case 2023-03-08 12:12:10 -05:00
Ed Minnix
c448481bf7 Added test expectations 2023-03-08 12:12:10 -05:00
Ed Minnix
12f78dbed4 Add DataFlow::PathGraph import 2023-03-08 12:12:10 -05:00
Ed Minnix
cd5a46123e Add a change note 2023-03-08 12:12:10 -05:00
Ed Minnix
2d6cdff14b Add period to alert message 2023-03-08 12:12:10 -05:00
Ed Minnix
0ec4df28f5 Add query metadata 2023-03-08 12:12:10 -05:00
Ed Minnix
3f589722c2 Refactor query to change returned source 2023-03-08 12:12:10 -05:00