Commit Graph

1091 Commits

Author SHA1 Message Date
Nick Rolfe
fdca896614 Ruby: improve handling of [g]sub!
rb/incomplete-sanitization has a few cases where we find flow from one
one string substitution call to another, e.g.

    a.sub(...).sub(...)

But this didn't find typical chained uses of the destructive variants,
e.g.

    a.sub!(...)
    a.sub!(...)

We now handle those cases by tracking flow from the post-update node for
the receiver of the first call.
2022-04-13 17:19:25 +01:00
Nick Rolfe
bbb8177176 Ruby: add rc/incomplete-sanitization query 2022-04-13 16:48:43 +01:00
Dave Bartolomeo
9f074cd8fd Bump a few more versions
Also fixes up some dependency declarations that should have been "*" because they refer to packs in the same workspace.
2022-04-08 13:01:41 -04:00
Edoardo Pirovano
f25618eed6 Bump minor version of all packs 2022-04-08 15:38:58 +01:00
Edoardo Pirovano
ce82c54b94 Merge branch 'main' into edoardo/3.5-mergeback 2022-04-08 15:30:58 +01:00
Anders Schack-Mulligen
7beed570f2 Dataflow: Sync. 2022-04-07 13:53:48 +02:00
Michael Nebel
72d4c97463 Merge pull request #8628 from michaelnebel/csharp/generatedkind
C#: Introduce generated flag as a part of the kind column for flow summaries
2022-04-07 08:43:30 +02:00
Tom Hvitved
4099d1318f Data flow: Tweak two join-orders
Before
```
[2022-04-06 13:19:29] (96s) Tuple counts for DataFlowImpl2::Stage1::revFlowConsCand#7ad53399#ff/2@i14#aa10f2wi after 4.4s:
                      10681    ~0%     {2} r1 = SCAN DataFlowImpl2::Stage1::revFlow#7ad53399#fff#prev_delta OUTPUT In.0, In.2 'config'
                      982      ~1%     {3} r2 = JOIN r1 WITH DataFlowImpl2::readSet#7ad53399#ffff_2301#join_rhs ON FIRST 2 OUTPUT Rhs.3, Lhs.1 'config', Rhs.2
                      83691528 ~2%     {3} r3 = JOIN r2 WITH DataFlowPublic::ContentSet::getAReadContent#dispred#f0820431#ff ON FIRST 1 OUTPUT Lhs.1 'config', Lhs.2, Rhs.1 'c'
                      83581763 ~2%     {3} r4 = r3 AND NOT DataFlowImpl2::Stage1::revFlowConsCand#7ad53399#ff#prev(Lhs.2 'c', Lhs.0 'config')
                      83581763 ~0%     {3} r5 = SCAN r4 OUTPUT In.2 'c', In.0 'config', In.1
                      0        ~0%     {3} r6 = JOIN r5 WITH DataFlowImpl2::Stage1::fwdFlowConsCand#7ad53399#ff ON FIRST 2 OUTPUT Lhs.2, Lhs.1 'config', Lhs.0 'c'
                      0        ~0%     {2} r7 = JOIN r6 WITH DataFlowImpl2::Stage1::fwdFlow#7ad53399#2#fff_02#join_rhs ON FIRST 2 OUTPUT Lhs.2 'c', Lhs.1 'config'
                                       return r7
```

After
```
[2022-04-06 13:44:38] (6s) Tuple counts for DataFlowImpl2::Stage1::revFlowConsCand#7ad53399#ff/2@i14#5abbf2wn after 6ms:
                      10681  ~0%     {2} r1 = SCAN DataFlowImpl2::Stage1::revFlow#7ad53399#fff#prev_delta OUTPUT In.0, In.2 'config'
                      982    ~1%     {3} r2 = JOIN r1 WITH DataFlowImpl2::readSet#7ad53399#ffff_2301#join_rhs ON FIRST 2 OUTPUT Rhs.3, Lhs.1 'config', Rhs.2
                      109765 ~0%     {3} r3 = JOIN r2 WITH DataFlowImpl2::Stage1::fwdFlowConsCandSet#7ad53399#fff#reorder_0_2_1 ON FIRST 2 OUTPUT Lhs.1 'config', Lhs.2, Rhs.2 'c'
                      0      ~0%     {3} r4 = r3 AND NOT DataFlowImpl2::Stage1::revFlowConsCand#7ad53399#ff#prev(Lhs.2 'c', Lhs.0 'config')
                      0      ~0%     {3} r5 = SCAN r4 OUTPUT In.1, In.0 'config', In.2 'c'
                      0      ~0%     {2} r6 = JOIN r5 WITH DataFlowImpl2::Stage1::fwdFlow#7ad53399#2#fff_02#join_rhs ON FIRST 2 OUTPUT Lhs.2 'c', Lhs.1 'config'
                                     return r6
```
2022-04-06 13:52:30 +02:00
Alex Ford
ccd7bb5e70 Merge pull request #8421 from alexrford/ruby/weak-cryptographic-algorithm
Ruby: Add `rb/weak-cryptographic-algorithm` query
2022-04-05 14:34:45 +01:00
Michael Nebel
784327c183 Java/Ruby: Hardcode generated flag to false. 2022-04-05 08:55:12 +02:00
Michael Nebel
de76df3988 C#: Only use generated summaries, if no handwritten model exist for a particular dataflow callable. 2022-04-05 08:55:12 +02:00
Harry Maclean
8f3578c92a Ruby: Include query results in test 2022-04-05 10:20:02 +12:00
Michael Nebel
3fe941aae2 C#: Add missing empty ext column in generated summaries. 2022-04-04 15:58:35 +02:00
Tom Hvitved
725d76e934 Ruby: Implement ContentSet 2022-04-04 13:51:44 +02:00
Tom Hvitved
c4fbc618a9 Data flow: Sync files 2022-04-04 13:51:44 +02:00
Tom Hvitved
309fd937c1 Data flow: Introduce ContentSet 2022-04-04 13:51:43 +02:00
Tom Hvitved
a5040fd0ce Ruby: Add data-flow test for reverse array stores 2022-04-04 13:51:43 +02:00
Tom Hvitved
50dc3820c6 Merge pull request #8589 from hvitved/regex/speedup-concretise 2022-04-03 17:56:07 +02:00
github-actions[bot]
6af568b16d Post-release preparation for codeql-cli-2.8.5 2022-04-01 16:22:14 +00:00
Chris Smowton
28fa49dcd6 dataflow -> data-flow 2022-04-01 13:22:58 +01:00
github-actions[bot]
ee746d20df Release preparation for version 2.8.5 2022-04-01 10:39:31 +00:00
Chris Smowton
3b0bd3bc0f Improve wording 2022-04-01 11:31:31 +01:00
Chris Smowton
99026a6071 Improve wording of isAdditionalFlow/TaintStep qldoc 2022-04-01 11:07:27 +01:00
Harry Maclean
ae60d40511 Ruby: Fix typo in rb/insecure-dependency qhelp
Co-authored-by: Nick Rolfe <nickrolfe@github.com>
2022-04-01 15:35:53 +13:00
Harry Maclean
5814db19d5 Ruby: Fix bug in rb/insecure-dependency query
Only look at the first component of strings for the prefix.

Co-authored-by: Nick Rolfe <nickrolfe@github.com>
2022-04-01 15:35:21 +13:00
Harry Maclean
3d96c5e6db Ruby: Add test case for rb/insecure-dependency
This tests that we recognise kwargs in hashrocket style:

    gem "foo", "1.2.3", :git => "..."

as well as the modern style:

    gem "foo", "1.2.3", git: "..."
2022-04-01 15:30:07 +13:00
Alex Ford
8b0ebbfecc Ruby: replace use of deprecated getStringOrSymbol() 2022-03-31 17:21:17 +01:00
Alex Ford
882f78c6f9 Merge remote-tracking branch 'origin/main' into ruby/weak-cryptographic-algorithm 2022-03-31 17:17:46 +01:00
Alex Ford
2b66dfa93e Ruby: replace a range field with instanceof 2022-03-31 15:39:11 +01:00
Tom Hvitved
46d69cf544 Regex: Further tweaks to concretise computations 2022-03-31 12:52:43 +02:00
Tom Hvitved
5052452ef9 SuperlinearBackTracking: Speedup concretise 2022-03-31 12:52:42 +02:00
Tom Hvitved
7efe698e56 Address review comment 2022-03-31 12:52:42 +02:00
Tom Hvitved
20f4d5a584 ExponentialBackTracking: Speedup concretise 2022-03-31 12:52:42 +02:00
Tom Hvitved
40986bfcb1 Ruby: Fix broken import 2022-03-31 12:32:03 +02:00
Arthur Baars
15c54f6100 Merge pull request #8354 from aibaars/incomplete-url-string-sanitization
Incomplete url string sanitization
2022-03-31 10:59:51 +02:00
Ian Lynagh
46c27dd20f Merge pull request #8514 from github/post-release-prep/codeql-cli-2.8.4
Post-release preparation for codeql-cli-2.8.4
2022-03-30 16:36:14 +01:00
Nick Rolfe
10b75bff76 Ruby: undo accidental revert of 13be9919 2022-03-30 16:02:12 +01:00
Arthur Baars
031d183bdf Merge pull request #8532 from aibaars/regex-refactor-2
JS/Ruby/Python: rename RegExpTreeView.qll to ReDoSUtilSpecific.qll
2022-03-30 16:38:47 +02:00
Nick Rolfe
a274af2b16 Merge pull request #7985 from github/nickrolfe/constant_regexp
Ruby: separate constant propagation of regexps from strings
2022-03-30 11:37:33 +01:00
Harry Maclean
167bda2d4e Ruby: Add QLDoc for InsecureDependencyQuery.qll 2022-03-30 13:50:12 +13:00
Harry Maclean
d13bbbaf35 Ruby: Add change note for rb/insecure-dependency 2022-03-30 13:39:35 +13:00
Harry Maclean
37cedda63a Ruby: Add InsecureDependencyResolution query
This query looks for places in a Gemfile where URLs with insecure
protocols (HTTP or FTP) are specified.
2022-03-30 13:39:15 +13:00
Tom Hvitved
f429dafb09 Address review comments 2022-03-29 10:23:59 +02:00
Tom Hvitved
15ef8c1d8f Ruby: Cache ConstantReadAccess::getValue 2022-03-29 10:23:54 +02:00
Tom Hvitved
fe50c2879e Ruby: Rework getConstantValue implementation 2022-03-29 10:23:49 +02:00
Tom Hvitved
ec82fb1221 Ruby: Fix bad join-order in RegExpTerm::hasLocationInfo
Before:
```
[2022-03-23 14:50:16] (776s) Tuple counts for RegExpTreeView::RegExpTerm::hasLocationInfo#dispred#f0820431#ffffff/6@5f6cf7if after 1m4s:
                      707103    ~7%     {2} r1 = SCAN Literal::StringlikeLiteral::getNumberOfComponents#dispred#f0820431#ff OUTPUT In.0, (In.1 - 1)
                      64721     ~0%     {5} r2 = JOIN r1 WITH RegExpTreeView::RegExpTerm#7783c185#ffff_1023#join_rhs ON FIRST 1 OUTPUT Lhs.0, Lhs.1, Rhs.1 'this', Rhs.2, Rhs.3
                      64721     ~0%     {5} r3 = JOIN r2 WITH Literal::StringlikeLiteral::getComponent#dispred#f0820431#fff ON FIRST 2 OUTPUT Rhs.2, Lhs.0, Lhs.2 'this', Lhs.3, Lhs.4
                      64721     ~0%     {5} r4 = JOIN r3 WITH AST::AstNode::getLocation#dispred#f0820431#bf ON FIRST 1 OUTPUT Rhs.1, Lhs.1, Lhs.2 'this', Lhs.3, Lhs.4
                      64721     ~3%     {6} r5 = JOIN r4 WITH Locations::Location::hasLocationInfo#dispred#f0820431#ffffff ON FIRST 1 OUTPUT Rhs.1 'filepath', Lhs.1, Lhs.2 'this', Lhs.3, Lhs.4, Rhs.4 'endline'
                      353247577 ~1%     {10} r6 = JOIN r5 WITH Locations::Location::hasLocationInfo#dispred#f0820431#ffffff_1023#join_rhs ON FIRST 1 OUTPUT Lhs.1, 0, Lhs.2 'this', Lhs.3, Lhs.4, Lhs.0 'filepath', Lhs.5 'endline', Rhs.1, Rhs.2 'startline', Rhs.3
                      353247577 ~0%     {9} r7 = JOIN r6 WITH Literal::StringlikeLiteral::getComponent#dispred#f0820431#fff ON FIRST 2 OUTPUT Rhs.2, Lhs.7, Lhs.2 'this', Lhs.3, Lhs.4, Lhs.5 'filepath', Lhs.6 'endline', Lhs.8 'startline', Lhs.9
                      64721     ~2%     {6} r8 = JOIN r7 WITH AST::AstNode::getLocation#dispred#f0820431#bf ON FIRST 2 OUTPUT Lhs.2 'this', Lhs.5 'filepath', Lhs.7 'startline', (Lhs.8 + Lhs.3), Lhs.6 'endline', ((Lhs.8 + Lhs.4) - 1)
                                        return r8
```

After:
```
[2022-03-23 14:58:35] (247s) Tuple counts for RegExpTreeView::RegExpTerm::componentHasLocationInfo#f0820431#fbfffff/7@de55ac7l after 1.1s:
                      12956   ~0%     {3} r1 = SCAN files OUTPUT In.0, 0, In.1 'filepath'
                      9850785 ~0%     {7} r2 = JOIN r1 WITH locations_default_102345#join_rhs ON FIRST 1 OUTPUT Rhs.1, 0, Lhs.2 'filepath', Rhs.2 'startline', Rhs.3 'startcolumn', Rhs.4 'endline', Rhs.5 'endcolumn'
                      823486  ~0%     {8} r3 = JOIN r2 WITH AST::AstNode::getLocation#dispred#f0820431#bf_10#join_rhs ON FIRST 1 OUTPUT 0, Rhs.1, 0, Lhs.2 'filepath', Lhs.3 'startline', Lhs.4 'startcolumn', Lhs.5 'endline', Lhs.6 'endcolumn'
                      698251  ~4%     {7} r4 = JOIN r3 WITH Literal::StringlikeLiteral::getComponent#dispred#f0820431#fff_120#join_rhs ON FIRST 2 OUTPUT Rhs.2, 0, Lhs.3 'filepath', Lhs.4 'startline', Lhs.5 'startcolumn', Lhs.6 'endline', Lhs.7 'endcolumn'
                      64721   ~1%     {7} r5 = JOIN r4 WITH RegExpTreeView::RegExpTerm#7783c185#ffff_10#join_rhs ON FIRST 1 OUTPUT Rhs.1 'this', 0, Lhs.2 'filepath', Lhs.3 'startline', Lhs.4 'startcolumn', Lhs.5 'endline', Lhs.6 'endcolumn'
                                      return r5

[2022-03-23 14:58:35] (247s) Tuple counts for RegExpTreeView::RegExpTerm::hasLocationInfo#dispred#f0820431#ffffff/6@ad66b12q after 53ms:
                      707103 ~7%     {2} r1 = SCAN Literal::StringlikeLiteral::getNumberOfComponents#dispred#f0820431#ff OUTPUT In.0, (In.1 - 1)
                      64721  ~0%     {4} r2 = JOIN r1 WITH RegExpTreeView::RegExpTerm#7783c185#ffff_1023#join_rhs ON FIRST 1 OUTPUT Rhs.1 'this', Lhs.1, Rhs.2, Rhs.3
                      64721  ~5%     {5} r3 = JOIN r2 WITH project#RegExpTreeView::RegExpTerm::componentHasLocationInfo#f0820431#bffffff ON FIRST 2 OUTPUT Lhs.0 'this', Rhs.2 'filepath', Lhs.2, Lhs.3, Rhs.3 'endline'
                      64721  ~2%     {6} r4 = JOIN r3 WITH project#RegExpTreeView::RegExpTerm::componentHasLocationInfo#f0820431#fbfffff ON FIRST 2 OUTPUT Lhs.0 'this', Lhs.1 'filepath', Rhs.2 'startline', (Rhs.3 + Lhs.2), Lhs.4 'endline', ((Rhs.3 + Lhs.3) - 1)
                                     return r4
```
2022-03-29 10:23:32 +02:00
Asger F
e5f2b830f3 Merge pull request #8577 from asgerf/fix-mad-warning
JS/Ruby: Fix regexp in MaD checking
2022-03-28 15:29:16 +02:00
Asger F
f22df765ed Merge pull request #8533 from asgerf/mad-receiver-token
JS/Ruby: Represent non-positional arguments with Argument/Parameter tokens
2022-03-28 15:28:52 +02:00
Nick Rolfe
9406aa2f29 Merge remote-tracking branch 'origin/main' into nickrolfe/constant_regexp 2022-03-28 13:05:34 +01:00
Asger Feldthaus
0b30ecf36a Ruby: add select clause back to Summaries.ql 2022-03-28 13:57:50 +02:00