Commit Graph

1155 Commits

Author SHA1 Message Date
Tom Hvitved
ec82fb1221 Ruby: Fix bad join-order in RegExpTerm::hasLocationInfo
Before:
```
[2022-03-23 14:50:16] (776s) Tuple counts for RegExpTreeView::RegExpTerm::hasLocationInfo#dispred#f0820431#ffffff/6@5f6cf7if after 1m4s:
                      707103    ~7%     {2} r1 = SCAN Literal::StringlikeLiteral::getNumberOfComponents#dispred#f0820431#ff OUTPUT In.0, (In.1 - 1)
                      64721     ~0%     {5} r2 = JOIN r1 WITH RegExpTreeView::RegExpTerm#7783c185#ffff_1023#join_rhs ON FIRST 1 OUTPUT Lhs.0, Lhs.1, Rhs.1 'this', Rhs.2, Rhs.3
                      64721     ~0%     {5} r3 = JOIN r2 WITH Literal::StringlikeLiteral::getComponent#dispred#f0820431#fff ON FIRST 2 OUTPUT Rhs.2, Lhs.0, Lhs.2 'this', Lhs.3, Lhs.4
                      64721     ~0%     {5} r4 = JOIN r3 WITH AST::AstNode::getLocation#dispred#f0820431#bf ON FIRST 1 OUTPUT Rhs.1, Lhs.1, Lhs.2 'this', Lhs.3, Lhs.4
                      64721     ~3%     {6} r5 = JOIN r4 WITH Locations::Location::hasLocationInfo#dispred#f0820431#ffffff ON FIRST 1 OUTPUT Rhs.1 'filepath', Lhs.1, Lhs.2 'this', Lhs.3, Lhs.4, Rhs.4 'endline'
                      353247577 ~1%     {10} r6 = JOIN r5 WITH Locations::Location::hasLocationInfo#dispred#f0820431#ffffff_1023#join_rhs ON FIRST 1 OUTPUT Lhs.1, 0, Lhs.2 'this', Lhs.3, Lhs.4, Lhs.0 'filepath', Lhs.5 'endline', Rhs.1, Rhs.2 'startline', Rhs.3
                      353247577 ~0%     {9} r7 = JOIN r6 WITH Literal::StringlikeLiteral::getComponent#dispred#f0820431#fff ON FIRST 2 OUTPUT Rhs.2, Lhs.7, Lhs.2 'this', Lhs.3, Lhs.4, Lhs.5 'filepath', Lhs.6 'endline', Lhs.8 'startline', Lhs.9
                      64721     ~2%     {6} r8 = JOIN r7 WITH AST::AstNode::getLocation#dispred#f0820431#bf ON FIRST 2 OUTPUT Lhs.2 'this', Lhs.5 'filepath', Lhs.7 'startline', (Lhs.8 + Lhs.3), Lhs.6 'endline', ((Lhs.8 + Lhs.4) - 1)
                                        return r8
```

After:
```
[2022-03-23 14:58:35] (247s) Tuple counts for RegExpTreeView::RegExpTerm::componentHasLocationInfo#f0820431#fbfffff/7@de55ac7l after 1.1s:
                      12956   ~0%     {3} r1 = SCAN files OUTPUT In.0, 0, In.1 'filepath'
                      9850785 ~0%     {7} r2 = JOIN r1 WITH locations_default_102345#join_rhs ON FIRST 1 OUTPUT Rhs.1, 0, Lhs.2 'filepath', Rhs.2 'startline', Rhs.3 'startcolumn', Rhs.4 'endline', Rhs.5 'endcolumn'
                      823486  ~0%     {8} r3 = JOIN r2 WITH AST::AstNode::getLocation#dispred#f0820431#bf_10#join_rhs ON FIRST 1 OUTPUT 0, Rhs.1, 0, Lhs.2 'filepath', Lhs.3 'startline', Lhs.4 'startcolumn', Lhs.5 'endline', Lhs.6 'endcolumn'
                      698251  ~4%     {7} r4 = JOIN r3 WITH Literal::StringlikeLiteral::getComponent#dispred#f0820431#fff_120#join_rhs ON FIRST 2 OUTPUT Rhs.2, 0, Lhs.3 'filepath', Lhs.4 'startline', Lhs.5 'startcolumn', Lhs.6 'endline', Lhs.7 'endcolumn'
                      64721   ~1%     {7} r5 = JOIN r4 WITH RegExpTreeView::RegExpTerm#7783c185#ffff_10#join_rhs ON FIRST 1 OUTPUT Rhs.1 'this', 0, Lhs.2 'filepath', Lhs.3 'startline', Lhs.4 'startcolumn', Lhs.5 'endline', Lhs.6 'endcolumn'
                                      return r5

[2022-03-23 14:58:35] (247s) Tuple counts for RegExpTreeView::RegExpTerm::hasLocationInfo#dispred#f0820431#ffffff/6@ad66b12q after 53ms:
                      707103 ~7%     {2} r1 = SCAN Literal::StringlikeLiteral::getNumberOfComponents#dispred#f0820431#ff OUTPUT In.0, (In.1 - 1)
                      64721  ~0%     {4} r2 = JOIN r1 WITH RegExpTreeView::RegExpTerm#7783c185#ffff_1023#join_rhs ON FIRST 1 OUTPUT Rhs.1 'this', Lhs.1, Rhs.2, Rhs.3
                      64721  ~5%     {5} r3 = JOIN r2 WITH project#RegExpTreeView::RegExpTerm::componentHasLocationInfo#f0820431#bffffff ON FIRST 2 OUTPUT Lhs.0 'this', Rhs.2 'filepath', Lhs.2, Lhs.3, Rhs.3 'endline'
                      64721  ~2%     {6} r4 = JOIN r3 WITH project#RegExpTreeView::RegExpTerm::componentHasLocationInfo#f0820431#fbfffff ON FIRST 2 OUTPUT Lhs.0 'this', Lhs.1 'filepath', Rhs.2 'startline', (Rhs.3 + Lhs.2), Lhs.4 'endline', ((Rhs.3 + Lhs.3) - 1)
                                     return r4
```
2022-03-29 10:23:32 +02:00
Asger F
e5f2b830f3 Merge pull request #8577 from asgerf/fix-mad-warning
JS/Ruby: Fix regexp in MaD checking
2022-03-28 15:29:16 +02:00
Asger F
f22df765ed Merge pull request #8533 from asgerf/mad-receiver-token
JS/Ruby: Represent non-positional arguments with Argument/Parameter tokens
2022-03-28 15:28:52 +02:00
Nick Rolfe
9406aa2f29 Merge remote-tracking branch 'origin/main' into nickrolfe/constant_regexp 2022-03-28 13:05:34 +01:00
Asger Feldthaus
0b30ecf36a Ruby: add select clause back to Summaries.ql 2022-03-28 13:57:50 +02:00
Arthur Baars
85c4daa2bf Address comments 2022-03-28 13:15:32 +02:00
Asger Feldthaus
d5bcd14733 Sync ApiGraphModels.qll 2022-03-28 12:43:55 +02:00
Nick Rolfe
a9eac19dac Ruby: address review feedback 2022-03-28 11:19:24 +01:00
Arthur Baars
b103679d8a JS/Ruby/Python: rename RegExpTreeView.qll to ReDoSUtilSpecific.qll 2022-03-28 12:17:26 +02:00
Arthur Baars
af1d949d06 Merge pull request #8489 from aibaars/regex-refactor
Ruby: refactor regex libraries
2022-03-28 12:17:00 +02:00
Arthur Baars
accdd9499a Ruby: drop unused predicates that do not exist in Python variant 2022-03-28 11:32:52 +02:00
Rasmus Lerchedahl Petersen
d39410aa2d python: backport review comment to Ruby 2022-03-28 07:35:14 +02:00
Nick Rolfe
c7ba7fd389 Ruby: add changenotes for regexp constant value changes 2022-03-25 11:08:01 +00:00
Asger Feldthaus
8e2ffc2508 Ruby: add the inline test expectations 2022-03-25 11:31:22 +01:00
Asger Feldthaus
5703f63afa Ruby: use InlineFlowTest in Summaries.ql test 2022-03-25 11:18:52 +01:00
Nick Rolfe
034fce0682 Ruby: show constant value type in tests 2022-03-25 08:25:07 +00:00
Nick Rolfe
0613fda57f Ruby: separate constant propagation of regexps from strings 2022-03-24 17:46:58 +00:00
Tom Hvitved
e12b6df118 Merge pull request #8484 from hvitved/ruby/constant-value-rework
Ruby: Rework `getConstantValue` implementation
2022-03-24 14:32:31 +01:00
Rasmus Wriedt Larsen
98c0d73ffe Merge pull request #8524 from RasmusWL/ruby-update-ssrf-concept
Ruby: Minor change of SSRF concept
2022-03-24 13:48:06 +01:00
Arthur Baars
15801fcc85 Apply suggestions from code review
Co-authored-by: Nick Rolfe <nickrolfe@github.com>
2022-03-24 11:37:03 +01:00
Arthur Baars
eef0da09bb Ruby: move RegExpTreeView.qll out of 'internal' 2022-03-24 11:37:03 +01:00
Arthur Baars
1a9aaf4543 Apply suggestions from code review
Co-authored-by: yoff <lerchedahl@gmail.com>
2022-03-24 11:37:03 +01:00
Arthur Baars
5f787144c0 Add change note 2022-03-24 11:37:03 +01:00
Arthur Baars
3c434931ec Ruby: make ParseRegExp.qll and RegExpTreeView.qll internal libraries 2022-03-24 11:37:03 +01:00
Arthur Baars
74aea81fe3 Ruby: refactor regex libraries 2022-03-24 11:37:02 +01:00
Arthur Baars
65f8f56095 Merge branch 'main' into incomplete-url-string-sanitization 2022-03-24 11:27:30 +01:00
Arthur Baars
496aab78a7 Merge pull request #8535 from aibaars/setter-method-arg-location
Ruby: fix location of setter-call argument
2022-03-24 11:26:13 +01:00
Tom Hvitved
eff7cf6396 Merge pull request #8538 from hvitved/ruby/regexpterm-location-perf
Ruby: Fix bad join-order in `RegExpTerm::hasLocationInfo`
2022-03-24 10:01:12 +01:00
Harry Maclean
28a430a2f2 Ruby: Fix bad name of lambda in test
This isn't the identity function, so it's confusing for it to be named
so.
2022-03-24 12:44:41 +13:00
Harry Maclean
3b4206cebf Merge pull request #8517 from hmac/hmac/lambda-captured-var
Ruby: fix bug with captured variable reads in lambdas
2022-03-24 10:00:19 +13:00
Asger Feldthaus
69eb24e748 Ruby: fix toCsv representation of argument/parameter positions 2022-03-23 18:11:09 +01:00
Asger Feldthaus
6870a19ace Ruby: autoformat 2022-03-23 18:06:12 +01:00
Asger Feldthaus
0d51804b5e Ruby: update a comment mentioning Receiver 2022-03-23 18:06:12 +01:00
Asger Feldthaus
ce54eb3c78 Ruby: Add Argument[foo:] syntax for keyword arguments 2022-03-23 18:06:12 +01:00
Asger Feldthaus
c923b9bb9b Ruby: Replace Receiver with Argument[self] 2022-03-23 18:06:12 +01:00
Asger Feldthaus
ec30a0f975 Ruby: replace BlockArgument with Argument[block] 2022-03-23 18:06:12 +01:00
Asger Feldthaus
6d84baf276 Ruby: Support self,block in Argument/Parameter tokens 2022-03-23 18:06:12 +01:00
Tom Hvitved
13be99196f Ruby: Fix bad join-order in RegExpTerm::hasLocationInfo
Before:
```
[2022-03-23 14:50:16] (776s) Tuple counts for RegExpTreeView::RegExpTerm::hasLocationInfo#dispred#f0820431#ffffff/6@5f6cf7if after 1m4s:
                      707103    ~7%     {2} r1 = SCAN Literal::StringlikeLiteral::getNumberOfComponents#dispred#f0820431#ff OUTPUT In.0, (In.1 - 1)
                      64721     ~0%     {5} r2 = JOIN r1 WITH RegExpTreeView::RegExpTerm#7783c185#ffff_1023#join_rhs ON FIRST 1 OUTPUT Lhs.0, Lhs.1, Rhs.1 'this', Rhs.2, Rhs.3
                      64721     ~0%     {5} r3 = JOIN r2 WITH Literal::StringlikeLiteral::getComponent#dispred#f0820431#fff ON FIRST 2 OUTPUT Rhs.2, Lhs.0, Lhs.2 'this', Lhs.3, Lhs.4
                      64721     ~0%     {5} r4 = JOIN r3 WITH AST::AstNode::getLocation#dispred#f0820431#bf ON FIRST 1 OUTPUT Rhs.1, Lhs.1, Lhs.2 'this', Lhs.3, Lhs.4
                      64721     ~3%     {6} r5 = JOIN r4 WITH Locations::Location::hasLocationInfo#dispred#f0820431#ffffff ON FIRST 1 OUTPUT Rhs.1 'filepath', Lhs.1, Lhs.2 'this', Lhs.3, Lhs.4, Rhs.4 'endline'
                      353247577 ~1%     {10} r6 = JOIN r5 WITH Locations::Location::hasLocationInfo#dispred#f0820431#ffffff_1023#join_rhs ON FIRST 1 OUTPUT Lhs.1, 0, Lhs.2 'this', Lhs.3, Lhs.4, Lhs.0 'filepath', Lhs.5 'endline', Rhs.1, Rhs.2 'startline', Rhs.3
                      353247577 ~0%     {9} r7 = JOIN r6 WITH Literal::StringlikeLiteral::getComponent#dispred#f0820431#fff ON FIRST 2 OUTPUT Rhs.2, Lhs.7, Lhs.2 'this', Lhs.3, Lhs.4, Lhs.5 'filepath', Lhs.6 'endline', Lhs.8 'startline', Lhs.9
                      64721     ~2%     {6} r8 = JOIN r7 WITH AST::AstNode::getLocation#dispred#f0820431#bf ON FIRST 2 OUTPUT Lhs.2 'this', Lhs.5 'filepath', Lhs.7 'startline', (Lhs.8 + Lhs.3), Lhs.6 'endline', ((Lhs.8 + Lhs.4) - 1)
                                        return r8
```

After:
```
[2022-03-23 14:58:35] (247s) Tuple counts for RegExpTreeView::RegExpTerm::componentHasLocationInfo#f0820431#fbfffff/7@de55ac7l after 1.1s:
                      12956   ~0%     {3} r1 = SCAN files OUTPUT In.0, 0, In.1 'filepath'
                      9850785 ~0%     {7} r2 = JOIN r1 WITH locations_default_102345#join_rhs ON FIRST 1 OUTPUT Rhs.1, 0, Lhs.2 'filepath', Rhs.2 'startline', Rhs.3 'startcolumn', Rhs.4 'endline', Rhs.5 'endcolumn'
                      823486  ~0%     {8} r3 = JOIN r2 WITH AST::AstNode::getLocation#dispred#f0820431#bf_10#join_rhs ON FIRST 1 OUTPUT 0, Rhs.1, 0, Lhs.2 'filepath', Lhs.3 'startline', Lhs.4 'startcolumn', Lhs.5 'endline', Lhs.6 'endcolumn'
                      698251  ~4%     {7} r4 = JOIN r3 WITH Literal::StringlikeLiteral::getComponent#dispred#f0820431#fff_120#join_rhs ON FIRST 2 OUTPUT Rhs.2, 0, Lhs.3 'filepath', Lhs.4 'startline', Lhs.5 'startcolumn', Lhs.6 'endline', Lhs.7 'endcolumn'
                      64721   ~1%     {7} r5 = JOIN r4 WITH RegExpTreeView::RegExpTerm#7783c185#ffff_10#join_rhs ON FIRST 1 OUTPUT Rhs.1 'this', 0, Lhs.2 'filepath', Lhs.3 'startline', Lhs.4 'startcolumn', Lhs.5 'endline', Lhs.6 'endcolumn'
                                      return r5

[2022-03-23 14:58:35] (247s) Tuple counts for RegExpTreeView::RegExpTerm::hasLocationInfo#dispred#f0820431#ffffff/6@ad66b12q after 53ms:
                      707103 ~7%     {2} r1 = SCAN Literal::StringlikeLiteral::getNumberOfComponents#dispred#f0820431#ff OUTPUT In.0, (In.1 - 1)
                      64721  ~0%     {4} r2 = JOIN r1 WITH RegExpTreeView::RegExpTerm#7783c185#ffff_1023#join_rhs ON FIRST 1 OUTPUT Rhs.1 'this', Lhs.1, Rhs.2, Rhs.3
                      64721  ~5%     {5} r3 = JOIN r2 WITH project#RegExpTreeView::RegExpTerm::componentHasLocationInfo#f0820431#bffffff ON FIRST 2 OUTPUT Lhs.0 'this', Rhs.2 'filepath', Lhs.2, Lhs.3, Rhs.3 'endline'
                      64721  ~2%     {6} r4 = JOIN r3 WITH project#RegExpTreeView::RegExpTerm::componentHasLocationInfo#f0820431#fbfffff ON FIRST 2 OUTPUT Lhs.0 'this', Lhs.1 'filepath', Rhs.2 'startline', (Rhs.3 + Lhs.2), Lhs.4 'endline', ((Rhs.3 + Lhs.3) - 1)
                                     return r4
```
2022-03-23 14:55:06 +01:00
Arthur Baars
06a99c3987 Ruby: fix location of setter-call argument 2022-03-23 12:55:52 +01:00
Rasmus Wriedt Larsen
671889372b Ruby: Update wording of change-note
Co-authored-by: Nick Rolfe <nickrolfe@github.com>
2022-03-23 11:26:41 +01:00
Rasmus Wriedt Larsen
e66932c728 Ruby: Make deprecated getURL work with new modeling
So an "old" query using the deprecated predicate, will still find the
same results, even when the modeling has been updated.
2022-03-23 11:22:34 +01:00
Tom Hvitved
4bcd4d75a9 Address review comments 2022-03-23 11:22:25 +01:00
Rasmus Wriedt Larsen
bbf60b875e Merge pull request #8476 from RasmusWL/shared-concepts-scaffolding
Python/JS/Ruby: Shared concepts scaffolding
2022-03-23 10:22:42 +01:00
Rasmus Wriedt Larsen
64a5c5d9aa Ruby: Keep getURL predicate for easier deprecation
Notice that we still don't fully keep our standard deprecation support,
since the new `getAUrlPart` is still abstract, and therefore will cause
compile errors if not implemented.
2022-03-22 16:48:14 +01:00
Tom Hvitved
c06508570a Ruby: Cache ConstantReadAccess::getValue 2022-03-22 14:15:07 +01:00
Rasmus Wriedt Larsen
9254b2904e Ruby: Adjust HttpClients test 2022-03-22 11:19:55 +01:00
Rasmus Wriedt Larsen
945b52fc46 Ruby: autoformat
😳
2022-03-22 10:59:26 +01:00
Michael Nebel
1d45996001 Merge pull request #8466 from michaelnebel/csharp/refactor-aspartial
C#: Refactor asPartial to allow re-use.
2022-03-22 10:54:54 +01:00
Rasmus Wriedt Larsen
68de6a4e3c Ruby: Add change-note 2022-03-22 10:53:55 +01:00
Rasmus Wriedt Larsen
9c2fc1b415 Ruby: client request: getUrl => getAUrlPart
This is a port of the same change in Python from
f8fc583af3

The description of that commit was:

> I think `getUrl` is a bit too misleading, since from the name, I would
> only ever expect ONE result for one request being made.
>
> `getAUrlPart` captures that there could be multiple results, and that
> they might not constitute a whole URl.
>
> Which is the same naming I used when I tried to model this a long time ago
> a80860cdc6/python/ql/lib/semmle/python/web/Http.qll (L102-L111)
2022-03-22 10:53:55 +01:00