Commit Graph

3367 Commits

Author SHA1 Message Date
Nick Rolfe
134cf4e0e1 Ruby: tweak join order in API::Impl::edge 2022-05-25 10:54:43 +01:00
Rasmus Wriedt Larsen
f7e58a9335 Ruby: Apply nomagic on parameterMatch instead 2022-05-25 10:07:02 +02:00
Anders Schack-Mulligen
673355df65 Fix markdown lists 2022-05-25 10:02:48 +02:00
Rasmus Wriedt Larsen
0bf0e0e16c Revert "Ruby: Fix performance for argumentPositionMatch"
as requested to use a different performance fix

and

Revert "Dataflow: Sync `DataFlowImplCommon`"

This reverts commit c9a833fc07
This reverts commit 911ddb9b2c.
2022-05-25 09:56:10 +02:00
github-actions[bot]
1f1b364feb Release preparation for version 2.9.3 2022-05-25 07:46:48 +00:00
Rasmus Wriedt Larsen
ae65af2c07 Ruby: Fix Argument[any] in Hash.qll
With this PR, `self` have to be explicitly added. A few edges were
removed, and I don't know why. It doesn't seem to affect results, so I
did not worry too much.
2022-05-24 18:09:52 +02:00
Rasmus Wriedt Larsen
04ac466189 Merge branch 'main' into ruby-mad-argument-self 2022-05-24 18:04:02 +02:00
Rasmus Wriedt Larsen
c9a833fc07 Ruby: Fix performance for argumentPositionMatch
before

[2022-05-24 17:29:07] (50s) Tuple counts for DataFlowImplCommon::argumentPositionMatch#4f8df883#fff/3@03b4073c after 35.8s:
                      156250456 ~2%     {4} r1 = JOIN DataFlowDispatch::Cached::TParameterPosition#36b84300#f WITH DataFlowImplCommon::ArgNode::argumentOf#dispred#f0820431#fff CARTESIAN PRODUCT OUTPUT Rhs.2, Lhs.0 'ppos', Rhs.0, Rhs.1 'call'

                      0         ~0%     {3} r2 = JOIN r1 WITH DataFlowDispatch::Cached::TAnyKeywordArgumentPosition#36b84300#f ON FIRST 1 OUTPUT Lhs.1 'ppos', Lhs.2 'arg', Lhs.3 'call'
                      0         ~0%     {3} r3 = JOIN r2 WITH DataFlowDispatch::Cached::TKeywordParameterPosition#36b84300#ff_1#join_rhs ON FIRST 1 OUTPUT Lhs.2 'call', Lhs.1 'arg', Lhs.0 'ppos'

                      156250456 ~2%     {4} r4 = JOIN DataFlowDispatch::Cached::TParameterPosition#36b84300#f WITH DataFlowImplCommon::ArgNode::argumentOf#dispred#f0820431#fff CARTESIAN PRODUCT OUTPUT Lhs.0 'ppos', Rhs.0, Rhs.1 'call', Rhs.2

                      252424    ~0%     {4} r5 = JOIN r4 WITH DataFlowDispatch::Cached::TSelfParameterPosition#36b84300#f ON FIRST 1 OUTPUT Lhs.3, Lhs.0 'ppos', Lhs.1 'arg', Lhs.2 'call'
                      121009    ~0%     {3} r6 = JOIN r5 WITH DataFlowDispatch::Cached::TSelfArgumentPosition#36b84300#f ON FIRST 1 OUTPUT Lhs.3 'call', Lhs.2 'arg', Lhs.1 'ppos'

                      121009    ~0%     {3} r7 = r3 UNION r6

                      252424    ~0%     {4} r8 = JOIN r4 WITH DataFlowDispatch::Cached::TBlockParameterPosition#36b84300#f ON FIRST 1 OUTPUT Lhs.3, Lhs.0 'ppos', Lhs.1 'arg', Lhs.2 'call'
                      11764     ~5%     {3} r9 = JOIN r8 WITH DataFlowDispatch::Cached::TBlockArgumentPosition#36b84300#f ON FIRST 1 OUTPUT Lhs.3 'call', Lhs.2 'arg', Lhs.1 'ppos'

                      252424    ~2%     {4} r10 = JOIN r4 WITH DataFlowDispatch::Cached::TAnyKeywordParameterPosition#36b84300#f ON FIRST 1 OUTPUT Lhs.3, Lhs.0 'ppos', Lhs.1 'arg', Lhs.2 'call'
                      20865     ~2%     {3} r11 = JOIN r10 WITH DataFlowDispatch::Cached::TKeywordArgumentPosition#36b84300#ff_1#join_rhs ON FIRST 1 OUTPUT Lhs.3 'call', Lhs.2 'arg', Lhs.1 'ppos'

                      32629     ~4%     {3} r12 = r9 UNION r11
                      153638    ~4%     {3} r13 = r7 UNION r12

                      252424    ~1%     {4} r14 = JOIN r4 WITH DataFlowDispatch::Cached::TAnyParameterPosition#36b84300#f ON FIRST 1 OUTPUT Lhs.0 'ppos', Lhs.1 'arg', Lhs.2 'call', Lhs.3
                      131415    ~0%     {4} r15 = r14 AND NOT DataFlowDispatch::Cached::TSelfArgumentPosition#36b84300#f(Lhs.3)
                      131415    ~0%     {3} r16 = SCAN r15 OUTPUT In.2 'call', In.1 'arg', In.0 'ppos'

                      0         ~0%     {4} r17 = JOIN r1 WITH DataFlowDispatch::Cached::TAnyArgumentPosition#36b84300#f ON FIRST 1 OUTPUT Lhs.1 'ppos', Lhs.2 'arg', Lhs.3 'call', Lhs.0
                      0         ~0%     {4} r18 = r17 AND NOT DataFlowDispatch::Cached::TSelfParameterPosition#36b84300#f(Lhs.0 'ppos')
                      0         ~0%     {3} r19 = SCAN r18 OUTPUT In.2 'call', In.1 'arg', In.0 'ppos'

                      131415    ~0%     {3} r20 = r16 UNION r19

                      5553328   ~5%     {5} r21 = JOIN r4 WITH DataFlowDispatch::Cached::TPositionalParameterPosition#36b84300#ff_10#join_rhs ON FIRST 1 OUTPUT Rhs.1, Lhs.3, Lhs.0 'ppos', Lhs.1 'arg', Lhs.2 'call'
                      98201     ~0%     {3} r22 = JOIN r21 WITH DataFlowDispatch::Cached::TPositionalArgumentPosition#36b84300#ff ON FIRST 2 OUTPUT Lhs.4 'call', Lhs.3 'arg', Lhs.2 'ppos'

                      149435008 ~0%     {5} r23 = JOIN r4 WITH DataFlowDispatch::Cached::TKeywordParameterPosition#36b84300#ff_10#join_rhs ON FIRST 1 OUTPUT Rhs.1, Lhs.3, Lhs.0 'ppos', Lhs.1 'arg', Lhs.2 'call'
                      17930     ~3%     {3} r24 = JOIN r23 WITH DataFlowDispatch::Cached::TKeywordArgumentPosition#36b84300#ff ON FIRST 2 OUTPUT Lhs.4 'call', Lhs.3 'arg', Lhs.2 'ppos'

                      252424    ~0%     {5} r25 = JOIN r4 WITH DataFlowDispatch::Cached::TPositionalParameterLowerBoundPosition#36b84300#ff_10#join_rhs ON FIRST 1 OUTPUT Lhs.3, Lhs.0 'ppos', Lhs.1 'arg', Lhs.2 'call', Rhs.1
                      98786     ~0%     {6} r26 = JOIN r25 WITH DataFlowDispatch::Cached::TPositionalArgumentPosition#36b84300#ff_10#join_rhs ON FIRST 1 OUTPUT Lhs.1 'ppos', Lhs.2 'arg', Lhs.3 'call', Lhs.0, Lhs.4, Rhs.1
                      98786     ~0%     {6} r27 = SELECT r26 ON In.5 >= In.4
                      98786     ~3%     {3} r28 = SCAN r27 OUTPUT In.2 'call', In.1 'arg', In.0 'ppos'

                      116716    ~0%     {3} r29 = r24 UNION r28
                      214917    ~0%     {3} r30 = r22 UNION r29
                      346332    ~0%     {3} r31 = r20 UNION r30
                      499970    ~1%     {3} r32 = r13 UNION r31
                                        return r32

now

[2022-05-24 17:26:06] (14s) Tuple counts for DataFlowImplCommon::argumentPositionMatch#4f8df883#fff/3@97d3444p after 149ms:
                      1000304 ~9%     {2} r1 = JOIN DataFlowDispatch::Cached::TParameterPosition#36b84300#f WITH DataFlowDispatch::Cached::TArgumentPosition#36b84300#f CARTESIAN PRODUCT OUTPUT Lhs.0 'ppos', Rhs.0

                      1616    ~0%     {2} r2 = JOIN r1 WITH DataFlowDispatch::Cached::TSelfParameterPosition#36b84300#f ON FIRST 1 OUTPUT Lhs.1, Lhs.0 'ppos'
                      1       ~0%     {2} r3 = JOIN r2 WITH DataFlowDispatch::Cached::TSelfArgumentPosition#36b84300#f ON FIRST 1 OUTPUT Lhs.0, Lhs.1 'ppos'

                      1616    ~5%     {2} r4 = JOIN r1 WITH DataFlowDispatch::Cached::TBlockParameterPosition#36b84300#f ON FIRST 1 OUTPUT Lhs.1, Lhs.0 'ppos'
                      1       ~0%     {2} r5 = JOIN r4 WITH DataFlowDispatch::Cached::TBlockArgumentPosition#36b84300#f ON FIRST 1 OUTPUT Lhs.0, Lhs.1 'ppos'

                      2       ~0%     {2} r6 = r3 UNION r5

                      1616    ~0%     {2} r7 = JOIN r1 WITH DataFlowDispatch::Cached::TAnyKeywordParameterPosition#36b84300#f ON FIRST 1 OUTPUT Lhs.1, Lhs.0 'ppos'
                      1533    ~0%     {2} r8 = JOIN r7 WITH DataFlowDispatch::Cached::TKeywordArgumentPosition#36b84300#ff_1#join_rhs ON FIRST 1 OUTPUT Lhs.0, Lhs.1 'ppos'

                      1000304 ~0%     {2} r9 = JOIN DataFlowDispatch::Cached::TParameterPosition#36b84300#f WITH DataFlowDispatch::Cached::TArgumentPosition#36b84300#f CARTESIAN PRODUCT OUTPUT Rhs.0, Lhs.0 'ppos'

                      619     ~0%     {2} r10 = JOIN r9 WITH DataFlowDispatch::Cached::TAnyKeywordArgumentPosition#36b84300#f ON FIRST 1 OUTPUT Lhs.1 'ppos', Lhs.0
                      592     ~0%     {2} r11 = JOIN r10 WITH DataFlowDispatch::Cached::TKeywordParameterPosition#36b84300#ff_1#join_rhs ON FIRST 1 OUTPUT Lhs.1, Lhs.0 'ppos'

                      2125    ~1%     {2} r12 = r8 UNION r11
                      2127    ~1%     {2} r13 = r6 UNION r12

                      1616    ~0%     {2} r14 = JOIN r1 WITH DataFlowDispatch::Cached::TAnyParameterPosition#36b84300#f ON FIRST 1 OUTPUT Lhs.0 'ppos', Lhs.1
                      1615    ~0%     {2} r15 = r14 AND NOT DataFlowDispatch::Cached::TSelfArgumentPosition#36b84300#f(Lhs.1)
                      1615    ~2%     {2} r16 = SCAN r15 OUTPUT In.1, In.0 'ppos'

                      619     ~0%     {2} r17 = JOIN r9 WITH DataFlowDispatch::Cached::TAnyArgumentPosition#36b84300#f ON FIRST 1 OUTPUT Lhs.1 'ppos', Lhs.0
                      618     ~0%     {2} r18 = r17 AND NOT DataFlowDispatch::Cached::TSelfParameterPosition#36b84300#f(Lhs.0 'ppos')
                      618     ~0%     {2} r19 = SCAN r18 OUTPUT In.1, In.0 'ppos'

                      2233    ~0%     {2} r20 = r16 UNION r19

                      35552   ~0%     {3} r21 = JOIN r1 WITH DataFlowDispatch::Cached::TPositionalParameterPosition#36b84300#ff_10#join_rhs ON FIRST 1 OUTPUT Rhs.1, Lhs.1, Lhs.0 'ppos'
                      22      ~0%     {2} r22 = JOIN r21 WITH DataFlowDispatch::Cached::TPositionalArgumentPosition#36b84300#ff ON FIRST 2 OUTPUT Lhs.1, Lhs.2 'ppos'

                      956672  ~0%     {3} r23 = JOIN r1 WITH DataFlowDispatch::Cached::TKeywordParameterPosition#36b84300#ff_10#join_rhs ON FIRST 1 OUTPUT Rhs.1, Lhs.1, Lhs.0 'ppos'
                      592     ~0%     {2} r24 = JOIN r23 WITH DataFlowDispatch::Cached::TKeywordArgumentPosition#36b84300#ff ON FIRST 2 OUTPUT Lhs.1, Lhs.2 'ppos'

                      1616    ~0%     {3} r25 = JOIN r1 WITH DataFlowDispatch::Cached::TPositionalParameterLowerBoundPosition#36b84300#ff_10#join_rhs ON FIRST 1 OUTPUT Lhs.1, Lhs.0 'ppos', Rhs.1
                      79      ~0%     {4} r26 = JOIN r25 WITH DataFlowDispatch::Cached::TPositionalArgumentPosition#36b84300#ff_10#join_rhs ON FIRST 1 OUTPUT Lhs.1 'ppos', Lhs.0, Lhs.2, Rhs.1
                      79      ~0%     {4} r27 = SELECT r26 ON In.3 >= In.2
                      79      ~1%     {2} r28 = SCAN r27 OUTPUT In.1, In.0 'ppos'

                      671     ~0%     {2} r29 = r24 UNION r28
                      693     ~0%     {2} r30 = r22 UNION r29
                      2926    ~0%     {2} r31 = r20 UNION r30
                      5053    ~0%     {2} r32 = r13 UNION r31
                      499970  ~6%     {3} r33 = JOIN r32 WITH DataFlowImplCommon::ArgNode::argumentOf#dispred#f0820431#fff_201#join_rhs ON FIRST 1 OUTPUT Rhs.2 'call', Rhs.1 'arg', Lhs.1 'ppos'
                                      return r33
2022-05-24 17:31:36 +02:00
Nick Rolfe
dd52a70454 Merge pull request #9292 from github/nickrolfe/cfg_scope
Ruby: rename CfgScope::Range_ to CfgScopeImpl
2022-05-24 15:53:16 +01:00
Michael Nebel
daace0fe68 Merge pull request #9270 from michaelnebel/csharp/summarized-callable-fix
C#: Summarized callable
2022-05-24 16:36:44 +02:00
Nick Rolfe
4b4a15c1b6 Ruby: rename CfgScope::Range_ to CfgScopeImpl 2022-05-24 14:34:44 +01:00
Tom Hvitved
728ccafe2b Merge pull request #9024 from hvitved/dataflow/content-flow-lib
Data flow: Introduce `ContentDataFlow.qll`
2022-05-24 15:09:16 +02:00
Tom Hvitved
1ae8087379 Update ruby/ql/lib/codeql/ruby/frameworks/core/Hash.qll
Co-authored-by: Nick Rolfe <nickrolfe@github.com>
2022-05-24 14:27:59 +02:00
Tom Hvitved
daf81ae90d Address review comments 2022-05-24 14:27:59 +02:00
Tom Hvitved
ab46c075f7 Ruby: Add change note 2022-05-24 14:27:58 +02:00
Tom Hvitved
63c70b9e7a Address review comments 2022-05-24 14:27:58 +02:00
Tom Hvitved
faf24a4f18 Ruby: Data-flow through hashes 2022-05-24 14:27:55 +02:00
Harry Maclean
334c43a2b7 Ruby: Add tests for ActiveSupport modelling 2022-05-24 09:35:26 +01:00
Harry Maclean
deff24e8e0 Fix singleton set literal 2022-05-24 09:35:26 +01:00
Harry Maclean
1fd54ed8c3 Ruby: Recognised ActiveSupport::TaggedLogging as a logger 2022-05-24 09:35:26 +01:00
Harry Maclean
dc4ddf6899 Ruby: Recognise ActiveSupport::Logger as a logger 2022-05-24 09:35:26 +01:00
Harry Maclean
14d2ff6528 Ruby: Model ActiveSupport extensions to Enumerable 2022-05-24 09:35:26 +01:00
Harry Maclean
ad2eaf0835 Ruby: Flow for ActiveSupport String extensions
Add taint flow summaries for ActiveSupport String extensions which
transform the string in various ways, for example `camelcase` and
`underscore`.

DCA suggests this increases the sensitivity of rb/code-injection,
catching cases such as

    params[:foo].camelcase.constantize
2022-05-24 09:35:26 +01:00
Michael Nebel
94664f11f5 C#/Java/Ruby: Sync files. 2022-05-24 08:21:39 +02:00
Arthur Baars
cf2eb0d3a1 Merge branch 'main' into instance-variable-flow 2022-05-23 18:48:51 +02:00
Harry Maclean
905a37c273 Merge pull request #9137 from hmac/hmac/cfg-ql-class
Ruby: Add getAPrimaryQlClass to CfgNodes classes
2022-05-23 15:37:51 +01:00
Harry Maclean
ae3a30256b Ruby: Add getAPrimaryQlClass to CfgNode 2022-05-23 14:02:23 +01:00
Rasmus Wriedt Larsen
85fa6fba63 Concepts: Move CryptographicOperation.isWeak to be Ruby specific 2022-05-23 14:39:06 +02:00
Tom Hvitved
64be958c52 Merge pull request #9262 from hvitved/ruby/local-source-node-antijoin
Ruby: Eliminate bad `isLocalSourceNode` antijoin
2022-05-23 14:36:03 +02:00
Rasmus Wriedt Larsen
3afa9425ef Ruby: Add TAnyKeywordArgumentPosition and TAnyKeywordParameterPosition 2022-05-23 14:03:45 +02:00
Arthur Baars
965f83e198 Reformat ControlFlowGraphImpl.qll 2022-05-23 12:22:47 +02:00
Arthur Baars
eabf2ed2d3 Apply suggestions from code review
Co-authored-by: Tom Hvitved <hvitved@github.com>
2022-05-23 12:18:48 +02:00
Tom Hvitved
d6b0772f7c Ruby: Improve performance of instanceVariableSelfSynthesis 2022-05-23 12:08:41 +02:00
Arthur Baars
d86983b7c8 Ruby: use InstanceVariableRead/WriteAccess CFG nodes 2022-05-23 12:03:11 +02:00
Arthur Baars
5fa4f07f7d Improve QLDoc 2022-05-23 11:59:28 +02:00
Arthur Baars
29ea1b2f24 Ruby: rename getSelfVariableAccess to getReceiver 2022-05-23 11:30:29 +02:00
Arthur Baars
f6ca3921f9 Add change note 2022-05-23 10:59:54 +02:00
Tom Hvitved
bbdedf5f14 Ruby: Eliminate bad isLocalSourceNode antijoin
Gets rid of
```
Tuple counts for DataFlowPrivate::Cached::isLocalSourceNode#462ff392#f#antijoin_rhs@dd2f927s:
        20905019     ~3%    {2} r1 = JOIN DataFlowPrivate::Cached::TExprNode#462ff392#ff_1#higher_order_body WITH boundedFastTC(DataFlowPrivate::Cached::localFlowStepTypeTracker#462ff392#ff_10#higher_order_body,DataFlowPrivate::Cached::TExprNode#462ff392#ff_1#higher_order_body) ON FIRST 1 OUTPUT Rhs.1, Lhs.0

        10420128  ~1496%    {1} r2 = JOIN r1 WITH DataFlowPrivate::Cached::TExprNode#462ff392#ff_1#higher_order_body ON FIRST 1 OUTPUT Lhs.1

          480918     ~8%    {1} r3 = JOIN r1 WITH DataFlowPrivate::Cached::entrySsaDefinition#462ff392#f ON FIRST 1 OUTPUT Lhs.1

        10901046  ~1218%    {1} r4 = r2 UNION r3
                            return r4
```
2022-05-23 10:54:17 +02:00
Michael Nebel
bf958ff5bb Merge pull request #9255 from michaelnebel/csharp/test-clearscontent
C#: Remove default clears content.
2022-05-23 10:30:30 +02:00
Anders Schack-Mulligen
f2218944f6 Merge pull request #9214 from hvitved/dataflow/lambda-fp-flow
Data flow: Do not discard call context when computing reverse lambda flow through jumps
2022-05-23 10:02:51 +02:00
Michael Nebel
ddde1d4607 C#: Remove default clears content. 2022-05-22 15:16:44 +02:00
Arthur Baars
d9c2b78aa2 Ruby: flow through instance variables 2022-05-20 16:30:58 +02:00
Anders Schack-Mulligen
8beef45599 Merge pull request #9195 from aschackmull/java/perf-local-flow
Java: Performance fixes for local flow relation
2022-05-20 12:38:02 +02:00
Michael Nebel
20af134ff0 Merge pull request #9210 from michaelnebel/dataflow/summarizedcallablerefactor
DataFlow - SummarizedCallable refactor
2022-05-20 09:32:30 +02:00
Alex Ford
6b7abef405 Ruby: remove unnecessary CryptographicOperation#isWeak override 2022-05-19 16:01:34 +01:00
Alex Ford
8b7bb7c358 Ruby: add missing qldoc 2022-05-19 15:55:48 +01:00
Alex Ford
d3662cf54a Deprecate CryptographicOperation#isWeak and add a default implementation 2022-05-19 15:46:13 +01:00
Alex Ford
3d66905dc6 Share the CryptographicOperation and BlockMode concepts between dynamic langs 2022-05-19 15:46:03 +01:00
Rasmus Wriedt Larsen
0879b6ae12 Ruby: Fix Argument[any,any-named] handling for path component in MaD 2022-05-19 15:51:30 +02:00
Rasmus Wriedt Larsen
7784b9f879 Ruby: WIP: Make Argument[any] and any-named work
It's not fully working I think the problem is that the code below ties
up `Argument[x]` with parameter positions, and `Parameter[x]` with
argument positions. This flip might be correct for flow-summaries, but
it does NOT seem to be correct for the `path` component  in MaD.

Specifically, quick-eval for ParameterPosition does NOT include `keyword key` while
quick-eval for ArgumentPosition DOES include `keyword key`!

For the test `Foo.sinkAnyNamedArg(key: tainted) # $ MISSING: hasValueFlow=tainted`

c8be8d30b3/ruby/ql/lib/codeql/ruby/frameworks/data/internal/ApiGraphModelsSpecific.qll (L130-L133)
2022-05-19 15:51:25 +02:00