Compare commits

...

1348 Commits

Author SHA1 Message Date
Felicity Chapman
e178626226 Merge pull request #7653 from github/felicitymay-patch-1
Port changes from main to rc/3.3 to avoid regression
2022-01-20 10:45:13 +00:00
Felicity Chapman
51e8b4c7ed Port changes from main to rc/3.3 to avoid regression 2022-01-19 14:26:52 +00:00
Felicity Chapman
c3ed74d63c Merge pull request #7604 from github/lgtm-1.29-docs
Update version numbers in CodeQL support notes for LGTM 1.29
2022-01-18 11:09:38 +00:00
Felicity Chapman
e0110bd25e FIx typo in new note 2022-01-17 17:20:00 +00:00
Felicity Chapman
e7dde79d50 Add note and link to main CodeQL CLI docs 2022-01-17 17:14:58 +00:00
Felicity Chapman
fdf77ad2b9 Update version numbers for LGTM 1.29 2022-01-14 15:07:29 +00:00
Arthur Baars
1327d7c8d5 Merge pull request #7043 from aibaars/fix-ql-tests-3.3
Ruby: Fix QL tests and Rust compilation error
2021-11-03 13:59:29 +01:00
Arthur Baars
aab8c64973 Ruby: fix compilation error 2021-11-03 12:32:45 +01:00
Arthur Baars
2c5d5ecdd8 Ruby: QLTest: fix pack search path for upgrades 2021-11-03 12:14:58 +01:00
Arthur Baars
32765e9bc1 Ruby: trigger jobs on workflow change 2021-11-03 12:14:58 +01:00
Dave Bartolomeo
d828ab7fd2 Merge pull request #6955 from github/codeql-ruby-3.3
RC 3.3: merge codeql-ruby repository into github/codeql
2021-11-02 09:57:49 -04:00
Arthur Baars
b79f8f1890 Fix CI jobs 2021-10-25 17:01:50 +02:00
shati-patel
8cd86ae8f5 Move queries.xml to src 2021-10-25 17:01:50 +02:00
shati-patel
b23b3c33f6 Add a queries.xml file (for CWE coverage) docs 2021-10-25 17:01:50 +02:00
Arthur Baars
de38570424 Merge identical-files.json 2021-10-25 17:01:44 +02:00
Arthur Baars
1bf4542c89 Remove github/codeql submodule 2021-10-25 16:42:45 +02:00
Arthur Baars
ddbba403f8 Update CodeSpaces configuration 2021-10-25 16:42:45 +02:00
Arthur Baars
aeb9ace694 Add ruby to CODEOWNERS 2021-10-25 16:42:45 +02:00
Arthur Baars
7741a72cc5 Merge remote-tracking branch 'codeql-ruby/rc/3.3' into codeql/rc/3.3 2021-10-25 16:41:36 +02:00
Arthur Baars
8ce7b287d1 Update dependabot config 2021-10-25 16:13:37 +02:00
Arthur Baars
3554e8d105 Drop LICENSE and CODE_OF_CONDUCT.md 2021-10-25 16:13:37 +02:00
Arthur Baars
2de757335f Update Ruby workflows 2021-10-25 16:13:35 +02:00
Arthur Baars
068beeff56 Move create-extractor-pack Action 2021-10-25 16:12:08 +02:00
Arthur Baars
d2ea732539 Remove CodeSpaces configuration 2021-10-25 16:12:08 +02:00
Arthur Baars
ba32c54038 Move files to ruby subfolder 2021-10-25 16:11:59 +02:00
Shati Patel
702c647556 Merge pull request #6904 from shati-patel/ruby-query-help
Docs: Add Ruby to query help pages
2021-10-18 16:13:50 +01:00
shati-patel
b9ede183b0 Docs: Add Ruby to query help pages 2021-10-18 11:48:24 +01:00
Dave Bartolomeo
91b2ee2f10 Merge pull request #6822 from github/lgtm.com
Make sure the lgtm.com branch is an ancestor of rc/3.3
2021-10-06 06:58:13 -04:00
Geoffrey White
4c6f4ef14b Revert "C++: change note" and "C++: Exclusion rules for system macros"
This reverts commit a055c86c4f.
This reverts commit 237a7d34b8.
2021-10-06 10:21:19 +01:00
Nick Rolfe
1d58f8cd50 Merge pull request #320 from github/rasmuswl/fix-hasLocationInfo-url 2021-09-29 13:23:08 +01:00
Tom Hvitved
c69762bc14 Merge pull request #317 from github/hvitved/disable-operation-resolution
Temporarily disable operation call resolution
2021-09-29 14:17:05 +02:00
Rasmus Wriedt Larsen
3a270abcdc Fix hasLocationInfo URL reference
Port of https://github.com/github/codeql/pull/6775
2021-09-29 14:04:25 +02:00
Tom Hvitved
10d19bf05b Temporarily disable operation call resolution 2021-09-29 09:40:41 +02:00
Tom Hvitved
5219b1a8b9 Merge pull request #310 from github/hvitved/more-instanceof
More uses of `instanceof` in the external/internal AST layer
2021-09-27 16:11:04 +02:00
Tom Hvitved
8018c1525d Merge pull request #314 from github/hvitved/setter-method-call-base
Strengthen the type of `SetterMethodCall`
2021-09-27 15:29:07 +02:00
Nick Rolfe
79c2f09585 Merge pull request #302 from github/rm_tokeninfo_idx
Remove unused columns from tokeninfo tables
2021-09-27 14:19:38 +01:00
Nick Rolfe
b2c4daecd5 Merge pull request #303 from github/nickrolfe/node_kind_id
Use integer comparisons instead of strings when scanning ERB files
2021-09-27 14:18:10 +01:00
Tom Hvitved
317303cdad Strengthen the type of SetterMethodCall 2021-09-27 14:05:28 +02:00
Arthur Baars
2a4747b27e Merge pull request #313 from github/hmac-remove-unicode-char
Remove unicode character from doc string
2021-09-27 12:57:21 +02:00
Harry Maclean
3e100bc2a9 Remove unicode character from doc string
We require that all source code is in ASCII.
2021-09-27 11:40:04 +01:00
Tom Hvitved
793368d670 More uses of instanceof in the external/internal AST layer 2021-09-24 15:55:15 +02:00
Harry Maclean
74982cb3aa Merge pull request #307 from github/hmac-outgoing-http-2
Model some more HTTP clients
2021-09-24 12:30:48 +01:00
Tom Hvitved
141f5f7605 Merge pull request #308 from github/hvitved/operation-method-call
Make `{Unary,Binary}Operation` a sub class of `MethodCall`
2021-09-24 12:51:07 +02:00
Tom Hvitved
30d2df53c6 Include MethodCall.getAChild in {Unary,Binary}Operation.getAChild 2021-09-24 12:08:54 +02:00
Tom Hvitved
edfdfb1fa4 Make {Unary,Binary}Operation a sub class of MethodCall 2021-09-23 19:13:55 +02:00
Harry Maclean
88885a222e Model the RestClient HTTP client 2021-09-23 16:32:15 +01:00
Harry Maclean
4cf520c2df Model the Faraday HTTP client 2021-09-23 16:32:15 +01:00
Harry Maclean
ee51298633 Model the Excon HTTP client 2021-09-23 16:32:15 +01:00
Tom Hvitved
ca2ff9a863 Merge pull request #305 from github/hvitved/desugar/array-literals
Desugar array literals to `::Array.[]`
2021-09-23 17:30:34 +02:00
Arthur Baars
40f0112e8a Merge pull request #297 from github/aibaars/alert-suppression
Alert suppression and file classifier query
2021-09-23 15:37:19 +02:00
Harry Maclean
4f9518a9c6 Merge pull request #293 from github/hmac-code-injection
Add query for Code Injection
2021-09-23 13:50:48 +01:00
Tom Hvitved
f347505542 Merge pull request #277 from github/hvitved/flow-summaries
Add support for flow summaries
2021-09-23 14:31:52 +02:00
Harry Maclean
41608ef47b Address review comments 2021-09-23 12:26:54 +01:00
Tom Hvitved
68d41f9f12 Address review comments 2021-09-23 12:39:47 +02:00
Harry Maclean
83705c5787 Merge pull request #306 from github/hmac-outgoing-http
Model outgoing HTTP requests as remote flow sources
2021-09-23 09:34:44 +01:00
Harry Maclean
5826f2c279 Move Net::HTTP modelling into http_clients module
This seems a more convenient place to keep all the HTTP client
modelling.
2021-09-23 09:04:20 +01:00
Harry Maclean
b658bacab3 Simplify Net::HTTP modelling 2021-09-23 09:04:01 +01:00
Harry Maclean
3000587849 Add Net::HTTP request modelling 2021-09-23 09:04:01 +01:00
Harry Maclean
2bdea01c8a Add HTTP::Client concept 2021-09-23 09:04:01 +01:00
Alex Ford
21e31a47d9 Merge pull request #283 from github/file-system-sources
Start modelling some file system access concepts
2021-09-22 16:45:13 +01:00
Alex Ford
b769aa67c2 test for IO.open as a way of creating an IO instance 2021-09-22 16:29:10 +01:00
Alex Ford
0092c0279b Apply suggestions from code review
Co-authored-by: Nick Rolfe <nickrolfe@github.com>
2021-09-22 14:28:15 +01:00
Tom Hvitved
e670fdbb82 Move two predicates in FlowSummaryImplSpecific.qll 2021-09-22 14:12:46 +02:00
Tom Hvitved
a37737d065 Replace string kind with boolean preservesValue 2021-09-22 09:28:55 +02:00
Tom Hvitved
888183f26d Desugar array literals to ::Array.[] 2021-09-21 21:27:29 +02:00
Alex Ford
70c2be8ca3 Files library tests 2021-09-21 19:08:03 +01:00
Alex Ford
05a04f4835 Files.qll library implementation 2021-09-21 19:07:55 +01:00
Alex Ford
6315621b16 use instanceof extensions for some filesystem concepts 2021-09-21 19:02:11 +01:00
Alex Ford
d1f2258d45 revamp weak file permissions query 2021-09-21 19:02:11 +01:00
Alex Ford
25300cb2b4 start modelling some file access concepts 2021-09-21 19:02:11 +01:00
Nick Rolfe
dd31473dff Merge pull request #301 from github/fix_source_archive
Fix filenames in source archives
2021-09-21 11:37:02 +01:00
Jonas Jensen
a055c86c4f C++: change note 2021-09-21 11:58:04 +02:00
Nick Rolfe
d60410e6b8 Use integer comparisons instead of strings when scanning ERB files 2021-09-21 10:50:04 +01:00
Jonas Jensen
237a7d34b8 C++: Exclusion rules for system macros
Unwanted results were reported for our JPL Rule 24 queries. Including
system headers with complex macros could lead to unpredictable alerts
from these rules.
2021-09-21 11:31:13 +02:00
Tom Hvitved
cdc359527a Resolve semantic conflicts after rebase 2021-09-21 11:14:11 +02:00
Tom Hvitved
564c76c41f Address review comments 2021-09-21 11:04:53 +02:00
Tom Hvitved
08dc6d79ef Add support for flow summaries 2021-09-21 11:04:53 +02:00
Nick Rolfe
3201f30098 Update dbscheme stats 2021-09-20 23:13:38 +01:00
Nick Rolfe
e97adff21d Add upgrade script to remove unused tokeninfo columns 2021-09-20 22:42:13 +01:00
Nick Rolfe
6a17dfd228 Remove file column from tokeninfo tables. 2021-09-20 22:42:13 +01:00
Nick Rolfe
6f059638d2 Remove idx column from tokeninfo tables. 2021-09-20 22:42:13 +01:00
Nick Rolfe
143256e673 Fix filenames in source archives 2021-09-20 22:17:45 +01:00
Nick Rolfe
c183e05c49 Merge pull request #300 from github/fix_tests
Fix tests
2021-09-20 16:19:40 +01:00
Nick Rolfe
d27f8a6d24 Add empty subpaths section to expected test output 2021-09-20 15:56:58 +01:00
Tom Hvitved
8aaabe8b1e Merge pull request #299 from github/hvitved/actions-reuse
Add two 'composite' actions for reusing logic
2021-09-20 15:55:28 +02:00
Nick Rolfe
6f7d4fef70 Merge pull request #287 from github/unsafe-deserialization
rb/unsafe-deserialization query
2021-09-20 14:23:30 +01:00
Nick Rolfe
8af12a164a Merge pull request #298 from github/trap_extension
Fix trap extension for source files without extensions
2021-09-20 14:23:01 +01:00
Tom Hvitved
e201dae672 Add two 'composite' actions for reusing logic 2021-09-20 14:52:02 +02:00
Nick Rolfe
c30c7b380d Replace if let with match. 2021-09-20 12:22:55 +01:00
Nick Rolfe
0936c4cd7b Fix trap extension for source files without extensions
We were writing files with names like `Gemfile..trap.gz`. Now fixed to
`Gemfile.trap.gz`.
2021-09-20 12:11:00 +01:00
Tom Hvitved
4bfbf62e13 Merge pull request #296 from github/hvitved/empty-location
Extract a special empty location
2021-09-20 13:05:27 +02:00
Tom Hvitved
1393dc9eb4 Update extractor/src/main.rs
Co-authored-by: Nick Rolfe <nickrolfe@github.com>
2021-09-20 12:50:24 +02:00
Harry Maclean
95e50cedad Add query for Code Injection
This query finds cases where user input flows to an argument to `eval`
or `send`, which can execute arbitrary Ruby code.
2021-09-20 11:35:45 +01:00
Harry Maclean
916b844557 Merge pull request #280 from github/hmac-cli-injection
Add CLI Injection query
2021-09-20 08:54:01 +01:00
Tom Hvitved
b2d0c60a02 Replace hasLocationInfo with getLocation in API::Node 2021-09-20 09:52:26 +02:00
Tom Hvitved
58d06715fc Extract a special empty location 2021-09-20 09:52:26 +02:00
Alex Ford
36289aa9d9 Merge pull request #255 from github/reflected-xss
rb/reflected-xss query
2021-09-17 18:32:48 +01:00
Harry Maclean
739661eb10 Test that KernelMethodCall is specific enough
Calls to `UnknownModule.system`, where `UnknownModule` is a module that
we know nothing about, should not be identified as instances of
`KernelMethodCall`.
2021-09-17 17:02:17 +01:00
Harry Maclean
64a8cedaa7 Generalise the concept of a Kernel method call 2021-09-17 17:02:17 +01:00
Harry Maclean
599dc28ffa Add another test for shell interpretation 2021-09-17 17:02:17 +01:00
Harry Maclean
f8359767bc Exclude non-shell interpreted args
Update the CommandInjection query to only consider sinks where the
argument is interpreted by a shell. If the argument is passed directly
to a subprocess then it's not vulnerable to shell injection.
2021-09-17 17:02:17 +01:00
Harry Maclean
c8e9a592f0 Update CLI injection tests
Cover more cases, like sinks after (but not guarded by) barrier guards.
2021-09-17 17:02:17 +01:00
Harry Maclean
d046fb0591 Separate open3 pipeline methods
These have a slightly different structure than the other open3 methods.
2021-09-17 17:02:17 +01:00
Harry Maclean
174ba25c66 Update SystemCommandExecution to new pattern
The new pattern is to use the new instanceof keyword in the class
definition, instead of constraining the "superclass" via a member field.
2021-09-17 17:02:17 +01:00
Harry Maclean
cbc14ccda9 Make KernelSystemCall more specific
Test that calls to`system` on modules other than `Kernel` are excluded,
such as in this example:

    module Foo
      def self.system(*args); end
    end

    # This is not a call to Kernel.system
    Foo.system("bar")
2021-09-17 17:02:17 +01:00
Harry Maclean
fb23a2e3bf Add SubshellHeredocExecution
This is a form of command execution:

    result = <<`EOF`
    echo foo bar #{baz}
    EOF
2021-09-17 17:02:17 +01:00
Harry Maclean
799ef4e4c9 Add barrier guards for CLI injection 2021-09-17 17:02:17 +01:00
Harry Maclean
4ecc78effc Kernel#system -> Kernel.system 2021-09-17 17:02:17 +01:00
Harry Maclean
8f65d78cb5 Add Shellwords.escape as CLI injection sanitizer 2021-09-17 17:02:17 +01:00
Harry Maclean
fe8fc0697b Add qhelp for CLI Injection query 2021-09-17 17:02:17 +01:00
Harry Maclean
4a0d7c528a Add top-level CLI injection query and tests 2021-09-17 17:02:17 +01:00
Harry Maclean
8440fe2ba9 Add CommandInjection dataflow config 2021-09-17 17:02:17 +01:00
Harry Maclean
a8f0bce1d1 Add SystemCommandExecution concept
A SystemCommandExecution is a method call or builtin that executes a
system command, either directly or via a subshell.
2021-09-17 17:02:17 +01:00
Nick Rolfe
3c05101961 Merge pull request #290 from github/extract_gemfile
Automatically extract Gemfiles
2021-09-17 16:42:30 +01:00
Nick Rolfe
3d23575a38 Merge pull request #292 from github/regexp_slash_az
Don't parse `\A` and `\Z` as `RegExpConstant`
2021-09-17 16:42:13 +01:00
Tom Hvitved
1fd91ab9bd Merge pull request #295 from github/hvitved/remove-numlines
No longer create redundant `numlines` relation
2021-09-16 13:21:20 +02:00
Tom Hvitved
464b50231b DB upgrade script 2021-09-16 12:57:32 +02:00
Tom Hvitved
fd04baa9fe No longer create redundant numlines relation 2021-09-16 11:43:13 +02:00
Alex Ford
e89d485bc0 update test output (subpaths) 2021-09-15 20:51:14 +01:00
Alex Ford
773291e4c3 Put exprNodeReturnedFrom predicate in DataFlowDispatch.qll 2021-09-15 20:50:46 +01:00
Alex Ford
e80faa017c Fix rb/reflected-xss flow from helper method return values 2021-09-15 20:50:46 +01:00
Alex Ford
35da921deb format 2021-09-15 20:50:46 +01:00
Alex Ford
50b0bb8b36 Restrict rb/reflected-xss instance variable taint edges 2021-09-15 20:50:46 +01:00
Alex Ford
5cfefb1027 Add some more test cases for rb/reflected-xss 2021-09-15 20:50:46 +01:00
Alex Ford
6cc82d46f3 Fix LinkToCallArgumentAsSink matching when link_to is passed a block 2021-09-15 20:50:46 +01:00
Alex Ford
200c8f2493 Add some HTMLEscaping implementations for Rails 2021-09-15 20:50:46 +01:00
Alex Ford
2e65f9b80e update some comments referencing view components 2021-09-15 20:50:46 +01:00
Alex Ford
98fd0e1c24 Update ql/src/queries/security/cwe-079/ReflectedXSS.qhelp
Co-authored-by: Nick Rolfe <nickrolfe@github.com>
2021-09-15 20:50:46 +01:00
Alex Ford
0689e6095e make a type more specific 2021-09-15 20:50:46 +01:00
Alex Ford
ed708c1903 Update ql/src/queries/security/cwe-079/ReflectedXSS.qhelp
Co-authored-by: Nick Rolfe <nickrolfe@github.com>
2021-09-15 20:50:46 +01:00
Alex Ford
eed87b3319 Apply suggestions from code review
Co-authored-by: Nick Rolfe <nickrolfe@github.com>
2021-09-15 20:50:46 +01:00
Alex Ford
205b141482 format 2021-09-15 20:50:46 +01:00
Alex Ford
76864a82be remove an incorrect test case 2021-09-15 20:50:46 +01:00
Alex Ford
3445a6a5e7 fix flow steps from controller instance var assignement to view read access 2021-09-15 20:50:46 +01:00
Alex Ford
b993723595 remove spurious ivar -> locals hash mapping (actionview/controller) 2021-09-15 20:50:46 +01:00
Alex Ford
3430a46440 fix some local variable mappings between view and controller 2021-09-15 20:50:46 +01:00
Alex Ford
b264a05288 Update ql/lib/codeql/ruby/security/ReflectedXSSCustomizations.qll
Co-authored-by: Harry Maclean <hmac@github.com>
2021-09-15 20:50:46 +01:00
Alex Ford
dbb239b04e reorder and format rb/reflected-xss qhelp 2021-09-15 20:50:46 +01:00
Alex Ford
d71dd3f6c7 rb/reflected-xss 2021-09-15 20:50:46 +01:00
Tom Hvitved
d3a1d0a62a Merge pull request #294 from github/bump-codeql
Bump `codeql` submodule
2021-09-15 16:24:04 +02:00
Tom Hvitved
9e67382f06 Bump codeql submodule 2021-09-15 14:59:42 +02:00
Nick Rolfe
961674e4a8 Update expected output now we extract the Gemfile 2021-09-14 18:23:57 +01:00
Nick Rolfe
ec13133317 Automatically extract .gemspec and Gemfile files
They are just Ruby code, after all.
2021-09-14 18:23:57 +01:00
Nick Rolfe
ebf23d00d1 Don't parse \A and \Z as RegExpConstant
Fixes some FPs for the ReDoS queries.
2021-09-14 16:49:35 +01:00
Harry Maclean
12723f0f13 Merge pull request #288 from github/hmac-barrier-guard-checks
Make barrier guards more specific
2021-09-14 16:16:20 +01:00
Arthur Baars
e03fe0fcd4 Add ClassifyFiles.ql 2021-09-14 16:30:34 +02:00
Tom Hvitved
f4e2c30d86 Merge pull request #291 from github/hvitved/regexp-multiples
Speedup `RegExp::multiples`
2021-09-14 14:22:20 +02:00
Tom Hvitved
8ac3dc29e0 Speedup RegExp::multiples
Use regexps to perform matching to avoid constructing sub strings.
2021-09-14 13:58:24 +02:00
Harry Maclean
4763312e55 Merge ConditionBlock and BarrierGuard 2021-09-14 11:11:12 +01:00
Arthur Baars
c2ec6407f5 Add AlertSuppression.ql 2021-09-14 11:53:53 +02:00
Harry Maclean
6f32401e5c Add unless x != test to barrier guards
This tests that the following call to `foo bar` is guarded:

    unless bar != "bar"
      foo bar
    end
2021-09-13 11:58:17 +01:00
Harry Maclean
800e18349f Add != to StringConstCompare
This means we treat != comparisons against strings as taint tracking guards:

    if foo != "A"
      foo         # still tainted
    else
      foo         # not tainted, because we know foo == "A"
    end
2021-09-10 16:42:45 +01:00
Harry Maclean
8f36b0d7fe Simplify guard in SQL injection tests
We don't (yet) properly sanitize taint in cases like this

    foo = "A" unless foo == "B"

So for now, use a simpler guard in the SQL injection test.
We can resurrect the old, more idiomatic guard when we can support it.
2021-09-10 16:27:57 +01:00
Harry Maclean
56983565fe Update ReDoS length guard
Changes to barrier guards in a previous commit mean we need to update
this guard to match.
2021-09-10 16:21:17 +01:00
Nick Rolfe
b51e741439 Merge pull request #289 from github/rust_warnings
Fix 'unused borrow that must be used' warnings.
2021-09-09 17:27:05 +01:00
Nick Rolfe
cf72bada3d Fix 'unused borrow that must be used' warnings.
I don't remember seeing this warning before upgrading to Rust 1.55
2021-09-09 17:03:10 +01:00
Nick Rolfe
6dbf6d7e82 Merge pull request #278 from github/aibaars/revert-hotfix
Revert "Use hotfixed version of `codeql/suite-helpers` with workaround for bug in released CLI"
2021-09-09 11:21:20 +01:00
Harry Maclean
b4c29425ea Make barrier guards more specific
Following examples from the other libraries, this change introduces a
member predicate `checks(CfgNode expr, boolean branch)` to
`BarrierGuard`, which holds if the guard validates `expr` for a
particular value of `branch`, which represents the value of the
condition in the guard.

For example, in the following guard...

    if foo == "foo"
      do_something foo
    else
      do_something_else foo
    end

...the variable `foo` is validated when the condition `foo == "foo"` is
true.

We also introduce the concept that a guard "controls" a code block based
on the value of `branch`. In the example above, the "then" branch of the
if statement is controlled when `branch` is true. The else branch is
not controlled because `foo` can take (almost) any value in that branch.

Based on these concepts, we define a guarded node to be a read of a
validated variable in a controlled block.

In the above example, the `foo` in `do_something foo` is guarded, but
the `foo` in `do_something_else foo` is not.
2021-09-09 11:04:52 +01:00
Nick Rolfe
2ddca2c0db Document and test YAML.safe_load 2021-09-08 18:22:31 +01:00
Nick Rolfe
760dbd739d Add test for rb/unsafe-deserialization 2021-09-08 17:49:23 +01:00
Nick Rolfe
9b9fc18605 Add taint step for Base64.decode64 2021-09-08 17:49:23 +01:00
Nick Rolfe
adceb0a2a1 Add query rb/unsafe-deserialization 2021-09-08 17:49:23 +01:00
Nick Rolfe
a62aa2b1b2 Merge pull request #269 from github/polynomial_redos
Polynomial ReDoS query
2021-09-07 18:31:04 +01:00
Nick Rolfe
414362db8d Rename .qll to match our naming scheme for other dataflow queries. 2021-09-07 17:38:08 +01:00
Nick Rolfe
7666d856b7 Merge remote-tracking branch 'origin/main' into polynomial_redos 2021-09-07 17:35:07 +01:00
Nick Rolfe
4d5928ae5a Add @security-severity tag 2021-09-07 12:15:44 +01:00
Nick Rolfe
8fbe5c0adf Merge pull request #261 from github/getPrimaryQlClasses
Implement getPrimaryQlClasses
2021-09-07 12:02:15 +01:00
Tom Hvitved
8ce7fdc59a Merge pull request #284 from github/hvitved/instanceof-test
Use `instanceof` base classes
2021-09-07 13:01:43 +02:00
Nick Rolfe
060060bc0b Merge remote-tracking branch 'origin/main' into getPrimaryQlClasses 2021-09-06 19:34:34 +01:00
Tom Hvitved
3594794875 Use instanceof base classes in range patterns 2021-09-06 16:15:52 +02:00
Tom Hvitved
9b3b9a731f Move instanceof check from charpred in CfgScope 2021-09-06 10:31:16 +02:00
Calum Grant
51d729a086 Merge pull request #282 from github/add-coc
Create CODE_OF_CONDUCT.md
2021-09-03 14:25:44 +01:00
Harry Maclean
36d5fda400 Merge pull request #260 from github/hmac-url-redirect
Add URLRedirect query
2021-09-03 13:36:54 +01:00
Pierre
12c1f43ceb Create CODE_OF_CONDUCT.md
Add COC based on the latest template.
2021-09-03 14:27:04 +02:00
Harry Maclean
87253032e2 Add a query for URL redirect vulnerabilities
This query finds instances of CWE-601: Redirection to Untrusted Site.

The structure is copied from a query of the same name in the Python
library. We add customisations specific to `ActionController`.
2021-09-03 13:17:14 +01:00
Calum Grant
799c0ff252 Merge pull request #281 from github/add-license
Add LICENSE
2021-09-03 13:14:15 +01:00
Pierre
bc85a1b825 Add LICENSE file
Required step for open-sourcing. This uses the same license at `codeql-ruby`.
2021-09-03 13:10:54 +02:00
Nick Rolfe
47e5a8fd09 Add test for polynomial ReDoS query 2021-09-02 17:57:56 +01:00
Nick Rolfe
cbe23661ed Rename exponential ReDoS test directory 2021-09-02 17:57:56 +01:00
Nick Rolfe
d62b41bdf4 Add query for polynomial ReDoS 2021-09-02 17:57:56 +01:00
Alex Ford
86073776b7 Merge pull request #249 from github/erb-lib
Add codeql_ruby.ast.Erb library
2021-09-02 16:26:52 +01:00
Arthur Baars
ab4cc753b0 Revert "Use hotfixed version of codeql/suite-helpers with workaround for bug in released CLI"
This reverts commit 9d7b77496e.
2021-09-02 16:01:51 +02:00
Tom Hvitved
b8ec5d7d31 Merge pull request #276 from github/hvitved/api-graphs-comment-typo
Fix typo in comment
2021-09-02 12:50:25 +02:00
Tom Hvitved
2d0febeb04 Fix typo in comment 2021-09-02 10:24:37 +02:00
Tom Hvitved
c176d344ab Merge pull request #274 from github/hvitved/cfg/may-raise
CFG: Model calls that may raise an exception
2021-09-01 17:42:13 +02:00
Tom Hvitved
6e23a9ae7a Merge pull request #275 from github/hvitved/api-graphs-fix
API graphs: Fix bug for resolvable modules
2021-09-01 17:10:27 +02:00
Tom Hvitved
03e91a22bc API graphs: Performance fixes 2021-09-01 16:57:56 +02:00
Tom Hvitved
ae70af01cd API graphs: Fix bug for resolvable modules 2021-09-01 16:57:52 +02:00
Tom Hvitved
031a73ff0f Add API graph test that exhibits a missing edge 2021-09-01 16:56:09 +02:00
Tom Hvitved
701eab7b74 Merge pull request #273 from github/hvitved/has-name
Add `hasName` predicates
2021-09-01 15:39:39 +02:00
Tom Hvitved
89e6c0e838 CFG: Model calls that may raise an exception
In order to avoid dead `rescue`s, we assume that any call that happens in a
`rescue`/`ensure` context may raise an exception.
2021-09-01 14:07:28 +02:00
Tom Hvitved
4eaa31d800 Add hasName predicates 2021-09-01 13:32:19 +02:00
Alex Ford
41e7ef11e6 add missing pragma back 2021-08-31 21:19:56 +01:00
Alex Ford
d47c8ee9a5 format 2021-08-31 21:04:43 +01:00
Tom Hvitved
2d08b0156a Merge pull request #271 from github/hvitved/cfg/shared
Adopt shared CFG library
2021-08-31 19:41:02 +02:00
Alex Ford
20b851a6e0 improve ErbExecutionDirective definition 2021-08-31 17:49:15 +01:00
Alex Ford
df9e0dfcb2 make strictlyBefore a member predicate on Location 2021-08-31 16:24:38 +01:00
Alex Ford
d84731bcc7 Add a library for working with the ERB AST 2021-08-31 16:24:38 +01:00
Harry Maclean
502ad3f9bd Merge pull request #247 from github/hmac-jump-to-def
Jump-to-definition
2021-08-31 16:00:43 +01:00
Harry Maclean
3490e328e1 codeql_ruby -> codeql.ruby 2021-08-31 15:43:02 +01:00
Harry Maclean
d3f683e573 Minor refactor of constantQualifiedName 2021-08-31 15:42:06 +01:00
Harry Maclean
34f02ee622 Fix constantQualifiedName
Exclude partial results

Co-authored-by: Alex Ford <alexrford@users.noreply.github.com>
2021-08-31 15:42:06 +01:00
Harry Maclean
91d56cd802 Use dataflow to find method call targets
This includes both local and non-local methods, and is also simpler than
the previous definition.
2021-08-31 15:42:06 +01:00
Harry Maclean
cd3192e8f1 Fix ordering for definitionOf
Actually select the lexicographically least location, not the greatest.
2021-08-31 15:42:06 +01:00
Harry Maclean
8901eba978 Include constants in jump-to-def query
The previous version of this query inadvertently excluded constants
which weren't classes or modules. This version includes them, by
introducing a laxer version of `resolveScopeExpr` that doesn't require
the result to be a `TResolved`.
2021-08-31 15:42:06 +01:00
Harry Maclean
155b385981 Simplify LocalVariable constraint in jump-to-def 2021-08-31 15:42:06 +01:00
Harry Maclean
e72f1399cb Include class variables in jump-to-def query 2021-08-31 15:42:06 +01:00
Harry Maclean
e84ebe2b94 Include instance variables in jump-to-def query
By convention, instance variables are considered to be "defined" in the
`#initialize` method of their containing class. If an instance variable
is written to in `#initialize` and then read elsewhere in the program,
we will point from the read to the write. If it is not written to in
`#initialize` then we won't provide any jump-to-definition information
for it.
2021-08-31 15:42:06 +01:00
Harry Maclean
a16cd8967b Ignore synthesised reads for jump-to-definition
We synthesise variables for things like tuple patterns. For example,
this Ruby code:

    a, b = ...

becomes:

    __synth__0 = ...
    a = __synth__0[0]
    b = __synth__0[1]

The `__synth__` variables should be ignored when calculating
jump-to-definition information, since they don't appear in the original
source code.
2021-08-31 15:42:05 +01:00
Harry Maclean
a814010665 Small refactor to constantQualifiedName 2021-08-31 15:42:05 +01:00
Harry Maclean
95e2b8a4a4 Simplify jump-to-def query
The expected output format is a tuple (a, b, k) where `a` and `b` are any
`AstNode` subclass and `k` is a string indicating the kind of
definition (e.g. variable, method, ...).

By ensuring that every value in `DefLoc` is a subclass of `Expr` (itself
a subclass of `AstNode`) we can simplify the query by removing all the
use of `getLocation()`.
2021-08-31 15:42:05 +01:00
Harry Maclean
19e135fb6f Remove redundant imports 2021-08-31 15:42:05 +01:00
Harry Maclean
2fbbabda2d First draft of a jump-to-definition query
TODO: flesh out this message
2021-08-31 15:42:05 +01:00
Nick Rolfe
d1171e08b1 Merge pull request #272 from github/fix_upgrade
Fix typo in db upgrade script
2021-08-31 15:34:55 +01:00
Nick Rolfe
ad66f03f90 Fix typo in db upgrade script 2021-08-31 15:23:16 +01:00
Tom Hvitved
eeb68a88b6 Add make target to run tests locally 2021-08-31 14:22:26 +02:00
Tom Hvitved
4677a0832f Adopt shared CFG library 2021-08-31 13:42:41 +02:00
Tom Hvitved
50158b82c8 Sync shared files 2021-08-31 13:42:25 +02:00
Tom Hvitved
b9745c8e27 Bump codeql submodule 2021-08-31 13:38:52 +02:00
Arthur Baars
60aca018a8 Merge pull request #254 from github/hvitved/drop-files-folders-columns
Drop redundant columns from `files` and `folders` relations
2021-08-31 12:30:05 +02:00
Tom Hvitved
c70407ae8c Update DB stats 2021-08-31 12:19:35 +02:00
Tom Hvitved
652d2a7a72 DB upgrade script 2021-08-31 12:19:35 +02:00
Tom Hvitved
7f03b87142 Drop redundant columns from files and folders relations 2021-08-31 12:16:26 +02:00
Arthur Baars
32253aa868 Merge pull request #266 from github/dbartol/refactor-packs
Refactor Ruby into library and query packs
2021-08-31 12:14:00 +02:00
Dave Bartolomeo
42629b969f Move initial dbscheme 2021-08-26 19:43:06 -04:00
Dave Bartolomeo
593f3b62fe Fix paths in upgrade script check 2021-08-26 19:26:26 -04:00
Dave Bartolomeo
9c03a02965 Update lock file for hotfix 2021-08-26 19:13:48 -04:00
Dave Bartolomeo
2c1620f25e Move missed library file 2021-08-26 18:59:58 -04:00
Dave Bartolomeo
9d7b77496e Use hotfixed version of codeql/suite-helpers with workaround for bug in released CLI 2021-08-26 18:50:04 -04:00
Dave Bartolomeo
11ad664bfb Updated pack versions and lock files 2021-08-26 18:50:04 -04:00
Dave Bartolomeo
eb412fb31e Fix PowerShell version of extractor pack script 2021-08-26 18:50:04 -04:00
Dave Bartolomeo
56332a676d Ignore .codeql output directories 2021-08-26 18:50:04 -04:00
Arthur Baars
ac2c315839 Fix merge conflicts during rebase 2021-08-26 18:48:53 -04:00
Arthur Baars
0afcb9cc86 Workaround for compilation failure 2021-08-26 18:42:06 -04:00
Arthur Baars
817f8747de Fix build 2021-08-26 18:42:02 -04:00
Arthur Baars
17fc6ab72c Refactor into separate library and query packs 2021-08-26 18:40:06 -04:00
Alex Ford
ee6c809281 Merge pull request #262 from github/action-view-1
Start modelling ActionView
2021-08-26 15:22:55 +01:00
Tom Hvitved
348b12c109 Merge pull request #268 from github/hvitved/db-upgrade-pr-check
Add DB upgrade script check
2021-08-26 16:06:06 +02:00
Tom Hvitved
42daf5b6d3 Add DB upgrade script check 2021-08-26 15:55:18 +02:00
Alex Ford
9571e7bccc drop ViewComponent parts from the ActionView library 2021-08-26 14:45:47 +01:00
Alex Ford
a3ae5bcec4 improve ActionControllerHelperMethod doc 2021-08-26 14:12:27 +01:00
Nick Rolfe
4ec30b2a4b Merge pull request #267 from github/erik-krogh/redosUnicode
use toUnicode in ReDoSUtil.qll
2021-08-26 11:08:31 +01:00
Erik Krogh Kristensen
ff27a0c894 use toUnicode in ReDoSUtil.qll 2021-08-26 08:46:51 +00:00
Alex Ford
4a4b2445dc Clean up how we map between Rails actions and default associated template files 2021-08-26 04:57:15 +01:00
Nick Rolfe
ffd80fcc88 Merge pull request #263 from github/bump_ts
Bump tree-sitter versions to pick up parsing fixes
2021-08-25 16:35:23 +01:00
Harry Maclean
4cbd848497 Merge pull request #264 from github/hmac-dependabot
Enable dependabot on the Rust projects
2021-08-25 16:34:29 +01:00
Harry Maclean
0bd7e5914f Enable dependabot on the Rust projects
Add a dependabot.yml file to trigger daily dependabot updates on the
four Rust projects in the codebase:

- `node_types`
- `generator`
- `extractor`
- `autobuilder`
2021-08-25 15:35:31 +01:00
Nick Rolfe
3b0055a7c0 Use published crate for tree-sitter-ruby 0.19 2021-08-25 14:32:01 +01:00
Nick Rolfe
bc06817611 Add ERB comment as regression test for parsing bug 2021-08-25 12:43:33 +01:00
Nick Rolfe
289b59d3b0 Bump tree-sitter versions to pick up parsing fixes
Particularly, in tree-siter-embedded-template
2021-08-25 11:58:56 +01:00
Alex Ford
abc283ee8a remove ErbFile refs 2021-08-24 17:22:35 +01:00
Alex Ford
e403fc77d3 tests 2021-08-24 17:21:22 +01:00
Alex Ford
d628716c42 extend ActionController tests 2021-08-24 17:21:22 +01:00
Alex Ford
41ff10c908 extend modelling of ActionController, and start modelling ActionView 2021-08-24 17:21:22 +01:00
Nick Rolfe
5e783e4798 Implement getPrimaryQlClasses 2021-08-24 14:49:56 +01:00
Nick Rolfe
9c17e00645 Merge pull request #256 from github/syncRedos
sync ReDoSUtil.qll with python/JS
2021-08-23 10:11:16 +01:00
Harry Maclean
a2115f41e8 Merge pull request #259 from github/hmac-print-ast
Don't include desugared nodes in the printed AST
2021-08-18 09:16:36 +01:00
Harry Maclean
e82c21d35d Don't include desugared nodes in the printed AST
The base `PrintAstConfiguration` class already has a predicate for
filtering out desugared nodes - this change just makes use of it in the
query.

This fixes https://github.com/github/codeql-team/issues/408, which was
caused by including nodes representing the desugaring of

    a[b] = c

in the query output. This would result in multiple edges to the same
target node (one from the surface AST and another from the desugared
AST), which the VSCode AST viewer cannot handle.
2021-08-17 15:20:30 +01:00
Arthur Baars
df4fb23f37 Merge pull request #246 from github/aibaars/tweaks
Add an example snippet query
2021-08-17 12:42:02 +02:00
Arthur Baars
9b877dc6e1 Add an example snippet query 2021-08-17 11:29:44 +01:00
Tom Hvitved
50cfd9c318 Merge pull request #257 from github/hvitved/cfg/erb
CFG: Allow `erb` top-level scopes
2021-08-17 11:21:44 +02:00
Arthur Baars
115a13f50c Merge pull request #258 from github/qltest-no-beta
Exclude beta releases of code-cli for qltest job
2021-08-17 11:09:53 +02:00
Alex Ford
8427a6bcee exclude beta releases of code-cli for qltest job 2021-08-17 09:57:52 +01:00
Tom Hvitved
394c27a279 CFG: Allow erb top-level scopes 2021-08-17 10:46:15 +02:00
Erik Krogh Kristensen
5e63b0b132 add RegExpSubPattern.getOperand 2021-08-16 12:14:53 +00:00
Erik Krogh Kristensen
8bd663a7ce sync ReDoSUtil.qll with python/JS 2021-08-16 12:04:22 +00:00
Alex Ford
0f6c464d27 Merge pull request #251 from github/aibaars/test
Add integration test
2021-08-11 16:54:47 +01:00
Arthur Baars
f26f8c1e05 Add integration test 2021-08-11 12:54:30 +02:00
Alex Ford
4d6d6a4016 Merge pull request #236 from github/more-concepts
Port some concepts to Concepts.qll
2021-08-10 12:42:40 +01:00
Calum Grant
e29e61fd3e Merge pull request #250 from github/aibaars-patch-2
Use strict 3 digit semantic version number
2021-08-10 11:41:15 +01:00
Arthur Baars
da464511ec Use strict 3 digit semantic version number 2021-08-10 12:02:54 +02:00
Aditya Sharad
0b64ef2579 Merge pull request #248 from github/hmakholm/pr/windows-autobuilder
attempt to fix Windows autobuilder script
2021-08-09 09:49:17 -07:00
Henning Makholm
d9880075cc attempt to fix Windows autobuilder script 2021-08-09 18:35:45 +02:00
Tom Hvitved
c0049bf161 Merge pull request #229 from github/hvitved/api-graphs/remove-mk-module
API graphs: Remove `MkModule`
2021-08-09 13:10:17 +02:00
Tom Hvitved
ae837d9f7a API graphs: Remove restriction on top-level constants 2021-08-09 12:59:36 +02:00
Arthur Baars
e8f6cb65b8 Merge pull request #245 from github/aibaars/tweaks
Move UseDetect.ql to experimental for now
2021-08-04 16:05:06 +02:00
Arthur Baars
23f423ad66 Merge pull request #242 from github/regex_parsing_fixes
Regex parsing fixes
2021-08-04 16:04:54 +02:00
Arthur Baars
9ca0e81953 Move UseDetect to experimental for now 2021-08-04 15:52:48 +02:00
Arthur Baars
8ded688b72 Add queries.xml for legacy tooling 2021-08-04 14:34:20 +02:00
Tom Hvitved
0eaeb3b5a6 Rename moduleImport to getTopLevelMember 2021-08-04 10:57:57 +02:00
Tom Hvitved
8451286754 API graphs: Remove MkModule 2021-08-04 10:28:30 +02:00
Nick Rolfe
78b64dad71 Merge pull request #244 from github/script_cleanup
Tidy up shell scripts
2021-08-03 11:27:32 +01:00
Nick Rolfe
52ecc2c152 fix path to create-extractor-pack.sh 2021-08-03 11:14:23 +01:00
Nick Rolfe
f2af68f8cf Clean up script file locations 2021-08-02 18:21:50 +01:00
Arthur Baars
2c8b1fa6da Merge pull request #231 from github/aibaars/makefile
Add makefile
2021-08-02 18:31:16 +02:00
Arthur Baars
38f82ffc3c Update Makefile
Co-authored-by: Nick Rolfe <nickrolfe@github.com>
2021-08-02 18:01:59 +02:00
Alex Ford
403dee279d add Node#getALocalSource predicate 2021-08-02 15:56:36 +01:00
Alex Ford
56139ccf93 port some concepts to Concepts.qll 2021-08-02 15:56:36 +01:00
Arthur Baars
58a6f5a783 Address comments 2021-08-02 16:12:50 +02:00
Arthur Baars
730b6d8e6c Add makefile 2021-08-02 16:12:50 +02:00
Arthur Baars
2f491a1924 Merge pull request #230 from github/redos-enable-tounicode
enable unicode parsing in the ReDoS query
2021-08-02 10:42:09 +02:00
Erik Krogh Kristensen
632ad518f0 enable unicode parsing in the ruby ReDoS query 2021-08-02 07:13:41 +00:00
Arthur Baars
d986bea317 Merge pull request #238 from github/aibaars/extract-erb
Extract ERB tags
2021-07-29 19:21:32 +02:00
Arthur Baars
00a0b93172 Add erb file 2021-07-29 19:09:56 +02:00
Nick Rolfe
4007e85991 Incorporate changes from Python PR 2021-07-29 17:25:39 +01:00
Nick Rolfe
3abe047cac Fix parsing of POSIX bracket expressions.
The docs are misleading. [[:alpha:]] is actually a character class
*containing* a POSIX bracket expression, and that means you can have
expressions like [[:alpha:][:digit:]_?!]
2021-07-29 17:24:51 +01:00
Nick Rolfe
5d336d8e1d Make some predicates/classes/imports private 2021-07-29 17:17:11 +01:00
Nick Rolfe
e757d2e654 Merge pull request #241 from github/fix_yml
Fix invalid file-type identifier
2021-07-29 12:05:10 +01:00
Arthur Baars
c568162256 Use a single TrapWriter
The output of two distinct TrapWriters should not be written to the
same TRAP file because this causes name clashes between TRAP labels.
2021-07-29 12:50:27 +02:00
Nick Rolfe
4aacdafb38 Fix invalid file-type identifier
Upper-case characters are not allowed.
2021-07-29 11:49:22 +01:00
Arthur Baars
cc1bdf1fc3 Add charpred to RubyFile class 2021-07-29 11:48:35 +02:00
Arthur Baars
fcf2d4cbd2 Apply suggestions from code review
Co-authored-by: Nick Rolfe <nickrolfe@github.com>
2021-07-29 09:02:57 +02:00
Arthur Baars
1d245b8d2e Merge pull request #237 from github/aibaars/rules-sarif
Build/Release: create rules.sarif file
2021-07-27 18:49:50 +02:00
Arthur Baars
dacd3f3d19 Update dbscheme stats 2021-07-27 18:43:51 +02:00
Arthur Baars
4d18ec226a Fix dataset_measure.yml 2021-07-27 18:43:51 +02:00
Arthur Baars
38eb6c112f Add ERB extraction 2021-07-27 18:43:51 +02:00
Arthur Baars
768a751271 Add upgrade script 2021-07-27 18:43:51 +02:00
Arthur Baars
866ff7b1f6 Replace Generated module with Ruby 2021-07-27 18:43:44 +02:00
Arthur Baars
02bf895a4a Update dbscheme type references 2021-07-27 18:42:21 +02:00
Arthur Baars
2e10f8f054 Prefix dbscheme entries with language name 2021-07-27 18:17:19 +02:00
Arthur Baars
fe868e4c05 Ruby-Generator: add --dbscheme and --library flags 2021-07-27 18:17:19 +02:00
Arthur Baars
fc8f5919f3 Remove Ruby specific parts from FileSystem.qll 2021-07-27 18:17:15 +02:00
Arthur Baars
58c93bfdca Build/Release: create rules.sarif file 2021-07-27 12:29:27 +02:00
Arthur Baars
3790611ca1 Merge pull request #233 from github/tausbn/bump-typetrackingnode-changes
Bump `codeql` submodule
2021-07-20 13:24:30 +02:00
Nick Rolfe
8d21f95ffc Merge pull request #235 from github/comment_fix
Move comment so it's not treated as part of the precision metadata
2021-07-19 12:39:13 +01:00
Nick Rolfe
ce35d74447 Move comment so it's not treated as part of the precision metadata 2021-07-19 12:29:16 +01:00
Calum Grant
8d71d09b94 Merge pull request #234 from github/calumgrant/security-severities
Add security-severity metadata
2021-07-16 15:40:03 +01:00
Calum Grant
46a03795c2 Add security-severity metadata 2021-07-16 14:05:54 +01:00
Taus
258f85d6d0 Add defaultImplicitTaintRead 2021-07-15 15:52:59 +00:00
Taus
dc4d353a01 Bump shared dataflow library 2021-07-15 15:08:59 +00:00
Taus
ec645725f0 Bump codeql submodule
Syncs up the shared type tracking implementation with Python.
2021-07-15 14:35:33 +00:00
Tom Hvitved
42c06bfde4 Merge pull request #226 from github/hvitved/const-flow
Data flow through constants
2021-07-14 13:21:07 +02:00
Tom Hvitved
9463927409 Address review comments 2021-07-14 11:05:55 +02:00
Arthur Baars
64a55ba6cf Merge pull request #232 from github/regexp_test_order
Stabilise node ordering for regexp parsing test
2021-07-13 17:36:21 +02:00
Nick Rolfe
1fe5162b67 Stabilise node ordering for regexp parsing test 2021-07-13 16:18:21 +01:00
Tom Hvitved
23447e6d58 Reduce size of lookupMethodOrConst 2021-07-02 14:02:26 +02:00
Tom Hvitved
bf696df788 Data flow through constants 2021-07-02 14:02:26 +02:00
Tom Hvitved
3b6e5881c8 Update constants.rb test 2021-07-02 14:02:26 +02:00
Arthur Baars
0eae89a41b Merge pull request #228 from github/qhelp
QHelp preview
2021-07-02 14:00:51 +02:00
Arthur Baars
5afd3c7846 Merge pull request #213 from github/aibaars/api-graphs2
First version of ApiGraphs
2021-07-02 13:58:00 +02:00
Arthur Baars
48ad0aa1ee Escape file paths 2021-07-02 13:51:22 +02:00
Arthur Baars
b2ba8e664c Handle .inc.qhelp files 2021-07-02 13:32:43 +02:00
Arthur Baars
20570eb1d1 QHelp preview 2021-07-02 13:10:51 +02:00
Tom Hvitved
703e9e726d Merge pull request #225 from github/hvitved/private-methods
Model private methods and "main objects"
2021-07-02 11:02:41 +02:00
Tom Hvitved
330b33638e Address review comments 2021-07-02 10:41:10 +02:00
Tom Hvitved
52529d590b Model private methods and "main objects" 2021-07-02 10:41:06 +02:00
Tom Hvitved
9de4ed4d4d Add tests for private methods 2021-07-02 10:39:49 +02:00
Tom Hvitved
8de1eedb41 Merge pull request #227 from github/hvitved/expose-call-graph 2021-07-01 18:29:14 +02:00
Tom Hvitved
c3cff3e113 Expose call graph through Call::getATarget() 2021-07-01 16:40:45 +02:00
Nick Rolfe
d99b5510e5 Merge pull request #219 from github/regex
Add regexp parser and exponential ReDoS query
2021-06-30 17:23:29 +01:00
Alex Ford
7cc6b3a7b0 Merge pull request #224 from github/sqli-override-fp
rb/sql-injection: fix FPs stemming from not accounting for overridden methods
2021-06-30 17:20:14 +01:00
Alex Ford
3f76075fe6 improve some rails framework tests 2021-06-29 13:56:28 +01:00
Alex Ford
31cbf818ab fix rb/sql-injection FPs due to not accounting for overridden ActiveRecord methods 2021-06-29 13:54:15 +01:00
Nick Rolfe
97ae9ed181 Add more qldoc comments from Python version
Co-authored-by: Alex Ford <alexrford@users.noreply.github.com>
2021-06-29 11:22:47 +01:00
Tom Hvitved
20f239fd0a Improve performance of seqChild/4
Gets rid of the following bad join-order
```
[2021-06-29 09:40:44] (5s) Starting to evaluate predicate RegExpTreeView::seqChild#fffff#reorder_0_1_2_4_3/5@i2#fe59dz (iteration 2)
[2021-06-29 09:46:34] (354s) Tuple counts for RegExpTreeView::seqChild#fffff#reorder_0_1_2_4_3/5@i2#fe59dz:
                      222277     ~0%     {5} r1 = SCAN RegExpTreeView::RegExpTerm#ffff#prev_delta OUTPUT In.1 're', In.2 'start', In.3, 0, In.0 'result'
                      207749     ~3%     {4} r2 = JOIN r1 WITH ParseRegExp::RegExp::item_dispred#fff ON FIRST 3 OUTPUT Lhs.0 're', Lhs.1 'start', 0, Lhs.4 'result'
                      11636      ~2%     {5} r3 = JOIN r2 WITH ParseRegExp::RegExp::sequence_dispred#fff ON FIRST 2 OUTPUT 0, Lhs.3 'result', Lhs.0 're', Lhs.1 'start', Rhs.2 'end'

                      222277     ~0%     {4} r4 = SCAN RegExpTreeView::RegExpTerm#ffff#prev_delta OUTPUT In.1 're', In.2 'start', In.3, In.0 'result'
                      207749     ~0%     {3} r5 = JOIN r4 WITH ParseRegExp::RegExp::item_dispred#fff ON FIRST 3 OUTPUT Lhs.1, Lhs.3 'result', Lhs.0 're'
                      902017671  ~2%     {3} r6 = JOIN r5 WITH RegExpTreeView::RegExpTerm#ffff#reorder_3_0_1_2#prev ON FIRST 1 OUTPUT Lhs.2 're', Lhs.1 'result', Rhs.1
                      1193975963 ~2%     {5} r7 = JOIN r6 WITH ParseRegExp::RegExp::sequence_dispred#fff ON FIRST 1 OUTPUT Lhs.0 're', Rhs.1, Rhs.2 'end', Lhs.2, Lhs.1 'result'
                      0          ~0%     {6} r8 = JOIN r7 WITH RegExpTreeView::seqChild#fffff#reorder_0_1_2_4_3#prev ON FIRST 4 OUTPUT Lhs.4 'result', Lhs.0 're', Lhs.1 'start', Lhs.2 'end', Rhs.4 're', (1 + Rhs.4 're')
                      0          ~0%     {6} r9 = SELECT r8 ON In.5 'i' > 0
                      0          ~0%     {5} r10 = SCAN r9 OUTPUT In.5 'i', In.0 'result', In.1 're', In.2 'start', In.3 'end'

                      11636      ~2%     {5} r11 = r3 UNION r10

                      222277     ~3%     {2} r12 = SCAN RegExpTreeView::RegExpTerm#ffff#prev_delta OUTPUT In.3, In.0 'result'
                      961948702  ~4%     {5} r13 = JOIN r12 WITH RegExpTreeView::RegExpTerm#ffff#reorder_2_1_0_3#prev ON FIRST 1 OUTPUT Rhs.1 're', Lhs.0, Rhs.3, Lhs.1, Rhs.2 'result'
                      902017671  ~0%     {3} r14 = JOIN r13 WITH ParseRegExp::RegExp::item_dispred#fff ON FIRST 3 OUTPUT Lhs.0 're', Lhs.3, Lhs.4 'result'
                      1193975963 ~2%     {5} r15 = JOIN r14 WITH ParseRegExp::RegExp::sequence_dispred#fff ON FIRST 1 OUTPUT Lhs.0 're', Rhs.1, Rhs.2 'end', Lhs.1, Lhs.2 'result'
                      0          ~0%     {6} r16 = JOIN r15 WITH RegExpTreeView::seqChild#fffff#reorder_0_1_2_4_3#prev ON FIRST 4 OUTPUT Lhs.4 'result', Lhs.0 're', Lhs.1 'start', Lhs.2 'end', Rhs.4 're', (1 + Rhs.4 're')
                      0          ~0%     {6} r17 = SELECT r16 ON In.5 'i' > 0
                      0          ~0%     {5} r18 = SCAN r17 OUTPUT In.5 'i', In.0 'result', In.1 're', In.2 'start', In.3 'end'

                      0          ~0%     {5} r19 = SCAN RegExpTreeView::seqChild#fffff#reorder_0_1_2_4_3#prev_delta OUTPUT In.0 're', In.1 'start', In.2 'end', In.4, In.3
                      0          ~0%     {6} r20 = JOIN r19 WITH ParseRegExp::RegExp::sequence_dispred#fff ON FIRST 3 OUTPUT Lhs.0 're', Lhs.1 'start', Lhs.2 'end', Lhs.3, Lhs.4, (1 + Lhs.3)
                      0          ~0%     {6} r21 = SELECT r20 ON In.5 'i' > 0
                      0          ~0%     {5} r22 = SCAN r21 OUTPUT In.4, In.0 're', In.1 'start', In.2 'end', In.5 'i'
                      0          ~0%     {5} r23 = JOIN r22 WITH RegExpTreeView::RegExpTerm#ffff#prev ON FIRST 1 OUTPUT Rhs.3, Lhs.1 're', Lhs.2 'start', Lhs.3 'end', Lhs.4 'i'
                      0          ~0%     {7} r24 = JOIN r23 WITH RegExpTreeView::RegExpTerm#ffff#reorder_2_1_0_3#prev ON FIRST 2 OUTPUT Lhs.1 're', Lhs.0, Rhs.3, Lhs.2 'start', Lhs.3 'end', Lhs.4 'i', Rhs.2 'result'
                      0          ~0%     {5} r25 = JOIN r24 WITH ParseRegExp::RegExp::item_dispred#fff ON FIRST 3 OUTPUT Lhs.5 'i', Lhs.6 'result', Lhs.0 're', Lhs.3 'start', Lhs.4 'end'

                      0          ~0%     {5} r26 = r18 UNION r25
                      11636      ~2%     {5} r27 = r11 UNION r26
                      11636      ~2%     {5} r28 = r27 AND NOT RegExpTreeView::seqChild#fffff#reorder_0_1_2_4_3#prev(Lhs.2 're', Lhs.3 'start', Lhs.4 'end', Lhs.1 'result', Lhs.0 'i')
                      11636      ~0%     {5} r29 = SCAN r28 OUTPUT In.2 're', In.3 'start', In.4 'end', In.1 'result', In.0 'i'
                                         return r29
```
2021-06-29 09:57:23 +02:00
Nick Rolfe
ba7021086b Merge remote-tracking branch 'origin/main' into regex 2021-06-25 15:00:26 +01:00
Nick Rolfe
bee94757dd Add query test for ReDoS.ql, ported from JS 2021-06-25 12:51:35 +01:00
Nick Rolfe
6142029fdc Recognise \t as not escaping t 2021-06-25 12:46:25 +01:00
Nick Rolfe
a77e7761fd Make \h and \H character class escapes 2021-06-25 12:27:39 +01:00
Nick Rolfe
a5dff79e51 Fix locations of regexp nodes in AST viewer 2021-06-25 12:00:38 +01:00
Arthur Baars
fa5e7cb9cc Merge pull request #223 from github/aibaars/mkdir-p
Create parent folders when copying qhelp and sample files
2021-06-25 11:29:27 +02:00
Alex Ford
5179e3e5d6 Merge pull request #209 from github/query-sql-injection 2021-06-25 09:59:50 +01:00
Arthur Baars
efde1f86d9 Fix test case 2021-06-25 10:59:10 +02:00
Arthur Baars
0d77f49f7c Create parent folders 2021-06-24 22:07:58 +02:00
Alex Ford
2a7d8bbc0a Apply suggestions from code review
Co-authored-by: Tom Hvitved <hvitved@github.com>
2021-06-24 19:43:35 +01:00
Arthur Baars
d4666ab099 Merge pull request #222 from github/aibaars/file-filters
Add support for LGTM_INDEX_FILTERS environment variable
2021-06-24 20:09:08 +02:00
Arthur Baars
e3b4e0a9a3 Add missing use statement 2021-06-24 20:00:41 +02:00
Arthur Baars
f92989350a Update autobuilder/src/main.rs
Co-authored-by: Nick Rolfe <nickrolfe@github.com>
2021-06-24 19:50:32 +02:00
Nick Rolfe
9ec503a3a5 Merge remote-tracking branch 'origin/main' into regex 2021-06-24 18:16:13 +01:00
Alex Ford
a45366e426 remove unnecessary ExprNodes prefix 2021-06-24 18:12:26 +01:00
Alex Ford
b27891b14e update ActiveRecord test output 2021-06-24 18:12:26 +01:00
Alex Ford
1f5a5181b9 StringInterpolationComponentCfgNode extends ExprNodes::StmtSequenceCfgNode 2021-06-24 18:12:26 +01:00
Alex Ford
a4a8f17a54 Update ql/src/codeql_ruby/dataflow/BarrierGuards.qll
Co-authored-by: Tom Hvitved <hvitved@github.com>
2021-06-24 18:12:26 +01:00
Alex Ford
9883a9b606 update SqlInjection tests 2021-06-24 18:12:26 +01:00
Alex Ford
d62f4f5bd4 Address review comments 2021-06-24 18:12:26 +01:00
Alex Ford
bc5a1b86ff Fix handling of arrays passed to ActiveRecord SQL methods 2021-06-24 18:12:26 +01:00
Alex Ford
fc8db88b66 Apply suggestions from code review
Co-authored-by: Tom Hvitved <hvitved@github.com>
2021-06-24 18:12:25 +01:00
Alex Ford
7415503772 update ActiveRecord test output 2021-06-24 18:12:25 +01:00
Alex Ford
12e4c9ee90 update SqlInjection tests 2021-06-24 18:12:25 +01:00
Alex Ford
734fe01867 Support named :conditions parameter to some SQL executing ActiveRecord calls 2021-06-24 18:12:25 +01:00
Alex Ford
91bde8d85d Support ActiveRecord SQL executing calls where there is a self receiver (implicit or explicit) 2021-06-24 18:12:25 +01:00
Alex Ford
5386c776b3 Implement rb/sql-injection 2021-06-24 18:12:25 +01:00
Alex Ford
957b29b5af Add more defaultAdditionalTaintSteps 2021-06-24 18:12:25 +01:00
Alex Ford
6e5665da8c Make ActiveRecord model flag more potentially dangerous SQL executions 2021-06-24 18:12:25 +01:00
Alex Ford
8761873cd1 Implement two common barrier guards 2021-06-24 18:12:25 +01:00
Alex Ford
98313d0a56 Convenience classes for wrapping some Exprs as ExprCfgNodes 2021-06-24 18:12:25 +01:00
Alex Ford
ad1d8420f3 Make BarrierGuard abstract 2021-06-24 18:12:25 +01:00
Alex Ford
adf32e973a Create Frameworks.qll to act as a container for all framework models 2021-06-24 18:12:25 +01:00
Nick Rolfe
17a59ef824 Add basic test for regex parsing 2021-06-24 18:06:08 +01:00
Arthur Baars
f69c5dc19b Merge pull request #221 from github/package-depend-on-compile-queries
make the package job depend on compile-queries
2021-06-24 19:03:44 +02:00
Arthur Baars
22990a938d Add support for LGTM_INDEX_FILTERS environment variable
* re-implement autobuilder script in Rust
* add additional --include/--exclude flags based on LGTM_INDEX_FILTERS
  environment variable
2021-06-24 18:45:31 +02:00
Nick Rolfe
51b0ffdaf8 Fix printAst to support adding edges in AstDesugar test 2021-06-24 17:14:23 +01:00
Nick Rolfe
f7e89f47fd Comment out temporarily-unused predicates 2021-06-24 17:06:41 +01:00
Alex Ford
58e9b69ea4 make the package job depend on compile-queries 2021-06-24 16:52:22 +01:00
Nick Rolfe
a6dd2fa0a1 Split ReDoS query into .ql and .qll, and add .qhelp 2021-06-24 16:32:45 +01:00
Arthur Baars
7574d1cad7 Merge pull request #220 from github/aibaars/update-build-yml
Update build.yml
2021-06-24 16:38:26 +02:00
Arthur Baars
be1d4c3d2c Address comment 2021-06-24 16:31:24 +02:00
Arthur Baars
ade36691b6 Remove unnecessary qualifier 2021-06-24 16:13:29 +02:00
Arthur Baars
dfc96de4cc Update build.yml 2021-06-24 16:09:45 +02:00
Arthur Baars
95399b2d0a Refactor ApiGraphs.qll 2021-06-24 15:58:02 +02:00
Arthur Baars
4f96834711 Add ConstantAccessCfgNode 2021-06-24 15:57:48 +02:00
Arthur Baars
6bed50a86b Rename predicate with snake cased name 2021-06-24 11:59:13 +02:00
Arthur Baars
b2be1c3b3d Update ql/src/codeql_ruby/ApiGraphs.qll
Co-authored-by: Tom Hvitved <hvitved@github.com>
2021-06-23 20:40:22 +02:00
Tom Hvitved
9438885776 Merge pull request #216 from github/hvitved/synthesis-location
AST synthesis: Move location information into a separate predicate
2021-06-23 16:50:17 +02:00
Nick Rolfe
c784e37089 Add regexp parser and exponential ReDoS query 2021-06-23 15:29:49 +01:00
Alex Ford
e5f0206c6d Merge pull request #208 from github/action-controller-1
Model accesses to `ActionController` parameters via `params` method
2021-06-23 14:21:55 +01:00
Alex Ford
0238c19085 remove TODO 2021-06-23 14:11:38 +01:00
Alex Ford
5941eb2be4 model some ActionController user input sources (params) 2021-06-23 14:11:38 +01:00
Alex Ford
9227f3a0c3 Add RemoteFlowSources.qll 2021-06-23 14:11:38 +01:00
Alex Ford
5163514d43 Merge pull request #218 from github/build-yml-debug
Fix `compile-queries` job
2021-06-23 14:04:33 +01:00
Alex Ford
8e1f2e6237 try fixing build.yml 2021-06-23 13:41:51 +01:00
Tom Hvitved
1dde5b8ef9 AST synthesis: Move location information into a separate predicate 2021-06-23 08:46:07 +02:00
Arthur Baars
f18e5030e0 Address comments by @tausbn 2021-06-22 17:25:34 +02:00
Alex Ford
dbf1805c8b Merge pull request #196 from github/active-record-1
Start modelling some potential SQL fragment sinks in ActiveRecord
2021-06-22 16:05:26 +01:00
Arthur Baars
bedd790d33 Merge pull request #217 from github/aibaars-patch-2
Remove ad-hoc entries from query suite
2021-06-22 15:48:22 +02:00
Arthur Baars
f7eee915da Remove ad-hoc queries 2021-06-22 15:35:30 +02:00
Arthur Baars
cdfe74959f Remove methodName field 2021-06-22 10:32:44 +02:00
Arthur Baars
7c3c1db462 Use ApiGraphs in WeakFilePermissions query 2021-06-22 10:25:56 +02:00
Arthur Baars
65d9327951 Add CallNode class 2021-06-22 10:25:56 +02:00
Arthur Baars
57d8ba649f Use flowsTo 2021-06-21 19:37:41 +02:00
Arthur Baars
d2e2901128 First version of ApiGraphs 2021-06-21 19:37:41 +02:00
Arthur Baars
f0c83288a7 Add test case for ApiGraph 2021-06-21 19:37:41 +02:00
Arthur Baars
4fa093048c Add inline expectations test framework 2021-06-21 19:37:41 +02:00
Arthur Baars
33c5312842 Merge pull request #215 from github/bump-codeql
Bump `codeql` submodule
2021-06-21 16:18:04 +02:00
Tom Hvitved
992d8faa06 Bump codeql submodule 2021-06-21 16:06:45 +02:00
Tom Hvitved
abe5e3d953 Merge pull request #210 from github/hvitved/dataflow/consistency
Data flow: Add consistency queries
2021-06-21 14:42:55 +02:00
Nick Rolfe
35eb4a3af4 Merge pull request #214 from github/regexp_naming
Use RegExp prefix instead of Regex, for consistency with other languages.
2021-06-21 11:06:19 +01:00
Tom Hvitved
b820f3f20d Merge pull request #212 from github/hvitved/ssa/assigns-pred
Add `Ssa::WriteDefinition::assigns/1` predicate
2021-06-21 10:46:48 +02:00
Nick Rolfe
65aa97c07c Use RegExp prefix instead of Regex, for consistency with other languages. 2021-06-18 15:56:19 +01:00
Tom Hvitved
7cc02e6d00 Add Ssa::WriteDefinition::assigns/1 predicate 2021-06-18 10:42:32 +02:00
Nick Rolfe
78db1bf045 Merge pull request #211 from github/smaller_trap
Tweaks to reduce size of TRAP output
2021-06-17 17:09:14 +01:00
Nick Rolfe
ab72b4e9e7 Use hexadecimal encoding for TRAP labels 2021-06-17 16:16:32 +01:00
Nick Rolfe
ed93233917 Remove unnecessary spaces in TRAP output 2021-06-17 16:16:06 +01:00
Alex Ford
7439ab5635 remove recvCls field from ActiveRecordModelClassMethodCall 2021-06-17 14:42:42 +01:00
Alex Ford
214532516b try to avoid a future merge conflict 2021-06-17 14:41:51 +01:00
Alex Ford
762656ee60 Add QLDoc to ActiveRecord.qll 2021-06-17 14:41:51 +01:00
Alex Ford
12a0af1d28 Tidy up PotentiallyUnsafeSqlExecutingMethodCall characteristic predicate
Co-authored-by: Nick Rolfe <nickrolfe@github.com>
2021-06-17 14:39:40 +01:00
Tom Hvitved
41ed9f3e1b Data flow: Fix inconsistencies 2021-06-17 10:48:32 +02:00
Tom Hvitved
00e544189e Data flow: Add consistency queries 2021-06-17 10:26:56 +02:00
Tom Hvitved
ad54f2e1f4 Bump codeql submodule 2021-06-17 10:24:19 +02:00
Tom Hvitved
872c7edfc8 Merge pull request #207 from github/bump-codeql
Bump `codeql` submodule
2021-06-16 12:33:40 +02:00
Tom Hvitved
84d79ccae9 Bump codeql submodule 2021-06-16 11:55:38 +02:00
Alex Ford
bf43a77df5 Include some more types of expressions as possible active record SQL sink arguments 2021-06-15 12:41:42 +01:00
Alex Ford
ea21c591af remove accidentally unbound variable 2021-06-15 11:39:48 +01:00
Alex Ford
c1b9952517 account for chained method calls when constructing ActiveRecord SQL queries 2021-06-15 11:39:48 +01:00
Alex Ford
f8a77b9854 format QL 2021-06-15 11:39:48 +01:00
Alex Ford
57c04266e3 rename SqlExecutingMethodCall as PotentiallyUnsafeSqlExecutingMethodCall 2021-06-15 11:39:48 +01:00
Alex Ford
2d4bb61789 limit SqlExecutingMethodCall to those that are called with a StringlikeLiteral argument 2021-06-15 11:39:48 +01:00
Alex Ford
2c15b60998 add ActiveRecord find_by_sql as an SQL executing method call 2021-06-15 11:39:48 +01:00
Alex Ford
c641d12259 add shell ActiveRecord library tests 2021-06-15 11:39:48 +01:00
Alex Ford
5b7df8578a cleanup ActiveRecord.qll 2021-06-15 11:39:48 +01:00
Alex Ford
7488d072d8 Model some SQL fragment sinks in ActiveRecord model classes 2021-06-15 11:39:48 +01:00
Alex Ford
743deee9ce add a class to represent ActiveRecord models 2021-06-15 11:39:48 +01:00
Alex Ford
7d3eaf40ff add base SqlExecution concepts 2021-06-15 11:39:48 +01:00
Tom Hvitved
3a37e321d5 Merge pull request #205 from github/hvitved/taint-tracking
Initial taint-tracking library
2021-06-15 09:30:59 +02:00
Tom Hvitved
5a9521372b Merge pull request #206 from github/tausbn/fix-identical-files 2021-06-15 07:31:07 +02:00
Taus
2bbcbb2200 Bump submodule pointer 2021-06-14 19:04:22 +00:00
Tom Hvitved
302b485f4c Merge pull request #204 from github/hvitved/cfg-nodes-perf
Improve performance of `ExprChildMapping::reachesBasicBlock()`
2021-06-14 20:14:17 +02:00
Taus
068b980517 Update identical-files.json
As of https://github.com/github/codeql/pull/6063 we have now started using the shared type tracking library in Python as well. 🎉
2021-06-14 19:01:24 +02:00
Tom Hvitved
8aa337ab01 Initial taint-tracking library 2021-06-14 14:19:34 +02:00
Tom Hvitved
b154c936c3 Improve performance of ExprChildMapping::reachesBasicBlock()
Since all expressions are now post-order, the logic of `reachesBasicBlock` can
be simplified, and performance can be improved as well.
2021-06-14 11:58:24 +02:00
Arthur Baars
88fb3c7097 Merge pull request #203 from github/aibaars/pack-qhelp-samples
Query pack: include .rb and .erb sample files from queries directory
2021-06-11 13:50:17 +02:00
Arthur Baars
909e6d5a62 Query pack: include .rb and .erb sample files from queries directory
These are required by the qhelp files.
2021-06-11 13:42:43 +02:00
Arthur Baars
78a6ed43c3 Merge pull request #202 from github/aibaars-patch-2
HardCodedCredentials: fix query metadata comment
2021-06-11 12:05:44 +02:00
Arthur Baars
661d6e8e38 HardCodedCredentials: fix query metadata comment 2021-06-11 11:59:46 +02:00
Tom Hvitved
8860b8adf0 Merge pull request #198 from github/hvitved/desugar-compound-assignment 2021-06-10 19:39:54 +02:00
Alex Ford
f74dff560b Merge pull request #187 from github/hardcoded-credentials
Add rb/hardcoded-credentials query
2021-06-10 16:12:32 +01:00
Alex Ford
8839d4c584 limit additional flow steps in rb/hardcoded-credentials to string concatenation 2021-06-10 14:59:28 +01:00
Alex Ford
fe45dadd55 set precision to high for rb/hardcoded-credentials 2021-06-10 14:52:26 +01:00
Alex Ford
e26afe91b5 move rb/hardcoded-credential alert location to the source 2021-06-07 14:53:04 +01:00
Alex Ford
5d79a8cec0 account for keyword args in rb/hardcoded-credentials and simplify query 2021-06-07 14:49:49 +01:00
Tom Hvitved
962768e7c0 Disambiguate toStrings for nested synthetic local variables 2021-06-04 19:20:11 +02:00
Tom Hvitved
82fbc03889 Merge pull request #200 from github/hvitved/dataflow/call-sensitivity
Data flow: Call-sensitive resolution of lambda/block calls
2021-06-04 16:25:13 +02:00
Alex Ford
ec326bfcb7 Merge pull request #201 from github/perm-file-report-source
Report rb/weak-file-permission alerts at source rather than sink and improve alert message
2021-06-04 14:52:48 +01:00
Alex Ford
8a3ffb6dca add missing toString 2021-06-04 13:25:03 +01:00
Alex Ford
b2d36babc4 report rb/weak-file-permission alerts at source rather than sink and improve alert message 2021-06-04 13:10:18 +01:00
Nick Rolfe
523a0b1f12 Merge pull request #197 from github/upgrade-pack 2021-06-04 13:03:39 +01:00
Nick Rolfe
6203c9019a Remove reference to deleted upgrades qlpack from manifest 2021-06-04 12:15:36 +01:00
Tom Hvitved
61e35ddae1 Data flow: Call-sensitive resolution of lambda/block calls 2021-06-04 12:58:38 +02:00
Tom Hvitved
77146e4e04 Data flow: Reduce caching
These predicates are now cached in the shared implementation.
2021-06-04 12:53:47 +02:00
Tom Hvitved
f9eecfb59f Bump codeql submodule 2021-06-04 12:52:05 +02:00
Tom Hvitved
6678ac0347 Desugar compound assignments 2021-06-04 10:39:06 +02:00
Tom Hvitved
da9adfbab4 Improve performance of desugaring transformations 2021-06-04 10:34:00 +02:00
Tom Hvitved
57eee0368d Add CFG tests for compound assignments 2021-06-04 10:34:00 +02:00
Tom Hvitved
dfcf4c90ab Merge pull request #199 from github/hvitved/splat-expr
Rename `(Hash)SplatArgument` to `(Hash)SplatExpr` and make them `UnaryOperation`s
2021-06-04 10:33:42 +02:00
Tom Hvitved
1007f2aaff Rename (Hash)SplatArgument to (Hash)SplatExpr and make them UnaryOperations 2021-06-04 10:04:06 +02:00
Tom Hvitved
372f8645a9 Add (hash)splat AST tests 2021-06-04 09:53:14 +02:00
Nick Rolfe
8b987757c6 Merge upgrades qlpack into ql/src 2021-06-03 18:28:20 +01:00
Tom Hvitved
2094aa983a Merge pull request #194 from github/hvitved/desugar-child 2021-06-03 18:07:33 +02:00
Arthur Baars
03ef1261d3 Merge pull request #192 from github/aibaars/release-workflow
Build workflow: create release
2021-06-03 16:52:50 +02:00
Tom Hvitved
908e9ff3b5 Include desugared node in AstDesugar.ql 2021-06-03 14:46:32 +02:00
Arthur Baars
63475dc692 Merge pull request #195 from github/escape_field_name
Escape field names with table storage
2021-06-01 14:55:46 +02:00
Nick Rolfe
1388d82f1d Escape field names with table storage 2021-06-01 13:32:13 +01:00
Nick Rolfe
9c199b6c2a Merge pull request #193 from github/tausbn/autogenerate-qldoc
Autogenerate QLDoc for `TreeSitter.qll`
2021-06-01 13:31:32 +01:00
Tom Hvitved
5bafc0c708 Merge pull request #183 from github/hvitved/assign-op-desugar
Desugar setter assignments
2021-06-01 14:00:04 +02:00
Alex Ford
f27dd45e4c run formatter 2021-06-01 12:29:45 +01:00
Alex Ford
907bb9b556 add a comment 2021-06-01 12:22:04 +01:00
Alex Ford
1f931d6f76 rb/hardcoded-credentials: fix bad bracketing 2021-06-01 12:22:04 +01:00
Alex Ford
fdd4f7f616 attempt to use typetracker in rb/hardcoded-credentials 2021-06-01 12:22:04 +01:00
Alex Ford
c530ba5b11 format ql 2021-06-01 12:22:04 +01:00
Alex Ford
f1303e0ced remove WIP files 2021-06-01 12:22:04 +01:00
Alex Ford
10175e1398 remove WIP files 2021-06-01 12:22:04 +01:00
Alex Ford
4fdd072603 WIP: HardcodedCredentials query 2021-06-01 12:22:04 +01:00
Taus
53b7492aa3 Generate QLDoc for getChild 2021-06-01 10:57:39 +00:00
Taus
6cf7a12c8c Undo field name escaping 2021-06-01 10:56:45 +00:00
Taus
d38520dc73 Escape field names correctly
This should make `field('unique', $.whatever)` valid again.
2021-05-31 20:56:29 +00:00
Taus
64090b086c Autogenerate QLDoc for TreeSitter.qll
It's not quite perfect, as there's still some QLDoc missing on the
various `getChild` methods, but it wasn't immediately clear to me how
to get this working (especially since the QLDoc would ideally be
different depending on whether there was a child index or not).

Then again, `getChild` probably has a pretty intuitive meaning...
2021-05-31 20:54:10 +00:00
Tom Hvitved
3ffef634d7 More synthesis refactoring
- Join `TElementReferenceSynth` and `TMethodCallSynth`.
- Move arity and setter information into `MethodCallKind`.
- Add `Synthesis::methodCall` for specifying which method calls need synthesis.
2021-05-31 16:29:41 +02:00
Tom Hvitved
e8841e6482 Simplify getSynthChild 2021-05-27 10:20:31 +02:00
Tom Hvitved
f8b99291a7 Improve desugaring of setter assignments 2021-05-26 18:41:21 +02:00
Arthur Baars
af6f050d06 Merge pull request #189 from github/aibaars/fix-lgtm-suite
Fix LGTM suites
2021-05-26 16:02:14 +02:00
Arthur Baars
3f210865b2 Build workflow: create release 2021-05-26 15:55:34 +02:00
Arthur Baars
ec905e0866 Merge pull request #168 from github/aibaars/typetrack-method
Call graph
2021-05-26 14:19:21 +02:00
Arthur Baars
4dc182d4a4 Merge pull request #191 from github/fixCap
fix snake_casing of camelCased identifiers
2021-05-26 13:39:52 +02:00
Arthur Baars
bacbd5e997 Address comments 2021-05-26 13:35:45 +02:00
Erik Krogh Kristensen
9c1b237e3b fix snake_casing of camelCased identifiers 2021-05-26 11:16:05 +00:00
Arthur Baars
a044f41aad Merge pull request #188 from github/aibaars/qlpack
Build Ruby bundle
2021-05-26 12:18:51 +02:00
Tom Hvitved
abcabeef06 Remove *Real predicates and enable recursive desugaring 2021-05-25 21:27:39 +02:00
Tom Hvitved
3f412e4fad Desugar setter assignment operations 2021-05-25 21:27:39 +02:00
Tom Hvitved
b173cc332a Desugar setter assignments 2021-05-25 21:27:39 +02:00
Tom Hvitved
b812012b71 Add CFG setter assignment test 2021-05-25 21:27:39 +02:00
Tom Hvitved
e85677a040 Adjust locations of synthesized AST nodes 2021-05-25 21:27:34 +02:00
Arthur Baars
aea0c6fc64 Merge pull request #190 from github/aibaars/fix-heredoc-parent
Fix Scope::parentOf for HeredocBody nodes
2021-05-25 11:58:21 +02:00
Arthur Baars
ce23ae33e7 Fix Scope::parentOf for HereDocBody 2021-05-25 11:27:45 +02:00
Arthur Baars
bb62564c9e Add test for heredoc with variables 2021-05-25 11:16:55 +02:00
Arthur Baars
86d57d3e26 Fix LGTM suites 2021-05-25 10:41:07 +02:00
Arthur Baars
73aae5dfd9 Use num_cpus-1 threads by default 2021-05-25 09:28:49 +02:00
Arthur Baars
4f404e9b11 Temporarily include some queries in the code scanning suite
This should be reverted once we have a decent set of default queries.
2021-05-25 09:21:40 +02:00
Arthur Baars
a02cfd27c9 Compile query packs with previous CodeQL versions too 2021-05-24 17:48:49 +02:00
Arthur Baars
78d9191526 Build query pack 2021-05-24 13:27:50 +02:00
Tom Hvitved
423a1b39e1 Improve call graph performance by forcing non-linear joins first 2021-05-20 14:36:56 +02:00
Tom Hvitved
492f41d399 Fix performance 2021-05-20 14:27:13 +02:00
Arthur Baars
0ccca47b01 Dataflow for implicit self argument of methods 2021-05-20 14:27:13 +02:00
Arthur Baars
eb8b2558da Add types of lambdas and methods 2021-05-20 14:27:13 +02:00
Arthur Baars
e787d99cd1 Resolve yield calls to blocks 2021-05-20 14:27:13 +02:00
Arthur Baars
66b2c39985 More tests 2021-05-20 14:27:13 +02:00
Arthur Baars
578b94453d Flow for captured local variables 2021-05-20 14:27:13 +02:00
Arthur Baars
e46755021b Add data flow steps for optional parameter values 2021-05-20 14:27:13 +02:00
Arthur Baars
da88661746 Add SSA flow step for parameters 2021-05-20 14:27:13 +02:00
Arthur Baars
84da0cb2f3 Track type of Classes/Modules and and self in singleton methods 2021-05-20 14:27:13 +02:00
Arthur Baars
f157f1f359 Fix superclass of Class 2021-05-20 14:27:13 +02:00
Arthur Baars
1ba94beb01 Fix types of true/false 2021-05-20 14:27:13 +02:00
Tom Hvitved
f63f5aba15 Fix performance 2021-05-20 14:27:13 +02:00
Arthur Baars
af19cc5fae Add test cases 2021-05-20 14:27:13 +02:00
Arthur Baars
a9806719f9 Toplevel 'self' 2021-05-20 14:27:13 +02:00
Arthur Baars
1a739b2fbf Resolve super calls 2021-05-20 14:27:13 +02:00
Arthur Baars
7f520e7899 Add types of literals 2021-05-20 14:27:13 +02:00
Arthur Baars
4951b7d378 Treat methods defined in a singleton class similar to single methods 2021-05-20 14:27:13 +02:00
Arthur Baars
8815bb7dbe Track calls to singleton methods 2021-05-20 14:27:13 +02:00
Arthur Baars
b13bae6a4e Resolve instance method calls 2021-05-20 14:27:13 +02:00
Arthur Baars
3c80b32ba0 Merge pull request #186 from github/bump-codeql
Bump `codeql` sub module
2021-05-20 14:26:24 +02:00
Tom Hvitved
16d34c7cd4 Sync files 2021-05-20 14:15:54 +02:00
Tom Hvitved
c73e6ff390 Bump codeql sub module 2021-05-20 14:15:33 +02:00
Tom Hvitved
1509584e27 Merge pull request #185 from github/hvitved/resolve-expr-perf
Improve performance of `internal/Module.qll`
2021-05-19 14:53:46 +02:00
Tom Hvitved
6b6aeb10c7 Improve performance of internal/Module.qll 2021-05-19 14:33:52 +02:00
Tom Hvitved
4798a1a008 Merge pull request #184 from github/cfg/singleton-method-abnormal
CFG: Add missing `propagatesAbnormal` overrides
2021-05-19 12:45:59 +02:00
Tom Hvitved
c866f88410 CFG: Add missing propagatesAbnormal overrides 2021-05-18 20:39:46 +02:00
Tom Hvitved
9871698cee Add more CFG tests 2021-05-18 20:39:08 +02:00
Nick Rolfe
b9b6ffe53e Merge pull request #178 from github/cfg_cleanup
Clean up CFG implementation
2021-05-18 10:53:44 +01:00
Nick Rolfe
778de741d0 Merge remote-tracking branch 'origin/main' into cfg_cleanup 2021-05-17 16:26:28 +01:00
Nick Rolfe
f3d831c25e Remove unnecessary superclass prefix 2021-05-17 15:26:53 +01:00
Nick Rolfe
9a2523e2f9 Make EndBlockTree extend StmtSequenceTree 2021-05-17 15:24:20 +01:00
Nick Rolfe
6d395230d4 Make BraceBlockTree extend StmtSequenceTree 2021-05-17 14:54:11 +01:00
Tom Hvitved
ad036f8af1 Merge pull request #179 from github/hvitved/synth-framework-take2
AST synthesis framework (take 2)
2021-05-17 15:36:56 +02:00
Tom Hvitved
25f226e9dc Add comment to getVariableReal 2021-05-17 15:02:40 +02:00
Tom Hvitved
b434d42d05 Rename ParenthesizedExprSynth to StmtSequenceSynth 2021-05-17 13:39:44 +02:00
Alex Ford
ca046c9af5 Merge pull request #182 from github/loc-query-tag 2021-05-14 17:42:21 +01:00
Alex Ford
1ba491a956 add lines-of-code tag to rb/summary/lines-of-code 2021-05-14 17:06:49 +01:00
Alex Ford
3c0f20cec8 Merge pull request #170 from github/weak-file-permissions
Add `rb/overly-permissive-file` query
2021-05-14 17:04:15 +01:00
Arthur Baars
6c382ccd4b Merge pull request #169 from github/aibaars/codespace
Add CodeSpace container
2021-05-14 18:00:51 +02:00
Alex Ford
e9090cec70 Merge pull request #181 from github/loc-description-improvements
LOC summary query improvements
2021-05-14 16:13:42 +01:00
Alex Ford
65b0ce204d restrict rb/summary/lines-of-code to the source root 2021-05-14 16:00:55 +01:00
Alex Ford
71234155b8 improve rb/summary/lines-of-code description 2021-05-14 15:59:07 +01:00
Alex Ford
7ff2ca4ffe improve rb/summary/lines-of-user-code name and description 2021-05-14 15:56:59 +01:00
Alex Ford
6bd2e4e4b7 Merge pull request #175 from github/loc-summary-queries-1
Summary queries for total LOC and user-code LOC
2021-05-14 15:51:45 +01:00
Arthur Baars
66bf13e77a Setup a CodeSpace 2021-05-13 21:03:40 +02:00
Arthur Baars
3547980f5b Update reference to tree-sitter-embedded-template 2021-05-13 21:03:40 +02:00
Arthur Baars
498e760b21 Add consistency queries to codeqlmanifest 2021-05-13 21:03:40 +02:00
Nick Rolfe
a46f45440a Create NamespaceTree to reduce duplication 2021-05-13 17:52:20 +01:00
Nick Rolfe
5e6dddad3e Replace count(getReceiver()) with 1 2021-05-13 16:59:05 +01:00
Alex Ford
11949c6b77 Merge pull request #176 from github/diagnostics-entries
Start writing diagnostics to the DB, and some basic summary/diagnostics queries
2021-05-13 14:31:01 +01:00
Alex Ford
15712df717 update ruby.dbscheme.stats 2021-05-13 13:50:53 +01:00
Alex Ford
dc3c5926f5 add a db upgrade for the diagnostics table 2021-05-13 13:45:02 +01:00
Alex Ford
277a6a020a diagnostics: use debug rather than hidden terminology, and leave gaps for other severities 2021-05-13 13:44:10 +01:00
Alex Ford
b2f2f786ac allow the WeakFilePermissions access predicate to return multiple values 2021-05-13 13:22:14 +01:00
Alex Ford
0d1c4a1290 document that the WeakFilePermissions access predicate should return at most one value 2021-05-13 13:06:45 +01:00
Alex Ford
89be8d8710 Apply suggestions from code review
Co-authored-by: Arthur Baars <aibaars@github.com>
2021-05-13 12:59:16 +01:00
Tom Hvitved
ff06e724b1 AST synthesis framework 2021-05-12 19:58:52 +02:00
Alex Ford
acdbd9859e simplify ExtractionError class defn 2021-05-12 16:45:31 +01:00
Alex Ford
11376bc411 note that severity 3 corresponds to an error diagnostic level 2021-05-12 16:39:51 +01:00
Alex Ford
0dad1a4779 use a case-split for diagnostic severity levels 2021-05-12 16:38:37 +01:00
Tom Hvitved
ea1c7b51ef Add more operator assignment tests 2021-05-12 17:24:11 +02:00
Alex Ford
0016146e11 limit summary queries to files from within the source directory 2021-05-11 21:07:08 +01:00
Alex Ford
49d9bb798c revamp the diagnostics tests 2021-05-11 19:53:00 +01:00
Alex Ford
9b115129fe move diagnostics queries to match other languages more closely 2021-05-11 19:53:00 +01:00
Alex Ford
1381d8d076 tidy up Diagnostics library 2021-05-11 19:28:31 +01:00
Alex Ford
9663b74e12 use severity level 3 to indicate an extraction error for a file 2021-05-11 19:23:05 +01:00
Alex Ford
d1d8cff915 tests for some more diagnostics queries 2021-05-11 19:14:22 +01:00
Alex Ford
de497dd1ba tests for NumberOfFiles* summary queries 2021-05-11 19:14:22 +01:00
Nick Rolfe
004147984b Simplify CFG classes for StmtSequences 2021-05-11 18:27:11 +01:00
Alex Ford
8ab95324eb dedupe some error reporting code 2021-05-11 14:09:10 +01:00
Alex Ford
0f3168f293 record more parse errors 2021-05-10 21:23:24 +01:00
Alex Ford
2154b7df30 add doc for IntegerLiteral.getValue 2021-05-10 11:02:48 +01:00
Alex Ford
48add9ffbc remove internal import in rb/overly-permissive-file 2021-05-10 11:00:59 +01:00
Alex Ford
269ae8331b record 'unknown table type' extraction errors 2021-05-07 17:56:50 +01:00
Nick Rolfe
94ceb3f237 Remove unused class 2021-05-07 17:20:51 +01:00
Nick Rolfe
9def7c2dfe Make CFG for TEnsure post-order 2021-05-07 17:15:10 +01:00
Nick Rolfe
7f6805c82f Make CFG for TDo post-order 2021-05-07 17:00:30 +01:00
Nick Rolfe
46c9f858c4 Make CFG for TElse post-order 2021-05-07 16:47:19 +01:00
Nick Rolfe
2569bf257f Make CFG for TThen post-order 2021-05-07 15:40:50 +01:00
Alex Ford
a7873f9023 rb/summary/number-of-files-extracted-with-errors 2021-05-07 00:24:13 +01:00
Alex Ford
31b8913ffd rb/summary/number-of-successfully-extracted-files FIXUP 2021-05-07 00:23:56 +01:00
Alex Ford
804198cd37 rb/summary/number-of-successfully-extracted-files 2021-05-07 00:22:22 +01:00
Alex Ford
e7285babf0 rb/diagnostics/successfully-extracted-files 2021-05-07 00:17:58 +01:00
Alex Ford
54266eca33 rb/diagnostics/files-extracted-with-errors 2021-05-07 00:17:12 +01:00
Alex Ford
d223851429 add Diagnostics.qll 2021-05-07 00:15:09 +01:00
Alex Ford
272aec27f2 clean up the parse_error writing code 2021-05-07 00:15:09 +01:00
Alex Ford
3a1dff1c95 start writing diagnostics entries for parse errors 2021-05-06 23:09:43 +01:00
Alex Ford
c38453305f add diagnostics table to dbscheme 2021-05-06 22:58:01 +01:00
Alex Ford
e5896047d8 summary LOC query tests 2021-05-06 19:54:23 +01:00
Alex Ford
98a4f4c5b9 rb/summary/lines-of-user-code 2021-05-06 19:54:23 +01:00
Alex Ford
f6c8b07f4f rb/summary/lines-of-code 2021-05-06 19:54:23 +01:00
Nick Rolfe
4e80b548c1 Make BeginBlock CFG post-order 2021-05-06 16:45:27 +01:00
Nick Rolfe
2c7f1e0c11 Remove unused class 2021-05-06 16:28:36 +01:00
Nick Rolfe
9185a93312 Make SingletonClassDeclarationTree post-order 2021-05-06 16:20:50 +01:00
Nick Rolfe
fd3d50f340 Make ModuleDeclarationTree post-order 2021-05-06 15:54:11 +01:00
Nick Rolfe
d623f47ba0 Make ClassDeclarationTree post-order 2021-05-06 15:36:25 +01:00
Arthur Baars
07c059cb2e Merge pull request #166 from github/type_tracking
Minimal implementation of shared type-tracking library
2021-05-06 10:59:45 +02:00
Nick Rolfe
a0084b7732 Simplify CFG tree classes for calls 2021-05-05 17:18:44 +01:00
Nick Rolfe
569063ca73 Make YieldCallTree post-order 2021-05-05 17:14:32 +01:00
Nick Rolfe
3a3586f14b Restrict type to MethodCallCfgNode 2021-05-05 14:49:24 +01:00
Arthur Baars
73b5699f32 Merge pull request #174 from github/escape_file_keys
Escape keys for files and folders
2021-05-05 15:02:04 +02:00
Nick Rolfe
c37f390efc Reserve more capacity for escaped key 2021-05-05 13:21:16 +01:00
Nick Rolfe
99ae17de03 Avoid copying key when it doesn't need escaping 2021-05-05 12:54:23 +01:00
Nick Rolfe
b16b95e2f7 Fix type-tracking load/store steps 2021-05-05 12:12:45 +01:00
Nick Rolfe
d2d5f31599 Escape keys for files and folders 2021-05-04 16:52:35 +01:00
Nick Rolfe
647c108c0b Merge remote-tracking branch 'origin/main' into type_tracking 2021-05-04 12:38:16 +01:00
Arthur Baars
1a94fb47b6 Merge pull request #172 from github/update-testoutput
Update expected test output
2021-05-04 13:37:37 +02:00
Arthur Baars
27538cb11d Update expected test output 2021-05-04 12:43:43 +02:00
Nick Rolfe
53deede8ab Remove unnecessary local flow inside type-tracking store step 2021-05-04 11:32:57 +01:00
Nick Rolfe
35ee62c689 Use splitting-aware nodes for type-tracking store/load steps 2021-05-04 11:31:03 +01:00
Arthur Baars
6adff6f195 Merge pull request #171 from github/self_nodes
Create synthetic `self` nodes for calls without explicit receivers
2021-05-03 12:59:11 +02:00
Nick Rolfe
5dc910d0db Move track predicate to LocalSourceNode 2021-04-30 15:05:12 +01:00
Nick Rolfe
37c8d8a252 Rename getCallable to getTarget 2021-04-30 14:41:50 +01:00
Nick Rolfe
fdccd5da7e Add AstNode::isSynthesized() 2021-04-30 11:58:54 +01:00
Alex Ford
2c8a4f833f make rb/overly-permissive-file a proper path-problem 2021-04-29 19:11:39 +01:00
Nick Rolfe
e87bf57bc5 Avoid recursion in IPA construction 2021-04-29 18:04:15 +01:00
Alex Ford
4375452866 more IntegerLiteral.getValue improvements 2021-04-29 17:08:33 +01:00
Alex Ford
05adfec03d account for more patterns in IntegerLiteral.getValue 2021-04-29 17:02:54 +01:00
Alex Ford
35d5bae10e run formatter 2021-04-29 16:16:09 +01:00
Alex Ford
efa323c304 rb/overly-permissive-file use QL bitwise operators 2021-04-29 16:08:42 +01:00
Alex Ford
46a14b2826 move parseInt logic into getValue method predicate on IntegerLiteral 2021-04-29 15:54:22 +01:00
Alex Ford
1c89bbe188 fix select format of rb/overly-permissive-file 2021-04-29 15:44:54 +01:00
Nick Rolfe
bd6fe41388 Merge IPA branches for implicit self 2021-04-29 15:38:58 +01:00
Alex Ford
2c0fc7d193 parse integer permission args as ints instead of using regex matches 2021-04-29 15:34:10 +01:00
Nick Rolfe
59c83b7b8f Add clarifying comment 2021-04-29 14:00:27 +01:00
Nick Rolfe
9540125771 Remove fromGeneratedInclSynth predicate 2021-04-29 13:58:16 +01:00
Arthur Baars
300a54384f Add TypeTracker to identical-files.json 2021-04-29 12:20:14 +02:00
Arthur Baars
f07c58ee07 Update codeql submodule 2021-04-29 12:13:11 +02:00
Nick Rolfe
96ddd55191 Apply suggestions from code review
Co-authored-by: Arthur Baars <aibaars@github.com>
2021-04-29 12:07:32 +02:00
Nick Rolfe
c1c437f020 Minimal implementation of shared type-tracking library 2021-04-29 12:07:32 +02:00
Nick Rolfe
f3852f9b56 Create synthetic self nodes for calls without explicit receivers 2021-04-28 16:43:40 +01:00
Alex Ford
0a6dc6f150 update WeakFilePermissions.expected 2021-04-28 16:31:07 +01:00
Alex Ford
7a72d8ec2f add qhelp for rb/overly-permissive-file 2021-04-28 15:51:08 +01:00
Alex Ford
e3d393b7c1 use full dataflow for permission args in rb/overly-permissive-file 2021-04-28 15:40:58 +01:00
Alex Ford
e5862a942f WIP rb/overly-permissive-file query 2021-04-27 21:22:17 +01:00
Arthur Baars
bc6aec7a99 Merge pull request #167 from github/alexrford/numlines
Implement FLines metrics queries
2021-04-21 14:42:18 +02:00
Alex Ford
240f0abf27 drop @tags from metrics queries 2021-04-21 13:00:48 +01:00
Alex Ford
15289dba34 simplify File.getNumberOfLines 2021-04-21 12:59:25 +01:00
Alex Ford
cc5bbfce0b Get -> Gets 2021-04-21 12:57:55 +01:00
Alex Ford
5a191692df Update ql/src/queries/metrics/FLinesOfComments.ql
Co-authored-by: Arthur Baars <aibaars@github.com>
2021-04-21 12:57:12 +01:00
Alex Ford
4e119cc085 consider empty files (no ruby tokens) to have 0 lines 2021-04-21 11:29:55 +01:00
Alex Ford
a8597025aa fixed logic for line counting 2021-04-21 11:29:09 +01:00
Alex Ford
bcc1be05de use explicit this prefixes in FileSystem.qll 2021-04-21 10:51:28 +01:00
Alex Ford
85ecacd858 make helper predicates private 2021-04-21 10:50:00 +01:00
Alex Ford
9d117d10b8 drop MetricFile class 2021-04-21 10:45:42 +01:00
Alex Ford
c6b6a83501 extend FLines* tests 2021-04-21 10:42:53 +01:00
Alex Ford
a1c91e28da move FLines* tests to a common directory 2021-04-21 10:34:58 +01:00
Alex Ford
fcd46025fe update metadata for FLines* queries 2021-04-21 10:28:20 +01:00
Arthur Baars
abb37e212a Merge pull request #165 from github/aibaars/methods
Implement method lookup
2021-04-21 11:24:20 +02:00
Arthur Baars
549e5ab9d6 Revert "Rename Method -> MethodDeclaration"
This reverts commit d361ef37af.
2021-04-21 10:50:47 +02:00
Arthur Baars
1245674df8 Add missing @id properties 2021-04-21 10:50:47 +02:00
Alex Ford
50a0f282bf add basic tests for FLines queries 2021-04-20 17:36:16 +01:00
Alex Ford
f0d1498c8c Revert "WIP: populate numlines table"
This reverts commit 62bf58b289.
2021-04-20 17:36:16 +01:00
Alex Ford
37cce23c26 add FLines.ql, FLinesOfComments.ql 2021-04-20 17:36:16 +01:00
Alex Ford
d6c7846089 put logic for determining line counts into MetricFile 2021-04-20 17:36:16 +01:00
Arthur Baars
122315db3f Remove 'Method' class 2021-04-20 13:41:11 +02:00
Alex Ford
28e46c8915 add FLinesOfCode.ql metric query 2021-04-20 10:12:52 +01:00
Alex Ford
7bfc61789d line count MetricFile predicates 2021-04-19 18:08:01 +01:00
Alex Ford
62bf58b289 WIP: populate numlines table 2021-04-19 18:06:35 +01:00
Arthur Baars
bf4f91e038 Address comments 2021-04-16 16:37:42 +02:00
Arthur Baars
07726fd979 Add some module and method tests 2021-04-16 11:07:57 +02:00
Arthur Baars
bf556a2b53 Implement method lookup 2021-04-15 11:32:43 +02:00
Arthur Baars
5837af0936 Add MethodBase::getMethod 2021-04-15 11:32:43 +02:00
Arthur Baars
d361ef37af Rename Method -> MethodDeclaration 2021-04-15 11:32:43 +02:00
Arthur Baars
3590a2c2ac Merge pull request #164 from github/aibaars/fix-modules
Improve module/class resolution
2021-04-15 11:32:28 +02:00
Arthur Baars
24bb11b20a Improve module/class resolution 2021-04-14 17:14:38 +02:00
Arthur Baars
12ee957331 Add test cases 2021-04-14 17:12:39 +02:00
Arthur Baars
3b73d41cc4 Merge pull request #163 from github/aibaars/modules-2
Ignore include/prepend statements in blocks
2021-04-14 17:09:34 +02:00
Arthur Baars
9afda342bc Address comments 2021-04-14 09:57:49 +02:00
Arthur Baars
754bfdd136 Ignore include/prepend statements in blocks
Include and prepend statements are rarely used in block in normal code and when
used in normal code they tend to be in blocks that are passed to methods like
`module_eval` which is a builtin method that evaluates a block in the context
of some other module (typically created with Module.new). We currently don't attempt
to track such "dynamically" constructed modules, and ignoring such modules
 and the `module_eval` calls on them seems fine for now.

Another, much more frequent use of include/prepend statements in blocks is in Rspec.describe and
Rspec.context method calls in tests. Rspec also evaluates those blocks in the context of some
special Rspec class. Precisely tracking such calls during the initial construction of the module/class
hierarchy would be really hard and there would be little benefit because the interesting modules and classes of
an application are not defined in test files.
2021-04-14 09:53:19 +02:00
Arthur Baars
280fe73063 Add test case with 'module_eval' call with block containing 'prepend' statement 2021-04-14 09:53:19 +02:00
Arthur Baars
caef2c36c7 Merge pull request #162 from github/aibaars/modules
Basic implementation of module resolution
2021-04-09 20:50:54 +02:00
Arthur Baars
cdfabbc95d Make Cached module private 2021-04-09 16:47:02 +02:00
Arthur Baars
a247544fc5 Add comments 2021-04-09 16:35:23 +02:00
Arthur Baars
7bc5be93ff Module: make main predicates cached 2021-04-09 13:29:27 +02:00
Arthur Baars
2db999d0da Improve module resolution 2021-04-09 09:51:24 +02:00
Arthur Baars
ceb2eb21d8 Address comments 2021-04-08 15:11:57 +02:00
Arthur Baars
039e8b36a5 Add some include/prepend tests 2021-04-07 17:27:33 +02:00
Arthur Baars
84f6e902ea AST: move some scope related methods to AstNode 2021-04-07 17:16:10 +02:00
Arthur Baars
063b085078 Address comments 2021-04-07 15:57:13 +02:00
Arthur Baars
50b8b6b257 Also resolve constants with respect to the ancestors
of the enclosing module.
2021-04-06 15:47:13 +02:00
Arthur Baars
f12e6ea8ea Avoid 'Object::' prefixes 2021-03-30 16:14:21 +02:00
Arthur Baars
b2c7185664 Add tests 2021-03-30 15:49:41 +02:00
Arthur Baars
201c1e4b81 Basic module resolution 2021-03-30 15:40:03 +02:00
Arthur Baars
ea9afcd4e1 AST: make some classes instance of Scope 2021-03-30 15:40:03 +02:00
Arthur Baars
eebbc7e505 AST: rename Class/Module to ClassDefinition/ModuleDefinition 2021-03-30 15:40:01 +02:00
Tom Hvitved
aad5d133d0 Merge pull request #161 from github/hvitved/cfg-remove-is-hidden
CFG: Remove `isHidden()` predicate
2021-03-25 15:08:17 +01:00
Tom Hvitved
0bb5007103 Reintroduce hidden then/else/do in AST; include all in CFG 2021-03-25 14:22:35 +01:00
Tom Hvitved
58ecd771d3 AST: Exclude empty then/else/do statements 2021-03-25 09:53:55 +01:00
Tom Hvitved
ca7c0584c7 CFG: Remove isHidden() predicate 2021-03-24 17:22:05 +01:00
Tom Hvitved
9472cef492 Merge pull request #160 from github/bump-codeql
Bump `codeql` sub module and implement new data-flow stubs
2021-03-24 15:34:42 +01:00
Tom Hvitved
6c00e66272 Update ql/src/codeql_ruby/dataflow/internal/DataFlowPrivate.qll
Co-authored-by: Nick Rolfe <nickrolfe@github.com>
2021-03-24 15:02:30 +01:00
Tom Hvitved
b8f65fb756 Bump codeql sub module and implement new data-flow stubs 2021-03-24 14:00:21 +01:00
Arthur Baars
d103acb04f Merge pull request #158 from github/hvitved/vscode-hide-codeql-submodule
Hide `codeql` sub module in VS Code workspace
2021-03-23 10:41:32 +01:00
Arthur Baars
6a26483fc7 Merge pull request #159 from github/hvitved/herdoc-body-rank-performance
Improve performance of `HereDoc::getBody()`
2021-03-23 10:40:28 +01:00
Tom Hvitved
2891d94f99 Improve performance of HereDoc::getBody()
Gets rid of
```
[2021-03-23 10:07:49] (138s) Tuple counts for Literal::HereDoc::getBody_dispred#ff#shared#1/4@1cc5b9:
                      11294    ~0%        {1} r1 = SCAN AST::Cached::THereDoc#ff@staged_ext OUTPUT In.0
                      11294    ~388%      {1} r2 = JOIN r1 WITH Literal::HereDoc::getBody_dispred#ff#join_rhs ON FIRST 1 OUTPUT Rhs.1 'arg1'
                      95514613 ~2080%     {4} r3 = JOIN r2 WITH locations_default_1023#join_rhs ON FIRST 1 OUTPUT Rhs.1 'arg0', Lhs.0 'arg1', Rhs.2 'arg2', Rhs.3 'arg3'
```
2021-03-23 10:31:48 +01:00
Tom Hvitved
1004363131 Hide codeql sub module in VS Code workspace 2021-03-23 09:55:56 +01:00
Nick Rolfe
b293522710 Merge pull request #150 from github/parent_child
Create `ast_node_parent` relation
2021-03-22 15:06:50 +00:00
Nick Rolfe
e7f1ae8c96 Merge remote-tracking branch 'origin/main' into parent_child 2021-03-22 14:58:33 +00:00
Nick Rolfe
3284a3fc1f Merge pull request #157 from github/cfg_impl
Port CFG implementation to public AST interface
2021-03-22 14:57:43 +00:00
Nick Rolfe
cf7ce911bc Combine CfgScope classes for BodyStmt ∩ Callable 2021-03-19 16:08:43 +00:00
Nick Rolfe
7667606b89 Replace some uses of Generated types 2021-03-19 14:31:17 +00:00
Nick Rolfe
21192bf43c Remove outdated comment 2021-03-19 14:28:26 +00:00
Nick Rolfe
f37c862c92 Rename MandatoryParameterTree to NonDefaultValueParameterTree 2021-03-19 14:27:29 +00:00
Nick Rolfe
c6958f64e4 Make CFG for AssignExpr visit left operand before right 2021-03-19 14:25:38 +00:00
Nick Rolfe
f381f94bc2 Rename ProgramScope to ToplevelScope 2021-03-19 14:02:54 +00:00
Nick Rolfe
5cedf7ee86 Remove unused import 2021-03-19 13:59:02 +00:00
Tom Hvitved
e175513293 Remove duplicate tuple patterns 2021-03-19 10:52:29 +01:00
Nick Rolfe
c0636bef29 Make CfgScope extend Scope 2021-03-18 19:08:51 +00:00
Nick Rolfe
6bcc433af3 Uncomment empty class and module in CFG test 2021-03-18 19:02:32 +00:00
Nick Rolfe
9493997e9d Make space in CFG test for two new lines in the middle
Commented out to make it easier to ignore the noise from line number
changes.
2021-03-18 19:01:11 +00:00
Nick Rolfe
37435764a0 Fix control-flow for empty classes and modules 2021-03-18 18:58:40 +00:00
Nick Rolfe
434d9e54a1 Fix complex symbols having multiple ControlFlowTree implementations 2021-03-18 14:48:08 +00:00
Nick Rolfe
4ce7faf868 Fix erroneous flow from 'raise' call to StmtSequence 2021-03-18 13:01:27 +00:00
Nick Rolfe
ceda7c8fd2 Generalise splitting of parenthesized exprs to all statement sequences 2021-03-18 11:21:11 +00:00
Nick Rolfe
c8eab42c1d Minor comment fixes 2021-03-18 11:09:21 +00:00
Tom Hvitved
3bb2c529a5 CFG: Revert change to mandatory parameters 2021-03-18 10:43:10 +01:00
Arthur Baars
d4030c66d8 Update Consistency.qll 2021-03-18 09:54:44 +01:00
Tom Hvitved
c761ab6882 Merge pull request #156 from github/hvitved/ipa-ast
Make external `AstNode` an IPA type
2021-03-17 22:23:05 +01:00
Nick Rolfe
32e2b257bf Port CFG implementation to public AST interface 2021-03-17 20:28:47 +00:00
Nick Rolfe
26c251f080 Order CFG nodes by column as well 2021-03-17 19:07:52 +00:00
Tom Hvitved
39aa2c6e53 Rework IPA injectors for constant accesses 2021-03-17 14:27:21 +01:00
Tom Hvitved
eb7610c55f Rename (to|from)TreeSitter to (to|from)Generated 2021-03-17 09:28:23 +01:00
Tom Hvitved
5724112513 Address review comments 2021-03-17 09:28:18 +01:00
Tom Hvitved
7eaf02a0bf Make external AstNode an IPA type 2021-03-16 12:50:20 +01:00
Arthur Baars
c672169621 Merge pull request #155 from github/aibaars/order-ast-test
AST: order edges by target node
2021-03-15 10:43:34 +01:00
Arthur Baars
d54db292f7 Move semmle.order property to printAst.qll 2021-03-15 10:33:10 +01:00
Arthur Baars
3e5ff1d042 AST: order edges by target node
When printing a tree CodeQL iterates over the nodes and
for each node prints the successor edges as children. If the
the successor edges are ordered by target node then the children
printe in the right order in the expected output.
2021-03-12 16:52:34 +01:00
Arthur Baars
cde496cc4c Merge pull request #152 from github/aibaars/fix-vars
Fix VariableRead/WriteAcess for instance and class variables
2021-03-11 17:05:56 +01:00
Calum Grant
bf873c8ad1 Merge pull request #147 from github/calumgrant/use-detect
Ruby: New query UseDetect
2021-03-10 14:39:37 +00:00
Calum Grant
cb977cb290 Ruby: Use getAUniqueRead TC 2021-03-10 10:56:33 +00:00
Arthur Baars
3966de6b2b Merge pull request #151 from github/aibaars/scopes-refactor
Add Scopes.qll and remove VariableScopes IPA type
2021-03-09 20:55:18 +01:00
Arthur Baars
6a284378d6 Update ql/src/codeql_ruby/ast/Scope.qll
Co-authored-by: Nick Rolfe <nickrolfe@github.com>
2021-03-09 18:57:24 +01:00
Arthur Baars
f28071ceb6 Fix VariableRead/WriteAcess for instance and class variables 2021-03-09 13:55:55 +01:00
Arthur Baars
600d9c66ae Remove VariableScope 2021-03-09 11:56:17 +01:00
Arthur Baars
86a89ab1fe Remove VariableScope IPA type 2021-03-09 11:48:18 +01:00
Calum Grant
855d190800 Ruby: Test local data flow 2021-03-09 10:25:24 +00:00
Calum Grant
5b4bf584a1 Ruby: Update qltest output for new select format 2021-03-09 10:20:23 +00:00
Calum Grant
0f829476f4 Ruby: Refactor EndCall to reduce number of classes 2021-03-09 10:13:07 +00:00
Arthur Baars
00260db58f Add Scope.qll 2021-03-09 09:46:42 +01:00
Nick Rolfe
56e03d7ed4 Remove old upgrades 2021-03-08 18:28:23 +00:00
Nick Rolfe
be102e24f6 Update stats 2021-03-08 18:25:37 +00:00
Nick Rolfe
f691ec9e2a Remove overrides of getParent[Index] 2021-03-08 18:25:37 +00:00
Nick Rolfe
9b96bc32cc Add ast_node_parent relation 2021-03-08 18:25:37 +00:00
Nick Rolfe
61b3aa8f27 Merge pull request #149 from github/manual_stats_workflow
Enable manual dispatch of stats workflow
2021-03-08 14:28:27 +00:00
Nick Rolfe
df8f7a30d7 Enable manual dispatch of stats workflow 2021-03-08 14:10:37 +00:00
Nick Rolfe
1818b68ea2 Merge pull request #148 from github/calumgrant/readme-qltest
Update README.md
2021-03-04 19:36:42 +00:00
Calum Grant
67416a6440 Update README.md
Co-authored-by: Nick Rolfe <nickrolfe@github.com>
2021-03-04 17:19:34 +00:00
Calum Grant
0be5c529ee Update README.md 2021-03-04 16:03:23 +00:00
Calum Grant
20a62d169a Ruby: Update query description 2021-03-04 15:48:09 +00:00
Calum Grant
ca497479c2 Ruby: Finish the test for UseDetect 2021-03-04 15:44:05 +00:00
Calum Grant
522bcff79d Ruby: Initial test case 2021-03-04 15:38:09 +00:00
Calum Grant
5854b831f3 Ruby: rb/use-detect query 2021-03-04 13:43:59 +00:00
Arthur Baars
ce69c912fd Merge pull request #145 from github/aibaars/fix
Fix regression in rb/unused-parameter
2021-03-01 12:26:47 +01:00
Arthur Baars
c9f86743bd Merge pull request #143 from github/aibaars/ast-test
AST: add printAST test case
2021-02-26 19:41:56 +01:00
Arthur Baars
b2fbeee794 CFG: hide all non-AstNodes 2021-02-26 19:04:33 +01:00
Arthur Baars
5f32b822e2 Remove use of AstNodes 2021-02-26 19:03:55 +01:00
Arthur Baars
dd4f297c37 Remove duplicate clause 2021-02-26 17:51:04 +01:00
Arthur Baars
39181ec871 AST: printAST: show all primary classes and method names 2021-02-25 15:25:49 +01:00
Arthur Baars
e2b2a450ac AST: add printAST test case 2021-02-25 15:25:49 +01:00
Arthur Baars
75883b94cd QLTest: ignore *.testproj folders 2021-02-25 15:25:42 +01:00
Arthur Baars
7ab147a7b8 Merge pull request #144 from github/aibaars/missing
AST: add missing getAPrimaryQlClass predicate
2021-02-25 15:18:29 +01:00
Arthur Baars
a6bb34c86d AST: add missing getAPrimaryQlClass predicate 2021-02-25 14:59:39 +01:00
Arthur Baars
fa7adee245 Merge pull request #142 from github/aibaars/clean-up
Remove as many references to TreeSitter::Generated
2021-02-25 14:28:09 +01:00
Arthur Baars
9800e3f930 Add some TODO comments 2021-02-25 13:43:36 +01:00
Arthur Baars
f3d1c804be Update test data 2021-02-25 12:57:18 +01:00
Arthur Baars
7c0ea7b3bc CFG: add AstNode for @in 2021-02-25 12:57:18 +01:00
Arthur Baars
b16d6bf5b4 CFG: make isValidFor work for hidden nodes 2021-02-25 12:57:18 +01:00
Arthur Baars
9fc5c43412 Clean-up Completion.qll 2021-02-25 12:57:18 +01:00
Arthur Baars
999b82ca73 Remove imports of TreeSitter 2021-02-25 12:57:18 +01:00
Arthur Baars
d30912611b Merge pull request #136 from github/aibaars/child-parent
Finish AST and add consistency query
2021-02-25 12:54:45 +01:00
Arthur Baars
27a2310840 CFG: sort expected output by file path and line 2021-02-25 12:27:11 +01:00
Arthur Baars
87b2c142bc Update qldoc 2021-02-25 10:23:29 +01:00
Arthur Baars
4ba0f3088a Use strictcount 2021-02-25 10:21:07 +01:00
Arthur Baars
0f940349ba AST: rename getExpr predicates to more meaningful names 2021-02-25 10:11:29 +01:00
Arthur Baars
1a73cf6cc4 AST: add ArgumentList 2021-02-24 19:07:16 +01:00
Arthur Baars
336b310668 AST: improve AST for special parameters 2021-02-24 19:07:16 +01:00
Arthur Baars
8913810bf0 AST: change return type of Assignment LHS to Pattern 2021-02-24 19:07:16 +01:00
Arthur Baars
190978cc56 AST: add consistency query 2021-02-24 19:07:16 +01:00
Arthur Baars
cb21e8edda CFG: hide nodes that are not proper AstNodes 2021-02-24 19:07:16 +01:00
Arthur Baars
14474d660b AST: change types to Stmt 2021-02-24 19:07:16 +01:00
Arthur Baars
3288070279 Merge pull request #131 from github/aibaars/pattern
AST: split method call into normal and setter calls
2021-02-24 19:03:55 +01:00
Arthur Baars
a7408dd262 Merge pull request #140 from github/aibaars/namespace
AST: introduce 'Namespace' as super class of Class/Module
2021-02-24 13:22:02 +01:00
Arthur Baars
242481c701 Apply suggestions from code review
Co-authored-by: Nick Rolfe <nickrolfe@github.com>
2021-02-24 13:13:47 +01:00
Arthur Baars
d6c0049a7e AST: make SetterMethodCall instance of MethodCall 2021-02-24 13:06:54 +01:00
Arthur Baars
7ae20f3b5b AST: add SetterMethodCall as instance of LhsExpr 2021-02-24 13:06:54 +01:00
Arthur Baars
79bb20b31f AST: add MethodCall as a subclass of Call 2021-02-24 13:06:53 +01:00
Arthur Baars
5fe7bd57fa AST: calls without method name 2021-02-24 13:02:22 +01:00
Arthur Baars
eaeabf19bf Merge pull request #141 from github/bump_ts
Add support for multiple statements in interpolations
2021-02-24 11:29:26 +01:00
Nick Rolfe
37253fd1f1 Update stats for dbscheme change to interpolation_child 2021-02-23 16:08:24 +00:00
Nick Rolfe
6c84f2c3dc Add test case for multiple statements in interpolation 2021-02-23 15:52:11 +00:00
Nick Rolfe
672148e5b4 Add support for multiple statements in interpolations 2021-02-23 15:36:14 +00:00
Arthur Baars
a7ddd642ea AST: introduce 'Namespace' as super class of Class/Module 2021-02-19 13:34:34 +01:00
Arthur Baars
098e0ac142 Merge pull request #139 from github/printast
printAst: use the user-facing AST library
2021-02-19 09:46:27 +01:00
Nick Rolfe
d52e439547 printAst: use the user-facing AST library 2021-02-18 18:25:57 +00:00
Arthur Baars
370135fab7 Merge pull request #138 from github/aibaars/part-1
AST: getChild/getParent
2021-02-18 19:00:08 +01:00
Arthur Baars
1c8a76f44a AST: make Assignment::getLeftOperand a Pattern again 2021-02-18 18:14:55 +01:00
Arthur Baars
c877eb4642 AST: add additional token-types to variable patterns 2021-02-18 14:37:58 +01:00
Arthur Baars
3ee83870b6 AST: add begin expressions 2021-02-18 14:37:58 +01:00
Arthur Baars
5659388ec0 AST: implement AstNode::child 2021-02-18 14:37:58 +01:00
Arthur Baars
c0b5ac760a AST: rename getLhs/getRhs to getLeftOperand/getRightOperand 2021-02-18 14:37:58 +01:00
Arthur Baars
095eb803b3 AST: improve type of getDefaultValue 2021-02-18 14:37:58 +01:00
Arthur Baars
e42d1ff936 Change Expr to LhsExpr for getVariableExpr 2021-02-18 14:37:58 +01:00
Arthur Baars
214f113016 AST: add getChild/getParent method 2021-02-18 14:37:53 +01:00
Nick Rolfe
ac3da22158 Merge pull request #137 from github/scope_tostring
Include file/class/method/module names in VariableScope::toString
2021-02-17 19:24:36 +00:00
Nick Rolfe
b8bbbe92f3 Include file/class/method/module names in VariableScope::toString 2021-02-17 18:10:03 +00:00
Nick Rolfe
aedf093e72 Merge pull request #135 from github/aibaars/heredoc
AST: HereDoc
2021-02-17 17:18:38 +00:00
Arthur Baars
cabe6df820 Add missing heredoc end token 2021-02-17 15:58:13 +01:00
Arthur Baars
e1047fad2c CFG: remove intermediate HeredocBody nodes 2021-02-17 13:10:18 +01:00
Arthur Baars
167574d82f AST: HereDoc 2021-02-17 13:10:18 +01:00
Arthur Baars
1e19904342 Merge pull request #134 from github/literals
Add and expand AST classes for literals
2021-02-17 13:09:02 +01:00
Nick Rolfe
c019da83f3 Address feedback on StringInterpolationComponent::getStmt 2021-02-17 10:57:01 +00:00
Nick Rolfe
97654eb338 Simplify bash script 2021-02-17 10:48:17 +00:00
Arthur Baars
4f5b1c06ac Merge branch 'main' into literals 2021-02-16 19:30:03 +01:00
Arthur Baars
3f4b4b360e Merge pull request #133 from github/aibaars/pattern-0
AST: RestAssignment and LhsExpr
2021-02-16 19:29:15 +01:00
Nick Rolfe
3978d6387e Update tree-sitter-ruby revision used 2021-02-16 16:49:59 +00:00
Nick Rolfe
4537e5d6f8 Update expected test output to match truncation of long strings 2021-02-16 16:21:49 +00:00
Nick Rolfe
02f853b8fd Add r suffix to RationalLiteral::getValueText() 2021-02-16 16:21:28 +00:00
Nick Rolfe
cd38b980a8 Update dbscheme stats 2021-02-16 16:13:00 +00:00
Nick Rolfe
fff5dad702 Truncate long strings in StringlikeLiteral::toString() 2021-02-16 16:11:41 +00:00
Arthur Baars
7dd429c945 Format Expr.qll 2021-02-16 15:41:44 +00:00
Nick Rolfe
1c869f6d85 Make merge_stats.py work in python3 2021-02-16 15:41:44 +00:00
Nick Rolfe
5e6ef5c8b5 Upgrade script for dbscheme changes to range_* 2021-02-16 15:41:44 +00:00
Nick Rolfe
2eb8757285 Update expected test output for toString changes 2021-02-16 15:41:05 +00:00
Nick Rolfe
f56f81f555 Add and expand AST classes for literals 2021-02-16 15:41:05 +00:00
Arthur Baars
c4b3c8bc28 More QLDoc for LhsExpr 2021-02-16 16:09:56 +01:00
Arthur Baars
e3cf226679 AST: make ConstantWriteAccess extend LhsExpr 2021-02-16 13:03:04 +01:00
Arthur Baars
9d449a90c2 AST: add LhsExpr 2021-02-16 13:02:02 +01:00
Arthur Baars
7778f1c21f AST: make Pattern:Range abstract 2021-02-16 13:01:00 +01:00
Arthur Baars
9c5da197ed AST: add Pattern::getRestIndex 2021-02-16 12:56:06 +01:00
Arthur Baars
eee12eecc9 Merge pull request #132 from github/rescue_naming
Rename {Rescue,RescueExpr} to {RescueExpr,RescueModifierExpr}
2021-02-16 12:54:19 +01:00
Nick Rolfe
04ad1f805a Update rust auto-formatting for 1.50 2021-02-16 11:47:24 +00:00
Nick Rolfe
0fc19ea7a9 Rename RescueExpr to RescueClause 2021-02-16 11:46:30 +00:00
Nick Rolfe
cf50006d68 Rename {Rescue,RescueExpr} to {RescueExpr,RescueModifierExpr} 2021-02-16 11:09:25 +00:00
Arthur Baars
90f59de589 Merge pull request #130 from github/aibaars/ast-5
AST: add ElementReference as call
2021-02-15 14:59:34 +01:00
Arthur Baars
ad6c916f01 Merge pull request #129 from github/aibaars/ast-4
AST: rescue modifier
2021-02-15 14:59:22 +01:00
Arthur Baars
c6c39ad04d Merge pull request #128 from github/aibaars/ast-3
AST: undef and alias
2021-02-15 14:59:12 +01:00
Arthur Baars
5b8c74eb5b AST: add SingletonMethod::getObject 2021-02-15 13:53:50 +01:00
Arthur Baars
e3f54411d8 AST: add ElementReference 2021-02-15 13:51:16 +01:00
Arthur Baars
d69a1731f9 Fix QL doc 2021-02-15 12:53:13 +01:00
Arthur Baars
ddea74265d AST: rescue modifier 2021-02-15 12:50:00 +01:00
Arthur Baars
9cb58be5cf AST: avoid multivalued results for MethodName::getValueText 2021-02-15 10:39:21 +01:00
Arthur Baars
8a4f27c052 Add test case 2021-02-12 19:23:13 +01:00
Arthur Baars
5f1907efc4 AST: undef and alias 2021-02-12 19:22:51 +01:00
Arthur Baars
392af7fe76 Merge pull request #127 from github/aibaars/ast-2
Some more AST
2021-02-12 18:40:24 +01:00
Arthur Baars
c0c155361f Address comments 2021-02-12 18:31:44 +01:00
Arthur Baars
874ac121d9 AST: Toplevel and BEGIN/ END blocks 2021-02-12 15:26:30 +01:00
Arthur Baars
015b581f57 AST: add redo, retry, empty-statement 2021-02-12 15:18:28 +01:00
Arthur Baars
64cba18c41 AST: add Self class 2021-02-12 14:09:00 +01:00
Arthur Baars
ce824f4adb Merge pull request #126 from github/aibaars/rescue
AST: rescue clauses
2021-02-12 14:08:31 +01:00
Arthur Baars
63f67aa04e AST: rename getVariable to getVariableExpr 2021-02-12 13:35:17 +01:00
Tom Hvitved
1aaebeea76 Merge pull request #125 from github/hvitved/cfg-to-string
CFG: Reintroduce `toString()`s
2021-02-11 18:46:26 +01:00
Arthur Baars
43b238f729 AST: rescue clauses 2021-02-11 18:40:29 +01:00
Tom Hvitved
c4ee79ed27 CFG: Reintroduce toString()s 2021-02-11 18:37:18 +01:00
Nick Rolfe
307db73c9c Merge pull request #124 from github/aibaars/ast-stmt-expr
AST: make Expr extend Stmt and change ExprSequence to StmtSequence
2021-02-11 17:00:21 +00:00
Arthur Baars
f9e9dc2304 Address comment
Co-authored-by: Nick Rolfe <nickrolfe@github.com>
2021-02-11 17:53:28 +01:00
Arthur Baars
c4e2c87d82 AST: some statement tests 2021-02-11 17:20:11 +01:00
Arthur Baars
d42b6b651e AST: rename ExprSequence to StmtSequence 2021-02-11 17:20:10 +01:00
Arthur Baars
fd6aeba9f5 AST: make Expr extend Stmt 2021-02-11 17:20:10 +01:00
Arthur Baars
f02d4a977d AST: some statement tests 2021-02-11 17:20:10 +01:00
Arthur Baars
d02d359c51 Merge pull request #122 from github/constants_scopes
Rework handling of scope resolution nodes, and add `ConstantAccess` class
2021-02-11 17:19:47 +01:00
Arthur Baars
ada652b6f0 Merge branch 'main' into constants_scopes 2021-02-11 17:00:50 +01:00
Nick Rolfe
885137dca2 Simplify representation of calls that use scope resolution operator.
Now, `Foo::bar` is a call where the receiver expr is `Foo`.
2021-02-11 15:29:42 +00:00
Arthur Baars
f8ce7276a3 Merge pull request #123 from github/aibaars/ast-ensure
AST: ensure and else blocks
2021-02-11 15:17:30 +01:00
Arthur Baars
a908f2fe86 Merge pull request #121 from github/aibaars/dataflow-2
Dataflow: identify ReturnNodes
2021-02-11 15:10:27 +01:00
Arthur Baars
426bf30822 AST: ensure and else blocks 2021-02-11 14:27:23 +01:00
Arthur Baars
4f3412fff9 Address comments 2021-02-11 13:46:34 +01:00
Nick Rolfe
23998e5f99 Accept CFG test changes
Some generated ScopeResolution nodes are no longer represented in the
user-facing AST. These should go away when we port the CFG to the
user-facing AST.
2021-02-11 12:38:13 +00:00
Nick Rolfe
6ff0ebb94a Add ConstantAccess class 2021-02-11 12:29:25 +00:00
Nick Rolfe
452a343e86 Remove ScopeResolution from AST
Now we handle it specially in calls and class/module names, so they have
predicate to get the scope expr.
2021-02-10 17:53:25 +00:00
Arthur Baars
0f6854301e Dataflow: identify ReturnNodes 2021-02-10 18:26:11 +01:00
Arthur Baars
d69aa96f23 More tests 2021-02-10 18:26:11 +01:00
Arthur Baars
6c63bd2586 Merge pull request #120 from github/aibaars/ast
AST: lambda and block bodies
2021-02-10 18:25:37 +01:00
Arthur Baars
635b6fb45b AST: lambda and brace block bodies 2021-02-10 14:45:14 +01:00
Arthur Baars
d4ebcbf18f Merge pull request #118 from github/aibaars/dataflow
More dataflow steps
2021-02-09 20:36:28 +01:00
Tom Hvitved
9cfc08319d Use Generated::AstNode in ExprChildMapping 2021-02-09 19:32:41 +01:00
Arthur Baars
1e64b264ba Fix compilation errors after merge 2021-02-09 18:50:30 +01:00
Arthur Baars
3e0b7c491a Merge remote-tracking branch 'origin/main' into aibaars/dataflow 2021-02-09 18:49:53 +01:00
Tom Hvitved
248f5cd648 Merge pull request #119 from github/hvitved/ast-range
Rangify `AstNode`
2021-02-09 16:47:50 +01:00
Tom Hvitved
85c13a1190 Make entries in RemoveWhenFullCoverage explicit 2021-02-09 16:34:25 +01:00
Arthur Baars
daa7bd7fd4 Move ReturningStmt::getValue implementation to internal library 2021-02-09 14:01:08 +01:00
Arthur Baars
e398837bdc Rename Statement to Stmt 2021-02-09 13:55:06 +01:00
Arthur Baars
bb89e134c4 Address comments 2021-02-09 13:54:46 +01:00
Tom Hvitved
32daf28b34 Rangify AstNode 2021-02-09 12:17:21 +01:00
Arthur Baars
a752491c5f Add flow steps for loop 'return' values 2021-02-08 19:06:07 +01:00
Arthur Baars
adb88df638 Add flow steps for conditional and case expressions 2021-02-08 19:00:47 +01:00
Arthur Baars
c991d550cd AST: add Statement and ReturningStatement 2021-02-08 19:00:47 +01:00
Arthur Baars
bde04d48a2 Merge pull request #116 from github/aibaars/cfg-loop-post-order
CFG: make loop expressions post order
2021-02-08 09:53:25 +01:00
Arthur Baars
f2a6f3aadc Update comments 2021-02-08 09:47:33 +01:00
Arthur Baars
37c4e6cbdf Merge pull request #98 from github/aibaars/erb-extractor
Quick and dirty ERB extraction
2021-02-05 18:45:47 +01:00
Arthur Baars
b553eb6964 CFG: make 'for .. in' post-order
Use the 'in' as the intermediate node that checks whether the Enumerable
has more elements.
2021-02-05 18:23:31 +01:00
Arthur Baars
4ae55a718a CFG: make 'while' post-order 2021-02-05 18:23:31 +01:00
Arthur Baars
5bb32b983c Merge pull request #115 from github/aibaars/dataflow
My first dataflow step
2021-02-05 14:13:38 +01:00
Arthur Baars
846173732b Add newline at the end of each code region in an ERB file 2021-02-05 09:49:25 +01:00
Arthur Baars
9d974bd56d Extract ERB files 2021-02-05 09:48:54 +01:00
Arthur Baars
bc55fa861e Merge pull request #114 from github/aibaars/fix-scopes
Correct the scope of class/method names etc.
2021-02-04 17:22:25 +01:00
Arthur Baars
83bcd26244 Add dataflow step tests 2021-02-04 16:09:50 +01:00
Arthur Baars
62802d53c8 Ensure module/class/methods and their headers belong to the right CfgScope 2021-02-04 15:32:20 +01:00
Arthur Baars
bfc5ee3149 Correct the scope of class/method names etc. 2021-02-04 15:30:58 +01:00
Arthur Baars
a998879897 Add local flow step for parenthesized expressions 2021-02-04 14:06:58 +01:00
Arthur Baars
f8cca01e6f Restrict assigment flow to normal assignments only 2021-02-04 14:04:58 +01:00
Arthur Baars
8368a39f00 QLDoc updates 2021-02-04 13:40:06 +01:00
Arthur Baars
da565875df Merge pull request #112 from github/hvitved/ssa/shared-sync
SSA: Sync with latest changes
2021-02-04 13:39:32 +01:00
Arthur Baars
8cec8699a7 Merge pull request #113 from github/aibaars/parenthesized-expr
AST: add ParenthesizedExpr
2021-02-04 13:36:47 +01:00
Arthur Baars
2035bc4d3a AST: add ParenthesizedExpr 2021-02-04 11:51:05 +01:00
Nick Rolfe
61d9669655 Merge pull request #110 from github/class_ast
Add AST classes for classes and modules
2021-02-03 19:32:55 +00:00
Tom Hvitved
16c4faef6a SSA: Sync with latest changes
Now that the shared SSA library supports uncertain/pseudo reads, we can simplify
the Ruby implementation.
2021-02-03 20:31:36 +01:00
Nick Rolfe
c5fca0cb6b Add ModuleBase base class and combine class/module tests 2021-02-03 16:13:59 +00:00
Arthur Baars
3c0f822369 Merge pull request #111 from github/hvitved/dataflow
Initial data flow library
2021-02-03 13:43:08 +01:00
Tom Hvitved
de77a7f96d Initial data-flow files 2021-02-03 10:57:14 +01:00
Nick Rolfe
8976cc556a Update test to match removal of Module::getAClass() 2021-02-02 18:02:16 +00:00
Nick Rolfe
ee03e84d7f Rename Class.qll to Module.qll 2021-02-02 18:00:29 +00:00
Nick Rolfe
645b8c2a8a Apply suggestions from code review
Co-authored-by: Arthur Baars <aibaars@github.com>
2021-02-02 17:54:00 +00:00
Tom Hvitved
f71505c29c Data flow: Sync files 2021-02-02 13:03:42 +01:00
Nick Rolfe
6331a33b23 Update dbscheme stats 2021-02-01 14:41:00 +00:00
Nick Rolfe
c667791bde Update expected test output to match toString() for classes and modules 2021-02-01 14:23:47 +00:00
Nick Rolfe
0649e6c3b0 Update CFG to handle separate superclass node 2021-02-01 14:23:47 +00:00
Nick Rolfe
fbc1c5e8c0 Add test for Module 2021-02-01 14:23:47 +00:00
Nick Rolfe
86bb8a246b Add test for Class and SingletonClass 2021-02-01 14:23:47 +00:00
Nick Rolfe
d26822ad23 Add upgrade script moving superclass exprs to own table 2021-02-01 14:23:47 +00:00
Nick Rolfe
443a992a90 Add AST classes for classes and modules 2021-02-01 14:23:41 +00:00
Arthur Baars
2770b4fef8 Merge pull request #104 from github/aibaars/variables
Simple implementation of class and instance variables
2021-01-29 18:28:25 +01:00
Arthur Baars
c33c3a1124 Address comments 2021-01-29 17:45:48 +01:00
Arthur Baars
6a7e3bfc10 Address comments 2021-01-29 17:45:48 +01:00
Arthur Baars
2921f72473 Implement class variables 2021-01-29 17:45:48 +01:00
Arthur Baars
a07e0fb0f7 Class variables boilerplate code 2021-01-29 17:45:44 +01:00
Arthur Baars
341bc5c888 Implement instance variables 2021-01-29 16:09:44 +01:00
Arthur Baars
e36795c82e Instance variables boilerplate code 2021-01-29 15:41:23 +01:00
Arthur Baars
184d42efe0 Remove unnecessary clause 2021-01-29 15:39:31 +01:00
Arthur Baars
b04391636d Fix qldoc comment 2021-01-29 15:39:31 +01:00
Tom Hvitved
f8790c81a8 Merge pull request #108 from github/hvitved/ssa
Add SSA library
2021-01-29 15:12:14 +01:00
Nick Rolfe
623ee59410 Merge pull request #106 from github/self 2021-01-28 20:16:48 +00:00
Nick Rolfe
30804f74e2 Remove redundant instanceof expression 2021-01-28 17:48:16 +00:00
Tom Hvitved
47fdee4bbe Sync SsaImplCommon.qll with C# implementation 2021-01-28 09:09:37 +01:00
Tom Hvitved
05b8a6c27b Apply suggestions from code review
Co-authored-by: Nick Rolfe <nickrolfe@github.com>
2021-01-28 08:49:42 +01:00
Nick Rolfe
640092352b RegularSuperCallRange::getReceiver() never holds 2021-01-27 18:49:37 +00:00
Nick Rolfe
743e627a8d Test calls to methods named 'super' 2021-01-27 18:45:08 +00:00
Nick Rolfe
70bbeaac3b Simplify, since super tokens are never variable accesses 2021-01-27 18:28:01 +00:00
Tom Hvitved
b9b4325b84 Add initial mapping of CFG nodes to AST nodes 2021-01-27 15:38:49 +01:00
Tom Hvitved
edc6e7eba8 Add UnusedParameter.ql query 2021-01-27 10:47:42 +01:00
Tom Hvitved
9dfea8006d Add UninitializedLocal.ql query 2021-01-27 10:44:49 +01:00
Tom Hvitved
8abedaee8a Add DeadStoreOfLocal.ql query 2021-01-27 10:42:02 +01:00
Tom Hvitved
2077ba4a1f Add SSA library 2021-01-27 10:39:19 +01:00
Nick Rolfe
6423ea3219 Merge pull request #107 from github/hvitved/index-files-working-dir
Add `--working-dir=.` to `index-files` call
2021-01-26 19:19:20 +00:00
Tom Hvitved
735eb24a33 Add --working-dir=. to index-files call 2021-01-26 19:31:16 +01:00
Nick Rolfe
7ac46bf8f8 Add SuperCall class for calls to super 2021-01-26 18:08:46 +00:00
Tom Hvitved
d19053deda Merge pull request #105 from github/hvitved/vcall 2021-01-25 18:41:36 +01:00
Tom Hvitved
2c6b9eceda Move vcall into internal/Variable.qll 2021-01-25 16:26:11 +01:00
Tom Hvitved
ce74208317 Merge pull request #97 from github/hvitved/var-access-categorization
Categorize variable accesses into reads and (implicit or explicit) writes
2021-01-25 16:25:35 +01:00
Tom Hvitved
979da623ed Merge pull request #103 from github/hvitved/cfg/params
CFG: Replace special parameters with their identifiers
2021-01-25 16:24:10 +01:00
Tom Hvitved
3a0c9a8104 CFG: Replace special parameters with their identifiers
For example, instead of including `**kwargs` in the CFG, we include `kwargs`.
This means that all variable accesses belonging to parameter definitions will
be included in the CFG.
2021-01-25 10:02:21 +01:00
Nick Rolfe
12fc0b914b Merge pull request #102 from github/hvitved/blocks-no-params
Recognize blocks without parameters
2021-01-22 15:44:14 +00:00
Tom Hvitved
586885f066 Recognize blocks without parameters 2021-01-22 16:16:01 +01:00
Tom Hvitved
0f3a4a1a60 Merge pull request #101 from github/stats
Update stats
2021-01-22 16:05:47 +01:00
Nick Rolfe
216b1de2dd Update stats 2021-01-22 14:35:43 +00:00
Nick Rolfe
858ca0b3bc Merge pull request #100 from github/call_ast
Add AST classes and tests for method calls
2021-01-22 14:33:10 +00:00
Nick Rolfe
243dfde72e Create ComplexSymbolRange class to deduplicate some predicates 2021-01-22 14:21:39 +00:00
Tom Hvitved
7e374c416a Categorize variable accesses into reads and (implicit or explicit) writes 2021-01-22 13:17:26 +01:00
Nick Rolfe
3939008fd5 Small tweaks based on PR feedback 2021-01-22 12:17:17 +00:00
Nick Rolfe
ccd8a2aae6 Merge remote-tracking branch 'origin/main' into call_ast 2021-01-22 11:48:32 +00:00
Tom Hvitved
08c655e4e3 Merge pull request #99 from github/hvitved/cfg/to-string
CFG: Use manual `toString()`s for `AstCfgNode` when available
2021-01-21 14:10:16 +01:00
Nick Rolfe
2e8d154f2b Add AST classes and tests for method calls 2021-01-20 18:34:25 +00:00
Tom Hvitved
bf7eb022a0 CFG: Use manual toString()s for AstCfgNode when available 2021-01-20 19:15:03 +01:00
Arthur Baars
78771ba4c2 Merge pull request #96 from github/hvitved/codeql-submodule-sync
Add `github/codeql` submodule and functionality for synchronizing files
2021-01-19 11:16:38 +01:00
Tom Hvitved
c11df1fe8c Add sync-identical-files.py 2021-01-18 17:34:51 +01:00
Tom Hvitved
a41eea4fd7 Merge pull request #95 from github/hvitved/cfg/not-bug
CFG: Fix bug in `LogicalNotTree`
2021-01-18 16:05:39 +01:00
Tom Hvitved
e9a8afe284 Add github/codeql as a sub module 2021-01-18 15:54:39 +01:00
Tom Hvitved
34fe416a85 CFG: Fix bug in LogicalNotTree 2021-01-18 15:03:58 +01:00
Tom Hvitved
3f31775252 CFG: Add test for constant condition 2021-01-18 15:01:41 +01:00
Arthur Baars
03d407e50d Merge pull request #82 from github/more_exprs
Add AST library for control expressions (conditionals and loops)
2021-01-11 10:35:37 +01:00
Nick Rolfe
6d7efab820 Add ConditionalLoop base class 2021-01-08 12:20:08 +00:00
Arthur Baars
c68f6a7f2e Merge pull request #84 from github/aibaars/codeql-threads
Actions: apply CODEQL_THREADS to all steps
2021-01-08 13:19:01 +01:00
Nick Rolfe
6465c90a16 Rename IfOrElsifExpr to IfExpr; remove child classes 2021-01-08 11:53:15 +00:00
Nick Rolfe
15785b4535 Add db base type for CaseExpr::Range 2021-01-08 11:31:43 +00:00
Arthur Baars
4ef4053385 Actions: apply CODEQL_THREADS to all steps 2021-01-08 10:25:25 +01:00
Nick Rolfe
6efebf1e36 Merge remote-tracking branch 'origin/main' into more_exprs 2021-01-07 19:02:50 +00:00
Nick Rolfe
6c0804c1af Address feedback on CFG change 2021-01-07 19:02:37 +00:00
Nick Rolfe
8cb8ead48e Address more feedback on ExprSequence 2021-01-07 19:02:14 +00:00
Nick Rolfe
19a4e63ac6 Move comment about getCondition from class to predicate 2021-01-07 18:01:38 +00:00
Nick Rolfe
9a71bdc993 Improvements from feedback on case/when classes. 2021-01-07 17:48:51 +00:00
Nick Rolfe
36c7d3fe5b Replace ConditionalExpr::get{Then,Else} with getBranch(boolean cond). 2021-01-07 17:32:41 +00:00
Nick Rolfe
e245382057 Merge pull request #83 from github/threads
Parallelize extraction
2021-01-07 17:14:41 +00:00
Nick Rolfe
f4abe7f4a1 Remove ThenExpr, ElseExpr, and DoExpr from public API 2021-01-07 15:56:31 +00:00
Nick Rolfe
83a28786a0 Use 4 threads for extraction and TRAP import in stats job 2021-01-07 11:17:07 +00:00
Nick Rolfe
1d3f06aca1 Simplify propagation of errors 2021-01-07 11:11:15 +00:00
Nick Rolfe
92c78e2b2d Simplify num_codeql_threads function slightly 2021-01-07 11:10:43 +00:00
Nick Rolfe
bb2bdc01b5 Have the extract function create the TS parser object 2021-01-07 10:56:23 +00:00
Nick Rolfe
bf4eac5113 Parallelize extraction
Use the Rayon library to do parallel iteration over the file list. The
number of threads used respects the CODEQL_THREADS environment variable.
2021-01-06 18:22:27 +00:00
Nick Rolfe
f484b573f2 update stats for dbscheme change 2021-01-05 16:25:46 +00:00
Nick Rolfe
7c503120ae Add AST library for control expressions (conditionals and loops) 2021-01-05 16:08:33 +00:00
Arthur Baars
c35283cefb Merge pull request #77 from github/aibaars/global-variables
Add global variables
2020-12-21 12:15:31 +01:00
Arthur Baars
f0ddeaa9f2 Merge pull request #81 from github/aibaars/revert-dup-code
Update ruby.dbscheme.stats
2020-12-21 12:15:10 +01:00
Arthur Baars
ad1782b620 Address comments 2020-12-21 11:01:46 +01:00
Arthur Baars
8469bd3688 Uncomment getAPrimaryQlClass() 2020-12-21 11:01:46 +01:00
Arthur Baars
dc0de9132e Add GlobalVariable 2020-12-21 11:01:46 +01:00
Arthur Baars
1ada9feda7 Make VariableAccess "abstract" 2020-12-21 11:01:46 +01:00
Arthur Baars
ebacec41d5 Update ruby.dbscheme.stats 2020-12-21 10:58:25 +01:00
Nick Rolfe
b1b2815c26 Merge pull request #80 from github/aibaars/revert-dup-code
Updates after CodeQL upgrade to 2.4.1
2020-12-21 09:57:59 +00:00
Arthur Baars
d4874641a3 Revert "Add duplicate code tables to dbscheme"
This reverts commit 4c699fcb32.
2020-12-21 10:45:59 +01:00
Arthur Baars
bf232f0582 Update formatting for CodeQL 2.4.1 2020-12-21 10:45:59 +01:00
Arthur Baars
ff8ea6d44f Merge pull request #79 from github/test_checks
Add all the TRAP check flags in qltest workflow
2020-12-21 10:20:47 +01:00
Nick Rolfe
5a54026bcc Add all the TRAP check flags in qltest workflow 2020-12-18 17:25:28 +00:00
Arthur Baars
dddf0a66d9 Merge pull request #78 from github/typo
fix typo in comment
2020-12-18 13:50:58 +01:00
Nick Rolfe
72319b538f fix typo in comment 2020-12-18 12:47:31 +00:00
Arthur Baars
8f1c916242 Merge pull request #66 from github/aibaars/cfg-2
CFG: make all simple nodes instance of StandardLeftToRight{Pre,Post}Tree
2020-12-18 13:26:05 +01:00
Nick Rolfe
c4ca537574 Merge pull request #75 from github/stmts_exprs
Add AST classes and tests for operations
2020-12-18 10:40:27 +00:00
Nick Rolfe
6c828214f7 Make import private 2020-12-18 10:23:19 +00:00
Nick Rolfe
53fbfc369d Make params test pass for now
- some toString improvements
- comment out getAPrimaryQlClass predicates that cause the test to fail
2020-12-18 10:13:13 +00:00
Nick Rolfe
4718de08b2 Address review feedback 2020-12-18 10:08:45 +00:00
Nick Rolfe
a87fe410af Simplify examples for unary plus/minus 2020-12-17 18:35:01 +00:00
Nick Rolfe
8b7af665b4 Simplify imports 2020-12-17 18:33:49 +00:00
Tom Hvitved
6893f57978 Merge pull request #74 from github/hvitved/cfg/fix-join-order
CFG: Fix bad join-order
2020-12-17 16:58:23 +01:00
Tom Hvitved
07c464b753 CFG: Fix bad join-order
Before:
```
[2020-12-17 11:33:46] (211s) Tuple counts for ControlFlowGraphImpl::Trees::RescueEnsureBlockTree::nestedEnsure_dispred#ff/2@2ea588:
                      11409019   ~0%     {2} r1 = SCAN ControlFlowGraphImpl::getScope#ff AS I OUTPUT I.<1>, I.<0> 'this'
                      3714296409 ~0%     {3} r2 = JOIN r1 WITH ControlFlowGraphImpl::Trees::getAChildInScope#fff_102#join_rhs AS R ON FIRST 1 OUTPUT r1.<1> 'this', R.<1>, R.<2>
                      2359       ~0%     {2} r3 = JOIN r2 WITH ControlFlowGraphImpl::Trees::RescueEnsureBlockTree::getAnEnsureDescendant#ff AS R ON FIRST 2 OUTPUT r2.<2>, r2.<0> 'this'
                      1          ~0%     {2} r4 = JOIN r3 WITH ControlFlowGraphImpl::Trees::RescueEnsureBlockTree::getEnsure_dispred#ff_10#join_rhs AS R ON FIRST 1 OUTPUT r3.<1> 'this', R.<1> 'innerBlock'
                                         return r4
```

After:
```
[2020-12-17 15:20:37] (51s) Tuple counts for ControlFlowGraphImpl::Trees::RescueEnsureBlockTree::nestedEnsure_dispred#ff/2@c4f57d:
                      635      ~1%     {3} r1 = JOIN ControlFlowGraphImpl::Trees::RescueEnsureBlockTree::getEnsure_dispred#ff_10#join_rhs AS L WITH ControlFlowGraphImpl::Trees::getAChildInScope#fff_201#join_rhs AS R ON FIRST 1 OUTPUT R.<1>, L.<1> 'innerBlock', R.<2>
                      1        ~0%     {3} r2 = JOIN r1 WITH ControlFlowGraphImpl::Trees::RescueEnsureBlockTree::getAnEnsureDescendant#ff_10#join_rhs AS R ON FIRST 1 OUTPUT R.<1> 'this', r1.<2>, r1.<1> 'innerBlock'
                      1        ~0%     {2} r3 = JOIN r2 WITH ControlFlowGraphImpl::getScope#ff AS R ON FIRST 2 OUTPUT r2.<0> 'this', r2.<2> 'innerBlock'
                                       return r3
```
2020-12-17 16:46:03 +01:00
Arthur Baars
ff751b97d2 CFG: make all simple nodes instance of StandardLeftToRight{Pre,Post}Tree 2020-12-17 16:39:54 +01:00
Arthur Baars
a15a066414 Merge pull request #72 from github/aibaars/fix-cfg
CFG improvements
2020-12-17 16:39:19 +01:00
Arthur Baars
b676c95218 Address comments 2020-12-17 16:35:51 +01:00
Nick Rolfe
73798312b9 Add classes and tests for operations 2020-12-17 15:16:37 +00:00
Tom Hvitved
46fc17da58 CFG: Fix multiple abnormal successors 2020-12-17 11:15:17 +01:00
Tom Hvitved
1033b8610a CFG: Add more tests 2020-12-17 11:14:10 +01:00
Arthur Baars
91ae237434 Use latest CodeQL for CI 2020-12-17 11:04:57 +01:00
Arthur Baars
dd954ea943 CFG: correct flow for lambda bodies
Lambda bodies are parsed as nested do-blocks or normal blocks.
This is actually incorrect, as the body of a lambda can't have
parameters. However, we can "inline" such blocks to get the
desired control flow.
2020-12-17 10:04:01 +01:00
Arthur Baars
eafec4331b CFG: add nodes for block arguments 2020-12-17 10:04:01 +01:00
Arthur Baars
d016e3cae0 CFG: methods are evaluated before their arguments 2020-12-17 10:04:01 +01:00
Arthur Baars
81c907a87a CFG: fix BEGIN and END blocks 2020-12-17 10:04:01 +01:00
Arthur Baars
f2fd1c7931 CFG: make def nodes visible 2020-12-17 10:04:01 +01:00
Arthur Baars
f2effce786 CFG: improve handling of block and lambda 2020-12-17 10:04:01 +01:00
Arthur Baars
30895e634c CFG: refactor CfgScope 2020-12-17 10:04:01 +01:00
Arthur Baars
bc47338b52 CFG: add test-case for conditional method declarations 2020-12-17 10:04:01 +01:00
Arthur Baars
69de81bdd5 CFG: have alternative flow for the definition and call of methods etc. 2020-12-17 10:04:01 +01:00
Arthur Baars
fd14770542 CFG: drop getObject from flow of singleton method 2020-12-17 09:59:30 +01:00
Arthur Baars
8501e30b6a CFG: fix linking heredoc start to heredoc body 2020-12-17 09:59:30 +01:00
Arthur Baars
edbd997f15 Merge pull request #71 from github/kinds
Create disjoint db types for different operators
2020-12-17 09:58:52 +01:00
Nick Rolfe
282d20d766 Remove redundant field on ChildNode struct 2020-12-16 20:57:06 +00:00
Nick Rolfe
a873cb9f3d Update dbscheme stats 2020-12-16 20:53:41 +00:00
Nick Rolfe
d1a9572b0e Merge remote-tracking branch 'origin/main' into kinds 2020-12-16 17:55:20 +00:00
Nick Rolfe
f5282edfc1 Simplifications based on PR feedback 2020-12-16 17:54:40 +00:00
Arthur Baars
381d6aafaa Merge pull request #73 from github/calls
Update tree-sitter-ruby to pick up improvements to calls
2020-12-16 14:00:53 +01:00
Nick Rolfe
0518d51b51 Update CFG: call receiers are evaluated before arguments 2020-12-16 12:40:57 +00:00
Nick Rolfe
e98a84c8b5 Update CFG to match changes to Call/MethodCall 2020-12-16 12:01:30 +00:00
Nick Rolfe
aa0c1491a6 Update tree-sitter-ruby to pick up improvements to calls 2020-12-16 10:13:45 +00:00
Arthur Baars
7971b243f1 Merge pull request #69 from github/hvitved/cfg/post-order-cond
CFG: Model `IfElsifAstNode` in post-order
2020-12-15 19:22:16 +01:00
Nick Rolfe
ddb71790e9 Fix formatting 2020-12-15 16:01:13 +00:00
Tom Hvitved
9aadeedeb9 CFG: Model IfElsifAstNode in post-order 2020-12-15 17:00:12 +01:00
Tom Hvitved
bb88858633 CFG: Add test for nested ifs 2020-12-15 16:46:55 +01:00
Nick Rolfe
3f5eab04b5 Create disjoint db types for different operators 2020-12-15 15:22:33 +00:00
Arthur Baars
ac9f439935 Merge pull request #70 from github/hvitved/cfg/rescue-part2
CFG: More adjustments for `rescue`/`ensure`
2020-12-15 16:06:26 +01:00
Tom Hvitved
16c25f2a4c CFG: Handle ensure blocks without body/rescues 2020-12-15 13:49:14 +01:00
Tom Hvitved
489b406e2a CFG: Change column order in succExit/hasExitScope 2020-12-15 13:45:22 +01:00
Tom Hvitved
e784640cca CFG: Add more test cases 2020-12-15 13:45:22 +01:00
Arthur Baars
5108b369e1 Merge pull request #64 from github/hvitved/cfg/rescue
Implement CFG logic for `rescue-ensure`
2020-12-15 11:43:14 +01:00
Tom Hvitved
a76e6848c7 CFG: Address more review comments 2020-12-14 20:45:57 +01:00
Tom Hvitved
ec4ead2117 Apply suggestions from code review
Co-authored-by: Arthur Baars <aibaars@github.com>
2020-12-14 14:53:35 +01:00
Nick Rolfe
b76f97d337 Merge pull request #68 from github/bump_ts
Bump tree-sitter-ruby revision to get operator_assignment field
2020-12-14 12:40:36 +00:00
Tom Hvitved
89fb2f8498 CFG: Add @kind graph to Cfg.ql, and remove labels from ordinary successor edges 2020-12-14 11:00:26 +01:00
Nick Rolfe
6bacac7598 Bump tree-sitter-ruby revision to get operator_assignment field 2020-12-08 18:28:54 +00:00
Tom Hvitved
b14a889f5f CFG: Use MatchingCompletion for parameters with default values 2020-12-08 13:47:32 +01:00
Tom Hvitved
80a59a81ed CFG: Use MatchingCompletion for patterns 2020-12-08 13:47:32 +01:00
Tom Hvitved
31b8d33a7c CFG: Mark redo edges out of for loops 2020-12-08 13:47:32 +01:00
Tom Hvitved
b6ea5c5eab CFG: Implement logic for rescue-ensure blocks 2020-12-08 13:47:32 +01:00
Nick Rolfe
53a1cbc492 Merge pull request #67 from github/getAPrimaryQlClass
Rename describeQlClass to getAPrimaryQlClass
2020-12-08 12:16:18 +00:00
Nick Rolfe
3145b3dde7 Rename describeQlClass to getAPrimaryQlClass 2020-12-08 11:09:18 +00:00
Tom Hvitved
5a0376f67e CFG: More tests 2020-12-08 11:06:15 +01:00
Arthur Baars
990ed34c02 Merge pull request #55 from github/aibaars/cfg
Control flow graph
2020-12-07 16:51:33 +01:00
Arthur Baars
9390cf0401 CFG: add test case for if-in-case 2020-12-07 16:46:52 +01:00
Arthur Baars
86e73afc74 CFG: extract HeredocBeginning::getName predicate 2020-12-07 16:31:17 +01:00
Arthur Baars
9883d7124e CFG: improve handling of redo 2020-12-07 16:20:42 +01:00
Arthur Baars
003f7230b2 Apply suggestions from code review
Co-authored-by: Tom Hvitved <hvitved@github.com>
2020-12-07 16:02:19 +01:00
Arthur Baars
024150b04b CFG: hide 'begin' 2020-12-07 16:02:19 +01:00
Arthur Baars
87451fd999 CFG: specialise return type instead of instanceof check 2020-12-07 15:36:09 +01:00
Arthur Baars
6aea3eff3e CFG: rename getBody{=>Node} and getCondition{=>Node} 2020-12-07 15:30:57 +01:00
Arthur Baars
6d12bcc2fe Make ConditionalSuccessor not abstract 2020-12-07 15:19:14 +01:00
Arthur Baars
044d14c8b4 Use private imports in generated code 2020-12-07 15:14:34 +01:00
Arthur Baars
ed3b102ecc Improve formatting 2020-12-07 15:12:43 +01:00
Arthur Baars
d25835c7d2 Merge pull request #61 from github/aibaars/code-nav
Add basic code navigation queries
2020-12-07 14:47:43 +01:00
Arthur Baars
2394b26636 CFG: skip Uninterpreted nodes 2020-12-07 13:11:21 +01:00
Arthur Baars
36f5a63c18 Improve handling of class, module, block and method 2020-12-07 13:11:21 +01:00
Arthur Baars
2124247d5e CFG: add samples of all syntactical constructs to cfg.rb 2020-12-07 13:11:21 +01:00
Arthur Baars
ebf3a31224 CFG: don't handle rescue, else, ensure for now 2020-12-07 13:11:21 +01:00
Arthur Baars
97d0220ffd CFG: Model nodes with simple flow 2020-12-07 13:11:21 +01:00
Arthur Baars
3807e1be38 CFG: flow for rescue-modifier 2020-12-07 13:11:21 +01:00
Arthur Baars
d619bdd8f9 CFG: Completions: fix definition of boolean constants 2020-12-07 13:11:21 +01:00
Arthur Baars
6c579ff608 CFG: link heredoc start to its body 2020-12-07 13:11:21 +01:00
Arthur Baars
49d11b1e09 CFG: don't hide Class and Module nodes 2020-12-07 13:11:21 +01:00
Arthur Baars
0852068bcd CFG: make lambda a CFG entry point 2020-12-07 13:11:21 +01:00
Arthur Baars
01066ea3bb CFG: case expression 2020-12-07 13:11:21 +01:00
Arthur Baars
2f238280dc CFG: model if-modifier and unless 2020-12-07 13:11:21 +01:00
Arthur Baars
5d6e77be28 CFG: model while, until and variants 2020-12-07 13:11:21 +01:00
Arthur Baars
6660cb4417 CFG: for-in loop 2020-12-07 13:11:21 +01:00
Arthur Baars
165b2b37dc Treat for variables and exception variables as declarations 2020-12-07 13:11:21 +01:00
Arthur Baars
b60ea74e8a Treat conditional expressions as if-then-else 2020-12-07 13:11:21 +01:00
Arthur Baars
97fab0d18b Assignments evaluate right-hand-side first 2020-12-07 13:11:21 +01:00
Arthur Baars
465c266b8a Classes and module are not CfgScopes 2020-12-07 13:11:21 +01:00
Arthur Baars
0959a4675f Merge pull request #65 from github/aibaars/dup-code
Add duplicate code tables to dbscheme
2020-12-07 13:10:52 +01:00
Arthur Baars
4c699fcb32 Add duplicate code tables to dbscheme 2020-12-07 13:06:26 +01:00
Arthur Baars
0a38d6801c Address review comments 2020-12-07 12:53:45 +01:00
Arthur Baars
d92d635103 Add basic code navigation queries 2020-12-04 15:01:43 +01:00
Arthur Baars
1d502cb40d Merge pull request #63 from github/aibaars/fix-warnings
Fix warnings and make imports private
2020-12-04 10:43:01 +01:00
Arthur Baars
c1f1efb16b Merge pull request #62 from github/aibaars/update-grammar
Update tree-sitter grammar
2020-12-03 19:14:13 +01:00
Arthur Baars
22fd8908c5 Use private imports
No need to have everyting re-export the entire AST
2020-12-03 19:13:05 +01:00
Arthur Baars
582b00ef07 Fix warnings 2020-12-03 19:05:49 +01:00
Arthur Baars
dd3f94a3e2 Update tree-sitter grammar 2020-12-03 18:50:47 +01:00
Nick Rolfe
b0227a7ee1 Merge pull request #60 from github/aibaars/osx-gnutar
Workaround for broken cache on OSX
2020-12-03 16:10:10 +00:00
Arthur Baars
c69f64fb4f Workaround for broken cache on OSX 2020-12-03 16:40:37 +01:00
Nick Rolfe
492f7d1987 Merge pull request #59 from github/bump_ts
Bump to latest tree-sitter-ruby revision
2020-12-02 20:04:12 +00:00
Nick Rolfe
d7c1231020 Bump to latest tree-sitter-ruby revision 2020-12-02 16:11:07 +00:00
Tom Hvitved
86a2cbc773 Merge pull request #58 from github/hvitved/pattern-get-a-variable
Add `Pattern::getAVariable()` and use `self` range field throughout
2020-12-02 12:57:52 +01:00
Tom Hvitved
9129e886b2 Update ql/src/codeql_ruby/ast/Parameter.qll
Co-authored-by: Arthur Baars <aibaars@github.com>
2020-12-02 12:07:13 +01:00
Tom Hvitved
77129e473a Adhere to ::Range pattern 2020-12-02 11:27:00 +01:00
Tom Hvitved
b2483069e0 Add Pattern::getAVariable() and use self range field througout 2020-12-02 10:36:33 +01:00
Arthur Baars
59263650b1 Merge pull request #57 from github/hvitved/rename-generated-qll
Move `Generated.qll` to `ast/internal/TreeSitter.qll`
2020-12-02 10:32:38 +01:00
Tom Hvitved
a370cd8bdf Move Generated.qll to ast/internal/TreeSitter.qll 2020-12-01 20:53:41 +01:00
Tom Hvitved
ba7a42328d Merge pull request #56 from github/hvitved/parameter-get-a-variable
Introduce `Parameter::getAVariable()`
2020-12-01 18:32:34 +01:00
Tom Hvitved
d50f5cc785 Address review comments 2020-12-01 15:14:14 +01:00
Tom Hvitved
9820dcb363 Generate VariableAccesses also for defining accesses 2020-12-01 14:39:41 +01:00
Tom Hvitved
bde9f59e0e Introduce Parameter::getAVariable() 2020-12-01 13:18:06 +01:00
Tom Hvitved
965b351cde Merge pull request #54 from github/hvitved/ast-final
Mark more AST predicates as `final`
2020-12-01 12:38:28 +01:00
Tom Hvitved
311a0b6b20 Mark more AST predicates as final 2020-12-01 10:24:33 +01:00
Tom Hvitved
11927a930f Merge pull request #53 from github/user-facing
Add some user-facing AST classes
2020-12-01 10:23:37 +01:00
Nick Rolfe
baf29ae56b Add qldoc comment and isOptional predicate to KeywordParameter 2020-11-30 13:42:02 +00:00
Tom Hvitved
c0dd89122c Handle parameters with overlapping names 2020-11-28 19:23:08 +01:00
Tom Hvitved
58baa33a3f Various changes to user-facing library
- Remove `abstract` classes from public API.
- Align `Variable.qll` with rest of library.
- Introduce `Callable` class.
- Make `Pattern` class cover everything that can be on the LHS of an assignment
  and in a pattern (except special parameters such as `**param`).
2020-11-27 17:07:03 +01:00
Tom Hvitved
59d45de118 Move AST files into ast folder 2020-11-27 14:45:15 +01:00
Tom Hvitved
00f3daabfe Rename Variables.qll to Variable.qll 2020-11-27 14:39:20 +01:00
Nick Rolfe
38b401f04f Fix import 2020-11-26 16:04:46 +00:00
Arthur Baars
f9c7ae78fe Merge pull request #52 from github/aibaars/db-stats
Collect database stats
2020-11-26 17:03:34 +01:00
Nick Rolfe
399170fd58 Add getParent(Index) to user-facing AstNode 2020-11-26 15:33:50 +00:00
Arthur Baars
c7986442d0 Update ruby.dbscheme.stats 2020-11-26 15:07:13 +01:00
Arthur Baars
49c97bd157 Collect database stats 2020-11-26 14:53:30 +01:00
Nick Rolfe
c598dc6b5c Initial work on user-facing AST library 2020-11-26 13:45:45 +00:00
Arthur Baars
2082171bdf Merge pull request #51 from github/aibaars/cfg-scopes
CFG: add more CfgScopeRanges
2020-11-26 12:13:53 +01:00
Tom Hvitved
8632cbec71 CFG: Do not descend into nested scopes 2020-11-26 10:58:23 +01:00
Arthur Baars
30cb2cc3e0 CFG: add more CfgScopeRanges 2020-11-26 10:58:23 +01:00
Arthur Baars
e181666a37 Merge pull request #49 from github/aibaars/parent
Add parent ref and parent_index fields to all AstNodes
2020-11-25 18:25:03 +01:00
Arthur Baars
083672744e Remove @file from @astnode 2020-11-25 17:37:58 +01:00
Arthur Baars
735aec9d34 Ensure top-level nodes have distinct parent_index values 2020-11-25 13:48:25 +01:00
Arthur Baars
00015b0022 Add #keyset[parent, parent_index] 2020-11-25 13:48:25 +01:00
Arthur Baars
89953fd87c Add parent_index field to @astnode 2020-11-25 13:48:25 +01:00
Arthur Baars
b72db8b6f1 Add parent field to AstNode 2020-11-25 13:48:25 +01:00
Arthur Baars
c7b07b7821 Merge pull request #47 from github/aibaars/name-resolution
Name resolution: handle the different types of parameters better
2020-11-25 13:44:42 +01:00
Arthur Baars
64ebf5b909 Address comments 2020-11-25 12:55:53 +01:00
Arthur Baars
7a13e8549b Merge pull request #50 from github/pin_ts_rev
Pin tree-sitter-ruby revision
2020-11-24 20:46:53 +01:00
Nick Rolfe
f612e05b34 Pin tree-sitter-ruby revision 2020-11-24 19:22:30 +00:00
Arthur Baars
bc5d7a3b74 Change modelling of Parameters 2020-11-24 19:22:40 +01:00
Arthur Baars
c745978ebb Fix inconsistent variable references 2020-11-24 19:22:40 +01:00
Arthur Baars
290d3decc8 Add consistency query for Variables
Test that VariableAccess.getVariable returns a unique Variable
2020-11-24 19:19:15 +01:00
Tom Hvitved
0616040f3c Merge pull request #48 from github/hvitved/ci-check-queries
Check query compilation and formatting in `qltest.yml`
2020-11-24 11:51:54 +01:00
Tom Hvitved
eceeb6a5fd Break up QL CI tests into separatly named steps 2020-11-24 11:47:59 +01:00
Tom Hvitved
966e1cdcd0 Apply old formatter to make CI check pass 2020-11-24 11:26:47 +01:00
Tom Hvitved
74f0a8fdb7 Check query compilation and formatting in qltest.yml 2020-11-24 11:20:16 +01:00
Tom Hvitved
d5582f3f48 Merge pull request #46 from github/hvitved/unique-parent
Add `unique` wrapper to `AstNode::getParent()`
2020-11-23 16:16:02 +01:00
Tom Hvitved
8132c4cafb Update generator/src/ql.rs
Co-authored-by: Arthur Baars <aibaars@github.com>
2020-11-23 16:12:31 +01:00
Tom Hvitved
d0257dda36 Add unique wrapper to AstNode::getParent() 2020-11-23 15:23:21 +01:00
Arthur Baars
41a76eeb01 Merge pull request #42 from github/aibaars/name-resolution
Local variable binding
2020-11-23 15:22:43 +01:00
Arthur Baars
3ea6cb40f8 Merge pull request #45 from github/hvitved/name-resolution-suggestions
Suggested changes to Variables.qll
2020-11-23 13:28:40 +01:00
Tom Hvitved
59624454d1 Suggested changes to Variables.qll
- Remove `abstract` predicates from public API.
- Cache core computations.
- Redefine `VariableScope::get[A]Variable` to only include variables declared
  directly in the scope.
2020-11-23 10:33:34 +01:00
Arthur Baars
bc423000ca Add variable to varaccess tests 2020-11-23 09:58:31 +01:00
Arthur Baars
49f1143133 Make Variable an IPA type and speed things up on large databases 2020-11-23 09:58:31 +01:00
Tom Hvitved
bb06c1ffeb Various minor changes to Variables.qll 2020-11-23 09:58:31 +01:00
Arthur Baars
c16a2e77d8 Model local variables 2020-11-23 09:58:31 +01:00
Arthur Baars
6bd476ff30 Add AstNode::getParent 2020-11-23 09:58:31 +01:00
Nick Rolfe
10411ef49e Merge pull request #43 from github/hvitved/unbreak-print-ast
Unbreak PrintAST query
2020-11-19 13:58:43 +00:00
Tom Hvitved
7716d53552 Unbreak PrintAST query 2020-11-19 14:48:14 +01:00
Tom Hvitved
100daacb94 Merge pull request #39 from github/hvitved/cfg-skeleton
Initial CFG skeleton code
2020-11-19 14:41:16 +01:00
Tom Hvitved
06a6a3feb0 Address review comments 2020-11-19 14:31:08 +01:00
Tom Hvitved
4626168969 CFG: Separate scope for method blocks 2020-11-19 09:29:15 +01:00
Tom Hvitved
4dd4373b53 Initial CFG skeleton code 2020-11-18 20:12:42 +01:00
Arthur Baars
f9c1bbd8f9 Merge pull request #41 from github/gitignore
Update .gitignore
2020-11-17 18:31:35 +01:00
Nick Rolfe
9d1eec8fe8 Update .gitignore 2020-11-17 16:45:10 +00:00
Nick Rolfe
12d4224e8e Merge pull request #40 from github/refactor
Move all naming decisions to shared library
2020-11-17 11:19:18 +00:00
Nick Rolfe
1a9663ff7d Replace single-branch match with if let 2020-11-16 18:43:54 +00:00
Nick Rolfe
68c97a2d13 Use .. to ignore fields
Co-authored-by: Arthur Baars <aibaars@github.com>
2020-11-16 18:41:18 +00:00
Nick Rolfe
ad61f7a0a6 Use references instead of owned strings in generator 2020-11-16 17:54:16 +00:00
Nick Rolfe
bbe7c70d34 more refactoring of names 2020-11-16 17:54:16 +00:00
Nick Rolfe
83a0e5fea6 Refactor to move naming decisions to shared library 2020-11-16 17:54:14 +00:00
Nick Rolfe
505d5c04d8 Merge pull request #31 from github/aibaars/drop-classes
Simplify generated QL classes
2020-11-16 14:16:02 +00:00
Arthur Baars
043c3fd2eb Simplify generated QL classes 2020-11-13 12:59:22 +01:00
Arthur Baars
f57d20f5c6 Merge pull request #36 from github/readme-build-dbs
Add README instructions for building databases
2020-11-13 12:57:09 +01:00
Nick Rolfe
c16390fd05 Merge remote-tracking branch 'origin/main' into readme-build-dbs 2020-11-13 11:37:28 +00:00
Nick Rolfe
8d46151a10 Merge pull request #37 from github/aibaars-patch-1
Change cache key
2020-11-13 11:33:31 +00:00
Arthur Baars
5fe3bf138c Change cache key 2020-11-12 19:11:04 +01:00
Arthur Baars
402c348e37 Merge pull request #33 from github/aibaars/qltest
Add QL test support
2020-11-12 15:10:39 +01:00
Nick Rolfe
0e1b54f061 Add instructions for building databases 2020-11-12 13:33:32 +00:00
Nick Rolfe
bb1d6f3bb8 Merge pull request #34 from github/aibaars/osx-fmt
Remove cargo fmt workaround on OSX
2020-11-12 13:03:41 +00:00
Nick Rolfe
056879eb97 Merge pull request #35 from github/aibaars/cargo-update
Run: cargo update
2020-11-12 10:34:50 +00:00
Arthur Baars
8d1ed4bf89 Run: cargo update
This pulls in improvements to the tree-sitter-ruby repository.
2020-11-12 10:25:40 +01:00
Arthur Baars
557d990a0d Remove cargo fmt workaround on OSX
The `fmt` component is now installed by default on OSX.
2020-11-12 09:29:26 +01:00
Arthur Baars
44150600ab Add QLTest workflow 2020-11-11 21:57:50 +01:00
Arthur Baars
080c56c9eb Add QL test support 2020-11-11 16:32:44 +01:00
Arthur Baars
db35abdf17 Merge pull request #32 from github/getFileBySourceArchiveName
Replace getEncodedFile with getFileBySourceArchiveName predicate
2020-11-11 13:46:10 +01:00
Nick Rolfe
5771e4790e Replace getEncodedFile with getFileBySourceArchiveName predicate
While also making it work with paths for databases created on Windows.
2020-11-10 16:50:10 +00:00
Arthur Baars
5f1e373355 Merge pull request #30 from github/string_contents
Get latest fixes from tree-sitter-ruby repo
2020-11-09 15:05:50 +01:00
Arthur Baars
81ceb22b14 Restore cache before running cargo fmt
It appears cargo fmt also downloads the git dependencies which takes quite a while. The cache should contain a copy of the cloned repo, so restoring the cache early should speed things up.
2020-11-09 14:25:54 +01:00
Nick Rolfe
6f72ba106e Get latest fixes from tree-sitter-ruby repo 2020-11-06 17:15:22 +00:00
Nick Rolfe
aec99746d6 Merge pull request #29 from github/aibaars/dedup
Deduplicate and sort union members
2020-11-05 18:00:07 +00:00
Arthur Baars
222af90790 Deduplicate and sort union members 2020-11-05 18:50:12 +01:00
Arthur Baars
f514655231 Merge pull request #28 from github/token_classes
Add classes for token kinds
2020-11-05 17:27:22 +01:00
Nick Rolfe
510621f018 Don't add 'Token' prefix to token subclass names 2020-11-05 16:21:33 +00:00
Nick Rolfe
4bda204118 Add classes for token kinds 2020-11-05 13:06:46 +00:00
Arthur Baars
296d4d0f47 Merge pull request #26 from github/aibaars/tokens
Store tokens into separate table
2020-11-05 14:03:26 +01:00
Arthur Baars
c565f323f6 Don't register extra tokens as children of the parent node 2020-11-05 12:53:58 +01:00
Arthur Baars
180df8a63d Make classes non-abstract 2020-11-04 18:18:45 +01:00
Nick Rolfe
69b1d7c0dc Make union-wrapping classes abstract to fix results for toString/describeQlClass 2020-11-04 16:01:51 +00:00
Arthur Baars
86aa05e3cb Address comments 2020-11-04 14:49:47 +01:00
Arthur Baars
c3e8d85f0b Tolerate tokens containing invalid UTF-8 2020-11-04 14:46:31 +01:00
Arthur Baars
8056186c3c Hide disconnected tokens 2020-11-04 13:35:24 +01:00
Arthur Baars
96423d2e8e Remove describeQlClass from union types
The descriptions of the underlying types are more interesting.
2020-11-04 13:35:24 +01:00
Arthur Baars
053c9f60a4 Store tokens in a separate table 2020-11-04 13:35:24 +01:00
Nick Rolfe
9e49991859 Merge pull request #27 from github/extractor-pack-script
Add scripts to create extractor pack locally
2020-11-04 12:20:45 +00:00
Nick Rolfe
b16588f058 Add powershell script to create extractor pack locally 2020-11-04 12:09:52 +00:00
Nick Rolfe
a83ac24652 Add bash script to create extractor pack locally 2020-11-04 11:59:17 +00:00
Arthur Baars
b92d789598 Merge pull request #25 from github/printAST
Implement basic `printAst` query
2020-11-03 19:13:44 +01:00
Nick Rolfe
41dcb19cd5 Implement basic printAst query 2020-11-03 13:47:54 +00:00
Arthur Baars
65c1f2c359 Merge pull request #20 from github/aibaars/extract-extra
Extract 'extra' nodes and their subtrees
2020-11-03 13:45:33 +01:00
Arthur Baars
d7e9178cda Merge pull request #24 from github/gzip
Add buffered writing and gzip compression for trap files
2020-11-03 13:45:19 +01:00
Arthur Baars
bfc05539ec Update library and dbscheme 2020-11-03 10:07:05 +01:00
Arthur Baars
25205a09a3 Update tree-sitter-ruby 2020-11-03 10:06:59 +01:00
Arthur Baars
dc3459de8e Extract 'extra' nodes and their subtrees 2020-11-03 10:03:11 +01:00
Nick Rolfe
27c3c88b3c Add buffered writing and gzip compression for trap files 2020-11-02 16:14:19 +00:00
Arthur Baars
0156de12ea Merge pull request #22 from github/aibaars/trapwriter
Add a TrapWriter
2020-11-02 15:00:38 +01:00
Arthur Baars
0ccd97639b Address comments 2020-11-02 13:30:46 +01:00
Arthur Baars
0ecab93d09 Merge pull request #23 from github/aibaars/locations-lib
Add Locations.qll and import FileSystem and Locations libraries in generated AST
2020-11-02 13:08:15 +01:00
Arthur Baars
f94b5ae412 Update QL code generator 2020-10-31 14:03:26 +01:00
Arthur Baars
1b502c161e Add Locations library and move language independent files to 'codeql' 2020-10-31 11:51:01 +01:00
Arthur Baars
63ca8212f6 Limit string sizes to 1MB 2020-10-31 11:36:01 +01:00
Arthur Baars
f265ccef59 TrapWriter: add global ID caching and populate folders 2020-10-31 11:35:57 +01:00
Arthur Baars
0de8b0c069 Add TrapWriter::comment 2020-10-31 11:35:22 +01:00
Arthur Baars
748dee64ae Escape label keys 2020-10-31 11:35:22 +01:00
Arthur Baars
57842e8a87 Add TrapWriter 2020-10-31 11:35:16 +01:00
Nick Rolfe
83667ab89a Merge pull request #19 from github/locations
Fix location handling to match common db schema requirements
2020-10-30 16:56:34 +00:00
Arthur Baars
c2c197dba5 Merge pull request #21 from github/aibaars/files-qll
Basic FileSystem.qll
2020-10-30 17:50:54 +01:00
Nick Rolfe
075c72e6ef Iterate through path components to 'normalize' paths on windows 2020-10-30 15:26:46 +00:00
Arthur Baars
3e12aa457f Basic FileSystem.qll 2020-10-30 15:40:29 +01:00
Nick Rolfe
e73500ef7c Cope with empty filenames/extensions 2020-10-30 14:38:24 +00:00
Nick Rolfe
0a754334cf Don't generate the QL File class 2020-10-30 13:41:27 +00:00
Nick Rolfe
35cb379db7 Fix name of table for locations 2020-10-30 13:24:16 +00:00
Nick Rolfe
a54f923a73 Normalize the absolute path in the files table 2020-10-30 13:22:58 +00:00
Nick Rolfe
4b8bbd101c Give locations full ids matching the common spec 2020-10-30 13:06:21 +00:00
Nick Rolfe
79d15051be Fix full ids for files to match common spec 2020-10-30 12:45:23 +00:00
Nick Rolfe
7f03206b52 Use a key id for file entities 2020-10-30 11:29:04 +00:00
Nick Rolfe
826b4571a0 Canonicalize source file paths in main 2020-10-30 11:21:51 +00:00
Nick Rolfe
d47bd32b58 Now that we also generate conjunctions, use parentheses in disjunctions 2020-10-30 10:34:42 +00:00
Nick Rolfe
f198dc530f Use fromSource = 1
Co-authored-by: Arthur Baars <aibaars@github.com>
2020-10-30 10:25:09 +00:00
Nick Rolfe
2232700428 Correct comment
Co-authored-by: Arthur Baars <aibaars@github.com>
2020-10-30 10:24:24 +00:00
Nick Rolfe
4d5d80c749 Fix location handling to match common db schema requirements 2020-10-29 19:44:16 +00:00
Nick Rolfe
556507cec7 Merge pull request #18 from github/optional_fields
Don't generate an index for optional fields that occur at most once
2020-10-29 15:35:27 +00:00
Nick Rolfe
547d12ca58 Add more info to error message 2020-10-29 15:13:04 +00:00
Nick Rolfe
11c9c18de4 Don't generate an index for optional fields that occur at most once 2020-10-29 13:04:26 +00:00
Arthur Baars
fbb075b477 Merge pull request #17 from github/aibaars/locations-2
TRAP locations: always fix-up empty ranges
2020-10-29 12:15:52 +01:00
Arthur Baars
3350d9d3d4 TRAP locations: always fix-up empty ranges 2020-10-29 10:45:07 +01:00
Arthur Baars
ca91e15a4b Merge pull request #16 from github/aibaars/locations
Fix locations in the
2020-10-28 18:09:58 +01:00
Arthur Baars
4c04b8bb15 Add comment 2020-10-28 17:40:01 +01:00
Arthur Baars
d2f42552f6 Adjust source locations
Tree-sitter row and column numbers are 0-based while CodeQL expects 1-based.
In addition tree-sitter location ranges end-points are exclusive while
CodeQL's ranges are inclusive.
2020-10-28 17:30:03 +01:00
Nick Rolfe
743eca7992 Merge pull request #15 from github/aibaars/ql-folder
Add QL folder structure
2020-10-28 13:11:50 +00:00
Arthur Baars
638fd91e50 Update generator to write the ast.qll file directly into ql/src 2020-10-28 14:04:36 +01:00
Arthur Baars
28a99cfe83 Update path of generated dbscheme 2020-10-28 14:04:36 +01:00
Arthur Baars
88acbc883c Copy dbscheme stats into extractor pack 2020-10-28 14:04:36 +01:00
Arthur Baars
030d957535 Update stats with values measured on bunch of ruby databases 2020-10-28 14:04:36 +01:00
Arthur Baars
5d3f2de685 Add dbscheme to QL folder 2020-10-28 14:04:36 +01:00
Arthur Baars
2e102b8cdf Add folder structure for QL code 2020-10-28 14:04:36 +01:00
Arthur Baars
553e1ab465 Merge pull request #13 from github/aibaars/improve-workflow
Check formatting and cache builds
2020-10-28 14:04:05 +01:00
Arthur Baars
7e6c30b121 Check formatting and cache builds 2020-10-28 13:55:52 +01:00
Nick Rolfe
29899485c7 Merge pull request #11 from github/ql_gen
Generate QL classes
2020-10-28 12:25:53 +00:00
Nick Rolfe
e03d5da8cd Rename a field to avoid using raw identifiers 2020-10-28 12:14:54 +00:00
Nick Rolfe
f4b9c0c71a Merge remote-tracking branch 'origin/main' into ql_gen 2020-10-28 11:41:18 +00:00
Nick Rolfe
24b4586ddd Merge pull request #14 from github/aibaars/remove-storage-index
Extractor: fix child index values
2020-10-28 11:37:38 +00:00
Nick Rolfe
11152583d5 Add get_name() method to simplify logic in field handling 2020-10-28 11:30:50 +00:00
Nick Rolfe
53de99e6af Regenerate QL with fix to Top::getAFieldOrChild 2020-10-28 11:22:21 +00:00
Nick Rolfe
7b51030dd4 Merge remote-tracking branch 'origin/ql_gen' into ql_gen 2020-10-28 11:20:58 +00:00
Nick Rolfe
b4f9599dd9 Simplify hashmap insertion 2020-10-28 11:20:47 +00:00
Nick Rolfe
679ca6d0f1 Update Actions workflow to generate ruby_ast.qll 2020-10-28 11:04:09 +00:00
Nick Rolfe
17820e017c Fix Top::getAFieldOrChild() so it doesn't take an index arg
Co-authored-by: Arthur Baars <aibaars@github.com>
2020-10-28 11:02:42 +00:00
Nick Rolfe
bc22631c32 Simplify QL model following review feedback 2020-10-28 11:00:40 +00:00
Nick Rolfe
77fdafdc95 Simplify error handling with if let 2020-10-28 10:35:33 +00:00
Nick Rolfe
59580d51bb Merge remote-tracking branch 'origin/main' into ql_gen 2020-10-28 10:30:36 +00:00
Arthur Baars
fe1d8ec15f Extractor: fix child index values 2020-10-27 22:32:53 +01:00
Arthur Baars
0c15783f2b Merge pull request #12 from github/crates-language
Use tree-sitter-ruby crate instead of vendoring it
2020-10-27 20:53:48 +01:00
Nick Rolfe
a41c3e36f9 Give node_types a static lifetime. 2020-10-27 19:11:05 +00:00
Nick Rolfe
5484ff3dcf Use tree_sitter_ruby crate in generator 2020-10-27 18:13:40 +00:00
Douglas Creager
2663de86fb Don't clone submodules in Actions workflow
Since we don't have any submodules anymore!
2020-10-27 14:02:15 -04:00
Nick Rolfe
ce8de3feba Update generator binary name in Actions workflow 2020-10-27 17:56:37 +00:00
Douglas Creager
5f985be2d9 Use tree-sitter-ruby crate instead of vendoring it 2020-10-27 13:54:56 -04:00
Nick Rolfe
e05bcf9fb7 Generate QL classes 2020-10-27 17:46:11 +00:00
Arthur Baars
3e1c378aba Merge pull request #8 from github/aibaars/actions
Improve extractor build and add GitHub Actions configuration
2020-10-27 18:21:20 +01:00
Arthur Baars
4b46a75c24 Merge pull request #10 from github/github/aibaars/escape-uppercase
DB scheme: convert uppercase to lowercase + underscore
2020-10-27 18:21:00 +01:00
Arthur Baars
bb2e7d841f DB scheme: convert uppercase to lowercase + underscore 2020-10-27 18:15:48 +01:00
Arthur Baars
53b97ff0fa Use release builds for the CodeQL package 2020-10-27 17:48:11 +01:00
Arthur Baars
bdff1fe9f4 Merge pull request #9 from github/aibaars/escape-column-names
DB scheme generator: escape column names
2020-10-27 17:44:39 +01:00
Arthur Baars
e3a1d426b8 DB scheme generator: escape column names 2020-10-27 17:31:10 +01:00
Arthur Baars
9e6ccf558e Preserve permissions of Linux and OSX binaries
The {upload,download}-artifact actions do not preserve
file permissions, so we need to patch things up.
2020-10-27 17:17:44 +01:00
Arthur Baars
048f19edc1 Build a CodeQL extractor pack 2020-10-27 17:02:08 +01:00
Arthur Baars
73a090501a Add GitHub actions configuration 2020-10-27 16:34:17 +01:00
Arthur Baars
7555141246 Extractor: include contents node-types.json as constant 2020-10-27 16:34:17 +01:00
Arthur Baars
74dd4dcc2c Build parser.c and scanner.cc separately 2020-10-27 16:34:17 +01:00
Arthur Baars
74e9829609 Merge pull request #7 from github/aibaars/refactor
Refactor dbscheme generator to use intermediate representation
2020-10-27 14:12:05 +01:00
Arthur Baars
1fd6fdd652 Address review comment from earlier pull-request 2020-10-27 13:43:59 +01:00
Arthur Baars
a50f79b401 Add logging to dbscheme generator 2020-10-27 13:36:58 +01:00
Arthur Baars
0439d4f674 Refactor dbscheme generator to use intermediate representation
* merge extractor/node_types.rs into node-types/lib.rs
* use intermediate representation in dbscheme generator
* move dbscheme naming and escaping functions to node-types so they can be shared
2020-10-27 13:27:45 +01:00
Arthur Baars
4c1682ef2e Merge pull request #5 from github/aibaars/logger
Add logging based on the tracing library
2020-10-27 13:24:34 +01:00
Nick Rolfe
63282eac60 Merge pull request #6 from github/windows_paths
Handle Windows path prefixes
2020-10-27 12:20:54 +00:00
Nick Rolfe
c02b735eec Handle Windows path prefixes 2020-10-27 12:09:46 +00:00
Arthur Baars
52035ef672 Add tracing logger 2020-10-27 11:29:21 +01:00
Arthur Baars
9c534209f7 Add tracing:0.1 2020-10-27 11:26:35 +01:00
Arthur Baars
467e32ade4 Merge pull request #2 from github/aibaars/extractor-rust
Rewrite extractor in rust
2020-10-27 10:16:58 +01:00
Arthur Baars
0f576fe29a Address review comments 2020-10-26 19:10:44 +01:00
Arthur Baars
1d36b5085a Do not recurse into 'extra' nodes for now 2020-10-26 18:39:10 +01:00
Arthur Baars
fd39524c5e Improve error messages
Include file path and line number and emit better descriptions
2020-10-26 18:37:29 +01:00
Arthur Baars
47ccc33ab3 Initial version of extractor based on tree-sitter grammar 2020-10-24 13:22:39 +02:00
Arthur Baars
d00c956028 Build with clang for non-windows platforms 2020-10-24 13:22:39 +02:00
Arthur Baars
f6292e437e Merge pull request #4 from github/shared_lib
Add library package for shared code
2020-10-23 14:18:42 +02:00
Nick Rolfe
849e109583 Add library package for shared code 2020-10-23 13:01:17 +01:00
Arthur Baars
305fd566a8 Merge pull request #3 from github/aibaars/codeql-extractor-yaml
Basic CodeQL extractor configuration and autobuild scripts
2020-10-22 22:23:44 +02:00
Arthur Baars
e16b85e511 Add codeql-extractor config 2020-10-22 18:30:57 +02:00
Nick Rolfe
12571dbe42 Merge pull request #1 from github/dbscheme
Basic dbscheme generation from `node-types.json`
2020-10-22 12:29:44 +01:00
Nick Rolfe
36823d7804 Move deserialization to node_types module; propagate errors to caller 2020-10-22 11:10:05 +01:00
Nick Rolfe
e018f3f20b Use if let instead of iterating over Option 2020-10-21 12:51:10 +01:00
Nick Rolfe
5e3544fcc3 Use fmt::Display trait for writing dbscheme 2020-10-21 12:45:54 +01:00
Nick Rolfe
a7a18b8b0f Gather all hard-coded Ruby-specific names/paths in one struct. 2020-10-21 11:29:25 +01:00
Nick Rolfe
47c8a3d6fb Simplify to std::io::Result 2020-10-21 11:26:23 +01:00
Nick Rolfe
fd1f8b22e2 Simplify keysets to Option<Vec<String>> 2020-10-21 11:06:53 +01:00
Nick Rolfe
97181d1c21 Basic dbscheme generation from node-types.json 2020-10-20 17:49:55 +01:00
Nick Rolfe
735fde7a22 Add README 2020-10-15 13:26:13 +01:00
Nick Rolfe
a837c65bc4 Add VSCode build task for cargo build 2020-10-15 13:21:12 +01:00
Nick Rolfe
ffbb57a8e2 Make VSCode default to unix line endings 2020-10-15 13:20:37 +01:00
Nick Rolfe
6c697bf9b5 Split into generator and extractor packages 2020-10-15 13:20:11 +01:00
Nick Rolfe
b677a91fea Add VSCode workspace 2020-10-14 11:16:28 +01:00
Nick Rolfe
89959b2e0d Add tree-sitter-ruby submodule 2020-10-14 11:15:59 +01:00
Nick Rolfe
d3ccb49273 Initial commit: cargo-generated boilerplate 2020-10-13 18:42:13 +01:00
480 changed files with 124679 additions and 32 deletions

View File

@@ -1,4 +1,5 @@
{ "provide": [ "*/ql/src/qlpack.yml",
{ "provide": [ "ruby/.codeqlmanifest.json",
"*/ql/src/qlpack.yml",
"*/ql/lib/qlpack.yml",
"*/ql/test/qlpack.yml",
"cpp/ql/test/query-tests/Security/CWE/CWE-190/semmle/tainted/qlpack.yml",

View File

@@ -1,9 +1,14 @@
{
"extensions": [
"rust-lang.rust",
"bungcip.better-toml",
"github.vscode-codeql",
"slevesque.vscode-zipexplorer"
],
"settings": {
"files.watcherExclude": {
"**/target/**": true
},
"codeQL.runningQueries.memory": 2048
}
}

14
.github/actions/fetch-codeql/action.yml vendored Normal file
View File

@@ -0,0 +1,14 @@
name: Fetch CodeQL
description: Fetches the latest version of CodeQL
runs:
using: composite
steps:
- name: Fetch CodeQL
shell: bash
run: |
LATEST=$(gh release list --repo https://github.com/github/codeql-cli-binaries | cut -f 1 | grep -v beta | sort --version-sort | tail -1)
gh release download --repo https://github.com/github/codeql-cli-binaries --pattern codeql-linux64.zip "$LATEST"
unzip -q codeql-linux64.zip
echo "${{ github.workspace }}/codeql" >> $GITHUB_PATH
env:
GITHUB_TOKEN: ${{ github.token }}

18
.github/dependabot.yml vendored Normal file
View File

@@ -0,0 +1,18 @@
version: 2
updates:
- package-ecosystem: "cargo"
directory: "ruby/node-types"
schedule:
interval: "daily"
- package-ecosystem: "cargo"
directory: "ruby/generator"
schedule:
interval: "daily"
- package-ecosystem: "cargo"
directory: "ruby/extractor"
schedule:
interval: "daily"
- package-ecosystem: "cargo"
directory: "ruby/autobuilder"
schedule:
interval: "daily"

39
.github/workflows/qhelp-pr-preview.yml vendored Normal file
View File

@@ -0,0 +1,39 @@
name: Query help preview
on:
pull_request:
branches:
- main
- 'rc/*'
paths:
- "ruby/**/*.qhelp"
jobs:
qhelp:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
with:
fetch-depth: 2
- name: Determine changed files
id: changes
run: |
echo -n "::set-output name=qhelp_files::"
(git diff --name-only --diff-filter=ACMRT HEAD~1 HEAD | grep .qhelp$ | grep -v .inc.qhelp;
git diff --name-only --diff-filter=ACMRT HEAD~1 HEAD | grep .inc.qhelp$ | xargs -d '\n' -rn1 basename | xargs -d '\n' -rn1 git grep -l) |
sort -u | xargs -d '\n' -n1 printf "'%s' "
- uses: ./.github/actions/fetch-codeql
- name: QHelp preview
if: ${{ steps.changes.outputs.qhelp_files }}
run: |
( echo "QHelp previews:";
for path in ${{ steps.changes.outputs.qhelp_files }} ; do
echo "<details> <summary>${path}</summary>"
echo
codeql generate query-help --format=markdown ${path}
echo "</details>"
done) | gh pr comment "${{ github.event.pull_request.number }}" -F -
env:
GITHUB_TOKEN: ${{ github.token }}

234
.github/workflows/ruby-build.yml vendored Normal file
View File

@@ -0,0 +1,234 @@
name: "Ruby: Build"
on:
push:
paths:
- "ruby/**"
- .github/workflows/ruby-build.yml
branches:
- main
- "rc/*"
pull_request:
paths:
- "ruby/**"
- .github/workflows/ruby-build.yml
branches:
- main
- "rc/*"
workflow_dispatch:
inputs:
tag:
description: "Version tag to create"
required: false
env:
CARGO_TERM_COLOR: always
defaults:
run:
working-directory: ruby
jobs:
build:
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest, macos-latest, windows-latest]
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@v2
- name: Install GNU tar
if: runner.os == 'macOS'
run: |
brew install gnu-tar
echo "/usr/local/opt/gnu-tar/libexec/gnubin" >> $GITHUB_PATH
- uses: actions/cache@v2
with:
path: |
~/.cargo/registry
~/.cargo/git
ruby/target
key: ${{ runner.os }}-rust-cargo-${{ hashFiles('**/Cargo.lock') }}
- name: Check formatting
run: cargo fmt --all -- --check
- name: Build
run: cargo build --verbose
- name: Run tests
run: cargo test --verbose
- name: Release build
run: cargo build --release
- name: Generate dbscheme
if: ${{ matrix.os == 'ubuntu-latest' }}
run: target/release/ruby-generator --dbscheme ql/lib/ruby.dbscheme --library ql/lib/codeql/ruby/ast/internal/TreeSitter.qll
- uses: actions/upload-artifact@v2
if: ${{ matrix.os == 'ubuntu-latest' }}
with:
name: ruby.dbscheme
path: ruby/ql/lib/ruby.dbscheme
- uses: actions/upload-artifact@v2
if: ${{ matrix.os == 'ubuntu-latest' }}
with:
name: TreeSitter.qll
path: ruby/ql/lib/codeql/ruby/ast/internal/TreeSitter.qll
- uses: actions/upload-artifact@v2
with:
name: extractor-${{ matrix.os }}
path: |
ruby/target/release/ruby-autobuilder
ruby/target/release/ruby-autobuilder.exe
ruby/target/release/ruby-extractor
ruby/target/release/ruby-extractor.exe
retention-days: 1
compile-queries:
runs-on: ubuntu-latest
env:
CODEQL_THREADS: 4 # TODO: remove this once it's set by the CLI
steps:
- uses: actions/checkout@v2
- name: Fetch CodeQL
run: |
LATEST=$(gh release list --repo https://github.com/github/codeql-cli-binaries | cut -f 1 | grep -v beta | sort --version-sort | tail -1)
gh release download --repo https://github.com/github/codeql-cli-binaries --pattern codeql-linux64.zip "$LATEST"
unzip -q codeql-linux64.zip
env:
GITHUB_TOKEN: ${{ github.token }}
- name: Build Query Pack
run: |
codeql/codeql pack create ql/lib --output target/packs
codeql/codeql pack install ql/src
codeql/codeql pack create ql/src --output target/packs
PACK_FOLDER=$(readlink -f target/packs/codeql/ruby-queries/*)
codeql/codeql generate query-help --format=sarifv2.1.0 --output="${PACK_FOLDER}/rules.sarif" ql/src
(cd ql/src; find queries \( -name '*.qhelp' -o -name '*.rb' -o -name '*.erb' \) -exec bash -c 'mkdir -p "'"${PACK_FOLDER}"'/$(dirname "{}")"' \; -exec cp "{}" "${PACK_FOLDER}/{}" \;)
- name: Compile with previous CodeQL versions
run: |
for version in $(gh release list --repo https://github.com/github/codeql-cli-binaries | cut -f 1 | sort --version-sort | tail -3 | head -2); do
rm -f codeql-linux64.zip
gh release download --repo https://github.com/github/codeql-cli-binaries --pattern codeql-linux64.zip "$version"
rm -rf codeql; unzip -q codeql-linux64.zip
codeql/codeql query compile target/packs/*
done
env:
GITHUB_TOKEN: ${{ github.token }}
- uses: actions/upload-artifact@v2
with:
name: codeql-ruby-queries
path: |
ruby/target/packs/*
retention-days: 1
package:
runs-on: ubuntu-latest
needs: [build, compile-queries]
steps:
- uses: actions/checkout@v2
- uses: actions/download-artifact@v2
with:
name: ruby.dbscheme
path: ruby/ruby
- uses: actions/download-artifact@v2
with:
name: extractor-ubuntu-latest
path: ruby/linux64
- uses: actions/download-artifact@v2
with:
name: extractor-windows-latest
path: ruby/win64
- uses: actions/download-artifact@v2
with:
name: extractor-macos-latest
path: ruby/osx64
- run: |
mkdir -p ruby
cp -r codeql-extractor.yml tools ql/lib/ruby.dbscheme.stats ruby/
mkdir -p ruby/tools/{linux64,osx64,win64}
cp linux64/ruby-autobuilder ruby/tools/linux64/autobuilder
cp osx64/ruby-autobuilder ruby/tools/osx64/autobuilder
cp win64/ruby-autobuilder.exe ruby/tools/win64/autobuilder.exe
cp linux64/ruby-extractor ruby/tools/linux64/extractor
cp osx64/ruby-extractor ruby/tools/osx64/extractor
cp win64/ruby-extractor.exe ruby/tools/win64/extractor.exe
chmod +x ruby/tools/{linux64,osx64}/{autobuilder,extractor}
zip -rq codeql-ruby.zip ruby
- uses: actions/upload-artifact@v2
with:
name: codeql-ruby-pack
path: ruby/codeql-ruby.zip
retention-days: 1
- uses: actions/download-artifact@v2
with:
name: codeql-ruby-queries
path: ruby/qlpacks
- run: |
echo '{
"provide": [
"ruby/codeql-extractor.yml",
"qlpacks/*/*/*/qlpack.yml"
]
}' > .codeqlmanifest.json
zip -rq codeql-ruby-bundle.zip .codeqlmanifest.json ruby qlpacks
- uses: actions/upload-artifact@v2
with:
name: codeql-ruby-bundle
path: ruby/codeql-ruby-bundle.zip
retention-days: 1
test:
defaults:
run:
working-directory: ${{ github.workspace }}
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest, macos-latest, windows-latest]
runs-on: ${{ matrix.os }}
needs: [package]
steps:
- uses: actions/checkout@v2
with:
repository: Shopify/example-ruby-app
ref: 67a0decc5eb550f3a9228eda53925c3afd40dfe9
- name: Fetch CodeQL
shell: bash
run: |
LATEST=$(gh release list --repo https://github.com/github/codeql-cli-binaries | cut -f 1 | grep -v beta | sort --version-sort | tail -1)
gh release download --repo https://github.com/github/codeql-cli-binaries --pattern codeql.zip "$LATEST"
unzip -q codeql.zip
env:
GITHUB_TOKEN: ${{ github.token }}
working-directory: ${{ runner.temp }}
- name: Download Ruby bundle
uses: actions/download-artifact@v2
with:
name: codeql-ruby-bundle
path: ${{ runner.temp }}
- name: Unzip Ruby bundle
shell: bash
run: unzip -q -d "${{ runner.temp }}/ruby-bundle" "${{ runner.temp }}/codeql-ruby-bundle.zip"
- name: Prepare test files
shell: bash
run: |
echo "import ruby select count(File f)" > "test.ql"
echo "| 4 |" > "test.expected"
echo 'name: sample-tests
version: 0.0.0
dependencies:
codeql/ruby-all: 0.0.1
extractor: ruby
tests: .
' > qlpack.yml
- name: Run QL test
shell: bash
run: |
"${{ runner.temp }}/codeql/codeql" test run --search-path "${{ runner.temp }}/ruby-bundle" --additional-packs "${{ runner.temp }}/ruby-bundle" .
- name: Create database
shell: bash
run: |
"${{ runner.temp }}/codeql/codeql" database create --search-path "${{ runner.temp }}/ruby-bundle" --language ruby --source-root . ../database
- name: Analyze database
shell: bash
run: |
"${{ runner.temp }}/codeql/codeql" database analyze --search-path "${{ runner.temp }}/ruby-bundle" --format=sarifv2.1.0 --output=out.sarif ../database ruby-code-scanning.qls

View File

@@ -0,0 +1,73 @@
name: "Ruby: Collect database stats"
on:
push:
branches:
- main
- "rc/*"
paths:
- ruby/ql/lib/ruby.dbscheme
- .github/workflows/ruby-dataset-measure.yml
pull_request:
branches:
- main
- "rc/*"
paths:
- ruby/ql/lib/ruby.dbscheme
- .github/workflows/ruby-dataset-measure.yml
workflow_dispatch:
jobs:
measure:
env:
CODEQL_THREADS: 4 # TODO: remove this once it's set by the CLI
strategy:
fail-fast: false
matrix:
repo: [rails/rails, discourse/discourse, spree/spree]
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- uses: ./.github/actions/fetch-codeql
- uses: ./ruby/actions/create-extractor-pack
- name: Checkout ${{ matrix.repo }}
uses: actions/checkout@v2
with:
repository: ${{ matrix.repo }}
path: ${{ github.workspace }}/repo
- name: Create database
run: |
codeql database create \
--search-path "${{ github.workspace }}/ruby" \
--threads 4 \
--language ruby --source-root "${{ github.workspace }}/repo" \
"${{ runner.temp }}/database"
- name: Measure database
run: |
mkdir -p "stats/${{ matrix.repo }}"
codeql dataset measure --threads 4 --output "stats/${{ matrix.repo }}/stats.xml" "${{ runner.temp }}/database/db-ruby"
- uses: actions/upload-artifact@v2
with:
name: measurements
path: stats
retention-days: 1
merge:
runs-on: ubuntu-latest
needs: measure
steps:
- uses: actions/checkout@v2
- uses: actions/download-artifact@v2
with:
name: measurements
path: stats
- run: |
python -m pip install --user lxml
find stats -name 'stats.xml' | sort | xargs python ruby/scripts/merge_stats.py --output ruby/ql/lib/ruby.dbscheme.stats --normalise ruby_tokeninfo
- uses: actions/upload-artifact@v2
with:
name: ruby.dbscheme.stats
path: ruby/ql/lib/ruby.dbscheme.stats

50
.github/workflows/ruby-qltest.yml vendored Normal file
View File

@@ -0,0 +1,50 @@
name: "Ruby: Run QL Tests"
on:
push:
paths:
- "ruby/**"
- .github/workflows/ruby-qltest.yml
branches:
- main
- "rc/*"
pull_request:
paths:
- "ruby/**"
- .github/workflows/ruby-qltest.yml
branches:
- main
- "rc/*"
env:
CARGO_TERM_COLOR: always
defaults:
run:
working-directory: ruby
jobs:
qltest:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- uses: ./.github/actions/fetch-codeql
- uses: ./ruby/actions/create-extractor-pack
- name: Run QL tests
run: |
codeql test run --check-databases --check-unused-labels --check-repeated-labels --check-redefined-labels --check-use-before-definition --search-path "${{ github.workspace }}/ruby" --additional-packs "${{ github.workspace }}" --consistency-queries ql/consistency-queries ql/test
env:
GITHUB_TOKEN: ${{ github.token }}
- name: Check QL formatting
run: find ql "(" -name "*.ql" -or -name "*.qll" ")" -print0 | xargs -0 codeql query format --check-only
- name: Check QL compilation
run: |
codeql query compile --check-only --threads=4 --warnings=error --search-path "${{ github.workspace }}/ruby" --additional-packs "${{ github.workspace }}" "ql/src" "ql/examples"
env:
GITHUB_TOKEN: ${{ github.token }}
- name: Check DB upgrade scripts
run: |
echo >empty.trap
codeql dataset import -S ql/lib/upgrades/initial/ruby.dbscheme testdb empty.trap
codeql dataset upgrade testdb --additional-packs ql/lib
diff -q testdb/ruby.dbscheme ql/lib/ruby.dbscheme

20
.github/workflows/sync-files.yml vendored Normal file
View File

@@ -0,0 +1,20 @@
name: Check synchronized files
on:
push:
branches:
- main
- 'rc/*'
pull_request:
branches:
- main
- 'rc/*'
jobs:
sync:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Check synchronized files
run: python config/sync-files.py

View File

@@ -3,6 +3,7 @@
/java/ @github/codeql-java
/javascript/ @github/codeql-javascript
/python/ @github/codeql-python
/ruby/ @github/codeql-ruby
# Make @xcorail (GitHub Security Lab) a code owner for experimental queries so he gets pinged when we promote a query out of experimental
/cpp/**/experimental/**/* @github/codeql-c-analysis @xcorail
@@ -10,6 +11,7 @@
/java/**/experimental/**/* @github/codeql-java @xcorail
/javascript/**/experimental/**/* @github/codeql-javascript @xcorail
/python/**/experimental/**/* @github/codeql-python @xcorail
/ruby/**/experimental/**/* @github/codeql-ruby @xcorail
# Notify members of codeql-go about PRs to the shared data-flow library files
/java/ql/src/semmle/code/java/dataflow/internal/DataFlowImpl.qll @github/codeql-java @github/codeql-go
@@ -22,4 +24,4 @@
/docs/codeql-cli/ @github/codeql-cli-reviewers
/docs/codeql-for-visual-studio-code/ @github/codeql-vscode-reviewers
/docs/ql-language-reference/ @github/codeql-frontend-reviewers
/docs/query-*-style-guide.md @github/codeql-analysis-reviewers
/docs/query-*-style-guide.md @github/codeql-analysis-reviewers

View File

@@ -24,14 +24,16 @@
"python/ql/lib/semmle/python/dataflow/new/internal/DataFlowImpl.qll",
"python/ql/lib/semmle/python/dataflow/new/internal/DataFlowImpl2.qll",
"python/ql/lib/semmle/python/dataflow/new/internal/DataFlowImpl3.qll",
"python/ql/lib/semmle/python/dataflow/new/internal/DataFlowImpl4.qll"
"python/ql/lib/semmle/python/dataflow/new/internal/DataFlowImpl4.qll",
"ruby/ql/lib/codeql/ruby/dataflow/internal/DataFlowImpl.qll"
],
"DataFlow Java/C++/C#/Python Common": [
"java/ql/lib/semmle/code/java/dataflow/internal/DataFlowImplCommon.qll",
"cpp/ql/lib/semmle/code/cpp/dataflow/internal/DataFlowImplCommon.qll",
"cpp/ql/lib/semmle/code/cpp/ir/dataflow/internal/DataFlowImplCommon.qll",
"csharp/ql/lib/semmle/code/csharp/dataflow/internal/DataFlowImplCommon.qll",
"python/ql/lib/semmle/python/dataflow/new/internal/DataFlowImplCommon.qll"
"python/ql/lib/semmle/python/dataflow/new/internal/DataFlowImplCommon.qll",
"ruby/ql/lib/codeql/ruby/dataflow/internal/DataFlowImplCommon.qll"
],
"TaintTracking::Configuration Java/C++/C#/Python": [
"cpp/ql/lib/semmle/code/cpp/dataflow/internal/tainttracking1/TaintTrackingImpl.qll",
@@ -49,18 +51,21 @@
"python/ql/lib/semmle/python/dataflow/new/internal/tainttracking1/TaintTrackingImpl.qll",
"python/ql/lib/semmle/python/dataflow/new/internal/tainttracking2/TaintTrackingImpl.qll",
"python/ql/lib/semmle/python/dataflow/new/internal/tainttracking3/TaintTrackingImpl.qll",
"python/ql/lib/semmle/python/dataflow/new/internal/tainttracking4/TaintTrackingImpl.qll"
"python/ql/lib/semmle/python/dataflow/new/internal/tainttracking4/TaintTrackingImpl.qll",
"ruby/ql/lib/codeql/ruby/dataflow/internal/tainttracking1/TaintTrackingImpl.qll"
],
"DataFlow Java/C++/C#/Python Consistency checks": [
"java/ql/lib/semmle/code/java/dataflow/internal/DataFlowImplConsistency.qll",
"cpp/ql/lib/semmle/code/cpp/dataflow/internal/DataFlowImplConsistency.qll",
"cpp/ql/lib/semmle/code/cpp/ir/dataflow/internal/DataFlowImplConsistency.qll",
"csharp/ql/lib/semmle/code/csharp/dataflow/internal/DataFlowImplConsistency.qll",
"python/ql/lib/semmle/python/dataflow/new/internal/DataFlowImplConsistency.qll"
"python/ql/lib/semmle/python/dataflow/new/internal/DataFlowImplConsistency.qll",
"ruby/ql/lib/codeql/ruby/dataflow/internal/DataFlowImplConsistency.qll"
],
"DataFlow Java/C# Flow Summaries": [
"java/ql/lib/semmle/code/java/dataflow/internal/FlowSummaryImpl.qll",
"csharp/ql/lib/semmle/code/csharp/dataflow/internal/FlowSummaryImpl.qll"
"csharp/ql/lib/semmle/code/csharp/dataflow/internal/FlowSummaryImpl.qll",
"ruby/ql/lib/codeql/ruby/dataflow/internal/FlowSummaryImpl.qll"
],
"SsaReadPosition Java/C#": [
"java/ql/lib/semmle/code/java/dataflow/internal/rangeanalysis/SsaReadPositionCommon.qll",
@@ -368,7 +373,8 @@
"Inline Test Expectations": [
"cpp/ql/test/TestUtilities/InlineExpectationsTest.qll",
"java/ql/test/TestUtilities/InlineExpectationsTest.qll",
"python/ql/test/TestUtilities/InlineExpectationsTest.qll"
"python/ql/test/TestUtilities/InlineExpectationsTest.qll",
"ruby/ql/test/TestUtilities/InlineExpectationsTest.qll"
],
"C++ ExternalAPIs": [
"cpp/ql/src/Security/CWE/CWE-020/ExternalAPIs.qll",
@@ -440,7 +446,8 @@
"csharp/ql/lib/semmle/code/csharp/dataflow/internal/SsaImplCommon.qll",
"csharp/ql/lib/semmle/code/csharp/controlflow/internal/pressa/SsaImplCommon.qll",
"csharp/ql/lib/semmle/code/csharp/dataflow/internal/basessa/SsaImplCommon.qll",
"csharp/ql/lib/semmle/code/cil/internal/SsaImplCommon.qll"
"csharp/ql/lib/semmle/code/cil/internal/SsaImplCommon.qll",
"ruby/ql/lib/codeql/ruby/dataflow/internal/SsaImplCommon.qll"
],
"CryptoAlgorithms Python/JS": [
"javascript/ql/lib/semmle/javascript/security/CryptoAlgorithms.qll",
@@ -460,6 +467,15 @@
],
"ReDoS Polynomial Python/JS": [
"javascript/ql/lib/semmle/javascript/security/performance/SuperlinearBackTracking.qll",
"python/ql/lib/semmle/python/security/performance/SuperlinearBackTracking.qll"
"python/ql/lib/semmle/python/security/performance/SuperlinearBackTracking.qll",
"ruby/ql/lib/codeql/ruby/regexp/SuperlinearBackTracking.qll"
],
"CFG": [
"csharp/ql/lib/semmle/code/csharp/controlflow/internal/ControlFlowGraphImplShared.qll",
"ruby/ql/lib/codeql/ruby/controlflow/internal/ControlFlowGraphImplShared.qll"
],
"TypeTracker": [
"python/ql/lib/semmle/python/dataflow/new/internal/TypeTracker.qll",
"ruby/ql/lib/codeql/ruby/typetracking/TypeTracker.qll"
]
}
}

View File

@@ -33,3 +33,4 @@ Note that the CWE coverage includes both "`supported queries <https://github.com
java-cwe
javascript-cwe
python-cwe
ruby-cwe

View File

@@ -9,6 +9,7 @@ View the query help for the queries included in the ``code-scanning``, ``securit
- :doc:`CodeQL query help for Java <java>`
- :doc:`CodeQL query help for JavaScript <javascript>`
- :doc:`CodeQL query help for Python <python>`
- :doc:`CodeQL query help for Ruby <ruby>`
.. pull-quote:: Information
@@ -33,5 +34,6 @@ For a full list of the CWEs covered by these queries, see ":doc:`CodeQL CWE cove
java
javascript
python
ruby
codeql-cwe-coverage

View File

@@ -0,0 +1,8 @@
# CWE coverage for Ruby
An overview of CWE coverage for Ruby in the latest release of CodeQL.
## Overview
<!-- autogenerated CWE coverage table will be added below -->

View File

@@ -0,0 +1,8 @@
CodeQL query help for Ruby
============================
.. include:: ../reusables/query-help-overview.rst
For shorter queries that you can use as building blocks when writing your own queries, see the `example queries in the CodeQL repository <https://github.com/github/codeql/tree/main/ruby/ql/examples>`__.
.. include:: toc-ruby.rst

View File

@@ -41,9 +41,12 @@ project = u'Supported languages and frameworks for LGTM Enterprise'
# The version info for this project, if different from version and release in main conf.py file.
# The short X.Y version.
version = u'1.27'
# The full version, including alpha/beta/rc tags.
release = u'1.27'
# LGTM Enterprise release
release = u'1.29'
# CodeQL CLI version used by LGTM Enterprise release
version = u'2.6.4'
# -- Project-specifc options for HTML output ----------------------------------------------

View File

@@ -1,7 +1,13 @@
Frameworks and libraries
########################
The libraries and queries in version |version| have been explicitly checked against the libraries and frameworks listed below.
LGTM Enterprise |release| includes CodeQL CLI |version|. The CodeQL libraries and queries used by this version of LGTM Enterprise have been explicitly checked against the libraries and frameworks listed below.
.. pull-quote::
Note
For details of framework and library support in the most recent release of the CodeQL CLI, see `Supported languages and frameworks <https://codeql.github.com/docs/codeql-overview/supported-languages-and-frameworks/>`__ in the CodeQL CLI documentation.
.. pull-quote::

View File

@@ -1,8 +1,13 @@
Languages and compilers
#######################
CodeQL and LGTM version |version| support analysis of the following languages compiled by the following compilers.
(CodeQL was previously known as QL.)
LGTM Enterprise |release| includes CodeQL CLI |version|. LGTM Enterprise supports analysis of the following languages compiled by the following compilers.
.. pull-quote::
Note
For details of language and compiler support in the most recent release of the CodeQL CLI, see `Supported languages and frameworks <https://codeql.github.com/docs/codeql-overview/supported-languages-and-frameworks/>`__ in the CodeQL CLI documentation.
Note that where there are several versions or dialects of a language, the supported variants are listed.
If your code requires a particular version of a compiler, check that this version is included below.

View File

@@ -25,7 +25,7 @@ When you have selected a presentation, use |arrow-r| and |arrow-l| to navigate b
Press **p** to view the additional notes on slides that have an information icon |info| in the top right corner, and press **f** to enter full-screen mode.
The presentations contain a number of query examples.
We recommend that you download `CodeQL for Visual Studio Code <https://help.semmle.com/codeql/codeql-for-vscode/procedures/setting-up.html>`__ and add the example database for each presentation so that you can find the bugs mentioned in the slides.
We recommend that you download `CodeQL for Visual Studio Code <https://codeql.github.com/docs/codeql-for-visual-studio-code/>`__ and add the example database for each presentation so that you can find the bugs mentioned in the slides.
.. pull-quote::
@@ -39,25 +39,25 @@ We recommend that you download `CodeQL for Visual Studio Code <https://help.semm
CodeQL and variant analysis for C/C++
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
- `Introduction to variant analysis: CodeQL for C/C++ <https://help.semmle.com/QL/ql-training/cpp/intro-ql-cpp.html>`__an introduction to variant analysis and CodeQL for C/C++ programmers.
- `Example: Bad overflow guard <https://help.semmle.com/QL/ql-training/cpp/bad-overflow-guard.html>`__an example of iterative query development to find bad overflow guards in a C++ project.
- `Program representation: CodeQL for C/C++ <https://help.semmle.com/QL/ql-training/cpp/program-representation-cpp.html>`__information on how CodeQL analysis represents C/C++ programs.
- `Introduction to local data flow <https://help.semmle.com/QL/ql-training/cpp/data-flow-cpp.html>`__an introduction to analyzing local data flow in C/C++ using CodeQL, including an example demonstrating how to develop a query to find a real CVE.
- `Exercise: snprintf overflow <https://help.semmle.com/QL/ql-training/cpp/snprintf.html>`__an example demonstrating how to develop a data flow query.
- `Introduction to global data flow <https://help.semmle.com/QL/ql-training/cpp/global-data-flow-cpp.html>`__an introduction to analyzing global data flow in C/C++ using CodeQL.
- `Analyzing control flow: CodeQL for C/C++ <https://help.semmle.com/QL/ql-training/cpp/control-flow-cpp.html>`__an introduction to analyzing control flow in C/C++ using CodeQL.
- `Introduction to variant analysis: CodeQL for C/C++ </QL/ql-training/cpp/intro-ql-cpp.html>`__an introduction to variant analysis and CodeQL for C/C++ programmers.
- `Example: Bad overflow guard </QL/ql-training/cpp/bad-overflow-guard.html>`__an example of iterative query development to find bad overflow guards in a C++ project.
- `Program representation: CodeQL for C/C++ </QL/ql-training/cpp/program-representation-cpp.html>`__information on how CodeQL analysis represents C/C++ programs.
- `Introduction to local data flow </QL/ql-training/cpp/data-flow-cpp.html>`__an introduction to analyzing local data flow in C/C++ using CodeQL, including an example demonstrating how to develop a query to find a real CVE.
- `Exercise: snprintf overflow </QL/ql-training/cpp/snprintf.html>`__an example demonstrating how to develop a data flow query.
- `Introduction to global data flow </QL/ql-training/cpp/global-data-flow-cpp.html>`__an introduction to analyzing global data flow in C/C++ using CodeQL.
- `Analyzing control flow: CodeQL for C/C++ </QL/ql-training/cpp/control-flow-cpp.html>`__an introduction to analyzing control flow in C/C++ using CodeQL.
CodeQL and variant analysis for Java
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
- `Introduction to variant analysis: CodeQL for Java <https://help.semmle.com/QL/ql-training/java/intro-ql-java.html>`__an introduction to variant analysis and CodeQL for Java programmers.
- `Example: Query injection <https://help.semmle.com/QL/ql-training/java/query-injection-java.html>`__an example of iterative query development to find unsanitized SPARQL injections in a Java project.
- `Program representation: CodeQL for Java <https://help.semmle.com/QL/ql-training/java/program-representation-java.html>`__information on how CodeQL analysis represents Java programs.
- `Introduction to local data flow <https://help.semmle.com/QL/ql-training/java/data-flow-java.html>`__an introduction to analyzing local data flow in Java using CodeQL, including an example demonstrating how to develop a query to find a real CVE.
- `Exercise: Apache Struts <https://help.semmle.com/QL/ql-training/java/apache-struts-java.html>`__an example demonstrating how to develop a data flow query.
- `Introduction to global data flow <https://help.semmle.com/QL/ql-training/java/global-data-flow-java.html>`__an introduction to analyzing global data flow in Java using CodeQL.
- `Introduction to variant analysis: CodeQL for Java </QL/ql-training/java/intro-ql-java.html>`__an introduction to variant analysis and CodeQL for Java programmers.
- `Example: Query injection </QL/ql-training/java/query-injection-java.html>`__an example of iterative query development to find unsanitized SPARQL injections in a Java project.
- `Program representation: CodeQL for Java </QL/ql-training/java/program-representation-java.html>`__information on how CodeQL analysis represents Java programs.
- `Introduction to local data flow </QL/ql-training/java/data-flow-java.html>`__an introduction to analyzing local data flow in Java using CodeQL, including an example demonstrating how to develop a query to find a real CVE.
- `Exercise: Apache Struts </QL/ql-training/java/apache-struts-java.html>`__an example demonstrating how to develop a data flow query.
- `Introduction to global data flow </QL/ql-training/java/global-data-flow-java.html>`__an introduction to analyzing global data flow in Java using CodeQL.
Further reading
~~~~~~~~~~~~~~~
- `GitHub Security Lab <https://securitylab.github.com/research>`__
- `GitHub Security Lab <https://securitylab.github.com/research>`__

10
ruby/.codeqlmanifest.json Normal file
View File

@@ -0,0 +1,10 @@
{
"provide": [
"ql/lib/qlpack.yml",
"ql/src/qlpack.yml",
"ql/consistency-queries/qlpack.yml",
"ql/test/qlpack.yml",
"ql/examples/qlpack.yml",
"extractor-pack/codeql-extractor.yml"
]
}

1
ruby/.gitattributes vendored Normal file
View File

@@ -0,0 +1 @@
Cargo.lock -diff -whitespace

8
ruby/.gitignore vendored Normal file
View File

@@ -0,0 +1,8 @@
/target
extractor-pack
.vscode/launch.json
.cache
ql/test/**/*.testproj
ql/test/**/*.actual
ql/test/**/CONSISTENCY
.codeql

14
ruby/.vscode/tasks.json vendored Normal file
View File

@@ -0,0 +1,14 @@
{
"version": "2.0.0",
"tasks": [
{
"type": "cargo",
"subcommand": "build",
"problemMatcher": [
"$rustc"
],
"group": "build",
"label": "Rust: cargo build"
}
]
}

BIN
ruby/Cargo.lock generated Normal file

Binary file not shown.

7
ruby/Cargo.toml Normal file
View File

@@ -0,0 +1,7 @@
[workspace]
members = [
"autobuilder",
"extractor",
"generator",
"node-types",
]

71
ruby/Makefile Normal file
View File

@@ -0,0 +1,71 @@
all: extractor dbscheme
ifeq ($(OS),Windows_NT)
EXE = .exe
CODEQL_PLATFORM = win64
else
EXE =
UNAME_S := $(shell uname -s)
ifeq ($(UNAME_S),Linux)
CODEQL_PLATFORM = linux64
endif
ifeq ($(UNAME_S),Darwin)
CODEQL_PLATFORM = osx64
endif
endif
FILES=codeql-extractor.yml\
tools/qltest.cmd\
tools/index-files.sh\
tools/index-files.cmd\
tools/autobuild.sh\
tools/qltest.sh\
tools/autobuild.cmd\
ql/lib/ruby.dbscheme.stats\
ql/lib/ruby.dbscheme
BIN_FILES=target/release/ruby-extractor$(EXE) target/release/ruby-autobuilder$(EXE)
extractor-common:
rm -rf build
mkdir build
mkdir build/codeql-extractor-ruby
cp codeql-extractor.yml ql/lib/ruby.dbscheme ql/lib/ruby.dbscheme.stats build/codeql-extractor-ruby
cp -r tools build/codeql-extractor-ruby/
.PHONY: tools
tools: $(BIN_FILES)
rm -rf tools/bin
mkdir tools/bin
cp -r target/release/ruby-autobuilder$(EXE) tools/bin/autobuilder$(EXE)
cp -r target/release/ruby-extractor$(EXE) tools/bin/extractor$(EXE)
target/release/%$(EXE):
cargo build --release --bin $(basename $(notdir $@))
dbscheme:
cargo build --bin ruby-generator
cargo run -p ruby-generator -- --dbscheme ql/lib/ruby.dbscheme --library ql/lib/codeql/ruby/ast/internal/TreeSitter.qll
codeql query format -i ql/lib/codeql/ruby/ast/internal/TreeSitter.qll
.PHONY: extractor
extractor: $(FILES) $(BIN_FILES)
rm -rf extractor-pack
mkdir extractor-pack
mkdir extractor-pack/tools
mkdir extractor-pack/tools/$(CODEQL_PLATFORM)
cp codeql-extractor.yml extractor-pack/codeql-extractor.yml
cp tools/qltest.cmd extractor-pack/tools/qltest.cmd
cp tools/index-files.sh extractor-pack/tools/index-files.sh
cp tools/index-files.cmd extractor-pack/tools/index-files.cmd
cp tools/autobuild.sh extractor-pack/tools/autobuild.sh
cp tools/qltest.sh extractor-pack/tools/qltest.sh
cp tools/autobuild.cmd extractor-pack/tools/autobuild.cmd
cp ql/lib/ruby.dbscheme.stats extractor-pack/ruby.dbscheme.stats
cp ql/lib/ruby.dbscheme extractor-pack/ruby.dbscheme
cp target/release/ruby-extractor$(EXE) extractor-pack/tools/$(CODEQL_PLATFORM)/extractor$(EXE)
cp target/release/ruby-autobuilder$(EXE) extractor-pack/tools/$(CODEQL_PLATFORM)/autobuilder$(EXE)
test: extractor dbscheme
codeql pack install ql/test
codeql test run --check-databases --check-unused-labels --check-repeated-labels --check-redefined-labels --check-use-before-definition --search-path . --consistency-queries ql/consistency-queries ql/test

47
ruby/README.md Normal file
View File

@@ -0,0 +1,47 @@
# Ruby analysis support for CodeQL
Under development.
## Building the tools from source
[Install Rust](https://www.rust-lang.org/tools/install), then run:
```bash
cargo build --release
```
## Generating the database schema and QL library
The generated `ql/lib/ruby.dbscheme` and `ql/lib/codeql/ruby/ast/internal/TreeSitter.qll` files are included in the repository, but they can be re-generated as follows:
```bash
# Run the generator
cargo run --release -p ruby-generator -- --dbscheme ql/lib/ruby.dbscheme --library ql/lib/codeql/ruby/ast/internal/TreeSitter.qll
# Then auto-format the QL library
codeql query format -i ql/lib/codeql/ruby/ast/internal/TreeSitter.qll
```
## Building a CodeQL database for a Ruby program
First, get an extractor pack. There are two options:
1. Either download the latest `codeql-ruby-pack` from Actions and unzip it twice, or
2. Run `scripts/create-extractor-pack.sh` (Linux/Mac) or `scripts\create-extractor-pack.ps1` (Windows PowerShell) and the pack will be created in the `extractor-pack` directory.
Then run
```bash
codeql database create <database-path> -l ruby -s <project-source-path> --search-path <extractor-pack-path>
```
## Running qltests
Run
```bash
codeql test run <test-path> --search-path <repository-root-path>
```
## Writing database upgrade scripts
See [this guide](doc/prepare-db-upgrade.md).

View File

@@ -0,0 +1,16 @@
name: Build Ruby CodeQL pack
description: Builds the Ruby CodeQL pack
runs:
using: composite
steps:
- uses: actions/cache@v2
with:
path: |
~/.cargo/registry
~/.cargo/git
ruby/target
key: ${{ runner.os }}-qltest-cargo-${{ hashFiles('ruby/**/Cargo.lock') }}
- name: Build Extractor
shell: bash
run: scripts/create-extractor-pack.sh
working-directory: ruby

View File

@@ -0,0 +1,9 @@
[package]
name = "ruby-autobuilder"
version = "0.1.0"
authors = ["GitHub"]
edition = "2018"
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[dependencies]

View File

@@ -0,0 +1,39 @@
use std::env;
use std::path::PathBuf;
use std::process::Command;
fn main() -> std::io::Result<()> {
let dist = env::var("CODEQL_DIST").expect("CODEQL_DIST not set");
let db = env::var("CODEQL_EXTRACTOR_RUBY_WIP_DATABASE")
.expect("CODEQL_EXTRACTOR_RUBY_WIP_DATABASE not set");
let codeql = if env::consts::OS == "windows" {
"codeql.exe"
} else {
"codeql"
};
let codeql: PathBuf = [&dist, codeql].iter().collect();
let mut cmd = Command::new(codeql);
cmd.arg("database")
.arg("index-files")
.arg("--include-extension=.rb")
.arg("--include-extension=.erb")
.arg("--include-extension=.gemspec")
.arg("--include=**/Gemfile")
.arg("--size-limit=5m")
.arg("--language=ruby")
.arg("--working-dir=.")
.arg(db);
for line in env::var("LGTM_INDEX_FILTERS")
.unwrap_or_default()
.split("\n")
{
if line.starts_with("include:") {
cmd.arg("--include").arg(&line[8..]);
} else if line.starts_with("exclude:") {
cmd.arg("--exclude").arg(&line[8..]);
}
}
let exit = &cmd.spawn()?.wait()?;
std::process::exit(exit.code().unwrap_or(1))
}

14
ruby/codeql-extractor.yml Normal file
View File

@@ -0,0 +1,14 @@
name: "ruby"
display_name: "Ruby"
version: 0.1.0
column_kind: "utf8"
legacy_qltest_extraction: true
file_types:
- name: ruby
display_name: Ruby files
extensions:
- .rb
- name: erb
display_name: Ruby templates
extensions:
- .erb

View File

@@ -0,0 +1,14 @@
{
"folders": [
{
"path": "."
}
],
"settings": {
"editor.formatOnSave": true,
"files.eol": "\n",
"files.exclude": {
"codeql": true
}
}
}

View File

@@ -0,0 +1,91 @@
# Upgrading the Ruby database schema
The schema (`ql/lib/ruby.dbscheme`) is automatically generated from tree-sitter's `node-types.json`. When the tree-sitter grammar changes, the database schema is likely to change as well, and we need to write an upgrade script. This document explains how to do that.
## Process Overview
1. Commit the change to `ruby.dbscheme` (along with any library updates required to work with the change).
2. Run `scripts/prepare-db-upgrade.sh`.
3. Fill in the details in `upgrade.properties`, and add any required upgrade queries.
It may be helpful to look at some of the existing upgrade scripts, to see how they work.
## Details
### Generating a Ruby database upgrade script
Schema changes need to be accompanied by scripts that allow us to upgrade databases that were generated with older versions of the tools, so they can use new query functionality (albeit with possibly degraded results).
#### The easy (mostly automatic) way
The easy way to generate an upgrade script is to run the `scripts/prepare-db-upgrade.sh` script. This will generate a skeleton upgrade directory, leaving you to fill out the details in the `upgrade.properties` file.
#### upgrade.properties
It will look something like:
```
description: what it does
compatibility: partial
some_relation.rel: run some_relation.qlo
```
The `description` field is a textual description of the aim of the upgrade.
The `compatibility` field takes one of four values:
* **full**: results from the upgraded snapshot will be identical to results from a snapshot built with the new version of the toolchain.
* **backwards**: the step is safe and preserves the meaning of the old database, but new features may not work correctly on the upgraded snapshot.
* **partial**: the step is safe and preserves the meaning of the old database, but you would get better results if you rebuilt the snapshot with the new version of the toolchain.
* **breaking**: the step is unsafe and will prevent certain queries from working.
The `some_relation.rel` line(s) are the actions required to do the database upgrade. Do a diff on the the new vs old `.dbscheme` file to get an idea of what they have to achieve. Sometimes you won't need any upgrade commands this happens when the dbscheme has changed in "cosmetic" ways, for example by adding/removing comments or changing union type relationships, but still retains the same on-disk format for all tables; the purpose of the upgrade script is then to document the fact that it's safe to replace the old dbscheme with the new one.
Some typical upgrade commands look like this:
```
// Delete a relation that has been replaced in the new scheme
obsolete.rel: delete
// Create a new version of a table by applying a simple RA expression to an
// existing table. The example duplicates the 'id' column of input.rel as
// the last column of etended.rel, perhaps to record our best guess at
// newly-populated "source declaration" information.
extended.rel: reorder input.rel (int id, string name, int parent) id name parent id
// Create relationname.rel by running relationname.qlo and writing the query
// results as a .rel file. The query file should be named relationname.ql and
// should be placed in the upgrade directory. It should avoid using the default
// QLL library, and will run in the context of the *old* dbscheme.
relationname.rel: run relationname.qlo
```
#### Testing your upgrade script
Upgrade scripts can be a little bit fiddly, so it's essential that you test them. You might do so as follows:
1. Create a snapshot of your favourite project using the old version of the code.
2. Switch to the new version of the code.
3. Try to run some queries that will depend on your upgrade script working correctly.
4. Observe the upgrade being performed in the query server log.
5. Verify that your queries produced sensible results.
#### Doing the upgrade manually
To create the upgrade directory manually, without using `scripts/prepare-db-upgrade.sh`:
1. Get a hash of the old `.dbscheme` file, from just before your changes. You can do this by checking out the code prior to your changes and running `git hash-object ql/lib/ruby.dbscheme`
2. Go back to your branch and create an upgrade directory with that hash as its name, for example: `mkdir ql/lib/upgrades/454f1e15151422355049dc4f1f0486a03baeffef`
3. Copy the old `.dbscheme` file to that directory, using the name old.dbscheme.
`cp ql/lib/ruby.dbscheme ql/lib/upgrades/454f1e15151422355049dc4f1f0486a03baeffef/old.dbscheme`
4. Put a copy of your new `.dbscheme` file in that directory and create an `upgrade.properties` file (as described above).

20
ruby/extractor/Cargo.toml Normal file
View File

@@ -0,0 +1,20 @@
[package]
name = "ruby-extractor"
version = "0.1.0"
authors = ["GitHub"]
edition = "2018"
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[dependencies]
flate2 = "1.0"
node-types = { path = "../node-types" }
tree-sitter = "0.19"
tree-sitter-embedded-template = "0.19"
tree-sitter-ruby = "0.19"
clap = "2.33"
tracing = "0.1"
tracing-subscriber = { version = "0.2", features = ["env-filter"] }
rayon = "1.5.0"
num_cpus = "1.13.0"
regex = "1.4.3"

View File

@@ -0,0 +1,844 @@
use node_types::{EntryKind, Field, NodeTypeMap, Storage, TypeName};
use std::borrow::Cow;
use std::collections::BTreeMap as Map;
use std::collections::BTreeSet as Set;
use std::fmt;
use std::io::Write;
use std::path::Path;
use tracing::{error, info, span, Level};
use tree_sitter::{Language, Node, Parser, Range, Tree};
pub struct TrapWriter {
/// The accumulated trap entries
trap_output: Vec<TrapEntry>,
/// A counter for generating fresh labels
counter: u32,
/// cache of global keys
global_keys: std::collections::HashMap<String, Label>,
}
pub fn new_trap_writer() -> TrapWriter {
TrapWriter {
counter: 0,
trap_output: Vec::new(),
global_keys: std::collections::HashMap::new(),
}
}
impl TrapWriter {
/// Gets a label that will hold the unique ID of the passed string at import time.
/// This can be used for incrementally importable TRAP files -- use globally unique
/// strings to compute a unique ID for table tuples.
///
/// Note: You probably want to make sure that the key strings that you use are disjoint
/// for disjoint column types; the standard way of doing this is to prefix (or append)
/// the column type name to the ID. Thus, you might identify methods in Java by the
/// full ID "methods_com.method.package.DeclaringClass.method(argumentList)".
fn fresh_id(&mut self) -> Label {
let label = Label(self.counter);
self.counter += 1;
self.trap_output.push(TrapEntry::FreshId(label));
label
}
fn global_id(&mut self, key: &str) -> (Label, bool) {
if let Some(label) = self.global_keys.get(key) {
return (*label, false);
}
let label = Label(self.counter);
self.counter += 1;
self.global_keys.insert(key.to_owned(), label);
self.trap_output
.push(TrapEntry::MapLabelToKey(label, key.to_owned()));
(label, true)
}
fn add_tuple(&mut self, table_name: &str, args: Vec<Arg>) {
self.trap_output
.push(TrapEntry::GenericTuple(table_name.to_owned(), args))
}
fn populate_file(&mut self, absolute_path: &Path) -> Label {
let (file_label, fresh) = self.global_id(&full_id_for_file(absolute_path));
if fresh {
self.add_tuple(
"files",
vec![
Arg::Label(file_label),
Arg::String(normalize_path(absolute_path)),
],
);
self.populate_parent_folders(file_label, absolute_path.parent());
}
file_label
}
fn populate_empty_file(&mut self) -> Label {
let (file_label, fresh) = self.global_id("empty;sourcefile");
if fresh {
self.add_tuple(
"files",
vec![Arg::Label(file_label), Arg::String("".to_string())],
);
}
file_label
}
pub fn populate_empty_location(&mut self) {
let file_label = self.populate_empty_file();
self.location(file_label, 0, 0, 0, 0);
}
fn populate_parent_folders(&mut self, child_label: Label, path: Option<&Path>) {
let mut path = path;
let mut child_label = child_label;
loop {
match path {
None => break,
Some(folder) => {
let (folder_label, fresh) = self.global_id(&full_id_for_folder(folder));
self.add_tuple(
"containerparent",
vec![Arg::Label(folder_label), Arg::Label(child_label)],
);
if fresh {
self.add_tuple(
"folders",
vec![
Arg::Label(folder_label),
Arg::String(normalize_path(folder)),
],
);
path = folder.parent();
child_label = folder_label;
} else {
break;
}
}
}
}
}
fn location(
&mut self,
file_label: Label,
start_line: usize,
start_column: usize,
end_line: usize,
end_column: usize,
) -> Label {
let (loc_label, fresh) = self.global_id(&format!(
"loc,{{{}}},{},{},{},{}",
file_label, start_line, start_column, end_line, end_column
));
if fresh {
self.add_tuple(
"locations_default",
vec![
Arg::Label(loc_label),
Arg::Label(file_label),
Arg::Int(start_line),
Arg::Int(start_column),
Arg::Int(end_line),
Arg::Int(end_column),
],
);
}
loc_label
}
fn comment(&mut self, text: String) {
self.trap_output.push(TrapEntry::Comment(text));
}
pub fn output(self, writer: &mut dyn Write) -> std::io::Result<()> {
write!(writer, "{}", Program(self.trap_output))
}
}
/// Extracts the source file at `path`, which is assumed to be canonicalized.
pub fn extract(
language: Language,
language_prefix: &str,
schema: &NodeTypeMap,
trap_writer: &mut TrapWriter,
path: &Path,
source: &Vec<u8>,
ranges: &[Range],
) -> std::io::Result<()> {
let span = span!(
Level::TRACE,
"extract",
file = %path.display()
);
let _enter = span.enter();
info!("extracting: {}", path.display());
let mut parser = Parser::new();
parser.set_language(language).unwrap();
parser.set_included_ranges(&ranges).unwrap();
let tree = parser.parse(&source, None).expect("Failed to parse file");
trap_writer.comment(format!("Auto-generated TRAP file for {}", path.display()));
let file_label = &trap_writer.populate_file(path);
let mut visitor = Visitor {
source: &source,
trap_writer: trap_writer,
// TODO: should we handle path strings that are not valid UTF8 better?
path: format!("{}", path.display()),
file_label: *file_label,
toplevel_child_counter: 0,
stack: Vec::new(),
language_prefix,
schema,
};
traverse(&tree, &mut visitor);
parser.reset();
Ok(())
}
/// Escapes a string for use in a TRAP key, by replacing special characters with
/// HTML entities.
fn escape_key<'a, S: Into<Cow<'a, str>>>(key: S) -> Cow<'a, str> {
fn needs_escaping(c: char) -> bool {
match c {
'&' => true,
'{' => true,
'}' => true,
'"' => true,
'@' => true,
'#' => true,
_ => false,
}
}
let key = key.into();
if key.contains(needs_escaping) {
let mut escaped = String::with_capacity(2 * key.len());
for c in key.chars() {
match c {
'&' => escaped.push_str("&amp;"),
'{' => escaped.push_str("&lbrace;"),
'}' => escaped.push_str("&rbrace;"),
'"' => escaped.push_str("&quot;"),
'@' => escaped.push_str("&commat;"),
'#' => escaped.push_str("&num;"),
_ => escaped.push(c),
}
}
Cow::Owned(escaped)
} else {
key
}
}
/// Normalizes the path according the common CodeQL specification. Assumes that
/// `path` has already been canonicalized using `std::fs::canonicalize`.
fn normalize_path(path: &Path) -> String {
if cfg!(windows) {
// The way Rust canonicalizes paths doesn't match the CodeQL spec, so we
// have to do a bit of work removing certain prefixes and replacing
// backslashes.
let mut components: Vec<String> = Vec::new();
for component in path.components() {
match component {
std::path::Component::Prefix(prefix) => match prefix.kind() {
std::path::Prefix::Disk(letter) | std::path::Prefix::VerbatimDisk(letter) => {
components.push(format!("{}:", letter as char));
}
std::path::Prefix::Verbatim(x) | std::path::Prefix::DeviceNS(x) => {
components.push(x.to_string_lossy().to_string());
}
std::path::Prefix::UNC(server, share)
| std::path::Prefix::VerbatimUNC(server, share) => {
components.push(server.to_string_lossy().to_string());
components.push(share.to_string_lossy().to_string());
}
},
std::path::Component::Normal(n) => {
components.push(n.to_string_lossy().to_string());
}
std::path::Component::RootDir => {}
std::path::Component::CurDir => {}
std::path::Component::ParentDir => {}
}
}
components.join("/")
} else {
// For other operating systems, we can use the canonicalized path
// without modifications.
format!("{}", path.display())
}
}
fn full_id_for_file(path: &Path) -> String {
format!("{};sourcefile", escape_key(&normalize_path(path)))
}
fn full_id_for_folder(path: &Path) -> String {
format!("{};folder", escape_key(&normalize_path(path)))
}
struct ChildNode {
field_name: Option<&'static str>,
label: Label,
type_name: TypeName,
}
struct Visitor<'a> {
/// The file path of the source code (as string)
path: String,
/// The label to use whenever we need to refer to the `@file` entity of this
/// source file.
file_label: Label,
/// The source code as a UTF-8 byte array
source: &'a Vec<u8>,
/// A TrapWriter to accumulate trap entries
trap_writer: &'a mut TrapWriter,
/// A counter for top-level child nodes
toplevel_child_counter: usize,
/// Language prefix
language_prefix: &'a str,
/// A lookup table from type name to node types
schema: &'a NodeTypeMap,
/// A stack for gathering information from child nodes. Whenever a node is
/// entered the parent's [Label], child counter, and an empty list is pushed.
/// All children append their data to the the list. When the visitor leaves a
/// node the list containing the child data is popped from the stack and
/// matched against the dbscheme for the node. If the expectations are met
/// the corresponding row definitions are added to the trap_output.
stack: Vec<(Label, usize, Vec<ChildNode>)>,
}
impl Visitor<'_> {
fn record_parse_error(
&mut self,
error_message: String,
full_error_message: String,
loc: Label,
) {
error!("{}", full_error_message);
let id = self.trap_writer.fresh_id();
self.trap_writer.add_tuple(
"diagnostics",
vec![
Arg::Label(id),
Arg::Int(40), // severity 40 = error
Arg::String("parse_error".to_string()),
Arg::String(error_message),
Arg::String(full_error_message),
Arg::Label(loc),
],
);
}
fn record_parse_error_for_node(
&mut self,
error_message: String,
full_error_message: String,
node: Node,
) {
let (start_line, start_column, end_line, end_column) = location_for(&self.source, node);
let loc = self.trap_writer.location(
self.file_label,
start_line,
start_column,
end_line,
end_column,
);
self.record_parse_error(error_message, full_error_message, loc);
}
fn enter_node(&mut self, node: Node) -> bool {
if node.is_error() || node.is_missing() {
let error_message = if node.is_missing() {
format!("parse error: expecting '{}'", node.kind())
} else {
"parse error".to_string()
};
let full_error_message = format!(
"{}:{}: {}",
&self.path,
node.start_position().row + 1,
error_message
);
self.record_parse_error_for_node(error_message, full_error_message, node);
return false;
}
let id = self.trap_writer.fresh_id();
self.stack.push((id, 0, Vec::new()));
return true;
}
fn leave_node(&mut self, field_name: Option<&'static str>, node: Node) {
if node.is_error() || node.is_missing() {
return;
}
let (id, _, child_nodes) = self.stack.pop().expect("Vistor: empty stack");
let (start_line, start_column, end_line, end_column) = location_for(&self.source, node);
let loc = self.trap_writer.location(
self.file_label,
start_line,
start_column,
end_line,
end_column,
);
let table = self
.schema
.get(&TypeName {
kind: node.kind().to_owned(),
named: node.is_named(),
})
.unwrap();
let mut valid = true;
let (parent_id, parent_index) = match self.stack.last_mut() {
Some(p) if !node.is_extra() => {
p.1 += 1;
(p.0, p.1 - 1)
}
_ => {
self.toplevel_child_counter += 1;
(self.file_label, self.toplevel_child_counter - 1)
}
};
match &table.kind {
EntryKind::Token { kind_id, .. } => {
self.trap_writer.add_tuple(
&format!("{}_ast_node_parent", self.language_prefix),
vec![
Arg::Label(id),
Arg::Label(parent_id),
Arg::Int(parent_index),
],
);
self.trap_writer.add_tuple(
&format!("{}_tokeninfo", self.language_prefix),
vec![
Arg::Label(id),
Arg::Int(*kind_id),
sliced_source_arg(self.source, node),
Arg::Label(loc),
],
);
}
EntryKind::Table {
fields,
name: table_name,
} => {
if let Some(args) = self.complex_node(&node, fields, &child_nodes, id) {
self.trap_writer.add_tuple(
&format!("{}_ast_node_parent", self.language_prefix),
vec![
Arg::Label(id),
Arg::Label(parent_id),
Arg::Int(parent_index),
],
);
let mut all_args = Vec::new();
all_args.push(Arg::Label(id));
all_args.extend(args);
all_args.push(Arg::Label(loc));
self.trap_writer.add_tuple(&table_name, all_args);
}
}
_ => {
let error_message = format!("unknown table type: '{}'", node.kind());
let full_error_message = format!(
"{}:{}: {}",
&self.path,
node.start_position().row + 1,
error_message
);
self.record_parse_error(error_message, full_error_message, loc);
valid = false;
}
}
if valid && !node.is_extra() {
// Extra nodes are independent root nodes and do not belong to the parent node
// Therefore we should not register them in the parent vector
if let Some(parent) = self.stack.last_mut() {
parent.2.push(ChildNode {
field_name,
label: id,
type_name: TypeName {
kind: node.kind().to_owned(),
named: node.is_named(),
},
});
};
}
}
fn complex_node(
&mut self,
node: &Node,
fields: &Vec<Field>,
child_nodes: &Vec<ChildNode>,
parent_id: Label,
) -> Option<Vec<Arg>> {
let mut map: Map<&Option<String>, (&Field, Vec<Arg>)> = Map::new();
for field in fields {
map.insert(&field.name, (field, Vec::new()));
}
for child_node in child_nodes {
if let Some((field, values)) = map.get_mut(&child_node.field_name.map(|x| x.to_owned()))
{
//TODO: handle error and missing nodes
if self.type_matches(&child_node.type_name, &field.type_info) {
if let node_types::FieldTypeInfo::ReservedWordInt(int_mapping) =
&field.type_info
{
// We can safely unwrap because type_matches checks the key is in the map.
let (int_value, _) = int_mapping.get(&child_node.type_name.kind).unwrap();
values.push(Arg::Int(*int_value));
} else {
values.push(Arg::Label(child_node.label));
}
} else if field.name.is_some() {
let error_message = format!(
"type mismatch for field {}::{} with type {:?} != {:?}",
node.kind(),
child_node.field_name.unwrap_or("child"),
child_node.type_name,
field.type_info
);
let full_error_message = format!(
"{}:{}: {}",
&self.path,
node.start_position().row + 1,
error_message
);
self.record_parse_error_for_node(error_message, full_error_message, *node);
}
} else {
if child_node.field_name.is_some() || child_node.type_name.named {
let error_message = format!(
"value for unknown field: {}::{} and type {:?}",
node.kind(),
&child_node.field_name.unwrap_or("child"),
&child_node.type_name
);
let full_error_message = format!(
"{}:{}: {}",
&self.path,
node.start_position().row + 1,
error_message
);
self.record_parse_error_for_node(error_message, full_error_message, *node);
}
}
}
let mut args = Vec::new();
let mut is_valid = true;
for field in fields {
let child_values = &map.get(&field.name).unwrap().1;
match &field.storage {
Storage::Column { name: column_name } => {
if child_values.len() == 1 {
args.push(child_values.first().unwrap().clone());
} else {
is_valid = false;
let error_message = format!(
"{} for field: {}::{}",
if child_values.is_empty() {
"missing value"
} else {
"too many values"
},
node.kind(),
column_name
);
let full_error_message = format!(
"{}:{}: {}",
&self.path,
node.start_position().row + 1,
error_message
);
self.record_parse_error_for_node(error_message, full_error_message, *node);
}
}
Storage::Table {
name: table_name,
has_index,
column_name: _,
} => {
for (index, child_value) in child_values.iter().enumerate() {
if !*has_index && index > 0 {
error!(
"{}:{}: too many values for field: {}::{}",
&self.path,
node.start_position().row + 1,
node.kind(),
table_name,
);
break;
}
let mut args = Vec::new();
args.push(Arg::Label(parent_id));
if *has_index {
args.push(Arg::Int(index))
}
args.push(child_value.clone());
self.trap_writer.add_tuple(&table_name, args);
}
}
}
}
if is_valid {
Some(args)
} else {
None
}
}
fn type_matches(&self, tp: &TypeName, type_info: &node_types::FieldTypeInfo) -> bool {
match type_info {
node_types::FieldTypeInfo::Single(single_type) => {
if tp == single_type {
return true;
}
match &self.schema.get(single_type).unwrap().kind {
EntryKind::Union { members } => {
if self.type_matches_set(tp, members) {
return true;
}
}
_ => {}
}
}
node_types::FieldTypeInfo::Multiple { types, .. } => {
return self.type_matches_set(tp, types);
}
node_types::FieldTypeInfo::ReservedWordInt(int_mapping) => {
return !tp.named && int_mapping.contains_key(&tp.kind)
}
}
false
}
fn type_matches_set(&self, tp: &TypeName, types: &Set<TypeName>) -> bool {
if types.contains(tp) {
return true;
}
for other in types.iter() {
if let EntryKind::Union { members } = &self.schema.get(other).unwrap().kind {
if self.type_matches_set(tp, members) {
return true;
}
}
}
false
}
}
// Emit a slice of a source file as an Arg.
fn sliced_source_arg(source: &Vec<u8>, n: Node) -> Arg {
let range = n.byte_range();
Arg::String(String::from_utf8_lossy(&source[range.start..range.end]).into_owned())
}
// Emit a pair of `TrapEntry`s for the provided node, appropriately calibrated.
// The first is the location and label definition, and the second is the
// 'Located' entry.
fn location_for<'a>(source: &Vec<u8>, n: Node) -> (usize, usize, usize, usize) {
// Tree-sitter row, column values are 0-based while CodeQL starts
// counting at 1. In addition Tree-sitter's row and column for the
// end position are exclusive while CodeQL's end positions are inclusive.
// This means that all values should be incremented by 1 and in addition the
// end position needs to be shift 1 to the left. In most cases this means
// simply incrementing all values except the end column except in cases where
// the end column is 0 (start of a line). In such cases the end position must be
// set to the end of the previous line.
let start_line = n.start_position().row + 1;
let start_col = n.start_position().column + 1;
let mut end_line = n.end_position().row + 1;
let mut end_col = n.end_position().column;
if start_line > end_line || start_line == end_line && start_col > end_col {
// the range is empty, clip it to sensible values
end_line = start_line;
end_col = start_col - 1;
} else if end_col == 0 {
// end_col = 0 means that we are at the start of a line
// unfortunately 0 is invalid as column number, therefore
// we should update the end location to be the end of the
// previous line
let mut index = n.end_byte();
if index > 0 && index <= source.len() {
index -= 1;
if source[index] != b'\n' {
error!("expecting a line break symbol, but none found while correcting end column value");
}
end_line -= 1;
end_col = 1;
while index > 0 && source[index - 1] != b'\n' {
index -= 1;
end_col += 1;
}
} else {
error!(
"cannot correct end column value: end_byte index {} is not in range [1,{}]",
index,
source.len()
);
}
}
(start_line, start_col, end_line, end_col)
}
fn traverse(tree: &Tree, visitor: &mut Visitor) {
let cursor = &mut tree.walk();
visitor.enter_node(cursor.node());
let mut recurse = true;
loop {
if recurse && cursor.goto_first_child() {
recurse = visitor.enter_node(cursor.node());
} else {
visitor.leave_node(cursor.field_name(), cursor.node());
if cursor.goto_next_sibling() {
recurse = visitor.enter_node(cursor.node());
} else if cursor.goto_parent() {
recurse = false;
} else {
break;
}
}
}
}
pub struct Program(Vec<TrapEntry>);
impl fmt::Display for Program {
fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
let mut text = String::new();
for trap_entry in &self.0 {
text.push_str(&format!("{}\n", trap_entry));
}
write!(f, "{}", text)
}
}
enum TrapEntry {
/// Maps the label to a fresh id, e.g. `#123=*`.
FreshId(Label),
/// Maps the label to a key, e.g. `#7=@"foo"`.
MapLabelToKey(Label, String),
/// foo_bar(arg*)
GenericTuple(String, Vec<Arg>),
Comment(String),
}
impl fmt::Display for TrapEntry {
fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
match self {
TrapEntry::FreshId(label) => write!(f, "{}=*", label),
TrapEntry::MapLabelToKey(label, key) => {
write!(f, "{}=@\"{}\"", label, key.replace("\"", "\"\""))
}
TrapEntry::GenericTuple(name, args) => {
write!(f, "{}(", name)?;
for (index, arg) in args.iter().enumerate() {
if index > 0 {
write!(f, ",")?;
}
write!(f, "{}", arg)?;
}
write!(f, ")")
}
TrapEntry::Comment(line) => write!(f, "// {}", line),
}
}
}
#[derive(Debug, Copy, Clone)]
// Identifiers of the form #0, #1...
struct Label(u32);
impl fmt::Display for Label {
fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
write!(f, "#{:x}", self.0)
}
}
// Numeric indices.
#[derive(Debug, Copy, Clone)]
struct Index(usize);
impl fmt::Display for Index {
fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
write!(f, "{}", self.0)
}
}
// Some untyped argument to a TrapEntry.
#[derive(Debug, Clone)]
enum Arg {
Label(Label),
Int(usize),
String(String),
}
const MAX_STRLEN: usize = 1048576;
impl fmt::Display for Arg {
fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
match self {
Arg::Label(x) => write!(f, "{}", x),
Arg::Int(x) => write!(f, "{}", x),
Arg::String(x) => write!(
f,
"\"{}\"",
limit_string(x, MAX_STRLEN).replace("\"", "\"\"")
),
}
}
}
/// Limit the length (in bytes) of a string. If the string's length in bytes is
/// less than or equal to the limit then the entire string is returned. Otherwise
/// the string is sliced at the provided limit. If there is a multi-byte character
/// at the limit then the returned slice will be slightly shorter than the limit to
/// avoid splitting that multi-byte character.
fn limit_string(string: &String, max_size: usize) -> &str {
if string.len() <= max_size {
return string;
}
let p = string.as_ptr();
let mut index = max_size;
// We want to clip the string at [max_size]; however, the character at that position
// may span several bytes. We need to find the first byte of the character. In UTF-8
// encoded data any byte that matches the bit pattern 10XXXXXX is not a start byte.
// Therefore we decrement the index as long as there are bytes matching this pattern.
// This ensures we cut the string at the border between one character and another.
while index > 0 && unsafe { (*p.offset(index as isize) & 0b11000000) == 0b10000000 } {
index -= 1;
}
&string[0..index]
}
#[test]
fn limit_string_test() {
assert_eq!("hello", limit_string(&"hello world".to_owned(), 5));
assert_eq!("hi ☹", limit_string(&"hi ☹☹".to_owned(), 6));
assert_eq!("hi ", limit_string(&"hi ☹☹".to_owned(), 5));
}
#[test]
fn escape_key_test() {
assert_eq!("foo!", escape_key("foo!"));
assert_eq!("foo&lbrace;&rbrace;", escape_key("foo{}"));
assert_eq!("&lbrace;&rbrace;", escape_key("{}"));
assert_eq!("", escape_key(""));
assert_eq!("/path/to/foo.rb", escape_key("/path/to/foo.rb"));
assert_eq!(
"/path/to/foo&amp;&lbrace;&rbrace;&quot;&commat;&num;.rb",
escape_key("/path/to/foo&{}\"@#.rb")
);
}

300
ruby/extractor/src/main.rs Normal file
View File

@@ -0,0 +1,300 @@
mod extractor;
extern crate num_cpus;
use clap;
use flate2::write::GzEncoder;
use rayon::prelude::*;
use std::fs;
use std::io::{BufRead, BufWriter};
use std::path::{Path, PathBuf};
use tree_sitter::{Language, Parser, Range};
enum TrapCompression {
None,
Gzip,
}
impl TrapCompression {
fn from_env() -> TrapCompression {
match std::env::var("CODEQL_RUBY_TRAP_COMPRESSION") {
Ok(method) => match TrapCompression::from_string(&method) {
Some(c) => c,
None => {
tracing::error!("Unknown compression method '{}'; using gzip.", &method);
TrapCompression::Gzip
}
},
// Default compression method if the env var isn't set:
Err(_) => TrapCompression::Gzip,
}
}
fn from_string(s: &str) -> Option<TrapCompression> {
match s.to_lowercase().as_ref() {
"none" => Some(TrapCompression::None),
"gzip" => Some(TrapCompression::Gzip),
_ => None,
}
}
fn extension(&self) -> &str {
match self {
TrapCompression::None => "trap",
TrapCompression::Gzip => "trap.gz",
}
}
}
/**
* Gets the number of threads the extractor should use, by reading the
* CODEQL_THREADS environment variable and using it as described in the
* extractor spec:
*
* "If the number is positive, it indicates the number of threads that should
* be used. If the number is negative or zero, it should be added to the number
* of cores available on the machine to determine how many threads to use
* (minimum of 1). If unspecified, should be considered as set to -1."
*/
fn num_codeql_threads() -> usize {
let threads_str = std::env::var("CODEQL_THREADS").unwrap_or("-1".to_owned());
match threads_str.parse::<i32>() {
Ok(num) if num <= 0 => {
let reduction = -num as usize;
std::cmp::max(1, num_cpus::get() - reduction)
}
Ok(num) => num as usize,
Err(_) => {
tracing::error!(
"Unable to parse CODEQL_THREADS value '{}'; defaulting to 1 thread.",
&threads_str
);
1
}
}
}
fn main() -> std::io::Result<()> {
tracing_subscriber::fmt()
.with_target(false)
.without_time()
.with_level(true)
.with_env_filter(tracing_subscriber::EnvFilter::from_default_env())
.init();
let num_threads = num_codeql_threads();
tracing::info!(
"Using {} {}",
num_threads,
if num_threads == 1 {
"thread"
} else {
"threads"
}
);
rayon::ThreadPoolBuilder::new()
.num_threads(num_threads)
.build_global()
.unwrap();
let matches = clap::App::new("Ruby extractor")
.version("1.0")
.author("GitHub")
.about("CodeQL Ruby extractor")
.args_from_usage(
"--source-archive-dir=<DIR> 'Sets a custom source archive folder'
--output-dir=<DIR> 'Sets a custom trap folder'
--file-list=<FILE_LIST> 'A text files containing the paths of the files to extract'",
)
.get_matches();
let src_archive_dir = matches
.value_of("source-archive-dir")
.expect("missing --source-archive-dir");
let src_archive_dir = PathBuf::from(src_archive_dir);
let trap_dir = matches
.value_of("output-dir")
.expect("missing --output-dir");
let trap_dir = PathBuf::from(trap_dir);
let trap_compression = TrapCompression::from_env();
let file_list = matches.value_of("file-list").expect("missing --file-list");
let file_list = fs::File::open(file_list)?;
let language = tree_sitter_ruby::language();
let erb = tree_sitter_embedded_template::language();
// Look up tree-sitter kind ids now, to avoid string comparisons when scanning ERB files.
let erb_directive_id = erb.id_for_node_kind("directive", true);
let erb_output_directive_id = erb.id_for_node_kind("output_directive", true);
let erb_code_id = erb.id_for_node_kind("code", true);
let schema = node_types::read_node_types_str("ruby", tree_sitter_ruby::NODE_TYPES)?;
let erb_schema =
node_types::read_node_types_str("erb", tree_sitter_embedded_template::NODE_TYPES)?;
let lines: std::io::Result<Vec<String>> = std::io::BufReader::new(file_list).lines().collect();
let lines = lines?;
lines
.par_iter()
.try_for_each(|line| {
let path = PathBuf::from(line).canonicalize()?;
let src_archive_file = path_for(&src_archive_dir, &path, "");
let mut source = std::fs::read(&path)?;
let code_ranges;
let mut trap_writer = extractor::new_trap_writer();
if path.extension().map_or(false, |x| x == "erb") {
tracing::info!("scanning: {}", path.display());
extractor::extract(
erb,
"erb",
&erb_schema,
&mut trap_writer,
&path,
&source,
&[],
)?;
let (ranges, line_breaks) = scan_erb(
erb,
&source,
erb_directive_id,
erb_output_directive_id,
erb_code_id,
);
for i in line_breaks {
if i < source.len() {
source[i] = b'\n';
}
}
code_ranges = ranges;
} else {
code_ranges = vec![];
}
extractor::extract(
language,
"ruby",
&schema,
&mut trap_writer,
&path,
&source,
&code_ranges,
)?;
std::fs::create_dir_all(&src_archive_file.parent().unwrap())?;
std::fs::copy(&path, &src_archive_file)?;
write_trap(&trap_dir, path, trap_writer, &trap_compression)
})
.expect("failed to extract files");
let path = PathBuf::from("extras");
let mut trap_writer = extractor::new_trap_writer();
trap_writer.populate_empty_location();
write_trap(&trap_dir, path, trap_writer, &trap_compression)
}
fn write_trap(
trap_dir: &PathBuf,
path: PathBuf,
trap_writer: extractor::TrapWriter,
trap_compression: &TrapCompression,
) -> std::io::Result<()> {
let trap_file = path_for(&trap_dir, &path, trap_compression.extension());
std::fs::create_dir_all(&trap_file.parent().unwrap())?;
let trap_file = std::fs::File::create(&trap_file)?;
let mut trap_file = BufWriter::new(trap_file);
match trap_compression {
TrapCompression::None => trap_writer.output(&mut trap_file),
TrapCompression::Gzip => {
let mut compressed_writer = GzEncoder::new(trap_file, flate2::Compression::fast());
trap_writer.output(&mut compressed_writer)
}
}
}
fn scan_erb(
erb: Language,
source: &Vec<u8>,
directive_id: u16,
output_directive_id: u16,
code_id: u16,
) -> (Vec<Range>, Vec<usize>) {
let mut parser = Parser::new();
parser.set_language(erb).unwrap();
let tree = parser.parse(&source, None).expect("Failed to parse file");
let mut result = Vec::new();
let mut line_breaks = vec![];
for n in tree.root_node().children(&mut tree.walk()) {
let kind_id = n.kind_id();
if kind_id == directive_id || kind_id == output_directive_id {
for c in n.children(&mut tree.walk()) {
if c.kind_id() == code_id {
let mut range = c.range();
if range.end_byte < source.len() {
line_breaks.push(range.end_byte);
range.end_byte += 1;
range.end_point.column += 1;
}
result.push(range);
}
}
}
}
if result.len() == 0 {
let root = tree.root_node();
// Add an empty range at the end of the file
result.push(Range {
start_byte: root.end_byte(),
end_byte: root.end_byte(),
start_point: root.end_position(),
end_point: root.end_position(),
});
}
(result, line_breaks)
}
fn path_for(dir: &Path, path: &Path, ext: &str) -> PathBuf {
let mut result = PathBuf::from(dir);
for component in path.components() {
match component {
std::path::Component::Prefix(prefix) => match prefix.kind() {
std::path::Prefix::Disk(letter) | std::path::Prefix::VerbatimDisk(letter) => {
result.push(format!("{}_", letter as char))
}
std::path::Prefix::Verbatim(x) | std::path::Prefix::DeviceNS(x) => {
result.push(x);
}
std::path::Prefix::UNC(server, share)
| std::path::Prefix::VerbatimUNC(server, share) => {
result.push("unc");
result.push(server);
result.push(share);
}
},
std::path::Component::RootDir => {
// skip
}
std::path::Component::Normal(_) => {
result.push(component);
}
std::path::Component::CurDir => {
// skip
}
std::path::Component::ParentDir => {
result.pop();
}
}
}
if !ext.is_empty() {
match result.extension() {
Some(x) => {
let mut new_ext = x.to_os_string();
new_ext.push(".");
new_ext.push(ext);
result.set_extension(new_ext);
}
None => {
result.set_extension(ext);
}
}
}
result
}

15
ruby/generator/Cargo.toml Normal file
View File

@@ -0,0 +1,15 @@
[package]
name = "ruby-generator"
version = "0.1.0"
authors = ["GitHub"]
edition = "2018"
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[dependencies]
clap = "2.33"
node-types = { path = "../node-types" }
tracing = "0.1"
tracing-subscriber = { version = "0.2", features = ["env-filter"] }
tree-sitter-embedded-template = "0.19"
tree-sitter-ruby = "0.19"

View File

@@ -0,0 +1,130 @@
use crate::ql;
use std::collections::BTreeSet as Set;
use std::fmt;
/// Represents a distinct entry in the database schema.
pub enum Entry<'a> {
/// An entry defining a database table.
Table(Table<'a>),
/// An entry defining a database table.
Case(Case<'a>),
/// An entry defining type that is a union of other types.
Union(Union<'a>),
}
/// A table in the database schema.
pub struct Table<'a> {
pub name: &'a str,
pub columns: Vec<Column<'a>>,
pub keysets: Option<Vec<&'a str>>,
}
/// A union in the database schema.
pub struct Union<'a> {
pub name: &'a str,
pub members: Set<&'a str>,
}
/// A table in the database schema.
pub struct Case<'a> {
pub name: &'a str,
pub column: &'a str,
pub branches: Vec<(usize, &'a str)>,
}
/// A column in a table.
pub struct Column<'a> {
pub db_type: DbColumnType,
pub name: &'a str,
pub unique: bool,
pub ql_type: ql::Type<'a>,
pub ql_type_is_ref: bool,
}
/// The database column type.
pub enum DbColumnType {
Int,
String,
}
impl<'a> fmt::Display for Case<'a> {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
writeln!(f, "case @{}.{} of", &self.name, &self.column)?;
let mut sep = " ";
for (c, tp) in &self.branches {
writeln!(f, "{} {} = @{}", sep, c, tp)?;
sep = "|";
}
writeln!(f, ";")
}
}
impl<'a> fmt::Display for Table<'a> {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
if let Some(keyset) = &self.keysets {
write!(f, "#keyset[")?;
for (key_index, key) in keyset.iter().enumerate() {
if key_index > 0 {
write!(f, ", ")?;
}
write!(f, "{}", key)?;
}
write!(f, "]\n")?;
}
write!(f, "{}(\n", self.name)?;
for (column_index, column) in self.columns.iter().enumerate() {
write!(f, " ")?;
if column.unique {
write!(f, "unique ")?;
}
write!(
f,
"{} ",
match column.db_type {
DbColumnType::Int => "int",
DbColumnType::String => "string",
}
)?;
write!(f, "{}: {}", column.name, column.ql_type)?;
if column.ql_type_is_ref {
write!(f, " ref")?;
}
if column_index + 1 != self.columns.len() {
write!(f, ",")?;
}
write!(f, "\n")?;
}
write!(f, ");")?;
Ok(())
}
}
impl<'a> fmt::Display for Union<'a> {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
write!(f, "@{} = ", self.name)?;
let mut first = true;
for member in &self.members {
if first {
first = false;
} else {
write!(f, " | ")?;
}
write!(f, "@{}", member)?;
}
Ok(())
}
}
/// Generates the dbscheme by writing the given dbscheme `entries` to the `file`.
pub fn write<'a>(file: &mut dyn std::io::Write, entries: &'a [Entry]) -> std::io::Result<()> {
for entry in entries {
match entry {
Entry::Case(case) => write!(file, "{}\n\n", case)?,
Entry::Table(table) => write!(file, "{}\n\n", table)?,
Entry::Union(union) => write!(file, "{}\n\n", union)?,
}
}
Ok(())
}

View File

@@ -0,0 +1,4 @@
pub struct Language {
pub name: String,
pub node_types: &'static str,
}

672
ruby/generator/src/main.rs Normal file
View File

@@ -0,0 +1,672 @@
mod dbscheme;
mod language;
mod ql;
mod ql_gen;
use clap;
use language::Language;
use std::collections::BTreeMap as Map;
use std::collections::BTreeSet as Set;
use std::fs::File;
use std::io::LineWriter;
use std::io::Write;
use std::path::PathBuf;
/// Given the name of the parent node, and its field information, returns a pair,
/// the first of which is the field's type. The second is an optional dbscheme
/// entry that should be added.
fn make_field_type<'a>(
parent_name: &'a str,
field: &'a node_types::Field,
nodes: &'a node_types::NodeTypeMap,
) -> (ql::Type<'a>, Option<dbscheme::Entry<'a>>) {
match &field.type_info {
node_types::FieldTypeInfo::Multiple {
types,
dbscheme_union,
ql_class: _,
} => {
// This field can have one of several types. Create an ad-hoc QL union
// type to represent them.
let members: Set<&str> = types
.iter()
.map(|t| nodes.get(t).unwrap().dbscheme_name.as_str())
.collect();
(
ql::Type::AtType(&dbscheme_union),
Some(dbscheme::Entry::Union(dbscheme::Union {
name: dbscheme_union,
members,
})),
)
}
node_types::FieldTypeInfo::Single(t) => {
let dbscheme_name = &nodes.get(&t).unwrap().dbscheme_name;
(ql::Type::AtType(dbscheme_name), None)
}
node_types::FieldTypeInfo::ReservedWordInt(int_mapping) => {
// The field will be an `int` in the db, and we add a case split to
// create other db types for each integer value.
let mut branches: Vec<(usize, &'a str)> = Vec::new();
for (_, (value, name)) in int_mapping {
branches.push((*value, name));
}
let case = dbscheme::Entry::Case(dbscheme::Case {
name: parent_name,
column: match &field.storage {
node_types::Storage::Column { name } => name,
node_types::Storage::Table { name, .. } => name,
},
branches,
});
(ql::Type::Int, Some(case))
}
}
}
fn add_field_for_table_storage<'a>(
field: &'a node_types::Field,
table_name: &'a str,
column_name: &'a str,
has_index: bool,
nodes: &'a node_types::NodeTypeMap,
) -> (dbscheme::Table<'a>, Option<dbscheme::Entry<'a>>) {
let parent_name = &nodes.get(&field.parent).unwrap().dbscheme_name;
// This field can appear zero or multiple times, so put
// it in an auxiliary table.
let (field_ql_type, field_type_entry) = make_field_type(parent_name, &field, nodes);
let parent_column = dbscheme::Column {
unique: !has_index,
db_type: dbscheme::DbColumnType::Int,
name: &parent_name,
ql_type: ql::Type::AtType(&parent_name),
ql_type_is_ref: true,
};
let index_column = dbscheme::Column {
unique: false,
db_type: dbscheme::DbColumnType::Int,
name: "index",
ql_type: ql::Type::Int,
ql_type_is_ref: true,
};
let field_column = dbscheme::Column {
unique: true,
db_type: dbscheme::DbColumnType::Int,
name: column_name,
ql_type: field_ql_type,
ql_type_is_ref: true,
};
let field_table = dbscheme::Table {
name: &table_name,
columns: if has_index {
vec![parent_column, index_column, field_column]
} else {
vec![parent_column, field_column]
},
// In addition to the field being unique, the combination of
// parent+index is unique, so add a keyset for them.
keysets: if has_index {
Some(vec![&parent_name, "index"])
} else {
None
},
};
(field_table, field_type_entry)
}
fn add_field_for_column_storage<'a>(
parent_name: &'a str,
field: &'a node_types::Field,
column_name: &'a str,
nodes: &'a node_types::NodeTypeMap,
) -> (dbscheme::Column<'a>, Option<dbscheme::Entry<'a>>) {
// This field must appear exactly once, so we add it as
// a column to the main table for the node type.
let (field_ql_type, field_type_entry) = make_field_type(parent_name, &field, nodes);
(
dbscheme::Column {
unique: false,
db_type: dbscheme::DbColumnType::Int,
name: column_name,
ql_type: field_ql_type,
ql_type_is_ref: true,
},
field_type_entry,
)
}
/// Converts the given tree-sitter node types into CodeQL dbscheme entries.
/// Returns a tuple containing:
///
/// 1. A vector of dbscheme entries.
/// 2. A set of names of the members of the `<lang>_ast_node` union.
/// 3. A map where the keys are the dbscheme names for token kinds, and the
/// values are their integer representations.
fn convert_nodes<'a>(
nodes: &'a node_types::NodeTypeMap,
) -> (Vec<dbscheme::Entry<'a>>, Set<&'a str>, Map<&'a str, usize>) {
let mut entries: Vec<dbscheme::Entry> = Vec::new();
let mut ast_node_members: Set<&str> = Set::new();
let token_kinds: Map<&str, usize> = nodes
.iter()
.filter_map(|(_, node)| match &node.kind {
node_types::EntryKind::Token { kind_id } => {
Some((node.dbscheme_name.as_str(), *kind_id))
}
_ => None,
})
.collect();
for (_, node) in nodes {
match &node.kind {
node_types::EntryKind::Union { members: n_members } => {
// It's a tree-sitter supertype node, for which we create a union
// type.
let members: Set<&str> = n_members
.iter()
.map(|n| nodes.get(n).unwrap().dbscheme_name.as_str())
.collect();
entries.push(dbscheme::Entry::Union(dbscheme::Union {
name: &node.dbscheme_name,
members,
}));
}
node_types::EntryKind::Table { name, fields } => {
// It's a product type, defined by a table.
let mut main_table = dbscheme::Table {
name: &name,
columns: vec![dbscheme::Column {
db_type: dbscheme::DbColumnType::Int,
name: "id",
unique: true,
ql_type: ql::Type::AtType(&node.dbscheme_name),
ql_type_is_ref: false,
}],
keysets: None,
};
ast_node_members.insert(&node.dbscheme_name);
// If the type also has fields or children, then we create either
// auxiliary tables or columns in the defining table for them.
for field in fields {
match &field.storage {
node_types::Storage::Column { name: column_name } => {
let (field_column, field_type_entry) = add_field_for_column_storage(
&node.dbscheme_name,
field,
column_name,
nodes,
);
if let Some(field_type_entry) = field_type_entry {
entries.push(field_type_entry);
}
main_table.columns.push(field_column);
}
node_types::Storage::Table {
name,
has_index,
column_name,
} => {
let (field_table, field_type_entry) = add_field_for_table_storage(
field,
name,
column_name,
*has_index,
nodes,
);
if let Some(field_type_entry) = field_type_entry {
entries.push(field_type_entry);
}
entries.push(dbscheme::Entry::Table(field_table));
}
}
}
if fields.is_empty() {
// There were no fields and no children, so it's a leaf node in
// the TS grammar. Add a column for the node text.
main_table.columns.push(dbscheme::Column {
unique: false,
db_type: dbscheme::DbColumnType::String,
name: "text",
ql_type: ql::Type::String,
ql_type_is_ref: true,
});
}
// Finally, the type's defining table also includes the location.
main_table.columns.push(dbscheme::Column {
unique: false,
db_type: dbscheme::DbColumnType::Int,
name: "loc",
ql_type: ql::Type::AtType("location"),
ql_type_is_ref: true,
});
entries.push(dbscheme::Entry::Table(main_table));
}
node_types::EntryKind::Token { .. } => {}
}
}
(entries, ast_node_members, token_kinds)
}
/// Creates a dbscheme table entry representing the parent relation for AST nodes.
///
/// # Arguments
/// - `name` - the name of both the table to create and the node parent type.
/// - `ast_node_name` - the name of the node child type.
fn create_ast_node_parent_table<'a>(name: &'a str, ast_node_name: &'a str) -> dbscheme::Table<'a> {
dbscheme::Table {
name,
columns: vec![
dbscheme::Column {
db_type: dbscheme::DbColumnType::Int,
name: "child",
unique: false,
ql_type: ql::Type::AtType(ast_node_name),
ql_type_is_ref: true,
},
dbscheme::Column {
db_type: dbscheme::DbColumnType::Int,
name: "parent",
unique: false,
ql_type: ql::Type::AtType(name),
ql_type_is_ref: true,
},
dbscheme::Column {
unique: false,
db_type: dbscheme::DbColumnType::Int,
name: "parent_index",
ql_type: ql::Type::Int,
ql_type_is_ref: true,
},
],
keysets: Some(vec!["parent", "parent_index"]),
}
}
fn create_tokeninfo<'a>(name: &'a str, type_name: &'a str) -> dbscheme::Table<'a> {
dbscheme::Table {
name,
keysets: None,
columns: vec![
dbscheme::Column {
db_type: dbscheme::DbColumnType::Int,
name: "id",
unique: true,
ql_type: ql::Type::AtType(type_name),
ql_type_is_ref: false,
},
dbscheme::Column {
unique: false,
db_type: dbscheme::DbColumnType::Int,
name: "kind",
ql_type: ql::Type::Int,
ql_type_is_ref: true,
},
dbscheme::Column {
unique: false,
db_type: dbscheme::DbColumnType::String,
name: "value",
ql_type: ql::Type::String,
ql_type_is_ref: true,
},
dbscheme::Column {
unique: false,
db_type: dbscheme::DbColumnType::Int,
name: "loc",
ql_type: ql::Type::AtType("location"),
ql_type_is_ref: true,
},
],
}
}
fn create_token_case<'a>(name: &'a str, token_kinds: Map<&'a str, usize>) -> dbscheme::Case<'a> {
let branches: Vec<(usize, &str)> = token_kinds
.iter()
.map(|(&name, kind_id)| (*kind_id, name))
.collect();
dbscheme::Case {
name: name,
column: "kind",
branches: branches,
}
}
fn create_location_union<'a>() -> dbscheme::Entry<'a> {
dbscheme::Entry::Union(dbscheme::Union {
name: "location",
members: vec!["location_default"].into_iter().collect(),
})
}
fn create_files_table<'a>() -> dbscheme::Entry<'a> {
dbscheme::Entry::Table(dbscheme::Table {
name: "files",
keysets: None,
columns: vec![
dbscheme::Column {
unique: true,
db_type: dbscheme::DbColumnType::Int,
name: "id",
ql_type: ql::Type::AtType("file"),
ql_type_is_ref: false,
},
dbscheme::Column {
db_type: dbscheme::DbColumnType::String,
name: "name",
unique: false,
ql_type: ql::Type::String,
ql_type_is_ref: true,
},
],
})
}
fn create_folders_table<'a>() -> dbscheme::Entry<'a> {
dbscheme::Entry::Table(dbscheme::Table {
name: "folders",
keysets: None,
columns: vec![
dbscheme::Column {
unique: true,
db_type: dbscheme::DbColumnType::Int,
name: "id",
ql_type: ql::Type::AtType("folder"),
ql_type_is_ref: false,
},
dbscheme::Column {
db_type: dbscheme::DbColumnType::String,
name: "name",
unique: false,
ql_type: ql::Type::String,
ql_type_is_ref: true,
},
],
})
}
fn create_locations_default_table<'a>() -> dbscheme::Entry<'a> {
dbscheme::Entry::Table(dbscheme::Table {
name: "locations_default",
keysets: None,
columns: vec![
dbscheme::Column {
unique: true,
db_type: dbscheme::DbColumnType::Int,
name: "id",
ql_type: ql::Type::AtType("location_default"),
ql_type_is_ref: false,
},
dbscheme::Column {
unique: false,
db_type: dbscheme::DbColumnType::Int,
name: "file",
ql_type: ql::Type::AtType("file"),
ql_type_is_ref: true,
},
dbscheme::Column {
unique: false,
db_type: dbscheme::DbColumnType::Int,
name: "start_line",
ql_type: ql::Type::Int,
ql_type_is_ref: true,
},
dbscheme::Column {
unique: false,
db_type: dbscheme::DbColumnType::Int,
name: "start_column",
ql_type: ql::Type::Int,
ql_type_is_ref: true,
},
dbscheme::Column {
unique: false,
db_type: dbscheme::DbColumnType::Int,
name: "end_line",
ql_type: ql::Type::Int,
ql_type_is_ref: true,
},
dbscheme::Column {
unique: false,
db_type: dbscheme::DbColumnType::Int,
name: "end_column",
ql_type: ql::Type::Int,
ql_type_is_ref: true,
},
],
})
}
fn create_container_union<'a>() -> dbscheme::Entry<'a> {
dbscheme::Entry::Union(dbscheme::Union {
name: "container",
members: vec!["folder", "file"].into_iter().collect(),
})
}
fn create_containerparent_table<'a>() -> dbscheme::Entry<'a> {
dbscheme::Entry::Table(dbscheme::Table {
name: "containerparent",
columns: vec![
dbscheme::Column {
unique: false,
db_type: dbscheme::DbColumnType::Int,
name: "parent",
ql_type: ql::Type::AtType("container"),
ql_type_is_ref: true,
},
dbscheme::Column {
unique: true,
db_type: dbscheme::DbColumnType::Int,
name: "child",
ql_type: ql::Type::AtType("container"),
ql_type_is_ref: true,
},
],
keysets: None,
})
}
fn create_source_location_prefix_table<'a>() -> dbscheme::Entry<'a> {
dbscheme::Entry::Table(dbscheme::Table {
name: "sourceLocationPrefix",
keysets: None,
columns: vec![dbscheme::Column {
unique: false,
db_type: dbscheme::DbColumnType::String,
name: "prefix",
ql_type: ql::Type::String,
ql_type_is_ref: true,
}],
})
}
fn create_diagnostics<'a>() -> (dbscheme::Case<'a>, dbscheme::Table<'a>) {
let table = dbscheme::Table {
name: "diagnostics",
keysets: None,
columns: vec![
dbscheme::Column {
unique: true,
db_type: dbscheme::DbColumnType::Int,
name: "id",
ql_type: ql::Type::AtType("diagnostic"),
ql_type_is_ref: false,
},
dbscheme::Column {
unique: false,
db_type: dbscheme::DbColumnType::Int,
name: "severity",
ql_type: ql::Type::Int,
ql_type_is_ref: true,
},
dbscheme::Column {
unique: false,
db_type: dbscheme::DbColumnType::String,
name: "error_tag",
ql_type: ql::Type::String,
ql_type_is_ref: true,
},
dbscheme::Column {
unique: false,
db_type: dbscheme::DbColumnType::String,
name: "error_message",
ql_type: ql::Type::String,
ql_type_is_ref: true,
},
dbscheme::Column {
unique: false,
db_type: dbscheme::DbColumnType::String,
name: "full_error_message",
ql_type: ql::Type::String,
ql_type_is_ref: true,
},
dbscheme::Column {
unique: false,
db_type: dbscheme::DbColumnType::Int,
name: "location",
ql_type: ql::Type::AtType("location_default"),
ql_type_is_ref: true,
},
],
};
let severities: Vec<(usize, &str)> = vec![
(10, "diagnostic_debug"),
(20, "diagnostic_info"),
(30, "diagnostic_warning"),
(40, "diagnostic_error"),
];
let case = dbscheme::Case {
name: "diagnostic",
column: "severity",
branches: severities,
};
(case, table)
}
fn main() -> std::io::Result<()> {
tracing_subscriber::fmt()
.with_target(false)
.without_time()
.with_level(true)
.with_env_filter(tracing_subscriber::EnvFilter::from_default_env())
.init();
let matches = clap::App::new("Ruby dbscheme generator")
.version("1.0")
.author("GitHub")
.about("CodeQL Ruby dbscheme generator")
.args_from_usage(
"--dbscheme=<FILE> 'Path of the generated dbscheme file'
--library=<FILE> 'Path of the generated QLL file'",
)
.get_matches();
let dbscheme_path = matches.value_of("dbscheme").expect("missing --dbscheme");
let dbscheme_path = PathBuf::from(dbscheme_path);
let ql_library_path = matches.value_of("library").expect("missing --library");
let ql_library_path = PathBuf::from(ql_library_path);
let languages = vec![
Language {
name: "Ruby".to_owned(),
node_types: tree_sitter_ruby::NODE_TYPES,
},
Language {
name: "Erb".to_owned(),
node_types: tree_sitter_embedded_template::NODE_TYPES,
},
];
let mut dbscheme_writer = LineWriter::new(File::create(dbscheme_path)?);
write!(
dbscheme_writer,
"// CodeQL database schema for {}\n\
// Automatically generated from the tree-sitter grammar; do not edit\n\n",
languages[0].name
)?;
let (diagnostics_case, diagnostics_table) = create_diagnostics();
dbscheme::write(
&mut dbscheme_writer,
&[
create_location_union(),
create_locations_default_table(),
create_files_table(),
create_folders_table(),
create_container_union(),
create_containerparent_table(),
create_source_location_prefix_table(),
dbscheme::Entry::Table(diagnostics_table),
dbscheme::Entry::Case(diagnostics_case),
],
)?;
let mut ql_writer = LineWriter::new(File::create(ql_library_path)?);
write!(
ql_writer,
"/*\n\
* CodeQL library for {}
* Automatically generated from the tree-sitter grammar; do not edit\n\
*/\n\n",
languages[0].name
)?;
ql::write(
&mut ql_writer,
&[
ql::TopLevel::Import("codeql.files.FileSystem"),
ql::TopLevel::Import("codeql.Locations"),
],
)?;
for language in languages {
let prefix = node_types::to_snake_case(&language.name);
let ast_node_name = format!("{}_ast_node", &prefix);
let ast_node_parent_name = format!("{}_ast_node_parent", &prefix);
let token_name = format!("{}_token", &prefix);
let tokeninfo_name = format!("{}_tokeninfo", &prefix);
let reserved_word_name = format!("{}_reserved_word", &prefix);
let nodes = node_types::read_node_types_str(&prefix, &language.node_types)?;
let (dbscheme_entries, mut ast_node_members, token_kinds) = convert_nodes(&nodes);
ast_node_members.insert(&token_name);
dbscheme::write(&mut dbscheme_writer, &dbscheme_entries)?;
let token_case = create_token_case(&token_name, token_kinds);
dbscheme::write(
&mut dbscheme_writer,
&[
dbscheme::Entry::Table(create_tokeninfo(&tokeninfo_name, &token_name)),
dbscheme::Entry::Case(token_case),
dbscheme::Entry::Union(dbscheme::Union {
name: &ast_node_name,
members: ast_node_members,
}),
dbscheme::Entry::Union(dbscheme::Union {
name: &ast_node_parent_name,
members: [&ast_node_name, "file"].iter().cloned().collect(),
}),
dbscheme::Entry::Table(create_ast_node_parent_table(
&ast_node_parent_name,
&ast_node_name,
)),
],
)?;
let mut body = vec![
ql::TopLevel::Class(ql_gen::create_ast_node_class(
&ast_node_name,
&ast_node_parent_name,
)),
ql::TopLevel::Class(ql_gen::create_token_class(&token_name, &tokeninfo_name)),
ql::TopLevel::Class(ql_gen::create_reserved_word_class(&reserved_word_name)),
];
body.append(&mut ql_gen::convert_nodes(&nodes));
ql::write(
&mut ql_writer,
&[ql::TopLevel::Module(ql::Module {
qldoc: None,
name: &language.name,
body,
})],
)?;
}
Ok(())
}

275
ruby/generator/src/ql.rs Normal file
View File

@@ -0,0 +1,275 @@
use std::collections::BTreeSet;
use std::fmt;
#[derive(Clone, Eq, PartialEq, Hash)]
pub enum TopLevel<'a> {
Class(Class<'a>),
Import(&'a str),
Module(Module<'a>),
}
impl<'a> fmt::Display for TopLevel<'a> {
fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
match self {
TopLevel::Import(x) => write!(f, "private import {}", x),
TopLevel::Class(cls) => write!(f, "{}", cls),
TopLevel::Module(m) => write!(f, "{}", m),
}
}
}
#[derive(Clone, Eq, PartialEq, Hash)]
pub struct Class<'a> {
pub qldoc: Option<String>,
pub name: &'a str,
pub is_abstract: bool,
pub supertypes: BTreeSet<Type<'a>>,
pub characteristic_predicate: Option<Expression<'a>>,
pub predicates: Vec<Predicate<'a>>,
}
impl<'a> fmt::Display for Class<'a> {
fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
if let Some(qldoc) = &self.qldoc {
write!(f, "/** {} */", qldoc)?;
}
if self.is_abstract {
write!(f, "abstract ")?;
}
write!(f, "class {} extends ", &self.name)?;
for (index, supertype) in self.supertypes.iter().enumerate() {
if index > 0 {
write!(f, ", ")?;
}
write!(f, "{}", supertype)?;
}
write!(f, " {{ \n")?;
if let Some(charpred) = &self.characteristic_predicate {
write!(
f,
" {}\n",
Predicate {
qldoc: None,
name: self.name.clone(),
overridden: false,
return_type: None,
formal_parameters: vec![],
body: charpred.clone(),
}
)?;
}
for predicate in &self.predicates {
write!(f, " {}\n", predicate)?;
}
write!(f, "}}")?;
Ok(())
}
}
#[derive(Clone, Eq, PartialEq, Hash)]
pub struct Module<'a> {
pub qldoc: Option<String>,
pub name: &'a str,
pub body: Vec<TopLevel<'a>>,
}
impl<'a> fmt::Display for Module<'a> {
fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
if let Some(qldoc) = &self.qldoc {
write!(f, "/** {} */", qldoc)?;
}
write!(f, "module {} {{ \n", self.name)?;
for decl in &self.body {
write!(f, " {}\n", decl)?;
}
write!(f, "}}")?;
Ok(())
}
}
// The QL type of a column.
#[derive(Clone, Eq, PartialEq, Hash, Ord, PartialOrd)]
pub enum Type<'a> {
/// Primitive `int` type.
Int,
/// Primitive `string` type.
String,
/// A database type that will need to be referred to with an `@` prefix.
AtType(&'a str),
/// A user-defined type.
Normal(&'a str),
}
impl<'a> fmt::Display for Type<'a> {
fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
match self {
Type::Int => write!(f, "int"),
Type::String => write!(f, "string"),
Type::Normal(name) => write!(f, "{}", name),
Type::AtType(name) => write!(f, "@{}", name),
}
}
}
#[derive(Clone, Eq, PartialEq, Hash)]
pub enum Expression<'a> {
Var(&'a str),
String(&'a str),
Integer(usize),
Pred(&'a str, Vec<Expression<'a>>),
And(Vec<Expression<'a>>),
Or(Vec<Expression<'a>>),
Equals(Box<Expression<'a>>, Box<Expression<'a>>),
Dot(Box<Expression<'a>>, &'a str, Vec<Expression<'a>>),
Aggregate {
name: &'a str,
vars: Vec<FormalParameter<'a>>,
range: Option<Box<Expression<'a>>>,
expr: Box<Expression<'a>>,
second_expr: Option<Box<Expression<'a>>>,
},
}
impl<'a> fmt::Display for Expression<'a> {
fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
match self {
Expression::Var(x) => write!(f, "{}", x),
Expression::String(s) => write!(f, "\"{}\"", s),
Expression::Integer(n) => write!(f, "{}", n),
Expression::Pred(n, args) => {
write!(f, "{}(", n)?;
for (index, arg) in args.iter().enumerate() {
if index > 0 {
write!(f, ", ")?;
}
write!(f, "{}", arg)?;
}
write!(f, ")")
}
Expression::And(conjuncts) => {
if conjuncts.is_empty() {
write!(f, "any()")
} else {
for (index, conjunct) in conjuncts.iter().enumerate() {
if index > 0 {
write!(f, " and ")?;
}
write!(f, "({})", conjunct)?;
}
Ok(())
}
}
Expression::Or(disjuncts) => {
if disjuncts.is_empty() {
write!(f, "none()")
} else {
for (index, disjunct) in disjuncts.iter().enumerate() {
if index > 0 {
write!(f, " or ")?;
}
write!(f, "({})", disjunct)?;
}
Ok(())
}
}
Expression::Equals(a, b) => write!(f, "{} = {}", a, b),
Expression::Dot(x, member_pred, args) => {
write!(f, "{}.{}(", x, member_pred)?;
for (index, arg) in args.iter().enumerate() {
if index > 0 {
write!(f, ", ")?;
}
write!(f, "{}", arg)?;
}
write!(f, ")")
}
Expression::Aggregate {
name,
vars,
range,
expr,
second_expr,
} => {
write!(f, "{}(", name)?;
if vars.len() > 0 {
for (index, var) in vars.iter().enumerate() {
if index > 0 {
write!(f, ", ")?;
}
write!(f, "{}", var)?;
}
write!(f, " | ")?;
}
if let Some(range) = range {
write!(f, "{} | ", range)?;
}
write!(f, "{}", expr)?;
if let Some(second_expr) = second_expr {
write!(f, ", {}", second_expr)?;
}
write!(f, ")")
}
}
}
}
#[derive(Clone, Eq, PartialEq, Hash)]
pub struct Predicate<'a> {
pub qldoc: Option<String>,
pub name: &'a str,
pub overridden: bool,
pub return_type: Option<Type<'a>>,
pub formal_parameters: Vec<FormalParameter<'a>>,
pub body: Expression<'a>,
}
impl<'a> fmt::Display for Predicate<'a> {
fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
if let Some(qldoc) = &self.qldoc {
write!(f, "/** {} */", qldoc)?;
}
if self.overridden {
write!(f, "override ")?;
}
match &self.return_type {
None => write!(f, "predicate ")?,
Some(return_type) => write!(f, "{} ", return_type)?,
}
write!(f, "{}(", self.name)?;
for (index, param) in self.formal_parameters.iter().enumerate() {
if index > 0 {
write!(f, ", ")?;
}
write!(f, "{}", param)?;
}
write!(f, ") {{ {} }}", self.body)?;
Ok(())
}
}
#[derive(Clone, Eq, PartialEq, Hash)]
pub struct FormalParameter<'a> {
pub name: &'a str,
pub param_type: Type<'a>,
}
impl<'a> fmt::Display for FormalParameter<'a> {
fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
write!(f, "{} {}", self.param_type, self.name)
}
}
/// Generates a QL library by writing the given `elements` to the `file`.
pub fn write<'a>(file: &mut dyn std::io::Write, elements: &'a [TopLevel]) -> std::io::Result<()> {
for element in elements {
write!(file, "{}\n\n", &element)?;
}
Ok(())
}

View File

@@ -0,0 +1,602 @@
use crate::ql;
use std::collections::BTreeSet;
/// Creates the hard-coded `AstNode` class that acts as a supertype of all
/// classes we generate.
pub fn create_ast_node_class<'a>(ast_node: &'a str, ast_node_parent: &'a str) -> ql::Class<'a> {
// Default implementation of `toString` calls `this.getAPrimaryQlClass()`
let to_string = ql::Predicate {
qldoc: Some(String::from(
"Gets a string representation of this element.",
)),
name: "toString",
overridden: false,
return_type: Some(ql::Type::String),
formal_parameters: vec![],
body: ql::Expression::Equals(
Box::new(ql::Expression::Var("result")),
Box::new(ql::Expression::Dot(
Box::new(ql::Expression::Var("this")),
"getAPrimaryQlClass",
vec![],
)),
),
};
let get_location = create_none_predicate(
Some(String::from("Gets the location of this element.")),
"getLocation",
false,
Some(ql::Type::Normal("Location")),
);
let get_a_field_or_child = create_none_predicate(
Some(String::from("Gets a field or child node of this node.")),
"getAFieldOrChild",
false,
Some(ql::Type::Normal("AstNode")),
);
let get_parent = ql::Predicate {
qldoc: Some(String::from("Gets the parent of this element.")),
name: "getParent",
overridden: false,
return_type: Some(ql::Type::Normal("AstNode")),
formal_parameters: vec![],
body: ql::Expression::Pred(
ast_node_parent,
vec![
ql::Expression::Var("this"),
ql::Expression::Var("result"),
ql::Expression::Var("_"),
],
),
};
let get_parent_index = ql::Predicate {
qldoc: Some(String::from(
"Gets the index of this node among the children of its parent.",
)),
name: "getParentIndex",
overridden: false,
return_type: Some(ql::Type::Int),
formal_parameters: vec![],
body: ql::Expression::Pred(
ast_node_parent,
vec![
ql::Expression::Var("this"),
ql::Expression::Var("_"),
ql::Expression::Var("result"),
],
),
};
let get_a_primary_ql_class = ql::Predicate {
qldoc: Some(String::from(
"Gets the name of the primary QL class for this element.",
)),
name: "getAPrimaryQlClass",
overridden: false,
return_type: Some(ql::Type::String),
formal_parameters: vec![],
body: ql::Expression::Equals(
Box::new(ql::Expression::Var("result")),
Box::new(ql::Expression::String("???")),
),
};
let get_primary_ql_classes = ql::Predicate {
qldoc: Some(
"Gets a comma-separated list of the names of the primary CodeQL \
classes to which this element belongs."
.to_owned(),
),
name: "getPrimaryQlClasses",
overridden: false,
return_type: Some(ql::Type::String),
formal_parameters: vec![],
body: ql::Expression::Equals(
Box::new(ql::Expression::Var("result")),
Box::new(ql::Expression::Aggregate {
name: "concat",
vars: vec![],
range: None,
expr: Box::new(ql::Expression::Pred("getAPrimaryQlClass", vec![])),
second_expr: Some(Box::new(ql::Expression::String(","))),
}),
),
};
ql::Class {
qldoc: Some(String::from("The base class for all AST nodes")),
name: "AstNode",
is_abstract: false,
supertypes: vec![ql::Type::AtType(ast_node)].into_iter().collect(),
characteristic_predicate: None,
predicates: vec![
to_string,
get_location,
get_parent,
get_parent_index,
get_a_field_or_child,
get_a_primary_ql_class,
get_primary_ql_classes,
],
}
}
pub fn create_token_class<'a>(token_type: &'a str, tokeninfo: &'a str) -> ql::Class<'a> {
let tokeninfo_arity = 4;
let get_value = ql::Predicate {
qldoc: Some(String::from("Gets the value of this token.")),
name: "getValue",
overridden: false,
return_type: Some(ql::Type::String),
formal_parameters: vec![],
body: create_get_field_expr_for_column_storage("result", tokeninfo, 1, tokeninfo_arity),
};
let get_location = ql::Predicate {
qldoc: Some(String::from("Gets the location of this token.")),
name: "getLocation",
overridden: true,
return_type: Some(ql::Type::Normal("Location")),
formal_parameters: vec![],
body: create_get_field_expr_for_column_storage("result", tokeninfo, 2, tokeninfo_arity),
};
let to_string = ql::Predicate {
qldoc: Some(String::from(
"Gets a string representation of this element.",
)),
name: "toString",
overridden: true,
return_type: Some(ql::Type::String),
formal_parameters: vec![],
body: ql::Expression::Equals(
Box::new(ql::Expression::Var("result")),
Box::new(ql::Expression::Pred("getValue", vec![])),
),
};
ql::Class {
qldoc: Some(String::from("A token.")),
name: "Token",
is_abstract: false,
supertypes: vec![ql::Type::AtType(token_type), ql::Type::Normal("AstNode")]
.into_iter()
.collect(),
characteristic_predicate: None,
predicates: vec![
get_value,
get_location,
to_string,
create_get_a_primary_ql_class("Token"),
],
}
}
// Creates the `ReservedWord` class.
pub fn create_reserved_word_class<'a>(db_name: &'a str) -> ql::Class<'a> {
let class_name = "ReservedWord";
let get_a_primary_ql_class = create_get_a_primary_ql_class(&class_name);
ql::Class {
qldoc: Some(String::from("A reserved word.")),
name: class_name,
is_abstract: false,
supertypes: vec![ql::Type::AtType(db_name), ql::Type::Normal("Token")]
.into_iter()
.collect(),
characteristic_predicate: None,
predicates: vec![get_a_primary_ql_class],
}
}
/// Creates a predicate whose body is `none()`.
fn create_none_predicate<'a>(
qldoc: Option<String>,
name: &'a str,
overridden: bool,
return_type: Option<ql::Type<'a>>,
) -> ql::Predicate<'a> {
ql::Predicate {
qldoc: qldoc,
name: name,
overridden,
return_type,
formal_parameters: Vec::new(),
body: ql::Expression::Pred("none", vec![]),
}
}
/// Creates an overridden `getAPrimaryQlClass` predicate that returns the given
/// name.
fn create_get_a_primary_ql_class<'a>(class_name: &'a str) -> ql::Predicate<'a> {
ql::Predicate {
qldoc: Some(String::from(
"Gets the name of the primary QL class for this element.",
)),
name: "getAPrimaryQlClass",
overridden: true,
return_type: Some(ql::Type::String),
formal_parameters: vec![],
body: ql::Expression::Equals(
Box::new(ql::Expression::Var("result")),
Box::new(ql::Expression::String(class_name)),
),
}
}
/// Creates the `getLocation` predicate.
///
/// # Arguments
///
/// `def_table` - the name of the table that defines the entity and its location.
/// `arity` - the total number of columns in the table
fn create_get_location_predicate<'a>(def_table: &'a str, arity: usize) -> ql::Predicate<'a> {
ql::Predicate {
qldoc: Some(String::from("Gets the location of this element.")),
name: "getLocation",
overridden: true,
return_type: Some(ql::Type::Normal("Location")),
formal_parameters: vec![],
// body of the form: foo_bar_def(_, _, ..., result)
body: ql::Expression::Pred(
def_table,
[
vec![ql::Expression::Var("this")],
vec![ql::Expression::Var("_"); arity - 2],
vec![ql::Expression::Var("result")],
]
.concat(),
),
}
}
/// Creates the `getText` predicate for a leaf node.
///
/// # Arguments
///
/// `def_table` - the name of the table that defines the entity and its text.
fn create_get_text_predicate<'a>(def_table: &'a str) -> ql::Predicate<'a> {
ql::Predicate {
qldoc: Some(String::from("Gets the text content of this element.")),
name: "getText",
overridden: false,
return_type: Some(ql::Type::String),
formal_parameters: vec![],
body: ql::Expression::Pred(
def_table,
vec![
ql::Expression::Var("this"),
ql::Expression::Var("result"),
ql::Expression::Var("_"),
],
),
}
}
/// Returns an expression to get a field that's defined as a column in the parent's table.
///
/// # Arguments
///
/// * `result_var_name` - the name of the variable to which the resulting value should be bound
/// * `table_name` - the name of parent's defining table
/// * `column_index` - the index in that table that defines the field
/// * `arity` - the total number of columns in the table
fn create_get_field_expr_for_column_storage<'a>(
result_var_name: &'a str,
table_name: &'a str,
column_index: usize,
arity: usize,
) -> ql::Expression<'a> {
let num_underscores_before = column_index;
let num_underscores_after = arity - 2 - num_underscores_before;
ql::Expression::Pred(
table_name,
[
vec![ql::Expression::Var("this")],
vec![ql::Expression::Var("_"); num_underscores_before],
vec![ql::Expression::Var(result_var_name)],
vec![ql::Expression::Var("_"); num_underscores_after],
]
.concat(),
)
}
/// Returns an expression to get the field with the given index from its
/// auxiliary table. The index name can be "_" so the expression will hold for
/// all indices.
fn create_get_field_expr_for_table_storage<'a>(
result_var_name: &'a str,
table_name: &'a str,
index_var_name: Option<&'a str>,
) -> ql::Expression<'a> {
ql::Expression::Pred(
table_name,
match index_var_name {
Some(index_var_name) => vec![
ql::Expression::Var("this"),
ql::Expression::Var(index_var_name),
ql::Expression::Var(result_var_name),
],
None => vec![ql::Expression::Var("this"), ql::Expression::Var("result")],
},
)
}
/// Creates a pair consisting of a predicate to get the given field, and an
/// optional expression that will get the same field. When the field can occur
/// multiple times, the predicate will take an index argument, while the
/// expression will use the "don't care" expression to hold for all occurrences.
///
/// # Arguments
///
/// `main_table_name` - the name of the defining table for the parent node
/// `main_table_arity` - the number of columns in the main table
/// `main_table_column_index` - a mutable reference to a column index indicating
/// where the field is in the main table. If this is used (i.e. the field has
/// column storage), then the index is incremented.
/// `parent_name` - the name of the parent node
/// `field` - the field whose getters we are creating
/// `field_type` - the db name of the field's type (possibly being a union we created)
fn create_field_getters<'a>(
main_table_name: &'a str,
main_table_arity: usize,
main_table_column_index: &mut usize,
field: &'a node_types::Field,
nodes: &'a node_types::NodeTypeMap,
) -> (ql::Predicate<'a>, Option<ql::Expression<'a>>) {
let return_type = match &field.type_info {
node_types::FieldTypeInfo::Single(t) => {
Some(ql::Type::Normal(&nodes.get(&t).unwrap().ql_class_name))
}
node_types::FieldTypeInfo::Multiple {
types: _,
dbscheme_union: _,
ql_class,
} => Some(ql::Type::Normal(&ql_class)),
node_types::FieldTypeInfo::ReservedWordInt(_) => Some(ql::Type::String),
};
let formal_parameters = match &field.storage {
node_types::Storage::Column { .. } => vec![],
node_types::Storage::Table { has_index, .. } => {
if *has_index {
vec![ql::FormalParameter {
name: "i",
param_type: ql::Type::Int,
}]
} else {
vec![]
}
}
};
// For the expression to get a value, what variable name should the result
// be bound to?
let get_value_result_var_name = match &field.type_info {
node_types::FieldTypeInfo::ReservedWordInt(_) => "value",
node_types::FieldTypeInfo::Single(_) => "result",
node_types::FieldTypeInfo::Multiple { .. } => "result",
};
// Two expressions for getting the value. One that's suitable use in the
// getter predicate (where there may be a specific index), and another for
// use in `getAFieldOrChild` (where we use a "don't care" expression to
// match any index).
let (get_value, get_value_any_index) = match &field.storage {
node_types::Storage::Column { name: _ } => {
let column_index = *main_table_column_index;
*main_table_column_index += 1;
(
create_get_field_expr_for_column_storage(
get_value_result_var_name,
&main_table_name,
column_index,
main_table_arity,
),
create_get_field_expr_for_column_storage(
get_value_result_var_name,
&main_table_name,
column_index,
main_table_arity,
),
)
}
node_types::Storage::Table {
name: field_table_name,
has_index,
column_name: _,
} => (
create_get_field_expr_for_table_storage(
get_value_result_var_name,
&field_table_name,
if *has_index { Some("i") } else { None },
),
create_get_field_expr_for_table_storage(
get_value_result_var_name,
&field_table_name,
if *has_index { Some("_") } else { None },
),
),
};
let (body, optional_expr) = match &field.type_info {
node_types::FieldTypeInfo::ReservedWordInt(int_mapping) => {
// Create an expression that binds the corresponding string to `result` for each `value`, e.g.:
// result = "foo" and value = 0 or
// result = "bar" and value = 1 or
// result = "baz" and value = 2
let disjuncts = int_mapping
.iter()
.map(|(token_str, (value, _))| {
ql::Expression::And(vec![
ql::Expression::Equals(
Box::new(ql::Expression::Var("result")),
Box::new(ql::Expression::String(token_str)),
),
ql::Expression::Equals(
Box::new(ql::Expression::Var("value")),
Box::new(ql::Expression::Integer(*value)),
),
])
})
.collect();
(
ql::Expression::Aggregate {
name: "exists",
vars: vec![ql::FormalParameter {
name: "value",
param_type: ql::Type::Int,
}],
range: Some(Box::new(get_value)),
expr: Box::new(ql::Expression::Or(disjuncts)),
second_expr: None,
},
// Since the getter returns a string and not an AstNode, it won't be part of getAFieldOrChild:
None,
)
}
node_types::FieldTypeInfo::Single(_) | node_types::FieldTypeInfo::Multiple { .. } => {
(get_value, Some(get_value_any_index))
}
};
let qldoc = match &field.name {
Some(name) => format!("Gets the node corresponding to the field `{}`.", name),
None => {
if formal_parameters.len() == 0 {
"Gets the child of this node.".to_owned()
} else {
"Gets the `i`th child of this node.".to_owned()
}
}
};
(
ql::Predicate {
qldoc: Some(qldoc),
name: &field.getter_name,
overridden: false,
return_type,
formal_parameters,
body,
},
optional_expr,
)
}
/// Converts the given node types into CodeQL classes wrapping the dbscheme.
pub fn convert_nodes<'a>(nodes: &'a node_types::NodeTypeMap) -> Vec<ql::TopLevel<'a>> {
let mut classes: Vec<ql::TopLevel> = Vec::new();
let mut token_kinds = BTreeSet::new();
for (type_name, node) in nodes {
if let node_types::EntryKind::Token { .. } = &node.kind {
if type_name.named {
token_kinds.insert(&type_name.kind);
}
}
}
for (type_name, node) in nodes {
match &node.kind {
node_types::EntryKind::Token { kind_id: _ } => {
if type_name.named {
let get_a_primary_ql_class = create_get_a_primary_ql_class(&node.ql_class_name);
let mut supertypes: BTreeSet<ql::Type> = BTreeSet::new();
supertypes.insert(ql::Type::AtType(&node.dbscheme_name));
supertypes.insert(ql::Type::Normal("Token"));
classes.push(ql::TopLevel::Class(ql::Class {
qldoc: Some(format!("A class representing `{}` tokens.", type_name.kind)),
name: &node.ql_class_name,
is_abstract: false,
supertypes,
characteristic_predicate: None,
predicates: vec![get_a_primary_ql_class],
}));
}
}
node_types::EntryKind::Union { members: _ } => {
// It's a tree-sitter supertype node, so we're wrapping a dbscheme
// union type.
classes.push(ql::TopLevel::Class(ql::Class {
qldoc: None,
name: &node.ql_class_name,
is_abstract: false,
supertypes: vec![
ql::Type::AtType(&node.dbscheme_name),
ql::Type::Normal("AstNode"),
]
.into_iter()
.collect(),
characteristic_predicate: None,
predicates: vec![],
}));
}
node_types::EntryKind::Table {
name: main_table_name,
fields,
} => {
// Count how many columns there will be in the main table.
// There will be:
// - one for the id
// - one for the location
// - one for each field that's stored as a column
// - if there are no fields, one for the text column.
let main_table_arity = 2 + if fields.is_empty() {
1
} else {
fields
.iter()
.filter(|&f| matches!(f.storage, node_types::Storage::Column { .. }))
.count()
};
let main_class_name = &node.ql_class_name;
let mut main_class = ql::Class {
qldoc: Some(format!("A class representing `{}` nodes.", type_name.kind)),
name: &main_class_name,
is_abstract: false,
supertypes: vec![
ql::Type::AtType(&node.dbscheme_name),
ql::Type::Normal("AstNode"),
]
.into_iter()
.collect(),
characteristic_predicate: None,
predicates: vec![
create_get_a_primary_ql_class(&main_class_name),
create_get_location_predicate(&main_table_name, main_table_arity),
],
};
if fields.is_empty() {
main_class
.predicates
.push(create_get_text_predicate(&main_table_name));
} else {
let mut main_table_column_index: usize = 0;
let mut get_child_exprs: Vec<ql::Expression> = Vec::new();
// Iterate through the fields, creating:
// - classes to wrap union types if fields need them,
// - predicates to access the fields,
// - the QL expressions to access the fields that will be part of getAFieldOrChild.
for field in fields {
let (get_pred, get_child_expr) = create_field_getters(
&main_table_name,
main_table_arity,
&mut main_table_column_index,
field,
nodes,
);
main_class.predicates.push(get_pred);
if let Some(get_child_expr) = get_child_expr {
get_child_exprs.push(get_child_expr)
}
}
main_class.predicates.push(ql::Predicate {
qldoc: Some(String::from("Gets a field or child node of this node.")),
name: "getAFieldOrChild",
overridden: true,
return_type: Some(ql::Type::Normal("AstNode")),
formal_parameters: vec![],
body: ql::Expression::Or(get_child_exprs),
});
}
classes.push(ql::TopLevel::Class(main_class));
}
}
}
classes
}

View File

@@ -0,0 +1,11 @@
[package]
name = "node-types"
version = "0.1.0"
authors = ["GitHub"]
edition = "2018"
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[dependencies]
serde = { version = "1.0", features = ["derive"] }
serde_json = "1.0"

441
ruby/node-types/src/lib.rs Normal file
View File

@@ -0,0 +1,441 @@
use serde::Deserialize;
use std::collections::BTreeMap;
use std::path::Path;
use std::collections::BTreeSet as Set;
use std::fs;
/// A lookup table from TypeName to Entry.
pub type NodeTypeMap = BTreeMap<TypeName, Entry>;
#[derive(Debug)]
pub struct Entry {
pub dbscheme_name: String,
pub ql_class_name: String,
pub kind: EntryKind,
}
#[derive(Debug)]
pub enum EntryKind {
Union { members: Set<TypeName> },
Table { name: String, fields: Vec<Field> },
Token { kind_id: usize },
}
#[derive(Debug, Ord, PartialOrd, Eq, PartialEq)]
pub struct TypeName {
pub kind: String,
pub named: bool,
}
#[derive(Debug)]
pub enum FieldTypeInfo {
/// The field has a single type.
Single(TypeName),
/// The field can take one of several types, so we also provide the name of
/// the database union type that wraps them, and the corresponding QL class
/// name.
Multiple {
types: Set<TypeName>,
dbscheme_union: String,
ql_class: String,
},
/// The field can be one of several tokens, so the db type will be an `int`
/// with a `case @foo.kind` for each possiblity.
ReservedWordInt(BTreeMap<String, (usize, String)>),
}
#[derive(Debug)]
pub struct Field {
pub parent: TypeName,
pub type_info: FieldTypeInfo,
/// The name of the field or None for the anonymous 'children'
/// entry from node_types.json
pub name: Option<String>,
/// The name of the predicate to get this field.
pub getter_name: String,
pub storage: Storage,
}
fn name_for_field_or_child(name: &Option<String>) -> String {
match name {
Some(name) => name.clone(),
None => "child".to_owned(),
}
}
#[derive(Debug)]
pub enum Storage {
/// the field is stored as a column in the parent table
Column { name: String },
/// the field is stored in a link table
Table {
/// the name of the table
name: String,
/// the name of the column for the field in the dbscheme
column_name: String,
/// does it have an associated index column?
has_index: bool,
},
}
pub fn read_node_types(prefix: &str, node_types_path: &Path) -> std::io::Result<NodeTypeMap> {
let file = fs::File::open(node_types_path)?;
let node_types = serde_json::from_reader(file)?;
Ok(convert_nodes(&prefix, &node_types))
}
pub fn read_node_types_str(prefix: &str, node_types_json: &str) -> std::io::Result<NodeTypeMap> {
let node_types = serde_json::from_str(node_types_json)?;
Ok(convert_nodes(&prefix, &node_types))
}
fn convert_type(node_type: &NodeType) -> TypeName {
TypeName {
kind: node_type.kind.to_string(),
named: node_type.named,
}
}
fn convert_types(node_types: &Vec<NodeType>) -> Set<TypeName> {
node_types.iter().map(convert_type).collect()
}
pub fn convert_nodes(prefix: &str, nodes: &Vec<NodeInfo>) -> NodeTypeMap {
let mut entries = NodeTypeMap::new();
let mut token_kinds = Set::new();
// First, find all the token kinds
for node in nodes {
if node.subtypes.is_none() {
if node.fields.as_ref().map_or(0, |x| x.len()) == 0 && node.children.is_none() {
let type_name = TypeName {
kind: node.kind.clone(),
named: node.named,
};
token_kinds.insert(type_name);
}
}
}
for node in nodes {
let flattened_name = &node_type_name(&node.kind, node.named);
let dbscheme_name = escape_name(&flattened_name);
let ql_class_name = dbscheme_name_to_class_name(&dbscheme_name);
let dbscheme_name = format!("{}_{}", prefix, &dbscheme_name);
if let Some(subtypes) = &node.subtypes {
// It's a tree-sitter supertype node, for which we create a union
// type.
entries.insert(
TypeName {
kind: node.kind.clone(),
named: node.named,
},
Entry {
dbscheme_name,
ql_class_name,
kind: EntryKind::Union {
members: convert_types(&subtypes),
},
},
);
} else if node.fields.as_ref().map_or(0, |x| x.len()) == 0 && node.children.is_none() {
// Token kind, handled above.
} else {
// It's a product type, defined by a table.
let type_name = TypeName {
kind: node.kind.clone(),
named: node.named,
};
let table_name = escape_name(&(format!("{}_def", &flattened_name)));
let table_name = format!("{}_{}", prefix, &table_name);
let mut fields = Vec::new();
// If the type also has fields or children, then we create either
// auxiliary tables or columns in the defining table for them.
if let Some(node_fields) = &node.fields {
for (field_name, field_info) in node_fields {
add_field(
&prefix,
&type_name,
Some(field_name.to_string()),
field_info,
&mut fields,
&token_kinds,
);
}
}
if let Some(children) = &node.children {
// Treat children as if they were a field called 'child'.
add_field(
&prefix,
&type_name,
None,
children,
&mut fields,
&token_kinds,
);
}
entries.insert(
type_name,
Entry {
dbscheme_name,
ql_class_name,
kind: EntryKind::Table {
name: table_name,
fields,
},
},
);
}
}
let mut counter = 0;
for type_name in token_kinds {
let entry = if type_name.named {
counter += 1;
let unprefixed_name = node_type_name(&type_name.kind, true);
Entry {
dbscheme_name: escape_name(&format!("{}_token_{}", &prefix, &unprefixed_name)),
ql_class_name: dbscheme_name_to_class_name(&escape_name(&unprefixed_name)),
kind: EntryKind::Token { kind_id: counter },
}
} else {
Entry {
dbscheme_name: format!("{}_reserved_word", &prefix),
ql_class_name: "ReservedWord".to_owned(),
kind: EntryKind::Token { kind_id: 0 },
}
};
entries.insert(type_name, entry);
}
entries
}
fn add_field(
prefix: &str,
parent_type_name: &TypeName,
field_name: Option<String>,
field_info: &FieldInfo,
fields: &mut Vec<Field>,
token_kinds: &Set<TypeName>,
) {
let parent_flattened_name = node_type_name(&parent_type_name.kind, parent_type_name.named);
let column_name = escape_name(&name_for_field_or_child(&field_name));
let storage = if !field_info.multiple && field_info.required {
// This field must appear exactly once, so we add it as
// a column to the main table for the node type.
Storage::Column { name: column_name }
} else {
// Put the field in an auxiliary table.
let has_index = field_info.multiple;
let field_table_name = escape_name(&format!(
"{}_{}_{}",
&prefix,
parent_flattened_name,
&name_for_field_or_child(&field_name)
));
Storage::Table {
has_index,
name: field_table_name,
column_name,
}
};
let converted_types = convert_types(&field_info.types);
let type_info = if field_info
.types
.iter()
.all(|t| !t.named && token_kinds.contains(&convert_type(t)))
{
// All possible types for this field are reserved words. The db
// representation will be an `int` with a `case @foo.field = ...` to
// enumerate the possible values.
let mut counter = 0;
let mut field_token_ints: BTreeMap<String, (usize, String)> = BTreeMap::new();
for t in converted_types {
let dbscheme_variant_name =
escape_name(&format!("{}_{}_{}", &prefix, parent_flattened_name, t.kind));
field_token_ints.insert(t.kind.to_owned(), (counter, dbscheme_variant_name));
counter += 1;
}
FieldTypeInfo::ReservedWordInt(field_token_ints)
} else if field_info.types.len() == 1 {
FieldTypeInfo::Single(converted_types.into_iter().next().unwrap())
} else {
// The dbscheme type for this field will be a union. In QL, it'll just be AstNode.
FieldTypeInfo::Multiple {
types: converted_types,
dbscheme_union: format!(
"{}_{}_{}_type",
&prefix,
&parent_flattened_name,
&name_for_field_or_child(&field_name)
),
ql_class: "AstNode".to_owned(),
}
};
let getter_name = format!(
"get{}",
dbscheme_name_to_class_name(&escape_name(&name_for_field_or_child(&field_name)))
);
fields.push(Field {
parent: TypeName {
kind: parent_type_name.kind.to_string(),
named: parent_type_name.named,
},
type_info,
name: field_name,
getter_name,
storage,
});
}
#[derive(Deserialize)]
pub struct NodeInfo {
#[serde(rename = "type")]
pub kind: String,
pub named: bool,
#[serde(skip_serializing_if = "Option::is_none")]
pub fields: Option<BTreeMap<String, FieldInfo>>,
#[serde(skip_serializing_if = "Option::is_none")]
pub children: Option<FieldInfo>,
#[serde(skip_serializing_if = "Option::is_none")]
pub subtypes: Option<Vec<NodeType>>,
}
#[derive(Deserialize)]
pub struct NodeType {
#[serde(rename = "type")]
pub kind: String,
pub named: bool,
}
#[derive(Deserialize)]
pub struct FieldInfo {
pub multiple: bool,
pub required: bool,
pub types: Vec<NodeType>,
}
/// Given a tree-sitter node type's (kind, named) pair, returns a single string
/// representing the (unescaped) name we'll use to refer to corresponding QL
/// type.
fn node_type_name(kind: &str, named: bool) -> String {
if named {
kind.to_string()
} else {
format!("{}_unnamed", kind)
}
}
const RESERVED_KEYWORDS: [&'static str; 14] = [
"boolean", "case", "date", "float", "int", "key", "of", "order", "ref", "string", "subtype",
"type", "unique", "varchar",
];
/// Returns a string that's a copy of `name` but suitably escaped to be a valid
/// QL identifier.
fn escape_name(name: &str) -> String {
let mut result = String::new();
// If there's a leading underscore, replace it with 'underscore_'.
if let Some(c) = name.chars().next() {
if c == '_' {
result.push_str("underscore");
}
}
for c in name.chars() {
match c {
'{' => result.push_str("lbrace"),
'}' => result.push_str("rbrace"),
'<' => result.push_str("langle"),
'>' => result.push_str("rangle"),
'[' => result.push_str("lbracket"),
']' => result.push_str("rbracket"),
'(' => result.push_str("lparen"),
')' => result.push_str("rparen"),
'|' => result.push_str("pipe"),
'=' => result.push_str("equal"),
'~' => result.push_str("tilde"),
'?' => result.push_str("question"),
'`' => result.push_str("backtick"),
'^' => result.push_str("caret"),
'!' => result.push_str("bang"),
'#' => result.push_str("hash"),
'%' => result.push_str("percent"),
'&' => result.push_str("ampersand"),
'.' => result.push_str("dot"),
',' => result.push_str("comma"),
'/' => result.push_str("slash"),
':' => result.push_str("colon"),
';' => result.push_str("semicolon"),
'"' => result.push_str("dquote"),
'*' => result.push_str("star"),
'+' => result.push_str("plus"),
'-' => result.push_str("minus"),
'@' => result.push_str("at"),
_ if c.is_uppercase() => {
result.push('_');
result.push_str(&c.to_lowercase().to_string())
}
_ => result.push(c),
}
}
for &keyword in &RESERVED_KEYWORDS {
if result == keyword {
result.push_str("__");
break;
}
}
result
}
pub fn to_snake_case(word: &str) -> String {
let mut prev_upper = true;
let mut result = String::new();
for c in word.chars() {
if c.is_uppercase() {
if !prev_upper {
result.push('_')
}
prev_upper = true;
result.push(c.to_ascii_lowercase());
} else {
prev_upper = false;
result.push(c);
}
}
result
}
/// Given a valid dbscheme name (i.e. in snake case), produces the equivalent QL
/// name (i.e. in CamelCase). For example, "foo_bar_baz" becomes "FooBarBaz".
fn dbscheme_name_to_class_name(dbscheme_name: &str) -> String {
fn to_title_case(word: &str) -> String {
let mut first = true;
let mut result = String::new();
for c in word.chars() {
if first {
first = false;
result.push(c.to_ascii_uppercase());
} else {
result.push(c);
}
}
result
}
dbscheme_name
.split('_')
.map(|word| to_title_case(word))
.collect::<Vec<String>>()
.join("")
}
#[test]
fn to_snake_case_test() {
assert_eq!("ruby", to_snake_case("Ruby"));
assert_eq!("erb", to_snake_case("ERB"));
assert_eq!("embedded_template", to_snake_case("EmbeddedTemplate"));
}

View File

@@ -0,0 +1,25 @@
import codeql.ruby.AST
import codeql.ruby.ast.internal.Synthesis
query predicate missingParent(AstNode node, string cls) {
not exists(node.getParent()) and
node.getLocation().getFile().getExtension() != "erb" and
not node instanceof Toplevel and
cls = node.getPrimaryQlClasses()
}
pragma[noinline]
private AstNode parent(AstNode child, int desugarLevel) {
result = child.getParent() and
desugarLevel = desugarLevel(result)
}
query predicate multipleParents(AstNode node, AstNode parent, string cls) {
parent = node.getParent() and
cls = parent.getPrimaryQlClasses() and
exists(AstNode one, AstNode two, int desugarLevel |
one = parent(node, desugarLevel) and
two = parent(node, desugarLevel) and
one != two
)
}

View File

@@ -0,0 +1 @@
import codeql.ruby.controlflow.internal.ControlFlowGraphImplShared::Consistency

View File

@@ -0,0 +1 @@
import codeql.ruby.dataflow.internal.DataFlowImplConsistency::Consistency

View File

@@ -0,0 +1,22 @@
import ruby
import codeql.ruby.dataflow.SSA
import codeql.ruby.controlflow.ControlFlowGraph
query predicate nonUniqueDef(CfgNode read, Ssa::Definition def) {
read = def.getARead() and
exists(Ssa::Definition other | read = other.getARead() and other != def)
}
query predicate readWithoutDef(LocalVariableReadAccess read) {
exists(CfgNode node |
node = read.getAControlFlowNode() and
not node = any(Ssa::Definition def).getARead()
)
}
query predicate deadDef(Ssa::Definition def, LocalVariable v) {
v = def.getSourceVariable() and
not v.isCaptured() and
not exists(def.getARead()) and
not def = any(Ssa::PhiNode phi).getAnInput()
}

View File

@@ -0,0 +1,6 @@
import codeql.ruby.ast.Variable
query predicate ambiguousVariable(VariableAccess access, Variable variable) {
access.getVariable() = variable and
count(access.getVariable()) > 1
}

View File

@@ -0,0 +1,5 @@
name: codeql/ruby-consistency-queries
version: 0.0.1
dependencies:
codeql/ruby-all: 0.0.1

View File

@@ -0,0 +1,37 @@
# Experimental CodeQL queries and libraries
In addition to our standard CodeQL queries and libraries, this repository may also contain queries and libraries of a more experimental nature. Experimental queries and libraries can be improved incrementally and may eventually reach a sufficient maturity to be included in our standard libraries and queries.
Experimental queries and libraries may not be actively maintained as the standard libraries evolve. They may also be changed in backwards-incompatible ways or may be removed entirely in the future without deprecation warnings.
## Requirements
1. **Directory structure**
- Experimental queries and libraries are stored in the `ql/src/experimental` subdirectory, and any corresponding tests in `ql/test/experimental`.
- The structure of an `experimental` subdirectory mirrors the structure of standard queries and libraries (or tests) in the parent directory.
2. **Query metadata**
- The query `@id` must not clash with any other queries in the repository.
- The query must have a `@name` and `@description` to explain its purpose.
- The query must have a `@kind` and `@problem.severity` as required by CodeQL tools.
For details, see the [guide on query metadata](https://github.com/github/codeql/blob/master/docs/query-metadata-style-guide.md).
3. **Formatting**
- The queries and libraries must be [autoformatted](https://help.semmle.com/codeql/codeql-for-vscode/reference/editor.html#autoformatting).
4. **Compilation**
- Compilation of the query and any associated libraries and tests must be resilient to future development of the standard libraries. This means that the functionality cannot use internal APIs, cannot depend on the output of `getAQlClass`, and cannot make use of regexp matching on `toString`.
- The query and any associated libraries and tests must not cause any compiler warnings to be emitted (such as use of deprecated functionality or missing `override` annotations).
5. **Results**
- The query must have at least one true positive result on some revision of a real project.
## Non-requirements
Other criteria typically required for our standard queries and libraries are not required for experimental queries and libraries. In particular, fully disciplined query [metadata](https://github.com/github/codeql/blob/master/docs/query-metadata-style-guide.md), query [help](https://github.com/github/codeql/blob/master/docs/query-help-style-guide.md), tests, a low false positive rate and performance tuning are not required (but nonetheless recommended).

View File

@@ -0,0 +1,4 @@
---
dependencies: {}
compiled: false
lockVersion: 1.0.0

View File

@@ -0,0 +1,4 @@
name: codeql/ruby-examples
version: 0.0.2
dependencies:
codeql/ruby-all: ^0.0.2

View File

@@ -0,0 +1 @@
<queries language="ruby"/>

View File

@@ -0,0 +1,18 @@
/**
* @name If statements with empty then branch
* @description Finds 'if' statements where the 'then' branch is
* an empty block statement
* @id ruby/examples/emptythen
* @tags if
* then
* empty
* conditional
* branch
* statement
*/
import ruby
from IfExpr i
where not exists(i.getThen().getAChild())
select i

View File

@@ -0,0 +1,19 @@
private import codeql.files.FileSystem
/**
* Returns an appropriately encoded version of a filename `name`
* passed by the VS Code extension in order to coincide with the
* output of `.getFile()` on locatable entities.
*/
cached
File getFileBySourceArchiveName(string name) {
// The name provided for a file in the source archive by the VS Code extension
// has some differences from the absolute path in the database:
// 1. colons are replaced by underscores
// 2. there's a leading slash, even for Windows paths: "C:/foo/bar" ->
// "/C_/foo/bar"
// 3. double slashes in UNC prefixes are replaced with a single slash
// We can handle 2 and 3 together by unconditionally adding a leading slash
// before replacing double slashes.
name = ("/" + result.getAbsolutePath().replaceAll(":", "_")).replaceAll("//", "/")
}

View File

@@ -0,0 +1,66 @@
/** Provides classes for working with locations. */
import files.FileSystem
/**
* A location as given by a file, a start line, a start column,
* an end line, and an end column.
*
* For more information about locations see [LGTM locations](https://codeql.github.com/docs/writing-codeql-queries/providing-locations-in-codeql-queries/).
*/
class Location extends @location {
/** Gets the file for this location. */
File getFile() { locations_default(this, result, _, _, _, _) }
/** Gets the 1-based line number (inclusive) where this location starts. */
int getStartLine() { locations_default(this, _, result, _, _, _) }
/** Gets the 1-based column number (inclusive) where this location starts. */
int getStartColumn() { locations_default(this, _, _, result, _, _) }
/** Gets the 1-based line number (inclusive) where this location ends. */
int getEndLine() { locations_default(this, _, _, _, result, _) }
/** Gets the 1-based column number (inclusive) where this location ends. */
int getEndColumn() { locations_default(this, _, _, _, _, result) }
/** Gets the number of lines covered by this location. */
int getNumLines() { result = getEndLine() - getStartLine() + 1 }
/** Gets a textual representation of this element. */
string toString() {
exists(string filepath, int startline, int startcolumn, int endline, int endcolumn |
hasLocationInfo(filepath, startline, startcolumn, endline, endcolumn) and
result = filepath + "@" + startline + ":" + startcolumn + ":" + endline + ":" + endcolumn
)
}
/**
* Holds if this element is at the specified location.
* The location spans column `startcolumn` of line `startline` to
* column `endcolumn` of line `endline` in file `filepath`.
* For more information, see
* [LGTM locations](https://codeql.github.com/docs/writing-codeql-queries/providing-locations-in-codeql-queries/).
*/
predicate hasLocationInfo(
string filepath, int startline, int startcolumn, int endline, int endcolumn
) {
exists(File f |
locations_default(this, f, startline, startcolumn, endline, endcolumn) and
filepath = f.getAbsolutePath()
)
}
/** Holds if this location starts strictly before the specified location. */
pragma[inline]
predicate strictlyBefore(Location other) {
this.getStartLine() < other.getStartLine()
or
this.getStartLine() = other.getStartLine() and this.getStartColumn() < other.getStartColumn()
}
}
/** An entity representing an empty location. */
class EmptyLocation extends Location {
EmptyLocation() { this.hasLocationInfo("", 0, 0, 0, 0) }
}

View File

@@ -0,0 +1,173 @@
/** Provides classes for working with files and folders. */
private import codeql.Locations
/** A file or folder. */
abstract class Container extends @container {
/** Gets a file or sub-folder in this container. */
Container getAChildContainer() { this = result.getParentContainer() }
/** Gets a file in this container. */
File getAFile() { result = getAChildContainer() }
/** Gets a sub-folder in this container. */
Folder getAFolder() { result = getAChildContainer() }
/**
* Gets the absolute, canonical path of this container, using forward slashes
* as path separator.
*
* The path starts with a _root prefix_ followed by zero or more _path
* segments_ separated by forward slashes.
*
* The root prefix is of one of the following forms:
*
* 1. A single forward slash `/` (Unix-style)
* 2. An upper-case drive letter followed by a colon and a forward slash,
* such as `C:/` (Windows-style)
* 3. Two forward slashes, a computer name, and then another forward slash,
* such as `//FileServer/` (UNC-style)
*
* Path segments are never empty (that is, absolute paths never contain two
* contiguous slashes, except as part of a UNC-style root prefix). Also, path
* segments never contain forward slashes, and no path segment is of the
* form `.` (one dot) or `..` (two dots).
*
* Note that an absolute path never ends with a forward slash, except if it is
* a bare root prefix, that is, the path has no path segments. A container
* whose absolute path has no segments is always a `Folder`, not a `File`.
*/
abstract string getAbsolutePath();
/**
* Gets the base name of this container including extension, that is, the last
* segment of its absolute path, or the empty string if it has no segments.
*
* Here are some examples of absolute paths and the corresponding base names
* (surrounded with quotes to avoid ambiguity):
*
* <table border="1">
* <tr><th>Absolute path</th><th>Base name</th></tr>
* <tr><td>"/tmp/tst.go"</td><td>"tst.go"</td></tr>
* <tr><td>"C:/Program Files (x86)"</td><td>"Program Files (x86)"</td></tr>
* <tr><td>"/"</td><td>""</td></tr>
* <tr><td>"C:/"</td><td>""</td></tr>
* <tr><td>"D:/"</td><td>""</td></tr>
* <tr><td>"//FileServer/"</td><td>""</td></tr>
* </table>
*/
string getBaseName() {
result = getAbsolutePath().regexpCapture(".*/(([^/]*?)(?:\\.([^.]*))?)", 1)
}
/**
* Gets the extension of this container, that is, the suffix of its base name
* after the last dot character, if any.
*
* In particular,
*
* - if the name does not include a dot, there is no extension, so this
* predicate has no result;
* - if the name ends in a dot, the extension is the empty string;
* - if the name contains multiple dots, the extension follows the last dot.
*
* Here are some examples of absolute paths and the corresponding extensions
* (surrounded with quotes to avoid ambiguity):
*
* <table border="1">
* <tr><th>Absolute path</th><th>Extension</th></tr>
* <tr><td>"/tmp/tst.go"</td><td>"go"</td></tr>
* <tr><td>"/tmp/.classpath"</td><td>"classpath"</td></tr>
* <tr><td>"/bin/bash"</td><td>not defined</td></tr>
* <tr><td>"/tmp/tst2."</td><td>""</td></tr>
* <tr><td>"/tmp/x.tar.gz"</td><td>"gz"</td></tr>
* </table>
*/
string getExtension() { result = getAbsolutePath().regexpCapture(".*/([^/]*?)(\\.([^.]*))?", 3) }
/** Gets the file in this container that has the given `baseName`, if any. */
File getFile(string baseName) {
result = getAFile() and
result.getBaseName() = baseName
}
/** Gets the sub-folder in this container that has the given `baseName`, if any. */
Folder getFolder(string baseName) {
result = getAFolder() and
result.getBaseName() = baseName
}
/** Gets the parent container of this file or folder, if any. */
Container getParentContainer() { containerparent(result, this) }
/**
* Gets the relative path of this file or folder from the root folder of the
* analyzed source location. The relative path of the root folder itself is
* the empty string.
*
* This has no result if the container is outside the source root, that is,
* if the root folder is not a reflexive, transitive parent of this container.
*/
string getRelativePath() {
exists(string absPath, string pref |
absPath = getAbsolutePath() and sourceLocationPrefix(pref)
|
absPath = pref and result = ""
or
absPath = pref.regexpReplaceAll("/$", "") + "/" + result and
not result.matches("/%")
)
}
/**
* Gets the stem of this container, that is, the prefix of its base name up to
* (but not including) the last dot character if there is one, or the entire
* base name if there is not.
*
* Here are some examples of absolute paths and the corresponding stems
* (surrounded with quotes to avoid ambiguity):
*
* <table border="1">
* <tr><th>Absolute path</th><th>Stem</th></tr>
* <tr><td>"/tmp/tst.go"</td><td>"tst"</td></tr>
* <tr><td>"/tmp/.classpath"</td><td>""</td></tr>
* <tr><td>"/bin/bash"</td><td>"bash"</td></tr>
* <tr><td>"/tmp/tst2."</td><td>"tst2"</td></tr>
* <tr><td>"/tmp/x.tar.gz"</td><td>"x.tar"</td></tr>
* </table>
*/
string getStem() { result = getAbsolutePath().regexpCapture(".*/([^/]*?)(?:\\.([^.]*))?", 1) }
/**
* Gets a URL representing the location of this container.
*
* For more information see https://codeql.github.com/docs/writing-codeql-queries/providing-locations-in-codeql-queries/#providing-urls.
*/
abstract string getURL();
/**
* Gets a textual representation of the path of this container.
*
* This is the absolute path of the container.
*/
string toString() { result = getAbsolutePath() }
}
/** A folder. */
class Folder extends Container, @folder {
override string getAbsolutePath() { folders(this, result) }
/** Gets the URL of this folder. */
override string getURL() { result = "folder://" + getAbsolutePath() }
}
/** A file. */
class File extends Container, @file {
override string getAbsolutePath() { files(this, result) }
/** Gets the URL of this file. */
override string getURL() { result = "file://" + this.getAbsolutePath() + ":0:0:0:0" }
/** Holds if this file was extracted from ordinary source code. */
predicate fromSource() { any() }
}

View File

@@ -0,0 +1,141 @@
import codeql.Locations
import ast.Call
import ast.Control
import ast.Constant
import ast.Erb
import ast.Expr
import ast.Literal
import ast.Method
import ast.Module
import ast.Parameter
import ast.Operation
import ast.Pattern
import ast.Scope
import ast.Statement
import ast.Variable
private import ast.internal.AST
private import ast.internal.Scope
private import ast.internal.Synthesis
private import ast.internal.TreeSitter
/**
* A node in the abstract syntax tree. This class is the base class for all Ruby
* program elements.
*/
class AstNode extends TAstNode {
/**
* Gets the name of a primary CodeQL class to which this node belongs.
*
* This predicate always has a result. If no primary class can be
* determined, the result is `"???"`. If multiple primary classes match,
* this predicate can have multiple results.
*/
string getAPrimaryQlClass() { result = "???" }
/**
* Gets a comma-separated list of the names of the primary CodeQL classes to
* which this element belongs.
*/
final string getPrimaryQlClasses() { result = concat(this.getAPrimaryQlClass(), ",") }
/** Gets the enclosing module, if any. */
ModuleBase getEnclosingModule() {
exists(Scope::Range s |
s = scopeOf(toGeneratedInclSynth(this)) and
toGeneratedInclSynth(result) = s.getEnclosingModule()
)
}
/** Gets the enclosing method, if any. */
MethodBase getEnclosingMethod() {
exists(Scope::Range s |
s = scopeOf(toGeneratedInclSynth(this)) and
toGeneratedInclSynth(result) = s.getEnclosingMethod()
)
}
/** Gets a textual representation of this node. */
cached
string toString() { none() }
/** Gets the location of this node. */
Location getLocation() { result = getLocation(this) }
/** Gets the file of this node. */
final File getFile() { result = this.getLocation().getFile() }
/** Gets a child node of this `AstNode`. */
final AstNode getAChild() { result = this.getAChild(_) }
/** Gets the parent of this `AstNode`, if this node is not a root node. */
final AstNode getParent() { result.getAChild() = this }
/**
* Gets a child of this node, which can also be retrieved using a predicate
* named `pred`.
*/
cached
AstNode getAChild(string pred) {
pred = "getDesugared" and
result = this.getDesugared()
}
/**
* Holds if this node was synthesized to represent an implicit AST node not
* present in the source code. In the following example method call, the
* receiver is an implicit `self` reference, for which there is a synthesized
* `Self` node.
*
* ```rb
* foo(123)
* ```
*/
final predicate isSynthesized() { this = getSynthChild(_, _) }
/**
* Gets the desugared version of this AST node, if any.
*
* For example, the desugared version of
*
* ```rb
* x += y
* ```
*
* is
*
* ```rb
* x = x + y
* ```
*
* when `x` is a variable. Whenever an AST node can be desugared,
* then the desugared version is used in the control-flow graph.
*/
final AstNode getDesugared() { result = getSynthChild(this, -1) }
}
/** A Ruby source file */
class RubyFile extends File {
RubyFile() { ruby_ast_node_parent(_, this, _) }
/** Gets a token in this file. */
private Ruby::Token getAToken() { result.getLocation().getFile() = this }
/** Holds if `line` contains a token. */
private predicate line(int line, boolean comment) {
exists(Ruby::Token token, Location l |
token = this.getAToken() and
l = token.getLocation() and
line in [l.getStartLine() .. l.getEndLine()] and
if token instanceof @ruby_token_comment then comment = true else comment = false
)
}
/** Gets the number of lines in this file. */
int getNumberOfLines() { result = max([0, this.getAToken().getLocation().getEndLine()]) }
/** Gets the number of lines of code in this file. */
int getNumberOfLinesOfCode() { result = count(int line | this.line(line, false)) }
/** Gets the number of lines of comments in this file. */
int getNumberOfLinesOfComments() { result = count(int line | this.line(line, true)) }
}

View File

@@ -0,0 +1,391 @@
/**
* Provides an implementation of _API graphs_, which are an abstract representation of the API
* surface used and/or defined by a code base.
*
* The nodes of the API graph represent definitions and uses of API components. The edges are
* directed and labeled; they specify how the components represented by nodes relate to each other.
*/
private import ruby
import codeql.ruby.DataFlow
import codeql.ruby.typetracking.TypeTracker
import codeql.ruby.ast.internal.Module
private import codeql.ruby.controlflow.CfgNodes
/**
* Provides classes and predicates for working with APIs used in a database.
*/
module API {
/**
* An abstract representation of a definition or use of an API component such as a Ruby module,
* or the result of a method call.
*/
class Node extends Impl::TApiNode {
/**
* Gets a data-flow node corresponding to a use of the API component represented by this node.
*
* For example, `Kernel.format "%s world!", "Hello"` is a use of the return of the `format` function of
* the `Kernel` module.
*
* This includes indirect uses found via data flow.
*/
DataFlow::Node getAUse() {
exists(DataFlow::LocalSourceNode src | Impl::use(this, src) |
Impl::trackUseNode(src).flowsTo(result)
)
}
/**
* Gets an immediate use of the API component represented by this node.
*
* Unlike `getAUse()`, this predicate only gets the immediate references, not the indirect uses
* found via data flow.
*/
DataFlow::LocalSourceNode getAnImmediateUse() { Impl::use(this, result) }
/**
* Gets a call to a method on the receiver represented by this API component.
*/
DataFlow::CallNode getAMethodCall(string method) {
result = getReturn(method).getAnImmediateUse()
}
/**
* Gets a node representing member `m` of this API component.
*
* For example, a member can be:
*
* - A submodule of a module
* - An attribute of an object
*/
bindingset[m]
bindingset[result]
Node getMember(string m) { result = getASuccessor(Label::member(m)) }
/**
* Gets a node representing a member of this API component where the name of the member is
* not known statically.
*/
Node getUnknownMember() { result = getASuccessor(Label::unknownMember()) }
/**
* Gets a node representing a member of this API component where the name of the member may
* or may not be known statically.
*/
Node getAMember() {
result = getASuccessor(Label::member(_)) or
result = getUnknownMember()
}
/**
* Gets a node representing an instance of this API component, that is, an object whose
* constructor is the function represented by this node.
*
* For example, if this node represents a use of some class `A`, then there might be a node
* representing instances of `A`, typically corresponding to expressions `new A()` at the
* source level.
*
* This predicate may have multiple results when there are multiple constructor calls invoking this API component.
* Consider using `getAnInstantiation()` if there is a need to distinguish between individual constructor calls.
*/
Node getInstance() { result = getASuccessor(Label::instance()) }
/**
* Gets a node representing the result of calling a method on the receiver represented by this node.
*/
Node getReturn(string method) { result = getASuccessor(Label::return(method)) }
/**
* Gets a `new` call to the function represented by this API component.
*/
DataFlow::Node getAnInstantiation() { result = getInstance().getAnImmediateUse() }
/**
* Gets a node representing a subclass of the class represented by this node.
*/
Node getASubclass() { result = getASuccessor(Label::subclass()) }
/**
* Gets a string representation of the lexicographically least among all shortest access paths
* from the root to this node.
*/
string getPath() { result = min(string p | p = getAPath(Impl::distanceFromRoot(this)) | p) }
/**
* Gets a node such that there is an edge in the API graph between this node and the other
* one, and that edge is labeled with `lbl`.
*/
Node getASuccessor(string lbl) { Impl::edge(this, lbl, result) }
/**
* Gets a node such that there is an edge in the API graph between that other node and
* this one, and that edge is labeled with `lbl`
*/
Node getAPredecessor(string lbl) { this = result.getASuccessor(lbl) }
/**
* Gets a node such that there is an edge in the API graph between this node and the other
* one.
*/
Node getAPredecessor() { result = getAPredecessor(_) }
/**
* Gets a node such that there is an edge in the API graph between that other node and
* this one.
*/
Node getASuccessor() { result = getASuccessor(_) }
/**
* Gets the data-flow node that gives rise to this node, if any.
*/
DataFlow::Node getInducingNode() { this = Impl::MkUse(result) }
/** Gets the location of this node. */
Location getLocation() {
result = this.getInducingNode().getLocation()
or
// For nodes that do not have a meaningful location, `path` is the empty string and all other
// parameters are zero.
not exists(getInducingNode()) and
result instanceof EmptyLocation
}
/**
* Gets a textual representation of this element.
*/
abstract string toString();
/**
* Gets a path of the given `length` from the root to this node.
*/
private string getAPath(int length) {
this instanceof Impl::MkRoot and
length = 0 and
result = ""
or
exists(Node pred, string lbl, string predpath |
Impl::edge(pred, lbl, this) and
lbl != "" and
predpath = pred.getAPath(length - 1) and
exists(string dot | if length = 1 then dot = "" else dot = "." |
result = predpath + dot + lbl and
// avoid producing strings longer than 1MB
result.length() < 1000 * 1000
)
) and
length in [1 .. Impl::distanceFromRoot(this)]
}
/** Gets the shortest distance from the root to this node in the API graph. */
int getDepth() { result = Impl::distanceFromRoot(this) }
}
/** The root node of an API graph. */
class Root extends Node, Impl::MkRoot {
override string toString() { result = "root" }
}
/** A node corresponding to the use of an API component. */
class Use extends Node, Impl::MkUse {
override string toString() {
exists(string type | this = Impl::MkUse(_) and type = "Use " |
result = type + getPath()
or
not exists(this.getPath()) and result = type + "with no path"
)
}
}
/** Gets the root node. */
Root root() { any() }
/**
* Gets a node corresponding to a top-level member `m` (typically a module).
*
* This is equivalent to `root().getAMember("m")`.
*
* Note: You should only use this predicate for top level modules or classes. If you want nodes corresponding to a nested module or class,
* you should use `.getMember` on the parent module/class. For example, for nodes corresponding to the class `Gem::Version`,
* use `getTopLevelMember("Gem").getMember("Version")`.
*/
Node getTopLevelMember(string m) { result = root().getMember(m) }
/**
* Provides the actual implementation of API graphs, cached for performance.
*
* Ideally, we'd like nodes to correspond to (global) access paths, with edge labels
* corresponding to extending the access path by one element. We also want to be able to map
* nodes to their definitions and uses in the data-flow graph, and this should happen modulo
* (inter-procedural) data flow.
*
* This, however, is not easy to implement, since access paths can have unbounded length
* and we need some way of recognizing cycles to avoid non-termination. Unfortunately, expressing
* a condition like "this node hasn't been involved in constructing any predecessor of
* this node in the API graph" without negative recursion is tricky.
*
* So instead most nodes are directly associated with a data-flow node, representing
* either a use or a definition of an API component. This ensures that we only have a finite
* number of nodes. However, we can now have multiple nodes with the same access
* path, which are essentially indistinguishable for a client of the API.
*
* On the other hand, a single node can have multiple access paths (which is, of
* course, unavoidable). We pick as canonical the alphabetically least access path with
* shortest length.
*/
cached
private module Impl {
cached
newtype TApiNode =
/** The root of the API graph. */
MkRoot() or
/** A use of an API member at the node `nd`. */
MkUse(DataFlow::Node nd) { use(_, _, nd) }
private string resolveTopLevel(ConstantReadAccess read) {
TResolved(result) = resolveScopeExpr(read) and
not result.matches("%::%")
}
/**
* Holds if `ref` is a use of a node that should have an incoming edge from `base` labeled
* `lbl` in the API graph.
*/
cached
predicate use(TApiNode base, string lbl, DataFlow::Node ref) {
base = MkRoot() and
exists(string name, ExprNodes::ConstantAccessCfgNode access, ConstantReadAccess read |
access = ref.asExpr() and
lbl = Label::member(read.getName()) and
read = access.getExpr()
|
name = resolveTopLevel(read)
or
name = read.getName() and
not exists(resolveTopLevel(read)) and
not exists(read.getScopeExpr())
)
or
exists(ExprCfgNode node |
// First, we find a predecessor of the node `ref` that we want to determine. The predecessor
// is any node that is a type-tracked use of a data flow node (`src`), which is itself a
// reference to the API node `base`. Thus, `pred` and `src` both represent uses of `base`.
//
// Once we have identified the predecessor, we define its relation to the successor `ref` as
// well as the label on the edge from `pred` to `ref`. This label describes the nature of
// the relationship between `pred` and `ref`.
useExpr(node, base)
|
// // Referring to an attribute on a node that is a use of `base`:
// pred = `Rails` part of `Rails::Whatever`
// lbl = `Whatever`
// ref = `Rails::Whatever`
exists(ExprNodes::ConstantAccessCfgNode c, ConstantReadAccess read |
not exists(resolveTopLevel(read)) and
node = c.getScopeExpr() and
lbl = Label::member(read.getName()) and
ref.asExpr() = c and
read = c.getExpr()
)
or
// Calling a method on a node that is a use of `base`
exists(ExprNodes::MethodCallCfgNode call, string name |
node = call.getReceiver() and
name = call.getExpr().getMethodName() and
lbl = Label::return(name) and
name != "new" and
ref.asExpr() = call
)
or
// Calling the `new` method on a node that is a use of `base`, which creates a new instance
exists(ExprNodes::MethodCallCfgNode call |
node = call.getReceiver() and
lbl = Label::instance() and
call.getExpr().getMethodName() = "new" and
ref.asExpr() = call
)
)
}
pragma[nomagic]
private predicate useExpr(ExprCfgNode node, TApiNode base) {
exists(DataFlow::LocalSourceNode src, DataFlow::LocalSourceNode pred |
use(base, src) and
pred = trackUseNode(src) and
pred.flowsTo(any(DataFlow::ExprNode n | n.getExprNode() = node))
)
}
/**
* Holds if `ref` is a use of node `nd`.
*/
cached
predicate use(TApiNode nd, DataFlow::Node ref) { nd = MkUse(ref) }
/**
* Gets a data-flow node to which `src`, which is a use of an API-graph node, flows.
*
* The flow from `src` to that node may be inter-procedural.
*/
private DataFlow::LocalSourceNode trackUseNode(DataFlow::Node src, TypeTracker t) {
// Declaring `src` to be a `LocalSourceNode` currently causes a redundant check in the
// recursive case, so instead we check it explicitly here.
src instanceof DataFlow::LocalSourceNode and
t.start() and
use(_, src) and
result = src
or
exists(TypeTracker t2 | result = trackUseNode(src, t2).track(t2, t))
}
/**
* Gets a data-flow node to which `src`, which is a use of an API-graph node, flows.
*
* The flow from `src` to that node may be inter-procedural.
*/
cached
DataFlow::LocalSourceNode trackUseNode(DataFlow::LocalSourceNode src) {
result = trackUseNode(src, TypeTracker::end())
}
/**
* Holds if there is an edge from `pred` to `succ` in the API graph that is labeled with `lbl`.
*/
cached
predicate edge(TApiNode pred, string lbl, TApiNode succ) {
/* Every node that is a use of an API component is itself added to the API graph. */
exists(DataFlow::LocalSourceNode ref |
use(pred, lbl, ref) and
succ = MkUse(ref)
)
}
/**
* Holds if there is an edge from `pred` to `succ` in the API graph.
*/
private predicate edge(TApiNode pred, TApiNode succ) { edge(pred, _, succ) }
/** Gets the shortest distance from the root to `nd` in the API graph. */
cached
int distanceFromRoot(TApiNode nd) = shortestDistances(MkRoot/0, edge/2)(_, nd, result)
}
}
private module Label {
/** Gets the `member` edge label for member `m`. */
bindingset[m]
bindingset[result]
string member(string m) { result = "getMember(\"" + m + "\")" }
/** Gets the `member` edge label for the unknown member. */
string unknownMember() { result = "getUnknownMember()" }
/** Gets the `instance` edge label. */
string instance() { result = "instance" }
/** Gets the `return` edge label. */
bindingset[m]
bindingset[result]
string return(string m) { result = "getReturn(\"" + m + "\")" }
string subclass() { result = "getASubclass()" }
}

View File

@@ -0,0 +1,5 @@
/** Provides classes representing the control flow graph. */
import controlflow.ControlFlowGraph
import controlflow.CfgNodes as CfgNodes
import controlflow.BasicBlocks

View File

@@ -0,0 +1,505 @@
/**
* Provides abstract classes representing generic concepts such as file system
* access or system command execution, for which individual framework libraries
* provide concrete subclasses.
*/
private import codeql.ruby.AST
private import codeql.ruby.CFG
private import codeql.ruby.DataFlow
private import codeql.ruby.Frameworks
private import codeql.ruby.dataflow.RemoteFlowSources
private import codeql.ruby.ApiGraphs
/**
* A data-flow node that executes SQL statements.
*
* Extend this class to refine existing API models. If you want to model new APIs,
* extend `SqlExecution::Range` instead.
*/
class SqlExecution extends DataFlow::Node instanceof SqlExecution::Range {
/** Gets the argument that specifies the SQL statements to be executed. */
DataFlow::Node getSql() { result = super.getSql() }
}
/** Provides a class for modeling new SQL execution APIs. */
module SqlExecution {
/**
* A data-flow node that executes SQL statements.
*
* Extend this class to model new APIs. If you want to refine existing API models,
* extend `SqlExecution` instead.
*/
abstract class Range extends DataFlow::Node {
/** Gets the argument that specifies the SQL statements to be executed. */
abstract DataFlow::Node getSql();
}
}
/**
* A data flow node that performs a file system access, including reading and writing data,
* creating and deleting files and folders, checking and updating permissions, and so on.
*
* Extend this class to refine existing API models. If you want to model new APIs,
* extend `FileSystemAccess::Range` instead.
*/
class FileSystemAccess extends DataFlow::Node instanceof FileSystemAccess::Range {
/** Gets an argument to this file system access that is interpreted as a path. */
DataFlow::Node getAPathArgument() { result = super.getAPathArgument() }
}
/** Provides a class for modeling new file system access APIs. */
module FileSystemAccess {
/**
* A data-flow node that performs a file system access, including reading and writing data,
* creating and deleting files and folders, checking and updating permissions, and so on.
*
* Extend this class to model new APIs. If you want to refine existing API models,
* extend `FileSystemAccess` instead.
*/
abstract class Range extends DataFlow::Node {
/** Gets an argument to this file system access that is interpreted as a path. */
abstract DataFlow::Node getAPathArgument();
}
}
/**
* A data flow node that reads data from the file system.
*
* Extend this class to refine existing API models. If you want to model new APIs,
* extend `FileSystemReadAccess::Range` instead.
*/
class FileSystemReadAccess extends FileSystemAccess instanceof FileSystemReadAccess::Range {
/**
* Gets a node that represents data read from the file system access.
*/
DataFlow::Node getADataNode() { result = FileSystemReadAccess::Range.super.getADataNode() }
}
/** Provides a class for modeling new file system reads. */
module FileSystemReadAccess {
/**
* A data flow node that reads data from the file system.
*
* Extend this class to model new APIs. If you want to refine existing API models,
* extend `FileSystemReadAccess` instead.
*/
abstract class Range extends FileSystemAccess::Range {
/**
* Gets a node that represents data read from the file system.
*/
abstract DataFlow::Node getADataNode();
}
}
/**
* A data flow node that sets the permissions for one or more files.
*
* Extend this class to refine existing API models. If you want to model new APIs,
* extend `FileSystemPermissionModification::Range` instead.
*/
class FileSystemPermissionModification extends DataFlow::Node instanceof FileSystemPermissionModification::Range {
/**
* Gets an argument to this permission modification that is interpreted as a
* set of permissions.
*/
DataFlow::Node getAPermissionNode() { result = super.getAPermissionNode() }
}
/** Provides a class for modeling new file system permission modifications. */
module FileSystemPermissionModification {
/**
* A data-flow node that sets permissions for a one or more files.
*
* Extend this class to model new APIs. If you want to refine existing API models,
* extend `FileSystemPermissionModification` instead.
*/
abstract class Range extends DataFlow::Node {
/**
* Gets an argument to this permission modification that is interpreted as a
* set of permissions.
*/
abstract DataFlow::Node getAPermissionNode();
}
}
/**
* A data flow node that contains a file name or an array of file names from the local file system.
*/
abstract class FileNameSource extends DataFlow::Node { }
/**
* A data-flow node that escapes meta-characters, which could be used to prevent
* injection attacks.
*
* Extend this class to refine existing API models. If you want to model new APIs,
* extend `Escaping::Range` instead.
*/
class Escaping extends DataFlow::Node instanceof Escaping::Range {
Escaping() {
// escapes that don't have _both_ input/output defined are not valid
exists(super.getAnInput()) and
exists(super.getOutput())
}
/** Gets an input that will be escaped. */
DataFlow::Node getAnInput() { result = super.getAnInput() }
/** Gets the output that contains the escaped data. */
DataFlow::Node getOutput() { result = super.getOutput() }
/**
* Gets the context that this function escapes for, such as `html`, or `url`.
*/
string getKind() { result = super.getKind() }
}
/** Provides a class for modeling new escaping APIs. */
module Escaping {
/**
* A data-flow node that escapes meta-characters, which could be used to prevent
* injection attacks.
*
* Extend this class to model new APIs. If you want to refine existing API models,
* extend `Escaping` instead.
*/
abstract class Range extends DataFlow::Node {
/** Gets an input that will be escaped. */
abstract DataFlow::Node getAnInput();
/** Gets the output that contains the escaped data. */
abstract DataFlow::Node getOutput();
/**
* Gets the context that this function escapes for.
*
* While kinds are represented as strings, this should not be relied upon. Use the
* predicates in the `Escaping` module, such as `getHtmlKind`.
*/
abstract string getKind();
}
/** Gets the escape-kind for escaping a string so it can safely be included in HTML. */
string getHtmlKind() { result = "html" }
}
/**
* An escape of a string so it can be safely included in
* the body of an HTML element, for example, replacing `{}` in
* `<p>{}</p>`.
*/
class HtmlEscaping extends Escaping {
HtmlEscaping() { super.getKind() = Escaping::getHtmlKind() }
}
/** Provides classes for modeling HTTP-related APIs. */
module HTTP {
/** Provides classes for modeling HTTP servers. */
module Server {
/**
* A data-flow node that sets up a route on a server.
*
* Extend this class to refine existing API models. If you want to model new APIs,
* extend `RouteSetup::Range` instead.
*/
class RouteSetup extends DataFlow::Node instanceof RouteSetup::Range {
/** Gets the URL pattern for this route, if it can be statically determined. */
string getUrlPattern() { result = super.getUrlPattern() }
/**
* Gets a function that will handle incoming requests for this route, if any.
*
* NOTE: This will be modified in the near future to have a `RequestHandler` result, instead of a `Method`.
*/
Method getARequestHandler() { result = super.getARequestHandler() }
/**
* Gets a parameter that will receive parts of the url when handling incoming
* requests for this route, if any. These automatically become a `RemoteFlowSource`.
*/
Parameter getARoutedParameter() { result = super.getARoutedParameter() }
/** Gets a string that identifies the framework used for this route setup. */
string getFramework() { result = super.getFramework() }
}
/** Provides a class for modeling new HTTP routing APIs. */
module RouteSetup {
/**
* A data-flow node that sets up a route on a server.
*
* Extend this class to model new APIs. If you want to refine existing API models,
* extend `RouteSetup` instead.
*/
abstract class Range extends DataFlow::Node {
/** Gets the argument used to set the URL pattern. */
abstract DataFlow::Node getUrlPatternArg();
/** Gets the URL pattern for this route, if it can be statically determined. */
string getUrlPattern() {
exists(CfgNodes::ExprNodes::StringlikeLiteralCfgNode strNode |
this.getUrlPatternArg().getALocalSource() = DataFlow::exprNode(strNode) and
result = strNode.getExpr().getValueText()
)
}
/**
* Gets a function that will handle incoming requests for this route, if any.
*
* NOTE: This will be modified in the near future to have a `RequestHandler` result, instead of a `Method`.
*/
abstract Method getARequestHandler();
/**
* Gets a parameter that will receive parts of the url when handling incoming
* requests for this route, if any. These automatically become a `RemoteFlowSource`.
*/
abstract Parameter getARoutedParameter();
/** Gets a string that identifies the framework used for this route setup. */
abstract string getFramework();
}
}
/**
* A function that will handle incoming HTTP requests.
*
* Extend this class to refine existing API models. If you want to model new APIs,
* extend `RequestHandler::Range` instead.
*/
class RequestHandler extends Method instanceof RequestHandler::Range {
/**
* Gets a parameter that could receive parts of the url when handling incoming
* requests, if any. These automatically become a `RemoteFlowSource`.
*/
Parameter getARoutedParameter() { result = super.getARoutedParameter() }
/** Gets a string that identifies the framework used for this route setup. */
string getFramework() { result = super.getFramework() }
}
/** Provides a class for modeling new HTTP request handlers. */
module RequestHandler {
/**
* A function that will handle incoming HTTP requests.
*
* Extend this class to model new APIs. If you want to refine existing API models,
* extend `RequestHandler` instead.
*
* Only extend this class if you can't provide a `RouteSetup`, since we handle that case automatically.
*/
abstract class Range extends Method {
/**
* Gets a parameter that could receive parts of the url when handling incoming
* requests, if any. These automatically become a `RemoteFlowSource`.
*/
abstract Parameter getARoutedParameter();
/** Gets a string that identifies the framework used for this request handler. */
abstract string getFramework();
}
}
private class RequestHandlerFromRouteSetup extends RequestHandler::Range {
RouteSetup rs;
RequestHandlerFromRouteSetup() { this = rs.getARequestHandler() }
override Parameter getARoutedParameter() {
result = rs.getARoutedParameter() and
result = this.getAParameter()
}
override string getFramework() { result = rs.getFramework() }
}
/** A parameter that will receive parts of the url when handling an incoming request. */
private class RoutedParameter extends RemoteFlowSource::Range, DataFlow::ParameterNode {
RequestHandler handler;
RoutedParameter() { this.getParameter() = handler.getARoutedParameter() }
override string getSourceType() { result = handler.getFramework() + " RoutedParameter" }
}
/**
* A data-flow node that creates a HTTP response on a server.
*
* Note: we don't require that this response must be sent to a client (a kind of
* "if a tree falls in a forest and nobody hears it" situation).
*
* Extend this class to refine existing API models. If you want to model new APIs,
* extend `HttpResponse::Range` instead.
*/
class HttpResponse extends DataFlow::Node instanceof HttpResponse::Range {
/** Gets the data-flow node that specifies the body of this HTTP response. */
DataFlow::Node getBody() { result = super.getBody() }
/** Gets the mimetype of this HTTP response, if it can be statically determined. */
string getMimetype() { result = super.getMimetype() }
}
/** Provides a class for modeling new HTTP response APIs. */
module HttpResponse {
/**
* A data-flow node that creates a HTTP response on a server.
*
* Note: we don't require that this response must be sent to a client (a kind of
* "if a tree falls in a forest and nobody hears it" situation).
*
* Extend this class to model new APIs. If you want to refine existing API models,
* extend `HttpResponse` instead.
*/
abstract class Range extends DataFlow::Node {
/** Gets the data-flow node that specifies the body of this HTTP response. */
abstract DataFlow::Node getBody();
/** Gets the data-flow node that specifies the content-type/mimetype of this HTTP response, if any. */
abstract DataFlow::Node getMimetypeOrContentTypeArg();
/** Gets the default mimetype that should be used if `getMimetypeOrContentTypeArg` has no results. */
abstract string getMimetypeDefault();
/** Gets the mimetype of this HTTP response, if it can be statically determined. */
string getMimetype() {
exists(CfgNodes::ExprNodes::StringlikeLiteralCfgNode strNode |
this.getMimetypeOrContentTypeArg().getALocalSource() = DataFlow::exprNode(strNode) and
result = strNode.getExpr().getValueText().splitAt(";", 0)
)
or
not exists(this.getMimetypeOrContentTypeArg()) and
result = this.getMimetypeDefault()
}
}
}
/**
* A data-flow node that creates a HTTP redirect response on a server.
*
* Note: we don't require that this redirect must be sent to a client (a kind of
* "if a tree falls in a forest and nobody hears it" situation).
*
* Extend this class to refine existing API models. If you want to model new APIs,
* extend `HttpRedirectResponse::Range` instead.
*/
class HttpRedirectResponse extends HttpResponse instanceof HttpRedirectResponse::Range {
/** Gets the data-flow node that specifies the location of this HTTP redirect response. */
DataFlow::Node getRedirectLocation() { result = super.getRedirectLocation() }
}
/** Provides a class for modeling new HTTP redirect response APIs. */
module HttpRedirectResponse {
/**
* A data-flow node that creates a HTTP redirect response on a server.
*
* Note: we don't require that this redirect must be sent to a client (a kind of
* "if a tree falls in a forest and nobody hears it" situation).
*
* Extend this class to model new APIs. If you want to refine existing API models,
* extend `HttpResponse` instead.
*/
abstract class Range extends HTTP::Server::HttpResponse::Range {
/** Gets the data-flow node that specifies the location of this HTTP redirect response. */
abstract DataFlow::Node getRedirectLocation();
}
}
}
/** Provides classes for modeling HTTP clients. */
module Client {
/**
* A method call that makes an outgoing HTTP request.
*
* Extend this class to refine existing API models. If you want to model new APIs,
* extend `Request::Range` instead.
*/
class Request extends MethodCall instanceof Request::Range {
/** Gets a node which returns the body of the response */
DataFlow::Node getResponseBody() { result = super.getResponseBody() }
/** Gets a string that identifies the framework used for this request. */
string getFramework() { result = super.getFramework() }
}
/** Provides a class for modeling new HTTP requests. */
module Request {
/**
* A method call that makes an outgoing HTTP request.
*
* Extend this class to model new APIs. If you want to refine existing API models,
* extend `Request` instead.
*/
abstract class Range extends MethodCall {
/** Gets a node which returns the body of the response */
abstract DataFlow::Node getResponseBody();
/** Gets a string that identifies the framework used for this request. */
abstract string getFramework();
}
}
/** The response body from an outgoing HTTP request, considered as a remote flow source */
private class RequestResponseBody extends RemoteFlowSource::Range, DataFlow::Node {
Request request;
RequestResponseBody() { this = request.getResponseBody() }
override string getSourceType() { result = request.getFramework() }
}
}
}
/**
* A data flow node that executes an operating system command,
* for instance by spawning a new process.
*/
class SystemCommandExecution extends DataFlow::Node instanceof SystemCommandExecution::Range {
/** Holds if a shell interprets `arg`. */
predicate isShellInterpreted(DataFlow::Node arg) { super.isShellInterpreted(arg) }
/** Gets an argument to this execution that specifies the command or an argument to it. */
DataFlow::Node getAnArgument() { result = super.getAnArgument() }
}
/** Provides a class for modeling new operating system command APIs. */
module SystemCommandExecution {
/**
* A data flow node that executes an operating system command, for instance by spawning a new
* process.
*
* Extend this class to model new APIs. If you want to refine existing API models,
* extend `SystemCommandExecution` instead.
*/
abstract class Range extends DataFlow::Node {
/** Gets an argument to this execution that specifies the command or an argument to it. */
abstract DataFlow::Node getAnArgument();
/** Holds if a shell interprets `arg`. */
predicate isShellInterpreted(DataFlow::Node arg) { none() }
}
}
/**
* A data-flow node that dynamically executes Ruby code.
*
* Extend this class to refine existing API models. If you want to model new APIs,
* extend `CodeExecution::Range` instead.
*/
class CodeExecution extends DataFlow::Node instanceof CodeExecution::Range {
/** Gets the argument that specifies the code to be executed. */
DataFlow::Node getCode() { result = super.getCode() }
}
/** Provides a class for modeling new dynamic code execution APIs. */
module CodeExecution {
/**
* A data-flow node that dynamically executes Ruby code.
*
* Extend this class to model new APIs. If you want to refine existing API models,
* extend `CodeExecution` instead.
*/
abstract class Range extends DataFlow::Node {
/** Gets the argument that specifies the code to be executed. */
abstract DataFlow::Node getCode();
}
}

View File

@@ -0,0 +1,7 @@
/**
* Provides classes for performing local (intra-procedural) and
* global (inter-procedural) data flow analyses.
*/
module DataFlow {
import codeql.ruby.dataflow.internal.DataFlowImpl
}

View File

@@ -0,0 +1,52 @@
private import codeql.Locations
/** A diagnostic emitted during extraction, such as a parse error */
class Diagnostic extends @diagnostic {
int severity;
string tag;
string message;
string fullMessage;
Location location;
Diagnostic() { diagnostics(this, severity, tag, message, fullMessage, location) }
/**
* Gets the numerical severity level associated with this diagnostic.
*/
int getSeverity() { result = severity }
/** Gets a string representation of the severity of this diagnostic. */
string getSeverityText() {
severity = 10 and result = "Debug"
or
severity = 20 and result = "Info"
or
severity = 30 and result = "Warning"
or
severity = 40 and result = "Error"
}
/** Gets the error code associated with this diagnostic, e.g. parse_error. */
string getTag() { result = tag }
/**
* Gets the error message text associated with this diagnostic.
*/
string getMessage() { result = message }
/**
* Gets the full error message text associated with this diagnostic.
*/
string getFullMessage() { result = fullMessage }
/** Gets the source location of this diagnostic. */
Location getLocation() { result = location }
/** Gets a textual representation of this diagnostic. */
string toString() { result = this.getMessage() }
}
/** A diagnostic relating to a particular error in extracting a file. */
class ExtractionError extends Diagnostic, @diagnostic_error {
ExtractionError() { this.getTag() = "parse_error" }
}

View File

@@ -0,0 +1,10 @@
/**
* Helper file that imports all framework modeling.
*/
private import codeql.ruby.frameworks.ActionController
private import codeql.ruby.frameworks.ActiveRecord
private import codeql.ruby.frameworks.ActionView
private import codeql.ruby.frameworks.StandardLibrary
private import codeql.ruby.frameworks.Files
private import codeql.ruby.frameworks.HTTPClients

View File

@@ -0,0 +1,7 @@
/**
* Provides classes for performing local (intra-procedural) and
* global (inter-procedural) taint-tracking analyses.
*/
module TaintTracking {
import codeql.ruby.dataflow.internal.tainttracking1.TaintTrackingImpl
}

View File

@@ -0,0 +1,199 @@
private import codeql.ruby.AST
private import internal.AST
private import internal.Call
private import internal.TreeSitter
private import codeql.ruby.dataflow.internal.DataFlowDispatch
private import codeql.ruby.dataflow.internal.DataFlowImplCommon
/**
* A call.
*/
class Call extends Expr instanceof CallImpl {
override string getAPrimaryQlClass() { result = "Call" }
/**
* Gets the `n`th argument of this method call. In the following example, the
* result for n=0 is the `IntegerLiteral` 0, while for n=1 the result is a
* `Pair` (whose `getKey` returns the `SymbolLiteral` for `bar`, and
* `getValue` returns the `IntegerLiteral` 1). Keyword arguments like this
* can be accessed more naturally using the
* `getKeywordArgument(string keyword)` predicate.
* ```rb
* foo(0, bar: 1)
* yield 0, bar: 1
* ```
*/
final Expr getArgument(int n) { result = super.getArgumentImpl(n) }
/**
* Gets an argument of this method call.
*/
final Expr getAnArgument() { result = this.getArgument(_) }
/**
* Gets the value of the keyword argument whose key is `keyword`, if any. For
* example, the result for `getKeywordArgument("qux")` in the following
* example is the `IntegerLiteral` 123.
* ```rb
* foo :bar "baz", qux: 123
* ```
*/
final Expr getKeywordArgument(string keyword) {
exists(Pair p |
p = this.getAnArgument() and
p.getKey().(SymbolLiteral).getValueText() = keyword and
result = p.getValue()
)
}
/**
* Gets the number of arguments of this method call.
*/
final int getNumberOfArguments() { result = super.getNumberOfArgumentsImpl() }
/** Gets a potential target of this call, if any. */
final Callable getATarget() {
exists(DataFlowCall c | this = c.asCall().getExpr() |
TCfgScope(result) = [viableCallable(c), viableCallableLambda(c, _)]
)
}
override AstNode getAChild(string pred) {
result = Expr.super.getAChild(pred)
or
pred = "getArgument" and result = this.getArgument(_)
}
}
/**
* A method call.
*/
class MethodCall extends Call instanceof MethodCallImpl {
override string getAPrimaryQlClass() { result = "MethodCall" }
/**
* Gets the receiver of this call, if any. For example:
*
* ```rb
* foo.bar
* Baz::qux
* corge()
* ```
*
* The result for the call to `bar` is the `Expr` for `foo`; the result for
* the call to `qux` is the `Expr` for `Baz`; for the call to `corge` there
* is no result.
*/
final Expr getReceiver() { result = super.getReceiverImpl() }
/**
* Gets the name of the method being called. For example, in:
*
* ```rb
* foo.bar x, y
* ```
*
* the result is `"bar"`.
*/
final string getMethodName() { result = super.getMethodNameImpl() }
/**
* Gets the block of this method call, if any.
* ```rb
* foo.each { |x| puts x }
* ```
*/
final Block getBlock() { result = super.getBlockImpl() }
override string toString() { result = "call to " + this.getMethodName() }
override AstNode getAChild(string pred) {
result = Call.super.getAChild(pred)
or
pred = "getReceiver" and result = this.getReceiver()
or
pred = "getBlock" and result = this.getBlock()
}
}
/**
* A call to a setter method.
* ```rb
* self.foo = 10
* a[0] = 10
* ```
*/
class SetterMethodCall extends MethodCall, TMethodCallSynth {
SetterMethodCall() { this = TMethodCallSynth(_, _, _, true, _) }
final override string getAPrimaryQlClass() { result = "SetterMethodCall" }
}
/**
* An element reference; a call to the `[]` method.
* ```rb
* a[0]
* ```
*/
class ElementReference extends MethodCall instanceof ElementReferenceImpl {
final override string getAPrimaryQlClass() { result = "ElementReference" }
final override string toString() { result = "...[...]" }
}
/**
* A call to `yield`.
* ```rb
* yield x, y
* ```
*/
class YieldCall extends Call instanceof YieldCallImpl {
final override string getAPrimaryQlClass() { result = "YieldCall" }
final override string toString() { result = "yield ..." }
}
/**
* A call to `super`.
* ```rb
* class Foo < Bar
* def baz
* super
* end
* end
* ```
*/
class SuperCall extends MethodCall instanceof SuperCallImpl {
final override string getAPrimaryQlClass() { result = "SuperCall" }
}
/**
* A block argument in a method call.
* ```rb
* foo(&block)
* ```
*/
class BlockArgument extends Expr, TBlockArgument {
private Ruby::BlockArgument g;
BlockArgument() { this = TBlockArgument(g) }
final override string getAPrimaryQlClass() { result = "BlockArgument" }
/**
* Gets the underlying expression representing the block. In the following
* example, the result is the `Expr` for `bar`:
* ```rb
* foo(&bar)
* ```
*/
final Expr getValue() { toGenerated(result) = g.getChild() }
final override string toString() { result = "&..." }
final override AstNode getAChild(string pred) {
result = super.getAChild(pred)
or
pred = "getValue" and result = this.getValue()
}
}

View File

@@ -0,0 +1,179 @@
private import codeql.ruby.AST
private import internal.AST
private import internal.Module
private import internal.Variable
private import internal.TreeSitter
/** An access to a constant. */
class ConstantAccess extends Expr, TConstantAccess {
/** Gets the name of the constant being accessed. */
string getName() { none() }
/** Holds if the name of the constant being accessed is `name`. */
final predicate hasName(string name) { this.getName() = name }
/**
* Gets the expression used in the access's scope resolution operation, if
* any. In the following example, the result is the `Call` expression for
* `foo()`.
*
* ```rb
* foo()::MESSAGE
* ```
*
* However, there is no result for the following example, since there is no
* scope resolution operation.
*
* ```rb
* MESSAGE
* ```
*/
Expr getScopeExpr() { none() }
/**
* Holds if the access uses the scope resolution operator to refer to the
* global scope, as in this example:
*
* ```rb
* ::MESSAGE
* ```
*/
predicate hasGlobalScope() { none() }
override string toString() { result = this.getName() }
override AstNode getAChild(string pred) {
result = super.getAChild(pred)
or
pred = "getScopeExpr" and result = this.getScopeExpr()
}
}
private class TokenConstantAccess extends ConstantAccess, TTokenConstantAccess {
private Ruby::Constant g;
TokenConstantAccess() { this = TTokenConstantAccess(g) }
final override string getName() { result = g.getValue() }
}
private class ScopeResolutionConstantAccess extends ConstantAccess, TScopeResolutionConstantAccess {
private Ruby::ScopeResolution g;
private Ruby::Constant constant;
ScopeResolutionConstantAccess() { this = TScopeResolutionConstantAccess(g, constant) }
final override string getName() { result = constant.getValue() }
final override Expr getScopeExpr() { toGenerated(result) = g.getScope() }
final override predicate hasGlobalScope() { not exists(g.getScope()) }
}
private class ConstantReadAccessSynth extends ConstantAccess, TConstantReadAccessSynth {
private string value;
ConstantReadAccessSynth() { this = TConstantReadAccessSynth(_, _, value) }
final override string getName() {
if this.hasGlobalScope() then result = value.suffix(2) else result = value
}
final override Expr getScopeExpr() { synthChild(this, 0, result) }
final override predicate hasGlobalScope() { value.matches("::%") }
}
/**
* A use (read) of a constant.
*
* For example, the right-hand side of the assignment in:
*
* ```rb
* x = Foo
* ```
*
* Or the superclass `Bar` in this example:
*
* ```rb
* class Foo < Bar
* end
* ```
*/
class ConstantReadAccess extends ConstantAccess {
ConstantReadAccess() {
not this instanceof ConstantWriteAccess
or
// `X` in `X ||= 10` is considered both a read and a write
this = any(AssignOperation a).getLeftOperand()
or
this instanceof TConstantReadAccessSynth
}
/**
* Gets the value being read, if any. For example, in
*
* ```rb
* module M
* CONST = "const"
* end
*
* puts M::CONST
* ```
*
* the value being read at `M::CONST` is `"const"`.
*/
Expr getValue() {
not exists(this.getScopeExpr()) and
result = lookupConst(this.getEnclosingModule+().getModule(), this.getName()) and
// For now, we restrict the scope of top-level declarations to their file.
// This may remove some plausible targets, but also removes a lot of
// implausible targets
if result.getEnclosingModule() instanceof Toplevel
then result.getFile() = this.getFile()
else any()
or
this.hasGlobalScope() and
result = lookupConst(TResolved("Object"), this.getName())
or
result = lookupConst(resolveScopeExpr(this.getScopeExpr()), this.getName())
}
final override string getAPrimaryQlClass() { result = "ConstantReadAccess" }
}
/**
* A definition of a constant.
*
* Examples:
*
* ```rb
* Foo = 1 # defines constant Foo as an integer
* M::Foo = 1 # defines constant Foo as an integer in module M
*
* class Bar; end # defines constant Bar as a class
* class M::Bar; end # defines constant Bar as a class in module M
*
* module Baz; end # defines constant Baz as a module
* module M::Baz; end # defines constant Baz as a module in module M
* ```
*/
class ConstantWriteAccess extends ConstantAccess {
ConstantWriteAccess() {
explicitAssignmentNode(toGenerated(this), _) or this instanceof TNamespace
}
override string getAPrimaryQlClass() { result = "ConstantWriteAccess" }
}
/**
* A definition of a constant via assignment. For example, the left-hand
* operand in the following example:
*
* ```rb
* MAX_SIZE = 100
* ```
*/
class ConstantAssignment extends ConstantWriteAccess, LhsExpr {
override string getAPrimaryQlClass() { result = "ConstantAssignment" }
}

View File

@@ -0,0 +1,611 @@
private import codeql.ruby.AST
private import internal.AST
private import internal.TreeSitter
/**
* A control expression that can be any of the following:
* - `case`
* - `if`/`unless` (including expression-modifier variants)
* - ternary-if (`?:`)
* - `while`/`until` (including expression-modifier variants)
* - `for`
*/
class ControlExpr extends Expr, TControlExpr { }
/**
* A conditional expression: `if`/`unless` (including expression-modifier
* variants), and ternary-if (`?:`) expressions.
*/
class ConditionalExpr extends ControlExpr, TConditionalExpr {
/**
* Gets the condition expression. For example, the result is `foo` in the
* following:
* ```rb
* if foo
* bar = 1
* end
* ```
*/
Expr getCondition() { none() }
/**
* Gets the branch of this conditional expression that is taken when the
* condition evaluates to `cond`, if any.
*/
Stmt getBranch(boolean cond) { none() }
override AstNode getAChild(string pred) {
result = super.getAChild(pred)
or
pred = "getCondition" and result = this.getCondition()
or
pred = "getBranch" and result = this.getBranch(_)
}
}
/**
* An `if` or `elsif` expression.
* ```rb
* if x
* a += 1
* elsif y
* a += 2
* end
* ```
*/
class IfExpr extends ConditionalExpr, TIfExpr {
final override string getAPrimaryQlClass() { result = "IfExpr" }
/** Holds if this is an `elsif` expression. */
predicate isElsif() { none() }
/** Gets the 'then' branch of this `if`/`elsif` expression. */
Stmt getThen() { none() }
/**
* Gets the `elsif`/`else` branch of this `if`/`elsif` expression, if any. In
* the following example, the result is a `StmtSequence` containing `b`.
* ```rb
* if foo
* a
* else
* b
* end
* ```
* But there is no result for the following:
* ```rb
* if foo
* a
* end
* ```
* There can be at most one result, since `elsif` branches nest. In the
* following example, `ifExpr.getElse()` returns an `ElsifExpr`, and the
* `else` branch is nested inside that. To get the `StmtSequence` for the
* `else` branch, i.e. the one containing `c`, use
* `getElse().(ElsifExpr).getElse()`.
* ```rb
* if foo
* a
* elsif bar
* b
* else
* c
* end
* ```
*/
Stmt getElse() { none() }
final override Stmt getBranch(boolean cond) {
cond = true and result = this.getThen()
or
cond = false and result = this.getElse()
}
override AstNode getAChild(string pred) {
result = super.getAChild(pred)
or
pred = "getThen" and result = this.getThen()
or
pred = "getElse" and result = this.getElse()
}
}
private class If extends IfExpr, TIf {
private Ruby::If g;
If() { this = TIf(g) }
final override Expr getCondition() { toGenerated(result) = g.getCondition() }
final override Stmt getThen() { toGenerated(result) = g.getConsequence() }
final override Stmt getElse() { toGenerated(result) = g.getAlternative() }
final override string toString() { result = "if ..." }
}
private class Elsif extends IfExpr, TElsif {
private Ruby::Elsif g;
Elsif() { this = TElsif(g) }
final override predicate isElsif() { any() }
final override Expr getCondition() { toGenerated(result) = g.getCondition() }
final override Stmt getThen() { toGenerated(result) = g.getConsequence() }
final override Stmt getElse() { toGenerated(result) = g.getAlternative() }
final override string toString() { result = "elsif ..." }
}
/**
* An `unless` expression.
* ```rb
* unless x == 0
* y /= x
* end
* ```
*/
class UnlessExpr extends ConditionalExpr, TUnlessExpr {
private Ruby::Unless g;
UnlessExpr() { this = TUnlessExpr(g) }
final override string getAPrimaryQlClass() { result = "UnlessExpr" }
final override Expr getCondition() { toGenerated(result) = g.getCondition() }
/**
* Gets the 'then' branch of this `unless` expression. In the following
* example, the result is the `StmtSequence` containing `foo`.
* ```rb
* unless a == b then
* foo
* else
* bar
* end
* ```
*/
final Stmt getThen() { toGenerated(result) = g.getConsequence() }
/**
* Gets the 'else' branch of this `unless` expression. In the following
* example, the result is the `StmtSequence` containing `bar`.
* ```rb
* unless a == b then
* foo
* else
* bar
* end
* ```
*/
final Stmt getElse() { toGenerated(result) = g.getAlternative() }
final override Expr getBranch(boolean cond) {
cond = false and result = getThen()
or
cond = true and result = getElse()
}
final override string toString() { result = "unless ..." }
override AstNode getAChild(string pred) {
result = ConditionalExpr.super.getAChild(pred)
or
pred = "getThen" and result = this.getThen()
or
pred = "getElse" and result = this.getElse()
}
}
/**
* An expression modified using `if`.
* ```rb
* foo if bar
* ```
*/
class IfModifierExpr extends ConditionalExpr, TIfModifierExpr {
private Ruby::IfModifier g;
IfModifierExpr() { this = TIfModifierExpr(g) }
final override string getAPrimaryQlClass() { result = "IfModifierExpr" }
final override Expr getCondition() { toGenerated(result) = g.getCondition() }
final override Stmt getBranch(boolean cond) { cond = true and result = this.getBody() }
/**
* Gets the statement that is conditionally evaluated. In the following
* example, the result is the `Expr` for `foo`.
* ```rb
* foo if bar
* ```
*/
final Stmt getBody() { toGenerated(result) = g.getBody() }
final override string toString() { result = "... if ..." }
override AstNode getAChild(string pred) {
result = ConditionalExpr.super.getAChild(pred)
or
pred = "getBody" and result = this.getBody()
}
}
/**
* An expression modified using `unless`.
* ```rb
* y /= x unless x == 0
* ```
*/
class UnlessModifierExpr extends ConditionalExpr, TUnlessModifierExpr {
private Ruby::UnlessModifier g;
UnlessModifierExpr() { this = TUnlessModifierExpr(g) }
final override string getAPrimaryQlClass() { result = "UnlessModifierExpr" }
final override Expr getCondition() { toGenerated(result) = g.getCondition() }
final override Stmt getBranch(boolean cond) { cond = false and result = this.getBody() }
/**
* Gets the statement that is conditionally evaluated. In the following
* example, the result is the `Expr` for `foo`.
* ```rb
* foo unless bar
* ```
*/
final Stmt getBody() { toGenerated(result) = g.getBody() }
final override string toString() { result = "... unless ..." }
override AstNode getAChild(string pred) {
result = ConditionalExpr.super.getAChild(pred)
or
pred = "getBody" and result = this.getBody()
}
}
/**
* A conditional expression using the ternary (`?:`) operator.
* ```rb
* (a > b) ? a : b
* ```
*/
class TernaryIfExpr extends ConditionalExpr, TTernaryIfExpr {
private Ruby::Conditional g;
TernaryIfExpr() { this = TTernaryIfExpr(g) }
final override string getAPrimaryQlClass() { result = "TernaryIfExpr" }
final override Expr getCondition() { toGenerated(result) = g.getCondition() }
/** Gets the 'then' branch of this ternary if expression. */
final Stmt getThen() { toGenerated(result) = g.getConsequence() }
/** Gets the 'else' branch of this ternary if expression. */
final Stmt getElse() { toGenerated(result) = g.getAlternative() }
final override Stmt getBranch(boolean cond) {
cond = true and result = getThen()
or
cond = false and result = getElse()
}
final override string toString() { result = "... ? ... : ..." }
override AstNode getAChild(string pred) {
result = ConditionalExpr.super.getAChild(pred)
or
pred = "getThen" and result = this.getThen()
or
pred = "getElse" and result = this.getElse()
}
}
class CaseExpr extends ControlExpr, TCaseExpr {
private Ruby::Case g;
CaseExpr() { this = TCaseExpr(g) }
final override string getAPrimaryQlClass() { result = "CaseExpr" }
/**
* Gets the expression being compared, if any. For example, `foo` in the following example.
* ```rb
* case foo
* when 0
* puts 'zero'
* when 1
* puts 'one'
* end
* ```
* There is no result for the following example:
* ```rb
* case
* when a then 0
* when b then 1
* else 2
* end
* ```
*/
final Expr getValue() { toGenerated(result) = g.getValue() }
/**
* Gets the `n`th branch of this case expression, either a `WhenExpr` or a
* `StmtSequence`.
*/
final Expr getBranch(int n) { toGenerated(result) = g.getChild(n) }
/**
* Gets a branch of this case expression, either a `WhenExpr` or an
* `ElseExpr`.
*/
final Expr getABranch() { result = this.getBranch(_) }
/** Gets a `when` branch of this case expression. */
final WhenExpr getAWhenBranch() { result = getABranch() }
/** Gets the `else` branch of this case expression, if any. */
final StmtSequence getElseBranch() { result = getABranch() }
/**
* Gets the number of branches of this case expression.
*/
final int getNumberOfBranches() { result = count(this.getBranch(_)) }
final override string toString() { result = "case ..." }
override AstNode getAChild(string pred) {
result = super.getAChild(pred)
or
pred = "getValue" and result = this.getValue()
or
pred = "getBranch" and result = this.getBranch(_)
}
}
/**
* A `when` branch of a `case` expression.
* ```rb
* case
* when a > b then x
* end
* ```
*/
class WhenExpr extends Expr, TWhenExpr {
private Ruby::When g;
WhenExpr() { this = TWhenExpr(g) }
final override string getAPrimaryQlClass() { result = "WhenExpr" }
/** Gets the body of this case-when expression. */
final Stmt getBody() { toGenerated(result) = g.getBody() }
/**
* Gets the `n`th pattern (or condition) in this case-when expression. In the
* following example, the 0th pattern is `x`, the 1st pattern is `y`, and the
* 2nd pattern is `z`.
* ```rb
* case foo
* when x, y, z
* puts 'x/y/z'
* end
* ```
*/
final Expr getPattern(int n) { toGenerated(result) = g.getPattern(n).getChild() }
/**
* Gets a pattern (or condition) in this case-when expression.
*/
final Expr getAPattern() { result = this.getPattern(_) }
/**
* Gets the number of patterns in this case-when expression.
*/
final int getNumberOfPatterns() { result = count(this.getPattern(_)) }
final override string toString() { result = "when ..." }
override AstNode getAChild(string pred) {
result = super.getAChild(pred)
or
pred = "getBody" and result = this.getBody()
or
pred = "getPattern" and result = this.getPattern(_)
}
}
/**
* A loop. That is, a `for` loop, a `while` or `until` loop, or their
* expression-modifier variants.
*/
class Loop extends ControlExpr, TLoop {
/** Gets the body of this loop. */
Stmt getBody() { none() }
override AstNode getAChild(string pred) {
result = super.getAChild(pred)
or
pred = "getBody" and result = this.getBody()
}
}
/**
* A loop using a condition expression. That is, a `while` or `until` loop, or
* their expression-modifier variants.
*/
class ConditionalLoop extends Loop, TConditionalLoop {
/** Gets the condition expression of this loop. */
Expr getCondition() { none() }
override AstNode getAChild(string pred) {
result = Loop.super.getAChild(pred)
or
pred = "getCondition" and result = this.getCondition()
}
/** Holds if the loop body is entered when the condition is `condValue`. */
predicate entersLoopWhenConditionIs(boolean condValue) { none() }
}
/**
* A `while` loop.
* ```rb
* while a < b
* p a
* a += 2
* end
* ```
*/
class WhileExpr extends ConditionalLoop, TWhileExpr {
private Ruby::While g;
WhileExpr() { this = TWhileExpr(g) }
final override string getAPrimaryQlClass() { result = "WhileExpr" }
/** Gets the body of this `while` loop. */
final override Stmt getBody() { toGenerated(result) = g.getBody() }
final override Expr getCondition() { toGenerated(result) = g.getCondition() }
/**
* Holds if the loop body is entered when the condition is `condValue`. For
* `while` loops, this holds when `condValue` is true.
*/
final override predicate entersLoopWhenConditionIs(boolean condValue) { condValue = true }
final override string toString() { result = "while ..." }
}
/**
* An `until` loop.
* ```rb
* until a >= b
* p a
* a += 1
* end
* ```
*/
class UntilExpr extends ConditionalLoop, TUntilExpr {
private Ruby::Until g;
UntilExpr() { this = TUntilExpr(g) }
final override string getAPrimaryQlClass() { result = "UntilExpr" }
/** Gets the body of this `until` loop. */
final override Stmt getBody() { toGenerated(result) = g.getBody() }
final override Expr getCondition() { toGenerated(result) = g.getCondition() }
/**
* Holds if the loop body is entered when the condition is `condValue`. For
* `until` loops, this holds when `condValue` is false.
*/
final override predicate entersLoopWhenConditionIs(boolean condValue) { condValue = false }
final override string toString() { result = "until ..." }
}
/**
* An expression looped using the `while` modifier.
* ```rb
* foo while bar
* ```
*/
class WhileModifierExpr extends ConditionalLoop, TWhileModifierExpr {
private Ruby::WhileModifier g;
WhileModifierExpr() { this = TWhileModifierExpr(g) }
final override Stmt getBody() { toGenerated(result) = g.getBody() }
final override Expr getCondition() { toGenerated(result) = g.getCondition() }
/**
* Holds if the loop body is entered when the condition is `condValue`. For
* `while`-modifier loops, this holds when `condValue` is true.
*/
final override predicate entersLoopWhenConditionIs(boolean condValue) { condValue = true }
final override string getAPrimaryQlClass() { result = "WhileModifierExpr" }
final override string toString() { result = "... while ..." }
}
/**
* An expression looped using the `until` modifier.
* ```rb
* foo until bar
* ```
*/
class UntilModifierExpr extends ConditionalLoop, TUntilModifierExpr {
private Ruby::UntilModifier g;
UntilModifierExpr() { this = TUntilModifierExpr(g) }
final override Stmt getBody() { toGenerated(result) = g.getBody() }
final override Expr getCondition() { toGenerated(result) = g.getCondition() }
/**
* Holds if the loop body is entered when the condition is `condValue`. For
* `until`-modifier loops, this holds when `condValue` is false.
*/
final override predicate entersLoopWhenConditionIs(boolean condValue) { condValue = false }
final override string getAPrimaryQlClass() { result = "UntilModifierExpr" }
final override string toString() { result = "... until ..." }
}
/**
* A `for` loop.
* ```rb
* for val in 1..n
* sum += val
* end
* ```
*/
class ForExpr extends Loop, TForExpr {
private Ruby::For g;
ForExpr() { this = TForExpr(g) }
final override string getAPrimaryQlClass() { result = "ForExpr" }
/** Gets the body of this `for` loop. */
final override Stmt getBody() { toGenerated(result) = g.getBody() }
/** Gets the pattern representing the iteration argument. */
final Pattern getPattern() { toGenerated(result) = g.getPattern() }
/**
* Gets the value being iterated over. In the following example, the result
* is the expression `1..10`:
* ```rb
* for n in 1..10 do
* puts n
* end
* ```
*/
final Expr getValue() { toGenerated(result) = g.getValue().getChild() }
final override string toString() { result = "for ... in ..." }
override AstNode getAChild(string pred) {
result = Loop.super.getAChild(pred)
or
pred = "getPattern" and result = this.getPattern()
or
pred = "getValue" and result = this.getValue()
}
}

View File

@@ -0,0 +1,267 @@
private import codeql.Locations
private import codeql.ruby.AST
private import internal.Erb
private import internal.TreeSitter
/**
* A node in the ERB abstract syntax tree. This class is the base class for all
* ERB elements.
*/
class ErbAstNode extends TAstNode {
/** Gets a textual representation of this node. */
cached
string toString() { none() }
/** Gets the location of this node. */
Location getLocation() { result = getLocation(this) }
/**
* Gets the name of a primary CodeQL class to which this node belongs.
*
* This predicate always has a result. If no primary class can be
* determined, the result is `"???"`. If multiple primary classes match,
* this predicate can have multiple results.
*/
string getAPrimaryQlClass() { result = "???" }
}
/**
* An ERB template. This can contain multiple directives to be executed when
* the template is compiled.
*/
class ErbTemplate extends TTemplate, ErbAstNode {
private Erb::Template g;
ErbTemplate() { this = TTemplate(g) }
override string toString() { result = "erb template" }
final override string getAPrimaryQlClass() { result = "ErbTemplate" }
ErbAstNode getAChildNode() { toGenerated(result) = g.getChild(_) }
}
// Truncate the token string value to 32 char max
bindingset[val]
private string displayToken(string val) {
val.length() <= 32 and result = val
or
val.length() > 32 and result = val.prefix(29) + "..."
}
/**
* An ERB token. This could be embedded code, a comment, or arbitrary text.
*/
class ErbToken extends TTokenNode, ErbAstNode {
override string toString() { result = displayToken(this.getValue()) }
/** Gets the string value of this token. */
string getValue() { exists(Erb::Token g | this = fromGenerated(g) | result = g.getValue()) }
override string getAPrimaryQlClass() { result = "ErbToken" }
}
/**
* An ERB token appearing within a comment directive.
*/
class ErbComment extends ErbToken {
private Erb::Comment g;
ErbComment() { this = TComment(g) }
override string getValue() { result = g.getValue() }
final override string getAPrimaryQlClass() { result = "ErbComment" }
}
/**
* An ERB token appearing within a code directive. This will typically be
* interpreted as Ruby code or a GraphQL query, depending on context.
*/
class ErbCode extends ErbToken {
private Erb::Code g;
ErbCode() { this = TCode(g) }
override string getValue() { result = g.getValue() }
final override string getAPrimaryQlClass() { result = "ErbCode" }
}
bindingset[line, col]
private predicate locationIncludesPosition(Location loc, int line, int col) {
// position between start and end line, exclusive
line > loc.getStartLine() and
line < loc.getEndLine()
or
// position on start line, multi line location
line = loc.getStartLine() and
not loc.getStartLine() = loc.getEndLine() and
col >= loc.getStartColumn()
or
// position on end line, multi line location
line = loc.getEndLine() and
not loc.getStartLine() = loc.getEndLine() and
col <= loc.getEndColumn()
or
// single line location, position between start and end column
line = loc.getStartLine() and
loc.getStartLine() = loc.getEndLine() and
col >= loc.getStartColumn() and
col <= loc.getEndColumn()
}
/**
* A directive in an ERB template.
*/
class ErbDirective extends TDirectiveNode, ErbAstNode {
private predicate containsStartOf(Location loc) {
loc.getFile() = this.getLocation().getFile() and
locationIncludesPosition(this.getLocation(), loc.getStartLine(), loc.getStartColumn())
}
private predicate containsStmtStart(Stmt s) {
this.containsStartOf(s.getLocation()) and
// `Toplevel` statements are not contained within individual directives,
// though their start location may appear within a directive location
not s instanceof Toplevel
}
/**
* Gets a statement that starts in directive that is not a child of any other
* statement starting in this directive.
*/
Stmt getAChildStmt() {
this.containsStmtStart(result) and
not this.containsStmtStart(result.getParent())
}
/**
* Gets the last child statement in this directive.
* See `getAChildStmt` for more details.
*/
Stmt getTerminalStmt() {
result = this.getAChildStmt() and
forall(Stmt s | s = this.getAChildStmt() and not s = result |
s.getLocation().strictlyBefore(result.getLocation())
)
}
/** Gets the child token of this directive. */
ErbToken getToken() {
exists(Erb::Directive g | this = fromGenerated(g) | toGenerated(result) = g.getChild())
}
override string toString() { result = "erb directive" }
override string getAPrimaryQlClass() { result = "ErbDirective" }
}
/**
* A comment directive in an ERB template.
* ```erb
* <%#= 2 + 2 %>
* <%# for x in xs do %>
* ```
*/
class ErbCommentDirective extends ErbDirective {
private Erb::CommentDirective g;
ErbCommentDirective() { this = TCommentDirective(g) }
override ErbComment getToken() { toGenerated(result) = g.getChild() }
final override string toString() { result = "<%#" + this.getToken().toString() + "%>" }
final override string getAPrimaryQlClass() { result = "ErbCommentDirective" }
}
/**
* A GraphQL directive in an ERB template.
* ```erb
* <%graphql
* fragment Foo on Bar {
* some {
* queryText
* moreProperties
* }
* }
* %>
* ```
*/
class ErbGraphqlDirective extends ErbDirective {
private Erb::GraphqlDirective g;
ErbGraphqlDirective() { this = TGraphqlDirective(g) }
override ErbCode getToken() { toGenerated(result) = g.getChild() }
final override string toString() { result = "<%graphql" + this.getToken().toString() + "%>" }
final override string getAPrimaryQlClass() { result = "ErbGraphqlDirective" }
}
/**
* An output directive in an ERB template.
* ```erb
* <%=
* fragment Foo on Bar {
* some {
* queryText
* moreProperties
* }
* }
* %>
* ```
*/
class ErbOutputDirective extends ErbDirective {
private Erb::OutputDirective g;
ErbOutputDirective() { this = TOutputDirective(g) }
override ErbCode getToken() { toGenerated(result) = g.getChild() }
final override string toString() { result = "<%=" + this.getToken().toString() + "%>" }
final override string getAPrimaryQlClass() { result = "ErbOutputDirective" }
}
/**
* An execution directive in an ERB template.
* This code will be executed as Ruby, but not rendered.
* ```erb
* <% books = author.books
* for book in books do %>
* ```
*/
class ErbExecutionDirective extends ErbDirective {
private Erb::Directive g;
ErbExecutionDirective() { this = TDirective(g) }
final override string toString() { result = "<%" + this.getToken().toString() + "%>" }
final override string getAPrimaryQlClass() { result = "ErbExecutionDirective" }
}
/**
* A `File` containing an Embedded Ruby template.
* This is typically a file containing snippets of Ruby code that can be
* evaluated to create a compiled version of the file.
*/
class ErbFile extends File {
private ErbTemplate template;
ErbFile() { this = template.getLocation().getFile() }
/**
* Holds if the file represents a partial to be rendered in the context of
* another template.
*/
predicate isPartial() { this.getStem().charAt(0) = "_" }
/**
* Gets the erb template contained within this file.
*/
ErbTemplate getTemplate() { result = template }
}

View File

@@ -0,0 +1,450 @@
private import codeql.ruby.AST
private import internal.AST
private import internal.TreeSitter
/**
* An expression.
*
* This is the root QL class for all expressions.
*/
class Expr extends Stmt, TExpr { }
/**
* A reference to the current object. For example:
* - `self == other`
* - `self.method_name`
* - `def self.method_name ... end`
*
* This also includes implicit references to the current object in method
* calls. For example, the method call `foo(123)` has an implicit `self`
* receiver, and is equivalent to the explicit `self.foo(123)`.
*/
class Self extends Expr, TSelf {
final override string getAPrimaryQlClass() { result = "Self" }
final override string toString() { result = "self" }
}
/**
* A sequence of expressions in the right-hand side of an assignment or
* a `return`, `break` or `next` statement.
* ```rb
* x = 1, *items, 3, *more
* return 1, 2
* next *list
* break **map
* return 1, 2, *items, k: 5, **map
* ```
*/
class ArgumentList extends Expr, TArgumentList {
private Ruby::AstNode g;
ArgumentList() { this = TArgumentList(g) }
/** Gets the `i`th element in this argument list. */
Expr getElement(int i) {
toGenerated(result) in [
g.(Ruby::ArgumentList).getChild(i), g.(Ruby::RightAssignmentList).getChild(i)
]
}
final override string getAPrimaryQlClass() { result = "ArgumentList" }
final override string toString() { result = "..., ..." }
final override AstNode getAChild(string pred) {
result = super.getAChild(pred)
or
pred = "getElement" and result = this.getElement(_)
}
}
/** A sequence of expressions. */
class StmtSequence extends Expr, TStmtSequence {
override string getAPrimaryQlClass() { result = "StmtSequence" }
/** Gets the `n`th statement in this sequence. */
Stmt getStmt(int n) { none() }
/** Gets a statement in this sequence. */
final Stmt getAStmt() { result = this.getStmt(_) }
/** Gets the last statement in this sequence, if any. */
final Stmt getLastStmt() { result = this.getStmt(this.getNumberOfStatements() - 1) }
/** Gets the number of statements in this sequence. */
final int getNumberOfStatements() { result = count(this.getAStmt()) }
/** Holds if this sequence has no statements. */
final predicate isEmpty() { this.getNumberOfStatements() = 0 }
override AstNode getAChild(string pred) {
result = super.getAChild(pred)
or
pred = "getStmt" and result = this.getStmt(_)
}
}
private class StmtSequenceSynth extends StmtSequence, TStmtSequenceSynth {
final override Stmt getStmt(int n) { synthChild(this, n, result) }
final override string toString() { result = "..." }
}
private class Then extends StmtSequence, TThen {
private Ruby::Then g;
Then() { this = TThen(g) }
override Stmt getStmt(int n) { toGenerated(result) = g.getChild(n) }
final override string toString() { result = "then ..." }
}
private class Else extends StmtSequence, TElse {
private Ruby::Else g;
Else() { this = TElse(g) }
override Stmt getStmt(int n) { toGenerated(result) = g.getChild(n) }
final override string toString() { result = "else ..." }
}
private class Do extends StmtSequence, TDo {
private Ruby::Do g;
Do() { this = TDo(g) }
override Stmt getStmt(int n) { toGenerated(result) = g.getChild(n) }
final override string toString() { result = "do ..." }
}
private class Ensure extends StmtSequence, TEnsure {
private Ruby::Ensure g;
Ensure() { this = TEnsure(g) }
override Stmt getStmt(int n) { toGenerated(result) = g.getChild(n) }
final override string toString() { result = "ensure ..." }
}
/**
* A sequence of statements representing the body of a method, class, module,
* or do-block. That is, any body that may also include rescue/ensure/else
* statements.
*/
class BodyStmt extends StmtSequence, TBodyStmt {
// Not defined by dispatch, as it should not be exposed
private Ruby::AstNode getChild(int i) {
result = any(Ruby::Method g | this = TMethod(g)).getChild(i)
or
result = any(Ruby::SingletonMethod g | this = TSingletonMethod(g)).getChild(i)
or
exists(Ruby::Lambda g | this = TLambda(g) |
result = g.getBody().(Ruby::DoBlock).getChild(i) or
result = g.getBody().(Ruby::Block).getChild(i)
)
or
result = any(Ruby::DoBlock g | this = TDoBlock(g)).getChild(i)
or
result = any(Ruby::Program g | this = TToplevel(g)).getChild(i) and
not result instanceof Ruby::BeginBlock
or
result = any(Ruby::Class g | this = TClassDeclaration(g)).getChild(i)
or
result = any(Ruby::SingletonClass g | this = TSingletonClass(g)).getChild(i)
or
result = any(Ruby::Module g | this = TModuleDeclaration(g)).getChild(i)
or
result = any(Ruby::Begin g | this = TBeginExpr(g)).getChild(i)
}
final override Stmt getStmt(int n) {
result =
rank[n + 1](AstNode node, int i |
toGenerated(node) = this.getChild(i) and
not node instanceof Else and
not node instanceof RescueClause and
not node instanceof Ensure
|
node order by i
)
}
/** Gets the `n`th rescue clause in this block. */
final RescueClause getRescue(int n) {
result =
rank[n + 1](RescueClause node, int i | toGenerated(node) = getChild(i) | node order by i)
}
/** Gets a rescue clause in this block. */
final RescueClause getARescue() { result = this.getRescue(_) }
/** Gets the `else` clause in this block, if any. */
final StmtSequence getElse() { result = unique(Else s | toGenerated(s) = getChild(_)) }
/** Gets the `ensure` clause in this block, if any. */
final StmtSequence getEnsure() { result = unique(Ensure s | toGenerated(s) = getChild(_)) }
final predicate hasEnsure() { exists(this.getEnsure()) }
override AstNode getAChild(string pred) {
result = StmtSequence.super.getAChild(pred)
or
pred = "getRescue" and result = this.getRescue(_)
or
pred = "getElse" and result = this.getElse()
or
pred = "getEnsure" and result = this.getEnsure()
}
}
/**
* A parenthesized expression sequence, typically containing a single expression:
* ```rb
* (x + 1)
* ```
* However, they can also contain multiple expressions (the value of the parenthesized
* expression is the last expression):
* ```rb
* (foo; bar)
* ```
* or even an empty sequence (value is `nil`):
* ```rb
* ()
* ```
*/
class ParenthesizedExpr extends StmtSequence, TParenthesizedExpr {
private Ruby::ParenthesizedStatements g;
ParenthesizedExpr() { this = TParenthesizedExpr(g) }
final override Stmt getStmt(int n) { toGenerated(result) = g.getChild(n) }
final override string getAPrimaryQlClass() { result = "ParenthesizedExpr" }
final override string toString() { result = "( ... )" }
}
/**
* A pair expression. For example, in a hash:
* ```rb
* { foo: bar }
* ```
* Or a keyword argument:
* ```rb
* baz(qux: 1)
* ```
*/
class Pair extends Expr, TPair {
private Ruby::Pair g;
Pair() { this = TPair(g) }
final override string getAPrimaryQlClass() { result = "Pair" }
/**
* Gets the key expression of this pair. For example, the `SymbolLiteral`
* representing the keyword `foo` in the following example:
* ```rb
* bar(foo: 123)
* ```
* Or the `StringLiteral` for `'foo'` in the following hash pair:
* ```rb
* { 'foo' => 123 }
* ```
*/
final Expr getKey() { toGenerated(result) = g.getKey() }
/**
* Gets the value expression of this pair. For example, the `InteralLiteral`
* 123 in the following hash pair:
* ```rb
* { 'foo' => 123 }
* ```
*/
final Expr getValue() { toGenerated(result) = g.getValue() }
final override string toString() { result = "Pair" }
override AstNode getAChild(string pred) {
result = super.getAChild(pred)
or
pred = "getKey" and result = this.getKey()
or
pred = "getValue" and result = this.getValue()
}
}
/**
* A rescue clause. For example:
* ```rb
* begin
* write_file
* rescue StandardError => msg
* puts msg
* end
*/
class RescueClause extends Expr, TRescueClause {
private Ruby::Rescue g;
RescueClause() { this = TRescueClause(g) }
final override string getAPrimaryQlClass() { result = "RescueClause" }
/**
* Gets the `n`th exception to match, if any. For example `FirstError` or `SecondError` in:
* ```rb
* begin
* do_something
* rescue FirstError, SecondError => e
* handle_error(e)
* end
* ```
*/
final Expr getException(int n) { toGenerated(result) = g.getExceptions().getChild(n) }
/**
* Gets an exception to match, if any. For example `FirstError` or `SecondError` in:
* ```rb
* begin
* do_something
* rescue FirstError, SecondError => e
* handle_error(e)
* end
* ```
*/
final Expr getAnException() { result = this.getException(_) }
/**
* Gets the variable to which to assign the matched exception, if any.
* For example `err` in:
* ```rb
* begin
* do_something
* rescue StandardError => err
* handle_error(err)
* end
* ```
*/
final LhsExpr getVariableExpr() { toGenerated(result) = g.getVariable().getChild() }
/**
* Gets the exception handler body.
*/
final StmtSequence getBody() { toGenerated(result) = g.getBody() }
final override string toString() { result = "rescue ..." }
override AstNode getAChild(string pred) {
result = super.getAChild(pred)
or
pred = "getException" and result = this.getException(_)
or
pred = "getVariableExpr" and result = this.getVariableExpr()
or
pred = "getBody" and result = this.getBody()
}
}
/**
* An expression with a `rescue` modifier. For example:
* ```rb
* contents = read_file rescue ""
* ```
*/
class RescueModifierExpr extends Expr, TRescueModifierExpr {
private Ruby::RescueModifier g;
RescueModifierExpr() { this = TRescueModifierExpr(g) }
final override string getAPrimaryQlClass() { result = "RescueModifierExpr" }
/**
* Gets the body of this `RescueModifierExpr`.
* ```rb
* body rescue handler
* ```
*/
final Stmt getBody() { toGenerated(result) = g.getBody() }
/**
* Gets the exception handler of this `RescueModifierExpr`.
* ```rb
* body rescue handler
* ```
*/
final Stmt getHandler() { toGenerated(result) = g.getHandler() }
final override string toString() { result = "... rescue ..." }
override AstNode getAChild(string pred) {
result = super.getAChild(pred)
or
pred = "getBody" and result = this.getBody()
or
pred = "getHandler" and result = this.getHandler()
}
}
/**
* A concatenation of string literals.
*
* ```rb
* "foo" "bar" "baz"
* ```
*/
class StringConcatenation extends Expr, TStringConcatenation {
private Ruby::ChainedString g;
StringConcatenation() { this = TStringConcatenation(g) }
final override string getAPrimaryQlClass() { result = "StringConcatenation" }
/** Gets the `n`th string literal in this concatenation. */
final StringLiteral getString(int n) { toGenerated(result) = g.getChild(n) }
/** Gets a string literal in this concatenation. */
final StringLiteral getAString() { result = this.getString(_) }
/** Gets the number of string literals in this concatenation. */
final int getNumberOfStrings() { result = count(this.getString(_)) }
/**
* Gets the result of concatenating all the string literals, if and only if
* they do not contain any interpolations.
*
* For the following example, the result is `"foobar"`:
*
* ```rb
* "foo" 'bar'
* ```
*
* And for the following example, where one of the string literals includes
* an interpolation, there is no result:
*
* ```rb
* "foo" "bar#{ n }"
* ```
*/
final string getConcatenatedValueText() {
forall(StringLiteral c | c = this.getString(_) | exists(c.getValueText())) and
result =
concat(string valueText, int i |
valueText = this.getString(i).getValueText()
|
valueText order by i
)
}
final override string toString() { result = "\"...\" \"...\"" }
override AstNode getAChild(string pred) {
result = super.getAChild(pred)
or
pred = "getString" and result = this.getString(_)
}
}

View File

@@ -0,0 +1,885 @@
private import codeql.ruby.AST
private import codeql.ruby.regexp.RegExpTreeView as RETV
private import internal.AST
private import internal.Scope
private import internal.TreeSitter
/**
* A literal.
*
* This is the QL root class for all literals.
*/
class Literal extends Expr, TLiteral {
/**
* Gets the source text for this literal, if this is a simple literal.
*
* For complex literals, such as arrays, hashes, and strings with
* interpolations, this predicate has no result.
*/
string getValueText() { none() }
}
/**
* A numeric literal, i.e. an integer, floating-point, rational, or complex
* value.
*
* ```rb
* 123
* 0xff
* 3.14159
* 1.0E2
* 7r
* 1i
* ```
*/
class NumericLiteral extends Literal, TNumericLiteral { }
/**
* An integer literal.
*
* ```rb
* 123
* 0xff
* ```
*/
class IntegerLiteral extends NumericLiteral, TIntegerLiteral {
/** Gets the numerical value of this integer literal. */
int getValue() { none() }
final override string toString() { result = this.getValueText() }
final override string getAPrimaryQlClass() { result = "IntegerLiteral" }
}
private class IntegerLiteralReal extends IntegerLiteral, TIntegerLiteralReal {
private Ruby::Integer g;
IntegerLiteralReal() { this = TIntegerLiteralReal(g) }
final override string getValueText() { result = g.getValue() }
final override int getValue() {
exists(string s, string values, string str |
s = this.getValueText().toLowerCase() and
(
s.matches("0b%") and
values = "01" and
str = s.suffix(2)
or
s.matches("0x%") and
values = "0123456789abcdef" and
str = s.suffix(2)
or
s.charAt(0) = "0" and
not s.charAt(1) = ["b", "x", "o"] and
values = "01234567" and
str = s.suffix(1)
or
s.matches("0o%") and
values = "01234567" and
str = s.suffix(2)
or
s.charAt(0) != "0" and values = "0123456789" and str = s
)
|
result =
sum(int index, string c, int v, int exp |
c = str.replaceAll("_", "").charAt(index) and
v = values.indexOf(c.toLowerCase()) and
exp = str.replaceAll("_", "").length() - index - 1
|
v * values.length().pow(exp)
)
)
}
}
private class IntegerLiteralSynth extends IntegerLiteral, TIntegerLiteralSynth {
private int value;
IntegerLiteralSynth() { this = TIntegerLiteralSynth(_, _, value) }
final override string getValueText() { result = value.toString() }
final override int getValue() { result = value }
}
/**
* A floating-point literal.
*
* ```rb
* 1.3
* 2.7e+5
* ```
*/
class FloatLiteral extends NumericLiteral, TFloatLiteral {
private Ruby::Float g;
FloatLiteral() { this = TFloatLiteral(g) }
final override string getValueText() { result = g.getValue() }
final override string toString() { result = this.getValueText() }
final override string getAPrimaryQlClass() { result = "FloatLiteral" }
}
/**
* A rational literal.
*
* ```rb
* 123r
* ```
*/
class RationalLiteral extends NumericLiteral, TRationalLiteral {
private Ruby::Rational g;
RationalLiteral() { this = TRationalLiteral(g) }
final override string getValueText() { result = g.getChild().(Ruby::Token).getValue() + "r" }
final override string toString() { result = this.getValueText() }
final override string getAPrimaryQlClass() { result = "RationalLiteral" }
}
/**
* A complex literal.
*
* ```rb
* 1i
* ```
*/
class ComplexLiteral extends NumericLiteral, TComplexLiteral {
private Ruby::Complex g;
ComplexLiteral() { this = TComplexLiteral(g) }
final override string getValueText() { result = g.getValue() }
final override string toString() { result = this.getValueText() }
final override string getAPrimaryQlClass() { result = "ComplexLiteral" }
}
/** A `nil` literal. */
class NilLiteral extends Literal, TNilLiteral {
private Ruby::Nil g;
NilLiteral() { this = TNilLiteral(g) }
final override string getValueText() { result = g.getValue() }
final override string toString() { result = this.getValueText() }
final override string getAPrimaryQlClass() { result = "NilLiteral" }
}
/**
* A Boolean literal.
* ```rb
* true
* false
* TRUE
* FALSE
* ```
*/
class BooleanLiteral extends Literal, TBooleanLiteral {
final override string getAPrimaryQlClass() { result = "BooleanLiteral" }
final override string toString() { result = this.getValueText() }
/** Holds if the Boolean literal is `true` or `TRUE`. */
predicate isTrue() { none() }
/** Holds if the Boolean literal is `false` or `FALSE`. */
predicate isFalse() { none() }
}
private class TrueLiteral extends BooleanLiteral, TTrueLiteral {
private Ruby::True g;
TrueLiteral() { this = TTrueLiteral(g) }
final override string getValueText() { result = g.getValue() }
final override predicate isTrue() { any() }
}
private class FalseLiteral extends BooleanLiteral, TFalseLiteral {
private Ruby::False g;
FalseLiteral() { this = TFalseLiteral(g) }
final override string getValueText() { result = g.getValue() }
final override predicate isFalse() { any() }
}
/**
* The base class for a component of a string: `StringTextComponent`,
* `StringEscapeSequenceComponent`, or `StringInterpolationComponent`.
*/
class StringComponent extends AstNode, TStringComponent {
/**
* Gets the source text for this string component. Has no result if this is
* a `StringInterpolationComponent`.
*/
string getValueText() { none() }
}
/**
* A component of a string (or string-like) literal that is simply text.
*
* For example, the following string literals all contain `StringTextComponent`
* components whose `getValueText()` returns `"foo"`:
*
* ```rb
* 'foo'
* "#{ bar() }foo"
* "foo#{ bar() } baz"
* ```
*/
class StringTextComponent extends StringComponent, TStringTextComponent {
private Ruby::Token g;
StringTextComponent() { this = TStringTextComponent(g) }
final override string toString() { result = g.getValue() }
final override string getValueText() { result = g.getValue() }
final override string getAPrimaryQlClass() { result = "StringTextComponent" }
}
/**
* An escape sequence component of a string or string-like literal.
*/
class StringEscapeSequenceComponent extends StringComponent, TStringEscapeSequenceComponent {
private Ruby::EscapeSequence g;
StringEscapeSequenceComponent() { this = TStringEscapeSequenceComponent(g) }
final override string toString() { result = g.getValue() }
final override string getValueText() { result = g.getValue() }
final override string getAPrimaryQlClass() { result = "StringEscapeSequenceComponent" }
}
/**
* An interpolation expression component of a string or string-like literal.
*/
class StringInterpolationComponent extends StringComponent, StmtSequence,
TStringInterpolationComponent {
private Ruby::Interpolation g;
StringInterpolationComponent() { this = TStringInterpolationComponent(g) }
final override string toString() { result = "#{...}" }
final override Stmt getStmt(int n) { toGenerated(result) = g.getChild(n) }
final override string getValueText() { none() }
final override string getAPrimaryQlClass() { result = "StringInterpolationComponent" }
}
/**
* A string, symbol, regexp, or subshell literal.
*/
class StringlikeLiteral extends Literal, TStringlikeLiteral {
/**
* Gets the `n`th component of this string or string-like literal. The result
* will be one of `StringTextComponent`, `StringInterpolationComponent`, and
* `StringEscapeSequenceComponent`.
*
* In the following example, the result for `n = 0` is the
* `StringTextComponent` for `foo_`, and the result for `n = 1` is the
* `StringInterpolationComponent` for `Time.now`.
*
* ```rb
* "foo_#{ Time.now }"
* ```
*/
StringComponent getComponent(int n) { none() }
/**
* Gets the number of components in this string or string-like literal.
*
* For the empty string `""`, the result is 0.
*
* For the string `"foo"`, the result is 1: there is a single
* `StringTextComponent`.
*
* For the following example, the result is 3: there is a
* `StringTextComponent` for the substring `"foo_"`; a
* `StringEscapeSequenceComponent` for the escaped quote; and a
* `StringInterpolationComponent` for the interpolation.
*
* ```rb
* "foo\"#{bar}"
* ```
*/
final int getNumberOfComponents() { result = count(this.getComponent(_)) }
private string getStartDelimiter() {
this instanceof TStringLiteral and
result = "\""
or
this instanceof TRegExpLiteral and
result = "/"
or
this instanceof TSimpleSymbolLiteral and
result = ":"
or
this instanceof TComplexSymbolLiteral and
result = ":\""
or
this instanceof THashKeySymbolLiteral and
result = ""
or
this instanceof TSubshellLiteral and
result = "`"
or
this instanceof THereDoc and
result = ""
}
private string getEndDelimiter() {
this instanceof TStringLiteral and
result = "\""
or
this instanceof TRegExpLiteral and
result = "/"
or
this instanceof TSimpleSymbolLiteral and
result = ""
or
this instanceof TComplexSymbolLiteral and
result = "\""
or
this instanceof THashKeySymbolLiteral and
result = ""
or
this instanceof TSubshellLiteral and
result = "`"
or
this instanceof THereDoc and
result = ""
}
override string getValueText() {
// 0 components should result in the empty string
// if there are any interpolations, there should be no result
// otherwise, concatenate all the components
forall(StringComponent c | c = this.getComponent(_) |
not c instanceof StringInterpolationComponent
) and
result =
concat(StringComponent c, int i | c = this.getComponent(i) | c.getValueText() order by i)
}
override string toString() {
exists(string full, string summary |
full =
concat(StringComponent c, int i, string s |
c = this.getComponent(i) and
(
s = toGenerated(c).(Ruby::Token).getValue()
or
not toGenerated(c) instanceof Ruby::Token and
s = "#{...}"
)
|
s order by i
) and
(
// summary should be 32 chars max (incl. ellipsis)
full.length() > 32 and summary = full.substring(0, 29) + "..."
or
full.length() <= 32 and summary = full
) and
result = this.getStartDelimiter() + summary + this.getEndDelimiter()
)
}
final override AstNode getAChild(string pred) {
result = super.getAChild(pred)
or
pred = "getComponent" and result = this.getComponent(_)
}
}
/**
* A string literal.
*
* ```rb
* 'hello'
* "hello, #{name}"
* ```
*/
class StringLiteral extends StringlikeLiteral, TStringLiteral {
final override string getAPrimaryQlClass() { result = "StringLiteral" }
}
private class RegularStringLiteral extends StringLiteral, TRegularStringLiteral {
private Ruby::String g;
RegularStringLiteral() { this = TRegularStringLiteral(g) }
final override StringComponent getComponent(int n) { toGenerated(result) = g.getChild(n) }
}
private class BareStringLiteral extends StringLiteral, TBareStringLiteral {
private Ruby::BareString g;
BareStringLiteral() { this = TBareStringLiteral(g) }
final override StringComponent getComponent(int n) { toGenerated(result) = g.getChild(n) }
}
/**
* A regular expression literal.
*
* ```rb
* /[a-z]+/
* ```
*/
class RegExpLiteral extends StringlikeLiteral, TRegExpLiteral {
private Ruby::Regex g;
RegExpLiteral() { this = TRegExpLiteral(g) }
final override string getAPrimaryQlClass() { result = "RegExpLiteral" }
final override StringComponent getComponent(int i) { toGenerated(result) = g.getChild(i) }
/**
* Gets the regexp flags as a string.
*
* ```rb
* /foo/ # => ""
* /foo/i # => "i"
* /foo/imxo # => "imxo"
*/
final string getFlagString() {
// For `/foo/i`, there should be an `/i` token in the database with `this`
// as its parents. Strip the delimiter, which can vary.
result =
max(Ruby::Token t | t.getParent() = g | t.getValue().suffix(1) order by t.getParentIndex())
}
/**
* Holds if the regexp was specified using the `i` flag to indicate case
* insensitivity, as in the following example:
*
* ```rb
* /foo/i
* ```
*/
final predicate hasCaseInsensitiveFlag() { this.getFlagString().charAt(_) = "i" }
/**
* Holds if the regex was specified using the `m` flag to indicate multiline
* mode. For example:
*
* ```rb
* /foo/m
* ```
*/
final predicate hasMultilineFlag() { this.getFlagString().charAt(_) = "m" }
/**
* Holds if the regex was specified using the `x` flag to indicate
* 'free-spacing' mode (also known as 'extended' mode), meaning that
* whitespace and comments in the pattern are ignored. For example:
*
* ```rb
* %r{
* [a-zA-Z_] # starts with a letter or underscore
* \w* # and then zero or more letters/digits/underscores
* }/x
* ```
*/
final predicate hasFreeSpacingFlag() { this.getFlagString().charAt(_) = "x" }
/** Returns the root node of the parse tree of this regular expression. */
final RETV::RegExpTerm getParsed() { result = RETV::getParsedRegExp(this) }
}
/**
* A symbol literal.
*
* ```rb
* :foo
* :"foo bar"
* :"foo bar #{baz}"
* ```
*/
class SymbolLiteral extends StringlikeLiteral, TSymbolLiteral {
final override string getAPrimaryQlClass() {
not this instanceof MethodName and result = "SymbolLiteral"
}
}
private class SimpleSymbolLiteral extends SymbolLiteral, TSimpleSymbolLiteral {
private Ruby::SimpleSymbol g;
SimpleSymbolLiteral() { this = TSimpleSymbolLiteral(g) }
// Tree-sitter gives us value text including the colon, which we skip.
final override string getValueText() { result = g.getValue().suffix(1) }
final override string toString() { result = g.getValue() }
}
private class ComplexSymbolLiteral extends SymbolLiteral, TComplexSymbolLiteral { }
private class DelimitedSymbolLiteral extends ComplexSymbolLiteral, TDelimitedSymbolLiteral {
private Ruby::DelimitedSymbol g;
DelimitedSymbolLiteral() { this = TDelimitedSymbolLiteral(g) }
final override StringComponent getComponent(int i) { toGenerated(result) = g.getChild(i) }
}
private class BareSymbolLiteral extends ComplexSymbolLiteral, TBareSymbolLiteral {
private Ruby::BareSymbol g;
BareSymbolLiteral() { this = TBareSymbolLiteral(g) }
final override StringComponent getComponent(int i) { toGenerated(result) = g.getChild(i) }
}
private class HashKeySymbolLiteral extends SymbolLiteral, THashKeySymbolLiteral {
private Ruby::HashKeySymbol g;
HashKeySymbolLiteral() { this = THashKeySymbolLiteral(g) }
final override string getValueText() { result = g.getValue() }
final override string toString() { result = ":" + this.getValueText() }
}
/**
* A subshell literal.
*
* ```rb
* `ls -l`
* %x(/bin/sh foo.sh)
* ```
*/
class SubshellLiteral extends StringlikeLiteral, TSubshellLiteral {
private Ruby::Subshell g;
SubshellLiteral() { this = TSubshellLiteral(g) }
final override string getAPrimaryQlClass() { result = "SubshellLiteral" }
final override StringComponent getComponent(int i) { toGenerated(result) = g.getChild(i) }
}
/**
* A character literal.
*
* ```rb
* ?a
* ?\u{61}
* ```
*/
class CharacterLiteral extends Literal, TCharacterLiteral {
private Ruby::Character g;
CharacterLiteral() { this = TCharacterLiteral(g) }
final override string getValueText() { result = g.getValue() }
final override string toString() { result = g.getValue() }
final override string getAPrimaryQlClass() { result = "CharacterLiteral" }
}
/**
* A "here document". For example:
* ```rb
* query = <<SQL
* SELECT * FROM person
* WHERE age > 21
* SQL
* ```
*/
class HereDoc extends StringlikeLiteral, THereDoc {
private Ruby::HeredocBeginning g;
HereDoc() { this = THereDoc(g) }
final override string getAPrimaryQlClass() { result = "HereDoc" }
/**
* Holds if this here document is executed in a subshell.
* ```rb
* <<`COMMAND`
* echo "Hello world!"
* COMMAND
* ```
*/
final predicate isSubShell() { getQuoteStyle() = "`" }
/**
* Gets the quotation mark (`"`, `'` or `` ` ``) that surrounds the here document identifier, if any.
* ```rb
* <<"IDENTIFIER"
* <<'IDENTIFIER'
* <<`IDENTIFIER`
* ```
*/
final string getQuoteStyle() {
exists(string s |
s = g.getValue() and
s.charAt(s.length() - 1) = result and
result = ["'", "`", "\""]
)
}
/**
* Gets the indentation modifier (`-` or `~`) of the here document identifier, if any.
* ```rb
* <<~IDENTIFIER
* <<-IDENTIFIER
* <<IDENTIFIER
* ```
*/
final string getIndentationModifier() {
exists(string s |
s = g.getValue() and
s.charAt(2) = result and
result = ["-", "~"]
)
}
final override StringComponent getComponent(int n) {
toGenerated(result) = getHereDocBody(g).getChild(n)
}
final override string toString() { result = g.getValue() }
}
/**
* An array literal.
*
* ```rb
* [123, 'foo', bar()]
* %w(foo bar)
* %i(foo bar)
* ```
*/
class ArrayLiteral extends Literal, TArrayLiteral {
final override string getAPrimaryQlClass() { result = "ArrayLiteral" }
/** Gets the `n`th element in this array literal. */
final Expr getElement(int n) { result = this.(ArrayLiteralImpl).getElementImpl(n) }
/** Gets an element in this array literal. */
final Expr getAnElement() { result = this.getElement(_) }
/** Gets the number of elements in this array literal. */
final int getNumberOfElements() { result = this.(ArrayLiteralImpl).getNumberOfElementsImpl() }
final override AstNode getAChild(string pred) {
result = super.getAChild(pred)
or
pred = "getElement" and result = this.getElement(_)
}
}
abstract private class ArrayLiteralImpl extends ArrayLiteral {
abstract Expr getElementImpl(int n);
abstract int getNumberOfElementsImpl();
}
private class RegularArrayLiteral extends ArrayLiteralImpl, TRegularArrayLiteral {
private Ruby::Array g;
RegularArrayLiteral() { this = TRegularArrayLiteral(g) }
final override Expr getElementImpl(int i) { toGenerated(result) = g.getChild(i) }
final override int getNumberOfElementsImpl() { result = count(g.getChild(_)) }
final override string toString() { result = "[...]" }
}
private class StringArrayLiteral extends ArrayLiteralImpl, TStringArrayLiteral {
private Ruby::StringArray g;
StringArrayLiteral() { this = TStringArrayLiteral(g) }
final override Expr getElementImpl(int i) { toGenerated(result) = g.getChild(i) }
final override int getNumberOfElementsImpl() { result = count(g.getChild(_)) }
final override string toString() { result = "%w(...)" }
}
private class SymbolArrayLiteral extends ArrayLiteralImpl, TSymbolArrayLiteral {
private Ruby::SymbolArray g;
SymbolArrayLiteral() { this = TSymbolArrayLiteral(g) }
final override Expr getElementImpl(int i) { toGenerated(result) = g.getChild(i) }
final override int getNumberOfElementsImpl() { result = count(g.getChild(_)) }
final override string toString() { result = "%i(...)" }
}
/**
* A hash literal.
*
* ```rb
* { foo: 123, bar: 456 }
* ```
*/
class HashLiteral extends Literal, THashLiteral {
private Ruby::Hash g;
HashLiteral() { this = THashLiteral(g) }
final override string getAPrimaryQlClass() { result = "HashLiteral" }
/**
* Gets the `n`th element in this array literal.
*
* In the following example, the 0th element is a `Pair`, and the 1st element
* is a `HashSplatExpr`.
*
* ```rb
* { foo: 123, **bar }
* ```
*/
final Expr getElement(int n) { toGenerated(result) = g.getChild(n) }
/** Gets an element in this array literal. */
final Expr getAnElement() { result = this.getElement(_) }
/** Gets a key-value `Pair` in this hash literal. */
final Pair getAKeyValuePair() { result = this.getAnElement() }
/** Gets the number of elements in this hash literal. */
final int getNumberOfElements() { result = count(this.getAnElement()) }
final override string toString() { result = "{...}" }
final override AstNode getAChild(string pred) {
result = super.getAChild(pred)
or
pred = "getElement" and result = this.getElement(_)
}
}
/**
* A range literal.
*
* ```rb
* (1..10)
* (1024...2048)
* ```
*/
class RangeLiteral extends Literal, TRangeLiteral {
final override string getAPrimaryQlClass() { result = "RangeLiteral" }
/** Gets the begin expression of this range, if any. */
Expr getBegin() { none() }
/** Gets the end expression of this range, if any. */
Expr getEnd() { none() }
/**
* Holds if the range is inclusive of the end value, i.e. uses the `..`
* operator.
*/
predicate isInclusive() { none() }
/**
* Holds if the range is exclusive of the end value, i.e. uses the `...`
* operator.
*/
predicate isExclusive() { none() }
final override AstNode getAChild(string pred) {
result = super.getAChild(pred)
or
pred = "getBegin" and result = this.getBegin()
or
pred = "getEnd" and result = this.getEnd()
}
final override string toString() {
exists(string op |
this.isInclusive() and op = ".."
or
this.isExclusive() and op = "..."
|
result = "_ " + op + " _"
)
}
}
private class RangeLiteralReal extends RangeLiteral, TRangeLiteralReal {
private Ruby::Range g;
RangeLiteralReal() { this = TRangeLiteralReal(g) }
final override Expr getBegin() { toGenerated(result) = g.getBegin() }
final override Expr getEnd() { toGenerated(result) = g.getEnd() }
final override predicate isInclusive() { g instanceof @ruby_range_dotdot }
final override predicate isExclusive() { g instanceof @ruby_range_dotdotdot }
}
private class RangeLiteralSynth extends RangeLiteral, TRangeLiteralSynth {
private boolean inclusive;
RangeLiteralSynth() { this = TRangeLiteralSynth(_, _, inclusive) }
final override Expr getBegin() { result = TIntegerLiteralSynth(this, 0, _) }
final override Expr getEnd() { result = TIntegerLiteralSynth(this, 1, _) }
final override predicate isInclusive() { inclusive = true }
final override predicate isExclusive() { inclusive = false }
}
/**
* A method name literal. For example:
* ```rb
* method_name # a normal name
* + # an operator
* :method_name # a symbol
* :"eval_#{name}" # a complex symbol
* ```
*/
class MethodName extends Literal {
MethodName() { MethodName::range(toGenerated(this)) }
final override string getAPrimaryQlClass() { result = "MethodName" }
}
private class TokenMethodName extends MethodName, TTokenMethodName {
private MethodName::Token g;
TokenMethodName() { this = TTokenMethodName(g) }
final override string getValueText() {
result = g.(Ruby::Token).getValue()
or
result = g.(Ruby::Setter).getName().getValue() + "="
}
final override string toString() { result = this.getValueText() }
}

View File

@@ -0,0 +1,228 @@
private import codeql.ruby.AST
private import codeql.ruby.controlflow.ControlFlowGraph
private import internal.AST
private import internal.TreeSitter
/** A callable. */
class Callable extends Expr, Scope, TCallable {
/** Gets the number of parameters of this callable. */
final int getNumberOfParameters() { result = count(this.getAParameter()) }
/** Gets a parameter of this callable. */
final Parameter getAParameter() { result = this.getParameter(_) }
/** Gets the `n`th parameter of this callable. */
Parameter getParameter(int n) { none() }
override AstNode getAChild(string pred) {
result = super.getAChild(pred)
or
pred = "getParameter" and result = this.getParameter(_)
}
}
/** A method. */
class MethodBase extends Callable, BodyStmt, Scope, TMethodBase {
/** Gets the name of this method. */
string getName() { none() }
/** Holds if the name of this method is `name`. */
final predicate hasName(string name) { this.getName() = name }
override AstNode getAChild(string pred) {
result = Callable.super.getAChild(pred)
or
result = BodyStmt.super.getAChild(pred)
}
}
/** A call to `private`. */
private class Private extends MethodCall {
Private() { this.getMethodName() = "private" }
}
/** A normal method. */
class Method extends MethodBase, TMethod {
private Ruby::Method g;
Method() { this = TMethod(g) }
final override string getAPrimaryQlClass() { result = "Method" }
final override string getName() {
result = g.getName().(Ruby::Token).getValue() or
result = g.getName().(Ruby::Setter).getName().getValue() + "="
}
/**
* Holds if this is a setter method, as in the following example:
* ```rb
* class Person
* def name=(n)
* @name = n
* end
* end
* ```
*/
final predicate isSetter() { g.getName() instanceof Ruby::Setter }
/**
* Holds if this method is private. All methods with the name prefix
* `private` are private below:
*
* ```rb
* class C
* private def private1
* end
*
* def public
* end
*
* def private2
* end
* private :private2
*
* private
*
* def private3
* end
*
* def private4
* end
* end
* ```
*/
predicate isPrivate() {
this = any(Private p).getArgument(0)
or
exists(ClassDeclaration c, Private p, SymbolLiteral s |
p.getArgument(0) = s and
p = c.getAStmt() and
this.getName() = s.getValueText() and
this = c.getAStmt()
)
or
exists(ClassDeclaration c, int i, int j |
c.getStmt(i).(Private).getNumberOfArguments() = 0 and
this = c.getStmt(j) and
j > i
)
or
// Top-level methods are private members of the Object class
this.getEnclosingModule() instanceof Toplevel
}
final override Parameter getParameter(int n) {
toGenerated(result) = g.getParameters().getChild(n)
}
final override string toString() { result = this.getName() }
}
/** A singleton method. */
class SingletonMethod extends MethodBase, TSingletonMethod {
private Ruby::SingletonMethod g;
SingletonMethod() { this = TSingletonMethod(g) }
final override string getAPrimaryQlClass() { result = "SingletonMethod" }
/** Gets the object of this singleton method. */
final Expr getObject() { toGenerated(result) = g.getObject() }
final override string getName() {
result = g.getName().(Ruby::Token).getValue()
or
result = g.getName().(Ruby::Setter).getName().getValue() + "="
}
final override Parameter getParameter(int n) {
toGenerated(result) = g.getParameters().getChild(n)
}
final override string toString() { result = this.getName() }
final override AstNode getAChild(string pred) {
result = super.getAChild(pred)
or
pred = "getObject" and result = this.getObject()
}
}
/**
* A lambda (anonymous method). For example:
* ```rb
* -> (x) { x + 1 }
* ```
*/
class Lambda extends Callable, BodyStmt, TLambda {
private Ruby::Lambda g;
Lambda() { this = TLambda(g) }
final override string getAPrimaryQlClass() { result = "Lambda" }
final override Parameter getParameter(int n) {
toGenerated(result) = g.getParameters().getChild(n)
}
final override string toString() { result = "-> { ... }" }
final override AstNode getAChild(string pred) {
result = Callable.super.getAChild(pred)
or
result = BodyStmt.super.getAChild(pred)
}
}
/** A block. */
class Block extends Callable, StmtSequence, Scope, TBlock {
override AstNode getAChild(string pred) {
result = Callable.super.getAChild(pred)
or
result = StmtSequence.super.getAChild(pred)
}
}
/** A block enclosed within `do` and `end`. */
class DoBlock extends Block, BodyStmt, TDoBlock {
private Ruby::DoBlock g;
DoBlock() { this = TDoBlock(g) }
final override Parameter getParameter(int n) {
toGenerated(result) = g.getParameters().getChild(n)
}
final override string toString() { result = "do ... end" }
final override AstNode getAChild(string pred) {
result = Block.super.getAChild(pred)
or
result = BodyStmt.super.getAChild(pred)
}
final override string getAPrimaryQlClass() { result = "DoBlock" }
}
/**
* A block defined using curly braces, e.g. in the following code:
* ```rb
* names.each { |name| puts name }
* ```
*/
class BraceBlock extends Block, TBraceBlock {
private Ruby::Block g;
BraceBlock() { this = TBraceBlock(g) }
final override Parameter getParameter(int n) {
toGenerated(result) = g.getParameters().getChild(n)
}
final override Stmt getStmt(int i) { toGenerated(result) = g.getChild(i) }
final override string toString() { result = "{ ... }" }
final override string getAPrimaryQlClass() { result = "BraceBlock" }
}

View File

@@ -0,0 +1,365 @@
private import codeql.ruby.AST
private import codeql.ruby.ast.Constant
private import internal.AST
private import internal.Module
private import internal.TreeSitter
/**
* A representation of a run-time `module` or `class` value.
*/
class Module extends TModule {
/** Gets a declaration of this module, if any. */
ModuleBase getADeclaration() { result.getModule() = this }
/** Gets the super class of this module, if any. */
Module getSuperClass() { result = getSuperClass(this) }
/** Gets a `prepend`ed module. */
Module getAPrependedModule() { result = getAPrependedModule(this) }
/** Gets an `include`d module. */
Module getAnIncludedModule() { result = getAnIncludedModule(this) }
/** Holds if this module is a class. */
pragma[noinline]
predicate isClass() { this.getADeclaration() instanceof ClassDeclaration }
/** Gets a textual representation of this module. */
string toString() {
this = TResolved(result)
or
exists(Namespace n | this = TUnresolved(n) and result = "...::" + n.toString())
}
/** Gets the location of this module. */
Location getLocation() {
exists(Namespace n | this = TUnresolved(n) and result = n.getLocation())
or
result =
min(Namespace n, string qName, Location loc, int weight |
this = TResolved(qName) and
qName = namespaceDeclaration(n) and
loc = n.getLocation() and
if exists(loc.getFile().getRelativePath()) then weight = 0 else weight = 1
|
loc
order by
weight, count(n.getAStmt()) desc, loc.getFile().getAbsolutePath(), loc.getStartLine(),
loc.getStartColumn()
)
}
}
/**
* The base class for classes, singleton classes, and modules.
*/
class ModuleBase extends BodyStmt, Scope, TModuleBase {
/** Gets a method defined in this module/class. */
MethodBase getAMethod() { result = this.getAStmt() }
/** Gets the method named `name` in this module/class, if any. */
MethodBase getMethod(string name) { result = this.getAMethod() and result.getName() = name }
/** Gets a class defined in this module/class. */
ClassDeclaration getAClass() { result = this.getAStmt() }
/** Gets the class named `name` in this module/class, if any. */
ClassDeclaration getClass(string name) { result = this.getAClass() and result.getName() = name }
/** Gets a module defined in this module/class. */
ModuleDeclaration getAModule() { result = this.getAStmt() }
/** Gets the module named `name` in this module/class, if any. */
ModuleDeclaration getModule(string name) {
result = this.getAModule() and result.getName() = name
}
/**
* Gets the value of the constant named `name`, if any.
*
* For example, the value of `CONST` is `"const"` in
* ```rb
* module M
* CONST = "const"
* end
* ```
*/
Expr getConstant(string name) {
exists(AssignExpr ae, ConstantWriteAccess w |
ae = this.getAStmt() and
w = ae.getLeftOperand() and
w.getName() = name and
not exists(w.getScopeExpr()) and
result = ae.getRightOperand()
)
}
/** Gets the representation of the run-time value of this module or class. */
Module getModule() { none() }
}
/**
* A Ruby source file.
*
* ```rb
* def main
* puts "hello world!"
* end
* main
* ```
*/
class Toplevel extends ModuleBase, TToplevel {
private Ruby::Program g;
Toplevel() { this = TToplevel(g) }
final override string getAPrimaryQlClass() { result = "Toplevel" }
/**
* Gets the `n`th `BEGIN` block.
*/
final BeginBlock getBeginBlock(int n) {
toGenerated(result) = rank[n + 1](int i, Ruby::BeginBlock b | b = g.getChild(i) | b order by i)
}
/**
* Gets a `BEGIN` block.
*/
final BeginBlock getABeginBlock() { result = getBeginBlock(_) }
final override AstNode getAChild(string pred) {
result = super.getAChild(pred)
or
pred = "getBeginBlock" and result = this.getBeginBlock(_)
}
final override Module getModule() { result = TResolved("Object") }
final override string toString() { result = g.getLocation().getFile().getBaseName() }
}
/**
* A class or module definition.
*
* ```rb
* class Foo
* def bar
* end
* end
* module Bar
* class Baz
* end
* end
* ```
*/
class Namespace extends ModuleBase, ConstantWriteAccess, TNamespace {
override string getAPrimaryQlClass() { result = "Namespace" }
/**
* Gets the name of the module/class. In the following example, the result is
* `"Foo"`.
* ```rb
* class Foo
* end
* ```
*
* N.B. in the following example, where the module/class name uses the scope
* resolution operator, the result is the name being resolved, i.e. `"Bar"`.
* Use `getScopeExpr` to get the `Foo` for `Foo`.
* ```rb
* module Foo::Bar
* end
* ```
*/
override string getName() { none() }
/**
* Gets the scope expression used in the module/class name's scope resolution
* operation, if any.
*
* In the following example, the result is the `Expr` for `Foo`.
*
* ```rb
* module Foo::Bar
* end
* ```
*
* However, there is no result for the following example, since there is no
* scope resolution operation.
*
* ```rb
* module Baz
* end
* ```
*/
override Expr getScopeExpr() { none() }
/**
* Holds if the module/class name uses the scope resolution operator to access the
* global scope, as in this example:
*
* ```rb
* class ::Foo
* end
* ```
*/
override predicate hasGlobalScope() { none() }
final override Module getModule() {
result = any(string qName | qName = namespaceDeclaration(this) | TResolved(qName))
or
result = TUnresolved(this)
}
override AstNode getAChild(string pred) {
result = ModuleBase.super.getAChild(pred) or
result = ConstantWriteAccess.super.getAChild(pred)
}
final override string toString() { result = ConstantWriteAccess.super.toString() }
}
/**
* A class definition.
*
* ```rb
* class Foo
* def bar
* end
* end
* ```
*/
class ClassDeclaration extends Namespace, TClassDeclaration {
private Ruby::Class g;
ClassDeclaration() { this = TClassDeclaration(g) }
final override string getAPrimaryQlClass() { result = "ClassDeclaration" }
/**
* Gets the `Expr` used as the superclass in the class definition, if any.
*
* In the following example, the result is a `ConstantReadAccess`.
* ```rb
* class Foo < Bar
* end
* ```
*
* In the following example, where the superclass is a call expression, the
* result is a `Call`.
* ```rb
* class C < foo()
* end
* ```
*/
final Expr getSuperclassExpr() { toGenerated(result) = g.getSuperclass().getChild() }
final override string getName() {
result = g.getName().(Ruby::Token).getValue() or
result = g.getName().(Ruby::ScopeResolution).getName().(Ruby::Token).getValue()
}
final override Expr getScopeExpr() {
toGenerated(result) = g.getName().(Ruby::ScopeResolution).getScope()
}
final override predicate hasGlobalScope() {
exists(Ruby::ScopeResolution sr |
sr = g.getName() and
not exists(sr.getScope())
)
}
final override AstNode getAChild(string pred) {
result = super.getAChild(pred)
or
pred = "getSuperclassExpr" and result = this.getSuperclassExpr()
}
}
/**
* A definition of a singleton class on an object.
*
* ```rb
* class << foo
* def bar
* p 'bar'
* end
* end
* ```
*/
class SingletonClass extends ModuleBase, TSingletonClass {
private Ruby::SingletonClass g;
SingletonClass() { this = TSingletonClass(g) }
final override string getAPrimaryQlClass() { result = "ClassDeclaration" }
/**
* Gets the expression resulting in the object on which the singleton class
* is defined. In the following example, the result is the `Expr` for `foo`:
*
* ```rb
* class << foo
* end
* ```
*/
final Expr getValue() { toGenerated(result) = g.getValue() }
final override string toString() { result = "class << ..." }
final override AstNode getAChild(string pred) {
result = super.getAChild(pred)
or
pred = "getValue" and result = this.getValue()
}
}
/**
* A module definition.
*
* ```rb
* module Foo
* class Bar
* end
* end
* ```
*
* N.B. this class represents a single instance of a module definition. In the
* following example, classes `Bar` and `Baz` are both defined in the module
* `Foo`, but in two syntactically distinct definitions, meaning that there
* will be two instances of `ModuleDeclaration` in the database.
*
* ```rb
* module Foo
* class Bar; end
* end
*
* module Foo
* class Baz; end
* end
* ```
*/
class ModuleDeclaration extends Namespace, TModuleDeclaration {
private Ruby::Module g;
ModuleDeclaration() { this = TModuleDeclaration(g) }
final override string getAPrimaryQlClass() { result = "ModuleDeclaration" }
final override string getName() {
result = g.getName().(Ruby::Token).getValue() or
result = g.getName().(Ruby::ScopeResolution).getName().(Ruby::Token).getValue()
}
final override Expr getScopeExpr() {
toGenerated(result) = g.getName().(Ruby::ScopeResolution).getScope()
}
final override predicate hasGlobalScope() {
exists(Ruby::ScopeResolution sr |
sr = g.getName() and
not exists(sr.getScope())
)
}
}

View File

@@ -0,0 +1,620 @@
private import codeql.ruby.AST
private import internal.AST
private import internal.TreeSitter
private import internal.Operation
/**
* An operation.
*
* This is the QL root class for all operations.
*/
class Operation extends Expr instanceof OperationImpl {
/** Gets the operator of this operation. */
final string getOperator() { result = super.getOperatorImpl() }
/** Gets an operand of this operation. */
final Expr getAnOperand() { result = super.getAnOperandImpl() }
override AstNode getAChild(string pred) {
result = Expr.super.getAChild(pred)
or
pred = "getAnOperand" and result = this.getAnOperand()
}
}
/** A unary operation. */
class UnaryOperation extends Operation, MethodCall instanceof UnaryOperationImpl {
/** Gets the operand of this unary operation. */
final Expr getOperand() { result = super.getOperandImpl() }
final override AstNode getAChild(string pred) {
result = Operation.super.getAChild(pred)
or
result = MethodCall.super.getAChild(pred)
or
pred = "getOperand" and result = this.getOperand()
}
final override string toString() { result = this.getOperator() + " ..." }
}
/** A unary logical operation. */
class UnaryLogicalOperation extends UnaryOperation, TUnaryLogicalOperation { }
/**
* A logical NOT operation, using either `!` or `not`.
* ```rb
* !x.nil?
* not params.empty?
* ```
*/
class NotExpr extends UnaryLogicalOperation, TNotExpr {
final override string getAPrimaryQlClass() { result = "NotExpr" }
}
/** A unary arithmetic operation. */
class UnaryArithmeticOperation extends UnaryOperation, TUnaryArithmeticOperation { }
/**
* A unary plus expression.
* ```rb
* + a
* ```
*/
class UnaryPlusExpr extends UnaryArithmeticOperation, TUnaryPlusExpr {
final override string getAPrimaryQlClass() { result = "UnaryPlusExpr" }
}
/**
* A unary minus expression.
* ```rb
* - a
* ```
*/
class UnaryMinusExpr extends UnaryArithmeticOperation, TUnaryMinusExpr {
final override string getAPrimaryQlClass() { result = "UnaryMinusExpr" }
}
/**
* A splat expression.
* ```rb
* foo(*args)
* ```
*/
class SplatExpr extends UnaryOperation, TSplatExpr {
final override string getAPrimaryQlClass() { result = "SplatExpr" }
}
/**
* A hash-splat (or 'double-splat') expression.
* ```rb
* foo(**options)
* ```
*/
class HashSplatExpr extends UnaryOperation, THashSplatExpr {
private Ruby::HashSplatArgument g;
HashSplatExpr() { this = THashSplatExpr(g) }
final override string getAPrimaryQlClass() { result = "HashSplatExpr" }
}
/** A unary bitwise operation. */
class UnaryBitwiseOperation extends UnaryOperation, TUnaryBitwiseOperation { }
/**
* A complement (bitwise NOT) expression.
* ```rb
* ~x
* ```
*/
class ComplementExpr extends UnaryBitwiseOperation, TComplementExpr {
final override string getAPrimaryQlClass() { result = "ComplementExpr" }
}
/**
* A call to the special `defined?` operator.
* ```rb
* defined? some_method
* ```
*/
class DefinedExpr extends UnaryOperation, TDefinedExpr {
final override string getAPrimaryQlClass() { result = "DefinedExpr" }
}
/** A binary operation. */
class BinaryOperation extends Operation, MethodCall instanceof BinaryOperationImpl {
final override string toString() { result = "... " + this.getOperator() + " ..." }
override AstNode getAChild(string pred) {
result = Operation.super.getAChild(pred)
or
result = MethodCall.super.getAChild(pred)
or
pred = "getLeftOperand" and result = this.getLeftOperand()
or
pred = "getRightOperand" and result = this.getRightOperand()
}
/** Gets the left operand of this binary operation. */
final Stmt getLeftOperand() { result = super.getLeftOperandImpl() }
/** Gets the right operand of this binary operation. */
final Stmt getRightOperand() { result = super.getRightOperandImpl() }
}
/**
* A binary arithmetic operation.
*/
class BinaryArithmeticOperation extends BinaryOperation, TBinaryArithmeticOperation { }
/**
* An add expression.
* ```rb
* x + 1
* ```
*/
class AddExpr extends BinaryArithmeticOperation, TAddExpr {
final override string getAPrimaryQlClass() { result = "AddExpr" }
}
/**
* A subtract expression.
* ```rb
* x - 3
* ```
*/
class SubExpr extends BinaryArithmeticOperation, TSubExpr {
final override string getAPrimaryQlClass() { result = "SubExpr" }
}
/**
* A multiply expression.
* ```rb
* x * 10
* ```
*/
class MulExpr extends BinaryArithmeticOperation, TMulExpr {
final override string getAPrimaryQlClass() { result = "MulExpr" }
}
/**
* A divide expression.
* ```rb
* x / y
* ```
*/
class DivExpr extends BinaryArithmeticOperation, TDivExpr {
final override string getAPrimaryQlClass() { result = "DivExpr" }
}
/**
* A modulo expression.
* ```rb
* x % 2
* ```
*/
class ModuloExpr extends BinaryArithmeticOperation, TModuloExpr {
final override string getAPrimaryQlClass() { result = "ModuloExpr" }
}
/**
* An exponent expression.
* ```rb
* x ** 2
* ```
*/
class ExponentExpr extends BinaryArithmeticOperation, TExponentExpr {
final override string getAPrimaryQlClass() { result = "ExponentExpr" }
}
/**
* A binary logical operation.
*/
class BinaryLogicalOperation extends BinaryOperation, TBinaryLogicalOperation { }
/**
* A logical AND operation, using either `and` or `&&`.
* ```rb
* x and y
* a && b
* ```
*/
class LogicalAndExpr extends BinaryLogicalOperation, TLogicalAndExpr {
final override string getAPrimaryQlClass() { result = "LogicalAndExpr" }
}
/**
* A logical OR operation, using either `or` or `||`.
* ```rb
* x or y
* a || b
* ```
*/
class LogicalOrExpr extends BinaryLogicalOperation, TLogicalOrExpr {
final override string getAPrimaryQlClass() { result = "LogicalOrExpr" }
}
/**
* A binary bitwise operation.
*/
class BinaryBitwiseOperation extends BinaryOperation, TBinaryBitwiseOperation { }
/**
* A left-shift operation.
* ```rb
* x << n
* ```
*/
class LShiftExpr extends BinaryBitwiseOperation, TLShiftExpr {
final override string getAPrimaryQlClass() { result = "LShiftExpr" }
}
/**
* A right-shift operation.
* ```rb
* x >> n
* ```
*/
class RShiftExpr extends BinaryBitwiseOperation, TRShiftExpr {
final override string getAPrimaryQlClass() { result = "RShiftExpr" }
}
/**
* A bitwise AND operation.
* ```rb
* x & 0xff
* ```
*/
class BitwiseAndExpr extends BinaryBitwiseOperation, TBitwiseAndExpr {
final override string getAPrimaryQlClass() { result = "BitwiseAndExpr" }
}
/**
* A bitwise OR operation.
* ```rb
* x | 0x01
* ```
*/
class BitwiseOrExpr extends BinaryBitwiseOperation, TBitwiseOrExpr {
final override string getAPrimaryQlClass() { result = "BitwiseOrExpr" }
}
/**
* An XOR (exclusive OR) operation.
* ```rb
* x ^ y
* ```
*/
class BitwiseXorExpr extends BinaryBitwiseOperation, TBitwiseXorExpr {
final override string getAPrimaryQlClass() { result = "BitwiseXorExpr" }
}
/**
* A comparison operation. That is, either an equality operation or a
* relational operation.
*/
class ComparisonOperation extends BinaryOperation, TComparisonOperation { }
/**
* An equality operation.
*/
class EqualityOperation extends ComparisonOperation, TEqualityOperation { }
/**
* An equals expression.
* ```rb
* x == y
* ```
*/
class EqExpr extends EqualityOperation, TEqExpr {
final override string getAPrimaryQlClass() { result = "EqExpr" }
}
/**
* A not-equals expression.
* ```rb
* x != y
* ```
*/
class NEExpr extends EqualityOperation, TNEExpr {
final override string getAPrimaryQlClass() { result = "NEExpr" }
}
/**
* A case-equality (or 'threequals') expression.
* ```rb
* String === "foo"
* ```
*/
class CaseEqExpr extends EqualityOperation, TCaseEqExpr {
final override string getAPrimaryQlClass() { result = "CaseEqExpr" }
}
/**
* A relational operation, that is, one of `<=`, `<`, `>`, or `>=`.
*/
class RelationalOperation extends ComparisonOperation, TRelationalOperation {
/** Gets the greater operand. */
Expr getGreaterOperand() { none() }
/** Gets the lesser operand. */
Expr getLesserOperand() { none() }
final override AstNode getAChild(string pred) {
result = super.getAChild(pred)
or
pred = "getGreaterOperand" and result = this.getGreaterOperand()
or
pred = "getLesserOperand" and result = this.getLesserOperand()
}
}
/**
* A greater-than expression.
* ```rb
* x > 0
* ```
*/
class GTExpr extends RelationalOperation, TGTExpr {
final override string getAPrimaryQlClass() { result = "GTExpr" }
final override Expr getGreaterOperand() { result = this.getLeftOperand() }
final override Expr getLesserOperand() { result = this.getRightOperand() }
}
/**
* A greater-than-or-equal expression.
* ```rb
* x >= 0
* ```
*/
class GEExpr extends RelationalOperation, TGEExpr {
final override string getAPrimaryQlClass() { result = "GEExpr" }
final override Expr getGreaterOperand() { result = this.getLeftOperand() }
final override Expr getLesserOperand() { result = this.getRightOperand() }
}
/**
* A less-than expression.
* ```rb
* x < 10
* ```
*/
class LTExpr extends RelationalOperation, TLTExpr {
final override string getAPrimaryQlClass() { result = "LTExpr" }
final override Expr getGreaterOperand() { result = this.getRightOperand() }
final override Expr getLesserOperand() { result = this.getLeftOperand() }
}
/**
* A less-than-or-equal expression.
* ```rb
* x <= 10
* ```
*/
class LEExpr extends RelationalOperation, TLEExpr {
final override string getAPrimaryQlClass() { result = "LEExpr" }
final override Expr getGreaterOperand() { result = this.getRightOperand() }
final override Expr getLesserOperand() { result = this.getLeftOperand() }
}
/**
* A three-way comparison ('spaceship') expression.
* ```rb
* a <=> b
* ```
*/
class SpaceshipExpr extends BinaryOperation, TSpaceshipExpr {
final override string getAPrimaryQlClass() { result = "SpaceshipExpr" }
}
/**
* A regexp match expression.
* ```rb
* input =~ /\d/
* ```
*/
class RegExpMatchExpr extends BinaryOperation, TRegExpMatchExpr {
final override string getAPrimaryQlClass() { result = "RegExpMatchExpr" }
}
/**
* A regexp-doesn't-match expression.
* ```rb
* input !~ /\d/
* ```
*/
class NoRegExpMatchExpr extends BinaryOperation, TNoRegExpMatchExpr {
final override string getAPrimaryQlClass() { result = "NoRegExpMatchExpr" }
}
/**
* A binary assignment operation, including `=`, `+=`, `&=`, etc.
*
* This is a QL base class for all assignments.
*/
class Assignment extends Operation instanceof AssignmentImpl {
/** Gets the left hand side of this assignment. */
final Pattern getLeftOperand() { result = super.getLeftOperandImpl() }
/** Gets the right hand side of this assignment. */
final Expr getRightOperand() { result = super.getRightOperandImpl() }
final override string toString() { result = "... " + this.getOperator() + " ..." }
override AstNode getAChild(string pred) {
result = Operation.super.getAChild(pred)
or
pred = "getLeftOperand" and result = getLeftOperand()
or
pred = "getRightOperand" and result = getRightOperand()
}
}
/**
* An assignment operation with the operator `=`.
* ```rb
* x = 123
* ```
*/
class AssignExpr extends Assignment, TAssignExpr {
final override string getAPrimaryQlClass() { result = "AssignExpr" }
}
/**
* A binary assignment operation other than `=`.
*/
class AssignOperation extends Assignment instanceof AssignOperationImpl { }
/**
* An arithmetic assignment operation: `+=`, `-=`, `*=`, `/=`, `**=`, and `%=`.
*/
class AssignArithmeticOperation extends AssignOperation, TAssignArithmeticOperation { }
/**
* A `+=` assignment expression.
* ```rb
* x += 1
* ```
*/
class AssignAddExpr extends AssignArithmeticOperation, TAssignAddExpr {
final override string getAPrimaryQlClass() { result = "AssignAddExpr" }
}
/**
* A `-=` assignment expression.
* ```rb
* x -= 3
* ```
*/
class AssignSubExpr extends AssignArithmeticOperation, TAssignSubExpr {
final override string getAPrimaryQlClass() { result = "AssignSubExpr" }
}
/**
* A `*=` assignment expression.
* ```rb
* x *= 10
* ```
*/
class AssignMulExpr extends AssignArithmeticOperation, TAssignMulExpr {
final override string getAPrimaryQlClass() { result = "AssignMulExpr" }
}
/**
* A `/=` assignment expression.
* ```rb
* x /= y
* ```
*/
class AssignDivExpr extends AssignArithmeticOperation, TAssignDivExpr {
final override string getAPrimaryQlClass() { result = "AssignDivExpr" }
}
/**
* A `%=` assignment expression.
* ```rb
* x %= 4
* ```
*/
class AssignModuloExpr extends AssignArithmeticOperation, TAssignModuloExpr {
final override string getAPrimaryQlClass() { result = "AssignModuloExpr" }
}
/**
* A `**=` assignment expression.
* ```rb
* x **= 2
* ```
*/
class AssignExponentExpr extends AssignArithmeticOperation, TAssignExponentExpr {
final override string getAPrimaryQlClass() { result = "AssignExponentExpr" }
}
/**
* A logical assignment operation: `&&=` and `||=`.
*/
class AssignLogicalOperation extends AssignOperation, TAssignLogicalOperation { }
/**
* A logical AND assignment operation.
* ```rb
* x &&= y.even?
* ```
*/
class AssignLogicalAndExpr extends AssignLogicalOperation, TAssignLogicalAndExpr {
final override string getAPrimaryQlClass() { result = "AssignLogicalAndExpr" }
}
/**
* A logical OR assignment operation.
* ```rb
* x ||= y
* ```
*/
class AssignLogicalOrExpr extends AssignLogicalOperation, TAssignLogicalOrExpr {
final override string getAPrimaryQlClass() { result = "AssignLogicalOrExpr" }
}
/**
* A bitwise assignment operation: `<<=`, `>>=`, `&=`, `|=` and `^=`.
*/
class AssignBitwiseOperation extends AssignOperation, TAssignBitwiseOperation { }
/**
* A left-shift assignment operation.
* ```rb
* x <<= 3
* ```
*/
class AssignLShiftExpr extends AssignBitwiseOperation, TAssignLShiftExpr {
final override string getAPrimaryQlClass() { result = "AssignLShiftExpr" }
}
/**
* A right-shift assignment operation.
* ```rb
* x >>= 3
* ```
*/
class AssignRShiftExpr extends AssignBitwiseOperation, TAssignRShiftExpr {
final override string getAPrimaryQlClass() { result = "AssignRShiftExpr" }
}
/**
* A bitwise AND assignment operation.
* ```rb
* x &= 0xff
* ```
*/
class AssignBitwiseAndExpr extends AssignBitwiseOperation, TAssignBitwiseAndExpr {
final override string getAPrimaryQlClass() { result = "AssignBitwiseAndExpr" }
}
/**
* A bitwise OR assignment operation.
* ```rb
* x |= 0x01
* ```
*/
class AssignBitwiseOrExpr extends AssignBitwiseOperation, TAssignBitwiseOrExpr {
final override string getAPrimaryQlClass() { result = "AssignBitwiseOrExpr" }
}
/**
* An XOR (exclusive OR) assignment operation.
* ```rb
* x ^= y
* ```
*/
class AssignBitwiseXorExpr extends AssignBitwiseOperation, TAssignBitwiseXorExpr {
final override string getAPrimaryQlClass() { result = "AssignBitwiseXorExpr" }
}

View File

@@ -0,0 +1,235 @@
private import codeql.ruby.AST
private import internal.AST
private import internal.Variable
private import internal.Parameter
private import internal.TreeSitter
/** A parameter. */
class Parameter extends AstNode, TParameter {
/** Gets the callable that this parameter belongs to. */
final Callable getCallable() { result.getAParameter() = this }
/** Gets the zero-based position of this parameter. */
final int getPosition() { this = any(Callable c).getParameter(result) }
/** Gets a variable introduced by this parameter. */
LocalVariable getAVariable() { none() }
/** Gets the variable named `name` introduced by this parameter. */
final LocalVariable getVariable(string name) {
result = this.getAVariable() and
result.getName() = name
}
}
/**
* A parameter defined using a pattern.
*
* This includes both simple parameters and tuple parameters.
*/
class PatternParameter extends Parameter, Pattern, TPatternParameter {
override LocalVariable getAVariable() { result = Pattern.super.getAVariable() }
}
/** A parameter defined using a tuple pattern. */
class TuplePatternParameter extends PatternParameter, TuplePattern, TTuplePatternParameter {
final override LocalVariable getAVariable() { result = TuplePattern.super.getAVariable() }
final override string getAPrimaryQlClass() { result = "TuplePatternParameter" }
override AstNode getAChild(string pred) { result = TuplePattern.super.getAChild(pred) }
}
/** A named parameter. */
class NamedParameter extends Parameter, TNamedParameter {
/** Gets the name of this parameter. */
string getName() { none() }
/** Holds if the name of this parameter is `name`. */
final predicate hasName(string name) { this.getName() = name }
/** Gets the variable introduced by this parameter. */
LocalVariable getVariable() { none() }
override LocalVariable getAVariable() { result = this.getVariable() }
/** Gets an access to this parameter. */
final VariableAccess getAnAccess() { result = this.getVariable().getAnAccess() }
/** Gets the access that defines the underlying local variable. */
final VariableAccess getDefiningAccess() { result = this.getVariable().getDefiningAccess() }
override AstNode getAChild(string pred) {
result = super.getAChild(pred)
or
pred = "getDefiningAccess" and
result = this.getDefiningAccess()
}
}
/** A simple (normal) parameter. */
class SimpleParameter extends NamedParameter, PatternParameter, VariablePattern, TSimpleParameter {
private Ruby::Identifier g;
SimpleParameter() { this = TSimpleParameter(g) }
final override string getName() { result = g.getValue() }
final override LocalVariable getVariable() { result = TLocalVariableReal(_, _, g) }
final override LocalVariable getAVariable() { result = this.getVariable() }
final override string getAPrimaryQlClass() { result = "SimpleParameter" }
final override string toString() { result = this.getName() }
}
/**
* A parameter that is a block. For example, `&bar` in the following code:
* ```rb
* def foo(&bar)
* bar.call if block_given?
* end
* ```
*/
class BlockParameter extends NamedParameter, TBlockParameter {
private Ruby::BlockParameter g;
BlockParameter() { this = TBlockParameter(g) }
final override string getName() { result = g.getName().getValue() }
final override LocalVariable getVariable() { result = TLocalVariableReal(_, _, g.getName()) }
final override string toString() { result = "&" + this.getName() }
final override string getAPrimaryQlClass() { result = "BlockParameter" }
}
/**
* A hash-splat (or double-splat) parameter. For example, `**options` in the
* following code:
* ```rb
* def foo(bar, **options)
* ...
* end
* ```
*/
class HashSplatParameter extends NamedParameter, THashSplatParameter {
private Ruby::HashSplatParameter g;
HashSplatParameter() { this = THashSplatParameter(g) }
final override string getAPrimaryQlClass() { result = "HashSplatParameter" }
final override LocalVariable getVariable() { result = TLocalVariableReal(_, _, g.getName()) }
final override string toString() { result = "**" + this.getName() }
final override string getName() { result = g.getName().getValue() }
}
/**
* A keyword parameter, including a default value if the parameter is optional.
* For example, in the following example, `foo` is a keyword parameter with a
* default value of `0`, and `bar` is a mandatory keyword parameter with no
* default value mandatory parameter).
* ```rb
* def f(foo: 0, bar:)
* foo * 10 + bar
* end
* ```
*/
class KeywordParameter extends NamedParameter, TKeywordParameter {
private Ruby::KeywordParameter g;
KeywordParameter() { this = TKeywordParameter(g) }
final override string getAPrimaryQlClass() { result = "KeywordParameter" }
final override LocalVariable getVariable() { result = TLocalVariableReal(_, _, g.getName()) }
/**
* Gets the default value, i.e. the value assigned to the parameter when one
* is not provided by the caller. If the parameter is mandatory and does not
* have a default value, this predicate has no result.
*/
final Expr getDefaultValue() { toGenerated(result) = g.getValue() }
/**
* Holds if the parameter is optional. That is, there is a default value that
* is used when the caller omits this parameter.
*/
final predicate isOptional() { exists(this.getDefaultValue()) }
final override string toString() { result = this.getName() }
final override string getName() { result = g.getName().getValue() }
final override Location getLocation() { result = g.getName().getLocation() }
final override AstNode getAChild(string pred) {
result = super.getAChild(pred)
or
pred = "getDefaultValue" and result = this.getDefaultValue()
}
}
/**
* An optional parameter. For example, the parameter `name` in the following
* code:
* ```rb
* def say_hello(name = 'Anon')
* puts "hello #{name}"
* end
* ```
*/
class OptionalParameter extends NamedParameter, TOptionalParameter {
private Ruby::OptionalParameter g;
OptionalParameter() { this = TOptionalParameter(g) }
final override string getAPrimaryQlClass() { result = "OptionalParameter" }
/**
* Gets the default value, i.e. the value assigned to the parameter when one
* is not provided by the caller.
*/
final Expr getDefaultValue() { toGenerated(result) = g.getValue() }
final override LocalVariable getVariable() { result = TLocalVariableReal(_, _, g.getName()) }
final override string toString() { result = this.getName() }
final override string getName() { result = g.getName().getValue() }
final override Location getLocation() { result = g.getName().getLocation() }
final override AstNode getAChild(string pred) {
result = super.getAChild(pred)
or
pred = "getDefaultValue" and result = this.getDefaultValue()
}
}
/**
* A splat parameter. For example, `*values` in the following code:
* ```rb
* def foo(bar, *values)
* ...
* end
* ```
*/
class SplatParameter extends NamedParameter, TSplatParameter {
private Ruby::SplatParameter g;
SplatParameter() { this = TSplatParameter(g) }
final override string getAPrimaryQlClass() { result = "SplatParameter" }
final override LocalVariable getVariable() { result = TLocalVariableReal(_, _, g.getName()) }
final override string toString() { result = "*" + this.getName() }
final override string getName() { result = g.getName().getValue() }
}

View File

@@ -0,0 +1,96 @@
private import codeql.ruby.AST
private import codeql.Locations
private import internal.AST
private import internal.Pattern
private import internal.TreeSitter
private import internal.Variable
/** A pattern. */
class Pattern extends AstNode {
Pattern() {
explicitAssignmentNode(toGenerated(this), _)
or
implicitAssignmentNode(toGenerated(this))
or
implicitParameterAssignmentNode(toGenerated(this), _)
or
this = getSynthChild(any(AssignExpr ae), 0)
}
/** Gets a variable used in (or introduced by) this pattern. */
Variable getAVariable() { none() }
}
private class LhsExpr_ =
TVariableAccess or TTokenConstantAccess or TScopeResolutionConstantAccess or TMethodCall or
TSimpleParameter;
/**
* A "left-hand-side" expression. An `LhsExpr` can occur on the left-hand side of
* operator assignments (`AssignOperation`), in patterns (`Pattern`) on the left-hand side of
* an assignment (`AssignExpr`) or for loop (`ForExpr`), and as the exception
* variable of a `rescue` clause (`RescueClause`).
*
* An `LhsExpr` can be a simple variable, a constant, a call, or an element reference:
* ```rb
* var = 1
* var += 1
* E = 1
* foo.bar = 1
* foo[0] = 1
* rescue E => var
* ```
*/
class LhsExpr extends Pattern, LhsExpr_, Expr {
override Variable getAVariable() { result = this.(VariableAccess).getVariable() }
}
private class TVariablePattern = TVariableAccess or TSimpleParameter;
/** A simple variable pattern. */
class VariablePattern extends Pattern, LhsExpr, TVariablePattern { }
/**
* A tuple pattern.
*
* This includes both tuple patterns in parameters and assignments. Example patterns:
* ```rb
* a, self.b = value
* (a, b), c[3] = value
* a, b, *rest, c, d = value
* ```
*/
class TuplePattern extends Pattern, TTuplePattern {
override string getAPrimaryQlClass() { result = "TuplePattern" }
private TuplePatternImpl getImpl() { result = toGenerated(this) }
private Ruby::AstNode getChild(int i) { result = this.getImpl().getChildNode(i) }
/** Gets the `i`th pattern in this tuple pattern. */
final Pattern getElement(int i) {
exists(Ruby::AstNode c | c = this.getChild(i) |
toGenerated(result) = c.(Ruby::RestAssignment).getChild()
or
toGenerated(result) = c
)
}
/** Gets a sub pattern in this tuple pattern. */
final Pattern getAnElement() { result = this.getElement(_) }
/**
* Gets the index of the pattern with the `*` marker on it, if it exists.
* In the example below the index is `2`.
* ```rb
* a, b, *rest, c, d = value
* ```
*/
final int getRestIndex() { result = this.getImpl().getRestIndex() }
override Variable getAVariable() { result = this.getElement(_).getAVariable() }
override string toString() { result = "(..., ...)" }
override AstNode getAChild(string pred) { pred = "getElement" and result = getElement(_) }
}

View File

@@ -0,0 +1,22 @@
private import codeql.ruby.AST
private import internal.AST
private import internal.Scope
private import internal.TreeSitter
class Scope extends AstNode, TScopeType {
private Scope::Range range;
Scope() { range = toGenerated(this) }
/** Gets the scope in which this scope is nested, if any. */
Scope getOuterScope() { toGenerated(result) = range.getOuterScope() }
/** Gets a variable that is declared in this scope. */
final Variable getAVariable() { result.getDeclaringScope() = this }
/** Gets the variable declared in this scope with the given name, if any. */
final Variable getVariable(string name) {
result = this.getAVariable() and
result.getName() = name
}
}

View File

@@ -0,0 +1,248 @@
private import codeql.ruby.AST
private import codeql.ruby.CFG
private import internal.AST
private import internal.TreeSitter
private import internal.Variable
private import codeql.ruby.controlflow.internal.ControlFlowGraphImpl
/**
* A statement.
*
* This is the root QL class for all statements.
*/
class Stmt extends AstNode, TStmt {
/** Gets a control-flow node for this statement, if any. */
CfgNodes::AstCfgNode getAControlFlowNode() { result.getNode() = this }
/** Gets the control-flow scope of this statement, if any. */
CfgScope getCfgScope() { result = getCfgScope(this) }
/** Gets the enclosing callable, if any. */
Callable getEnclosingCallable() { result = this.getCfgScope() }
}
/**
* An empty statement (`;`).
*/
class EmptyStmt extends Stmt, TEmptyStmt {
final override string getAPrimaryQlClass() { result = "EmptyStmt" }
final override string toString() { result = ";" }
}
/**
* A `begin` statement.
* ```rb
* begin
* puts "hello world"
* end
* ```
*/
class BeginExpr extends BodyStmt, TBeginExpr {
final override string getAPrimaryQlClass() { result = "BeginExpr" }
final override string toString() { result = "begin ... " }
}
/**
* A `BEGIN` block.
* ```rb
* BEGIN { puts "starting ..." }
* ```
*/
class BeginBlock extends StmtSequence, TBeginBlock {
private Ruby::BeginBlock g;
BeginBlock() { this = TBeginBlock(g) }
final override string getAPrimaryQlClass() { result = "BeginBlock" }
final override string toString() { result = "BEGIN { ... }" }
final override Stmt getStmt(int n) { toGenerated(result) = g.getChild(n) }
}
/**
* An `END` block.
* ```rb
* END { puts "shutting down" }
* ```
*/
class EndBlock extends StmtSequence, TEndBlock {
private Ruby::EndBlock g;
EndBlock() { this = TEndBlock(g) }
final override string getAPrimaryQlClass() { result = "EndBlock" }
final override string toString() { result = "END { ... }" }
final override Stmt getStmt(int n) { toGenerated(result) = g.getChild(n) }
}
/**
* An `undef` statement. For example:
* ```rb
* - undef method_name
* - undef &&, :method_name
* - undef :"method_#{ name }"
* ```
*/
class UndefStmt extends Stmt, TUndefStmt {
private Ruby::Undef g;
UndefStmt() { this = TUndefStmt(g) }
/** Gets the `n`th method name to undefine. */
final MethodName getMethodName(int n) { toGenerated(result) = g.getChild(n) }
/** Gets a method name to undefine. */
final MethodName getAMethodName() { result = getMethodName(_) }
final override string getAPrimaryQlClass() { result = "UndefStmt" }
final override string toString() { result = "undef ..." }
final override AstNode getAChild(string pred) {
result = super.getAChild(pred)
or
pred = "getMethodName" and result = this.getMethodName(_)
}
}
/**
* An `alias` statement. For example:
* ```rb
* - alias alias_name method_name
* - alias foo :method_name
* - alias bar :"method_#{ name }"
* ```
*/
class AliasStmt extends Stmt, TAliasStmt {
private Ruby::Alias g;
AliasStmt() { this = TAliasStmt(g) }
/** Gets the new method name. */
final MethodName getNewName() { toGenerated(result) = g.getName() }
/** Gets the original method name. */
final MethodName getOldName() { toGenerated(result) = g.getAlias() }
final override string getAPrimaryQlClass() { result = "AliasStmt" }
final override string toString() { result = "alias ..." }
final override AstNode getAChild(string pred) {
result = super.getAChild(pred)
or
pred = "getNewName" and result = this.getNewName()
or
pred = "getOldName" and result = this.getOldName()
}
}
/**
* A statement that may return a value: `return`, `break` and `next`.
*
* ```rb
* return
* return value
* break
* break value
* next
* next value
* ```
*/
class ReturningStmt extends Stmt, TReturningStmt {
private Ruby::ArgumentList getArgumentList() {
result = any(Ruby::Return g | this = TReturnStmt(g)).getChild()
or
result = any(Ruby::Break g | this = TBreakStmt(g)).getChild()
or
result = any(Ruby::Next g | this = TNextStmt(g)).getChild()
}
/** Gets the returned value, if any. */
final Expr getValue() {
toGenerated(result) =
any(Ruby::AstNode res |
exists(Ruby::ArgumentList a, int c |
a = this.getArgumentList() and c = count(a.getChild(_))
|
res = a.getChild(0) and c = 1
or
res = a and c > 1
)
)
}
final override AstNode getAChild(string pred) {
result = super.getAChild(pred)
or
pred = "getValue" and result = this.getValue()
}
}
/**
* A `return` statement.
* ```rb
* return
* return value
* ```
*/
class ReturnStmt extends ReturningStmt, TReturnStmt {
final override string getAPrimaryQlClass() { result = "ReturnStmt" }
final override string toString() { result = "return" }
}
/**
* A `break` statement.
* ```rb
* break
* break value
* ```
*/
class BreakStmt extends ReturningStmt, TBreakStmt {
final override string getAPrimaryQlClass() { result = "BreakStmt" }
final override string toString() { result = "break" }
}
/**
* A `next` statement.
* ```rb
* next
* next value
* ```
*/
class NextStmt extends ReturningStmt, TNextStmt {
final override string getAPrimaryQlClass() { result = "NextStmt" }
final override string toString() { result = "next" }
}
/**
* A `redo` statement.
* ```rb
* redo
* ```
*/
class RedoStmt extends Stmt, TRedoStmt {
final override string getAPrimaryQlClass() { result = "RedoStmt" }
final override string toString() { result = "redo" }
}
/**
* A `retry` statement.
* ```rb
* retry
* ```
*/
class RetryStmt extends Stmt, TRetryStmt {
final override string getAPrimaryQlClass() { result = "RetryStmt" }
final override string toString() { result = "retry" }
}

View File

@@ -0,0 +1,187 @@
/** Provides classes for modeling program variables. */
private import codeql.ruby.AST
private import codeql.Locations
private import internal.AST
private import internal.TreeSitter
private import internal.Variable
/** A variable declared in a scope. */
class Variable instanceof VariableImpl {
/** Gets the name of this variable. */
final string getName() { result = super.getNameImpl() }
/** Holds if the name of this variable is `name`. */
final predicate hasName(string name) { this.getName() = name }
/** Gets a textual representation of this variable. */
final string toString() { result = this.getName() }
/** Gets the location of this variable. */
final Location getLocation() { result = super.getLocationImpl() }
/** Gets the scope this variable is declared in. */
final Scope getDeclaringScope() {
toGenerated(result) = this.(VariableReal).getDeclaringScopeImpl()
}
/** Gets an access to this variable. */
VariableAccess getAnAccess() { result.getVariable() = this }
}
/** A local variable. */
class LocalVariable extends Variable, TLocalVariable {
override LocalVariableAccess getAnAccess() { result.getVariable() = this }
/** Gets the access where this local variable is first introduced. */
VariableAccess getDefiningAccess() { result = this.(LocalVariableReal).getDefiningAccessImpl() }
/**
* Holds if this variable is captured. For example in
*
* ```rb
* def m x
* x.times do |y|
* puts x
* end
* puts x
* end
* ```
*
* `x` is a captured variable, whereas `y` is not.
*/
predicate isCaptured() { this.getAnAccess().isCapturedAccess() }
}
/** A global variable. */
class GlobalVariable extends Variable instanceof GlobalVariableImpl {
final override GlobalVariableAccess getAnAccess() { result.getVariable() = this }
}
/** An instance variable. */
class InstanceVariable extends Variable instanceof InstanceVariableImpl {
/** Holds is this variable is a class instance variable. */
final predicate isClassInstanceVariable() { super.isClassInstanceVariable() }
final override InstanceVariableAccess getAnAccess() { result.getVariable() = this }
}
/** A class variable. */
class ClassVariable extends Variable instanceof ClassVariableImpl {
final override ClassVariableAccess getAnAccess() { result.getVariable() = this }
}
/** An access to a variable. */
class VariableAccess extends Expr instanceof VariableAccessImpl {
/** Gets the variable this identifier refers to. */
final Variable getVariable() { result = super.getVariableImpl() }
/**
* Holds if this access is a write access belonging to the explicit
* assignment `assignment`. For example, in
*
* ```rb
* a, b = foo
* ```
*
* both `a` and `b` are write accesses belonging to the same assignment.
*/
predicate isExplicitWrite(AstNode assignment) {
explicitWriteAccess(toGenerated(this), toGenerated(assignment))
or
this = assignment.(AssignExpr).getLeftOperand()
}
/**
* Holds if this access is a write access belonging to an implicit assignment.
* For example, in
*
* ```rb
* def m elements
* for e in elements do
* puts e
* end
* end
* ```
*
* the access to `elements` in the parameter list is an implicit assignment,
* as is the first access to `e`.
*/
predicate isImplicitWrite() { implicitWriteAccess(toGenerated(this)) }
final override string toString() { result = VariableAccessImpl.super.toString() }
}
/** An access to a variable where the value is updated. */
class VariableWriteAccess extends VariableAccess {
VariableWriteAccess() {
this.isExplicitWrite(_) or
this.isImplicitWrite()
}
}
/** An access to a variable where the value is read. */
class VariableReadAccess extends VariableAccess {
VariableReadAccess() { not this instanceof VariableWriteAccess }
}
/** An access to a local variable. */
class LocalVariableAccess extends VariableAccess instanceof LocalVariableAccessImpl {
final override string getAPrimaryQlClass() { result = "LocalVariableAccess" }
/**
* Holds if this access is a captured variable access. For example in
*
* ```rb
* def m x
* x.times do |y|
* puts x
* end
* puts x
* end
* ```
*
* the access to `x` in the first `puts x` is a captured access, while
* the access to `x` in the second `puts x` is not.
*/
final predicate isCapturedAccess() { isCapturedAccess(this) }
}
/** An access to a local variable where the value is updated. */
class LocalVariableWriteAccess extends LocalVariableAccess, VariableWriteAccess { }
/** An access to a local variable where the value is read. */
class LocalVariableReadAccess extends LocalVariableAccess, VariableReadAccess { }
/** An access to a global variable. */
class GlobalVariableAccess extends VariableAccess instanceof GlobalVariableAccessImpl {
final override string getAPrimaryQlClass() { result = "GlobalVariableAccess" }
}
/** An access to a global variable where the value is updated. */
class GlobalVariableWriteAccess extends GlobalVariableAccess, VariableWriteAccess { }
/** An access to a global variable where the value is read. */
class GlobalVariableReadAccess extends GlobalVariableAccess, VariableReadAccess { }
/** An access to an instance variable. */
class InstanceVariableAccess extends VariableAccess instanceof InstanceVariableAccessImpl {
final override string getAPrimaryQlClass() { result = "InstanceVariableAccess" }
}
/** An access to an instance variable where the value is updated. */
class InstanceVariableWriteAccess extends InstanceVariableAccess, VariableWriteAccess { }
/** An access to an instance variable where the value is read. */
class InstanceVariableReadAccess extends InstanceVariableAccess, VariableReadAccess { }
/** An access to a class variable. */
class ClassVariableAccess extends VariableAccess instanceof ClassVariableAccessRealImpl {
final override string getAPrimaryQlClass() { result = "ClassVariableAccess" }
}
/** An access to a class variable where the value is updated. */
class ClassVariableWriteAccess extends ClassVariableAccess, VariableWriteAccess { }
/** An access to a class variable where the value is read. */
class ClassVariableReadAccess extends ClassVariableAccess, VariableReadAccess { }

View File

@@ -0,0 +1,699 @@
import codeql.Locations
private import TreeSitter
private import codeql.ruby.ast.internal.Call
private import codeql.ruby.ast.internal.Parameter
private import codeql.ruby.ast.internal.Variable
private import codeql.ruby.AST as AST
private import Synthesis
module MethodName {
predicate range(Ruby::UnderscoreMethodName g) {
exists(Ruby::Undef u | u.getChild(_) = g)
or
exists(Ruby::Alias a | a.getName() = g or a.getAlias() = g)
}
class Token =
@ruby_setter or @ruby_token_class_variable or @ruby_token_constant or
@ruby_token_global_variable or @ruby_token_identifier or @ruby_token_instance_variable or
@ruby_token_operator;
}
private predicate mkSynthChild(SynthKind kind, AST::AstNode parent, int i) {
any(Synthesis s).child(parent, i, SynthChild(kind))
}
cached
private module Cached {
cached
newtype TAstNode =
TAddExprReal(Ruby::Binary g) { g instanceof @ruby_binary_plus } or
TAddExprSynth(AST::AstNode parent, int i) { mkSynthChild(AddExprKind(), parent, i) } or
TAliasStmt(Ruby::Alias g) or
TArgumentList(Ruby::AstNode g) {
(
g.getParent() instanceof Ruby::Break or
g.getParent() instanceof Ruby::Return or
g.getParent() instanceof Ruby::Next or
g.getParent() instanceof Ruby::Assignment or
g.getParent() instanceof Ruby::OperatorAssignment
) and
(
strictcount(g.(Ruby::ArgumentList).getChild(_)) > 1
or
g instanceof Ruby::RightAssignmentList
)
} or
TAssignAddExpr(Ruby::OperatorAssignment g) { g instanceof @ruby_operator_assignment_plusequal } or
TAssignBitwiseAndExpr(Ruby::OperatorAssignment g) {
g instanceof @ruby_operator_assignment_ampersandequal
} or
TAssignBitwiseOrExpr(Ruby::OperatorAssignment g) {
g instanceof @ruby_operator_assignment_pipeequal
} or
TAssignBitwiseXorExpr(Ruby::OperatorAssignment g) {
g instanceof @ruby_operator_assignment_caretequal
} or
TAssignDivExpr(Ruby::OperatorAssignment g) { g instanceof @ruby_operator_assignment_slashequal } or
TAssignExponentExpr(Ruby::OperatorAssignment g) {
g instanceof @ruby_operator_assignment_starstarequal
} or
TAssignExprReal(Ruby::Assignment g) or
TAssignExprSynth(AST::AstNode parent, int i) { mkSynthChild(AssignExprKind(), parent, i) } or
TAssignLShiftExpr(Ruby::OperatorAssignment g) {
g instanceof @ruby_operator_assignment_langlelangleequal
} or
TAssignLogicalAndExpr(Ruby::OperatorAssignment g) {
g instanceof @ruby_operator_assignment_ampersandampersandequal
} or
TAssignLogicalOrExpr(Ruby::OperatorAssignment g) {
g instanceof @ruby_operator_assignment_pipepipeequal
} or
TAssignModuloExpr(Ruby::OperatorAssignment g) {
g instanceof @ruby_operator_assignment_percentequal
} or
TAssignMulExpr(Ruby::OperatorAssignment g) { g instanceof @ruby_operator_assignment_starequal } or
TAssignRShiftExpr(Ruby::OperatorAssignment g) {
g instanceof @ruby_operator_assignment_ranglerangleequal
} or
TAssignSubExpr(Ruby::OperatorAssignment g) { g instanceof @ruby_operator_assignment_minusequal } or
TBareStringLiteral(Ruby::BareString g) or
TBareSymbolLiteral(Ruby::BareSymbol g) or
TBeginBlock(Ruby::BeginBlock g) or
TBeginExpr(Ruby::Begin g) or
TBitwiseAndExprReal(Ruby::Binary g) { g instanceof @ruby_binary_ampersand } or
TBitwiseAndExprSynth(AST::AstNode parent, int i) {
mkSynthChild(BitwiseAndExprKind(), parent, i)
} or
TBitwiseOrExprReal(Ruby::Binary g) { g instanceof @ruby_binary_pipe } or
TBitwiseOrExprSynth(AST::AstNode parent, int i) { mkSynthChild(BitwiseOrExprKind(), parent, i) } or
TBitwiseXorExprReal(Ruby::Binary g) { g instanceof @ruby_binary_caret } or
TBitwiseXorExprSynth(AST::AstNode parent, int i) {
mkSynthChild(BitwiseXorExprKind(), parent, i)
} or
TBlockArgument(Ruby::BlockArgument g) or
TBlockParameter(Ruby::BlockParameter g) or
TBraceBlock(Ruby::Block g) { not g.getParent() instanceof Ruby::Lambda } or
TBreakStmt(Ruby::Break g) or
TCaseEqExpr(Ruby::Binary g) { g instanceof @ruby_binary_equalequalequal } or
TCaseExpr(Ruby::Case g) or
TCharacterLiteral(Ruby::Character g) or
TClassDeclaration(Ruby::Class g) or
TClassVariableAccessReal(Ruby::ClassVariable g, AST::ClassVariable v) {
ClassVariableAccess::range(g, v)
} or
TClassVariableAccessSynth(AST::AstNode parent, int i, AST::ClassVariable v) {
mkSynthChild(ClassVariableAccessKind(v), parent, i)
} or
TComplementExpr(Ruby::Unary g) { g instanceof @ruby_unary_tilde } or
TComplexLiteral(Ruby::Complex g) or
TConstantReadAccessSynth(AST::AstNode parent, int i, string value) {
mkSynthChild(ConstantReadAccessKind(value), parent, i)
} or
TDefinedExpr(Ruby::Unary g) { g instanceof @ruby_unary_definedquestion } or
TDelimitedSymbolLiteral(Ruby::DelimitedSymbol g) or
TDestructuredLeftAssignment(Ruby::DestructuredLeftAssignment g) {
not strictcount(int i | exists(g.getParent().(Ruby::LeftAssignmentList).getChild(i))) = 1
} or
TDivExprReal(Ruby::Binary g) { g instanceof @ruby_binary_slash } or
TDivExprSynth(AST::AstNode parent, int i) { mkSynthChild(DivExprKind(), parent, i) } or
TDo(Ruby::Do g) or
TDoBlock(Ruby::DoBlock g) { not g.getParent() instanceof Ruby::Lambda } or
TElementReference(Ruby::ElementReference g) or
TElse(Ruby::Else g) or
TElsif(Ruby::Elsif g) or
TEmptyStmt(Ruby::EmptyStatement g) or
TEndBlock(Ruby::EndBlock g) or
TEnsure(Ruby::Ensure g) or
TEqExpr(Ruby::Binary g) { g instanceof @ruby_binary_equalequal } or
TExponentExprReal(Ruby::Binary g) { g instanceof @ruby_binary_starstar } or
TExponentExprSynth(AST::AstNode parent, int i) { mkSynthChild(ExponentExprKind(), parent, i) } or
TFalseLiteral(Ruby::False g) or
TFloatLiteral(Ruby::Float g) { not any(Ruby::Rational r).getChild() = g } or
TForExpr(Ruby::For g) or
TForIn(Ruby::In g) or // TODO REMOVE
TGEExpr(Ruby::Binary g) { g instanceof @ruby_binary_rangleequal } or
TGTExpr(Ruby::Binary g) { g instanceof @ruby_binary_rangle } or
TGlobalVariableAccessReal(Ruby::GlobalVariable g, AST::GlobalVariable v) {
GlobalVariableAccess::range(g, v)
} or
TGlobalVariableAccessSynth(AST::AstNode parent, int i, AST::GlobalVariable v) {
mkSynthChild(GlobalVariableAccessKind(v), parent, i)
} or
THashKeySymbolLiteral(Ruby::HashKeySymbol g) or
THashLiteral(Ruby::Hash g) or
THashSplatExpr(Ruby::HashSplatArgument g) or
THashSplatParameter(Ruby::HashSplatParameter g) or
THereDoc(Ruby::HeredocBeginning g) or
TIdentifierMethodCall(Ruby::Identifier g) { isIdentifierMethodCall(g) } or
TIf(Ruby::If g) or
TIfModifierExpr(Ruby::IfModifier g) or
TInstanceVariableAccessReal(Ruby::InstanceVariable g, AST::InstanceVariable v) {
InstanceVariableAccess::range(g, v)
} or
TInstanceVariableAccessSynth(AST::AstNode parent, int i, AST::InstanceVariable v) {
mkSynthChild(InstanceVariableAccessKind(v), parent, i)
} or
TIntegerLiteralReal(Ruby::Integer g) { not any(Ruby::Rational r).getChild() = g } or
TIntegerLiteralSynth(AST::AstNode parent, int i, int value) {
mkSynthChild(IntegerLiteralKind(value), parent, i)
} or
TKeywordParameter(Ruby::KeywordParameter g) or
TLEExpr(Ruby::Binary g) { g instanceof @ruby_binary_langleequal } or
TLShiftExprReal(Ruby::Binary g) { g instanceof @ruby_binary_langlelangle } or
TLShiftExprSynth(AST::AstNode parent, int i) { mkSynthChild(LShiftExprKind(), parent, i) } or
TLTExpr(Ruby::Binary g) { g instanceof @ruby_binary_langle } or
TLambda(Ruby::Lambda g) or
TLeftAssignmentList(Ruby::LeftAssignmentList g) or
TLocalVariableAccessReal(Ruby::Identifier g, AST::LocalVariable v) {
LocalVariableAccess::range(g, v)
} or
TLocalVariableAccessSynth(AST::AstNode parent, int i, AST::LocalVariable v) {
mkSynthChild(LocalVariableAccessRealKind(v), parent, i)
or
mkSynthChild(LocalVariableAccessSynthKind(v), parent, i)
} or
TLogicalAndExprReal(Ruby::Binary g) {
g instanceof @ruby_binary_and or g instanceof @ruby_binary_ampersandampersand
} or
TLogicalAndExprSynth(AST::AstNode parent, int i) {
mkSynthChild(LogicalAndExprKind(), parent, i)
} or
TLogicalOrExprReal(Ruby::Binary g) {
g instanceof @ruby_binary_or or g instanceof @ruby_binary_pipepipe
} or
TLogicalOrExprSynth(AST::AstNode parent, int i) { mkSynthChild(LogicalOrExprKind(), parent, i) } or
TMethod(Ruby::Method g) or
TMethodCallSynth(AST::AstNode parent, int i, string name, boolean setter, int arity) {
mkSynthChild(MethodCallKind(name, setter, arity), parent, i)
} or
TModuleDeclaration(Ruby::Module g) or
TModuloExprReal(Ruby::Binary g) { g instanceof @ruby_binary_percent } or
TModuloExprSynth(AST::AstNode parent, int i) { mkSynthChild(ModuloExprKind(), parent, i) } or
TMulExprReal(Ruby::Binary g) { g instanceof @ruby_binary_star } or
TMulExprSynth(AST::AstNode parent, int i) { mkSynthChild(MulExprKind(), parent, i) } or
TNEExpr(Ruby::Binary g) { g instanceof @ruby_binary_bangequal } or
TNextStmt(Ruby::Next g) or
TNilLiteral(Ruby::Nil g) or
TNoRegExpMatchExpr(Ruby::Binary g) { g instanceof @ruby_binary_bangtilde } or
TNotExpr(Ruby::Unary g) { g instanceof @ruby_unary_bang or g instanceof @ruby_unary_not } or
TOptionalParameter(Ruby::OptionalParameter g) or
TPair(Ruby::Pair g) or
TParenthesizedExpr(Ruby::ParenthesizedStatements g) or
TRShiftExprReal(Ruby::Binary g) { g instanceof @ruby_binary_ranglerangle } or
TRShiftExprSynth(AST::AstNode parent, int i) { mkSynthChild(RShiftExprKind(), parent, i) } or
TRangeLiteralReal(Ruby::Range g) or
TRangeLiteralSynth(AST::AstNode parent, int i, boolean inclusive) {
mkSynthChild(RangeLiteralKind(inclusive), parent, i)
} or
TRationalLiteral(Ruby::Rational g) or
TRedoStmt(Ruby::Redo g) or
TRegExpLiteral(Ruby::Regex g) or
TRegExpMatchExpr(Ruby::Binary g) { g instanceof @ruby_binary_equaltilde } or
TRegularArrayLiteral(Ruby::Array g) or
TRegularMethodCall(Ruby::Call g) { isRegularMethodCall(g) } or
TRegularStringLiteral(Ruby::String g) or
TRegularSuperCall(Ruby::Call g) { g.getMethod() instanceof Ruby::Super } or
TRescueClause(Ruby::Rescue g) or
TRescueModifierExpr(Ruby::RescueModifier g) or
TRetryStmt(Ruby::Retry g) or
TReturnStmt(Ruby::Return g) or
TScopeResolutionConstantAccess(Ruby::ScopeResolution g, Ruby::Constant constant) {
constant = g.getName() and
(
// A tree-sitter `scope_resolution` node with a `constant` name field is a
// read of that constant in any context where an identifier would be a
// vcall.
vcall(g)
or
explicitAssignmentNode(g, _)
)
} or
TScopeResolutionMethodCall(Ruby::ScopeResolution g, Ruby::Identifier i) {
isScopeResolutionMethodCall(g, i)
} or
TSelfReal(Ruby::Self g) or
TSelfSynth(AST::AstNode parent, int i) { mkSynthChild(SelfKind(), parent, i) } or
TSimpleParameter(Ruby::Identifier g) { g instanceof Parameter::Range } or
TSimpleSymbolLiteral(Ruby::SimpleSymbol g) or
TSingletonClass(Ruby::SingletonClass g) or
TSingletonMethod(Ruby::SingletonMethod g) or
TSpaceshipExpr(Ruby::Binary g) { g instanceof @ruby_binary_langleequalrangle } or
TSplatExprReal(Ruby::SplatArgument g) or
TSplatExprSynth(AST::AstNode parent, int i) { mkSynthChild(SplatExprKind(), parent, i) } or
TSplatParameter(Ruby::SplatParameter g) or
TStmtSequenceSynth(AST::AstNode parent, int i) { mkSynthChild(StmtSequenceKind(), parent, i) } or
TStringArrayLiteral(Ruby::StringArray g) or
TStringConcatenation(Ruby::ChainedString g) or
TStringEscapeSequenceComponent(Ruby::EscapeSequence g) or
TStringInterpolationComponent(Ruby::Interpolation g) or
TStringTextComponent(Ruby::Token g) {
g instanceof Ruby::StringContent or g instanceof Ruby::HeredocContent
} or
TSubExprReal(Ruby::Binary g) { g instanceof @ruby_binary_minus } or
TSubExprSynth(AST::AstNode parent, int i) { mkSynthChild(SubExprKind(), parent, i) } or
TSubshellLiteral(Ruby::Subshell g) or
TSymbolArrayLiteral(Ruby::SymbolArray g) or
TTernaryIfExpr(Ruby::Conditional g) or
TThen(Ruby::Then g) or
TTokenConstantAccess(Ruby::Constant g) {
// A tree-sitter `constant` token is a read of that constant in any context
// where an identifier would be a vcall.
vcall(g)
or
explicitAssignmentNode(g, _)
} or
TTokenMethodName(MethodName::Token g) { MethodName::range(g) } or
TTokenSuperCall(Ruby::Super g) { vcall(g) } or
TToplevel(Ruby::Program g) or
TTrueLiteral(Ruby::True g) or
TTuplePatternParameter(Ruby::DestructuredParameter g) or
TUnaryMinusExpr(Ruby::Unary g) { g instanceof @ruby_unary_minus } or
TUnaryPlusExpr(Ruby::Unary g) { g instanceof @ruby_unary_plus } or
TUndefStmt(Ruby::Undef g) or
TUnlessExpr(Ruby::Unless g) or
TUnlessModifierExpr(Ruby::UnlessModifier g) or
TUntilExpr(Ruby::Until g) or
TUntilModifierExpr(Ruby::UntilModifier g) or
TWhenExpr(Ruby::When g) or
TWhileExpr(Ruby::While g) or
TWhileModifierExpr(Ruby::WhileModifier g) or
TYieldCall(Ruby::Yield g)
/**
* Gets the underlying TreeSitter entity for a given AST node. This does not
* include synthesized AST nodes, because they are not the primary AST node
* for any given generated node.
*/
cached
Ruby::AstNode toGenerated(AST::AstNode n) {
n = TAddExprReal(result) or
n = TAliasStmt(result) or
n = TArgumentList(result) or
n = TAssignAddExpr(result) or
n = TAssignBitwiseAndExpr(result) or
n = TAssignBitwiseOrExpr(result) or
n = TAssignBitwiseXorExpr(result) or
n = TAssignDivExpr(result) or
n = TAssignExponentExpr(result) or
n = TAssignExprReal(result) or
n = TAssignLShiftExpr(result) or
n = TAssignLogicalAndExpr(result) or
n = TAssignLogicalOrExpr(result) or
n = TAssignModuloExpr(result) or
n = TAssignMulExpr(result) or
n = TAssignRShiftExpr(result) or
n = TAssignSubExpr(result) or
n = TBareStringLiteral(result) or
n = TBareSymbolLiteral(result) or
n = TBeginBlock(result) or
n = TBeginExpr(result) or
n = TBitwiseAndExprReal(result) or
n = TBitwiseOrExprReal(result) or
n = TBitwiseXorExprReal(result) or
n = TBlockArgument(result) or
n = TBlockParameter(result) or
n = TBraceBlock(result) or
n = TBreakStmt(result) or
n = TCaseEqExpr(result) or
n = TCaseExpr(result) or
n = TCharacterLiteral(result) or
n = TClassDeclaration(result) or
n = TClassVariableAccessReal(result, _) or
n = TComplementExpr(result) or
n = TComplexLiteral(result) or
n = TDefinedExpr(result) or
n = TDelimitedSymbolLiteral(result) or
n = TDestructuredLeftAssignment(result) or
n = TDivExprReal(result) or
n = TDo(result) or
n = TDoBlock(result) or
n = TElementReference(result) or
n = TElse(result) or
n = TElsif(result) or
n = TEmptyStmt(result) or
n = TEndBlock(result) or
n = TEnsure(result) or
n = TEqExpr(result) or
n = TExponentExprReal(result) or
n = TFalseLiteral(result) or
n = TFloatLiteral(result) or
n = TForExpr(result) or
n = TForIn(result) or // TODO REMOVE
n = TGEExpr(result) or
n = TGTExpr(result) or
n = TGlobalVariableAccessReal(result, _) or
n = THashKeySymbolLiteral(result) or
n = THashLiteral(result) or
n = THashSplatExpr(result) or
n = THashSplatParameter(result) or
n = THereDoc(result) or
n = TIdentifierMethodCall(result) or
n = TIf(result) or
n = TIfModifierExpr(result) or
n = TInstanceVariableAccessReal(result, _) or
n = TIntegerLiteralReal(result) or
n = TKeywordParameter(result) or
n = TLEExpr(result) or
n = TLShiftExprReal(result) or
n = TLTExpr(result) or
n = TLambda(result) or
n = TLeftAssignmentList(result) or
n = TLocalVariableAccessReal(result, _) or
n = TLogicalAndExprReal(result) or
n = TLogicalOrExprReal(result) or
n = TMethod(result) or
n = TModuleDeclaration(result) or
n = TModuloExprReal(result) or
n = TMulExprReal(result) or
n = TNEExpr(result) or
n = TNextStmt(result) or
n = TNilLiteral(result) or
n = TNoRegExpMatchExpr(result) or
n = TNotExpr(result) or
n = TOptionalParameter(result) or
n = TPair(result) or
n = TParenthesizedExpr(result) or
n = TRShiftExprReal(result) or
n = TRangeLiteralReal(result) or
n = TRationalLiteral(result) or
n = TRedoStmt(result) or
n = TRegExpLiteral(result) or
n = TRegExpMatchExpr(result) or
n = TRegularArrayLiteral(result) or
n = TRegularMethodCall(result) or
n = TRegularStringLiteral(result) or
n = TRegularSuperCall(result) or
n = TRescueClause(result) or
n = TRescueModifierExpr(result) or
n = TRetryStmt(result) or
n = TReturnStmt(result) or
n = TScopeResolutionConstantAccess(result, _) or
n = TScopeResolutionMethodCall(result, _) or
n = TSelfReal(result) or
n = TSimpleParameter(result) or
n = TSimpleSymbolLiteral(result) or
n = TSingletonClass(result) or
n = TSingletonMethod(result) or
n = TSpaceshipExpr(result) or
n = TSplatExprReal(result) or
n = TSplatParameter(result) or
n = TStringArrayLiteral(result) or
n = TStringConcatenation(result) or
n = TStringEscapeSequenceComponent(result) or
n = TStringInterpolationComponent(result) or
n = TStringTextComponent(result) or
n = TSubExprReal(result) or
n = TSubshellLiteral(result) or
n = TSymbolArrayLiteral(result) or
n = TTernaryIfExpr(result) or
n = TThen(result) or
n = TTokenConstantAccess(result) or
n = TTokenMethodName(result) or
n = TTokenSuperCall(result) or
n = TToplevel(result) or
n = TTrueLiteral(result) or
n = TTuplePatternParameter(result) or
n = TUnaryMinusExpr(result) or
n = TUnaryPlusExpr(result) or
n = TUndefStmt(result) or
n = TUnlessExpr(result) or
n = TUnlessModifierExpr(result) or
n = TUntilExpr(result) or
n = TUntilModifierExpr(result) or
n = TWhenExpr(result) or
n = TWhileExpr(result) or
n = TWhileModifierExpr(result) or
n = TYieldCall(result)
}
/** Gets the `i`th synthesized child of `parent`. */
cached
AST::AstNode getSynthChild(AST::AstNode parent, int i) {
result = TAddExprSynth(parent, i)
or
result = TAssignExprSynth(parent, i)
or
result = TBitwiseAndExprSynth(parent, i)
or
result = TBitwiseOrExprSynth(parent, i)
or
result = TBitwiseXorExprSynth(parent, i)
or
result = TClassVariableAccessSynth(parent, i, _)
or
result = TConstantReadAccessSynth(parent, i, _)
or
result = TDivExprSynth(parent, i)
or
result = TExponentExprSynth(parent, i)
or
result = TGlobalVariableAccessSynth(parent, i, _)
or
result = TInstanceVariableAccessSynth(parent, i, _)
or
result = TIntegerLiteralSynth(parent, i, _)
or
result = TLShiftExprSynth(parent, i)
or
result = TLocalVariableAccessSynth(parent, i, _)
or
result = TLogicalAndExprSynth(parent, i)
or
result = TLogicalOrExprSynth(parent, i)
or
result = TMethodCallSynth(parent, i, _, _, _)
or
result = TModuloExprSynth(parent, i)
or
result = TMulExprSynth(parent, i)
or
result = TRangeLiteralSynth(parent, i, _)
or
result = TRShiftExprSynth(parent, i)
or
result = TSelfSynth(parent, i)
or
result = TSplatExprSynth(parent, i)
or
result = TStmtSequenceSynth(parent, i)
or
result = TSubExprSynth(parent, i)
}
/**
* Holds if the `i`th child of `parent` is `child`. Either `parent` or
* `child` (or both) is a synthesized node.
*/
cached
predicate synthChild(AST::AstNode parent, int i, AST::AstNode child) {
child = getSynthChild(parent, i)
or
any(Synthesis s).child(parent, i, RealChild(child))
}
/**
* Like `toGenerated`, but also returns generated nodes for synthesized AST
* nodes.
*/
cached
Ruby::AstNode toGeneratedInclSynth(AST::AstNode n) {
result = toGenerated(n)
or
not exists(toGenerated(n)) and
exists(AST::AstNode parent |
synthChild(parent, _, n) and
result = toGeneratedInclSynth(parent)
)
}
cached
Location getLocation(AST::AstNode n) {
synthLocation(n, result)
or
n.isSynthesized() and
not synthLocation(n, _) and
result = getLocation(n.getParent())
or
result = toGenerated(n).getLocation()
}
}
import Cached
TAstNode fromGenerated(Ruby::AstNode n) { n = toGenerated(result) }
class TCall = TMethodCall or TYieldCall;
class TMethodCall =
TMethodCallSynth or TIdentifierMethodCall or TScopeResolutionMethodCall or TRegularMethodCall or
TElementReference or TSuperCall or TUnaryOperation or TBinaryOperation;
class TSuperCall = TTokenSuperCall or TRegularSuperCall;
class TConstantAccess =
TTokenConstantAccess or TScopeResolutionConstantAccess or TNamespace or TConstantReadAccessSynth;
class TControlExpr = TConditionalExpr or TCaseExpr or TLoop;
class TConditionalExpr =
TIfExpr or TUnlessExpr or TIfModifierExpr or TUnlessModifierExpr or TTernaryIfExpr;
class TIfExpr = TIf or TElsif;
class TConditionalLoop = TWhileExpr or TUntilExpr or TWhileModifierExpr or TUntilModifierExpr;
class TLoop = TConditionalLoop or TForExpr;
class TSelf = TSelfReal or TSelfSynth;
class TExpr =
TSelf or TArgumentList or TRescueClause or TRescueModifierExpr or TPair or TStringConcatenation or
TCall or TBlockArgument or TConstantAccess or TControlExpr or TWhenExpr or TLiteral or
TCallable or TVariableAccess or TStmtSequence or TOperation or TSimpleParameter;
class TSplatExpr = TSplatExprReal or TSplatExprSynth;
class TStmtSequence =
TBeginBlock or TEndBlock or TThen or TElse or TDo or TEnsure or TStringInterpolationComponent or
TBlock or TBodyStmt or TParenthesizedExpr or TStmtSequenceSynth;
class TBodyStmt = TBeginExpr or TModuleBase or TMethod or TLambda or TDoBlock or TSingletonMethod;
class TLiteral =
TNumericLiteral or TNilLiteral or TBooleanLiteral or TStringlikeLiteral or TCharacterLiteral or
TArrayLiteral or THashLiteral or TRangeLiteral or TTokenMethodName;
class TNumericLiteral = TIntegerLiteral or TFloatLiteral or TRationalLiteral or TComplexLiteral;
class TIntegerLiteral = TIntegerLiteralReal or TIntegerLiteralSynth;
class TBooleanLiteral = TTrueLiteral or TFalseLiteral;
class TStringComponent =
TStringTextComponent or TStringEscapeSequenceComponent or TStringInterpolationComponent;
class TStringlikeLiteral =
TStringLiteral or TRegExpLiteral or TSymbolLiteral or TSubshellLiteral or THereDoc;
class TStringLiteral = TRegularStringLiteral or TBareStringLiteral;
class TSymbolLiteral = TSimpleSymbolLiteral or TComplexSymbolLiteral or THashKeySymbolLiteral;
class TComplexSymbolLiteral = TDelimitedSymbolLiteral or TBareSymbolLiteral;
class TArrayLiteral = TRegularArrayLiteral or TStringArrayLiteral or TSymbolArrayLiteral;
class TCallable = TMethodBase or TLambda or TBlock;
class TMethodBase = TMethod or TSingletonMethod;
class TBlock = TDoBlock or TBraceBlock;
class TModuleBase = TToplevel or TNamespace or TSingletonClass;
class TNamespace = TClassDeclaration or TModuleDeclaration;
class TOperation = TUnaryOperation or TBinaryOperation or TAssignment;
class TUnaryOperation =
TUnaryLogicalOperation or TUnaryArithmeticOperation or TUnaryBitwiseOperation or TDefinedExpr or
TSplatExpr or THashSplatExpr;
class TUnaryLogicalOperation = TNotExpr;
class TUnaryArithmeticOperation = TUnaryPlusExpr or TUnaryMinusExpr;
class TUnaryBitwiseOperation = TComplementExpr;
class TBinaryOperation =
TBinaryArithmeticOperation or TBinaryLogicalOperation or TBinaryBitwiseOperation or
TComparisonOperation or TSpaceshipExpr or TRegExpMatchExpr or TNoRegExpMatchExpr;
class TBinaryArithmeticOperation =
TAddExpr or TSubExpr or TMulExpr or TDivExpr or TModuloExpr or TExponentExpr;
class TAddExpr = TAddExprReal or TAddExprSynth;
class TSubExpr = TSubExprReal or TSubExprSynth;
class TMulExpr = TMulExprReal or TMulExprSynth;
class TDivExpr = TDivExprReal or TDivExprSynth;
class TModuloExpr = TModuloExprReal or TModuloExprSynth;
class TExponentExpr = TExponentExprReal or TExponentExprSynth;
class TBinaryLogicalOperation = TLogicalAndExpr or TLogicalOrExpr;
class TLogicalAndExpr = TLogicalAndExprReal or TLogicalAndExprSynth;
class TLogicalOrExpr = TLogicalOrExprReal or TLogicalOrExprSynth;
class TBinaryBitwiseOperation =
TLShiftExpr or TRShiftExpr or TBitwiseAndExpr or TBitwiseOrExpr or TBitwiseXorExpr;
class TLShiftExpr = TLShiftExprReal or TLShiftExprSynth;
class TRangeLiteral = TRangeLiteralReal or TRangeLiteralSynth;
class TRShiftExpr = TRShiftExprReal or TRShiftExprSynth;
class TBitwiseAndExpr = TBitwiseAndExprReal or TBitwiseAndExprSynth;
class TBitwiseOrExpr = TBitwiseOrExprReal or TBitwiseOrExprSynth;
class TBitwiseXorExpr = TBitwiseXorExprReal or TBitwiseXorExprSynth;
class TComparisonOperation = TEqualityOperation or TRelationalOperation;
class TEqualityOperation = TEqExpr or TNEExpr or TCaseEqExpr;
class TRelationalOperation = TGTExpr or TGEExpr or TLTExpr or TLEExpr;
class TAssignExpr = TAssignExprReal or TAssignExprSynth;
class TAssignment = TAssignExpr or TAssignOperation;
class TAssignOperation =
TAssignArithmeticOperation or TAssignLogicalOperation or TAssignBitwiseOperation;
class TAssignArithmeticOperation =
TAssignAddExpr or TAssignSubExpr or TAssignMulExpr or TAssignDivExpr or TAssignModuloExpr or
TAssignExponentExpr;
class TAssignLogicalOperation = TAssignLogicalAndExpr or TAssignLogicalOrExpr;
class TAssignBitwiseOperation =
TAssignLShiftExpr or TAssignRShiftExpr or TAssignBitwiseAndExpr or TAssignBitwiseOrExpr or
TAssignBitwiseXorExpr;
class TStmt =
TEmptyStmt or TBodyStmt or TStmtSequence or TUndefStmt or TAliasStmt or TReturningStmt or
TRedoStmt or TRetryStmt or TExpr;
class TReturningStmt = TReturnStmt or TBreakStmt or TNextStmt;
class TParameter =
TPatternParameter or TBlockParameter or THashSplatParameter or TKeywordParameter or
TOptionalParameter or TSplatParameter;
class TPatternParameter = TSimpleParameter or TTuplePatternParameter;
class TNamedParameter =
TSimpleParameter or TBlockParameter or THashSplatParameter or TKeywordParameter or
TOptionalParameter or TSplatParameter;
class TTuplePattern = TTuplePatternParameter or TDestructuredLeftAssignment or TLeftAssignmentList;
class TVariableAccess =
TLocalVariableAccess or TGlobalVariableAccess or TInstanceVariableAccess or TClassVariableAccess;
class TLocalVariableAccess = TLocalVariableAccessReal or TLocalVariableAccessSynth;
class TGlobalVariableAccess = TGlobalVariableAccessReal or TGlobalVariableAccessSynth;
class TInstanceVariableAccess = TInstanceVariableAccessReal or TInstanceVariableAccessSynth;
class TClassVariableAccess = TClassVariableAccessReal or TClassVariableAccessSynth;

View File

@@ -0,0 +1,186 @@
private import TreeSitter
private import Variable
private import codeql.ruby.AST
private import codeql.ruby.ast.internal.AST
predicate isIdentifierMethodCall(Ruby::Identifier g) { vcall(g) and not access(g, _) }
predicate isRegularMethodCall(Ruby::Call g) { not g.getMethod() instanceof Ruby::Super }
predicate isScopeResolutionMethodCall(Ruby::ScopeResolution g, Ruby::Identifier i) {
i = g.getName() and
not exists(Ruby::Call c | c.getMethod() = g)
}
abstract class CallImpl extends Expr, TCall {
abstract AstNode getArgumentImpl(int n);
/**
* It is not possible to define this predicate as
*
* ```ql
* result = count(this.getArgumentImpl(_))
* ```
*
* since that will result in a non-monotonicity error.
*/
abstract int getNumberOfArgumentsImpl();
}
abstract class MethodCallImpl extends CallImpl, TMethodCall {
abstract AstNode getReceiverImpl();
abstract string getMethodNameImpl();
abstract Block getBlockImpl();
}
class MethodCallSynth extends MethodCallImpl, TMethodCallSynth {
final override string getMethodNameImpl() {
exists(boolean setter, string name | this = TMethodCallSynth(_, _, name, setter, _) |
setter = true and result = name + "="
or
setter = false and result = name
)
}
final override AstNode getReceiverImpl() { synthChild(this, 0, result) }
final override AstNode getArgumentImpl(int n) { synthChild(this, n + 1, result) and n >= 0 }
final override int getNumberOfArgumentsImpl() { this = TMethodCallSynth(_, _, _, _, result) }
final override Block getBlockImpl() { none() }
}
class IdentifierMethodCall extends MethodCallImpl, TIdentifierMethodCall {
private Ruby::Identifier g;
IdentifierMethodCall() { this = TIdentifierMethodCall(g) }
final override string getMethodNameImpl() { result = g.getValue() }
final override AstNode getReceiverImpl() { result = TSelfSynth(this, 0) }
final override Expr getArgumentImpl(int n) { none() }
final override int getNumberOfArgumentsImpl() { result = 0 }
final override Block getBlockImpl() { none() }
}
class ScopeResolutionMethodCall extends MethodCallImpl, TScopeResolutionMethodCall {
private Ruby::ScopeResolution g;
private Ruby::Identifier i;
ScopeResolutionMethodCall() { this = TScopeResolutionMethodCall(g, i) }
final override string getMethodNameImpl() { result = i.getValue() }
final override Expr getReceiverImpl() { toGenerated(result) = g.getScope() }
final override Expr getArgumentImpl(int n) { none() }
final override int getNumberOfArgumentsImpl() { result = 0 }
final override Block getBlockImpl() { none() }
}
class RegularMethodCall extends MethodCallImpl, TRegularMethodCall {
private Ruby::Call g;
RegularMethodCall() { this = TRegularMethodCall(g) }
final override Expr getReceiverImpl() {
toGenerated(result) = g.getReceiver()
or
not exists(g.getReceiver()) and
toGenerated(result) = g.getMethod().(Ruby::ScopeResolution).getScope()
or
result = TSelfSynth(this, 0)
}
final override string getMethodNameImpl() {
isRegularMethodCall(g) and
(
result = "call" and g.getMethod() instanceof Ruby::ArgumentList
or
result = g.getMethod().(Ruby::Token).getValue()
or
result = g.getMethod().(Ruby::ScopeResolution).getName().(Ruby::Token).getValue()
)
}
final override Expr getArgumentImpl(int n) {
toGenerated(result) = g.getArguments().getChild(n)
or
toGenerated(result) = g.getMethod().(Ruby::ArgumentList).getChild(n)
}
final override int getNumberOfArgumentsImpl() {
result =
count(g.getArguments().getChild(_)) + count(g.getMethod().(Ruby::ArgumentList).getChild(_))
}
final override Block getBlockImpl() { toGenerated(result) = g.getBlock() }
}
class ElementReferenceImpl extends MethodCallImpl, TElementReference {
private Ruby::ElementReference g;
ElementReferenceImpl() { this = TElementReference(g) }
final override Expr getReceiverImpl() { toGenerated(result) = g.getObject() }
final override Expr getArgumentImpl(int n) { toGenerated(result) = g.getChild(n) }
final override int getNumberOfArgumentsImpl() { result = count(g.getChild(_)) }
final override string getMethodNameImpl() { result = "[]" }
final override Block getBlockImpl() { none() }
}
abstract class SuperCallImpl extends MethodCallImpl, TSuperCall { }
class TokenSuperCall extends SuperCallImpl, TTokenSuperCall {
private Ruby::Super g;
TokenSuperCall() { this = TTokenSuperCall(g) }
final override string getMethodNameImpl() { result = g.getValue() }
final override Expr getReceiverImpl() { none() }
final override Expr getArgumentImpl(int n) { none() }
final override int getNumberOfArgumentsImpl() { result = 0 }
final override Block getBlockImpl() { none() }
}
class RegularSuperCall extends SuperCallImpl, TRegularSuperCall {
private Ruby::Call g;
RegularSuperCall() { this = TRegularSuperCall(g) }
final override string getMethodNameImpl() { result = g.getMethod().(Ruby::Super).getValue() }
final override Expr getReceiverImpl() { none() }
final override Expr getArgumentImpl(int n) { toGenerated(result) = g.getArguments().getChild(n) }
final override int getNumberOfArgumentsImpl() { result = count(g.getArguments().getChild(_)) }
final override Block getBlockImpl() { toGenerated(result) = g.getBlock() }
}
class YieldCallImpl extends CallImpl, TYieldCall {
Ruby::Yield g;
YieldCallImpl() { this = TYieldCall(g) }
final override Expr getArgumentImpl(int n) { toGenerated(result) = g.getChild().getChild(n) }
final override int getNumberOfArgumentsImpl() { result = count(g.getChild().getChild(_)) }
}

View File

@@ -0,0 +1,43 @@
import codeql.Locations
private import TreeSitter
private import codeql.ruby.ast.Erb
cached
private module Cached {
cached
newtype TAstNode =
TCommentDirective(Erb::CommentDirective g) or
TDirective(Erb::Directive g) or
TGraphqlDirective(Erb::GraphqlDirective g) or
TOutputDirective(Erb::OutputDirective g) or
TTemplate(Erb::Template g) or
TToken(Erb::Token g) or
TComment(Erb::Comment g) or
TCode(Erb::Code g)
/**
* Gets the underlying TreeSitter entity for a given erb AST node.
*/
cached
Erb::AstNode toGenerated(ErbAstNode n) {
n = TCommentDirective(result) or
n = TDirective(result) or
n = TGraphqlDirective(result) or
n = TOutputDirective(result) or
n = TTemplate(result) or
n = TToken(result) or
n = TComment(result) or
n = TCode(result)
}
cached
Location getLocation(ErbAstNode n) { result = toGenerated(n).getLocation() }
}
import Cached
TAstNode fromGenerated(Erb::AstNode n) { n = toGenerated(result) }
class TDirectiveNode = TCommentDirective or TDirective or TGraphqlDirective or TOutputDirective;
class TTokenNode = TToken or TComment or TCode;

View File

@@ -0,0 +1,409 @@
private import codeql.Locations
private import codeql.ruby.AST
private import codeql.ruby.ast.Call
private import codeql.ruby.ast.Constant
private import codeql.ruby.ast.Expr
private import codeql.ruby.ast.Module
private import codeql.ruby.ast.Operation
private import codeql.ruby.ast.Scope
// Names of built-in modules and classes
private string builtin() {
result =
[
"Object", "Kernel", "BasicObject", "Class", "Module", "NilClass", "FalseClass", "TrueClass",
"Numeric", "Integer", "Float", "Rational", "Complex", "Array", "Hash", "Symbol", "Proc"
]
}
cached
private module Cached {
cached
newtype TModule =
TResolved(string qName) {
qName = builtin()
or
qName = namespaceDeclaration(_)
} or
TUnresolved(Namespace n) { not exists(namespaceDeclaration(n)) }
cached
string namespaceDeclaration(Namespace n) {
isToplevel(n) and result = n.getName()
or
not isToplevel(n) and
not exists(n.getScopeExpr()) and
result = scopeAppend(namespaceDeclaration(n.getEnclosingModule()), n.getName())
or
exists(string container |
TResolved(container) = resolveScopeExpr(n.getScopeExpr()) and
result = scopeAppend(container, n.getName())
)
}
cached
Module getSuperClass(Module cls) {
cls = TResolved("Object") and result = TResolved("BasicObject")
or
cls = TResolved(["Module", "Numeric", "Array", "Hash", "FalseClass", "TrueClass", "NilClass"]) and
result = TResolved("Object")
or
cls = TResolved(["Integer", "Float", "Rational", "Complex"]) and
result = TResolved("Numeric")
or
cls = TResolved("Class") and
result = TResolved("Module")
or
not cls = TResolved(builtin()) and
(
exists(ClassDeclaration d |
d = cls.getADeclaration() and
result = resolveScopeExpr(d.getSuperclassExpr())
)
or
result = TResolved("Object") and
forex(ClassDeclaration d | d = cls.getADeclaration() |
not exists(resolveScopeExpr(d.getSuperclassExpr()))
)
)
}
cached
Module getAnIncludedModule(Module m) {
m = TResolved("Object") and result = TResolved("Kernel")
or
exists(IncludeOrPrependCall c |
c.getMethodName() = "include" and
(
m = resolveScopeExpr(c.getReceiver())
or
m = enclosingModule(c).getModule() and
c.getReceiver() instanceof Self
) and
result = resolveScopeExpr(c.getAnArgument())
)
}
cached
Module getAPrependedModule(Module m) {
exists(IncludeOrPrependCall c |
c.getMethodName() = "prepend" and
(
m = resolveScopeExpr(c.getReceiver())
or
m = enclosingModule(c).getModule() and
c.getReceiver() instanceof Self
) and
result = resolveScopeExpr(c.getAnArgument())
)
}
/**
* Resolve class or module read access to a qualified module name.
*/
cached
TResolved resolveScopeExpr(ConstantReadAccess r) {
exists(string qname | qname = resolveConstant(r) and result = TResolved(qname))
}
/**
* Resolve constant access (class, module or otherwise) to a qualified module name.
* `resolveScopeExpr/1` picks the best (lowest priority number) result of
* `resolveScopeExpr/2` that resolves to a constant definition. If the constant
* definition is a Namespace then it is returned, if it's a constant assignment then
* the right-hand side of the assignment is resolved.
*/
cached
string resolveConstant(ConstantReadAccess r) {
exists(string qname |
qname =
min(string qn, int p |
isDefinedConstant(qn) and
qn = resolveScopeExpr(r, p) and
// prevent classes/modules that contain/extend themselves
not exists(ConstantWriteAccess w | qn = constantDefinition0(w) |
r = w.getScopeExpr()
or
r = w.(ClassDeclaration).getSuperclassExpr()
)
|
qn order by p
)
|
result = qname
or
exists(ConstantAssignment a |
qname = constantDefinition0(a) and
result = resolveConstant(a.getParent().(Assignment).getRightOperand())
)
)
}
cached
Method lookupMethod(Module m, string name) { TMethod(result) = lookupMethodOrConst(m, name) }
cached
Expr lookupConst(Module m, string name) {
TExpr(result) = lookupMethodOrConst(m, name)
or
exists(AssignExpr ae, ConstantWriteAccess w |
w = ae.getLeftOperand() and
w.getName() = name and
m = resolveScopeExpr(w.getScopeExpr()) and
result = ae.getRightOperand()
)
}
}
import Cached
private predicate isToplevel(ConstantAccess n) {
not exists(n.getScopeExpr()) and
(
n.hasGlobalScope()
or
n.getEnclosingModule() instanceof Toplevel
)
}
private predicate isDefinedConstant(string qualifiedModuleName) {
qualifiedModuleName = [builtin(), constantDefinition0(_)]
}
private int maxDepth() { result = 1 + max(int level | exists(enclosing(_, level))) }
private ModuleBase enclosing(ModuleBase m, int level) {
result = m and level = 0
or
result = enclosing(m.getEnclosingModule(), level - 1)
}
pragma[noinline]
private Namespace enclosingNameSpaceConstantReadAccess(
ConstantReadAccess c, int priority, string name
) {
result = enclosing(c.getEnclosingModule(), priority) and
name = c.getName()
}
/**
* Resolve constant read access (typically a scope expression) to a qualified name. The
* `priority` value indicates the precedence of the solution with respect to the lookup order.
* A constant name without scope specifier is resolved against its enclosing modules (inner-most first);
* if the constant is not found in any of the enclosing modules, then the constant will be resolved
* with respect to the ancestors (prepends, includes, super classes, and their ancestors) of the
* directly enclosing module.
*/
private string resolveScopeExpr(ConstantReadAccess c, int priority) {
c.hasGlobalScope() and result = c.getName() and priority = 0
or
exists(string name |
result = qualifiedModuleName(resolveScopeExprConstantReadAccess(c, priority, name), name)
)
or
not exists(c.getScopeExpr()) and
not c.hasGlobalScope() and
(
exists(string name |
exists(Namespace n |
n = enclosingNameSpaceConstantReadAccess(c, priority, name) and
result = qualifiedModuleName(constantDefinition0(n), name)
)
or
result =
qualifiedModuleName(ancestors(qualifiedModuleNameConstantReadAccess(c, name),
priority - maxDepth()), name)
)
or
priority = maxDepth() + 4 and
qualifiedModuleNameConstantReadAccess(c, result) != "BasicObject"
)
}
pragma[nomagic]
private string resolveScopeExprConstantReadAccess(ConstantReadAccess c, int priority, string name) {
result = resolveScopeExpr(c.getScopeExpr(), priority) and
name = c.getName()
}
bindingset[qualifier, name]
private string scopeAppend(string qualifier, string name) {
if qualifier = "Object" then result = name else result = qualifier + "::" + name
}
private string qualifiedModuleName(ModuleBase m) {
result = "Object" and m instanceof Toplevel
or
result = constantDefinition0(m)
}
pragma[noinline]
private string qualifiedModuleNameConstantWriteAccess(ConstantWriteAccess c, string name) {
result = qualifiedModuleName(c.getEnclosingModule()) and
name = c.getName()
}
pragma[noinline]
private string qualifiedModuleNameConstantReadAccess(ConstantReadAccess c, string name) {
result = qualifiedModuleName(c.getEnclosingModule()) and
name = c.getName()
}
/**
* Get a qualified name for a constant definition. May return multiple qualified
* names because we over-approximate when resolving scope resolutions and ignore
* lookup order precedence. Taking lookup order into account here would lead to
* non-monotonic recursion.
*/
private string constantDefinition0(ConstantWriteAccess c) {
c.hasGlobalScope() and result = c.getName()
or
result = scopeAppend(resolveScopeExpr(c.getScopeExpr(), _), c.getName())
or
not exists(c.getScopeExpr()) and
not c.hasGlobalScope() and
exists(string name | result = scopeAppend(qualifiedModuleNameConstantWriteAccess(c, name), name))
}
/**
* The qualified names of the ancestors of a class/module. The ancestors should be an ordered list
* of the ancestores of `prepend`ed modules, the module itself , the ancestors or `include`d modules
* and the ancestors of the super class. The priority value only distinguishes the kind of ancestor,
* it does not order the ancestors within a group of the same kind. This is an over-approximation, however,
* computing the precise order is tricky because it depends on the evaluation/file loading order.
*/
// TODO: the order of super classes can be determined more precisely even without knowing the evaluation
// order, so we should be able to make this more precise.
private string ancestors(string qname, int priority) {
result = ancestors(prepends(qname), _) and priority = 0
or
result = qname and priority = 1 and isDefinedConstant(qname)
or
result = ancestors(includes(qname), _) and priority = 2
or
result = ancestors(superclass(qname), _) and priority = 3
}
private class IncludeOrPrependCall extends MethodCall {
IncludeOrPrependCall() { this.getMethodName() = ["include", "prepend"] }
string getAModule() { result = resolveScopeExpr(this.getAnArgument(), _) }
string getTarget() {
result = resolveScopeExpr(this.getReceiver(), _)
or
result = qualifiedModuleName(enclosingModule(this)) and
(
this.getReceiver() instanceof Self
or
not exists(this.getReceiver())
)
}
}
/**
* A variant of AstNode::getEnclosingModule that excludes
* results that are enclosed in a block. This is a bit wrong because
* it could lead to false negatives. However, `include` statements in
* blocks are very rare in normal code. The majority of cases are in calls
* to methods like `module_eval` and `Rspec.describe` / `Rspec.context`. These
* methods evaluate the block in the context of some other module/class instead of
* the enclosing one.
*/
private ModuleBase enclosingModule(AstNode node) { result = parent*(node).getParent() }
private AstNode parent(AstNode n) {
result = n.getParent() and
not result instanceof ModuleBase and
not result instanceof Block
}
private string prepends(string qname) {
exists(IncludeOrPrependCall m |
m.getMethodName() = "prepend" and
qname = m.getTarget() and
result = m.getAModule()
)
}
private string includes(string qname) {
qname = "Object" and
result = "Kernel"
or
exists(IncludeOrPrependCall m |
m.getMethodName() = "include" and
qname = m.getTarget() and
result = m.getAModule()
)
}
private Expr superexpr(string qname) {
exists(ClassDeclaration c | qname = constantDefinition0(c) and result = c.getSuperclassExpr())
}
private string superclass(string qname) {
qname = "Object" and result = "BasicObject"
or
result = resolveScopeExpr(superexpr(qname), _)
}
private string qualifiedModuleName(string container, string name) {
isDefinedConstant(result) and
(
container = result.regexpCapture("(.+)::([^:]+)", 1) and
name = result.regexpCapture("(.+)::([^:]+)", 2)
or
container = "Object" and name = result
)
}
private Module getAncestors(Module m) {
result = m or
result = getAncestors(m.getAnIncludedModule()) or
result = getAncestors(m.getAPrependedModule())
}
private newtype TMethodOrExpr =
TMethod(Method m) or
TExpr(Expr e)
private TMethodOrExpr getMethodOrConst(TModule owner, string name) {
exists(ModuleBase m | m.getModule() = owner |
result = TMethod(m.getMethod(name))
or
result = TExpr(m.getConstant(name))
)
}
module ExposedForTestingOnly {
Method getMethod(TModule owner, string name) { TMethod(result) = getMethodOrConst(owner, name) }
Expr getConst(TModule owner, string name) { TExpr(result) = getMethodOrConst(owner, name) }
}
private TMethodOrExpr lookupMethodOrConst0(Module m, string name) {
result = lookupMethodOrConst0(m.getAPrependedModule(), name)
or
not exists(getMethodOrConst(getAncestors(m.getAPrependedModule()), name)) and
(
result = getMethodOrConst(m, name)
or
not exists(getMethodOrConst(m, name)) and
result = lookupMethodOrConst0(m.getAnIncludedModule(), name)
)
}
private AstNode getNode(TMethodOrExpr e) { e = TMethod(result) or e = TExpr(result) }
private TMethodOrExpr lookupMethodOrConst(Module m, string name) {
result = lookupMethodOrConst0(m, name)
or
not exists(lookupMethodOrConst0(m, name)) and
result = lookupMethodOrConst(m.getSuperClass(), name) and
// For now, we restrict the scope of top-level declarations to their file.
// This may remove some plausible targets, but also removes a lot of
// implausible targets
if getNode(result).getEnclosingModule() instanceof Toplevel
then getNode(result).getFile() = m.getADeclaration().getFile()
else any()
}

View File

@@ -0,0 +1,198 @@
private import codeql.ruby.AST
private import AST
private import TreeSitter
private import Call
abstract class OperationImpl extends Expr, TOperation {
abstract string getOperatorImpl();
abstract Expr getAnOperandImpl();
}
abstract class UnaryOperationImpl extends OperationImpl, MethodCallImpl, TUnaryOperation {
abstract Expr getOperandImpl();
final override Expr getAnOperandImpl() { result = this.getOperandImpl() }
final override string getMethodNameImpl() { result = this.getOperatorImpl() }
final override AstNode getReceiverImpl() { result = this.getOperandImpl() }
final override Expr getArgumentImpl(int n) { none() }
final override int getNumberOfArgumentsImpl() { result = 0 }
final override Block getBlockImpl() { none() }
}
class UnaryOperationGenerated extends UnaryOperationImpl {
private Ruby::Unary g;
UnaryOperationGenerated() { g = toGenerated(this) }
final override Expr getOperandImpl() { toGenerated(result) = g.getOperand() }
final override string getOperatorImpl() { result = g.getOperator() }
}
class SplatExprReal extends UnaryOperationImpl, TSplatExprReal {
private Ruby::SplatArgument g;
SplatExprReal() { this = TSplatExprReal(g) }
final override string getOperatorImpl() { result = "*" }
final override Expr getOperandImpl() { toGenerated(result) = g.getChild() }
}
class SplatExprSynth extends UnaryOperationImpl, TSplatExprSynth {
final override string getOperatorImpl() { result = "*" }
final override Expr getOperandImpl() { synthChild(this, 0, result) }
}
class HashSplatExprImpl extends UnaryOperationImpl, THashSplatExpr {
private Ruby::HashSplatArgument g;
HashSplatExprImpl() { this = THashSplatExpr(g) }
final override Expr getOperandImpl() { toGenerated(result) = g.getChild() }
final override string getOperatorImpl() { result = "**" }
}
abstract class BinaryOperationImpl extends OperationImpl, MethodCallImpl, TBinaryOperation {
abstract Stmt getLeftOperandImpl();
abstract Stmt getRightOperandImpl();
final override Expr getAnOperandImpl() {
result = this.getLeftOperandImpl()
or
result = this.getRightOperandImpl()
}
final override string getMethodNameImpl() { result = this.getOperatorImpl() }
final override AstNode getReceiverImpl() { result = this.getLeftOperandImpl() }
final override Expr getArgumentImpl(int n) { n = 0 and result = this.getRightOperandImpl() }
final override int getNumberOfArgumentsImpl() { result = 1 }
final override Block getBlockImpl() { none() }
}
class BinaryOperationReal extends BinaryOperationImpl {
private Ruby::Binary g;
BinaryOperationReal() { g = toGenerated(this) }
final override string getOperatorImpl() { result = g.getOperator() }
final override Stmt getLeftOperandImpl() { toGenerated(result) = g.getLeft() }
final override Stmt getRightOperandImpl() { toGenerated(result) = g.getRight() }
}
abstract class BinaryOperationSynth extends BinaryOperationImpl {
final override Stmt getLeftOperandImpl() { synthChild(this, 0, result) }
final override Stmt getRightOperandImpl() { synthChild(this, 1, result) }
}
class AddExprSynth extends BinaryOperationSynth, TAddExprSynth {
final override string getOperatorImpl() { result = "+" }
}
class SubExprSynth extends BinaryOperationSynth, TSubExprSynth {
final override string getOperatorImpl() { result = "-" }
}
class MulExprSynth extends BinaryOperationSynth, TMulExprSynth {
final override string getOperatorImpl() { result = "*" }
}
class DivExprSynth extends BinaryOperationSynth, TDivExprSynth {
final override string getOperatorImpl() { result = "/" }
}
class ModuloExprSynth extends BinaryOperationSynth, TModuloExprSynth {
final override string getOperatorImpl() { result = "%" }
}
class ExponentExprSynth extends BinaryOperationSynth, TExponentExprSynth {
final override string getOperatorImpl() { result = "**" }
}
class LogicalAndExprSynth extends BinaryOperationSynth, TLogicalAndExprSynth {
final override string getOperatorImpl() { result = "&&" }
}
class LogicalOrExprSynth extends BinaryOperationSynth, TLogicalOrExprSynth {
final override string getOperatorImpl() { result = "||" }
}
class LShiftExprSynth extends BinaryOperationSynth, TLShiftExprSynth {
final override string getOperatorImpl() { result = "<<" }
}
class RShiftExprSynth extends BinaryOperationSynth, TRShiftExprSynth {
final override string getOperatorImpl() { result = ">>" }
}
class BitwiseAndSynthExpr extends BinaryOperationSynth, TBitwiseAndExprSynth {
final override string getOperatorImpl() { result = "&" }
}
class BitwiseOrSynthExpr extends BinaryOperationSynth, TBitwiseOrExprSynth {
final override string getOperatorImpl() { result = "|" }
}
class BitwiseXorSynthExpr extends BinaryOperationSynth, TBitwiseXorExprSynth {
final override string getOperatorImpl() { result = "^" }
}
abstract class AssignmentImpl extends OperationImpl, TAssignment {
abstract Pattern getLeftOperandImpl();
abstract Expr getRightOperandImpl();
final override Expr getAnOperandImpl() {
result = this.getLeftOperandImpl()
or
result = this.getRightOperandImpl()
}
}
class AssignExprReal extends AssignmentImpl, TAssignExprReal {
private Ruby::Assignment g;
AssignExprReal() { this = TAssignExprReal(g) }
final override string getOperatorImpl() { result = "=" }
final override Pattern getLeftOperandImpl() { toGenerated(result) = g.getLeft() }
final override Expr getRightOperandImpl() { toGenerated(result) = g.getRight() }
}
class AssignExprSynth extends AssignmentImpl, TAssignExprSynth {
final override string getOperatorImpl() { result = "=" }
final override Pattern getLeftOperandImpl() { synthChild(this, 0, result) }
final override Expr getRightOperandImpl() { synthChild(this, 1, result) }
}
class AssignOperationImpl extends AssignmentImpl, TAssignOperation {
Ruby::OperatorAssignment g;
AssignOperationImpl() { g = toGenerated(this) }
final override string getOperatorImpl() { result = g.getOperator() }
final override Pattern getLeftOperandImpl() { toGenerated(result) = g.getLeft() }
final override Expr getRightOperandImpl() { toGenerated(result) = g.getRight() }
}

View File

@@ -0,0 +1,19 @@
private import codeql.ruby.AST
private import AST
private import TreeSitter
module Parameter {
class Range extends Ruby::AstNode {
private int pos;
Range() {
this = any(Ruby::BlockParameters bp).getChild(pos)
or
this = any(Ruby::MethodParameters mp).getChild(pos)
or
this = any(Ruby::LambdaParameters lp).getChild(pos)
}
int getPosition() { result = pos }
}
}

View File

@@ -0,0 +1,32 @@
private import codeql.ruby.AST
private import AST
private import TreeSitter
abstract class TuplePatternImpl extends Ruby::AstNode {
abstract Ruby::AstNode getChildNode(int i);
final int getRestIndex() {
result = unique(int i | this.getChildNode(i) instanceof Ruby::RestAssignment)
}
}
class TuplePatternParameterImpl extends TuplePatternImpl, Ruby::DestructuredParameter {
override Ruby::AstNode getChildNode(int i) { result = this.getChild(i) }
}
class DestructuredLeftAssignmentImpl extends TuplePatternImpl, Ruby::DestructuredLeftAssignment {
override Ruby::AstNode getChildNode(int i) { result = this.getChild(i) }
}
class LeftAssignmentListImpl extends TuplePatternImpl, Ruby::LeftAssignmentList {
override Ruby::AstNode getChildNode(int i) {
this =
any(Ruby::LeftAssignmentList lal |
if
strictcount(int j | exists(lal.getChild(j))) = 1 and
lal.getChild(0) instanceof Ruby::DestructuredLeftAssignment
then result = lal.getChild(0).(Ruby::DestructuredLeftAssignment).getChild(i)
else result = lal.getChild(i)
)
}
}

View File

@@ -0,0 +1,109 @@
private import TreeSitter
private import codeql.ruby.ast.Scope
private import codeql.ruby.ast.internal.AST
private import codeql.ruby.ast.internal.Parameter
class TScopeType = TMethodBase or TModuleLike or TBlockLike;
private class TBlockLike = TDoBlock or TLambda or TBlock or TEndBlock;
private class TModuleLike = TToplevel or TModuleDeclaration or TClassDeclaration or TSingletonClass;
module Scope {
class TypeRange = Callable::TypeRange or ModuleBase::TypeRange or @ruby_end_block;
class Range extends Ruby::AstNode, TypeRange {
Range() { not this = any(Ruby::Lambda l).getBody() }
ModuleBase::Range getEnclosingModule() {
result = this
or
not this instanceof ModuleBase::Range and result = this.getOuterScope().getEnclosingModule()
}
MethodBase::Range getEnclosingMethod() {
result = this
or
not this instanceof MethodBase::Range and
not this instanceof ModuleBase::Range and
result = this.getOuterScope().getEnclosingMethod()
}
Range getOuterScope() { result = scopeOf(this) }
}
}
module MethodBase {
class TypeRange = @ruby_method or @ruby_singleton_method;
class Range extends Scope::Range, TypeRange { }
}
module Callable {
class TypeRange = MethodBase::TypeRange or @ruby_do_block or @ruby_lambda or @ruby_block;
class Range extends Scope::Range, TypeRange {
Parameter::Range getParameter(int i) {
result = this.(Ruby::Method).getParameters().getChild(i) or
result = this.(Ruby::SingletonMethod).getParameters().getChild(i) or
result = this.(Ruby::DoBlock).getParameters().getChild(i) or
result = this.(Ruby::Lambda).getParameters().getChild(i) or
result = this.(Ruby::Block).getParameters().getChild(i)
}
}
}
module ModuleBase {
class TypeRange = @ruby_program or @ruby_module or @ruby_class or @ruby_singleton_class;
class Range extends Scope::Range, TypeRange { }
}
pragma[noinline]
private predicate rankHeredocBody(File f, Ruby::HeredocBody b, int i) {
b =
rank[i](Ruby::HeredocBody b0 |
f = b0.getLocation().getFile()
|
b0 order by b0.getLocation().getStartLine(), b0.getLocation().getStartColumn()
)
}
Ruby::HeredocBody getHereDocBody(Ruby::HeredocBeginning g) {
exists(int i, File f |
g =
rank[i](Ruby::HeredocBeginning b |
f = b.getLocation().getFile()
|
b order by b.getLocation().getStartLine(), b.getLocation().getStartColumn()
) and
rankHeredocBody(f, result, i)
)
}
private Ruby::AstNode parentOf(Ruby::AstNode n) {
n = getHereDocBody(result)
or
exists(Ruby::AstNode parent | parent = n.getParent() |
if
n =
[
parent.(Ruby::Module).getName(), parent.(Ruby::Class).getName(),
parent.(Ruby::Class).getSuperclass(), parent.(Ruby::SingletonClass).getValue(),
parent.(Ruby::Method).getName(), parent.(Ruby::SingletonMethod).getName(),
parent.(Ruby::SingletonMethod).getObject()
]
then result = parent.getParent()
else result = parent
)
}
/** Gets the enclosing scope of a node */
cached
Scope::Range scopeOf(Ruby::AstNode n) {
exists(Ruby::AstNode p | p = parentOf(n) |
p = result
or
not p instanceof Scope::Range and result = scopeOf(p)
)
}

View File

@@ -0,0 +1,797 @@
/** Provides predicates for synthesizing AST nodes. */
private import AST
private import TreeSitter
private import codeql.ruby.ast.internal.Call
private import codeql.ruby.ast.internal.Variable
private import codeql.ruby.ast.internal.Pattern
private import codeql.ruby.AST
/** A synthesized AST node kind. */
newtype SynthKind =
AddExprKind() or
AssignExprKind() or
BitwiseAndExprKind() or
BitwiseOrExprKind() or
BitwiseXorExprKind() or
ClassVariableAccessKind(ClassVariable v) or
DivExprKind() or
ExponentExprKind() or
GlobalVariableAccessKind(GlobalVariable v) or
InstanceVariableAccessKind(InstanceVariable v) or
IntegerLiteralKind(int i) { i in [-1000 .. 1000] } or
LShiftExprKind() or
LocalVariableAccessRealKind(LocalVariableReal v) or
LocalVariableAccessSynthKind(TLocalVariableSynth v) or
LogicalAndExprKind() or
LogicalOrExprKind() or
MethodCallKind(string name, boolean setter, int arity) {
any(Synthesis s).methodCall(name, setter, arity)
} or
ModuloExprKind() or
MulExprKind() or
RangeLiteralKind(boolean inclusive) { inclusive in [false, true] } or
RShiftExprKind() or
SplatExprKind() or
StmtSequenceKind() or
SelfKind() or
SubExprKind() or
ConstantReadAccessKind(string value) { any(Synthesis s).constantReadAccess(value) }
/**
* An AST child.
*
* Either a new synthesized node or a reference to an existing node.
*/
newtype Child =
SynthChild(SynthKind k) or
RealChild(AstNode n)
private newtype TSynthesis = MkSynthesis()
/** A class used for synthesizing AST nodes. */
class Synthesis extends TSynthesis {
/**
* Holds if a node should be synthesized as the `i`th child of `parent`, or if
* a non-synthesized node should be the `i`th child of synthesized node `parent`.
*
* `i = -1` is used to represent that the synthesized node is a desugared version
* of its parent.
*/
predicate child(AstNode parent, int i, Child child) { none() }
/**
* Holds if synthesized node `n` should have location `l`. Synthesized nodes for
* which this predicate does not hold, inherit their location (recursively) from
* their parent node.
*/
predicate location(AstNode n, Location l) { none() }
/**
* Holds if a local variable, identified by `i`, should be synthesized for AST
* node `n`.
*/
predicate localVariable(AstNode n, int i) { none() }
/**
* Holds if a method call to `name` with arity `arity` is needed.
*/
predicate methodCall(string name, boolean setter, int arity) { none() }
/**
* Holds if a constant read access of `name` is needed.
*/
predicate constantReadAccess(string name) { none() }
/**
* Holds if `n` should be excluded from `ControlFlowTree` in the CFG construction.
*/
predicate excludeFromControlFlowTree(AstNode n) { none() }
final string toString() { none() }
}
private class Desugared extends AstNode {
Desugared() { this = any(AstNode sugar).getDesugared() }
AstNode getADescendant() { result = this.getAChild*() }
}
/**
* Gets the desugaring level of `n`. That is, the number of desugaring
* transformations required before the context in which `n` occurs is
* fully desugared.
*/
int desugarLevel(AstNode n) { result = count(Desugared desugared | n = desugared.getADescendant()) }
/**
* Use this predicate in `Synthesis::child` to generate an assignment of `value` to
* synthesized variable `v`, where the assignment is a child of `assignParent` at
* index `assignIndex`.
*/
bindingset[v, assignParent, assignIndex, value]
private predicate assign(
AstNode parent, int i, Child child, TLocalVariableSynth v, AstNode assignParent, int assignIndex,
AstNode value
) {
parent = assignParent and
i = assignIndex and
child = SynthChild(AssignExprKind())
or
parent = TAssignExprSynth(assignParent, assignIndex) and
(
i = 0 and
child = SynthChild(LocalVariableAccessSynthKind(v))
or
i = 1 and
child = RealChild(value)
)
}
/** Holds if synthesized node `n` should have location `l`. */
predicate synthLocation(AstNode n, Location l) {
n.isSynthesized() and any(Synthesis s).location(n, l)
}
private predicate hasLocation(AstNode n, Location l) {
l = toGenerated(n).getLocation()
or
synthLocation(n, l)
}
private module ImplicitSelfSynthesis {
pragma[nomagic]
private predicate identifierMethodCallSelfSynthesis(AstNode mc, int i, Child child) {
child = SynthChild(SelfKind()) and
mc = TIdentifierMethodCall(_) and
i = 0
}
private class IdentifierMethodCallSelfSynthesis extends Synthesis {
final override predicate child(AstNode parent, int i, Child child) {
identifierMethodCallSelfSynthesis(parent, i, child)
}
}
pragma[nomagic]
private predicate regularMethodCallSelfSynthesis(TRegularMethodCall mc, int i, Child child) {
exists(Ruby::AstNode g |
mc = TRegularMethodCall(g) and
// If there's no explicit receiver (or scope resolution that acts like a
// receiver), then the receiver is implicitly `self`. N.B. `::Foo()` is
// not valid Ruby.
not exists(g.(Ruby::Call).getReceiver()) and
not exists(g.(Ruby::Call).getMethod().(Ruby::ScopeResolution).getScope())
) and
child = SynthChild(SelfKind()) and
i = 0
}
private class RegularMethodCallSelfSynthesis extends Synthesis {
final override predicate child(AstNode parent, int i, Child child) {
regularMethodCallSelfSynthesis(parent, i, child)
}
}
}
private module SetterDesugar {
/** An assignment where the left-hand side is a method call. */
private class SetterAssignExpr extends AssignExpr {
private MethodCall mc;
pragma[nomagic]
SetterAssignExpr() { mc = this.getLeftOperand() }
MethodCall getMethodCall() { result = mc }
pragma[nomagic]
MethodCallKind getCallKind(boolean setter, int arity) {
result = MethodCallKind(mc.getMethodName(), setter, arity)
}
pragma[nomagic]
Expr getReceiver() { result = mc.getReceiver() }
pragma[nomagic]
Expr getArgument(int i) { result = mc.getArgument(i) }
pragma[nomagic]
int getNumberOfArguments() { result = mc.getNumberOfArguments() }
pragma[nomagic]
Location getMethodCallLocation() { hasLocation(mc, result) }
}
pragma[nomagic]
private predicate setterMethodCallSynthesis(AstNode parent, int i, Child child) {
exists(SetterAssignExpr sae |
parent = sae and
i = -1 and
child = SynthChild(StmtSequenceKind())
or
exists(AstNode seq | seq = TStmtSequenceSynth(sae, -1) |
parent = seq and
i = 0 and
child = SynthChild(sae.getCallKind(true, sae.getNumberOfArguments() + 1))
or
exists(AstNode call | call = TMethodCallSynth(seq, 0, _, _, _) |
parent = call and
i = 0 and
child = RealChild(sae.getReceiver())
or
parent = call and
child = RealChild(sae.getArgument(i - 1))
or
exists(int valueIndex | valueIndex = sae.getNumberOfArguments() + 1 |
parent = call and
i = valueIndex and
child = SynthChild(AssignExprKind())
or
parent = TAssignExprSynth(call, valueIndex) and
(
i = 0 and
child = SynthChild(LocalVariableAccessSynthKind(TLocalVariableSynth(sae, 0)))
or
i = 1 and
child = RealChild(sae.getRightOperand())
)
)
)
or
parent = seq and
i = 1 and
child = SynthChild(LocalVariableAccessSynthKind(TLocalVariableSynth(sae, 0)))
)
)
}
/**
* ```rb
* x.foo = y
* ```
*
* desugars to
*
* ```rb
* x.foo=(__synth_0 = y);
* __synth_0;
* ```
*/
private class SetterMethodCallSynthesis extends Synthesis {
final override predicate child(AstNode parent, int i, Child child) {
setterMethodCallSynthesis(parent, i, child)
}
final override predicate location(AstNode n, Location l) {
exists(SetterAssignExpr sae, StmtSequence seq |
seq = sae.getDesugared() and
l = sae.getMethodCallLocation() and
n = seq.getAStmt()
)
}
final override predicate excludeFromControlFlowTree(AstNode n) {
n = any(SetterAssignExpr sae).getMethodCall()
}
final override predicate localVariable(AstNode n, int i) {
n instanceof SetterAssignExpr and
i = 0
}
final override predicate methodCall(string name, boolean setter, int arity) {
exists(SetterAssignExpr sae |
name = sae.getMethodCall().getMethodName() and
setter = true and
arity = sae.getNumberOfArguments() + 1
)
}
}
}
private module AssignOperationDesugar {
/**
* Gets the operator kind to synthesize for operator assignment `ao`.
*/
private SynthKind getKind(AssignOperation ao) {
ao instanceof AssignAddExpr and result = AddExprKind()
or
ao instanceof AssignSubExpr and result = SubExprKind()
or
ao instanceof AssignMulExpr and result = MulExprKind()
or
ao instanceof AssignDivExpr and result = DivExprKind()
or
ao instanceof AssignModuloExpr and result = ModuloExprKind()
or
ao instanceof AssignExponentExpr and result = ExponentExprKind()
or
ao instanceof AssignLogicalAndExpr and result = LogicalAndExprKind()
or
ao instanceof AssignLogicalOrExpr and result = LogicalOrExprKind()
or
ao instanceof AssignLShiftExpr and result = LShiftExprKind()
or
ao instanceof AssignRShiftExpr and result = RShiftExprKind()
or
ao instanceof AssignBitwiseAndExpr and result = BitwiseAndExprKind()
or
ao instanceof AssignBitwiseOrExpr and result = BitwiseOrExprKind()
or
ao instanceof AssignBitwiseXorExpr and result = BitwiseXorExprKind()
}
private Location getAssignOperationLocation(AssignOperation ao) {
exists(Ruby::OperatorAssignment g, Ruby::Token op |
g = toGenerated(ao) and
op.getParent() = g and
op.getParentIndex() = 1 and
result = op.getLocation()
)
}
/** An assignment operation where the left-hand side is a variable. */
private class VariableAssignOperation extends AssignOperation {
private Variable v;
pragma[nomagic]
VariableAssignOperation() { v = this.getLeftOperand().(VariableAccess).getVariable() }
pragma[nomagic]
SynthKind getVariableAccessKind() {
result in [
LocalVariableAccessRealKind(v).(SynthKind), InstanceVariableAccessKind(v),
ClassVariableAccessKind(v), GlobalVariableAccessKind(v)
]
}
}
pragma[nomagic]
private predicate variableAssignOperationSynthesis(AstNode parent, int i, Child child) {
exists(VariableAssignOperation vao |
parent = vao and
i = -1 and
child = SynthChild(AssignExprKind())
or
exists(AstNode assign | assign = TAssignExprSynth(vao, -1) |
parent = assign and
i = 0 and
child = RealChild(vao.getLeftOperand())
or
parent = assign and
i = 1 and
child = SynthChild(getKind(vao))
or
parent = getSynthChild(assign, 1) and
(
i = 0 and
child = SynthChild(vao.getVariableAccessKind())
or
i = 1 and
child = RealChild(vao.getRightOperand())
)
)
)
}
/**
* ```rb
* x += y
* ```
*
* desugars to
*
* ```rb
* x = x + y
* ```
*
* when `x` is a variable.
*/
private class VariableAssignOperationSynthesis extends Synthesis {
final override predicate child(AstNode parent, int i, Child child) {
variableAssignOperationSynthesis(parent, i, child)
}
final override predicate location(AstNode n, Location l) {
exists(VariableAssignOperation vao, BinaryOperation bo |
bo = vao.getDesugared().(AssignExpr).getRightOperand()
|
n = bo and
l = getAssignOperationLocation(vao)
or
n = bo.getLeftOperand() and
hasLocation(vao.getLeftOperand(), l)
)
}
}
/** An assignment operation where the left-hand side is a method call. */
private class SetterAssignOperation extends AssignOperation {
private MethodCall mc;
pragma[nomagic]
SetterAssignOperation() { mc = this.getLeftOperand() }
MethodCall getMethodCall() { result = mc }
pragma[nomagic]
MethodCallKind getCallKind(boolean setter, int arity) {
result = MethodCallKind(mc.getMethodName(), setter, arity)
}
pragma[nomagic]
Expr getReceiver() { result = mc.getReceiver() }
pragma[nomagic]
Expr getArgument(int i) { result = mc.getArgument(i) }
pragma[nomagic]
int getNumberOfArguments() { result = mc.getNumberOfArguments() }
pragma[nomagic]
Location getMethodCallLocation() { hasLocation(mc, result) }
}
pragma[nomagic]
private predicate methodCallAssignOperationSynthesis(AstNode parent, int i, Child child) {
exists(SetterAssignOperation sao |
parent = sao and
i = -1 and
child = SynthChild(StmtSequenceKind())
or
exists(AstNode seq | seq = TStmtSequenceSynth(sao, -1) |
// `__synth__0 = foo`
assign(parent, i, child, TLocalVariableSynth(sao, 0), seq, 0, sao.getReceiver())
or
// `__synth__1 = bar`
exists(Expr arg, int j | arg = sao.getArgument(j - 1) |
assign(parent, i, child, TLocalVariableSynth(sao, j), seq, j, arg)
)
or
// `__synth__2 = __synth__0.[](__synth__1) + y`
exists(int opAssignIndex | opAssignIndex = sao.getNumberOfArguments() + 1 |
parent = seq and
i = opAssignIndex and
child = SynthChild(AssignExprKind())
or
exists(AstNode assign | assign = TAssignExprSynth(seq, opAssignIndex) |
parent = assign and
i = 0 and
child =
SynthChild(LocalVariableAccessSynthKind(TLocalVariableSynth(sao, opAssignIndex)))
or
parent = assign and
i = 1 and
child = SynthChild(getKind(sao))
or
// `__synth__0.[](__synth__1) + y`
exists(AstNode op | op = getSynthChild(assign, 1) |
parent = op and
i = 0 and
child = SynthChild(sao.getCallKind(false, sao.getNumberOfArguments()))
or
parent = TMethodCallSynth(op, 0, _, _, _) and
child = SynthChild(LocalVariableAccessSynthKind(TLocalVariableSynth(sao, i))) and
i in [0 .. sao.getNumberOfArguments()]
or
parent = op and
i = 1 and
child = RealChild(sao.getRightOperand())
)
)
or
// `__synth__0.[]=(__synth__1, __synth__2);`
parent = seq and
i = opAssignIndex + 1 and
child = SynthChild(sao.getCallKind(true, opAssignIndex))
or
exists(AstNode setter | setter = TMethodCallSynth(seq, opAssignIndex + 1, _, _, _) |
parent = setter and
child = SynthChild(LocalVariableAccessSynthKind(TLocalVariableSynth(sao, i))) and
i in [0 .. sao.getNumberOfArguments()]
or
parent = setter and
i = opAssignIndex + 1 and
child =
SynthChild(LocalVariableAccessSynthKind(TLocalVariableSynth(sao, opAssignIndex)))
)
or
parent = seq and
i = opAssignIndex + 2 and
child = SynthChild(LocalVariableAccessSynthKind(TLocalVariableSynth(sao, opAssignIndex)))
)
)
)
}
/**
* ```rb
* foo[bar] += y
* ```
*
* desugars to
*
* ```rb
* __synth__0 = foo;
* __synth__1 = bar;
* __synth__2 = __synth__0.[](__synth__1) + y;
* __synth__0.[]=(__synth__1, __synth__2);
* __synth__2;
* ```
*/
private class MethodCallAssignOperationSynthesis extends Synthesis {
final override predicate child(AstNode parent, int i, Child child) {
methodCallAssignOperationSynthesis(parent, i, child)
}
final override predicate location(AstNode n, Location l) {
exists(SetterAssignOperation sao, StmtSequence seq | seq = sao.getDesugared() |
n = seq.getStmt(0) and
hasLocation(sao.getReceiver(), l)
or
exists(int i |
n = seq.getStmt(i + 1) and
hasLocation(sao.getArgument(i), l)
)
or
exists(AssignExpr ae, int opAssignIndex |
opAssignIndex = sao.getNumberOfArguments() + 1 and
ae = seq.getStmt(opAssignIndex)
|
l = getAssignOperationLocation(sao) and
n = ae
or
exists(BinaryOperation bo | bo = ae.getRightOperand() |
n = bo.getLeftOperand() and
l = sao.getMethodCallLocation()
or
exists(MethodCall mc | mc = bo.getLeftOperand() |
n = mc.getReceiver() and
hasLocation(sao.getReceiver(), l)
or
exists(int i |
n = mc.getArgument(i) and
hasLocation(sao.getArgument(i), l)
)
)
)
or
exists(MethodCall mc | mc = seq.getStmt(opAssignIndex + 1) |
n = mc and
l = sao.getMethodCallLocation()
or
n = mc.getReceiver() and
hasLocation(sao.getReceiver(), l)
or
exists(int i | n = mc.getArgument(i) |
hasLocation(sao.getArgument(i), l)
or
i = opAssignIndex and
l = getAssignOperationLocation(sao)
)
)
or
n = seq.getStmt(opAssignIndex + 2) and
l = getAssignOperationLocation(sao)
)
)
}
final override predicate localVariable(AstNode n, int i) {
n = any(SetterAssignOperation sao | i in [0 .. sao.getNumberOfArguments() + 1])
}
final override predicate methodCall(string name, boolean setter, int arity) {
exists(SetterAssignOperation sao | name = sao.getMethodCall().getMethodName() |
setter = false and
arity = sao.getNumberOfArguments()
or
setter = true and
arity = sao.getNumberOfArguments() + 1
)
}
final override predicate excludeFromControlFlowTree(AstNode n) {
n = any(SetterAssignOperation sao).getMethodCall()
}
}
}
private module CompoundAssignDesugar {
/** An assignment where the left-hand side is a tuple pattern. */
private class TupleAssignExpr extends AssignExpr {
private TuplePattern tp;
pragma[nomagic]
TupleAssignExpr() { tp = this.getLeftOperand() }
TuplePattern getTuplePattern() { result = tp }
pragma[nomagic]
Pattern getElement(int i) { result = tp.getElement(i) }
pragma[nomagic]
int getNumberOfElements() {
toGenerated(tp) = any(TuplePatternImpl impl | result = count(impl.getChildNode(_)))
}
pragma[nomagic]
int getRestIndexOrNumberOfElements() {
result = tp.getRestIndex()
or
toGenerated(tp) = any(TuplePatternImpl impl | not exists(impl.getRestIndex())) and
result = this.getNumberOfElements()
}
}
pragma[nomagic]
private predicate compoundAssignSynthesis(AstNode parent, int i, Child child) {
exists(TupleAssignExpr tae |
parent = tae and
i = -1 and
child = SynthChild(StmtSequenceKind())
or
exists(AstNode seq | seq = TStmtSequenceSynth(tae, -1) |
parent = seq and
i = 0 and
child = SynthChild(AssignExprKind())
or
exists(AstNode assign | assign = TAssignExprSynth(seq, 0) |
parent = assign and
i = 0 and
child = SynthChild(LocalVariableAccessSynthKind(TLocalVariableSynth(tae, 0)))
or
parent = assign and
i = 1 and
child = SynthChild(SplatExprKind())
or
parent = TSplatExprSynth(assign, 1) and
i = 0 and
child = RealChild(tae.getRightOperand())
)
or
exists(Pattern p, int j, int restIndex |
p = tae.getElement(j) and
restIndex = tae.getRestIndexOrNumberOfElements()
|
parent = seq and
i = j + 1 and
child = SynthChild(AssignExprKind())
or
exists(AstNode assign | assign = TAssignExprSynth(seq, j + 1) |
parent = assign and
i = 0 and
child = RealChild(p)
or
parent = assign and
i = 1 and
child = SynthChild(MethodCallKind("[]", false, 1))
or
parent = TMethodCallSynth(assign, 1, _, _, _) and
i = 0 and
child = SynthChild(LocalVariableAccessSynthKind(TLocalVariableSynth(tae, 0)))
or
j < restIndex and
parent = TMethodCallSynth(assign, 1, _, _, _) and
i = 1 and
child = SynthChild(IntegerLiteralKind(j))
or
j = restIndex and
(
parent = TMethodCallSynth(assign, 1, _, _, _) and
i = 1 and
child = SynthChild(RangeLiteralKind(true))
or
exists(AstNode call |
call = TMethodCallSynth(assign, 1, _, _, _) and
parent = TRangeLiteralSynth(call, 1, _)
|
i = 0 and
child = SynthChild(IntegerLiteralKind(j))
or
i = 1 and
child = SynthChild(IntegerLiteralKind(restIndex - tae.getNumberOfElements()))
)
)
or
j > restIndex and
parent = TMethodCallSynth(assign, 1, _, _, _) and
i = 1 and
child = SynthChild(IntegerLiteralKind(j - tae.getNumberOfElements()))
)
)
)
)
}
/**
* ```rb
* x, *y, z = w
* ```
* desugars to
*
* ```rb
* __synth__0 = *w;
* x = __synth__0[0];
* y = __synth__0[1..-2];
* z = __synth__0[-1];
* ```
*/
private class CompoundAssignSynthesis extends Synthesis {
final override predicate child(AstNode parent, int i, Child child) {
compoundAssignSynthesis(parent, i, child)
}
final override predicate location(AstNode n, Location l) {
exists(TupleAssignExpr tae, StmtSequence seq | seq = tae.getDesugared() |
n = seq.getStmt(0) and
hasLocation(tae.getRightOperand(), l)
or
exists(Pattern p, int j |
p = tae.getElement(j) and
n = seq.getStmt(j + 1) and
hasLocation(p, l)
)
)
}
final override predicate localVariable(AstNode n, int i) {
n instanceof TupleAssignExpr and
i = 0
}
final override predicate methodCall(string name, boolean setter, int arity) {
name = "[]" and
setter = false and
arity = 1
}
final override predicate excludeFromControlFlowTree(AstNode n) {
n = any(TupleAssignExpr tae).getTuplePattern()
}
}
}
private module ArrayLiteralDesugar {
pragma[nomagic]
private predicate arrayLiteralSynthesis(AstNode parent, int i, Child child) {
exists(ArrayLiteral al |
parent = al and
i = -1 and
child = SynthChild(MethodCallKind("[]", false, al.getNumberOfElements() + 1))
or
exists(AstNode mc | mc = TMethodCallSynth(al, -1, _, _, _) |
parent = mc and
i = 0 and
child = SynthChild(ConstantReadAccessKind("::Array"))
or
parent = mc and
child = RealChild(al.getElement(i - 1))
)
)
}
/**
* ```rb
* [1, 2, 3]
* ```
* desugars to
*
* ```rb
* ::Array.[](1, 2, 3)
* ```
*/
private class CompoundAssignSynthesis extends Synthesis {
final override predicate child(AstNode parent, int i, Child child) {
arrayLiteralSynthesis(parent, i, child)
}
final override predicate methodCall(string name, boolean setter, int arity) {
name = "[]" and
setter = false and
arity = any(ArrayLiteral al).getNumberOfElements() + 1
}
final override predicate constantReadAccess(string name) { name = "::Array" }
}
}

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,604 @@
private import TreeSitter
private import codeql.Locations
private import codeql.ruby.AST
private import codeql.ruby.ast.internal.AST
private import codeql.ruby.ast.internal.Parameter
private import codeql.ruby.ast.internal.Scope
private import codeql.ruby.ast.internal.Synthesis
/**
* Holds if `n` is in the left-hand-side of an explicit assignment `assignment`.
*/
predicate explicitAssignmentNode(Ruby::AstNode n, Ruby::AstNode assignment) {
n = assignment.(Ruby::Assignment).getLeft()
or
n = assignment.(Ruby::OperatorAssignment).getLeft()
or
exists(Ruby::AstNode parent |
parent = n.getParent() and
explicitAssignmentNode(parent, assignment)
|
parent instanceof Ruby::DestructuredLeftAssignment
or
parent instanceof Ruby::LeftAssignmentList
or
parent instanceof Ruby::RestAssignment
)
}
/** Holds if `n` is inside an implicit assignment. */
predicate implicitAssignmentNode(Ruby::AstNode n) {
n = any(Ruby::ExceptionVariable ev).getChild()
or
n = any(Ruby::For for).getPattern()
or
implicitAssignmentNode(n.getParent())
}
/** Holds if `n` is inside a parameter. */
predicate implicitParameterAssignmentNode(Ruby::AstNode n, Callable::Range c) {
n = c.getParameter(_)
or
implicitParameterAssignmentNode(n.getParent().(Ruby::DestructuredParameter), c)
}
private predicate instanceVariableAccess(
Ruby::InstanceVariable var, string name, Scope::Range scope, boolean instance
) {
name = var.getValue() and
scope = enclosingModuleOrClass(var) and
if hasEnclosingMethod(var) then instance = true else instance = false
}
private predicate classVariableAccess(Ruby::ClassVariable var, string name, Scope::Range scope) {
name = var.getValue() and
scope = enclosingModuleOrClass(var)
}
private predicate hasEnclosingMethod(Ruby::AstNode node) {
exists(Scope::Range s | scopeOf(node) = s and exists(s.getEnclosingMethod()))
}
private ModuleBase::Range enclosingModuleOrClass(Ruby::AstNode node) {
exists(Scope::Range s | scopeOf(node) = s and result = s.getEnclosingModule())
}
private predicate parameterAssignment(Callable::Range scope, string name, Ruby::Identifier i) {
implicitParameterAssignmentNode(i, scope) and
name = i.getValue()
}
/** Holds if `scope` defines `name` in its parameter declaration at `i`. */
private predicate scopeDefinesParameterVariable(
Callable::Range scope, string name, Ruby::Identifier i
) {
// In case of overlapping parameter names (e.g. `_`), only the first
// parameter will give rise to a variable
i =
min(Ruby::Identifier other |
parameterAssignment(scope, name, other)
|
other order by other.getLocation().getStartLine(), other.getLocation().getStartColumn()
)
or
exists(Parameter::Range p |
p = scope.getParameter(_) and
name = i.getValue()
|
i = p.(Ruby::BlockParameter).getName() or
i = p.(Ruby::HashSplatParameter).getName() or
i = p.(Ruby::KeywordParameter).getName() or
i = p.(Ruby::OptionalParameter).getName() or
i = p.(Ruby::SplatParameter).getName()
)
}
/** Holds if `name` is assigned in `scope` at `i`. */
private predicate scopeAssigns(Scope::Range scope, string name, Ruby::Identifier i) {
(explicitAssignmentNode(i, _) or implicitAssignmentNode(i)) and
name = i.getValue() and
scope = scopeOf(i)
}
cached
private module Cached {
cached
newtype TVariable =
TGlobalVariable(string name) { name = any(Ruby::GlobalVariable var).getValue() } or
TClassVariable(Scope::Range scope, string name, Ruby::AstNode decl) {
decl =
min(Ruby::ClassVariable other |
classVariableAccess(other, name, scope)
|
other order by other.getLocation().getStartLine(), other.getLocation().getStartColumn()
)
} or
TInstanceVariable(Scope::Range scope, string name, boolean instance, Ruby::AstNode decl) {
decl =
min(Ruby::InstanceVariable other |
instanceVariableAccess(other, name, scope, instance)
|
other order by other.getLocation().getStartLine(), other.getLocation().getStartColumn()
)
} or
TLocalVariableReal(Scope::Range scope, string name, Ruby::Identifier i) {
scopeDefinesParameterVariable(scope, name, i)
or
i =
min(Ruby::Identifier other |
scopeAssigns(scope, name, other)
|
other order by other.getLocation().getStartLine(), other.getLocation().getStartColumn()
) and
not scopeDefinesParameterVariable(scope, name, _) and
not inherits(scope, name, _)
} or
TLocalVariableSynth(AstNode n, int i) { any(Synthesis s).localVariable(n, i) }
// Db types that can be vcalls
private class VcallToken =
@ruby_scope_resolution or @ruby_token_constant or @ruby_token_identifier or @ruby_token_super;
/**
* Holds if `i` is an `identifier` node occurring in the context where it
* should be considered a VCALL. VCALL is the term that MRI/Ripper uses
* internally when there's an identifier without arguments or parentheses,
* i.e. it *might* be a method call, but it might also be a variable access,
* depending on the bindings in the current scope.
* ```rb
* foo # in MRI this is a VCALL, and the predicate should hold for this
* bar() # in MRI this would be an FCALL. Tree-sitter gives us a `call` node,
* # and the `method` field will be an `identifier`, but this predicate
* # will not hold for that identifier.
* ```
*/
cached
predicate vcall(VcallToken i) {
i = any(Ruby::ArgumentList x).getChild(_)
or
i = any(Ruby::Array x).getChild(_)
or
i = any(Ruby::Assignment x).getRight()
or
i = any(Ruby::Begin x).getChild(_)
or
i = any(Ruby::BeginBlock x).getChild(_)
or
i = any(Ruby::Binary x).getLeft()
or
i = any(Ruby::Binary x).getRight()
or
i = any(Ruby::Block x).getChild(_)
or
i = any(Ruby::BlockArgument x).getChild()
or
i = any(Ruby::Call x).getReceiver()
or
i = any(Ruby::Case x).getValue()
or
i = any(Ruby::Class x).getChild(_)
or
i = any(Ruby::Conditional x).getCondition()
or
i = any(Ruby::Conditional x).getConsequence()
or
i = any(Ruby::Conditional x).getAlternative()
or
i = any(Ruby::Do x).getChild(_)
or
i = any(Ruby::DoBlock x).getChild(_)
or
i = any(Ruby::ElementReference x).getChild(_)
or
i = any(Ruby::ElementReference x).getObject()
or
i = any(Ruby::Else x).getChild(_)
or
i = any(Ruby::Elsif x).getCondition()
or
i = any(Ruby::EndBlock x).getChild(_)
or
i = any(Ruby::Ensure x).getChild(_)
or
i = any(Ruby::Exceptions x).getChild(_)
or
i = any(Ruby::HashSplatArgument x).getChild()
or
i = any(Ruby::If x).getCondition()
or
i = any(Ruby::IfModifier x).getCondition()
or
i = any(Ruby::IfModifier x).getBody()
or
i = any(Ruby::In x).getChild()
or
i = any(Ruby::Interpolation x).getChild(_)
or
i = any(Ruby::KeywordParameter x).getValue()
or
i = any(Ruby::Method x).getChild(_)
or
i = any(Ruby::Module x).getChild(_)
or
i = any(Ruby::OperatorAssignment x).getRight()
or
i = any(Ruby::OptionalParameter x).getValue()
or
i = any(Ruby::Pair x).getKey()
or
i = any(Ruby::Pair x).getValue()
or
i = any(Ruby::ParenthesizedStatements x).getChild(_)
or
i = any(Ruby::Pattern x).getChild()
or
i = any(Ruby::Program x).getChild(_)
or
i = any(Ruby::Range x).getBegin()
or
i = any(Ruby::Range x).getEnd()
or
i = any(Ruby::RescueModifier x).getBody()
or
i = any(Ruby::RescueModifier x).getHandler()
or
i = any(Ruby::RightAssignmentList x).getChild(_)
or
i = any(Ruby::ScopeResolution x).getScope()
or
i = any(Ruby::SingletonClass x).getValue()
or
i = any(Ruby::SingletonClass x).getChild(_)
or
i = any(Ruby::SingletonMethod x).getChild(_)
or
i = any(Ruby::SingletonMethod x).getObject()
or
i = any(Ruby::SplatArgument x).getChild()
or
i = any(Ruby::Superclass x).getChild()
or
i = any(Ruby::Then x).getChild(_)
or
i = any(Ruby::Unary x).getOperand()
or
i = any(Ruby::Unless x).getCondition()
or
i = any(Ruby::UnlessModifier x).getCondition()
or
i = any(Ruby::UnlessModifier x).getBody()
or
i = any(Ruby::Until x).getCondition()
or
i = any(Ruby::UntilModifier x).getCondition()
or
i = any(Ruby::UntilModifier x).getBody()
or
i = any(Ruby::While x).getCondition()
or
i = any(Ruby::WhileModifier x).getCondition()
or
i = any(Ruby::WhileModifier x).getBody()
}
cached
predicate access(Ruby::Identifier access, VariableReal variable) {
exists(string name |
variable.getNameImpl() = name and
name = access.getValue()
|
variable.getDeclaringScopeImpl() = scopeOf(access) and
not access.getLocation().strictlyBefore(variable.getLocationImpl()) and
// In case of overlapping parameter names, later parameters should not
// be considered accesses to the first parameter
if parameterAssignment(_, _, access)
then scopeDefinesParameterVariable(_, _, access)
else any()
or
exists(Scope::Range declScope |
variable.getDeclaringScopeImpl() = declScope and
inherits(scopeOf(access), name, declScope)
)
)
}
private class Access extends Ruby::Token {
Access() {
access(this, _) or
this instanceof Ruby::GlobalVariable or
this instanceof Ruby::InstanceVariable or
this instanceof Ruby::ClassVariable
}
}
cached
predicate explicitWriteAccess(Access access, Ruby::AstNode assignment) {
explicitAssignmentNode(access, assignment)
}
cached
predicate implicitWriteAccess(Access access) {
implicitAssignmentNode(access)
or
scopeDefinesParameterVariable(_, _, access)
}
cached
predicate isCapturedAccess(LocalVariableAccess access) {
toGenerated(access.getVariable().getDeclaringScope()) != scopeOf(toGenerated(access))
}
cached
predicate instanceVariableAccess(Ruby::InstanceVariable var, InstanceVariable v) {
exists(string name, Scope::Range scope, boolean instance |
v = TInstanceVariable(scope, name, instance, _) and
instanceVariableAccess(var, name, scope, instance)
)
}
cached
predicate classVariableAccess(Ruby::ClassVariable var, ClassVariable variable) {
exists(Scope::Range scope, string name |
variable = TClassVariable(scope, name, _) and
classVariableAccess(var, name, scope)
)
}
}
import Cached
/** Holds if this scope inherits `name` from an outer scope `outer`. */
private predicate inherits(Scope::Range scope, string name, Scope::Range outer) {
(scope instanceof Ruby::Block or scope instanceof Ruby::DoBlock) and
not scopeDefinesParameterVariable(scope, name, _) and
(
outer = scope.getOuterScope() and
(
scopeDefinesParameterVariable(outer, name, _)
or
exists(Ruby::Identifier i |
scopeAssigns(outer, name, i) and
i.getLocation().strictlyBefore(scope.getLocation())
)
)
or
inherits(scope.getOuterScope(), name, outer)
)
}
abstract class VariableImpl extends TVariable {
abstract string getNameImpl();
final string toString() { result = this.getNameImpl() }
abstract Location getLocationImpl();
}
class TVariableReal = TGlobalVariable or TClassVariable or TInstanceVariable or TLocalVariableReal;
class TLocalVariable = TLocalVariableReal or TLocalVariableSynth;
/**
* This class only exists to avoid negative recursion warnings. Ideally,
* we would use `VariableImpl` directly, but that results in incorrect
* negative recursion warnings. Adding new root-defs for the predicates
* below works around this.
*/
abstract class VariableReal extends TVariableReal {
abstract string getNameImpl();
abstract Location getLocationImpl();
abstract Scope::Range getDeclaringScopeImpl();
final string toString() { result = this.getNameImpl() }
}
// Convert extensions of `VariableReal` into extensions of `VariableImpl`
private class VariableRealAdapter extends VariableImpl, TVariableReal instanceof VariableReal {
final override string getNameImpl() { result = VariableReal.super.getNameImpl() }
final override Location getLocationImpl() { result = VariableReal.super.getLocationImpl() }
}
class LocalVariableReal extends VariableReal, TLocalVariableReal {
private Scope::Range scope;
private string name;
private Ruby::Identifier i;
LocalVariableReal() { this = TLocalVariableReal(scope, name, i) }
final override string getNameImpl() { result = name }
final override Location getLocationImpl() { result = i.getLocation() }
final override Scope::Range getDeclaringScopeImpl() { result = scope }
final VariableAccess getDefiningAccessImpl() { toGenerated(result) = i }
}
class LocalVariableSynth extends VariableImpl, TLocalVariableSynth {
private AstNode n;
private int i;
LocalVariableSynth() { this = TLocalVariableSynth(n, i) }
final override string getNameImpl() {
exists(int level | level = desugarLevel(n) |
if level > 0 then result = "__synth__" + i + "__" + level else result = "__synth__" + i
)
}
final override Location getLocationImpl() { result = n.getLocation() }
}
class GlobalVariableImpl extends VariableReal, TGlobalVariable {
private string name;
GlobalVariableImpl() { this = TGlobalVariable(name) }
final override string getNameImpl() { result = name }
final override Location getLocationImpl() { none() }
final override Scope::Range getDeclaringScopeImpl() { none() }
}
class InstanceVariableImpl extends VariableReal, TInstanceVariable {
private ModuleBase::Range scope;
private boolean instance;
private string name;
private Ruby::AstNode decl;
InstanceVariableImpl() { this = TInstanceVariable(scope, name, instance, decl) }
final override string getNameImpl() { result = name }
final predicate isClassInstanceVariable() { instance = false }
final override Location getLocationImpl() { result = decl.getLocation() }
final override Scope::Range getDeclaringScopeImpl() { result = scope }
}
class ClassVariableImpl extends VariableReal, TClassVariable {
private ModuleBase::Range scope;
private string name;
private Ruby::AstNode decl;
ClassVariableImpl() { this = TClassVariable(scope, name, decl) }
final override string getNameImpl() { result = name }
final override Location getLocationImpl() { result = decl.getLocation() }
final override Scope::Range getDeclaringScopeImpl() { result = scope }
}
abstract class VariableAccessImpl extends Expr, TVariableAccess {
abstract VariableImpl getVariableImpl();
}
module LocalVariableAccess {
predicate range(Ruby::Identifier id, LocalVariable v) {
access(id, v) and
(
explicitWriteAccess(id, _)
or
implicitWriteAccess(id)
or
vcall(id)
)
}
}
class TVariableAccessReal =
TLocalVariableAccessReal or TGlobalVariableAccess or TInstanceVariableAccess or
TClassVariableAccess;
abstract class LocalVariableAccessImpl extends VariableAccessImpl, TLocalVariableAccess { }
private class LocalVariableAccessReal extends LocalVariableAccessImpl, TLocalVariableAccessReal {
private Ruby::Identifier g;
private LocalVariable v;
LocalVariableAccessReal() { this = TLocalVariableAccessReal(g, v) }
final override LocalVariable getVariableImpl() { result = v }
final override string toString() { result = g.getValue() }
}
private class LocalVariableAccessSynth extends LocalVariableAccessImpl, TLocalVariableAccessSynth {
private LocalVariable v;
LocalVariableAccessSynth() { this = TLocalVariableAccessSynth(_, _, v) }
final override LocalVariable getVariableImpl() { result = v }
final override string toString() { result = v.getName() }
}
module GlobalVariableAccess {
predicate range(Ruby::GlobalVariable n, GlobalVariableImpl v) { n.getValue() = v.getNameImpl() }
}
abstract class GlobalVariableAccessImpl extends VariableAccessImpl, TGlobalVariableAccess { }
private class GlobalVariableAccessReal extends GlobalVariableAccessImpl, TGlobalVariableAccessReal {
private Ruby::GlobalVariable g;
private GlobalVariable v;
GlobalVariableAccessReal() { this = TGlobalVariableAccessReal(g, v) }
final override GlobalVariable getVariableImpl() { result = v }
final override string toString() { result = g.getValue() }
}
private class GlobalVariableAccessSynth extends GlobalVariableAccessImpl, TGlobalVariableAccessSynth {
private GlobalVariable v;
GlobalVariableAccessSynth() { this = TGlobalVariableAccessSynth(_, _, v) }
final override GlobalVariable getVariableImpl() { result = v }
final override string toString() { result = v.getName() }
}
module InstanceVariableAccess {
predicate range(Ruby::InstanceVariable n, InstanceVariable v) { instanceVariableAccess(n, v) }
}
abstract class InstanceVariableAccessImpl extends VariableAccessImpl, TInstanceVariableAccess { }
private class InstanceVariableAccessReal extends InstanceVariableAccessImpl,
TInstanceVariableAccessReal {
private Ruby::InstanceVariable g;
private InstanceVariable v;
InstanceVariableAccessReal() { this = TInstanceVariableAccessReal(g, v) }
final override InstanceVariable getVariableImpl() { result = v }
final override string toString() { result = g.getValue() }
}
private class InstanceVariableAccessSynth extends InstanceVariableAccessImpl,
TInstanceVariableAccessSynth {
private InstanceVariable v;
InstanceVariableAccessSynth() { this = TInstanceVariableAccessSynth(_, _, v) }
final override InstanceVariable getVariableImpl() { result = v }
final override string toString() { result = v.getName() }
}
module ClassVariableAccess {
predicate range(Ruby::ClassVariable n, ClassVariable v) { classVariableAccess(n, v) }
}
abstract class ClassVariableAccessRealImpl extends VariableAccessImpl, TClassVariableAccess { }
private class ClassVariableAccessReal extends ClassVariableAccessRealImpl, TClassVariableAccessReal {
private Ruby::ClassVariable g;
private ClassVariable v;
ClassVariableAccessReal() { this = TClassVariableAccessReal(g, v) }
final override ClassVariable getVariableImpl() { result = v }
final override string toString() { result = g.getValue() }
}
private class ClassVariableAccessSynth extends ClassVariableAccessRealImpl,
TClassVariableAccessSynth {
private ClassVariable v;
ClassVariableAccessSynth() { this = TClassVariableAccessSynth(_, _, v) }
final override ClassVariable getVariableImpl() { result = v }
final override string toString() { result = v.getName() }
}

View File

@@ -0,0 +1,414 @@
/** Provides classes representing basic blocks. */
private import codeql.Locations
private import codeql.ruby.AST
private import codeql.ruby.ast.internal.AST
private import codeql.ruby.ast.internal.TreeSitter
private import codeql.ruby.controlflow.ControlFlowGraph
private import internal.ControlFlowGraphImpl
private import CfgNodes
private import SuccessorTypes
/**
* A basic block, that is, a maximal straight-line sequence of control flow nodes
* without branches or joins.
*/
class BasicBlock extends TBasicBlockStart {
/** Gets the scope of this basic block. */
CfgScope getScope() { result = this.getAPredecessor().getScope() }
/** Gets an immediate successor of this basic block, if any. */
BasicBlock getASuccessor() { result = this.getASuccessor(_) }
/** Gets an immediate successor of this basic block of a given type, if any. */
BasicBlock getASuccessor(SuccessorType t) {
result.getFirstNode() = this.getLastNode().getASuccessor(t)
}
/** Gets an immediate predecessor of this basic block, if any. */
BasicBlock getAPredecessor() { result.getASuccessor() = this }
/** Gets an immediate predecessor of this basic block of a given type, if any. */
BasicBlock getAPredecessor(SuccessorType t) { result.getASuccessor(t) = this }
/** Gets the control flow node at a specific (zero-indexed) position in this basic block. */
CfgNode getNode(int pos) { bbIndex(this.getFirstNode(), result, pos) }
/** Gets a control flow node in this basic block. */
CfgNode getANode() { result = this.getNode(_) }
/** Gets the first control flow node in this basic block. */
CfgNode getFirstNode() { this = TBasicBlockStart(result) }
/** Gets the last control flow node in this basic block. */
CfgNode getLastNode() { result = this.getNode(this.length() - 1) }
/** Gets the length of this basic block. */
int length() { result = strictcount(this.getANode()) }
/**
* Holds if this basic block immediately dominates basic block `bb`.
*
* That is, all paths reaching basic block `bb` from some entry point
* basic block must go through this basic block (which is an immediate
* predecessor of `bb`).
*
* Example:
*
* ```rb
* def m b
* if b
* return 0
* end
* return 1
* end
* ```
*
* The basic block starting on line 2 immediately dominates the
* basic block on line 5 (all paths from the entry point of `m`
* to `return 1` must go through the `if` block).
*/
predicate immediatelyDominates(BasicBlock bb) { bbIDominates(this, bb) }
/**
* Holds if this basic block strictly dominates basic block `bb`.
*
* That is, all paths reaching basic block `bb` from some entry point
* basic block must go through this basic block (which must be different
* from `bb`).
*
* Example:
*
* ```rb
* def m b
* if b
* return 0
* end
* return 1
* end
* ```
*
* The basic block starting on line 2 strictly dominates the
* basic block on line 5 (all paths from the entry point of `m`
* to `return 1` must go through the `if` block).
*/
predicate strictlyDominates(BasicBlock bb) { bbIDominates+(this, bb) }
/**
* Holds if this basic block dominates basic block `bb`.
*
* That is, all paths reaching basic block `bb` from some entry point
* basic block must go through this basic block.
*
* Example:
*
* ```rb
* def m b
* if b
* return 0
* end
* return 1
* end
* ```
*
* The basic block starting on line 2 dominates the basic
* basic block on line 5 (all paths from the entry point of `m`
* to `return 1` must go through the `if` block).
*/
predicate dominates(BasicBlock bb) {
bb = this or
this.strictlyDominates(bb)
}
/**
* Holds if `df` is in the dominance frontier of this basic block.
* That is, this basic block dominates a predecessor of `df`, but
* does not dominate `df` itself.
*
* Example:
*
* ```rb
* def m x
* if x < 0
* x = -x
* if x > 10
* x = x - 1
* end
* end
* puts x
* end
* ```
*
* The basic block on line 8 is in the dominance frontier
* of the basic block starting on line 3 because that block
* dominates the basic block on line 4, which is a predecessor of
* `puts x`. Also, the basic block starting on line 3 does not
* dominate the basic block on line 8.
*/
predicate inDominanceFrontier(BasicBlock df) {
this.dominatesPredecessor(df) and
not strictlyDominates(df)
}
/**
* Holds if this basic block dominates a predecessor of `df`.
*/
private predicate dominatesPredecessor(BasicBlock df) { this.dominates(df.getAPredecessor()) }
/**
* Gets the basic block that immediately dominates this basic block, if any.
*
* That is, all paths reaching this basic block from some entry point
* basic block must go through the result, which is an immediate basic block
* predecessor of this basic block.
*
* Example:
*
* ```rb
* def m b
* if b
* return 0
* end
* return 1
* end
* ```
*
* The basic block starting on line 2 is an immediate dominator of
* the basic block on line 5 (all paths from the entry point of `m`
* to `return 1` must go through the `if` block, and the `if` block
* is an immediate predecessor of `return 1`).
*/
BasicBlock getImmediateDominator() { bbIDominates(result, this) }
/**
* Holds if this basic block strictly post-dominates basic block `bb`.
*
* That is, all paths reaching a normal exit point basic block from basic
* block `bb` must go through this basic block (which must be different
* from `bb`).
*
* Example:
*
* ```rb
* def m b
* if b
* puts "b"
* end
* puts "m"
* end
* ```
*
* The basic block on line 5 strictly post-dominates the basic block on
* line 3 (all paths to the exit point of `m` from `puts "b"` must go
* through `puts "m"`).
*/
predicate strictlyPostDominates(BasicBlock bb) { bbIPostDominates+(this, bb) }
/**
* Holds if this basic block post-dominates basic block `bb`.
*
* That is, all paths reaching a normal exit point basic block from basic
* block `bb` must go through this basic block.
*
* Example:
*
* ```rb
* def m b
* if b
* puts "b"
* end
* puts "m"
* end
* ```
*
* The basic block on line 5 post-dominates the basic block on line 3
* (all paths to the exit point of `m` from `puts "b"` must go through
* `puts "m"`).
*/
predicate postDominates(BasicBlock bb) {
this.strictlyPostDominates(bb) or
this = bb
}
/** Holds if this basic block is in a loop in the control flow graph. */
predicate inLoop() { this.getASuccessor+() = this }
/** Gets a textual representation of this basic block. */
string toString() { result = this.getFirstNode().toString() }
/** Gets the location of this basic block. */
Location getLocation() { result = this.getFirstNode().getLocation() }
}
cached
private module Cached {
/** Internal representation of basic blocks. */
cached
newtype TBasicBlock = TBasicBlockStart(CfgNode cfn) { startsBB(cfn) }
/** Holds if `cfn` starts a new basic block. */
private predicate startsBB(CfgNode cfn) {
not exists(cfn.getAPredecessor()) and exists(cfn.getASuccessor())
or
cfn.isJoin()
or
cfn.getAPredecessor().isBranch()
}
/**
* Holds if `succ` is a control flow successor of `pred` within
* the same basic block.
*/
private predicate intraBBSucc(CfgNode pred, CfgNode succ) {
succ = pred.getASuccessor() and
not startsBB(succ)
}
/**
* Holds if `cfn` is the `i`th node in basic block `bb`.
*
* In other words, `i` is the shortest distance from a node `bb`
* that starts a basic block to `cfn` along the `intraBBSucc` relation.
*/
cached
predicate bbIndex(CfgNode bbStart, CfgNode cfn, int i) =
shortestDistances(startsBB/1, intraBBSucc/2)(bbStart, cfn, i)
/**
* Holds if the first node of basic block `succ` is a control flow
* successor of the last node of basic block `pred`.
*/
private predicate succBB(BasicBlock pred, BasicBlock succ) { succ = pred.getASuccessor() }
/** Holds if `dom` is an immediate dominator of `bb`. */
cached
predicate bbIDominates(BasicBlock dom, BasicBlock bb) =
idominance(entryBB/1, succBB/2)(_, dom, bb)
/** Holds if `pred` is a basic block predecessor of `succ`. */
private predicate predBB(BasicBlock succ, BasicBlock pred) { succBB(pred, succ) }
/** Holds if `bb` is an exit basic block that represents normal exit. */
private predicate normalExitBB(BasicBlock bb) { bb.getANode().(AnnotatedExitNode).isNormal() }
/** Holds if `dom` is an immediate post-dominator of `bb`. */
cached
predicate bbIPostDominates(BasicBlock dom, BasicBlock bb) =
idominance(normalExitBB/1, predBB/2)(_, dom, bb)
/**
* Gets the `i`th predecessor of join block `jb`, with respect to some
* arbitrary order.
*/
cached
JoinBlockPredecessor getJoinBlockPredecessor(JoinBlock jb, int i) {
result =
rank[i + 1](JoinBlockPredecessor jbp |
jbp = jb.getAPredecessor()
|
jbp order by JoinBlockPredecessors::getId(jbp), JoinBlockPredecessors::getSplitString(jbp)
)
}
}
private import Cached
/** Holds if `bb` is an entry basic block. */
private predicate entryBB(BasicBlock bb) { bb.getFirstNode() instanceof EntryNode }
/**
* An entry basic block, that is, a basic block whose first node is
* an entry node.
*/
class EntryBasicBlock extends BasicBlock {
EntryBasicBlock() { entryBB(this) }
override CfgScope getScope() { this.getFirstNode() = TEntryNode(result) }
}
/**
* An annotated exit basic block, that is, a basic block whose last node is
* an annotated exit node.
*/
class AnnotatedExitBasicBlock extends BasicBlock {
private boolean normal;
AnnotatedExitBasicBlock() {
exists(AnnotatedExitNode n |
n = this.getANode() and
if n.isNormal() then normal = true else normal = false
)
}
/** Holds if this block represent a normal exit. */
final predicate isNormal() { normal = true }
}
/**
* An exit basic block, that is, a basic block whose last node is
* an exit node.
*/
class ExitBasicBlock extends BasicBlock {
ExitBasicBlock() { this.getLastNode() instanceof ExitNode }
}
private module JoinBlockPredecessors {
private predicate id(Ruby::AstNode x, Ruby::AstNode y) { x = y }
private predicate idOf(Ruby::AstNode x, int y) = equivalenceRelation(id/2)(x, y)
int getId(JoinBlockPredecessor jbp) {
idOf(toGeneratedInclSynth(jbp.getFirstNode().(AstCfgNode).getNode()), result)
or
idOf(toGeneratedInclSynth(jbp.(EntryBasicBlock).getScope()), result)
}
string getSplitString(JoinBlockPredecessor jbp) {
result = jbp.getFirstNode().(AstCfgNode).getSplitsString()
or
not exists(jbp.getFirstNode().(AstCfgNode).getSplitsString()) and
result = ""
}
}
/** A basic block with more than one predecessor. */
class JoinBlock extends BasicBlock {
JoinBlock() { getFirstNode().isJoin() }
/**
* Gets the `i`th predecessor of this join block, with respect to some
* arbitrary order.
*/
JoinBlockPredecessor getJoinBlockPredecessor(int i) { result = getJoinBlockPredecessor(this, i) }
}
/** A basic block that is an immediate predecessor of a join block. */
class JoinBlockPredecessor extends BasicBlock {
JoinBlockPredecessor() { this.getASuccessor() instanceof JoinBlock }
}
/** A basic block that terminates in a condition, splitting the subsequent control flow. */
class ConditionBlock extends BasicBlock {
ConditionBlock() { this.getLastNode().isCondition() }
/**
* Holds if basic block `succ` is immediately controlled by this basic
* block with conditional value `s`. That is, `succ` is an immediate
* successor of this block, and `succ` can only be reached from
* the callable entry point by going via the `s` edge out of this basic block.
*/
pragma[nomagic]
predicate immediatelyControls(BasicBlock succ, BooleanSuccessor s) {
succ = this.getASuccessor(s) and
forall(BasicBlock pred | pred = succ.getAPredecessor() and pred != this | succ.dominates(pred))
}
/**
* Holds if basic block `controlled` is controlled by this basic block with
* conditional value `s`. That is, `controlled` can only be reached from
* the callable entry point by going via the `s` edge out of this basic block.
*/
predicate controls(BasicBlock controlled, BooleanSuccessor s) {
exists(BasicBlock succ | this.immediatelyControls(succ, s) | succ.dominates(controlled))
}
}

View File

@@ -0,0 +1,418 @@
/** Provides classes representing nodes in a control flow graph. */
private import codeql.ruby.AST
private import codeql.ruby.controlflow.BasicBlocks
private import ControlFlowGraph
private import internal.ControlFlowGraphImpl
private import internal.Splitting
/** An entry node for a given scope. */
class EntryNode extends CfgNode, TEntryNode {
private CfgScope scope;
EntryNode() { this = TEntryNode(scope) }
final override EntryBasicBlock getBasicBlock() { result = CfgNode.super.getBasicBlock() }
final override Location getLocation() { result = scope.getLocation() }
final override string toString() { result = "enter " + scope }
}
/** An exit node for a given scope, annotated with the type of exit. */
class AnnotatedExitNode extends CfgNode, TAnnotatedExitNode {
private CfgScope scope;
private boolean normal;
AnnotatedExitNode() { this = TAnnotatedExitNode(scope, normal) }
/** Holds if this node represent a normal exit. */
final predicate isNormal() { normal = true }
final override AnnotatedExitBasicBlock getBasicBlock() { result = CfgNode.super.getBasicBlock() }
final override Location getLocation() { result = scope.getLocation() }
final override string toString() {
exists(string s |
normal = true and s = "normal"
or
normal = false and s = "abnormal"
|
result = "exit " + scope + " (" + s + ")"
)
}
}
/** An exit node for a given scope. */
class ExitNode extends CfgNode, TExitNode {
private CfgScope scope;
ExitNode() { this = TExitNode(scope) }
final override Location getLocation() { result = scope.getLocation() }
final override string toString() { result = "exit " + scope }
}
/**
* A node for an AST node.
*
* Each AST node maps to zero or more `AstCfgNode`s: zero when the node in unreachable
* (dead) code or not important for control flow, and multiple when there are different
* splits for the AST node.
*/
class AstCfgNode extends CfgNode, TElementNode {
private Splits splits;
private AstNode n;
AstCfgNode() { this = TElementNode(n, splits) }
final override AstNode getNode() { result = n }
override Location getLocation() { result = n.getLocation() }
final override string toString() {
exists(string s | s = n.(AstNode).toString() |
result = "[" + this.getSplitsString() + "] " + s
or
not exists(this.getSplitsString()) and result = s
)
}
/** Gets a comma-separated list of strings for each split in this node, if any. */
final string getSplitsString() {
result = splits.toString() and
result != ""
}
/** Gets a split for this control flow node, if any. */
final Split getASplit() { result = splits.getASplit() }
}
/** A control-flow node that wraps an AST expression. */
class ExprCfgNode extends AstCfgNode {
Expr e;
ExprCfgNode() { e = this.getNode() }
/** Gets the underlying expression. */
Expr getExpr() { result = e }
}
/** A control-flow node that wraps a return-like statement. */
class ReturningCfgNode extends AstCfgNode {
ReturningStmt s;
ReturningCfgNode() { s = this.getNode() }
/** Gets the node of the returned value, if any. */
ExprCfgNode getReturnedValueNode() {
result = this.getAPredecessor() and
result.getNode() = s.getValue()
}
}
/** A control-flow node that wraps a `StringComponent` AST expression. */
class StringComponentCfgNode extends AstCfgNode {
StringComponentCfgNode() { this.getNode() instanceof StringComponent }
}
private Expr desugar(Expr n) {
result = n.getDesugared()
or
not exists(n.getDesugared()) and
result = n
}
/**
* A class for mapping parent-child AST nodes to parent-child CFG nodes.
*/
abstract private class ExprChildMapping extends Expr {
/**
* Holds if `child` is a (possibly nested) child of this expression
* for which we would like to find a matching CFG child.
*/
abstract predicate relevantChild(Expr child);
pragma[nomagic]
private predicate reachesBasicBlock(Expr child, CfgNode cfn, BasicBlock bb) {
this.relevantChild(child) and
cfn = this.getAControlFlowNode() and
bb.getANode() = cfn
or
exists(BasicBlock mid |
this.reachesBasicBlock(child, cfn, mid) and
bb = mid.getAPredecessor() and
not mid.getANode().getNode() = child
)
}
/**
* Holds if there is a control-flow path from `cfn` to `cfnChild`, where `cfn`
* is a control-flow node for this expression, and `cfnChild` is a control-flow
* node for `child`.
*
* The path never escapes the syntactic scope of this expression.
*/
cached
predicate hasCfgChild(Expr child, CfgNode cfn, CfgNode cfnChild) {
this.reachesBasicBlock(child, cfn, cfnChild.getBasicBlock()) and
cfnChild = desugar(child).getAControlFlowNode()
}
}
/** Provides classes for control-flow nodes that wrap AST expressions. */
module ExprNodes {
// TODO: Add more classes
private class AssignExprChildMapping extends ExprChildMapping, AssignExpr {
override predicate relevantChild(Expr e) { e = this.getAnOperand() }
}
/** A control-flow node that wraps an `AssignExpr` AST expression. */
class AssignExprCfgNode extends ExprCfgNode {
override AssignExprChildMapping e;
final override AssignExpr getExpr() { result = ExprCfgNode.super.getExpr() }
/** Gets the LHS of this assignment. */
final ExprCfgNode getLhs() { e.hasCfgChild(e.getLeftOperand(), this, result) }
/** Gets the RHS of this assignment. */
final ExprCfgNode getRhs() { e.hasCfgChild(e.getRightOperand(), this, result) }
}
private class OperationExprChildMapping extends ExprChildMapping, Operation {
override predicate relevantChild(Expr e) { e = this.getAnOperand() }
}
/** A control-flow node that wraps an `Operation` AST expression. */
class OperationCfgNode extends ExprCfgNode {
override OperationExprChildMapping e;
override Operation getExpr() { result = super.getExpr() }
/** Gets an operand of this operation. */
final ExprCfgNode getAnOperand() { e.hasCfgChild(e.getAnOperand(), this, result) }
}
/** A control-flow node that wraps a `BinaryOperation` AST expression. */
class BinaryOperationCfgNode extends OperationCfgNode {
private BinaryOperation bo;
BinaryOperationCfgNode() { e = bo }
override BinaryOperation getExpr() { result = super.getExpr() }
/** Gets the left operand of this binary operation. */
final ExprCfgNode getLeftOperand() { e.hasCfgChild(bo.getLeftOperand(), this, result) }
/** Gets the right operand of this binary operation. */
final ExprCfgNode getRightOperand() { e.hasCfgChild(bo.getRightOperand(), this, result) }
}
private class BlockArgumentChildMapping extends ExprChildMapping, BlockArgument {
override predicate relevantChild(Expr e) { e = this.getValue() }
}
/** A control-flow node that wraps a `BlockArgument` AST expression. */
class BlockArgumentCfgNode extends ExprCfgNode {
override BlockArgumentChildMapping e;
final override BlockArgument getExpr() { result = ExprCfgNode.super.getExpr() }
/** Gets the value of this block argument. */
final ExprCfgNode getValue() { e.hasCfgChild(e.getValue(), this, result) }
}
private class CallExprChildMapping extends ExprChildMapping, Call {
override predicate relevantChild(Expr e) {
e = [this.getAnArgument(), this.(MethodCall).getReceiver(), this.(MethodCall).getBlock()]
}
}
/** A control-flow node that wraps a `Call` AST expression. */
class CallCfgNode extends ExprCfgNode {
override CallExprChildMapping e;
override Call getExpr() { result = super.getExpr() }
/** Gets the `n`th argument of this call. */
final ExprCfgNode getArgument(int n) { e.hasCfgChild(e.getArgument(n), this, result) }
/** Gets the the keyword argument whose key is `keyword` of this call. */
final ExprCfgNode getKeywordArgument(string keyword) {
e.hasCfgChild(e.getKeywordArgument(keyword), this, result)
}
/** Gets the number of arguments of this call. */
final int getNumberOfArguments() { result = e.getNumberOfArguments() }
/** Gets the receiver of this call. */
final ExprCfgNode getReceiver() { e.hasCfgChild(e.(MethodCall).getReceiver(), this, result) }
/** Gets the block of this call. */
final ExprCfgNode getBlock() { e.hasCfgChild(e.(MethodCall).getBlock(), this, result) }
}
private class CaseExprChildMapping extends ExprChildMapping, CaseExpr {
override predicate relevantChild(Expr e) { e = this.getValue() or e = this.getBranch(_) }
}
/** A control-flow node that wraps a `MethodCall` AST expression. */
class MethodCallCfgNode extends CallCfgNode {
MethodCallCfgNode() { super.getExpr() instanceof MethodCall }
override MethodCall getExpr() { result = super.getExpr() }
}
/** A control-flow node that wraps a `CaseExpr` AST expression. */
class CaseExprCfgNode extends ExprCfgNode {
override CaseExprChildMapping e;
final override CaseExpr getExpr() { result = ExprCfgNode.super.getExpr() }
/** Gets the expression being compared, if any. */
final ExprCfgNode getValue() { e.hasCfgChild(e.getValue(), this, result) }
/**
* Gets the `n`th branch of this case expression.
*/
final ExprCfgNode getBranch(int n) { e.hasCfgChild(e.getBranch(n), this, result) }
}
private class ConditionalExprChildMapping extends ExprChildMapping, ConditionalExpr {
override predicate relevantChild(Expr e) { e = this.getCondition() or e = this.getBranch(_) }
}
/** A control-flow node that wraps a `ConditionalExpr` AST expression. */
class ConditionalExprCfgNode extends ExprCfgNode {
override ConditionalExprChildMapping e;
final override ConditionalExpr getExpr() { result = ExprCfgNode.super.getExpr() }
/** Gets the condition expression. */
final ExprCfgNode getCondition() { e.hasCfgChild(e.getCondition(), this, result) }
/**
* Gets the branch of this conditional expression that is taken when the condition
* evaluates to cond, if any.
*/
final ExprCfgNode getBranch(boolean cond) { e.hasCfgChild(e.getBranch(cond), this, result) }
}
private class ConstantAccessChildMapping extends ExprChildMapping, ConstantAccess {
override predicate relevantChild(Expr e) { e = this.getScopeExpr() }
}
/** A control-flow node that wraps a `ConditionalExpr` AST expression. */
class ConstantAccessCfgNode extends ExprCfgNode {
override ConstantAccessChildMapping e;
final override ConstantAccess getExpr() { result = super.getExpr() }
/** Gets the scope expression. */
final ExprCfgNode getScopeExpr() { e.hasCfgChild(e.getScopeExpr(), this, result) }
}
private class StmtSequenceChildMapping extends ExprChildMapping, StmtSequence {
override predicate relevantChild(Expr e) { e = this.getLastStmt() }
}
/** A control-flow node that wraps a `StmtSequence` AST expression. */
class StmtSequenceCfgNode extends ExprCfgNode {
override StmtSequenceChildMapping e;
final override StmtSequence getExpr() { result = ExprCfgNode.super.getExpr() }
/** Gets the last statement in this sequence, if any. */
final ExprCfgNode getLastStmt() { e.hasCfgChild(e.getLastStmt(), this, result) }
}
private class ForExprChildMapping extends ExprChildMapping, ForExpr {
override predicate relevantChild(Expr e) { e = this.getValue() }
}
/** A control-flow node that wraps a `ForExpr` AST expression. */
class ForExprCfgNode extends ExprCfgNode {
override ForExprChildMapping e;
final override ForExpr getExpr() { result = ExprCfgNode.super.getExpr() }
/** Gets the value being iterated over. */
final ExprCfgNode getValue() { e.hasCfgChild(e.getValue(), this, result) }
}
/** A control-flow node that wraps a `ParenthesizedExpr` AST expression. */
class ParenthesizedExprCfgNode extends StmtSequenceCfgNode {
ParenthesizedExprCfgNode() { this.getExpr() instanceof ParenthesizedExpr }
}
/** A control-flow node that wraps a `VariableReadAccess` AST expression. */
class VariableReadAccessCfgNode extends ExprCfgNode {
override VariableReadAccess e;
final override VariableReadAccess getExpr() { result = ExprCfgNode.super.getExpr() }
}
/** A control-flow node that wraps a `InstanceVariableWriteAccess` AST expression. */
class InstanceVariableWriteAccessCfgNode extends ExprCfgNode {
override InstanceVariableWriteAccess e;
final override InstanceVariableWriteAccess getExpr() { result = ExprCfgNode.super.getExpr() }
}
/** A control-flow node that wraps a `StringInterpolationComponent` AST expression. */
class StringInterpolationComponentCfgNode extends StmtSequenceCfgNode {
StringInterpolationComponentCfgNode() { this.getNode() instanceof StringInterpolationComponent }
}
private class StringlikeLiteralChildMapping extends ExprChildMapping, StringlikeLiteral {
override predicate relevantChild(Expr e) { e = this.getComponent(_) }
}
/** A control-flow node that wraps a `StringlikeLiteral` AST expression. */
class StringlikeLiteralCfgNode extends ExprCfgNode {
override StringlikeLiteralChildMapping e;
final override StringlikeLiteral getExpr() { result = super.getExpr() }
/** Gets a component of this `StringlikeLiteral` */
StringComponentCfgNode getAComponent() { e.hasCfgChild(e.getComponent(_), this, result) }
}
/** A control-flow node that wraps a `StringLiteral` AST expression. */
class StringLiteralCfgNode extends ExprCfgNode {
override StringLiteral e;
final override StringLiteral getExpr() { result = super.getExpr() }
}
/** A control-flow node that wraps a `RegExpLiteral` AST expression. */
class RegExpLiteralCfgNode extends ExprCfgNode {
override RegExpLiteral e;
final override RegExpLiteral getExpr() { result = super.getExpr() }
}
/** A control-flow node that wraps a `ComparisonOperation` AST expression. */
class ComparisonOperationCfgNode extends BinaryOperationCfgNode {
ComparisonOperationCfgNode() { e instanceof ComparisonOperation }
override ComparisonOperation getExpr() { result = super.getExpr() }
}
/** A control-flow node that wraps a `RelationalOperation` AST expression. */
class RelationalOperationCfgNode extends ComparisonOperationCfgNode {
RelationalOperationCfgNode() { e instanceof RelationalOperation }
final override RelationalOperation getExpr() { result = super.getExpr() }
}
/** A control-flow node that wraps an `ElementReference` AST expression. */
class ElementReferenceCfgNode extends MethodCallCfgNode {
ElementReferenceCfgNode() { e instanceof ElementReference }
final override ElementReference getExpr() { result = super.getExpr() }
}
}

View File

@@ -0,0 +1,341 @@
/** Provides classes representing the control flow graph. */
private import codeql.Locations
private import codeql.ruby.AST
private import codeql.ruby.controlflow.BasicBlocks
private import SuccessorTypes
private import internal.ControlFlowGraphImpl
private import internal.Splitting
private import internal.Completion
/** An AST node with an associated control-flow graph. */
class CfgScope extends Scope instanceof CfgScope::Range_ {
/** Gets the CFG scope that this scope is nested under, if any. */
final CfgScope getOuterCfgScope() {
exists(AstNode parent |
parent = this.getParent() and
result = getCfgScope(parent)
)
}
}
/**
* A control flow node.
*
* A control flow node is a node in the control flow graph (CFG). There is a
* many-to-one relationship between CFG nodes and AST nodes.
*
* Only nodes that can be reached from an entry point are included in the CFG.
*/
class CfgNode extends TNode {
/** Gets a textual representation of this control flow node. */
string toString() { none() }
/** Gets the AST node that this node corresponds to, if any. */
AstNode getNode() { none() }
/** Gets the location of this control flow node. */
Location getLocation() { none() }
/** Gets the file of this control flow node. */
final File getFile() { result = this.getLocation().getFile() }
/** Holds if this control flow node has conditional successors. */
final predicate isCondition() { exists(this.getASuccessor(any(BooleanSuccessor bs))) }
/** Gets the scope of this node. */
final CfgScope getScope() { result = this.getBasicBlock().getScope() }
/** Gets the basic block that this control flow node belongs to. */
BasicBlock getBasicBlock() { result.getANode() = this }
/** Gets a successor node of a given type, if any. */
final CfgNode getASuccessor(SuccessorType t) { result = getASuccessor(this, t) }
/** Gets an immediate successor, if any. */
final CfgNode getASuccessor() { result = this.getASuccessor(_) }
/** Gets an immediate predecessor node of a given flow type, if any. */
final CfgNode getAPredecessor(SuccessorType t) { result.getASuccessor(t) = this }
/** Gets an immediate predecessor, if any. */
final CfgNode getAPredecessor() { result = this.getAPredecessor(_) }
/** Holds if this node has more than one predecessor. */
final predicate isJoin() { strictcount(this.getAPredecessor()) > 1 }
/** Holds if this node has more than one successor. */
final predicate isBranch() { strictcount(this.getASuccessor()) > 1 }
}
/** The type of a control flow successor. */
class SuccessorType extends TSuccessorType {
/** Gets a textual representation of successor type. */
string toString() { none() }
}
/** Provides different types of control flow successor types. */
module SuccessorTypes {
/** A normal control flow successor. */
class NormalSuccessor extends SuccessorType, TSuccessorSuccessor {
final override string toString() { result = "successor" }
}
/**
* A conditional control flow successor. Either a Boolean successor (`BooleanSuccessor`),
* an emptiness successor (`EmptinessSuccessor`), or a matching successor
* (`MatchingSuccessor`)
*/
class ConditionalSuccessor extends SuccessorType {
boolean value;
ConditionalSuccessor() {
this = TBooleanSuccessor(value) or
this = TEmptinessSuccessor(value) or
this = TMatchingSuccessor(value)
}
/** Gets the Boolean value of this successor. */
final boolean getValue() { result = value }
override string toString() { result = getValue().toString() }
}
/**
* A Boolean control flow successor.
*
* For example, in
*
* ```rb
* if x >= 0
* puts "positive"
* else
* puts "negative"
* end
* ```
*
* `x >= 0` has both a `true` successor and a `false` successor.
*/
class BooleanSuccessor extends ConditionalSuccessor, TBooleanSuccessor { }
/**
* An emptiness control flow successor.
*
* For example, this program fragment:
*
* ```rb
* for arg in args do
* puts arg
* end
* puts "done";
* ```
*
* has a control flow graph containing emptiness successors:
*
* ```
* args
* |
* for------<-----
* / \ \
* / \ |
* / \ |
* / \ |
* empty non-empty |
* | \ |
* puts "done" \ |
* arg |
* | |
* puts arg |
* \___/
* ```
*/
class EmptinessSuccessor extends ConditionalSuccessor, TEmptinessSuccessor {
override string toString() { if value = true then result = "empty" else result = "non-empty" }
}
/**
* A matching control flow successor.
*
* For example, this program fragment:
*
* ```rb
* case x
* when 1 then puts "one"
* else puts "not one"
* end
* ```
*
* has a control flow graph containing matching successors:
*
* ```
* x
* |
* 1
* / \
* / \
* / \
* / \
* match non-match
* | |
* puts "one" puts "not one"
* ```
*/
class MatchingSuccessor extends ConditionalSuccessor, TMatchingSuccessor {
override string toString() { if value = true then result = "match" else result = "no-match" }
}
/**
* A `return` control flow successor.
*
* Example:
*
* ```rb
* def sum(x,y)
* return x + y
* end
* ```
*
* The exit node of `sum` is a `return` successor of the `return x + y`
* statement.
*/
class ReturnSuccessor extends SuccessorType, TReturnSuccessor {
final override string toString() { result = "return" }
}
/**
* A `break` control flow successor.
*
* Example:
*
* ```rb
* def m
* while x >= 0
* x -= 1
* if num > 100
* break
* end
* end
* puts "done"
* end
* ```
*
* The node `puts "done"` is `break` successor of the node `break`.
*/
class BreakSuccessor extends SuccessorType, TBreakSuccessor {
final override string toString() { result = "break" }
}
/**
* A `next` control flow successor.
*
* Example:
*
* ```rb
* def m
* while x >= 0
* x -= 1
* if num > 100
* next
* end
* end
* puts "done"
* end
* ```
*
* The node `x >= 0` is `next` successor of the node `next`.
*/
class NextSuccessor extends SuccessorType, TNextSuccessor {
final override string toString() { result = "next" }
}
/**
* A `redo` control flow successor.
*
* Example:
*
* Example:
*
* ```rb
* def m
* while x >= 0
* x -= 1
* if num > 100
* redo
* end
* end
* puts "done"
* end
* ```
*
* The node `x -= 1` is `redo` successor of the node `redo`.
*/
class RedoSuccessor extends SuccessorType, TRedoSuccessor {
final override string toString() { result = "redo" }
}
/**
* A `retry` control flow successor.
*
* Example:
*
* Example:
*
* ```rb
* def m
* begin
* puts "Retry"
* raise
* rescue
* retry
* end
* end
* ```
*
* The node `puts "Retry"` is `retry` successor of the node `retry`.
*/
class RetrySuccessor extends SuccessorType, TRetrySuccessor {
final override string toString() { result = "retry" }
}
/**
* An exceptional control flow successor.
*
* Example:
*
* ```rb
* def m x
* if x > 2
* raise "x > 2"
* end
* puts "x <= 2"
* end
* ```
*
* The exit node of `m` is an exceptional successor of the node
* `raise "x > 2"`.
*/
class RaiseSuccessor extends SuccessorType, TRaiseSuccessor {
final override string toString() { result = "raise" }
}
/**
* An exit control flow successor.
*
* Example:
*
* ```rb
* def m x
* if x > 2
* exit 1
* end
* puts "x <= 2"
* end
* ```
*
* The exit node of `m` is an exit successor of the node
* `exit 1`.
*/
class ExitSuccessor extends SuccessorType, TExitSuccessor {
final override string toString() { result = "exit" }
}
}

View File

@@ -0,0 +1,507 @@
/**
* Provides classes representing control flow completions.
*
* A completion represents how a statement or expression terminates.
*/
private import codeql.ruby.AST
private import codeql.ruby.ast.internal.AST
private import codeql.ruby.controlflow.ControlFlowGraph
private import ControlFlowGraphImpl
private import NonReturning
private import SuccessorTypes
private newtype TCompletion =
TSimpleCompletion() or
TBooleanCompletion(boolean b) { b in [false, true] } or
TEmptinessCompletion(boolean isEmpty) { isEmpty in [false, true] } or
TMatchingCompletion(boolean isMatch) { isMatch in [false, true] } or
TReturnCompletion() or
TBreakCompletion() or
TNextCompletion() or
TRedoCompletion() or
TRetryCompletion() or
TRaiseCompletion() or // TODO: Add exception type?
TExitCompletion() or
TNestedCompletion(Completion inner, Completion outer, int nestLevel) {
inner = TBreakCompletion() and
outer instanceof NonNestedNormalCompletion and
nestLevel = 0
or
inner instanceof NormalCompletion and
nestedEnsureCompletion(outer, nestLevel)
}
pragma[noinline]
private predicate nestedEnsureCompletion(Completion outer, int nestLevel) {
(
outer = TReturnCompletion()
or
outer = TBreakCompletion()
or
outer = TNextCompletion()
or
outer = TRedoCompletion()
or
outer = TRetryCompletion()
or
outer = TRaiseCompletion()
or
outer = TExitCompletion()
) and
nestLevel = any(Trees::BodyStmtTree t).getNestLevel()
}
pragma[noinline]
private predicate completionIsValidForStmt(AstNode n, Completion c) {
n = TForIn(_) and
c instanceof EmptinessCompletion
or
n instanceof BreakStmt and
c = TBreakCompletion()
or
n instanceof NextStmt and
c = TNextCompletion()
or
n instanceof RedoStmt and
c = TRedoCompletion()
or
n instanceof ReturnStmt and
c = TReturnCompletion()
}
/**
* Holds if `c` happens in an exception-aware context, that is, it may be
* `rescue`d or `ensure`d. In such cases, we assume that the target of `c`
* may raise an exception (in addition to evaluating normally).
*/
private predicate mayRaise(Call c) {
exists(Trees::BodyStmtTree bst | c = bst.getBodyChild(_, true).getAChild*() |
exists(bst.getARescue())
or
exists(bst.getEnsure())
)
}
/** A completion of a statement or an expression. */
abstract class Completion extends TCompletion {
/** Holds if this completion is valid for node `n`. */
predicate isValidFor(AstNode n) {
this = n.(NonReturningCall).getACompletion()
or
completionIsValidForStmt(n, this)
or
mustHaveBooleanCompletion(n) and
(
exists(boolean value | isBooleanConstant(n, value) | this = TBooleanCompletion(value))
or
not isBooleanConstant(n, _) and
this = TBooleanCompletion(_)
)
or
mustHaveMatchingCompletion(n) and
this = TMatchingCompletion(_)
or
n = any(RescueModifierExpr parent).getBody() and this = TRaiseCompletion()
or
mayRaise(n) and
this = TRaiseCompletion()
or
not n instanceof NonReturningCall and
not completionIsValidForStmt(n, _) and
not mustHaveBooleanCompletion(n) and
not mustHaveMatchingCompletion(n) and
this = TSimpleCompletion()
}
/**
* Holds if this completion will continue a loop when it is the completion
* of a loop body.
*/
predicate continuesLoop() {
this instanceof NormalCompletion or
this instanceof NextCompletion
}
/**
* Gets the inner completion. This is either the inner completion,
* when the completion is nested, or the completion itself.
*/
Completion getInnerCompletion() { result = this }
/**
* Gets the outer completion. This is either the outer completion,
* when the completion is nested, or the completion itself.
*/
Completion getOuterCompletion() { result = this }
/** Gets a successor type that matches this completion. */
abstract SuccessorType getAMatchingSuccessorType();
/** Gets a textual representation of this completion. */
abstract string toString();
}
/** Holds if node `n` has the Boolean constant value `value`. */
private predicate isBooleanConstant(AstNode n, boolean value) {
mustHaveBooleanCompletion(n) and
(
n.(BooleanLiteral).isTrue() and
value = true
or
n.(BooleanLiteral).isFalse() and
value = false
)
}
/**
* Holds if a normal completion of `n` must be a Boolean completion.
*/
private predicate mustHaveBooleanCompletion(AstNode n) {
inBooleanContext(n) and
not n instanceof NonReturningCall
}
/**
* Holds if `n` is used in a Boolean context. That is, the value
* that `n` evaluates to determines a true/false branch successor.
*/
private predicate inBooleanContext(AstNode n) {
exists(ConditionalExpr i |
n = i.getCondition()
or
inBooleanContext(i) and
n = i.getBranch(_)
)
or
n = any(ConditionalLoop parent).getCondition()
or
exists(LogicalAndExpr parent |
n = parent.getLeftOperand()
or
inBooleanContext(parent) and
n = parent.getRightOperand()
)
or
exists(LogicalOrExpr parent |
n = parent.getLeftOperand()
or
inBooleanContext(parent) and
n = parent.getRightOperand()
)
or
n = any(NotExpr parent | inBooleanContext(parent)).getOperand()
or
n = any(StmtSequence parent | inBooleanContext(parent)).getLastStmt()
or
exists(CaseExpr c, WhenExpr w |
not exists(c.getValue()) and
c.getAWhenBranch() = w and
w.getPattern(_) = n
)
}
/**
* Holds if a normal completion of `n` must be a matching completion.
*/
private predicate mustHaveMatchingCompletion(AstNode n) {
inMatchingContext(n) and
not n instanceof NonReturningCall
}
/**
* Holds if `n` is used in a matching context. That is, whether or
* not the value of `n` matches, determines the successor.
*/
private predicate inMatchingContext(AstNode n) {
n = any(RescueClause r).getException(_)
or
exists(CaseExpr c, WhenExpr w |
exists(c.getValue()) and
c.getAWhenBranch() = w and
w.getPattern(_) = n
)
or
n.(Trees::DefaultValueParameterTree).hasDefaultValue()
}
/**
* A completion that represents normal evaluation of a statement or an
* expression.
*/
abstract class NormalCompletion extends Completion { }
abstract private class NonNestedNormalCompletion extends NormalCompletion { }
/** A simple (normal) completion. */
class SimpleCompletion extends NonNestedNormalCompletion, TSimpleCompletion {
override NormalSuccessor getAMatchingSuccessorType() { any() }
override string toString() { result = "simple" }
}
/**
* A completion that represents evaluation of an expression, whose value determines
* the successor. Either a Boolean completion (`BooleanCompletion`), an emptiness
* completion (`EmptinessCompletion`), or a matching completion (`MatchingCompletion`).
*/
abstract class ConditionalCompletion extends NonNestedNormalCompletion {
boolean value;
bindingset[value]
ConditionalCompletion() { any() }
/** Gets the Boolean value of this conditional completion. */
final boolean getValue() { result = value }
}
/**
* A completion that represents evaluation of an expression
* with a Boolean value.
*/
class BooleanCompletion extends ConditionalCompletion, TBooleanCompletion {
BooleanCompletion() { this = TBooleanCompletion(value) }
/** Gets the dual Boolean completion. */
BooleanCompletion getDual() { result = TBooleanCompletion(value.booleanNot()) }
override BooleanSuccessor getAMatchingSuccessorType() { result.getValue() = value }
override string toString() { result = value.toString() }
}
/** A Boolean `true` completion. */
class TrueCompletion extends BooleanCompletion {
TrueCompletion() { this.getValue() = true }
}
/** A Boolean `false` completion. */
class FalseCompletion extends BooleanCompletion {
FalseCompletion() { this.getValue() = false }
}
/**
* A completion that represents evaluation of an emptiness test, for example
* a test in a `for in` statement.
*/
class EmptinessCompletion extends ConditionalCompletion, TEmptinessCompletion {
EmptinessCompletion() { this = TEmptinessCompletion(value) }
override EmptinessSuccessor getAMatchingSuccessorType() { result.getValue() = value }
override string toString() { if value = true then result = "empty" else result = "non-empty" }
}
/**
* A completion that represents evaluation of a matching test, for example
* a test in a `rescue` statement.
*/
class MatchingCompletion extends ConditionalCompletion, TMatchingCompletion {
MatchingCompletion() { this = TMatchingCompletion(value) }
override MatchingSuccessor getAMatchingSuccessorType() { result.getValue() = value }
override string toString() { if value = true then result = "match" else result = "no-match" }
}
/**
* A completion that represents evaluation of a statement or an
* expression resulting in a return.
*/
class ReturnCompletion extends Completion {
ReturnCompletion() {
this = TReturnCompletion() or
this = TNestedCompletion(_, TReturnCompletion(), _)
}
override ReturnSuccessor getAMatchingSuccessorType() { any() }
override string toString() {
// `NestedCompletion` defines `toString()` for the other case
this = TReturnCompletion() and result = "return"
}
}
/**
* A completion that represents evaluation of a statement or an
* expression resulting in a break from a loop.
*/
class BreakCompletion extends Completion {
BreakCompletion() {
this = TBreakCompletion() or
this = TNestedCompletion(_, TBreakCompletion(), _)
}
override BreakSuccessor getAMatchingSuccessorType() { any() }
override string toString() {
// `NestedCompletion` defines `toString()` for the other case
this = TBreakCompletion() and result = "break"
}
}
/**
* A completion that represents evaluation of a statement or an
* expression resulting in a continuation of a loop.
*/
class NextCompletion extends Completion {
NextCompletion() {
this = TNextCompletion() or
this = TNestedCompletion(_, TNextCompletion(), _)
}
override NextSuccessor getAMatchingSuccessorType() { any() }
override string toString() {
// `NestedCompletion` defines `toString()` for the other case
this = TNextCompletion() and result = "next"
}
}
/**
* A completion that represents evaluation of a statement or an
* expression resulting in a redo of a loop iteration.
*/
class RedoCompletion extends Completion {
RedoCompletion() {
this = TRedoCompletion() or
this = TNestedCompletion(_, TRedoCompletion(), _)
}
override RedoSuccessor getAMatchingSuccessorType() { any() }
override string toString() {
// `NestedCompletion` defines `toString()` for the other case
this = TRedoCompletion() and result = "redo"
}
}
/**
* A completion that represents evaluation of a statement or an
* expression resulting in a retry.
*/
class RetryCompletion extends Completion {
RetryCompletion() {
this = TRetryCompletion() or
this = TNestedCompletion(_, TRetryCompletion(), _)
}
override RetrySuccessor getAMatchingSuccessorType() { any() }
override string toString() {
// `NestedCompletion` defines `toString()` for the other case
this = TRetryCompletion() and result = "retry"
}
}
/**
* A completion that represents evaluation of a statement or an
* expression resulting in a thrown exception.
*/
class RaiseCompletion extends Completion {
RaiseCompletion() {
this = TRaiseCompletion() or
this = TNestedCompletion(_, TRaiseCompletion(), _)
}
override RaiseSuccessor getAMatchingSuccessorType() { any() }
override string toString() {
// `NestedCompletion` defines `toString()` for the other case
this = TRaiseCompletion() and result = "raise"
}
}
/**
* A completion that represents evaluation of a statement or an
* expression resulting in an abort/exit.
*/
class ExitCompletion extends Completion {
ExitCompletion() {
this = TExitCompletion() or
this = TNestedCompletion(_, TExitCompletion(), _)
}
override ExitSuccessor getAMatchingSuccessorType() { any() }
override string toString() {
// `NestedCompletion` defines `toString()` for the other case
this = TExitCompletion() and result = "exit"
}
}
/**
* A nested completion. For example, in
*
* ```rb
* def m
* while x >= 0
* x -= 1
* if num > 100
* break
* end
* end
* puts "done"
* end
* ```
*
* the `while` loop can have a nested completion where the inner completion
* is a `break` and the outer completion is a simple successor.
*/
abstract class NestedCompletion extends Completion, TNestedCompletion {
Completion inner;
Completion outer;
int nestLevel;
NestedCompletion() { this = TNestedCompletion(inner, outer, nestLevel) }
/** Gets a completion that is compatible with the inner completion. */
abstract Completion getAnInnerCompatibleCompletion();
/** Gets the level of this nested completion. */
final int getNestLevel() { result = nestLevel }
override string toString() { result = outer + " [" + inner + "] (" + nestLevel + ")" }
}
class NestedBreakCompletion extends NormalCompletion, NestedCompletion {
NestedBreakCompletion() {
inner = TBreakCompletion() and
outer instanceof NonNestedNormalCompletion
}
override BreakCompletion getInnerCompletion() { result = inner }
override NonNestedNormalCompletion getOuterCompletion() { result = outer }
override Completion getAnInnerCompatibleCompletion() {
result = inner and
outer = TSimpleCompletion()
or
result = TNestedCompletion(outer, inner, _)
}
override SuccessorType getAMatchingSuccessorType() {
outer instanceof SimpleCompletion and
result instanceof BreakSuccessor
or
result = outer.(ConditionalCompletion).getAMatchingSuccessorType()
}
}
class NestedEnsureCompletion extends NestedCompletion {
NestedEnsureCompletion() {
inner instanceof NormalCompletion and
nestedEnsureCompletion(outer, nestLevel)
}
override NormalCompletion getInnerCompletion() { result = inner }
override Completion getOuterCompletion() { result = outer }
override Completion getAnInnerCompatibleCompletion() {
result.getOuterCompletion() = this.getInnerCompletion()
}
override SuccessorType getAMatchingSuccessorType() { none() }
}

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,945 @@
/** Provides language-independent definitions for AST-to-CFG construction. */
private import ControlFlowGraphImplSpecific
/** An element with associated control flow. */
abstract class ControlFlowTree extends ControlFlowTreeBase {
/** Holds if `first` is the first element executed within this element. */
pragma[nomagic]
abstract predicate first(ControlFlowElement first);
/**
* Holds if `last` with completion `c` is a potential last element executed
* within this element.
*/
pragma[nomagic]
abstract predicate last(ControlFlowElement last, Completion c);
/** Holds if abnormal execution of `child` should propagate upwards. */
abstract predicate propagatesAbnormal(ControlFlowElement child);
/**
* Holds if `succ` is a control flow successor for `pred`, given that `pred`
* finishes with completion `c`.
*/
pragma[nomagic]
abstract predicate succ(ControlFlowElement pred, ControlFlowElement succ, Completion c);
}
/**
* Holds if `first` is the first element executed within control flow
* element `cft`.
*/
predicate first(ControlFlowTree cft, ControlFlowElement first) { cft.first(first) }
/**
* Holds if `last` with completion `c` is a potential last element executed
* within control flow element `cft`.
*/
predicate last(ControlFlowTree cft, ControlFlowElement last, Completion c) {
cft.last(last, c)
or
exists(ControlFlowElement cfe |
cft.propagatesAbnormal(cfe) and
last(cfe, last, c) and
not completionIsNormal(c)
)
}
/**
* Holds if `succ` is a control flow successor for `pred`, given that `pred`
* finishes with completion `c`.
*/
pragma[nomagic]
predicate succ(ControlFlowElement pred, ControlFlowElement succ, Completion c) {
any(ControlFlowTree cft).succ(pred, succ, c)
}
/** An element that is executed in pre-order. */
abstract class PreOrderTree extends ControlFlowTree {
final override predicate first(ControlFlowElement first) { first = this }
}
/** An element that is executed in post-order. */
abstract class PostOrderTree extends ControlFlowTree {
override predicate last(ControlFlowElement last, Completion c) {
last = this and
completionIsValidFor(c, last)
}
}
/**
* An element where the children are evaluated following a standard left-to-right
* evaluation. The actual evaluation order is determined by the predicate
* `getChildElement()`.
*/
abstract class StandardTree extends ControlFlowTree {
/** Gets the `i`th child element, in order of evaluation. */
abstract ControlFlowElement getChildElement(int i);
private ControlFlowElement getChildElementRanked(int i) {
result =
rank[i + 1](ControlFlowElement child, int j |
child = this.getChildElement(j)
|
child order by j
)
}
/** Gets the first child node of this element. */
final ControlFlowElement getFirstChildElement() { result = this.getChildElementRanked(0) }
/** Gets the last child node of this node. */
final ControlFlowElement getLastChildElement() {
exists(int last |
result = this.getChildElementRanked(last) and
not exists(this.getChildElementRanked(last + 1))
)
}
/** Holds if this element has no children. */
predicate isLeafElement() { not exists(this.getFirstChildElement()) }
override predicate propagatesAbnormal(ControlFlowElement child) {
child = this.getChildElement(_)
}
pragma[nomagic]
override predicate succ(ControlFlowElement pred, ControlFlowElement succ, Completion c) {
exists(int i |
last(this.getChildElementRanked(i), pred, c) and
completionIsNormal(c) and
first(this.getChildElementRanked(i + 1), succ)
)
}
}
/** A standard element that is executed in pre-order. */
abstract class StandardPreOrderTree extends StandardTree, PreOrderTree {
override predicate last(ControlFlowElement last, Completion c) {
last(this.getLastChildElement(), last, c)
or
this.isLeafElement() and
completionIsValidFor(c, this) and
last = this
}
override predicate succ(ControlFlowElement pred, ControlFlowElement succ, Completion c) {
StandardTree.super.succ(pred, succ, c)
or
pred = this and
first(this.getFirstChildElement(), succ) and
completionIsSimple(c)
}
}
/** A standard element that is executed in post-order. */
abstract class StandardPostOrderTree extends StandardTree, PostOrderTree {
override predicate first(ControlFlowElement first) {
first(this.getFirstChildElement(), first)
or
not exists(this.getFirstChildElement()) and
first = this
}
override predicate succ(ControlFlowElement pred, ControlFlowElement succ, Completion c) {
StandardTree.super.succ(pred, succ, c)
or
last(this.getLastChildElement(), pred, c) and
succ = this and
completionIsNormal(c)
}
}
/** An element that is a leaf in the control flow graph. */
abstract class LeafTree extends PreOrderTree, PostOrderTree {
override predicate propagatesAbnormal(ControlFlowElement child) { none() }
override predicate succ(ControlFlowElement pred, ControlFlowElement succ, Completion c) { none() }
}
/**
* Holds if split kinds `sk1` and `sk2` may overlap. That is, they may apply
* to at least one common AST node inside `scope`.
*/
private predicate overlapping(CfgScope scope, SplitKind sk1, SplitKind sk2) {
exists(ControlFlowElement e |
sk1.appliesTo(e) and
sk2.appliesTo(e) and
scope = getCfgScope(e)
)
}
/**
* A split kind. Each control flow node can have at most one split of a
* given kind.
*/
abstract class SplitKind extends SplitKindBase {
/** Gets a split of this kind. */
SplitImpl getASplit() { result.getKind() = this }
/** Holds if some split of this kind applies to AST node `n`. */
predicate appliesTo(ControlFlowElement n) { this.getASplit().appliesTo(n) }
/**
* Gets a unique integer representing this split kind. The integer is used
* to represent sets of splits as ordered lists.
*/
abstract int getListOrder();
/** Gets the rank of this split kind among all overlapping kinds for `c`. */
private int getRank(CfgScope scope) {
this = rank[result](SplitKind sk | overlapping(scope, this, sk) | sk order by sk.getListOrder())
}
/**
* Holds if this split kind is enabled for AST node `n`. For performance reasons,
* the number of splits is restricted by the `maxSplits()` predicate.
*/
predicate isEnabled(ControlFlowElement n) {
this.appliesTo(n) and
this.getRank(getCfgScope(n)) <= maxSplits()
}
/**
* Gets the rank of this split kind among all the split kinds that apply to
* AST node `n`. The rank is based on the order defined by `getListOrder()`.
*/
int getListRank(ControlFlowElement n) {
this.isEnabled(n) and
this = rank[result](SplitKind sk | sk.appliesTo(n) | sk order by sk.getListOrder())
}
/** Gets a textual representation of this split kind. */
abstract string toString();
}
/** Provides the interface for implementing an entity to split on. */
abstract class SplitImpl extends Split {
/** Gets the kind of this split. */
abstract SplitKind getKind();
/**
* Holds if this split is entered when control passes from `pred` to `succ` with
* completion `c`.
*
* Invariant: `hasEntry(pred, succ, c) implies succ(pred, succ, c)`.
*/
abstract predicate hasEntry(ControlFlowElement pred, ControlFlowElement succ, Completion c);
/**
* Holds if this split is entered when control passes from `scope` to the entry point
* `first`.
*
* Invariant: `hasEntryScope(scope, first) implies scopeFirst(scope, first)`.
*/
abstract predicate hasEntryScope(CfgScope scope, ControlFlowElement first);
/**
* Holds if this split is left when control passes from `pred` to `succ` with
* completion `c`.
*
* Invariant: `hasExit(pred, succ, c) implies succ(pred, succ, c)`.
*/
abstract predicate hasExit(ControlFlowElement pred, ControlFlowElement succ, Completion c);
/**
* Holds if this split is left when control passes from `last` out of the enclosing
* scope `scope` with completion `c`.
*
* Invariant: `hasExitScope(scope, last, c) implies scopeLast(scope, last, c)`
*/
abstract predicate hasExitScope(CfgScope scope, ControlFlowElement last, Completion c);
/**
* Holds if this split is maintained when control passes from `pred` to `succ` with
* completion `c`.
*
* Invariant: `hasSuccessor(pred, succ, c) implies succ(pred, succ, c)`
*/
abstract predicate hasSuccessor(ControlFlowElement pred, ControlFlowElement succ, Completion c);
/** Holds if this split applies to control flow element `cfe`. */
final predicate appliesTo(ControlFlowElement cfe) {
this.hasEntry(_, cfe, _)
or
this.hasEntryScope(_, cfe)
or
exists(ControlFlowElement pred | this.appliesTo(pred) | this.hasSuccessor(pred, cfe, _))
}
/** The `succ` relation restricted to predecessors `pred` that this split applies to. */
pragma[noinline]
final predicate appliesSucc(ControlFlowElement pred, ControlFlowElement succ, Completion c) {
this.appliesTo(pred) and
succ(pred, succ, c)
}
}
/**
* A set of control flow node splits. The set is represented by a list of splits,
* ordered by ascending rank.
*/
class Splits extends TSplits {
/** Gets a textual representation of this set of splits. */
string toString() { result = splitsToString(this) }
/** Gets a split belonging to this set of splits. */
SplitImpl getASplit() {
exists(SplitImpl head, Splits tail | this = TSplitsCons(head, tail) |
result = head
or
result = tail.getASplit()
)
}
}
private predicate succEntrySplitsFromRank(
CfgScope pred, ControlFlowElement succ, Splits splits, int rnk
) {
splits = TSplitsNil() and
scopeFirst(pred, succ) and
rnk = 0
or
exists(SplitImpl head, Splits tail | succEntrySplitsCons(pred, succ, head, tail, rnk) |
splits = TSplitsCons(head, tail)
)
}
private predicate succEntrySplitsCons(
CfgScope pred, ControlFlowElement succ, SplitImpl head, Splits tail, int rnk
) {
succEntrySplitsFromRank(pred, succ, tail, rnk - 1) and
head.hasEntryScope(pred, succ) and
rnk = head.getKind().getListRank(succ)
}
/**
* Holds if `succ` with splits `succSplits` is the first element that is executed
* when entering callable `pred`.
*/
pragma[noinline]
private predicate succEntrySplits(
CfgScope pred, ControlFlowElement succ, Splits succSplits, SuccessorType t
) {
exists(int rnk |
scopeFirst(pred, succ) and
successorTypeIsSimple(t) and
succEntrySplitsFromRank(pred, succ, succSplits, rnk)
|
rnk = 0 and
not any(SplitImpl split).hasEntryScope(pred, succ)
or
rnk = max(SplitImpl split | split.hasEntryScope(pred, succ) | split.getKind().getListRank(succ))
)
}
/**
* Holds if `pred` with splits `predSplits` can exit the enclosing callable
* `succ` with type `t`.
*/
private predicate succExitSplits(
ControlFlowElement pred, Splits predSplits, CfgScope succ, SuccessorType t
) {
exists(Reachability::SameSplitsBlock b, Completion c | pred = b.getAnElement() |
b.isReachable(predSplits) and
t = getAMatchingSuccessorType(c) and
scopeLast(succ, pred, c) and
forall(SplitImpl predSplit | predSplit = predSplits.getASplit() |
predSplit.hasExitScope(succ, pred, c)
)
)
}
/**
* Provides a predicate for the successor relation with split information,
* as well as logic used to construct the type `TSplits` representing sets
* of splits. Only sets of splits that can be reached are constructed, hence
* the predicates are mutually recursive.
*
* For the successor relation
*
* ```ql
* succSplits(ControlFlowElement pred, Splits predSplits, ControlFlowElement succ, Splits succSplits, Completion c)
* ```
*
* the following invariants are maintained:
*
* 1. `pred` is reachable with split set `predSplits`.
* 2. For all `split` in `predSplits`:
* - If `split.hasSuccessor(pred, succ, c)` then `split` in `succSplits`.
* 3. For all `split` in `predSplits`:
* - If `split.hasExit(pred, succ, c)` and not `split.hasEntry(pred, succ, c)` then
* `split` not in `succSplits`.
* 4. For all `split` with kind not in `predSplits`:
* - If `split.hasEntry(pred, succ, c)` then `split` in `succSplits`.
* 5. For all `split` in `succSplits`:
* - `split.hasSuccessor(pred, succ, c)` and `split` in `predSplits`, or
* - `split.hasEntry(pred, succ, c)`.
*
* The algorithm divides into four cases:
*
* 1. The set of splits for the successor is the same as the set of splits
* for the predecessor:
* a) The successor is in the same `SameSplitsBlock` as the predecessor.
* b) The successor is *not* in the same `SameSplitsBlock` as the predecessor.
* 2. The set of splits for the successor is different from the set of splits
* for the predecessor:
* a) The set of splits for the successor is *maybe* non-empty.
* b) The set of splits for the successor is *always* empty.
*
* Only case 2a may introduce new sets of splits, so only predicates from
* this case are used in the definition of `TSplits`.
*
* The predicates in this module are named after the cases above.
*/
private module SuccSplits {
private predicate succInvariant1(
Reachability::SameSplitsBlock b, ControlFlowElement pred, Splits predSplits,
ControlFlowElement succ, Completion c
) {
pred = b.getAnElement() and
b.isReachable(predSplits) and
succ(pred, succ, c)
}
private predicate case1b0(
ControlFlowElement pred, Splits predSplits, ControlFlowElement succ, Completion c
) {
exists(Reachability::SameSplitsBlock b |
// Invariant 1
succInvariant1(b, pred, predSplits, succ, c)
|
(succ = b.getAnElement() implies succ = b) and
// Invariant 4
not exists(SplitImpl split | split.hasEntry(pred, succ, c))
)
}
/**
* Case 1b.
*
* Invariants 1 and 4 hold in the base case, and invariants 2, 3, and 5 are
* maintained for all splits in `predSplits` (= `succSplits`), except
* possibly for the splits in `except`.
*
* The predicate is written using explicit recursion, as opposed to a `forall`,
* to avoid negative recursion.
*/
private predicate case1bForall(
ControlFlowElement pred, Splits predSplits, ControlFlowElement succ, Completion c, Splits except
) {
case1b0(pred, predSplits, succ, c) and
except = predSplits
or
exists(SplitImpl split |
case1bForallCons(pred, predSplits, succ, c, split, except) and
split.hasSuccessor(pred, succ, c)
)
}
pragma[noinline]
private predicate case1bForallCons(
ControlFlowElement pred, Splits predSplits, ControlFlowElement succ, Completion c,
SplitImpl exceptHead, Splits exceptTail
) {
case1bForall(pred, predSplits, succ, c, TSplitsCons(exceptHead, exceptTail))
}
private predicate case1(
ControlFlowElement pred, Splits predSplits, ControlFlowElement succ, Completion c
) {
// Case 1a
exists(Reachability::SameSplitsBlock b | succInvariant1(b, pred, predSplits, succ, c) |
succ = b.getAnElement() and
not succ = b
)
or
// Case 1b
case1bForall(pred, predSplits, succ, c, TSplitsNil())
}
pragma[noinline]
private SplitImpl succInvariant1GetASplit(
Reachability::SameSplitsBlock b, ControlFlowElement pred, Splits predSplits,
ControlFlowElement succ, Completion c
) {
succInvariant1(b, pred, predSplits, succ, c) and
result = predSplits.getASplit()
}
private predicate case2aux(
ControlFlowElement pred, Splits predSplits, ControlFlowElement succ, Completion c
) {
exists(Reachability::SameSplitsBlock b |
succInvariant1(b, pred, predSplits, succ, c) and
(succ = b.getAnElement() implies succ = b)
|
succInvariant1GetASplit(b, pred, predSplits, succ, c).hasExit(pred, succ, c)
or
any(SplitImpl split).hasEntry(pred, succ, c)
)
}
/**
* Holds if `succSplits` should not inherit a split of kind `sk` from
* `predSplits`, except possibly because of a split in `except`.
*
* The predicate is written using explicit recursion, as opposed to a `forall`,
* to avoid negative recursion.
*/
private predicate case2aNoneInheritedOfKindForall(
ControlFlowElement pred, Splits predSplits, ControlFlowElement succ, Completion c, SplitKind sk,
Splits except
) {
case2aux(pred, predSplits, succ, c) and
sk.appliesTo(succ) and
except = predSplits
or
exists(Splits mid, SplitImpl split |
case2aNoneInheritedOfKindForall(pred, predSplits, succ, c, sk, mid) and
mid = TSplitsCons(split, except)
|
split.getKind() = any(SplitKind sk0 | sk0 != sk and sk0.appliesTo(succ))
or
split.hasExit(pred, succ, c)
)
}
pragma[nomagic]
private predicate entryOfKind(
ControlFlowElement pred, ControlFlowElement succ, Completion c, SplitImpl split, SplitKind sk
) {
split.hasEntry(pred, succ, c) and
sk = split.getKind()
}
/** Holds if `succSplits` should not have a split of kind `sk`. */
pragma[nomagic]
private predicate case2aNoneOfKind(
ControlFlowElement pred, Splits predSplits, ControlFlowElement succ, Completion c, SplitKind sk
) {
// None inherited from predecessor
case2aNoneInheritedOfKindForall(pred, predSplits, succ, c, sk, TSplitsNil()) and
// None newly entered into
not entryOfKind(pred, succ, c, _, sk)
}
/** Holds if `succSplits` should not have a split of kind `sk` at rank `rnk`. */
pragma[nomagic]
private predicate case2aNoneAtRank(
ControlFlowElement pred, Splits predSplits, ControlFlowElement succ, Completion c, int rnk
) {
exists(SplitKind sk | case2aNoneOfKind(pred, predSplits, succ, c, sk) |
rnk = sk.getListRank(succ)
)
}
pragma[nomagic]
private SplitImpl case2auxGetAPredecessorSplit(
ControlFlowElement pred, Splits predSplits, ControlFlowElement succ, Completion c
) {
case2aux(pred, predSplits, succ, c) and
result = predSplits.getASplit()
}
/** Gets a split that should be in `succSplits`. */
pragma[nomagic]
private SplitImpl case2aSome(
ControlFlowElement pred, Splits predSplits, ControlFlowElement succ, Completion c, SplitKind sk
) {
(
// Inherited from predecessor
result = case2auxGetAPredecessorSplit(pred, predSplits, succ, c) and
result.hasSuccessor(pred, succ, c)
or
// Newly entered into
exists(SplitKind sk0 |
case2aNoneInheritedOfKindForall(pred, predSplits, succ, c, sk0, TSplitsNil())
|
entryOfKind(pred, succ, c, result, sk0)
)
) and
sk = result.getKind()
}
/** Gets a split that should be in `succSplits` at rank `rnk`. */
pragma[nomagic]
SplitImpl case2aSomeAtRank(
ControlFlowElement pred, Splits predSplits, ControlFlowElement succ, Completion c, int rnk
) {
exists(SplitKind sk | result = case2aSome(pred, predSplits, succ, c, sk) |
rnk = sk.getListRank(succ)
)
}
/**
* Case 2a.
*
* As opposed to the other cases, in this case we need to construct a new set
* of splits `succSplits`. Since this involves constructing the very IPA type,
* we cannot recurse directly over the structure of `succSplits`. Instead, we
* recurse over the ranks of all splits that *might* be in `succSplits`.
*
* - Invariant 1 holds in the base case,
* - invariant 2 holds for all splits with rank at least `rnk`,
* - invariant 3 holds for all splits in `predSplits`,
* - invariant 4 holds for all splits in `succSplits` with rank at least `rnk`,
* and
* - invariant 4 holds for all splits in `succSplits` with rank at least `rnk`.
*/
predicate case2aFromRank(
ControlFlowElement pred, Splits predSplits, ControlFlowElement succ, Splits succSplits,
Completion c, int rnk
) {
case2aux(pred, predSplits, succ, c) and
succSplits = TSplitsNil() and
rnk = max(any(SplitKind sk).getListRank(succ)) + 1
or
case2aFromRank(pred, predSplits, succ, succSplits, c, rnk + 1) and
case2aNoneAtRank(pred, predSplits, succ, c, rnk)
or
exists(Splits mid, SplitImpl split | split = case2aCons(pred, predSplits, succ, mid, c, rnk) |
succSplits = TSplitsCons(split, mid)
)
}
pragma[noinline]
private SplitImpl case2aCons(
ControlFlowElement pred, Splits predSplits, ControlFlowElement succ, Splits succSplits,
Completion c, int rnk
) {
case2aFromRank(pred, predSplits, succ, succSplits, c, rnk + 1) and
result = case2aSomeAtRank(pred, predSplits, succ, c, rnk)
}
/**
* Case 2b.
*
* Invariants 1, 4, and 5 hold in the base case, and invariants 2 and 3 are
* maintained for all splits in `predSplits`, except possibly for the splits
* in `except`.
*
* The predicate is written using explicit recursion, as opposed to a `forall`,
* to avoid negative recursion.
*/
private predicate case2bForall(
ControlFlowElement pred, Splits predSplits, ControlFlowElement succ, Completion c, Splits except
) {
// Invariant 1
case2aux(pred, predSplits, succ, c) and
// Invariants 4 and 5
not any(SplitKind sk).appliesTo(succ) and
except = predSplits
or
exists(SplitImpl split | case2bForallCons(pred, predSplits, succ, c, split, except) |
// Invariants 2 and 3
split.hasExit(pred, succ, c)
)
}
pragma[noinline]
private predicate case2bForallCons(
ControlFlowElement pred, Splits predSplits, ControlFlowElement succ, Completion c,
SplitImpl exceptHead, Splits exceptTail
) {
case2bForall(pred, predSplits, succ, c, TSplitsCons(exceptHead, exceptTail))
}
private predicate case2(
ControlFlowElement pred, Splits predSplits, ControlFlowElement succ, Splits succSplits,
Completion c
) {
case2aFromRank(pred, predSplits, succ, succSplits, c, 1)
or
case2bForall(pred, predSplits, succ, c, TSplitsNil()) and
succSplits = TSplitsNil()
}
/**
* Holds if `succ` with splits `succSplits` is a successor of type `t` for `pred`
* with splits `predSplits`.
*/
predicate succSplits(
ControlFlowElement pred, Splits predSplits, ControlFlowElement succ, Splits succSplits,
Completion c
) {
case1(pred, predSplits, succ, c) and
succSplits = predSplits
or
case2(pred, predSplits, succ, succSplits, c)
}
}
import SuccSplits
/** Provides logic for calculating reachable control flow nodes. */
private module Reachability {
/**
* Holds if `cfe` is a control flow element where the set of possible splits may
* be different from the set of possible splits for one of `cfe`'s predecessors.
* That is, `cfe` starts a new block of elements with the same set of splits.
*/
private predicate startsSplits(ControlFlowElement cfe) {
scopeFirst(_, cfe)
or
exists(SplitImpl s |
s.hasEntry(_, cfe, _)
or
s.hasExit(_, cfe, _)
)
or
exists(ControlFlowElement pred, SplitImpl split, Completion c | succ(pred, cfe, c) |
split.appliesTo(pred) and
not split.hasSuccessor(pred, cfe, c)
)
}
private predicate intraSplitsSucc(ControlFlowElement pred, ControlFlowElement succ) {
succ(pred, succ, _) and
not startsSplits(succ)
}
private predicate splitsBlockContains(ControlFlowElement start, ControlFlowElement cfe) =
fastTC(intraSplitsSucc/2)(start, cfe)
/**
* A block of control flow elements where the set of splits is guaranteed
* to remain unchanged, represented by the first element in the block.
*/
class SameSplitsBlock extends ControlFlowElement {
SameSplitsBlock() { startsSplits(this) }
/** Gets a control flow element in this block. */
ControlFlowElement getAnElement() {
splitsBlockContains(this, result)
or
result = this
}
pragma[noinline]
private SameSplitsBlock getASuccessor(Splits predSplits, Splits succSplits) {
exists(ControlFlowElement pred | pred = this.getAnElement() |
succSplits(pred, predSplits, result, succSplits, _)
)
}
/**
* Holds if the elements of this block are reachable from a callable entry
* point, with the splits `splits`.
*/
predicate isReachable(Splits splits) {
// Base case
succEntrySplits(_, this, splits, _)
or
// Recursive case
exists(SameSplitsBlock pred, Splits predSplits | pred.isReachable(predSplits) |
this = pred.getASuccessor(predSplits, splits)
)
}
}
}
cached
private module Cached {
/**
* If needed, call this predicate from `ControlFlowGraphImplSpecific.qll` in order to
* force a stage-dependency on the `ControlFlowGraphImplShared.qll` stage and therby
* collapsing the two stages.
*/
cached
predicate forceCachingInSameStage() { any() }
cached
newtype TSplits =
TSplitsNil() or
TSplitsCons(SplitImpl head, Splits tail) {
exists(
ControlFlowElement pred, Splits predSplits, ControlFlowElement succ, Completion c, int rnk
|
case2aFromRank(pred, predSplits, succ, tail, c, rnk + 1) and
head = case2aSomeAtRank(pred, predSplits, succ, c, rnk)
)
or
succEntrySplitsCons(_, _, head, tail, _)
}
cached
string splitsToString(Splits splits) {
splits = TSplitsNil() and
result = ""
or
exists(SplitImpl head, Splits tail, string headString, string tailString |
splits = TSplitsCons(head, tail)
|
headString = head.toString() and
tailString = tail.toString() and
if tailString = ""
then result = headString
else
if headString = ""
then result = tailString
else result = headString + ", " + tailString
)
}
/**
* Internal representation of control flow nodes in the control flow graph.
* The control flow graph is pruned for unreachable nodes.
*/
cached
newtype TNode =
TEntryNode(CfgScope scope) { succEntrySplits(scope, _, _, _) } or
TAnnotatedExitNode(CfgScope scope, boolean normal) {
exists(Reachability::SameSplitsBlock b, SuccessorType t | b.isReachable(_) |
succExitSplits(b.getAnElement(), _, scope, t) and
if isAbnormalExitType(t) then normal = false else normal = true
)
} or
TExitNode(CfgScope scope) {
exists(Reachability::SameSplitsBlock b | b.isReachable(_) |
succExitSplits(b.getAnElement(), _, scope, _)
)
} or
TElementNode(ControlFlowElement cfe, Splits splits) {
exists(Reachability::SameSplitsBlock b | b.isReachable(splits) | cfe = b.getAnElement())
}
/** Gets a successor node of a given flow type, if any. */
cached
TNode getASuccessor(TNode pred, SuccessorType t) {
// Callable entry node -> callable body
exists(ControlFlowElement succElement, Splits succSplits, CfgScope scope |
result = TElementNode(succElement, succSplits) and
pred = TEntryNode(scope) and
succEntrySplits(scope, succElement, succSplits, t)
)
or
exists(ControlFlowElement predElement, Splits predSplits |
pred = TElementNode(predElement, predSplits)
|
// Element node -> callable exit (annotated)
exists(CfgScope scope, boolean normal |
result = TAnnotatedExitNode(scope, normal) and
succExitSplits(predElement, predSplits, scope, t) and
if isAbnormalExitType(t) then normal = false else normal = true
)
or
// Element node -> element node
exists(ControlFlowElement succElement, Splits succSplits, Completion c |
result = TElementNode(succElement, succSplits)
|
succSplits(predElement, predSplits, succElement, succSplits, c) and
t = getAMatchingSuccessorType(c)
)
)
or
// Callable exit (annotated) -> callable exit
exists(CfgScope scope |
pred = TAnnotatedExitNode(scope, _) and
result = TExitNode(scope) and
successorTypeIsSimple(t)
)
}
/**
* Gets a first control flow element executed within `cfe`.
*/
cached
ControlFlowElement getAControlFlowEntryNode(ControlFlowElement cfe) { first(cfe, result) }
/**
* Gets a potential last control flow element executed within `cfe`.
*/
cached
ControlFlowElement getAControlFlowExitNode(ControlFlowElement cfe) { last(cfe, result, _) }
}
import Cached
/**
* Import this module into a `.ql` file of `@kind graph` to render a CFG. The
* graph is restricted to nodes from `RelevantNode`.
*/
module TestOutput {
abstract class RelevantNode extends Node { }
query predicate nodes(RelevantNode n, string attr, string val) {
attr = "semmle.order" and
val =
any(int i |
n =
rank[i](RelevantNode p, Location l |
l = p.getLocation()
|
p
order by
l.getFile().getBaseName(), l.getFile().getAbsolutePath(), l.getStartLine(),
l.getStartColumn()
)
).toString()
}
query predicate edges(RelevantNode pred, RelevantNode succ, string attr, string val) {
exists(SuccessorType t | succ = getASuccessor(pred, t) |
attr = "semmle.label" and
if successorTypeIsSimple(t) then val = "" else val = t.toString()
)
}
}
/** Provides a set of splitting-related consistency queries. */
module Consistency {
query predicate nonUniqueSetRepresentation(Splits s1, Splits s2) {
forex(Split s | s = s1.getASplit() | s = s2.getASplit()) and
forex(Split s | s = s2.getASplit() | s = s1.getASplit()) and
s1 != s2
}
query predicate breakInvariant2(
ControlFlowElement pred, Splits predSplits, ControlFlowElement succ, Splits succSplits,
SplitImpl split, Completion c
) {
succSplits(pred, predSplits, succ, succSplits, c) and
split = predSplits.getASplit() and
split.hasSuccessor(pred, succ, c) and
not split = succSplits.getASplit()
}
query predicate breakInvariant3(
ControlFlowElement pred, Splits predSplits, ControlFlowElement succ, Splits succSplits,
SplitImpl split, Completion c
) {
succSplits(pred, predSplits, succ, succSplits, c) and
split = predSplits.getASplit() and
split.hasExit(pred, succ, c) and
not split.hasEntry(pred, succ, c) and
split = succSplits.getASplit()
}
query predicate breakInvariant4(
ControlFlowElement pred, Splits predSplits, ControlFlowElement succ, Splits succSplits,
SplitImpl split, Completion c
) {
succSplits(pred, predSplits, succ, succSplits, c) and
split.hasEntry(pred, succ, c) and
not split.getKind() = predSplits.getASplit().getKind() and
not split = succSplits.getASplit()
}
query predicate breakInvariant5(
ControlFlowElement pred, Splits predSplits, ControlFlowElement succ, Splits succSplits,
SplitImpl split, Completion c
) {
succSplits(pred, predSplits, succ, succSplits, c) and
split = succSplits.getASplit() and
not (split.hasSuccessor(pred, succ, c) and split = predSplits.getASplit()) and
not split.hasEntry(pred, succ, c)
}
query predicate multipleSuccessors(Node node, SuccessorType t, Node successor) {
not node instanceof TEntryNode and
strictcount(getASuccessor(node, t)) > 1 and
successor = getASuccessor(node, t)
}
}

View File

@@ -0,0 +1,74 @@
private import ruby as rb
private import ControlFlowGraphImpl as Impl
private import Completion as Comp
private import codeql.ruby.ast.internal.Synthesis
private import Splitting as Splitting
private import codeql.ruby.CFG as CFG
/** The base class for `ControlFlowTree`. */
class ControlFlowTreeBase extends rb::AstNode {
ControlFlowTreeBase() { not any(Synthesis s).excludeFromControlFlowTree(this) }
}
class ControlFlowElement = rb::AstNode;
class Completion = Comp::Completion;
/**
* Hold if `c` represents normal evaluation of a statement or an
* expression.
*/
predicate completionIsNormal(Completion c) { c instanceof Comp::NormalCompletion }
/**
* Hold if `c` represents simple (normal) evaluation of a statement or an
* expression.
*/
predicate completionIsSimple(Completion c) { c instanceof Comp::SimpleCompletion }
/** Holds if `c` is a valid completion for `e`. */
predicate completionIsValidFor(Completion c, ControlFlowElement e) { c.isValidFor(e) }
class CfgScope = CFG::CfgScope;
predicate getCfgScope = Impl::getCfgScope/1;
/** Holds if `first` is first executed when entering `scope`. */
predicate scopeFirst(CfgScope scope, ControlFlowElement first) {
scope.(Impl::CfgScope::Range_).entry(first)
}
/** Holds if `scope` is exited when `last` finishes with completion `c`. */
predicate scopeLast(CfgScope scope, ControlFlowElement last, Completion c) {
scope.(Impl::CfgScope::Range_).exit(last, c)
}
/** The maximum number of splits allowed for a given node. */
int maxSplits() { result = 5 }
class SplitKindBase = Splitting::TSplitKind;
class Split = Splitting::Split;
class SuccessorType = CFG::SuccessorType;
/** Gets a successor type that matches completion `c`. */
SuccessorType getAMatchingSuccessorType(Completion c) { result = c.getAMatchingSuccessorType() }
/**
* Hold if `c` represents simple (normal) evaluation of a statement or an
* expression.
*/
predicate successorTypeIsSimple(SuccessorType t) {
t instanceof CFG::SuccessorTypes::NormalSuccessor
}
/** Holds if `t` is an abnormal exit type out of a CFG scope. */
predicate isAbnormalExitType(SuccessorType t) {
t instanceof CFG::SuccessorTypes::RaiseSuccessor or
t instanceof CFG::SuccessorTypes::ExitSuccessor
}
class Location = rb::Location;
class Node = CFG::CfgNode;

View File

@@ -0,0 +1,22 @@
/** Provides a simple analysis for identifying calls that will not return. */
private import codeql.ruby.AST
private import Completion
/** A call that definitely does not return (conservative analysis). */
abstract class NonReturningCall extends MethodCall {
/** Gets a valid completion for this non-returning call. */
abstract Completion getACompletion();
}
private class RaiseCall extends NonReturningCall {
RaiseCall() { this.getMethodName() = "raise" }
override RaiseCompletion getACompletion() { not result instanceof NestedCompletion }
}
private class ExitCall extends NonReturningCall {
ExitCall() { this.getMethodName() in ["abort", "exit"] }
override ExitCompletion getACompletion() { not result instanceof NestedCompletion }
}

View File

@@ -0,0 +1,336 @@
/**
* Provides classes and predicates relevant for splitting the control flow graph.
*/
private import codeql.ruby.AST
private import Completion
private import ControlFlowGraphImpl
private import SuccessorTypes
private import codeql.ruby.controlflow.ControlFlowGraph
cached
private module Cached {
cached
newtype TSplitKind =
TConditionalCompletionSplitKind() { forceCachingInSameStage() } or
TEnsureSplitKind(int nestLevel) { nestLevel = any(Trees::BodyStmtTree t).getNestLevel() }
cached
newtype TSplit =
TConditionalCompletionSplit(ConditionalCompletion c) or
TEnsureSplit(EnsureSplitting::EnsureSplitType type, int nestLevel) {
nestLevel = any(Trees::BodyStmtTree t).getNestLevel()
}
}
import Cached
/** A split for a control flow node. */
class Split extends TSplit {
/** Gets a textual representation of this split. */
string toString() { none() }
}
private module ConditionalCompletionSplitting {
/**
* A split for conditional completions. For example, in
*
* ```rb
* def method x
* if x < 2 and x > 0
* puts "x is 1"
* end
* end
* ```
*
* we record whether `x < 2` and `x > 0` evaluate to `true` or `false`, and
* restrict the edges out of `x < 2 and x > 0` accordingly.
*/
class ConditionalCompletionSplit extends Split, TConditionalCompletionSplit {
ConditionalCompletion completion;
ConditionalCompletionSplit() { this = TConditionalCompletionSplit(completion) }
override string toString() { result = completion.toString() }
}
private class ConditionalCompletionSplitKind extends SplitKind, TConditionalCompletionSplitKind {
override int getListOrder() { result = 0 }
override predicate isEnabled(AstNode n) { this.appliesTo(n) }
override string toString() { result = "ConditionalCompletion" }
}
int getNextListOrder() { result = 1 }
private class ConditionalCompletionSplitImpl extends SplitImpl, ConditionalCompletionSplit {
override ConditionalCompletionSplitKind getKind() { any() }
override predicate hasEntry(AstNode pred, AstNode succ, Completion c) {
succ(pred, succ, c) and
last(succ, _, completion) and
(
last(succ.(NotExpr).getOperand(), pred, c) and
completion.(BooleanCompletion).getDual() = c
or
last(succ.(LogicalAndExpr).getAnOperand(), pred, c) and
completion = c
or
last(succ.(LogicalOrExpr).getAnOperand(), pred, c) and
completion = c
or
last(succ.(StmtSequence).getLastStmt(), pred, c) and
completion = c
or
last(succ.(ConditionalExpr).getBranch(_), pred, c) and
completion = c
)
}
override predicate hasEntryScope(CfgScope scope, AstNode succ) { none() }
override predicate hasExit(AstNode pred, AstNode succ, Completion c) {
this.appliesTo(pred) and
succ(pred, succ, c) and
if c instanceof ConditionalCompletion then completion = c else any()
}
override predicate hasExitScope(CfgScope scope, AstNode last, Completion c) {
this.appliesTo(last) and
succExit(scope, last, c) and
if c instanceof ConditionalCompletion then completion = c else any()
}
override predicate hasSuccessor(AstNode pred, AstNode succ, Completion c) { none() }
}
}
module EnsureSplitting {
/**
* The type of a split `ensure` node.
*
* The type represents one of the possible ways of entering an `ensure`
* block. For example, if a block ends with a `return` statement, then
* the `ensure` block must end with a `return` as well (provided that
* the `ensure` block executes normally).
*/
class EnsureSplitType extends SuccessorType {
EnsureSplitType() { not this instanceof ConditionalSuccessor }
/** Holds if this split type matches entry into an `ensure` block with completion `c`. */
predicate isSplitForEntryCompletion(Completion c) {
if c instanceof NormalCompletion
then
// If the entry into the `ensure` block completes with any normal completion,
// it simply means normal execution after the `ensure` block
this instanceof NormalSuccessor
else this = c.getAMatchingSuccessorType()
}
}
/** A node that belongs to an `ensure` block. */
private class EnsureNode extends AstNode {
private Trees::BodyStmtTree block;
EnsureNode() { this = block.getAnEnsureDescendant() }
int getNestLevel() { result = block.getNestLevel() }
/** Holds if this node is the entry node in the `ensure` block it belongs to. */
predicate isEntryNode() { first(block.getEnsure(), this) }
}
/**
* A split for nodes belonging to an `ensure` block, which determines how to
* continue execution after leaving the `ensure` block. For example, in
*
* ```rb
* begin
* if x
* raise "Exception"
* end
* ensure
* puts "Ensure"
* end
* ```
*
* all control flow nodes in the `ensure` block have two splits: one representing
* normal execution of the body (when `x` evaluates to `true`), and one representing
* exceptional execution of the body (when `x` evaluates to `false`).
*/
class EnsureSplit extends Split, TEnsureSplit {
private EnsureSplitType type;
private int nestLevel;
EnsureSplit() { this = TEnsureSplit(type, nestLevel) }
/**
* Gets the type of this `ensure` split, that is, how to continue execution after the
* `ensure` block.
*/
EnsureSplitType getType() { result = type }
/** Gets the nesting level. */
int getNestLevel() { result = nestLevel }
override string toString() {
if type instanceof NormalSuccessor
then result = ""
else
if nestLevel > 0
then result = "ensure(" + nestLevel + "): " + type.toString()
else result = "ensure: " + type.toString()
}
}
private int getListOrder(EnsureSplitKind kind) {
result = ConditionalCompletionSplitting::getNextListOrder() + kind.getNestLevel()
}
int getNextListOrder() {
result = max([getListOrder(_) + 1, ConditionalCompletionSplitting::getNextListOrder()])
}
private class EnsureSplitKind extends SplitKind, TEnsureSplitKind {
private int nestLevel;
EnsureSplitKind() { this = TEnsureSplitKind(nestLevel) }
/** Gets the nesting level. */
int getNestLevel() { result = nestLevel }
override int getListOrder() { result = getListOrder(this) }
override string toString() { result = "ensure (" + nestLevel + ")" }
}
pragma[noinline]
private predicate hasEntry0(AstNode pred, EnsureNode succ, int nestLevel, Completion c) {
succ.isEntryNode() and
nestLevel = succ.getNestLevel() and
succ(pred, succ, c)
}
private class EnsureSplitImpl extends SplitImpl, EnsureSplit {
override EnsureSplitKind getKind() { result.getNestLevel() = this.getNestLevel() }
override predicate hasEntry(AstNode pred, AstNode succ, Completion c) {
hasEntry0(pred, succ, this.getNestLevel(), c) and
this.getType().isSplitForEntryCompletion(c)
}
override predicate hasEntryScope(CfgScope scope, AstNode first) { none() }
/**
* Holds if this split applies to `pred`, where `pred` is a valid predecessor.
*/
private predicate appliesToPredecessor(AstNode pred) {
this.appliesTo(pred) and
(succ(pred, _, _) or succExit(_, pred, _))
}
pragma[noinline]
private predicate exit0(AstNode pred, Trees::BodyStmtTree block, int nestLevel, Completion c) {
this.appliesToPredecessor(pred) and
nestLevel = block.getNestLevel() and
block.lastInner(pred, c)
}
/**
* Holds if `pred` may exit this split with completion `c`. The Boolean
* `inherited` indicates whether `c` is an inherited completion from the
* body.
*/
private predicate exit(Trees::BodyStmtTree block, AstNode pred, Completion c, boolean inherited) {
exists(EnsureSplitType type |
exit0(pred, block, this.getNestLevel(), c) and
type = this.getType()
|
if last(block.getEnsure(), pred, c)
then
// `ensure` block can itself exit with completion `c`: either `c` must
// match this split, `c` must be an abnormal completion, or this split
// does not require another completion to be recovered
inherited = false and
(
type = c.getAMatchingSuccessorType()
or
not c instanceof NormalCompletion
or
type instanceof NormalSuccessor
)
else (
// `ensure` block can exit with inherited completion `c`, which must
// match this split
inherited = true and
type = c.getAMatchingSuccessorType() and
not type instanceof NormalSuccessor
)
)
or
// If this split is normal, and an outer split can exit based on an inherited
// completion, we need to exit this split as well. For example, in
//
// ```rb
// def m(b1, b2)
// if b1
// return
// end
// ensure
// begin
// if b2
// raise "Exception"
// end
// ensure
// puts "inner ensure"
// end
// end
// ```
//
// if the outer split for `puts "inner ensure"` is `return` and the inner split
// is "normal" (corresponding to `b1 = true` and `b2 = false`), then the inner
// split must be able to exit with a `return` completion.
this.appliesToPredecessor(pred) and
exists(EnsureSplitImpl outer |
outer.getNestLevel() = this.getNestLevel() - 1 and
outer.exit(_, pred, c, inherited) and
this.getType() instanceof NormalSuccessor and
inherited = true
)
}
override predicate hasExit(AstNode pred, AstNode succ, Completion c) {
succ(pred, succ, c) and
(
exit(_, pred, c, _)
or
exit(_, pred, c.(NestedBreakCompletion).getAnInnerCompatibleCompletion(), _)
)
}
override predicate hasExitScope(CfgScope scope, AstNode last, Completion c) {
succExit(scope, last, c) and
(
exit(_, last, c, _)
or
exit(_, last, c.(NestedBreakCompletion).getAnInnerCompatibleCompletion(), _)
)
}
override predicate hasSuccessor(AstNode pred, AstNode succ, Completion c) {
this.appliesToPredecessor(pred) and
succ(pred, succ, c) and
succ =
any(EnsureNode en |
if en.isEntryNode()
then
// entering a nested `ensure` block
en.getNestLevel() > this.getNestLevel()
else
// staying in the same (possibly nested) `ensure` block as `pred`
en.getNestLevel() >= this.getNestLevel()
)
}
}
}

Some files were not shown because too many files have changed in this diff Show More