Taus b5cac9285e Python: Fix bad join in getOuterVariable
Much sadness:

```
Tuple counts for ImportTime::ImportTimeScope::getOuterVariable#dispred#f0820431#fff/3@64d04d33 after 7.6s:
19624    ~1%     {1} r1 = SCAN py_Classes OUTPUT In.0 'this'
19531    ~1%     {1} r2 = JOIN r1 WITH ImportTime::ImportTimeScope#class#7851b601#f ON FIRST 1 OUTPUT Lhs.0 'this'
19531    ~0%     {2} r3 = JOIN r2 WITH Scope::Scope::getEnclosingModule#dispred#f0820431#ff ON FIRST 1 OUTPUT Lhs.0 'this', Rhs.1
296389   ~0%     {3} r4 = JOIN r3 WITH Variables::Variable::getScope#dispred#f0820431#ff_10#join_rhs ON FIRST 1 OUTPUT Rhs.1 'var', Lhs.0 'this', Lhs.1
296389   ~0%     {3} r5 = JOIN r4 WITH Variables::LocalVariable#3aa06bbf#f ON FIRST 1 OUTPUT Lhs.0 'var', Lhs.1 'this', Lhs.2
296389   ~1%     {4} r6 = JOIN r5 WITH Variables::Variable::getId#dispred#f0820431#ff ON FIRST 1 OUTPUT Lhs.2, Lhs.1 'this', Lhs.0 'var', Rhs.1
62294919 ~0%     {4} r7 = JOIN r6 WITH Variables::Variable::getScope#dispred#f0820431#ff_10#join_rhs ON FIRST 1 OUTPUT Rhs.1 'var', Lhs.1 'this', Lhs.2 'var', Lhs.3
62294919 ~0%     {4} r8 = JOIN r7 WITH Variables::GlobalVariable#class#3aa06bbf#f ON FIRST 1 OUTPUT Lhs.0 'result', Lhs.3, Lhs.1 'this', Lhs.2 'var'
639      ~0%     {3} r9 = JOIN r8 WITH Variables::Variable::getId#dispred#f0820431#ff ON FIRST 2 OUTPUT Lhs.2 'this', Lhs.3 'var', Lhs.0 'result'
                return r9
```

Clearly we _shouldn't_ be joining on `getId` as the last thing, as this
means we're building tuples of completely unrelated variables (not even
with the same name!) which obviously blows up.

A standard way of fixing this is to correlate as much information about
these variables as possible in a `nomagic`ked helper predicate. This is
what we do here, grouping together the variable with its scope and name
(both of which are uniquely determined by the variable). This results
in a much nicer join order:

```
Tuple counts for ImportTime::ImportTimeScope::getOuterVariable#dispred#f0820431#fff/3@82866b6p after 42ms:
23867  ~4%     {2} r1 = JOIN Scope::Scope::getEnclosingModule#dispred#f0820431#ff WITH ImportTime::ImportTimeScope#class#7851b601#f ON FIRST 1 OUTPUT Lhs.0 'this', Lhs.1
296389 ~0%     {4} r2 = JOIN r1 WITH ImportTime::class_var_scope#7851b601#fff ON FIRST 1 OUTPUT Rhs.1, Lhs.1, Lhs.0 'this', Rhs.2 'var'
639    ~0%     {3} r3 = JOIN r2 WITH ImportTime::global_var_scope#7851b601#fff ON FIRST 2 OUTPUT Lhs.2 'this', Lhs.3 'var', Rhs.2 'result'
                return r3
```
```
Tuple counts for ImportTime::class_var_scope#7851b601#fff/3@366258vr after 47ms:
19624  ~1%     {1} r1 = SCAN py_Classes OUTPUT In.0 'scope'
296743 ~0%     {2} r2 = JOIN r1 WITH Variables::Variable::getScope#dispred#f0820431#ff_10#join_rhs ON FIRST 1 OUTPUT Rhs.1 'var', Lhs.0 'scope'
296743 ~0%     {2} r3 = JOIN r2 WITH Variables::LocalVariable#3aa06bbf#f ON FIRST 1 OUTPUT Lhs.0 'var', Lhs.1 'scope'
296743 ~2%     {3} r4 = JOIN r3 WITH Variables::Variable::getId#dispred#f0820431#ff ON FIRST 1 OUTPUT Lhs.1 'scope', Rhs.1 'name', Lhs.0 'var'
                return r4
```
```
Tuple counts for ImportTime::global_var_scope#7851b601#fff/3@718e4bpm after 18ms:
108173 ~0%     {2} r1 = JOIN Variables::GlobalVariable#class#3aa06bbf#f WITH Variables::Variable::getId#dispred#f0820431#ff ON FIRST 1 OUTPUT Lhs.0 'var', Rhs.1 'name'
108173 ~0%     {3} r2 = JOIN r1 WITH Variables::Variable::getScope#dispred#f0820431#ff ON FIRST 1 OUTPUT Lhs.1 'name', Rhs.1 'scope', Lhs.0 'var'
                return r2
```

(You may be wondering what's up with the order of arguments for the two
helper predicates. By ordering the arguments this way, there's no need
to reorder the resulting relations when used in `getOuterVariable.)
2022-07-19 17:14:37 +00:00
2022-06-22 11:05:30 +02:00
2022-07-12 21:49:19 -07:00
2022-07-14 13:20:52 +02:00
2022-04-28 13:17:05 +02:00
2022-04-12 12:40:59 +02:00
2018-09-23 16:24:31 -07:00
2022-06-17 14:40:22 +02:00
2022-04-12 12:40:59 +02:00
2022-04-12 12:40:59 +02:00
2020-04-07 12:03:26 +01:00

CodeQL

This open source repository contains the standard CodeQL libraries and queries that power GitHub Advanced Security and the other application security products that GitHub makes available to its customers worldwide.

How do I learn CodeQL and run queries?

There is extensive documentation on getting started with writing CodeQL. You can use the CodeQL for Visual Studio Code extension or the interactive query console on LGTM.com (Semmle Legacy product) to try out your queries on any open source project that's currently being analyzed.

Contributing

We welcome contributions to our standard library and standard checks. Do you have an idea for a new check, or how to improve an existing query? Then please go ahead and open a pull request! Before you do, though, please take the time to read our contributing guidelines. You can also consult our style guides to learn how to format your code for consistency and clarity, how to write query metadata, and how to write query help documentation for your query.

License

The code in this repository is licensed under the MIT License by GitHub.

The CodeQL CLI (including the CodeQL engine) is hosted in a different repository and is licensed separately. If you'd like to use the CodeQL CLI to analyze closed-source code, you will need a separate commercial license; please contact us for further help.

Visual Studio Code integration

If you use Visual Studio Code to work in this repository, there are a few integration features to make development easier.

CodeQL for Visual Studio Code

You can install the CodeQL for Visual Studio Code extension to get syntax highlighting, IntelliSense, and code navigation for the QL language, as well as unit test support for testing CodeQL libraries and queries.

Tasks

The .vscode/tasks.json file defines custom tasks specific to working in this repository. To invoke one of these tasks, select the Terminal | Run Task... menu option, and then select the desired task from the dropdown. You can also invoke the Tasks: Run Task command from the command palette.

Description
CodeQL: the libraries and queries that power security researchers around the world, as well as code scanning in GitHub Advanced Security
Readme MIT 15 GiB
Languages
CodeQL 32.3%
Kotlin 27.5%
C# 17.1%
Java 7.7%
Python 4.6%
Other 10.6%