Commit Graph

40697 Commits

Author SHA1 Message Date
Alex Denisov
906ce34e2f Swift: generate QL files properly 2022-04-25 17:09:29 +02:00
Mathias Vorreiter Pedersen
8e1d079db7 Merge pull request #8858 from github/erik-krogh/enable-ql-for-ql-swift
QL: add swift to QL-for-QL
2022-04-25 16:00:56 +01:00
Alex Denisov
355504a86a Swift: use File from QL library 2022-04-25 16:58:30 +02:00
James Fletcher
5a7043f528 Update analyzing-databases-with-the-codeql-cli.rst 2022-04-25 15:57:18 +01:00
Alex Denisov
6f0ddaa431 Merge branch 'main' into alexdenisov/swift-first-extractor-test 2022-04-25 16:53:22 +02:00
Taus
d4fc096ea8 Python: Use local flow in Pythagorean.ql
The hand-rolled notion of flow was causing some severe performance
issues (on a few databases):

```
Tuple counts for Pythagorean::square#168e234a#f#loop_invariant_prefix/2@c86989kr after 6m35s:
175000     ~5%     {2} r1 = JOIN SSA::SsaVariable::getDefinition#dispred#f0820431#ff_10#join_rhs WITH Flow::ControlFlowNode::getNode#dispred#f0820431#bf ON FIRST 1 OUTPUT Lhs.1, Rhs.1 'arg0'
174500     ~6%     {2} r2 = JOIN r1 WITH SSA::SsaVariable::getVariable#dispred#f0820431#ff ON FIRST 1 OUTPUT Rhs.1, Lhs.1 'arg0'
1467782500 ~5%     {3} r3 = JOIN r2 WITH AstGenerated::Name_::getVariable#dispred#f0820431#ff_10#join_rhs ON FIRST 1 OUTPUT 3, Rhs.1 'arg1', Lhs.1 'arg0'
1467553000 ~0%     {2} r4 = JOIN r3 WITH py_expr_contexts_12#join_rhs ON FIRST 2 OUTPUT Lhs.2 'arg0', Lhs.1 'arg1'
                    return r4
```

Rewriting it to use the data flow library made all of this go away. 🎉
2022-04-25 14:35:37 +00:00
Taus
b2cc91369a Python: Fix bad join in firstUse
This was what it looked like (at the point when I killed the evaluation):

```
Tuple counts for SsaCompute::SsaComputeImpl::AdjacentUsesImpl::firstUse#c5fa2be7#ff/2@i1#be98bwif after 1m50s:
274000     ~7%     {4} r1 = SCAN SsaCompute::SsaComputeImpl::AdjacentUsesImpl::definesAt#c5fa2be7#ffff OUTPUT In.1, In.0 'def', In.2, In.3
2731768000 ~1%     {7} r2 = JOIN r1 WITH SsaCompute::SsaComputeImpl::AdjacentUsesImpl::variableSourceUse#c5fa2be7#ffff ON FIRST 1 OUTPUT Rhs.0, Lhs.2, Lhs.3, Rhs.2, Rhs.3, Rhs.1 'use', Lhs.1 'def'
178000     ~4%     {2} r3 = JOIN r2 WITH SsaCompute::SsaComputeImpl::AdjacentUsesImpl::adjacentVarRefs#c5fa2be7#fffff ON FIRST 5 OUTPUT Lhs.6 'def', Lhs.5 'use'
                    return r3
```

And this is what it looks like now:

```
Tuple counts for SsaCompute::SsaComputeImpl::AdjacentUsesImpl::firstUse#c5fa2be7#ff/2@i1#f9d6ewsi after 207ms:
931353  ~2%     {4} r1 = SCAN SsaCompute::SsaComputeImpl::AdjacentUsesImpl::variableSourceUse#c5fa2be7#ffff OUTPUT In.0, In.2, In.3, In.1 'use'
1050477 ~0%     {4} r2 = JOIN r1 WITH SsaCompute::SsaComputeImpl::AdjacentUsesImpl::adjacentVarRefs#c5fa2be7#fffff_03412#join_rhs ON FIRST 3 OUTPUT Lhs.0, Rhs.3, Rhs.4, Lhs.3 'use'
506626  ~0%     {2} r3 = JOIN r2 WITH SsaCompute::SsaComputeImpl::AdjacentUsesImpl::definesAt#c5fa2be7#ffff_1230#join_rhs ON FIRST 3 OUTPUT Rhs.3 'def', Lhs.3 'use'
                return r3
```
2022-04-25 14:33:31 +00:00
Erik Krogh Kristensen
f5e1aa7c98 QL: add swift to QL-for-QL 2022-04-25 16:29:44 +02:00
Taus
49233268a9 Python: Fix bad join in getValue
We were building essentially a CP of all control flow nodes:

```
Tuple counts for Essa::AssignmentDefinition::getValue#dispred#f0820431#ff/2@dd1f67vl after 2m45s:
733365     ~6%     {3} r1 = JOIN Essa::TEssaNodeDefinition#24e22a14#ffff_30#join_rhs WITH Essa::EssaNodeDefinition::getDefiningNode#dispred#f0820431#ff ON FIRST 1 OUTPUT Lhs.1, Rhs.1, Rhs.0
376588     ~0%     {2} r2 = JOIN r1 WITH SsaDefinitions::SsaSource::assignment_definition#9197156e#fff ON FIRST 2 OUTPUT Lhs.2 'this', Rhs.2 'result'
376588     ~0%     {3} r3 = JOIN r2 WITH Essa::TEssaNodeDefinition#24e22a14#ffff_30#join_rhs ON FIRST 1 OUTPUT Rhs.1, Lhs.0 'this', Lhs.1 'result'
6965593033 ~2%     {3} r4 = JOIN r3 WITH project#SsaDefinitions::SsaSource::assignment_definition#9197156e ON FIRST 1 OUTPUT Lhs.1 'this', Rhs.1, Lhs.2 'result'
376588     ~0%     {2} r5 = JOIN r4 WITH Essa::EssaNodeDefinition::getDefiningNode#dispred#f0820431#ff ON FIRST 2 OUTPUT Lhs.0 'this', Lhs.2 'result'
                    return r5
```

We first tried preventing the join on `result`, but this caused the
characteristic predicate to blow up instead. Finally, we figured just
putting the `value` part in a field would be sufficient, and this did
the trick.
2022-04-25 14:28:00 +00:00
Tony Torralba
85d5b122f7 Merge pull request #8817 from atorralba/atorralba/cleartext-storage-sharedprefs-improvs
Java: Add value-preserving flow steps for Android's SharedPreferences
2022-04-25 16:16:46 +02:00
Erik Krogh Kristensen
0a26e891a2 include startsWith/endsWith checks in js/missing-origin-check 2022-04-25 15:28:50 +02:00
Erik Krogh Kristensen
17005dde2d QL: fix query-id, and add description 2022-04-25 15:21:35 +02:00
Mathias Vorreiter Pedersen
3199a690aa Merge pull request #8854 from redsun82/swift-ql-gen
Swift: QL generation script
2022-04-25 14:05:25 +01:00
Paolo Tranquilli
643471f400 add temporary exception for Swift for QLdoc 2022-04-25 14:24:22 +02:00
Erik Krogh Kristensen
fe3d71ebc2 fix qhelp: the window, not the origin, is sending the message
Co-authored-by: Esben Sparre Andreasen <esbena@github.com>
2022-04-25 14:07:01 +02:00
Paolo Tranquilli
cdb10a2151 Swift: fix codegen check 2022-04-25 14:03:48 +02:00
Paolo Tranquilli
9bf4c72085 Swift: split codegen action and fix dependencies 2022-04-25 14:00:41 +02:00
Anders Schack-Mulligen
60eb341b49 Merge pull request #8851 from aschackmull/shared/accesspathsyntax-cleanup
Minor clean-up in AccessPathSyntax.qll
2022-04-25 13:47:22 +02:00
Mathias Vorreiter Pedersen
faaa2cedcd Merge pull request #8849 from JohnMcSandwich/main
C++: add new Windows pool allocation functions in `Allocation.qll`
2022-04-25 12:46:31 +01:00
Paolo Tranquilli
de0fa9e456 Swift: QL generation script
Also added code generation to the swift checks.
2022-04-25 13:23:36 +02:00
Anders Schack-Mulligen
c06efa1f42 Dataflow: Sync. 2022-04-25 13:11:04 +02:00
Anders Schack-Mulligen
f4d93f089a Dataflow: Fix join-on-config producing a CP. 2022-04-25 13:10:31 +02:00
Tony Torralba
f1e5e57d76 Update java/ql/lib/semmle/code/java/frameworks/android/SharedPreferences.qll 2022-04-25 12:39:01 +02:00
Anders Schack-Mulligen
40a16325a9 Minor clean-up in AccessPathSyntax. 2022-04-25 12:27:48 +02:00
Alex Denisov
5b20d580be Swift: use parenthesis instead of curly braces 2022-04-25 12:26:34 +02:00
Alex Ford
b956616a56 Ruby: fix alert 2022-04-25 11:25:57 +01:00
Anders Schack-Mulligen
cbdd4927ce Merge pull request #8582 from Marcono1234/marcono1234/JumpStmt-superclass
Java: Make `JumpStmt` a proper superclass
2022-04-25 12:22:20 +02:00
Anders Schack-Mulligen
fd2904d49c Merge pull request #8760 from Marcono1234/patch-1
Clarify `min`, `max` and `rank` documentation
2022-04-25 12:20:00 +02:00
Tom Hvitved
bffa8fa7cb Merge pull request #8641 from hvitved/dataflow/interpret-read-store
Data flow: Introduce `ContentSet`
2022-04-25 12:17:34 +02:00
Mathias Vorreiter Pedersen
516ef8d27a Update cpp/ql/lib/change-notes/2022-04-25-windows-pool-allocation-functions.md
Co-authored-by: Jeroen Ketema <93738568+jketema@users.noreply.github.com>
2022-04-25 10:54:12 +01:00
Mathias Vorreiter Pedersen
12c8d9c60e C++: Add change note. 2022-04-25 10:51:49 +01:00
Alex Denisov
8bcdfb2e4f Swift: initialize LLVM
No need to shutdown LLVM, it's done by the PROGRAM_START macro
2022-04-25 11:49:21 +02:00
Alex Denisov
462133e0f0 Swift: add more comments 2022-04-25 11:48:51 +02:00
Erik Krogh Kristensen
b5193d99d7 have getSourceType() depend on which kind of event it is 2022-04-25 11:32:52 +02:00
Anders Schack-Mulligen
b21f077e8e Update java/ql/lib/semmle/code/java/Statement.qll 2022-04-25 11:02:23 +02:00
Jeroen Ketema
ba2a884a45 Merge pull request #8818 from jketema/links
Replace `help.semmle.com` links by `codeql.github.com` links
2022-04-25 10:25:42 +02:00
JohnMcSandwich
b3dff77d1a C++: add new Windows pool allocation functions in Allocation.qll
Add:
 - ExAllocatePool2
 - ExAllocatePool3
 - ExAllocatePoolZero
2022-04-25 10:21:42 +02:00
Mathias Vorreiter Pedersen
e8b6bfbe0e Merge pull request #8813 from jketema/buffer
C++: Cover variable sized member arrays without a size in `Buffer.qll`
2022-04-25 09:20:31 +01:00
Tom Hvitved
2466288656 Data flow: Simplify revFlowStore 2022-04-25 10:11:54 +02:00
Tom Hvitved
cf0a1e748a Add change notes 2022-04-25 09:17:40 +02:00
Alex Ford
869d827768 Revert "Ruby: extract rbi files"
This reverts commit ba9342e0f38f4468d04bad987a8f69e95fdbdd5f.
2022-04-24 22:48:52 +01:00
Alex Ford
de35bd987a Ruby: extract rbi files 2022-04-24 22:48:52 +01:00
Alex Ford
ad3a9b19e4 Ruby: test files for RBI library 2022-04-24 22:48:52 +01:00
Alex Ford
e03ce8f9f2 Ruby: add experimental library to support RBI files 2022-04-24 22:48:52 +01:00
Alex Ford
e3e02c98ea Ruby: Add ExprNodes::CallableCfgNode and ExprNodes::MethodBaseCfgNode 2022-04-24 22:27:20 +01:00
Jeroen Ketema
79164056d1 Replace help.semmle.com links by codeql.github.com links 2022-04-22 20:42:11 +02:00
Jeroen Ketema
a4711206c8 Merge pull request #8769 from felickz/patch-1
Docs - Supported Queries - Fixing broken link
2022-04-22 18:29:27 +02:00
Tony Torralba
d982aeaf6f Add change note 2022-04-22 17:50:47 +02:00
Mathias Vorreiter Pedersen
1e61fdde8b Merge pull request #8815 from jketema/unreachable-test-case
C++: Add non-returning function test case using `__builtin_expect`
2022-04-22 16:47:44 +01:00
Tony Torralba
f1c08bc492 Add value-preserving steps for SharedPreferences 2022-04-22 17:44:59 +02:00