Commit Graph

41418 Commits

Author SHA1 Message Date
Jeroen Ketema
5b0603a5b9 C++: Artificial block insertion for loops and switches has changed in frontend
There are now more artificial blocks containing more than one instruction
(artificial blocks containing a single instruction have the extractor only
emit that instruction and not the block). The second instruction in each case
is the label for breaking out of a loop or switch.
2022-04-25 21:13:52 +02:00
Jeroen Ketema
d43ae9b7ed C++: More artificially inserted blocks carry location information in frontend 2022-04-25 21:13:29 +02:00
Jeroen Ketema
a546e6e01c C++: Update test to run with C++14
The test uses an `auto` return type without a trailing return type, which is
a C++14 feature.
2022-04-25 21:13:24 +02:00
Jeroen Ketema
b9e7045a2f C++: Update vector type test to reflect it uses old-clang features
These features are no longer available and the frontend does stricter checking
on this.
2022-04-25 21:13:04 +02:00
Robert Marsh
d0fc348ad9 C++: autoformat 2022-04-25 14:17:49 -04:00
Mathias Vorreiter Pedersen
2c33a7cdd8 Merge pull request #8862 from github/add-swift-to-labeler
Include Swift in `labeler.yml`
2022-04-25 17:10:38 +01:00
Jean Helie
47fdb79cf8 Merge pull request #8751 from github/jhelie/add-gitkeep-to-model-resources
ML: add .gitkeep to resources dir in which ML models are to be found
2022-04-25 18:08:24 +02:00
Mathias Vorreiter Pedersen
00b1e4b3dd Include Swift in labeler.yml 2022-04-25 16:57:47 +01:00
Mathias Vorreiter Pedersen
8869038b4f Swift: Update schema.yml and regenerate files. 2022-04-25 16:15:37 +01:00
Alex Denisov
906ce34e2f Swift: generate QL files properly 2022-04-25 17:09:29 +02:00
Mathias Vorreiter Pedersen
8e1d079db7 Merge pull request #8858 from github/erik-krogh/enable-ql-for-ql-swift
QL: add swift to QL-for-QL
2022-04-25 16:00:56 +01:00
Alex Denisov
355504a86a Swift: use File from QL library 2022-04-25 16:58:30 +02:00
James Fletcher
5a7043f528 Update analyzing-databases-with-the-codeql-cli.rst 2022-04-25 15:57:18 +01:00
Alex Denisov
6f0ddaa431 Merge branch 'main' into alexdenisov/swift-first-extractor-test 2022-04-25 16:53:22 +02:00
Taus
d4fc096ea8 Python: Use local flow in Pythagorean.ql
The hand-rolled notion of flow was causing some severe performance
issues (on a few databases):

```
Tuple counts for Pythagorean::square#168e234a#f#loop_invariant_prefix/2@c86989kr after 6m35s:
175000     ~5%     {2} r1 = JOIN SSA::SsaVariable::getDefinition#dispred#f0820431#ff_10#join_rhs WITH Flow::ControlFlowNode::getNode#dispred#f0820431#bf ON FIRST 1 OUTPUT Lhs.1, Rhs.1 'arg0'
174500     ~6%     {2} r2 = JOIN r1 WITH SSA::SsaVariable::getVariable#dispred#f0820431#ff ON FIRST 1 OUTPUT Rhs.1, Lhs.1 'arg0'
1467782500 ~5%     {3} r3 = JOIN r2 WITH AstGenerated::Name_::getVariable#dispred#f0820431#ff_10#join_rhs ON FIRST 1 OUTPUT 3, Rhs.1 'arg1', Lhs.1 'arg0'
1467553000 ~0%     {2} r4 = JOIN r3 WITH py_expr_contexts_12#join_rhs ON FIRST 2 OUTPUT Lhs.2 'arg0', Lhs.1 'arg1'
                    return r4
```

Rewriting it to use the data flow library made all of this go away. 🎉
2022-04-25 14:35:37 +00:00
Taus
b2cc91369a Python: Fix bad join in firstUse
This was what it looked like (at the point when I killed the evaluation):

```
Tuple counts for SsaCompute::SsaComputeImpl::AdjacentUsesImpl::firstUse#c5fa2be7#ff/2@i1#be98bwif after 1m50s:
274000     ~7%     {4} r1 = SCAN SsaCompute::SsaComputeImpl::AdjacentUsesImpl::definesAt#c5fa2be7#ffff OUTPUT In.1, In.0 'def', In.2, In.3
2731768000 ~1%     {7} r2 = JOIN r1 WITH SsaCompute::SsaComputeImpl::AdjacentUsesImpl::variableSourceUse#c5fa2be7#ffff ON FIRST 1 OUTPUT Rhs.0, Lhs.2, Lhs.3, Rhs.2, Rhs.3, Rhs.1 'use', Lhs.1 'def'
178000     ~4%     {2} r3 = JOIN r2 WITH SsaCompute::SsaComputeImpl::AdjacentUsesImpl::adjacentVarRefs#c5fa2be7#fffff ON FIRST 5 OUTPUT Lhs.6 'def', Lhs.5 'use'
                    return r3
```

And this is what it looks like now:

```
Tuple counts for SsaCompute::SsaComputeImpl::AdjacentUsesImpl::firstUse#c5fa2be7#ff/2@i1#f9d6ewsi after 207ms:
931353  ~2%     {4} r1 = SCAN SsaCompute::SsaComputeImpl::AdjacentUsesImpl::variableSourceUse#c5fa2be7#ffff OUTPUT In.0, In.2, In.3, In.1 'use'
1050477 ~0%     {4} r2 = JOIN r1 WITH SsaCompute::SsaComputeImpl::AdjacentUsesImpl::adjacentVarRefs#c5fa2be7#fffff_03412#join_rhs ON FIRST 3 OUTPUT Lhs.0, Rhs.3, Rhs.4, Lhs.3 'use'
506626  ~0%     {2} r3 = JOIN r2 WITH SsaCompute::SsaComputeImpl::AdjacentUsesImpl::definesAt#c5fa2be7#ffff_1230#join_rhs ON FIRST 3 OUTPUT Rhs.3 'def', Lhs.3 'use'
                return r3
```
2022-04-25 14:33:31 +00:00
Erik Krogh Kristensen
f5e1aa7c98 QL: add swift to QL-for-QL 2022-04-25 16:29:44 +02:00
Taus
49233268a9 Python: Fix bad join in getValue
We were building essentially a CP of all control flow nodes:

```
Tuple counts for Essa::AssignmentDefinition::getValue#dispred#f0820431#ff/2@dd1f67vl after 2m45s:
733365     ~6%     {3} r1 = JOIN Essa::TEssaNodeDefinition#24e22a14#ffff_30#join_rhs WITH Essa::EssaNodeDefinition::getDefiningNode#dispred#f0820431#ff ON FIRST 1 OUTPUT Lhs.1, Rhs.1, Rhs.0
376588     ~0%     {2} r2 = JOIN r1 WITH SsaDefinitions::SsaSource::assignment_definition#9197156e#fff ON FIRST 2 OUTPUT Lhs.2 'this', Rhs.2 'result'
376588     ~0%     {3} r3 = JOIN r2 WITH Essa::TEssaNodeDefinition#24e22a14#ffff_30#join_rhs ON FIRST 1 OUTPUT Rhs.1, Lhs.0 'this', Lhs.1 'result'
6965593033 ~2%     {3} r4 = JOIN r3 WITH project#SsaDefinitions::SsaSource::assignment_definition#9197156e ON FIRST 1 OUTPUT Lhs.1 'this', Rhs.1, Lhs.2 'result'
376588     ~0%     {2} r5 = JOIN r4 WITH Essa::EssaNodeDefinition::getDefiningNode#dispred#f0820431#ff ON FIRST 2 OUTPUT Lhs.0 'this', Lhs.2 'result'
                    return r5
```

We first tried preventing the join on `result`, but this caused the
characteristic predicate to blow up instead. Finally, we figured just
putting the `value` part in a field would be sufficient, and this did
the trick.
2022-04-25 14:28:00 +00:00
Tony Torralba
85d5b122f7 Merge pull request #8817 from atorralba/atorralba/cleartext-storage-sharedprefs-improvs
Java: Add value-preserving flow steps for Android's SharedPreferences
2022-04-25 16:16:46 +02:00
Erik Krogh Kristensen
0a26e891a2 include startsWith/endsWith checks in js/missing-origin-check 2022-04-25 15:28:50 +02:00
Erik Krogh Kristensen
17005dde2d QL: fix query-id, and add description 2022-04-25 15:21:35 +02:00
Mathias Vorreiter Pedersen
3199a690aa Merge pull request #8854 from redsun82/swift-ql-gen
Swift: QL generation script
2022-04-25 14:05:25 +01:00
Paolo Tranquilli
643471f400 add temporary exception for Swift for QLdoc 2022-04-25 14:24:22 +02:00
Erik Krogh Kristensen
fe3d71ebc2 fix qhelp: the window, not the origin, is sending the message
Co-authored-by: Esben Sparre Andreasen <esbena@github.com>
2022-04-25 14:07:01 +02:00
Paolo Tranquilli
cdb10a2151 Swift: fix codegen check 2022-04-25 14:03:48 +02:00
Paolo Tranquilli
9bf4c72085 Swift: split codegen action and fix dependencies 2022-04-25 14:00:41 +02:00
Anders Schack-Mulligen
60eb341b49 Merge pull request #8851 from aschackmull/shared/accesspathsyntax-cleanup
Minor clean-up in AccessPathSyntax.qll
2022-04-25 13:47:22 +02:00
Mathias Vorreiter Pedersen
faaa2cedcd Merge pull request #8849 from JohnMcSandwich/main
C++: add new Windows pool allocation functions in `Allocation.qll`
2022-04-25 12:46:31 +01:00
Paolo Tranquilli
de0fa9e456 Swift: QL generation script
Also added code generation to the swift checks.
2022-04-25 13:23:36 +02:00
Anders Schack-Mulligen
c06efa1f42 Dataflow: Sync. 2022-04-25 13:11:04 +02:00
Anders Schack-Mulligen
f4d93f089a Dataflow: Fix join-on-config producing a CP. 2022-04-25 13:10:31 +02:00
Tony Torralba
f1e5e57d76 Update java/ql/lib/semmle/code/java/frameworks/android/SharedPreferences.qll 2022-04-25 12:39:01 +02:00
Anders Schack-Mulligen
40a16325a9 Minor clean-up in AccessPathSyntax. 2022-04-25 12:27:48 +02:00
Alex Denisov
5b20d580be Swift: use parenthesis instead of curly braces 2022-04-25 12:26:34 +02:00
Alex Ford
b956616a56 Ruby: fix alert 2022-04-25 11:25:57 +01:00
Anders Schack-Mulligen
cbdd4927ce Merge pull request #8582 from Marcono1234/marcono1234/JumpStmt-superclass
Java: Make `JumpStmt` a proper superclass
2022-04-25 12:22:20 +02:00
Anders Schack-Mulligen
fd2904d49c Merge pull request #8760 from Marcono1234/patch-1
Clarify `min`, `max` and `rank` documentation
2022-04-25 12:20:00 +02:00
Tom Hvitved
bffa8fa7cb Merge pull request #8641 from hvitved/dataflow/interpret-read-store
Data flow: Introduce `ContentSet`
2022-04-25 12:17:34 +02:00
Mathias Vorreiter Pedersen
516ef8d27a Update cpp/ql/lib/change-notes/2022-04-25-windows-pool-allocation-functions.md
Co-authored-by: Jeroen Ketema <93738568+jketema@users.noreply.github.com>
2022-04-25 10:54:12 +01:00
Mathias Vorreiter Pedersen
12c8d9c60e C++: Add change note. 2022-04-25 10:51:49 +01:00
Alex Denisov
8bcdfb2e4f Swift: initialize LLVM
No need to shutdown LLVM, it's done by the PROGRAM_START macro
2022-04-25 11:49:21 +02:00
Alex Denisov
462133e0f0 Swift: add more comments 2022-04-25 11:48:51 +02:00
Erik Krogh Kristensen
b5193d99d7 have getSourceType() depend on which kind of event it is 2022-04-25 11:32:52 +02:00
Anders Schack-Mulligen
b21f077e8e Update java/ql/lib/semmle/code/java/Statement.qll 2022-04-25 11:02:23 +02:00
Jeroen Ketema
ba2a884a45 Merge pull request #8818 from jketema/links
Replace `help.semmle.com` links by `codeql.github.com` links
2022-04-25 10:25:42 +02:00
JohnMcSandwich
b3dff77d1a C++: add new Windows pool allocation functions in Allocation.qll
Add:
 - ExAllocatePool2
 - ExAllocatePool3
 - ExAllocatePoolZero
2022-04-25 10:21:42 +02:00
Mathias Vorreiter Pedersen
e8b6bfbe0e Merge pull request #8813 from jketema/buffer
C++: Cover variable sized member arrays without a size in `Buffer.qll`
2022-04-25 09:20:31 +01:00
Tom Hvitved
2466288656 Data flow: Simplify revFlowStore 2022-04-25 10:11:54 +02:00
Tom Hvitved
cf0a1e748a Add change notes 2022-04-25 09:17:40 +02:00
Alex Ford
869d827768 Revert "Ruby: extract rbi files"
This reverts commit ba9342e0f38f4468d04bad987a8f69e95fdbdd5f.
2022-04-24 22:48:52 +01:00