Commit Graph

48840 Commits

Author SHA1 Message Date
Tom Hvitved
8b8a662c76 Ruby: Fix bad join in parameterMatch
Before
```
Evaluated relational algebra for predicate DataFlowDispatch#36b84300::parameterMatch#2#ff@281bdfu5 with tuple counts:
        23338949   ~0%    {2} r1 = JOIN DataFlowDispatch#36b84300::Cached::TParameterPosition#f WITH DataFlowDispatch#36b84300::Cached::TArgumentPosition#f CARTESIAN PRODUCT OUTPUT Lhs.0, Rhs.0

           65011   ~0%    {2} r2 = JOIN r1 WITH DataFlowDispatch#36b84300::Cached::TAnyParameterPosition#f ON FIRST 1 OUTPUT Lhs.0, Lhs.1
           65010   ~0%    {2} r3 = r2 AND NOT DataFlowDispatch#36b84300::Cached::TSelfArgumentPosition#f(Lhs.1)

        23338949   ~0%    {2} r4 = JOIN DataFlowDispatch#36b84300::Cached::TParameterPosition#f WITH DataFlowDispatch#36b84300::Cached::TArgumentPosition#f CARTESIAN PRODUCT OUTPUT Rhs.0, Lhs.0

             359   ~3%    {2} r5 = JOIN r4 WITH DataFlowDispatch#36b84300::Cached::TAnyArgumentPosition#f ON FIRST 1 OUTPUT Lhs.1, Lhs.0
             358   ~3%    {2} r6 = r5 AND NOT DataFlowDispatch#36b84300::Cached::TSelfParameterPosition#f(Lhs.0)

           65368   ~0%    {2} r7 = r3 UNION r6

           65011   ~0%    {2} r8 = JOIN r1 WITH DataFlowDispatch#36b84300::Cached::TSelfParameterPosition#f ON FIRST 1 OUTPUT Lhs.1, Lhs.0
               1   ~0%    {2} r9 = JOIN r8 WITH DataFlowDispatch#36b84300::Cached::TSelfArgumentPosition#f ON FIRST 1 OUTPUT Lhs.1, Lhs.0

           65011   ~0%    {2} r10 = JOIN r1 WITH DataFlowDispatch#36b84300::Cached::TBlockParameterPosition#f ON FIRST 1 OUTPUT Lhs.1, Lhs.0
               1   ~0%    {2} r11 = JOIN r10 WITH DataFlowDispatch#36b84300::Cached::TBlockArgumentPosition#f ON FIRST 1 OUTPUT Lhs.1, Lhs.0

           65011   ~3%    {2} r12 = JOIN r1 WITH DataFlowDispatch#36b84300::Cached::THashSplatParameterPosition#f ON FIRST 1 OUTPUT Lhs.1, Lhs.0
               1   ~0%    {2} r13 = JOIN r12 WITH DataFlowDispatch#36b84300::Cached::THashSplatArgumentPosition#f ON FIRST 1 OUTPUT Lhs.1, Lhs.0

               2   ~0%    {2} r14 = r11 UNION r13
               3   ~0%    {2} r15 = r9 UNION r14
           65371   ~0%    {2} r16 = r7 UNION r15

           65011   ~0%    {2} r17 = JOIN r1 WITH DataFlowDispatch#36b84300::Cached::TAnyKeywordParameterPosition#f ON FIRST 1 OUTPUT Lhs.1, Lhs.0
            1645   ~1%    {2} r18 = JOIN r17 WITH DataFlowDispatch#36b84300::Cached::TKeywordArgumentPosition#ff_1#join_rhs ON FIRST 1 OUTPUT Lhs.1, Lhs.0

             359   ~0%    {2} r19 = JOIN r4 WITH DataFlowDispatch#36b84300::Cached::TAnyKeywordArgumentPosition#f ON FIRST 1 OUTPUT Lhs.1, Lhs.0
             320   ~0%    {2} r20 = JOIN r19 WITH DataFlowDispatch#36b84300::Cached::TKeywordParameterPosition#ff_1#join_rhs ON FIRST 1 OUTPUT Lhs.0, Lhs.1

            1965   ~1%    {2} r21 = r18 UNION r20

        20803520   ~1%    {3} r22 = JOIN r1 WITH DataFlowDispatch#36b84300::Cached::TKeywordParameterPosition#ff_10#join_rhs ON FIRST 1 OUTPUT Rhs.1, Lhs.1, Lhs.0
             320   ~0%    {2} r23 = JOIN r22 WITH DataFlowDispatch#36b84300::Cached::TKeywordArgumentPosition#ff ON FIRST 2 OUTPUT Lhs.2, Lhs.1

         2145363   ~0%    {3} r24 = JOIN r1 WITH DataFlowDispatch#36b84300::Cached::TPositionalParameterPosition#ff_10#join_rhs ON FIRST 1 OUTPUT Rhs.1, Lhs.1, Lhs.0
              33   ~0%    {2} r25 = JOIN r24 WITH DataFlowDispatch#36b84300::Cached::TPositionalArgumentPosition#ff ON FIRST 2 OUTPUT Lhs.2, Lhs.1

           65011   ~0%    {3} r26 = JOIN r1 WITH DataFlowDispatch#36b84300::Cached::TPositionalParameterLowerBoundPosition#ff_10#join_rhs ON FIRST 1 OUTPUT Lhs.1, Lhs.0, Rhs.1
           63361   ~0%    {4} r27 = JOIN r26 WITH DataFlowDispatch#36b84300::Cached::TPositionalArgumentPosition#ff_10#join_rhs ON FIRST 1 OUTPUT Lhs.1, Lhs.0, Lhs.2, Rhs.1
           63360   ~0%    {4} r28 = SELECT r27 ON In.3 >= In.2
           63360   ~0%    {2} r29 = SCAN r28 OUTPUT In.0, In.1

           63393   ~0%    {2} r30 = r25 UNION r29
           63713   ~0%    {2} r31 = r23 UNION r30
           65678   ~0%    {2} r32 = r21 UNION r31
          131049   ~0%    {2} r33 = r16 UNION r32
                          return r33
```

After
```
Evaluated relational algebra for predicate DataFlowDispatch#36b84300::parameterMatch#2#ff@698b99ci with tuple counts:
             1   ~0%    {2} r1 = JOIN DataFlowDispatch#36b84300::Cached::TSelfParameterPosition#f WITH DataFlowDispatch#36b84300::Cached::TSelfArgumentPosition#f CARTESIAN PRODUCT OUTPUT Lhs.0, Rhs.0

             1   ~0%    {2} r2 = JOIN DataFlowDispatch#36b84300::Cached::TBlockParameterPosition#f WITH DataFlowDispatch#36b84300::Cached::TBlockArgumentPosition#f CARTESIAN PRODUCT OUTPUT Lhs.0, Rhs.0

             2   ~0%    {2} r3 = r1 UNION r2

             1   ~0%    {2} r4 = JOIN DataFlowDispatch#36b84300::Cached::THashSplatParameterPosition#f WITH DataFlowDispatch#36b84300::Cached::THashSplatArgumentPosition#f CARTESIAN PRODUCT OUTPUT Lhs.0, Rhs.0

         65010   ~0%    {2} r5 = JOIN DataFlowDispatch#36b84300::Cached::TAnyParameterPosition#f WITH DataFlowDispatch#36b84300::argumentPositionIsNotSelf#1#f CARTESIAN PRODUCT OUTPUT Lhs.0, Rhs.0

           358   ~3%    {2} r6 = JOIN DataFlowDispatch#36b84300::Cached::TAnyArgumentPosition#f WITH DataFlowDispatch#36b84300::parameterPositionIsNotSelf#1#f CARTESIAN PRODUCT OUTPUT Rhs.0, Lhs.0

         65368   ~0%    {2} r7 = r5 UNION r6
         65369   ~0%    {2} r8 = r4 UNION r7
         65371   ~0%    {2} r9 = r3 UNION r8

          1645   ~1%    {2} r10 = JOIN DataFlowDispatch#36b84300::Cached::TAnyKeywordParameterPosition#f WITH DataFlowDispatch#36b84300::Cached::TKeywordArgumentPosition#ff CARTESIAN PRODUCT OUTPUT Lhs.0, Rhs.1

           320   ~0%    {2} r11 = JOIN DataFlowDispatch#36b84300::Cached::TAnyKeywordArgumentPosition#f WITH DataFlowDispatch#36b84300::Cached::TKeywordParameterPosition#ff CARTESIAN PRODUCT OUTPUT Rhs.1, Lhs.0

          1965   ~1%    {2} r12 = r10 UNION r11

            33   ~0%    {2} r13 = JOIN DataFlowDispatch#36b84300::Cached::TPositionalParameterPosition#ff WITH DataFlowDispatch#36b84300::Cached::TPositionalArgumentPosition#ff ON FIRST 1 OUTPUT Lhs.1, Rhs.1

           320   ~0%    {2} r14 = JOIN DataFlowDispatch#36b84300::Cached::TKeywordParameterPosition#ff WITH DataFlowDispatch#36b84300::Cached::TKeywordArgumentPosition#ff ON FIRST 1 OUTPUT Lhs.1, Rhs.1

         63361   ~1%    {4} r15 = JOIN DataFlowDispatch#36b84300::Cached::TPositionalParameterLowerBoundPosition#ff WITH DataFlowDispatch#36b84300::Cached::TPositionalArgumentPosition#ff CARTESIAN PRODUCT OUTPUT Lhs.0, Lhs.1, Rhs.0, Rhs.1
         63360   ~1%    {4} r16 = SELECT r15 ON In.2 >= In.0
         63360   ~0%    {2} r17 = SCAN r16 OUTPUT In.1, In.3

         63680   ~0%    {2} r18 = r14 UNION r17
         63713   ~0%    {2} r19 = r13 UNION r18
         65678   ~0%    {2} r20 = r12 UNION r19
        131049   ~0%    {2} r21 = r9 UNION r20
                        return r21
```
2022-09-06 13:02:36 +02:00
Erik Krogh Kristensen
c76b6d1782 Merge pull request #10309 from erik-krogh/leftoverTodo
JS: fix leftover todo in js/insecure-temporary-file
2022-09-06 12:31:29 +02:00
erik-krogh
24f2e3cc07 update alert-messages of the sensitive data queries to match #10314 2022-09-06 12:25:36 +02:00
Taus
810568cf02 Merge pull request #10171 from RasmusWL/variable-accesss
Python: Fixes for variable access
2022-09-06 12:18:37 +02:00
Rasmus Wriedt Larsen
5f6e3dcc2e Python: Revert changes to sensitive data query alert messages
This partly reverts the changes from https://github.com/github/codeql/pull/10252

Although consistency is nice, the new messages didn't sound as natural.

New alert message would read

> Insecure hashing algorithm (md5) depends on sensitive data (password). (...)

I'm not sure what it means that a hashing algorithm depends on data. So
for me, the original text below is much easier to understand.

> Sensitive data (password) is used in a hashing algorithm (md5) that is insecure (...)

Same goes for the other sensitive data queries.
2022-09-06 12:01:24 +02:00
Mathias Vorreiter Pedersen
02c18e714b C++: Mention 'range-based for-loops' in the QLDoc for 'IRDeclarationEntry'. 2022-09-06 10:40:13 +01:00
Mathias Vorreiter Pedersen
10f962f341 C++: Rename 'PseudoDeclarationEntry' to 'IRDeclarationEntry'. 2022-09-06 10:36:38 +01:00
Tamas Vajk
826bbdf834 Kotlin: Fix vararg extraction outside of method call 2022-09-06 11:32:32 +02:00
Tamas Vajk
cb3c53dee7 Kotlin: Add test case for unexpected vararg extraction error 2022-09-06 11:32:24 +02:00
Mathias Vorreiter Pedersen
ed3fff0eba Update cpp/ql/lib/semmle/code/cpp/ir/implementation/raw/internal/TranslatedElement.qll
Co-authored-by: Jeroen Ketema <93738568+jketema@users.noreply.github.com>
2022-09-06 10:32:16 +01:00
Tony Torralba
b94e0d3e69 Merge pull request #10251 from atorralba/atorralba/implicit-pendingintent-sinks
Java: Add new AlarmManager sinks to Use of implicit PendingIntents
2022-09-06 11:31:27 +02:00
Rasmus Wriedt Larsen
7895a5859d Ruby: Autoformat 2022-09-06 11:01:06 +02:00
Rasmus Wriedt Larsen
a9e1e72196 Merge branch 'main' into shared-http-client-request 2022-09-06 10:52:27 +02:00
Rasmus Wriedt Larsen
528ef0eeaa Ruby: Use separate dataflow copy for HTTP client libs
As discussed with @hvitved offline. This helps out to ensrue we don't
needlessly evaluate dataflow for configurations that are not needed
anyway. That is, if other library modeling also used the same dataflow
configuration, which ends up being used in query A, then dataflow for
all the `DataFlowImplForLibraries` configurations would be computeted at
once. When we get to evaluate the query `RequestWithoutValidation.ql`
these results mgith have been forgotten since the predicates are not
cached, and everything will have to be computeted again.

In principle we could be added a dataflow copy for each framework.
However, since we know that the `disablesCertificateValidation`
member-predicates for all the HTTP client libraries will all be used at
the same time, and only for the one query, we only add ONE additional
copy.

Note that the only use of `DataFlowImplForLibraries` before this PR is
using `tainttrackingforlibraries.TaintTrackingImpl` (based on
DataFlowImplForLibraries) for regex computation.
c904ba1d16/ruby/ql/lib/codeql/ruby/Regexp.qll (L153)
Since this is currently transitively imported from Frameworks.qll
(through Core.qll, and core/String.qll), the previous approach didn't
actually violate the assumption about all configurations always being in
scope, but it might have been more by accident, than by purpose.
2022-09-06 10:43:36 +02:00
Rasmus Wriedt Larsen
25d09cd6d6 Ruby: Simplify getKeywordArgumentIncludeHashArgument
As suggested by @hvitved in review
2022-09-06 10:24:22 +02:00
Tom Hvitved
eff3747eb9 Docs: No longer mention required /p:UseSharedCompilation=false 2022-09-06 10:13:29 +02:00
Rasmus Wriedt Larsen
07457b2b5f Python: Apply suggestions from code review
Co-authored-by: Taus <tausbn@github.com>
2022-09-06 10:11:37 +02:00
Rasmus Wriedt Larsen
d708abfc80 Python: Accept more .expected changes 2022-09-06 10:11:37 +02:00
Rasmus Wriedt Larsen
c9cd809ef2 Python: Add change-note 2022-09-06 10:11:37 +02:00
Rasmus Wriedt Larsen
e979dffc08 Python: Fix variable access from extractor-change
These changes are from internal PR.
2022-09-06 10:11:37 +02:00
Rasmus Wriedt Larsen
985e87ccde Python: Add variable scope example with subclass 2022-09-06 10:11:37 +02:00
Rasmus Wriedt Larsen
0e3d520712 Python: Add variables regression test
As illustrated when running the python file, the non qualified reads in
the `use` method all refer to the global variables, whereas `ex =
func(baz)` are to the things defined on the class.

The important part of the .expected changes is that the _global_
variable `bar` is used inside the function, whereas it's the local
variable for `foo` (on class scope) that is used inside the function
(which is wrong).
2022-09-06 10:11:37 +02:00
Rasmus Wriedt Larsen
98db1af898 Python: Also show variable access 2022-09-06 10:11:37 +02:00
Rasmus Wriedt Larsen
fd4f60dd1b Python: Adjust variables tests 2022-09-06 10:11:37 +02:00
Rasmus Wriedt Larsen
ebd97f4496 Python: Add type-tracking regession example 2022-09-06 10:11:36 +02:00
erik-krogh
0776687991 fix leftover todo in js/insecure-temporary-file 2022-09-06 10:05:50 +02:00
Philip Ginsbach
6674e07eaa Merge pull request #10088 from github/parameterisedModules
parameterised modules in the QL language reference
2022-09-06 08:59:31 +01:00
Tom Hvitved
12015928c1 Merge pull request #10295 from hvitved/csharp/code-analysis-shared-compilation
C#: No longer manually disable shared compilation in `codeql-analysis.yml`
2022-09-06 09:45:31 +02:00
Asger F
5ef69628b3 Ruby: remove exists that ql4ql is unhappy about 2022-09-06 09:36:06 +02:00
Michael Nebel
0e52d2c9d3 C#: Add missing bi-directional imports. 2022-09-06 09:34:13 +02:00
Asger F
f07e0592d0 JS: Drive-by fix for accidental recursion 2022-09-06 09:30:02 +02:00
Asger F
c7c3ad29cd Python: sync with Python 2022-09-06 09:30:02 +02:00
Asger F
2cbba65617 JS: Sync with JS
fixup JS
2022-09-06 09:30:02 +02:00
Tom Hvitved
62986a23f3 C#: Add dotnet build integration test 2022-09-06 09:24:54 +02:00
Asger F
afd00161e8 Ruby: introduce getExtraNodeFromType
Using getExtraNodeFromPath with n=0 was a bit of a hack. In principle, the CodeQL libraries might care about the type, even though there are no relevant paths starting at that type.
2022-09-06 09:17:34 +02:00
Asger F
b99e9a58e7 Update ruby/ql/lib/codeql/ruby/frameworks/data/internal/ApiGraphModels.qll
Co-authored-by: Tom Hvitved <hvitved@github.com>
2022-09-06 09:10:24 +02:00
Tom Hvitved
9fd9a04c2f Merge pull request #10277 from hvitved/csharp/dotnet-publish-inject
C#: Also inject `/p:UseSharedCompilation=false` into `dotnet publish`
2022-09-06 09:02:00 +02:00
Tamás Vajk
5f841f71db Merge pull request #10291 from tamasvajk/kotlin-fix-array-set
Kotlin: Fix array `set` operator extraction
2022-09-06 09:01:05 +02:00
Arthur Baars
604af4f7b3 Merge pull request #10302 from github/rc/3.7
Merge 3.7 into main
2022-09-06 08:42:44 +02:00
Mathias Vorreiter Pedersen
885bf427bd C++: Accept test changes. 2022-09-05 23:26:03 +01:00
Mathias Vorreiter Pedersen
f6654e3d69 C++: As a bonus we now no longer need the 'TTranslatedRangeBasedForVariableDeclaration' IPA branch. This previously existed only to account for the missing 'DeclarationEntry's happening in range-based for loops. But these are now also handled by 'PseudoDeclarationEntry's. 2022-09-05 22:18:41 +01:00
Mathias Vorreiter Pedersen
fc85ef76ed C++: Introduce 'PseudoDeclarationEntry's to handle missing 'DeclarationEntry's during IR construction. 2022-09-05 22:17:14 +01:00
Arthur Baars
e8d13d156d Merge pull request #10298 from aibaars/suppress-require
Ruby: exclude 'require' and 'require_relative' definitions from call graph
2022-09-05 20:58:38 +02:00
Rasmus Lerchedahl Petersen
af08c6eb08 Python: remove repeated test file 2022-09-05 20:44:55 +02:00
Philip Ginsbach
cec63e4522 Update docs/codeql/ql-language-reference/modules.rst
Co-authored-by: Felicity Chapman <felicitymay@github.com>
2022-09-05 19:17:11 +01:00
Philip Ginsbach
aa539454b5 Update docs/codeql/ql-language-reference/index.rst
Co-authored-by: Felicity Chapman <felicitymay@github.com>
2022-09-05 19:17:06 +01:00
Mathias Vorreiter Pedersen
2da46ad691 C++: Add two testcases demonstrating malformed IR from missing declaration entries from decl stmts. 2022-09-05 17:49:20 +01:00
Geoffrey White
b6d5b6731a C++: Make QLDoc meet style guide. 2022-09-05 17:17:57 +01:00
Geoffrey White
234da636a4 C++: Change note. 2022-09-05 17:01:25 +01:00
Geoffrey White
008d583da8 C++: Modernize cpp/cleartext-storage-database. 2022-09-05 16:47:14 +01:00