Commit Graph

32095 Commits

Author SHA1 Message Date
Rasmus Wriedt Larsen
a4bb0cc5d8 Python: Run tests for fieldflow/test.py 2022-02-01 15:32:07 +01:00
Rasmus Wriedt Larsen
1390f034f3 Python: Delete duplicated tests
All the same tests are present in `fieldflow/test.py`
2022-02-01 15:31:30 +01:00
Rasmus Wriedt Larsen
1394b38032 Python: Improve customSanitizer tests
Before we didn't show how we treated the value _after_ the check. But we
do actually handle this nicely 💪
2022-02-01 15:09:29 +01:00
yoff
8df04c58e9 Merge pull request #7793 from tausbn/python-fix-bad-TPythonTuple-join-order
Python: Fix bad join order in `TPythonTuple`
2022-01-31 22:39:58 +01:00
Harry Maclean
e5b7478028 Merge pull request #7780 from github/hmac/split-tests
Ruby: Split up CI jobs
2022-02-01 09:10:01 +13:00
Taus
4a29095e3b Python: Fix bad join order in TPythonTuple
TL;DR: Something introduced the following bad join order:
```
(227s) Tuple counts for dom#TObject::TPythonTuple#ff/2@i2#8f58670w after 3m46s:
25000      ~0%     {2} r1 = SCAN PointsToContext::PointsToContext::appliesToScope_dispred#ff#prev_delta OUTPUT In.1, In.0 'context'
24000      ~1%     {2} r2 = JOIN r1 WITH @py_scope#f ON FIRST 1 OUTPUT Lhs.1 'context', Lhs.0
1076876712 ~6%     {3} r3 = JOIN r2 WITH Flow::TupleNode#class#f CARTESIAN PRODUCT OUTPUT Rhs.0, Lhs.0 'context', Lhs.1
870129666  ~0%     {3} r4 = JOIN r3 WITH Flow::ControlFlowNode::isLoad_dispred#f ON FIRST 1 OUTPUT Lhs.1 'context', Lhs.2, Lhs.0 'origin'
870129000  ~0%     {3} r5 = r4 AND NOT dom#TObject::TPythonTuple#ff#prev(Lhs.2 'origin', Lhs.0 'context')
870129000  ~1%     {3} r6 = SCAN r5 OUTPUT In.2 'origin', In.1, In.0 'context'
9000       ~0%     {2} r7 = JOIN r6 WITH Flow::ControlFlowNode::getScope_dispred#ff ON FIRST 2 OUTPUT Lhs.0 'origin', Lhs.2 'context'
                    return r7
```
(...the above being the tuple counts _at the point when I cancelled the
query_!)

Rewriting the code to force a join between `TupleNode#class` and
`getScope` results in the following join orders:

```
(0s) Tuple counts for TObject::scope_loads_tuplenode#ff/2@b3cf0bo5 after 13ms:
37369 ~3%     {1} r1 = JOIN Flow::TupleNode#class#f WITH Flow::ControlFlowNode::isLoad_dispred#f ON FIRST 1 OUTPUT Lhs.0 'origin'
37369 ~3%     {2} r2 = JOIN r1 WITH Flow::ControlFlowNode::getScope_dispred#ff ON FIRST 1 OUTPUT Rhs.1 's', Lhs.0 'origin'
            return r2
```
and
```
(78s) Tuple counts for dom#TObject::TPythonTuple#ff/2@i53#121c440w after 6ms:
34736 ~3%     {2} r1 = SCAN PointsToContext::PointsToContext::appliesToScope_dispred#ff#prev_delta OUTPUT In.1, In.0 'context'
7370  ~5%     {2} r2 = JOIN r1 WITH TObject::scope_loads_tuplenode#ff ON FIRST 1 OUTPUT Lhs.1 'context', Rhs.1 'origin'
7370  ~5%     {2} r3 = r2 AND NOT dom#TObject::TPythonTuple#ff#prev(Lhs.1 'origin', Lhs.0 'context')
7370  ~1%     {2} r4 = SCAN r3 OUTPUT In.1 'origin', In.0 'context'
            return r4
```
the latter being the largest iteration of `dom#TPythonTuple` throughout
the log.

No other major performance issues were observed.
2022-01-31 16:59:50 +00:00
Tom Hvitved
5503abc73d Merge pull request #7772 from hvitved/csharp/event-accessor-event-null
C#: Guard against `AssociatedSymbol` not being an `IEventSymbol`
2022-01-31 14:52:02 +01:00
Michael Nebel
56ac99039f Merge pull request #7720 from michaelnebel/csharp/extended-prop-patterns
C#: Desugar property patterns that uses member access syntax.
2022-01-31 13:24:24 +01:00
Tom Hvitved
2354281721 C#: Add DB down/upgrade scripts 2022-01-31 11:46:10 +01:00
Tom Hvitved
32e58add7b C#: Extend compiler_generated to include event accessors 2022-01-31 11:45:23 +01:00
Michael Nebel
7cbeffc8a7 C#: Refactor and use new language features. 2022-01-31 09:24:31 +01:00
Mathias Vorreiter Pedersen
bb2feda8fb Merge pull request #7703 from geoffw0/getslocal 2022-01-28 19:35:15 +00:00
Mathias Vorreiter Pedersen
0f239e315c Merge pull request #7782 from geoffw0/clrtxt7
C++: Fix FPs for cpp/cleartext-storage-file
2022-01-28 17:24:05 +00:00
Geoffrey White
0396a84c3c C++: Remove empty predicate / extends. 2022-01-28 17:11:38 +00:00
Geoffrey White
af09dd8af1 C++: Fixes to gets models. 2022-01-28 16:04:23 +00:00
Geoffrey White
036e1495b8 Merge branch 'main' into getslocal 2022-01-28 15:58:13 +00:00
Geoffrey White
a695f02af4 C++: Add change note. 2022-01-28 12:38:27 +00:00
Tom Hvitved
82cceb0a29 C#: Mark event accessors without bodies as compiler generated 2022-01-28 13:11:34 +01:00
Geoffrey White
b73dc98191 C++: Exclude write to stdout etc. 2022-01-28 11:57:31 +00:00
Tom Hvitved
864b61a804 Merge pull request #7766 from hvitved/csharp/extractor/type-param-constraints
C#: Make `TypeParameterConstraints` a `CachedEntity`
2022-01-28 12:39:31 +01:00
Tom Hvitved
28702dff82 Merge pull request #7779 from hvitved/csharp/initial-downgrade-scheme
C#: Add initial downgrade DB scheme for use in tests
2022-01-28 12:38:07 +01:00
Nick Rolfe
588e60e230 Merge pull request #7775 from github/nickrolfe/graph_test_ordering
Ruby/C#: more stable graph test ordering
2022-01-28 11:16:02 +00:00
Erik Krogh Kristensen
7aa59ca233 Merge pull request #7633 from erik-krogh/CWE-300
JS: add js/http-dependency query
2022-01-28 12:10:14 +01:00
Taus
47a57e0c0a Merge pull request #7635 from github/python/support-match
Python/support match
2022-01-28 11:55:46 +01:00
yoff
74d57bbb1a Update python/ql/lib/semmle/python/dataflow/new/internal/DataFlowPrivate.qll
Co-authored-by: Taus <tausbn@github.com>
2022-01-28 11:38:29 +01:00
Erik Krogh Kristensen
b5198bdaca apply suggestions from doc review
Co-authored-by: mc <42146119+mchammer01@users.noreply.github.com>
2022-01-28 10:46:27 +01:00
Tony Torralba
f3e034b2be Merge pull request #7764 from github/workflow/coverage/update
Update CSV framework coverage reports
2022-01-28 09:35:54 +01:00
Harry Maclean
0428b8ee20 Split Ruby CI into multiple parallel jobs
Run format, compile and db upgrade checks in parallel, along with the
main tests, which run in two parallel halves.
2022-01-28 21:23:34 +13:00
Esben Sparre Andreasen
ee52774e90 Merge pull request #7760 from erik-krogh/CWE-184
JS: add CWE-184 to incomplete-scheme-check and bad-tag-filter
2022-01-28 09:18:41 +01:00
Tom Hvitved
ee5495ce65 C#: Add initial downgrade DB scheme for use in tests 2022-01-28 09:05:42 +01:00
github-actions[bot]
c6130ea2d4 Add changed framework coverage reports 2022-01-28 00:11:49 +00:00
Dave Bartolomeo
cca74e925f Merge pull request #7724 from github/aeisenberg/examples-groups
Add new groups for examples packs
2022-01-27 12:11:26 -05:00
Rasmus Lerchedahl Petersen
c60df7d69c Merge branch 'main' of github.com:github/codeql into python/support-match 2022-01-27 16:45:17 +01:00
yoff
4632c14280 Merge pull request #7654 from RasmusWL/remove-old-pointsto-queries
Python: Cleanup: Remove old points-to versions of queries
2022-01-27 16:39:01 +01:00
Nick Rolfe
cd5010fe11 C#: sync changes from Ruby to improve ordering of graph test output 2022-01-27 15:34:01 +00:00
Tom Hvitved
b7fb9e8b95 Merge pull request #7768 from hvitved/csharp/extractor-diagnostics-query
C#: Add internal extractor diagnostics query
2022-01-27 16:33:32 +01:00
Chris Smowton
17656fc12b Merge pull request #7771 from Dig2/main
Fix typo in CodeQL-query-help-for-JavaScript
2022-01-27 15:03:35 +00:00
Mathias Vorreiter Pedersen
b3f4357dc8 Merge pull request #7742 from geoffw0/clrtxt6
C++: Upgrade cpp/cleartext-storage-buffer
2022-01-27 14:40:40 +00:00
Rasmus Lerchedahl Petersen
b93c04bb79 python: Add reverse flow in some patterns
Particularly in value and literal patterns.
This is getting a little bit into the guards aspect of matching.
We could similarly add reverse flow in terms of
sub-patterns storing to a sequence pattern,
a flow step from alternatives to an-or-pattern, etc..
It does not seem too likely that sources are embedded in patterns
to begin with, but for secrets perhaps?

It is illustrated by the literal test. The value test still fails.
I believe we miss flow in general from the static attribute.
2022-01-27 15:20:23 +01:00
Tom Hvitved
cdfe239016 C#: Guard against AssociatedSymbol not being an IEventSymbol
Apply same logic as for property/indexer accessors to account for cases where
the associated event cannot be determined. I have not been able to reproduce
such cases locally, though we have seen reports of it happening.
2022-01-27 15:14:03 +01:00
Nick Rolfe
6f06263d49 Ruby: add more properties for ordering nodes in graph tests 2022-01-27 13:57:43 +00:00
Dig2
516bed391a Fix CodeQL-query-help-for-JavaScript typo 2022-01-27 21:33:20 +08:00
Geoffrey White
2e1b09fd75 C++: Modernize flow sources. 2022-01-27 13:19:09 +00:00
Geoffrey White
47528dd8c0 C++: Autoformat. 2022-01-27 12:56:16 +00:00
Tom Hvitved
d9a1046e0e Merge pull request #7683 from hvitved/ruby/qltest-4-threads
Ruby: Use multiple threads in QL test CI job
2022-01-27 13:11:39 +01:00
Geoffrey White
1bf9c19638 C++: Autoformat. 2022-01-27 11:26:18 +00:00
Geoffrey White
f090a3b440 C++: Add to and clarify some taint library QLDoc. 2022-01-27 11:26:00 +00:00
Tom Hvitved
1e39259e26 Merge pull request #7750 from hvitved/ruby/desugar-hash-literals
Ruby: Desugar hash literals
2022-01-27 12:01:06 +01:00
Geoffrey White
d9a2347178 C++: Switch back to IR taint. 2022-01-27 10:50:22 +00:00
Tom Hvitved
e2ae327a74 C#: Add internal extractor diagnostics query 2022-01-27 11:19:31 +01:00