hohn/codeql
1
0
Fork 0
mirror of https://github.com/github/codeql.git synced 2026-02-11 20:51:06 +01:00
Code Issues Packages Projects Releases Wiki Activity
24,066 Commits 1,689 Branches 160 Tags
a73e382dfe0ffcd697185fe334207cebb7c41b9a
Commit Graph

24066 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Taus
a73e382dfe Python: Prevent bad join in hashlib model 
I'm not entirely sure what triggered this bad join order, but some
combination of the use of abstract classes and the exclusion of `new`
caused this to go really wrong:

```
WeakSensitiveDataHashing.ql-15:Stdlib::Stdlib::HashlibDataPassedToHashClass#class#ffff ......... 15.5s
```

with the following tuple counts:
```
[2021-07-12 13:20:15] (16s) Tuple counts for Stdlib::Stdlib::HashlibDataPassedToHashClass#class#ffff/4@217901:
148810  ~3%        {3} r1 = JOIN DataFlowPublic::CallCfgNode#class#ff#shared WITH project#DataFlowPublic::CallCfgNode::getArg_dispred#fff ON FIRST 1 OUTPUT "hashlib", Lhs.1 'node', Lhs.0 'this'
148810  ~4%        {3} r2 = JOIN r1 WITH ApiGraphs::API::Impl::MkModuleImport#ff@staged_ext ON FIRST 1 OUTPUT Rhs.1, Lhs.1 'node', Lhs.2 'this'
7589310 ~486%      {4} r3 = JOIN r2 WITH ApiGraphs::API::Impl::edge#2#fff@staged_ext ON FIRST 1 OUTPUT Lhs.1 'node', Lhs.2 'this', Rhs.1, InverseAppend("getMember(\"","\")",Rhs.1)
6994070 ~490%      {4} r4 = SELECT r3 ON In.3 != "new"
6994070 ~4503%     {2} r5 = SCAN r4 OUTPUT In.1 'this', In.0 'node'

22      ~4%        {3} r6 = JOIN DataFlowPublic::CallCfgNode#class#ff#shared WITH project#DataFlowPublic::CallCfgNode::getArgByName_dispred#fff ON FIRST 1 OUTPUT "hashlib", Lhs.1 'node', Lhs.0 'this'
22      ~0%        {3} r7 = JOIN r6 WITH ApiGraphs::API::Impl::MkModuleImport#ff@staged_ext ON FIRST 1 OUTPUT Rhs.1, Lhs.1 'node', Lhs.2 'this'
1122    ~437%      {4} r8 = JOIN r7 WITH ApiGraphs::API::Impl::edge#2#fff@staged_ext ON FIRST 1 OUTPUT Lhs.1 'node', Lhs.2 'this', Rhs.1, InverseAppend("getMember(\"","\")",Rhs.1)
1034    ~460%      {4} r9 = SELECT r8 ON In.3 != "new"
1034    ~4549%     {2} r10 = SCAN r9 OUTPUT In.1 'this', In.0 'node'

6995104 ~4503%     {2} r11 = r5 UNION r10
5213851 ~4683%     {3} r12 = JOIN r11 WITH ApiGraphs::API::Node::getACall_dispred#ff_10#join_rhs ON FIRST 1 OUTPUT Rhs.1 'hashClass', Lhs.1 'node', Lhs.0 'this'
6478480 ~4646%     {6} r13 = JOIN r12 WITH ApiGraphs::API::Impl::edge#2#fff_201#join_rhs ON FIRST 1 OUTPUT "hashlib", Rhs.1, Lhs.1 'node', Lhs.2 'this', Lhs.0 'hashClass', Rhs.2
1410    ~4693%     {5} r14 = JOIN r13 WITH ApiGraphs::API::Impl::MkModuleImport#ff@staged_ext ON FIRST 2 OUTPUT Lhs.2 'node', Lhs.3 'this', Lhs.4 'hashClass', Lhs.5, InverseAppend("getMember(\"","\")",Lhs.5)
1222    ~4540%     {5} r15 = SELECT r14 ON In.4 'hashName' != "new"
1222    ~4540%     {4} r16 = SCAN r15 OUTPUT In.1 'this', In.4 'hashName', In.2 'hashClass', In.0 'node'
```

By factoring out the insides, the biggest iteration now looks like

```
[2021-07-12 14:17:36] (0s) Tuple counts for Stdlib::Stdlib::HashlibDataPassedToHashClass#class#ffff/4@85bb21:
148810 ~0%     {2} r1 = JOIN DataFlowPublic::CallCfgNode#class#ff#shared WITH project#DataFlowPublic::CallCfgNode::getArg_dispred#fff ON FIRST 1 OUTPUT Lhs.1 'node', Lhs.0 'this'
148810 ~0%     {2} r2 = JOIN r1 WITH Stdlib::Stdlib::hashlibMember#ff#nonempty CARTESIAN PRODUCT OUTPUT Lhs.1 'this', Lhs.0 'node'

22     ~0%     {2} r3 = JOIN DataFlowPublic::CallCfgNode#class#ff#shared WITH project#DataFlowPublic::CallCfgNode::getArgByName_dispred#fff ON FIRST 1 OUTPUT Lhs.1 'node', Lhs.0 'this'
22     ~0%     {2} r4 = JOIN r3 WITH Stdlib::Stdlib::hashlibMember#ff#nonempty CARTESIAN PRODUCT OUTPUT Lhs.1 'this', Lhs.0 'node'

148832 ~0%     {2} r5 = r2 UNION r4
110933 ~2%     {3} r6 = JOIN r5 WITH ApiGraphs::API::Node::getACall_dispred#ff_10#join_rhs ON FIRST 1 OUTPUT Rhs.1 'hashClass', Lhs.1 'node', Lhs.0 'this'
26     ~0%     {4} r7 = JOIN r6 WITH Stdlib::Stdlib::hashlibMember#ff_10#join_rhs ON FIRST 1 OUTPUT Lhs.2 'this', Rhs.1 'hashName', Lhs.0 'hashClass', Lhs.1 'node'
               return r7
```

(The tuple counts themselves are not directly comparable.)
 2021-07-12 14:22:21 +00:00
Mathias Vorreiter Pedersen
c47d680d65 Merge pull request #6168 from criemen/fix-warning 
C++: Fix warning from compile-query.
 2021-07-12 12:41:29 +02:00
Cornelius Riemenschneider
d34f7b941a C++: Address code review. 2021-07-12 11:43:43 +02:00
Cornelius Riemenschneider
e821b8be99 C++: Fix warning from compile-query. 2021-07-12 11:43:43 +02:00
Tom Hvitved
db4c8dfd3c Merge pull request #6208 from hvitved/csharp/query-modules 
C#: Add `Query` suffix to libraries that should only be imported by queries
 2021-07-12 10:26:45 +02:00
Anders Schack-Mulligen
0e913a19aa Merge pull request #6220 from github/workflow/coverage/update 
Update CSV framework coverage reports
 2021-07-12 09:54:18 +02:00
github-actions[bot]
56419bc74b Add changed framework coverage reports 2021-07-12 00:06:55 +00:00
Tom Hvitved
4de4753c67 C#: Remove Query.qll top-level modules 2021-07-04 09:35:27 +02:00
Tom Hvitved
c812d4e4e8 C#: Add Query suffix to libraries that should only be imported by queries 2021-07-04 09:35:26 +02:00
CodeQL CI
1d56748eed Merge pull request #6200 from yoff/pythonJS-make-expbtlib-private 
Approved by RasmusWL, esbena
 2021-07-02 09:09:18 -07:00
CodeQL CI
a25933aa56 Merge pull request #5926 from RasmusWL/small-cleanups 
Approved by tausbn
 2021-07-02 04:59:54 -07:00
Rasmus Wriedt Larsen
3c8c2d1da1 Merge pull request #6209 from yoff/python-add-redos-queryhelp 
Python: port redos .qhelp from js
 2021-07-02 13:42:39 +02:00
Rasmus Wriedt Larsen
81fab487a4 Python: Apply suggestions from code review 
Co-authored-by: Taus <tausbn@github.com>
 2021-07-02 13:27:41 +02:00
Rasmus Wriedt Larsen
22c155687e Python: Fix code after removing getPostUpdateNode 2021-07-02 13:25:25 +02:00
Rasmus Wriedt Larsen
7a6eee50ff Revert "Python: Add getPostUpdateNode to DataFlow::Node" 
This reverts commit 9137f04bd3.
 2021-07-02 13:23:02 +02:00
Rasmus Wriedt Larsen
e56dfe75bd Python: AttrRef getOjbect/1 -> accesses/2 
See this thread for discussion:
https://github.com/github/codeql/pull/5926#discussion_r635384981
 2021-07-02 13:21:12 +02:00
CodeQL CI
38f763dd6a Merge pull request #6192 from asgerf/js/string-literals-as-source-nodes 
Approved by esbena
 2021-07-02 03:47:20 -07:00
Rasmus Lerchedahl Petersen
6f2642607e Python: make the import of RedosUtil public 
This mirrors `SuperlinearBacktracking.qll`
An alternative is to keep it private and import it again
in the query files.
 2021-07-02 12:32:04 +02:00
Chris Smowton
6823855e9c Merge pull request #6203 from smowton/smowton/admin/avoid-config-imports-from-qlls 
Java: Reduce DataFlow Configuration pollution from Random.qll and JexlInjection.qll
 2021-07-02 11:27:27 +01:00
Chris Smowton
ca1bf7791e Merge pull request #6210 from tamasvajk/fix/large-coverage-comment 
Fix markdown link in framework coverage PR comment
 2021-07-02 11:27:17 +01:00
Rasmus Lerchedahl Petersen
77c329fb0f Python/JS: Make much more private 2021-07-02 12:13:52 +02:00
Tamás Vajk
4a5fe75d8c Merge pull request #6207 from github/workflow/coverage/update 
Update CSV framework coverage reports
 2021-07-02 12:00:31 +02:00
Asger Feldthaus
c3b7d85341 JS: Update test output after rebasing 2021-07-02 11:57:45 +02:00
Tamas Vajk
f3f069fed5 Fix markdown link in framework coverage PR comment 2021-07-02 11:56:00 +02:00
Asger Feldthaus
7249d2892a JS: Add comment to VueTemplateSink class 2021-07-02 11:55:56 +02:00
Asger Feldthaus
0105b829c4 JS: Update test output 2021-07-02 11:55:56 +02:00
Asger Feldthaus
6d9b96f6e8 JS: Dont use getALocalSource() when marking Vue template sinks 2021-07-02 11:55:56 +02:00
Asger Feldthaus
472b41f5e1 JS: Update React to handle string literals being SourceNodes 2021-07-02 11:55:56 +02:00
Asger Feldthaus
39c204ac39 JS: Treat string literals as source nodes 2021-07-02 11:55:56 +02:00
Rasmus Lerchedahl Petersen
1fc9638486 Python: port redos .qhelp from js 2021-07-02 11:36:46 +02:00
Chris Smowton
a51154a8ef Deduplicate Jexl configuration 2021-07-02 10:02:28 +01:00
Chris Smowton
d022c57903 Add change note 2021-07-02 10:02:28 +01:00
Chris Smowton
bbd3ecb768 Add docs to RandomQuery.qll 2021-07-02 10:02:28 +01:00
Chris Smowton
e661fc08d3 Split Android XSS sink defintions out of XSS.qll 
This removes one of the routes by which XSS.qll is always in scope, and so its dataflow configuration is too -- however it is still always in scope because JaxWS.qll imports it.
 2021-07-02 10:02:25 +01:00
Chris Smowton
747a8e4157 Split up JexlInjection.qll 
This avoids a DataFlow2::Configuration being in scope for all queries via the import from ExternalFlow.qll
 2021-07-02 10:01:51 +01:00
Chris Smowton
643f7dfb87 Split up Random.qll 
This prevents bringing a dataflow config into scope from utility libraries.
 2021-07-02 10:00:49 +01:00
Anders Schack-Mulligen
80124df78e Merge pull request #5487 from joefarebrother/sql-sinks 
Java: Convert SQL sinks to CSV format
 2021-07-02 10:51:09 +02:00
CodeQL CI
61ee193dc0 Merge pull request #6197 from asgerf/js/recompose 
Approved by esbena
 2021-07-02 00:58:06 -07:00
Esben Sparre Andreasen
0cf9c95981 Merge pull request #6193 from esbena/esbena/mootools-xss 
JS: add Mootools XSS sinks
 2021-07-02 09:24:56 +02:00
Anders Schack-Mulligen
4e1155cfd2 Merge pull request #6202 from smowton/smowton/admin/cleanup-duplicated-experimental-query 
Deduplicate shared body of regular and experimental versions of `java/command-line-injection` query.
 2021-07-02 09:23:50 +02:00
Anders Schack-Mulligen
f9da044e54 Merge pull request #6185 from aschackmull/java/perf-fix-request-forgery 
Java: Fix bad magic.
 2021-07-02 09:07:07 +02:00
github-actions[bot]
55aff21587 Add changed framework coverage reports 2021-07-02 00:09:02 +00:00
Taus
f151338def Merge pull request #6198 from RasmusWL/fix-cleartext-logging 
Python: Some minor fixes to `py/clear-text-logging-sensitive-data`
 2021-07-01 18:28:25 +02:00
Chris Smowton
8b7db8a8cc Merge pull request #5408 from p0wn4j/urlclassloader-webclient-ssrf-sinks 
Java: Add URLClassLoader, WebClient SSRF sinks
 2021-07-01 16:14:22 +01:00
Tamás Vajk
05842dcdb3 Merge pull request #6181 from tamasvajk/feature/test-options-files 
C#: Start using 'options' files in tests
 2021-07-01 17:03:27 +02:00
Joe Farebrother
1e82c607ef Mark failing tests as missing 2021-07-01 15:29:47 +01:00
Tamas Vajk
5e2770339f Add adjusted expected files 2021-07-01 16:09:11 +02:00
Tamas Vajk
03d1a3e0ad Trim test files + remove duplicate newlines 2021-07-01 16:09:11 +02:00
Tamas Vajk
4900ecfabe Manual fixes 2021-07-01 16:09:11 +02:00
Tamas Vajk
c29d11087b C#: Start using 'options' files in tests 2021-07-01 16:08:47 +02:00