Commit Graph

17246 Commits

Author SHA1 Message Date
yoff
ea4ea6b3e6 Merge pull request #4529 from tausbn/python-remove-cartesian-product-in-tkwoverflownode
Python: Remove bad join in `getCallableScope`
2020-10-21 15:36:38 +02:00
Jonas Jensen
2479402a9d Merge pull request #4522 from geoffw0/odasa704
C++: Non-constant format string test case
2020-10-21 14:58:02 +02:00
Jonas Jensen
6a80f3380e Merge pull request #4527 from geoffw0/odasa3940
C++: Improve SizeCheck queries
2020-10-21 14:55:57 +02:00
Taus Brock-Nannestad
31c169daac Python: Remove bad join in getCallableScope
Clause timing report had this suspicious entry

```
CommandInjection.ql-12:DataFlowPublic::Node::getCallableScope#bbf .................. 7.2s
    (4 evaluations with max 6.4s in DataFlowPublic::Node::getCallableScope#bbf/3@i3#119d7b)
```

which indeed was a bad join:
```
Tuple counts for DataFlowPublic::Node::getCallableScope#bbf:
293509   ~2%     {3} r1 = JOIN DataFlowPublic::Node::getCallableScope#bbf#prev_delta AS L WITH DataFlowPublic::TNode#f AS R ON FIRST 1 OUTPUT L.<1>, L.<0>, L.<2>
22337162 ~0%     {3} r2 = JOIN r1 WITH Scope::Scope::getEnclosingScope_dispred#ff_10#join_rhs AS R ON FIRST 1 OUTPUT r1.<1>, r1.<2>, R.<1>
22337162 ~0%     {3} r3 = r2 AND NOT DataFlowPublic::Node::getCallableScope#bbf#prev AS R(r2.<0>, r2.<2>, r2.<1>)
22337162 ~0%     {3} r4 = SCAN r3 OUTPUT r3.<0>, r3.<2>, r3.<1>
722      ~1%     {3} r5 = JOIN r4 WITH m#DataFlowPublic::Node::getCallableScope#bbf AS R ON FIRST 2 OUTPUT r4.<0>, r4.<1>, r4.<2>
722      ~1%     {3} r6 = JOIN r5 WITH m#DataFlowPublic::Node::getCallableScope#bbf AS R ON FIRST 2 OUTPUT r5.<0>, r5.<2>, r5.<1>
722      ~1%     {3} r7 = r6 AND NOT project#DataFlowPrivate::DataFlowCallable::getScope_dispred#ff AS R(r6.<2>)
722      ~1%     {3} r8 = SCAN r7 OUTPUT r7.<0>, r7.<2>, r7.<1>
                 return r8
```

In this case, the join went away by simply moving the helper predicate
out of the class it was situated in (and since it doesn't mention
`this`, it didn't really belong there in the first place).

Result:
```
DataFlowPublic.qll-8:DataFlowPublic::getCallableScope#ff ........................... 26ms
    (4 evaluations with max 15ms in DataFlowPublic::getCallableScope#ff/2@i3#709a9e)
```
2020-10-21 14:45:05 +02:00
yoff
9bd808c924 Merge pull request #4528 from tausbn/python-remove-cartesian-product-in-tkwoverflownode
Python: Prevent early join on `argName` in `getArg`
2020-10-21 14:43:39 +02:00
yoff
75357727c4 Merge pull request #4490 from RasmusWL/python-model-django-sources
Python: model Django HttpRequest as RemoteFlowSource
2020-10-21 13:46:51 +02:00
Taus Brock-Nannestad
eb3333c0ce Python: Prevent early join on argName in getArg 2020-10-21 13:23:09 +02:00
Geoffrey White
ba29591b83 C++: Change note. 2020-10-21 11:21:53 +01:00
CodeQL CI
da58306f2d Merge pull request #4506 from asgerf/js/separate-jquery-config
Approved by esbena
2020-10-21 03:13:42 -07:00
Rasmus Wriedt Larsen
5874a7b422 Merge pull request #4488 from yoff/SharedDataflow_ArgumentPassingTests
Python: Shared dataflow, argument passing tests
2020-10-21 12:01:37 +02:00
Geoffrey White
6d91d55669 C++: Use the AllocationExpr model. 2020-10-21 11:00:15 +01:00
CodeQL CI
9faf675f1f Merge pull request #4486 from erik-krogh/lessTokens
Approved by asgerf
2020-10-21 02:56:38 -07:00
Mathias Vorreiter Pedersen
3fe2a17933 Merge pull request #4509 from geoffw0/intmultlong
C++: Add IntMultToLong test cases.
2020-10-21 11:03:13 +02:00
CodeQL CI
eaed93fa7d Merge pull request #4513 from RasmusWL/python-model-fabric
Approved by yoff
2020-10-21 01:58:19 -07:00
Rasmus Lerchedahl Petersen
9ee5a01d7e Python: Reword comment on isBarrierIn 2020-10-21 10:30:40 +02:00
Rasmus Wriedt Larsen
ba9caacee1 Merge pull request #4524 from tausbn/python-remove-cartesian-product-in-tkwoverflownode
Python: Remove cartesian product in `TKwOverflowNode`
2020-10-21 10:22:45 +02:00
Rasmus Wriedt Larsen
b0af0b94d0 Python: Fix grammar
Co-authored-by: yoff <lerchedahl@gmail.com>
2020-10-21 09:58:37 +02:00
CodeQL CI
897d8de65a Merge pull request #4523 from erik-krogh/optionalPromise
Approved by asgerf
2020-10-21 00:34:12 -07:00
Erik Krogh Kristensen
996488511c Merge pull request #4520 from erik-krogh/LessTests
JS: Aggregate tests cases
2020-10-20 19:03:47 +02:00
Geoffrey White
17ff34267a C++: Resolve the overlap. 2020-10-20 17:47:49 +01:00
Geoffrey White
7db2b2ce37 C++: Make the two queries more alike. 2020-10-20 17:46:47 +01:00
Geoffrey White
4630c69950 C++: Add a test case resembling the example from ODASA-3940. 2020-10-20 17:39:32 +01:00
Geoffrey White
857a4d8a3f C++: Merge the two tests, so that overlaps are obvious. 2020-10-20 17:33:15 +01:00
Rasmus Wriedt Larsen
6053157b09 Python: Fix QLDoc to not mention sudo method on fabric Group 2020-10-20 17:30:09 +02:00
Taus Brock-Nannestad
0e2ec0dfb4 Python: Remove cartesian product in TKwOverflowNode
With CP:

(0s) Tuple counts for dom#DataFlowPublic::TKwOverflowNode#ff:
1209    ~0%       {2} r1 = JOIN project#AstGenerated::Function_::getKwarg_dispred#ff AS L WITH ObjectAPI::CallableValue::getScope_dispred#ff_10#join_rhs AS R ON FIRST 1 OUTPUT R.<1>, L.<0>
4329    ~0%       {3} r2 = JOIN r1 WITH DataFlowPrivate::ArgumentPassing::connects#bb_10#join_rhs AS R ON FIRST 1 OUTPUT R.<1>, r1.<1>, r1.<0>
7819    ~2%       {4} r3 = JOIN r2 WITH Flow::CallNode::getArgByName_dispred#fff AS R ON FIRST 1 OUTPUT r2.<1>, r2.<2>, r2.<0>, R.<1>
7114    ~1%       {4} r4 = r3 AND NOT Function::Function::getArgByName_dispred#fff_01#antijoin_rhs AS R(r3.<0>, r3.<3>)
7114    ~76%      {2} r5 = SCAN r4 OUTPUT r4.<2>, r4.<1>
1123    ~0%       {1} r6 = JOIN project#Exprs::Call::getKwargs_dispred#ff AS L WITH py_flow_bb_node_10#join_rhs AS R ON FIRST 1 OUTPUT R.<1>
1123    ~0%       {1} r7 = JOIN r6 WITH Flow::CallNode#class#f AS R ON FIRST 1 OUTPUT r6.<0>
1357707 ~0%       {2} r8 = JOIN r7 WITH project#AstGenerated::Function_::getKwarg_dispred#ff AS R CARTESIAN PRODUCT OUTPUT R.<0>, r7.<0>
1357707 ~0%       {2} r9 = JOIN r8 WITH ObjectAPI::CallableValue::getScope_dispred#ff_10#join_rhs AS R ON FIRST 1 OUTPUT r8.<1>, R.<1>
1364821 ~0%       {2} r10 = r5 \/ r9
                  return r10

Without CP:

(13s) Tuple counts for dom#DataFlowPublic::TKwOverflowNode#ff:
1209    ~0%       {2} r1 = JOIN project#AstGenerated::Function_::getKwarg_dispred#ff AS L WITH ObjectAPI::CallableValue::getScope_dispred#ff_10#join_rhs AS R ON FIRST 1 OUTPUT R.<1>, L.<0>
19175   ~4%       {3} r2 = JOIN r1 WITH DataFlowPrivate::ArgumentPassing::connects#ff_10#join_rhs AS R ON FIRST 1 OUTPUT R.<1>, r1.<1>, r1.<0>
7819    ~2%       {4} r3 = JOIN r2 WITH Flow::CallNode::getArgByName_dispred#fff AS R ON FIRST 1 OUTPUT r2.<1>, r2.<2>, r2.<0>, R.<1>
7114    ~1%       {4} r4 = r3 AND NOT Function::Function::getArgByName_dispred#fff_01#antijoin_rhs AS R(r3.<0>, r3.<3>)
7114    ~76%      {2} r5 = SCAN r4 OUTPUT r4.<2>, r4.<1>
1123    ~0%       {1} r6 = JOIN project#Exprs::Call::getKwargs_dispred#ff AS L WITH py_flow_bb_node_10#join_rhs AS R ON FIRST 1 OUTPUT R.<1>
574     ~0%       {2} r7 = JOIN r6 WITH DataFlowPrivate::ArgumentPassing::connects#ff AS R ON FIRST 1 OUTPUT R.<1>, r6.<0>
524     ~1%       {3} r8 = JOIN r7 WITH ObjectAPI::CallableValue::getScope_dispred#ff AS R ON FIRST 1 OUTPUT R.<1>, r7.<1>, r7.<0>
291     ~0%       {2} r9 = JOIN r8 WITH project#AstGenerated::Function_::getKwarg_dispred#ff AS R ON FIRST 1 OUTPUT r8.<1>, r8.<2>
7405    ~72%      {2} r10 = r5 \/ r9
                  return r10
2020-10-20 17:29:26 +02:00
Rasmus Wriedt Larsen
cd116fc5f2 Python: Fix grammar in QLDoc
Co-authored-by: yoff <lerchedahl@gmail.com>
2020-10-20 17:26:33 +02:00
Geoffrey White
a372578571 C++: Move the SizeCheck*.ql tests to the standard location. 2020-10-20 16:02:54 +01:00
Erik Krogh Kristensen
bdbc8f5c91 add support for OptionalUse in js/missing-await 2020-10-20 16:52:57 +02:00
Geoffrey White
8198b96eb2 C++: Add test cases. 2020-10-20 15:33:57 +01:00
Geoffrey White
540af7866c C++: Test spacing. 2020-10-20 15:23:15 +01:00
CodeQL CI
7ea8652f49 Merge pull request #4521 from erik-krogh/moreMiddle
Approved by asgerf
2020-10-20 07:14:14 -07:00
Geoffrey White
cbf55ce51a Merge branch 'main' into intmultlong 2020-10-20 14:55:53 +01:00
Asger Feldthaus
5436bb154a JS: Mention Xss.ql in change note 2020-10-20 14:49:20 +01:00
Rasmus Wriedt Larsen
6920f3012c Python: Django route handlers in different file now works
Fixed by https://github.com/github/codeql/pull/4514
2020-10-20 15:41:14 +02:00
Rasmus Wriedt Larsen
c8441dc4fb Merge branch 'main' into python-model-django-sources 2020-10-20 15:38:20 +02:00
Erik Krogh Kristensen
e061c6a006 add support for more custom CSRF checking middlewares 2020-10-20 15:16:14 +02:00
yoff
17155b64f5 Merge pull request #4514 from tausbn/python-add-module-boundary-flow-steps
Python: Add module boundary flow steps
2020-10-20 14:36:10 +02:00
Erik Krogh Kristensen
05d4858d4f autoformat 2020-10-20 14:27:09 +02:00
CodeQL CI
d2282fc474 Merge pull request #4517 from erik-krogh/logAssign
Approved by esbena
2020-10-20 05:24:49 -07:00
Erik Krogh Kristensen
1096cb0708 aggregate the tests in library-tests/InterProceduralFlow into a single .ql file 2020-10-20 14:08:55 +02:00
Erik Krogh Kristensen
234cb5c67a aggregate the tests in library-tests/JSLint into a single .ql file 2020-10-20 14:08:55 +02:00
Erik Krogh Kristensen
6b86b53a2c aggregate the tests in library-tests/LocalObjects into a single .ql file 2020-10-20 14:08:55 +02:00
Erik Krogh Kristensen
3e3e9de45f aggregate the tests in library-tests/TaintBarriers into a single .ql file 2020-10-20 14:08:55 +02:00
Erik Krogh Kristensen
f8d38227e8 aggregate the tests in library-tests/TypeInference/AnalyzedModule into a single .ql file 2020-10-20 14:08:55 +02:00
Erik Krogh Kristensen
036187cd70 aggregate the tests in library-tests/Util into a single .ql file 2020-10-20 14:08:55 +02:00
Erik Krogh Kristensen
90bb38b522 aggregate the tests in library-tests/variables into a single .ql file 2020-10-20 14:08:55 +02:00
Erik Krogh Kristensen
2707531ddd aggregate the tests in library-tests/Closure into a single .ql file 2020-10-20 14:08:54 +02:00
Erik Krogh Kristensen
15fe0bc38a aggregate the tests in library-tests/Comprehensions into a single .ql file 2020-10-20 14:08:54 +02:00
Erik Krogh Kristensen
0ac55bd521 aggregate the tests in library-tests/Flow into a single .ql file 2020-10-20 14:08:54 +02:00
Erik Krogh Kristensen
b43e21a4c7 aggregate the tests in library-tests/HTML/HTMLElementAndHTMLAttribute into a single .ql file 2020-10-20 14:08:54 +02:00