Commit Graph

33872 Commits

Author SHA1 Message Date
Benjamin Muskalla
1a4fd7bc7d Allow camelcase names 2021-11-10 16:30:20 +01:00
Benjamin Muskalla
2b2ac82fb7 Fix bug in sink detection 2021-11-10 16:30:20 +01:00
Benjamin Muskalla
c616eb1473 Fix finding more sources 2021-11-10 16:30:19 +01:00
Benjamin Muskalla
38579ef25b Add proper metadata to queries 2021-11-10 16:30:19 +01:00
Benjamin Muskalla
a80d50cbc0 Simplify field flow 2021-11-10 16:30:19 +01:00
Benjamin Muskalla
a1d8dfb524 Initial support for source models 2021-11-10 16:30:19 +01:00
Benjamin Muskalla
c844f5382f Add script to generate flow models 2021-11-10 16:30:19 +01:00
Benjamin Muskalla
f9fea15a52 Initial support for capturing sink models 2021-11-10 16:30:18 +01:00
Benjamin Muskalla
364de55b8d Support parameter->parameter flow 2021-11-10 16:30:18 +01:00
Benjamin Muskalla
cd11ef3bf6 Support outgoing taint flow from fields 2021-11-10 16:30:18 +01:00
Benjamin Muskalla
c3462be2c9 Capture argument->return value flows 2021-11-10 16:30:18 +01:00
Benjamin Muskalla
4ca006ba3d Only expose visible innner classes 2021-11-10 16:30:18 +01:00
Benjamin Muskalla
88032afdc3 Add test for final class 2021-11-10 16:30:17 +01:00
Benjamin Muskalla
ec772fb6b2 Add support for qualifier flow 2021-11-10 16:30:17 +01:00
Benjamin Muskalla
32ef40c77b Add scaffolding for summary model generator 2021-11-10 16:30:17 +01:00
Erik Krogh Kristensen
2d907f825e have the aliasPropertyPresenceStep step over extend calls 2021-11-10 16:26:00 +01:00
Geoffrey White
ae622bd482 C++: Use hasGlobalOrStdName. 2021-11-10 14:57:07 +00:00
Tom Hvitved
198b321158 Java: Hide parameters of summarized callables 2021-11-10 15:13:32 +01:00
Tom Hvitved
34fdf11b4b Ruby: Update expected test output 2021-11-10 15:11:13 +01:00
Tom Hvitved
82abab1510 C#: Hide parameters of summarized callables 2021-11-10 15:11:13 +01:00
Anders Schack-Mulligen
6d9fb3ca43 Dataflow: Sync. 2021-11-10 15:11:13 +01:00
Anders Schack-Mulligen
678a21e532 Dataflow: Support hidden return nodes in subpaths. 2021-11-10 15:11:13 +01:00
Mathias Vorreiter Pedersen
e0b876d2f6 Merge pull request #7102 from MathiasVP/fix-map-test
C++: Fix a testcase
2021-11-10 13:51:10 +00:00
yoff
d23a920ed4 Merge branch 'main' into python/model-aiomysql 2021-11-10 14:32:36 +01:00
Rasmus Lerchedahl Petersen
57e7bfbdba Python: model aiomysql 2021-11-10 14:29:39 +01:00
Rasmus Lerchedahl Petersen
047cff0749 Python: test aiomysql 2021-11-10 14:24:45 +01:00
Mathias Vorreiter Pedersen
ccdaf49464 C++: Fix the same bug in the test for ordered maps. 2021-11-10 13:24:27 +00:00
Erik Krogh Kristensen
55434653f5 add CWE-532 to the clear-text-logging query 2021-11-10 14:15:49 +01:00
Erik Krogh Kristensen
98da532c46 dont extract regular expressions from strings that are leaves in a string concat 2021-11-10 14:11:48 +01:00
Mathias Vorreiter Pedersen
86d78b34aa C++: Use the correct variable in the 'test'. 2021-11-10 13:04:48 +00:00
Rasmus Wriedt Larsen
de926dc2a1 Merge pull request #7085 from yoff/python/model-aiopg
Python: model aiopg
2021-11-10 13:10:30 +01:00
Rasmus Wriedt Larsen
1f90dcadf1 Python: Use backtracker for verify arg 2021-11-10 12:54:23 +01:00
Geoffrey White
2f39c64cc2 C++: Fix character in qhelp. 2021-11-10 11:23:57 +00:00
Benjamin Muskalla
0f086056a1 Merge pull request #7100 from bmuskalla/bmuskalla/ioAsFile
Java: Extract Commons IO into seperate file
2021-11-10 12:04:12 +01:00
Max Schaefer
a8c4455b20 Factor out an auxiliary predicate. 2021-11-10 10:17:59 +00:00
Rasmus Lerchedahl Petersen
92a7114b72 Python: Add API references 2021-11-10 11:06:58 +01:00
Rasmus Lerchedahl Petersen
c6d285dd2a Python: Fix test 2021-11-10 11:06:45 +01:00
yoff
a856395d56 Apply suggestions from code review
Co-authored-by: Rasmus Wriedt Larsen <rasmuswriedtlarsen@gmail.com>
2021-11-10 10:51:40 +01:00
Rasmus Wriedt Larsen
77099fe9d0 Python: Always use @id py/
The two queries in CWE-020 are used for manual evaluation (is my
understanding), and the two IDE queries should work based on their tags,
and not on the query-id.
2021-11-10 10:40:17 +01:00
Benjamin Muskalla
f9fa22c14d Removed unused import 2021-11-10 10:21:54 +01:00
Benjamin Muskalla
1a751608de Extract Commons IO into seperate file 2021-11-10 10:15:27 +01:00
Mathias Vorreiter Pedersen
e2ab1c8c5e Merge branch 'main' into use-range-analysis-in-buffer-write 2021-11-10 08:28:43 +00:00
Tony Torralba
4da1dce811 Merge pull request #7099 from github/workflow/coverage/update
Update CSV framework coverage reports
2021-11-10 08:54:11 +01:00
Erik Krogh Kristensen
ab5d9459c7 Update javascript/ql/src/Security/CWE-384/SessionFixation.qhelp
Co-authored-by: Ethan Palm <56270045+ethanpalm@users.noreply.github.com>
2021-11-10 08:24:46 +01:00
ihsinme
a0448240aa Update InsecureTemporaryFile.expected 2021-11-10 09:23:51 +03:00
ihsinme
7514fe2b45 Update test.cpp 2021-11-10 09:22:58 +03:00
ihsinme
289d58745a Update InsecureTemporaryFile.ql 2021-11-10 09:22:03 +03:00
github-actions[bot]
f5426336c3 Add changed framework coverage reports 2021-11-10 00:09:06 +00:00
Taus
33135e909a Python: Add magic to named_argument_transfer
This predicate was materialised as a _big_, _cached_ relation:

```
(169s) Tuple counts for PointsTo::InterProceduralPointsTo::named_argument_transfer#ffff#join_rhs/4@38ce07 after 53.4s:
25212     ~4%     {3} r1 = SCAN Function::Function::getArgByName_dispred#fff OUTPUT In.1, In.0 'arg1', In.2 'arg2'
159751200 ~0%     {4} r2 = JOIN r1 WITH Flow::CallNode::getArgByName_dispred#fff_102#join_rhs ON FIRST 1 OUTPUT Rhs.1 'arg0', Lhs.1 'arg1', Lhs.2 'arg2', Rhs.2 'arg3'
                  return r2
```

... However it's only used in a single place (where it is immediately
joined with the points-to relation to relate the caller and argument),
none of these joins were ever larger than 2000 tuples. This made it
pretty clear that we could gain something by pushing in that points-to
join as a bit of manual magic.

However, doing so didn't actually fix anything, since the join-orderer
then decided to join `func.getArgByName(name)` with
`call.getArgByName(name)` on `name` as the first thing (which caused a
join of the same size as above).

Unbinding didn't work, since `name` would then be an unbound `string`,
so instead I factored out relating the function, parameter, and name
thereof into its own predicate. (I could also have done this with the
call, but I would expect there to be more calls than function
definitions in general.)

Overall, this resulted in going from

```
(709s)
Definitions.ql-7:PointsTo::InterProceduralPointsTo::named_argument_transfer#ffff#join_rhs ......... 53.5s
Definitions.ql-7:Instances::InstanceObject::initializer_dispred#fbf ............................... 35.3s (456 evaluations with max 136ms in Instances::InstanceObject::initializer_dispred#fbf/3@i110#0508e8)
Definitions.ql-10:DefinitionTracking::jump_to_defn_attribute#fbf .................................. 27s (100 evaluations with max 12.8s in DefinitionTracking::jump_to_defn_attribute#fbf/3@i1#fc1f7x)
Definitions.ql-7:PointsTo::PointsToInternal::pointsTo#ffff ........................................ 16.1s (681 evaluations with max 2.5s in PointsTo::PointsToInternal::pointsTo#ffff/4@i4#0508eg)
Definitions.ql-7:Constants::ConstantObjectInternal::attribute#ffff ................................ 13.4s (505 evaluations with max 50ms in Constants::ConstantObjectInternal::attribute#ffff/4@i153#0508e5)
Definitions.ql-10:DefinitionTracking::assignment_jump_to_defn_attribute#fbf ....................... 12.4s (99 evaluations with max 11.8s in DefinitionTracking::assignment_jump_to_defn_attribute#fbf/3@i2#fc1f
7z)
...
```

to

```
(668s)
Definitions.ql-7:Instances::InstanceObject::initializer_dispred#fbf ................... 35.4s (456 evaluations with max 140ms in Instances::InstanceObject::initializer_dispred#fbf/3@i110#bf4328)
Definitions.ql-10:DefinitionTracking::jump_to_defn_attribute#fbf ...................... 27.4s (100 evaluations with max 13.3s in DefinitionTracking::jump_to_defn_attribute#fbf/3@i1#679d7x)
Definitions.ql-7:PointsTo::PointsToInternal::pointsTo#ffff ............................ 16.1s (681 evaluations with max 2.5s in PointsTo::PointsToInternal::pointsTo#ffff/4@i4#bf432g)
Definitions.ql-7:Constants::ConstantObjectInternal::attribute#ffff .................... 14.4s (505 evaluations with max 51ms in Constants::ConstantObjectInternal::attribute#ffff/4@i140#bf4325)
Definitions.ql-10:DefinitionTracking::assignment_jump_to_defn_attribute#fbf ........... 12.3s (99 evaluations with max 11.7s in DefinitionTracking::assignment_jump_to_defn_attribute#fbf/3@i2#679d
7z)
...
```
2021-11-09 21:39:32 +00:00
Taus
e2f79d8516 Python: Fix several bad getScope joins
It seems the optimiser has started getting the wrong end of the stick
whenever we write `foo.getScope() = bar.getScope()` for some expressions
`foo` and `bar`.

This lead to things like

```
(196s) Tuple counts for Definitions::ModuleVariable::global_variable_callnode#ff/2@5ab278 after 2m33s:
2952757013 ~0%     {2} r1 = JOIN Definitions::ModuleVariable::global_variable_callnode#ff#shared WITH Variables::Variable::getScope_dispred#ff_10#join_rhs ON FIRST 1 OUTPUT Rhs.1 'this', Lhs.1 'result'
495693     ~0%     {2} r2 = JOIN r1 WITH Variables::GlobalVariable#class#f ON FIRST 1 OUTPUT Lhs.0 'this', Lhs.1 'result'
453589     ~0%     {2} r3 = JOIN r2 WITH Definitions::ModuleVariable#f ON FIRST 1 OUTPUT Lhs.0 'this', Lhs.1 'result'
                   return r3
```

and

```
(315s) Tuple counts for Definitions::SsaSourceVariable::getAUse_dispred#ff/2@a39328 after 1m57s:
...
1785275    ~3%       {2} r24 = Definitions::ModuleVariable::global_variable_callnode#ff#shared UNION Definitions::SsaSourceVariable::getAUse_dispred#ff#shared
3008614987 ~0%       {2} r25 = JOIN r24 WITH Variables::Variable::getScope_dispred#ff_10#join_rhs ON FIRST 1 OUTPUT Rhs.1 'this', Lhs.1 'result'
127        ~1%       {2} r26 = JOIN r25 WITH Definitions::NonLocalVariable#class#f ON FIRST 1 OUTPUT Lhs.0 'this', Lhs.1 'result'
127        ~1%       {2} r27 = JOIN r26 WITH Variables::LocalVariable#f ON FIRST 1 OUTPUT Lhs.0 'this', Lhs.1 'result'
...
```

(Note the timings: 2m33s and 1m57s.)

Now we have the much more reasonable

```
(38s) Tuple counts for Definitions::ModuleVariable::global_variable_callnode#ff/2@c53031 after 42ms:
453589 ~0%     {2} r1 = JOIN Definitions::ModuleVariable::global_variable_callnode#ff#shared WITH Definitions::ModuleVariable::scope_as_global_variable#ff_10#join_rhs ON FIRST 1 OUTPUT Rhs.1 'this', Lhs.1 'result'
               return r1
```

and

```
(46s) Tuple counts for Definitions::SsaSourceVariable::getAUse_dispred#ff/2@4b19de after 375ms:
...
```
2021-11-09 20:54:41 +00:00