Commit Graph

41812 Commits

Author SHA1 Message Date
Chris Smowton
1ebb47feb3 Fix filename spelling error 2021-11-10 09:29:50 +00:00
Chris Smowton
2953a44b36 Revert changes to go.sum 2021-11-10 09:25:40 +00:00
Benjamin Muskalla
f9fa22c14d Removed unused import 2021-11-10 10:21:54 +01:00
Benjamin Muskalla
1a751608de Extract Commons IO into seperate file 2021-11-10 10:15:27 +01:00
Mathias Vorreiter Pedersen
e2ab1c8c5e Merge branch 'main' into use-range-analysis-in-buffer-write 2021-11-10 08:28:43 +00:00
Tony Torralba
4da1dce811 Merge pull request #7099 from github/workflow/coverage/update
Update CSV framework coverage reports
2021-11-10 08:54:11 +01:00
Erik Krogh Kristensen
ab5d9459c7 Update javascript/ql/src/Security/CWE-384/SessionFixation.qhelp
Co-authored-by: Ethan Palm <56270045+ethanpalm@users.noreply.github.com>
2021-11-10 08:24:46 +01:00
ihsinme
a0448240aa Update InsecureTemporaryFile.expected 2021-11-10 09:23:51 +03:00
ihsinme
7514fe2b45 Update test.cpp 2021-11-10 09:22:58 +03:00
ihsinme
289d58745a Update InsecureTemporaryFile.ql 2021-11-10 09:22:03 +03:00
pupiles
4d9ce49816 use stubs libs && add heuristic sanitizers 2021-11-10 14:12:45 +08:00
github-actions[bot]
f5426336c3 Add changed framework coverage reports 2021-11-10 00:09:06 +00:00
Taus
33135e909a Python: Add magic to named_argument_transfer
This predicate was materialised as a _big_, _cached_ relation:

```
(169s) Tuple counts for PointsTo::InterProceduralPointsTo::named_argument_transfer#ffff#join_rhs/4@38ce07 after 53.4s:
25212     ~4%     {3} r1 = SCAN Function::Function::getArgByName_dispred#fff OUTPUT In.1, In.0 'arg1', In.2 'arg2'
159751200 ~0%     {4} r2 = JOIN r1 WITH Flow::CallNode::getArgByName_dispred#fff_102#join_rhs ON FIRST 1 OUTPUT Rhs.1 'arg0', Lhs.1 'arg1', Lhs.2 'arg2', Rhs.2 'arg3'
                  return r2
```

... However it's only used in a single place (where it is immediately
joined with the points-to relation to relate the caller and argument),
none of these joins were ever larger than 2000 tuples. This made it
pretty clear that we could gain something by pushing in that points-to
join as a bit of manual magic.

However, doing so didn't actually fix anything, since the join-orderer
then decided to join `func.getArgByName(name)` with
`call.getArgByName(name)` on `name` as the first thing (which caused a
join of the same size as above).

Unbinding didn't work, since `name` would then be an unbound `string`,
so instead I factored out relating the function, parameter, and name
thereof into its own predicate. (I could also have done this with the
call, but I would expect there to be more calls than function
definitions in general.)

Overall, this resulted in going from

```
(709s)
Definitions.ql-7:PointsTo::InterProceduralPointsTo::named_argument_transfer#ffff#join_rhs ......... 53.5s
Definitions.ql-7:Instances::InstanceObject::initializer_dispred#fbf ............................... 35.3s (456 evaluations with max 136ms in Instances::InstanceObject::initializer_dispred#fbf/3@i110#0508e8)
Definitions.ql-10:DefinitionTracking::jump_to_defn_attribute#fbf .................................. 27s (100 evaluations with max 12.8s in DefinitionTracking::jump_to_defn_attribute#fbf/3@i1#fc1f7x)
Definitions.ql-7:PointsTo::PointsToInternal::pointsTo#ffff ........................................ 16.1s (681 evaluations with max 2.5s in PointsTo::PointsToInternal::pointsTo#ffff/4@i4#0508eg)
Definitions.ql-7:Constants::ConstantObjectInternal::attribute#ffff ................................ 13.4s (505 evaluations with max 50ms in Constants::ConstantObjectInternal::attribute#ffff/4@i153#0508e5)
Definitions.ql-10:DefinitionTracking::assignment_jump_to_defn_attribute#fbf ....................... 12.4s (99 evaluations with max 11.8s in DefinitionTracking::assignment_jump_to_defn_attribute#fbf/3@i2#fc1f
7z)
...
```

to

```
(668s)
Definitions.ql-7:Instances::InstanceObject::initializer_dispred#fbf ................... 35.4s (456 evaluations with max 140ms in Instances::InstanceObject::initializer_dispred#fbf/3@i110#bf4328)
Definitions.ql-10:DefinitionTracking::jump_to_defn_attribute#fbf ...................... 27.4s (100 evaluations with max 13.3s in DefinitionTracking::jump_to_defn_attribute#fbf/3@i1#679d7x)
Definitions.ql-7:PointsTo::PointsToInternal::pointsTo#ffff ............................ 16.1s (681 evaluations with max 2.5s in PointsTo::PointsToInternal::pointsTo#ffff/4@i4#bf432g)
Definitions.ql-7:Constants::ConstantObjectInternal::attribute#ffff .................... 14.4s (505 evaluations with max 51ms in Constants::ConstantObjectInternal::attribute#ffff/4@i140#bf4325)
Definitions.ql-10:DefinitionTracking::assignment_jump_to_defn_attribute#fbf ........... 12.3s (99 evaluations with max 11.7s in DefinitionTracking::assignment_jump_to_defn_attribute#fbf/3@i2#679d
7z)
...
```
2021-11-09 21:39:32 +00:00
Taus
e2f79d8516 Python: Fix several bad getScope joins
It seems the optimiser has started getting the wrong end of the stick
whenever we write `foo.getScope() = bar.getScope()` for some expressions
`foo` and `bar`.

This lead to things like

```
(196s) Tuple counts for Definitions::ModuleVariable::global_variable_callnode#ff/2@5ab278 after 2m33s:
2952757013 ~0%     {2} r1 = JOIN Definitions::ModuleVariable::global_variable_callnode#ff#shared WITH Variables::Variable::getScope_dispred#ff_10#join_rhs ON FIRST 1 OUTPUT Rhs.1 'this', Lhs.1 'result'
495693     ~0%     {2} r2 = JOIN r1 WITH Variables::GlobalVariable#class#f ON FIRST 1 OUTPUT Lhs.0 'this', Lhs.1 'result'
453589     ~0%     {2} r3 = JOIN r2 WITH Definitions::ModuleVariable#f ON FIRST 1 OUTPUT Lhs.0 'this', Lhs.1 'result'
                   return r3
```

and

```
(315s) Tuple counts for Definitions::SsaSourceVariable::getAUse_dispred#ff/2@a39328 after 1m57s:
...
1785275    ~3%       {2} r24 = Definitions::ModuleVariable::global_variable_callnode#ff#shared UNION Definitions::SsaSourceVariable::getAUse_dispred#ff#shared
3008614987 ~0%       {2} r25 = JOIN r24 WITH Variables::Variable::getScope_dispred#ff_10#join_rhs ON FIRST 1 OUTPUT Rhs.1 'this', Lhs.1 'result'
127        ~1%       {2} r26 = JOIN r25 WITH Definitions::NonLocalVariable#class#f ON FIRST 1 OUTPUT Lhs.0 'this', Lhs.1 'result'
127        ~1%       {2} r27 = JOIN r26 WITH Variables::LocalVariable#f ON FIRST 1 OUTPUT Lhs.0 'this', Lhs.1 'result'
...
```

(Note the timings: 2m33s and 1m57s.)

Now we have the much more reasonable

```
(38s) Tuple counts for Definitions::ModuleVariable::global_variable_callnode#ff/2@c53031 after 42ms:
453589 ~0%     {2} r1 = JOIN Definitions::ModuleVariable::global_variable_callnode#ff#shared WITH Definitions::ModuleVariable::scope_as_global_variable#ff_10#join_rhs ON FIRST 1 OUTPUT Rhs.1 'this', Lhs.1 'result'
               return r1
```

and

```
(46s) Tuple counts for Definitions::SsaSourceVariable::getAUse_dispred#ff/2@4b19de after 375ms:
...
```
2021-11-09 20:54:41 +00:00
Taus
24000a50e6 Python: Fix bad join in py/redundant-assignment
A minor thing, but still best avoided:
```
Tuple counts for RedundantAssignment::pyflakes_commented_line#bb/2@0477c7 after 14.2s:
160826   ~0%     {2} r1 = SCAN py_comments OUTPUT In.2, In.1
160826   ~0%     {3} r2 = JOIN r1 WITH project#Files::Location::hasLocationInfo_dispred#bfffff ON FIRST 1 OUTPUT Rhs.1, Lhs.1, Rhs.2 'result'
160826   ~3%     {3} r3 = JOIN r2 WITH files_10#join_rhs ON FIRST 1 OUTPUT Lhs.2 'result', Lhs.1, Rhs.1 'file'
46672419 ~4%     {4} r4 = JOIN r3 WITH Files::Location::getStartLine_dispred#bf_10#join_rhs ON FIRST 1 OUTPUT Rhs.1, Lhs.2 'file', Lhs.1, Lhs.0 'result'
3951     ~0%     {4} r5 = JOIN r4 WITH Files::Location::getFile_dispred#bf ON FIRST 2 OUTPUT Lhs.2, Lhs.3 'result', Lhs.1 'file', toLower(Lhs.2)
0        ~0%     {4} r6 = SELECT r5 ON In.3 contains "pyflakes"
0        ~0%     {2} r7 = SCAN r6 OUTPUT In.2 'file', In.1 'result'
                return r7
```
2021-11-09 20:31:44 +00:00
Geoffrey White
26e9adcc34 C++: Change note. 2021-11-09 19:39:21 +00:00
Geoffrey White
ef21d1b512 C++: Add a model for curl as well. 2021-11-09 19:32:43 +00:00
Geoffrey White
6388ac5f1d C++: Add tests. 2021-11-09 18:41:57 +00:00
Geoffrey White
d2b18d952d C++: Add qhelp. 2021-11-09 18:41:56 +00:00
Geoffrey White
bd1e708c5d C++: First version of cpp/non-https-url. 2021-11-09 18:33:49 +00:00
Rasmus Wriedt Larsen
985cd1ebdb Python: Port py/request-without-cert-validation to use API graphs 2021-11-09 16:37:50 +01:00
Rasmus Wriedt Larsen
59581690fd Python: Add py/request-without-cert-validation tests 2021-11-09 16:29:57 +01:00
Rasmus Wriedt Larsen
9710aeecbf Python/C#: Add CWE-1333 to redos queries
As is already done in JS and Ruby.
2021-11-09 16:10:38 +01:00
Anders Schack-Mulligen
1efe1e0d10 Java: Improve algorithm for subtyping of parameterized types. 2021-11-09 15:49:17 +01:00
Tom Hvitved
7178a98e45 Ruby: Rename pruneUseNode{Fwd,Rev} 2021-11-09 15:16:36 +01:00
Tom Hvitved
30251740e3 Ruby: Prune nodes before computing trackUseNode 2021-11-09 15:16:36 +01:00
Tom Hvitved
8195ebf4b3 Merge pull request #7059 from hvitved/ruby/basic-store-step-postupdate
Ruby: Fix `basicStoreStep`
2021-11-09 15:16:07 +01:00
Alex Ford
556cdbaa21 ruby: QL format 2021-11-09 14:09:11 +00:00
Alex Ford
37775407a9 ruby: drop a redundant bit of documentation 2021-11-09 14:07:00 +00:00
Alex Ford
340897f262 ruby: drop unnecessary variable 2021-11-09 14:06:21 +00:00
Alex Ford
a23750a9c7 ruby: inline some predicates 2021-11-09 14:06:21 +00:00
Benjamin Muskalla
40e47c0ea3 Merge pull request #7082 from bmuskalla/filterOutputStream
Java: Model taint for `FilterOutputStream`
2021-11-09 15:06:15 +01:00
Alex Ford
c65d1d9a50 ruby: CSRFProtectionDisabled.qhelp fixes
Co-authored-by: Harry Maclean <hmac@github.com>
2021-11-09 14:05:41 +00:00
Rasmus Wriedt Larsen
f70e4fea55 Python: Add interesting path-injection FP 2021-11-09 14:53:32 +01:00
Mathias Vorreiter Pedersen
10bca3544c C++: Change 'annotate_path_to_sink' so that you now annotate a ir-path with the previous node (instead of its source). This gives a better overview of the path. 2021-11-09 13:49:12 +00:00
jorgectf
5b46b90e10 Fix additional taint step variables 2021-11-09 14:41:35 +01:00
pupiles
70a268dc6d remove redundant reference lib 2021-11-09 21:35:46 +08:00
pupiles
97d4359881 add test code 2021-11-09 21:31:35 +08:00
Benjamin Muskalla
bfe2e2e0b9 Model taint for FilterOutputStream 2021-11-09 14:21:50 +01:00
Rasmus Wriedt Larsen
1e31416049 Merge pull request #7031 from yoff/python/taint-through-with
Python: Taint through `async with`
2021-11-09 14:08:07 +01:00
Alex Ford
c708b6b76f Merge pull request #7077 from github/ruby/downgrade-hardcoded-credentials
Ruby: Downgrade `rb/hardcoded-credentials` precision from high to medium
2021-11-09 12:08:10 +00:00
Rasmus Lerchedahl Petersen
ac5a46f24f Python: split test as suggested in review 2021-11-09 13:04:52 +01:00
yoff
5f4aad40c1 Update python/ql/test/experimental/meta/InlineTaintTest.qll
Co-authored-by: Rasmus Wriedt Larsen <rasmuswriedtlarsen@gmail.com>
2021-11-09 13:00:35 +01:00
Rasmus Lerchedahl Petersen
aa1541a5c3 Python: add changenote 2021-11-09 12:57:36 +01:00
Rasmus Lerchedahl Petersen
a58c47b07b Python: model aiopg.sa 2021-11-09 12:49:57 +01:00
Rasmus Lerchedahl Petersen
f53314019a Python: test aiopg.sa 2021-11-09 12:42:03 +01:00
CodeQL CI
d9d304fc13 Merge pull request #7076 from asgerf/js/tainted-path-regexp-guard2
Approved by erik-krogh
2021-11-09 03:40:37 -08:00
Chris Smowton
f7e6b0ad5d Merge pull request #603 from github/criemen/lua-tracing-config
Add port of the existing compiler-tracing.spec files to the new Lua tracing infrastructure.
2021-11-09 11:36:03 +00:00
Rasmus Lerchedahl Petersen
cd332a75fc Python: model aiopg 2021-11-09 12:32:21 +01:00
Erik Krogh Kristensen
56a7c8b163 fix typo in change note
Co-authored-by: Asger F <asgerf@github.com>
2021-11-09 12:06:29 +01:00