Commit Graph

31240 Commits

Author SHA1 Message Date
Alex Ford
ede1503cc6 Merge pull request #7328 from github/ruby/customizations
Ruby: add `Customizations.qll` file
2021-12-08 10:54:23 +00:00
Tom Hvitved
283173ad02 Address review comments 2021-12-08 11:26:44 +01:00
Erik Krogh Kristensen
3145e8f9b7 add upgrade script 2021-12-08 10:53:47 +01:00
Erik Krogh Kristensen
1956405d17 Merge pull request #7284 from erik-krogh/myApply-part1
JS: remove paths without unmatched returns from polynomial-redos
2021-12-08 10:46:03 +01:00
Tom Hvitved
5735bb698d Ruby: Hide desugared nodes in data-flow paths 2021-12-08 09:00:16 +01:00
yoff
0e33f730b1 Merge pull request #7329 from tausbn/tausbn/python-fix-syntax-error-locations
Python: Fix syntax error locations
2021-12-07 22:45:35 +01:00
Geoffrey White
5ee9684435 C++: Change note. 2021-12-07 20:42:36 +00:00
Geoffrey White
122f6385e6 C++: Improve recognition of stdin, stdout etc. 2021-12-07 20:42:35 +00:00
Geoffrey White
6896b20dcd C++: Redesign and fix results that appear to be encrypted. 2021-12-07 20:42:13 +00:00
Alex Ford
ea7063f3c6 Ruby: make Customizations import private 2021-12-07 19:43:08 +00:00
Tom Hvitved
490872173a Data flow: Sync files 2021-12-07 20:29:18 +01:00
Tom Hvitved
07ca1c2ec0 Data flow: Adjust parameterMatch join-orders 2021-12-07 20:29:00 +01:00
Erik Krogh Kristensen
fe1a473f07 Merge pull request #138 from github/erik-krogh/bump-this
bump the severity of `ql/implicit-this`
2021-12-07 20:12:21 +01:00
Erik Krogh Kristensen
a722aeb5e3 QL: Merge pull request #138 from github/erik-krogh/bump-this
bump the severity of `ql/implicit-this`
2021-12-07 20:12:21 +01:00
Taus
e7c298d903 Python: Fix bad scope_entry_points_to join
From `pritomrajkhowa/LoopBound`:

```
Definitions.ql-7:PointsTo::PointsToInternal::scope_entry_points_to#ffff#antijoin_rhs#2 ........... 55.1s
```

specifically

```
(443s) Tuple counts for PointsTo::PointsToInternal::scope_entry_points_to#ffff#antijoin_rhs#2/3@74a7cart after 55.1s:
184070    ~0%        {3} r1 = JOIN PointsTo::PointsToInternal::scope_entry_points_to#ffff#shared#1 WITH Variables::GlobalVariable#class#f ON FIRST 1 OUTPUT Lhs.0 'arg2', Lhs.1 'arg0', Lhs.2 'arg1'
184070    ~0%        {3} r2 = STREAM DEDUP r1
919966523 ~2%        {4} r3 = JOIN r2 WITH Essa::EssaDefinition::getSourceVariable_dispred#ff_10#join_rhs ON FIRST 1 OUTPUT Rhs.1, Lhs.1 'arg0', Lhs.2 'arg1', Lhs.0 'arg2'
4281779   ~2293%     {3} r4 = JOIN r3 WITH Essa::EssaVariable::getScope_dispred#ff ON FIRST 2 OUTPUT Lhs.1 'arg0', Lhs.2 'arg1', Lhs.3 'arg2'
                    return r4
```

First, this is an `antijoin`, so there's likely some negation involved.
Also, there's mention of `GlobalVariable`, `getScope`, and
`getSourceVariable`, none of which appear in `scope_entry_points_to`, so
it's likely that something got inlined.

Taking a closer look at the predicates mentioned in the body, we spot
`undefined_variable` as a likely culprit.

Evaluating this predicate in isolation reveals that it's not terribly
big, so we could try just marking it with `pragma[noinline]` (I opted
for the slightly more solid `nomagic`) and see how that fares. I also
checked that `builtin_not_in_outer_scope` was similarly small, and
made that one un-inlineable as well.

The result? Well, I can't even show you. Both `scope_entry_points_to`
and `undefined_variable` are so fast that they don't appear in the
clause timing report (so they can at most take 3.5s each to evaluate, as
that is the smallest timing in the list).
2021-12-07 18:51:44 +00:00
Taus
b502ca1ea7 Python: Fix bad callsite_points_to join
From `pritomrajkhowa/LoopBound`:

```
Definitions.ql-7:PointsTo::InterProceduralPointsTo::callsite_points_to#ffff#join_rhs#3 ........... 5m53s
```

specifically

```
(767s) Tuple counts for PointsTo::InterProceduralPointsTo::callsite_points_to#ffff#join_rhs#3/3@f8f86764 after 5m53s:
832806293 ~0%     {4} r1 = JOIN PointsTo::InterProceduralPointsTo::callsite_points_to#ffff#shared#1 WITH PointsTo::InterProceduralPointsTo::var_at_exit#fff ON FIRST 1 OUTPUT Lhs.0, Lhs.1 'arg1', Rhs.1 'arg2', Rhs.2 'arg0'
832806293 ~0%     {3} r2 = JOIN r1 WITH Essa::TEssaNodeRefinement#ffff_03#join_rhs ON FIRST 2 OUTPUT Lhs.3 'arg0', Lhs.1 'arg1', Lhs.2 'arg2'
                return r2
```

This one is a bit tricky to unpack. Where is this `shared#1` defined?

```
EVALUATE NONRECURSIVE RELATION:
SYNTHETIC PointsTo::InterProceduralPointsTo::callsite_points_to#ffff#shared#1(int arg0, numbered_tuple arg1) :-
    SENTINEL PointsTo::InterProceduralPointsTo::callsite_points_to#ffff#shared
    SENTINEL Definitions::EscapingAssignmentGlobalVariable#class#f
    SENTINEL Essa::TEssaNodeRefinement#ffff_03#join_rhs
    {2} r1 = JOIN PointsTo::InterProceduralPointsTo::callsite_points_to#ffff#shared WITH Definitions::EscapingAssignmentGlobalVariable#class#f ON FIRST 1 OUTPUT Lhs.0 'arg0', Lhs.1 'arg1'
    {2} r2 = STREAM DEDUP r1
    {2} r3 = JOIN r2 WITH Essa::TEssaNodeRefinement#ffff_03#join_rhs ON FIRST 2 OUTPUT Lhs.0 'arg0', Lhs.1 'arg1'
    {2} r4 = STREAM DEDUP r3
    return r4
```

Looking at `callsite_points_to`, we see a likely candidate in `srcvar`.
It is guarded with an `instanceof` check for
`EscapingAssignmentGlobalVariable` (which lines up nicely with the
sentinel on its charpred) and `getSourceVariable` is just a projection
of `TEssaNodeRefinement`.

So let's try unbinding `srcvar` to prevent an early join.

The timing is now:

```
Definitions.ql-7:PointsTo::InterProceduralPointsTo::callsite_points_to#ffff ...................... 31.3s (2554 evaluations with max 101ms in PointsTo::InterProceduralPointsTo::callsite_points_to#ffff/4@i516#581fap5w)
```
(Showing the tuple counts doesn't make sense here, since all of the
`shared` and `join_rhs` predicates have been smooshed around.)
2021-12-07 18:25:53 +00:00
Taus
a716482c1f Python: Fix bad join in SSA
On `pritomrajkhowa/LoopBound`:

```
Definitions.ql-3:SsaCompute::SsaComputeImpl::AdjacentUsesImpl::adjacentUseUse#ff ................. 4m35s
```

specifically

```
(376s) Tuple counts for SsaCompute::SsaComputeImpl::AdjacentUsesImpl::adjacentUseUse#ff/2@be04e9kp after 4m58s:
388843     ~0%     {4} r1 = JOIN Essa::TPhiFunction#fff_2#join_rhs WITH SsaCompute::SsaComputeImpl::AdjacentUsesImpl::definesAt#ffff ON FIRST 1 OUTPUT Rhs.1, Lhs.0, Rhs.2, Rhs.3
3629812090 ~1%     {7} r2 = JOIN r1 WITH SsaCompute::SsaComputeImpl::variableUse#ffff ON FIRST 1 OUTPUT Lhs.0, Rhs.2, Rhs.3, Lhs.2, Lhs.3, Lhs.1, Rhs.1 'use1'
0          ~0%     {2} r3 = JOIN r2 WITH SsaCompute::SsaComputeImpl::AdjacentUsesImpl::adjacentVarRefs#fffff ON FIRST 5 OUTPUT Lhs.5, Lhs.6 'use1'
0          ~0%     {2} r4 = JOIN r3 WITH SsaCompute::SsaComputeImpl::AdjacentUsesImpl::firstUse#ff ON FIRST 1 OUTPUT Lhs.1 'use1', Rhs.1 'use2'

897141     ~0%     {2} r5 = SsaCompute::SsaComputeImpl::AdjacentUsesImpl::adjacentUseUseSameVar#ff UNION r4
                    return r5
```

Clearly we do not want to join on the variable so soon. So we unbind it
and get

```
(78s) Tuple counts for SsaCompute::SsaComputeImpl::AdjacentUsesImpl::adjacentUseUse#ff/2@40e0e6uv after 434ms:
3377959 ~2%     {4} r1 = SCAN SsaCompute::SsaComputeImpl::variableUse#ffff OUTPUT In.0, In.2, In.3, In.1 'use1'
1026855 ~2%     {4} r2 = JOIN r1 WITH SsaCompute::SsaComputeImpl::AdjacentUsesImpl::adjacentVarRefs#fffff ON FIRST 3 OUTPUT Lhs.0, Rhs.3, Rhs.4, Lhs.3 'use1'
129484  ~0%     {2} r3 = JOIN r2 WITH SsaCompute::SsaComputeImpl::AdjacentUsesImpl::definesAt#ffff_1230#join_rhs ON FIRST 3 OUTPUT Rhs.3, Lhs.3 'use1'
0       ~0%     {2} r4 = JOIN r3 WITH Essa::TPhiFunction#fff_2#join_rhs ON FIRST 1 OUTPUT Lhs.0, Lhs.1 'use1'
0       ~0%     {2} r5 = JOIN r4 WITH SsaCompute::SsaComputeImpl::AdjacentUsesImpl::firstUse#ff ON FIRST 1 OUTPUT Lhs.1 'use1', Rhs.1 'use2'

897141  ~0%     {2} r6 = SsaCompute::SsaComputeImpl::AdjacentUsesImpl::adjacentUseUseSameVar#ff UNION r5
                return r6
```
2021-12-07 18:19:47 +00:00
Taus
59bac04d8f Python: Fix Python 2 failures 2021-12-07 18:00:46 +00:00
Taus
ffc858e34d Python: Add missing file 2021-12-07 17:29:35 +00:00
Alex Ford
bf0ecded04 Merge pull request #7326 from github/ginsbach/FixInstanceof
fix dependency cycle by removing superfluous classes
2021-12-07 17:05:26 +00:00
Taus
7437cd4d85 Python: Fix syntax error locations 2021-12-07 16:51:33 +00:00
Geoffrey White
511bee7a1a C++: Fix results that flow to/from encryption routines. 2021-12-07 15:44:18 +00:00
Alex Ford
f85a47d41f Ruby: add Customizations.qll file 2021-12-07 15:37:04 +00:00
Tom Hvitved
a2dc505c26 Merge pull request #7317 from hvitved/ruby/param-node-refactor
Ruby: Restructure `ParameterNode(Impl)`
2021-12-07 16:29:49 +01:00
Tom Hvitved
5183290439 Merge pull request #7315 from hvitved/ruby/inline-flow-test
Ruby: Add `InlineFlowTest.qll`
2021-12-07 16:29:34 +01:00
Henry Mercer
322e39446d JS: Autoformat 2021-12-07 14:17:11 +00:00
Henry Mercer
016727d6b6 JS: Fix occasional duplicate body tokens
0e31439 introduces some occasional duplicate tokens due to duplicate AST
node attributes. The long-term fix is to update `CodeToFeatures.qll`,
but for the short-term, we update the concatenation to concatenate
unique (location, token) pairs.
2021-12-07 14:16:48 +00:00
Philip Ginsbach
b2c1b55c0c rephrase extensions as aliases 2021-12-07 13:09:25 +00:00
Tom Hvitved
b17a93eaad Merge pull request #7316 from hvitved/ruby/is-private-join
Ruby: Tweak `Method::isPrivate` join-orders
2021-12-07 13:58:19 +01:00
Tom Hvitved
4d797d6b3d Merge pull request #7324 from github/hmac/empty-else-cfg
Ruby: Include empty StmtSequences in CFG
2021-12-07 13:19:15 +01:00
Philip Ginsbach
da43984ba4 fix dependency cycle by removing superfluous classes 2021-12-07 11:59:04 +00:00
Geoffrey White
b82425a35c C++: Add various new test cases. 2021-12-07 11:58:56 +00:00
Geoffrey White
2d4a2e0d44 C++: Test spacing. 2021-12-07 11:58:06 +00:00
Rasmus Wriedt Larsen
ee23799a59 Merge pull request #7319 from RasmusWL/js-cwe-328
JS: Tag queries with CWE-328
2021-12-07 11:40:33 +01:00
Anders Schack-Mulligen
6c739b67fa Merge pull request #7318 from RasmusWL/java-cwe-328
Java: Tag queries with CWE-328
2021-12-07 11:39:48 +01:00
Asger Feldthaus
23480b2d8f JS: Remove stray TODO 2021-12-07 10:49:14 +01:00
Asger F
614c80706f Apply suggestions from code review
Co-authored-by: Erik Krogh Kristensen <erik-krogh@github.com>
2021-12-07 10:48:44 +01:00
Asger Feldthaus
5559681916 JS: Change note 2021-12-07 10:46:18 +01:00
Asger Feldthaus
635ac0a209 JS: Fix perf issue in data flow step generation 2021-12-07 10:46:18 +01:00
Asger Feldthaus
da8e67b7ee JS: Use routing trees to detect deeply tainted req.body 2021-12-07 10:46:18 +01:00
Asger Feldthaus
7492293c5b JS: Add test with route handler indirection 2021-12-07 10:46:18 +01:00
Asger Feldthaus
3cbe94ac0a JS: Add consistency checks to TemplateObjectInjection test 2021-12-07 10:46:18 +01:00
Asger Feldthaus
64db70f3ac JS: Add explicit body-parsers to TemplateObjectInjection test 2021-12-07 10:46:18 +01:00
Asger Feldthaus
8af430d40f JS: Shift line numbers in TemplateObjectInjection test 2021-12-07 10:46:17 +01:00
Asger Feldthaus
5f8ea3965d JS: Do not flag auth endpoints that are immune to Login CSRF 2021-12-07 10:46:17 +01:00
Asger Feldthaus
66b1612e5e JS: Treat non-cookie based auth as CSRF preventer 2021-12-07 10:46:17 +01:00
Asger Feldthaus
b73219392b JS: Improve precision of missing CSRF middleware 2021-12-07 10:46:17 +01:00
Asger Feldthaus
d0e94e655d JS: Exclude error handling from auth calls 2021-12-07 10:46:17 +01:00
Asger Feldthaus
400bf10cc3 JS: Move fastify-specific route handler step into extension point 2021-12-07 10:46:17 +01:00
Asger Feldthaus
71820569e1 JS: Instantiate for Fastify 2021-12-07 10:46:15 +01:00