Commit Graph

2229 Commits

Author SHA1 Message Date
CodeQL CI
eaed93fa7d Merge pull request #4513 from RasmusWL/python-model-fabric
Approved by yoff
2020-10-21 01:58:19 -07:00
Rasmus Lerchedahl Petersen
9ee5a01d7e Python: Reword comment on isBarrierIn 2020-10-21 10:30:40 +02:00
Rasmus Wriedt Larsen
b0af0b94d0 Python: Fix grammar
Co-authored-by: yoff <lerchedahl@gmail.com>
2020-10-21 09:58:37 +02:00
Rasmus Lerchedahl Petersen
90d0cff384 Python: Use flask routing 2020-10-21 00:30:16 +02:00
Rasmus Lerchedahl Petersen
383d846396 Python: address review
- smooth out future merge
- keyword argument for execute
2020-10-21 00:15:05 +02:00
Rasmus Lerchedahl Petersen
e1dfbc0486 Python: address review 2020-10-20 23:59:44 +02:00
Rasmus Wriedt Larsen
6920f3012c Python: Django route handlers in different file now works
Fixed by https://github.com/github/codeql/pull/4514
2020-10-20 15:41:14 +02:00
Rasmus Wriedt Larsen
c8441dc4fb Merge branch 'main' into python-model-django-sources 2020-10-20 15:38:20 +02:00
yoff
17155b64f5 Merge pull request #4514 from tausbn/python-add-module-boundary-flow-steps
Python: Add module boundary flow steps
2020-10-20 14:36:10 +02:00
Rasmus Wriedt Larsen
80adbdfbc1 Python: Mark unhandled django route handlers with f-:
That is playing more nicely with the expected usage of the inline-tests.
2020-10-20 13:44:34 +02:00
Taus Brock-Nannestad
a21c29507c Python: Fix false negative
I'm slightly suspicious of this fix -- it seems to work, but it makes
me wonder if we're potentially missing other kinds of flow, by not
handling other kinds of definitions.

Also, I feel like this should really be attached to an appropriate
post-update node of the given argument. As it is written now, the flow
will go from the argument _before_ the call, which obviously misses a
step if the argument is modified by the call. In practice, I would
expect this to be rather rare.
2020-10-20 13:16:54 +02:00
Taus Brock-Nannestad
860cafed4d Python: Mark failing test as false negative 2020-10-20 13:11:06 +02:00
Rasmus Wriedt Larsen
045a6c3cb5 Python: Add test for tricky module member for type-tracking
Local testing shows that the `getDefinition` result for this is a `SSA filter definition`,
and not an `AssignmentDefinition`.
2020-10-20 12:20:35 +02:00
Rasmus Lerchedahl Petersen
5990241c8f Python: Support django models (with some caveats) 2020-10-20 03:20:00 +02:00
Rasmus Lerchedahl Petersen
d7308bddf2 Python: Add django sink with concept test 2020-10-19 21:34:55 +02:00
Taus Brock-Nannestad
7755993dd3 Python: Add jump steps for module attribute reads.
This is the quick-and-dirty solution, as discussed.

An even quicker-and-dirtier solution would have used
`ModuleValue::attr` and take the `getOrigin` of that as the source of
the jump step. However, this turns out to be a bad choice, since
`attr` might fail to have a value for the given attribute (for a
variety of reasons). Thus, we instead appeal to a helper predicate
that keeps track of which names are defined by which right-hand-sides
in a given module. (Observe that type tracking works correctly for `x`
in `mymodule.py`, even though `x` is never assigned a value in the
eyes of the Value API.)

This means that points-to is only used to actually figure out if the
object we're looking an attribute up on is a module or not. This is
the next thing to replace in order to eliminate the dependence on
points-to, but this will require some care to ensure that all module
lookups are handled correctly.

Only two test files needed to be changed for the tests to pass. The
first was the fixed false negative in the type tracker, and the other
was a bunch of missing flow in the regression test. I have manually
removed the `# Flow not found` annotations to make them consistent
with the output. Pay particular attention to the annotation on line
117 -- I believe it was misplaced and should have been on line 106
instead (where, indeed, we now have flow where none appeared before).
2020-10-19 19:13:32 +02:00
Rasmus Wriedt Larsen
98691fe8ec Python: Model fabric Group execution (version 2.x)
This required some thought for how to model that we're interested in subclasses
of `fabric.group.Group`, and not so much that class itself. Some thoughts:

---

After initially using this in `module Group`

    /** A reference to a subclass of `fabric.group.Group` */
    abstract class SubclassRef extends DataFlow::Node { }

    private class SubclassInstantiation extends SubclassInstanceSource, DataFlow::CfgNode {
      override CallNode node;

      SubclassInstantiation() { node.getFunction() = any(SubclassRef ref).asCfgNode() }
    }

with this in `module SerialGroup` and `module ThreadingGroup`:

    class ClassRef extends DataFlow::Node, fabric::group::Group::SubclassRef {
      ClassRef() { this = classRef(DataFlow::TypeTracker::end()) }
    }

I wasn't too much of fan of that approach. Since we probably need the `SubclassInstanceSource` anyway, and don't really have a specific use for `SubclassRef`, I just went with concrete (QL) subclasses of `SubclassInstanceSource` in each of the modules for the Python subclasses.

I really don't know what the best approach is, so I'm very open to suggestions. I think we'll really have to flesh this out for handling Django responses, since we're interested in the fact that some subclasses provide default values for the content-type, and keeping track of that is important for XSS (since there is no XSS if response is `text/plain`)
2020-10-19 18:09:11 +02:00
Rasmus Wriedt Larsen
f10456e35f Python: Model fabric task decorator (version 2.x) 2020-10-19 18:03:03 +02:00
Rasmus Wriedt Larsen
c671017252 Python: Model fabric Connection (version 2.x) 2020-10-19 18:03:02 +02:00
Rasmus Wriedt Larsen
f7502386e7 Python: Model fabric package (version 1.x) 2020-10-19 18:03:01 +02:00
Rasmus Wriedt Larsen
6b30198d59 Python: Port old fabric tests
For v1 tests, just extended with explicit calls that use keyword arguments.

For v2 tests, rewrote pretty much everything to what it 100% explicit what we support
2020-10-19 14:34:22 +02:00
Rasmus Lerchedahl Petersen
646ced2a1d Python: Add concept test scaffold 2020-10-19 10:58:57 +02:00
Rasmus Lerchedahl Petersen
f17720f587 Python: Add test and fix filename 2020-10-19 10:58:57 +02:00
Taus Brock-Nannestad
60fcb5e7d5 Python: Add PEP-328 test example
Based on https://www.python.org/dev/peps/pep-0328/#guido-s-decision

Original "code" is in the Public Domain.
2020-10-16 12:03:43 +02:00
Rasmus Wriedt Larsen
86798063a3 Python: Model tainted attributes of django HttpRequest 2020-10-16 11:12:20 +02:00
Rasmus Wriedt Larsen
a3cdbf2052 Python: Basic modeling of Django HttpRequest 2020-10-16 11:12:19 +02:00
Rasmus Wriedt Larsen
f547b6010b Python: Implement routed parameter for django.urls.re_path 2020-10-16 11:12:17 +02:00
Rasmus Wriedt Larsen
ff8708df67 Python: Implement routed parameter for django.urls.path
Matching current implementation in
f07a7bf8cf/python/ql/src/semmle/python/web/django/General.qll (L125-L133)
2020-10-16 11:12:16 +02:00
Rasmus Wriedt Larsen
44683f2959 Python: Identify route handlers for django
Not including class based handlers
2020-10-16 11:12:14 +02:00
Rasmus Wriedt Larsen
c0d71f767a Python: Add taint test for django v2/v3 2020-10-16 11:12:13 +02:00
Rasmus Wriedt Larsen
09a2a6cdfd Python: Fix django re_path kwarg test
Since it was using the wrong keyword argument name before :D
2020-10-16 11:12:12 +02:00
Rasmus Wriedt Larsen
b28d022be9 Python: Add simpel model of a django path/re_path route setup
Also had to change the annotation to not include the `r` prefix for the
raw-string... not sure why that isn't replicated, but ¯\_(ツ)_/¯
2020-10-16 11:12:11 +02:00
Rasmus Wriedt Larsen
979dc471ac Python: Port old routing tests 2020-10-16 11:12:09 +02:00
Rasmus Wriedt Larsen
ca60132e24 Python: Django test: Add simple route handler and annotations 2020-10-16 11:12:09 +02:00
Rasmus Wriedt Larsen
44b9b7f084 Python: Django test: Enable app
and add a bits of use docs
2020-10-16 11:12:08 +02:00
Rasmus Wriedt Larsen
6506e5d646 Python: Django test: Add testapp 2020-10-16 11:12:07 +02:00
Rasmus Wriedt Larsen
c71c41b759 Python: Django test: Disable DB (for now) 2020-10-16 11:12:06 +02:00
Rasmus Wriedt Larsen
f704c566b9 Python: Add real django 3.1 project as base of tests 2020-10-16 11:12:05 +02:00
Rasmus Lerchedahl Petersen
cced335284 Python: Switch tests to path queries 2020-10-16 09:59:30 +02:00
Tom Hvitved
27fc610c0d Python: Update expected test output 2020-10-16 09:09:06 +02:00
Rasmus Lerchedahl Petersen
41b8877579 Python: Improve test configuration 2020-10-16 08:32:05 +02:00
Rasmus Wriedt Larsen
5142bfaf01 Merge pull request #4453 from yoff/python-port-unsafe-deserialization
Python: port unsafe deserialization
2020-10-15 17:26:31 +02:00
Rasmus Wriedt Larsen
58baec5b06 Merge pull request #4364 from yoff/SharedDataflow_ArgumentPassing
Python: Shared dataflow, argument passing
2020-10-15 17:10:59 +02:00
Rasmus Lerchedahl Petersen
ef32488596 Merge branch 'main' of github.com:github/codeql into python-port-unsafe-deserialization 2020-10-15 15:45:35 +02:00
Erik Krogh Kristensen
1d2ca42d49 update expected output 2020-10-15 13:42:13 +02:00
Rasmus Lerchedahl Petersen
172e058438 Python: unsafe -> mayExecuteInput 2020-10-15 12:56:29 +02:00
Rasmus Lerchedahl Petersen
9c8e968cba Python: Fix bad merge 2020-10-15 11:47:34 +02:00
Rasmus Wriedt Larsen
ce967e1249 Merge branch 'main' into python-model-python2-specific-command-execution 2020-10-15 10:00:02 +02:00
Rasmus Lerchedahl Petersen
0766eef49b Merge branch 'main' of github.com:github/codeql into SharedDataflow_ArgumentPassing 2020-10-15 09:49:21 +02:00
Taus
466c22f4a8 Merge pull request #4435 from RasmusWL/python-port-code-injection
Python: port code injection query
2020-10-14 16:41:42 +02:00