Commit Graph

2409 Commits

Author SHA1 Message Date
Rasmus Wriedt Larsen
87f31a96d7 Python: Add flask_attr helper 2020-10-23 14:31:28 +02:00
Rasmus Wriedt Larsen
bfc29bb349 Python: Add annotations for flask response tests
The fact that we need to add routeSetup and routeHandler annotations is sort of
annoying :|
2020-10-23 14:31:27 +02:00
Rasmus Wriedt Larsen
47dcc09992 Python: Add tests for creating HTTP responses in flask
Which is runnable, if you have flask installed locally
2020-10-23 14:31:26 +02:00
Rasmus Wriedt Larsen
8aaa36bd99 Python: Port ReflectedXss query (and tests) 2020-10-23 14:31:25 +02:00
Rasmus Wriedt Larsen
df6fd53a7e Python: Add HttpResponse concept
We might need to rework this a bit when we also start to handle redirects. I
could see a world where we simply allow http redirects to be subclasses of http
responses, and need to manually exclude them from queries (or create
HttpContentResponse to model the HttpResponses that will contain a body). Let us
see where the wind will take us.

I looked through JS and Go libraries, but I didn't feel their modeling would map
very well to Python.
2020-10-23 14:31:25 +02:00
Rasmus Wriedt Larsen
0d6165883c Python: Fix spelling for Server::RouteSetup documentation 2020-10-23 14:31:24 +02:00
CodeQL CI
bbda22c769 Merge pull request #4534 from RasmusWL/python-update-flask-modeling
Approved by tausbn
2020-10-23 13:28:19 +01:00
Rasmus Wriedt Larsen
b3e53f8d0a Python: Model django.conf.urls.url (v 1.x) 2020-10-23 14:26:37 +02:00
Taus Brock-Nannestad
6d81ca12c4 Python: Fix bad join order in adjacentUseUseSameVar 2020-10-23 14:08:45 +02:00
Rasmus Wriedt Larsen
ed0fe29d7d Python: Fix grammar
Co-authored-by: Taus <tausbn@github.com>
2020-10-23 13:53:16 +02:00
Rasmus Wriedt Larsen
be166d9c02 Python: Expand Django 2/3 routing tests with 1.x way
Added it to the `testapp` so it's easy to run the server to SEE that it works.

Added it to `routing_test` so it's obvious this is supported by our modeling
when we _know_ it's running Django 2/3.
2020-10-23 13:43:27 +02:00
yoff
462e839a83 Update python/ql/src/experimental/Security-new-dataflow/CWE-022/PathInjection.ql
Co-authored-by: Rasmus Wriedt Larsen <rasmuswriedtlarsen@gmail.com>
2020-10-23 13:35:13 +02:00
Rasmus Wriedt Larsen
ae60ac211b Python: Annotate django v1 routing tests
Again need to remove trailing $, since inline-expectation tests still don't
handle $
2020-10-23 12:05:05 +02:00
Rasmus Wriedt Larsen
78ab637b54 Python: Port django v1 tests 2020-10-23 12:00:27 +02:00
Rasmus Lerchedahl Petersen
f88cc3c98e Python: Use custom PathGraph 2020-10-23 01:10:21 +02:00
Rasmus Wriedt Larsen
41ec4f8166 Python: Use FlaskModel as workaround name
As suggested by Taus 👍
2020-10-22 19:13:26 +02:00
Rasmus Lerchedahl Petersen
8ce5f41366 Python: Fix source of second part of path 2020-10-22 12:20:50 +02:00
Rasmus Lerchedahl Petersen
8549c9cfde Python: Rewrite logic to split on nomalization 2020-10-22 11:35:55 +02:00
Rasmus Lerchedahl Petersen
391925532d Python: PathCheck StartswithCall
Should this use `Value::named`?
2020-10-22 11:34:07 +02:00
Rasmus Lerchedahl Petersen
6e361c7793 Python: Make PathCheck a BarrierGuard 2020-10-22 11:32:18 +02:00
Erik Krogh Kristensen
e89e99deaa Merge pull request #4461 from erik-krogh/pyPrint
Python: implement printAst for Python
2020-10-22 09:37:10 +02:00
Rasmus Lerchedahl Petersen
f8dba85e0f Python: PathNormalization os.path.normpath 2020-10-21 22:21:40 +02:00
Rasmus Lerchedahl Petersen
17273dd27e Python: Add FileSystemAccess open 2020-10-21 22:01:49 +02:00
Rasmus Lerchedahl Petersen
4570c29a11 Python: port query 2020-10-21 21:40:42 +02:00
Rasmus Lerchedahl Petersen
eb5ed23354 Python: Add TaintTracking2 2020-10-21 21:39:50 +02:00
Rasmus Lerchedahl Petersen
da77cbb3d0 Python: concepts PathCheck and PathNormalization
Should they be in a module?
2020-10-21 21:37:43 +02:00
Rasmus Wriedt Larsen
278c071fe6 Python: Modernise werkzeug FileStorage modeling 2020-10-21 20:36:40 +02:00
Rasmus Wriedt Larsen
d0fdb542e5 Python: Modernise werkzeug MultiDict modeling 2020-10-21 20:30:20 +02:00
Rasmus Wriedt Larsen
b6bd70a5da Python: Modernise flask library modeling
Two interesting things happened while doing this:

1. I found out that you can't use the same name to define a submodule as any
parent module. So we need give unique names to the top-level module, and the
module for modeling the `flask.Flask` class. I randomly choose a new name for
the top-level module to get things moving (and not be stuck in bikeshedding
forever).

2. With this new setup, I wanted to expose the `route` and `add_url_rule`
methods on instances of `flask.Flask`. It wasn't quite obvious how to do so. I
simply lumped them next to `classRef()` and `instance()`, without too much
care. I did consider putting them inside a `instance` module, which would allow
you to access them by `flask::Flask::instance::route()`, but I wasn't quite
sure, and just did something easy to get moving.
2020-10-21 20:30:14 +02:00
Rasmus Wriedt Larsen
62d665ecb3 Python: Fix shared QLDoc for InstanceSource 2020-10-21 19:55:40 +02:00
Rasmus Wriedt Larsen
047a326183 Python: Remove reference to old PR
These have been added to internal tracking issue instead
2020-10-21 19:53:25 +02:00
Rasmus Lerchedahl Petersen
2e8cbbd866 Python: Add concept FileSystemAccess 2020-10-21 17:35:21 +02:00
Rasmus Wriedt Larsen
a6abee9b3a Merge pull request #4476 from yoff/python-port-sql-injection
Python: Port SqlInjection
2020-10-21 15:55:19 +02:00
yoff
ea4ea6b3e6 Merge pull request #4529 from tausbn/python-remove-cartesian-product-in-tkwoverflownode
Python: Remove bad join in `getCallableScope`
2020-10-21 15:36:38 +02:00
Rasmus Lerchedahl Petersen
060481053a Python: Add note about incompleteness
I was going to do this in an issue, but it makes sense
to have it in the code. We could still add an issue as well.
2020-10-21 15:15:19 +02:00
Rasmus Lerchedahl Petersen
c57c798bfa Python: Add TODO 2020-10-21 15:10:40 +02:00
yoff
ee5221abb4 Apply suggestions from code review
Co-authored-by: Rasmus Wriedt Larsen <rasmuswriedtlarsen@gmail.com>
2020-10-21 15:08:16 +02:00
Taus Brock-Nannestad
31c169daac Python: Remove bad join in getCallableScope
Clause timing report had this suspicious entry

```
CommandInjection.ql-12:DataFlowPublic::Node::getCallableScope#bbf .................. 7.2s
    (4 evaluations with max 6.4s in DataFlowPublic::Node::getCallableScope#bbf/3@i3#119d7b)
```

which indeed was a bad join:
```
Tuple counts for DataFlowPublic::Node::getCallableScope#bbf:
293509   ~2%     {3} r1 = JOIN DataFlowPublic::Node::getCallableScope#bbf#prev_delta AS L WITH DataFlowPublic::TNode#f AS R ON FIRST 1 OUTPUT L.<1>, L.<0>, L.<2>
22337162 ~0%     {3} r2 = JOIN r1 WITH Scope::Scope::getEnclosingScope_dispred#ff_10#join_rhs AS R ON FIRST 1 OUTPUT r1.<1>, r1.<2>, R.<1>
22337162 ~0%     {3} r3 = r2 AND NOT DataFlowPublic::Node::getCallableScope#bbf#prev AS R(r2.<0>, r2.<2>, r2.<1>)
22337162 ~0%     {3} r4 = SCAN r3 OUTPUT r3.<0>, r3.<2>, r3.<1>
722      ~1%     {3} r5 = JOIN r4 WITH m#DataFlowPublic::Node::getCallableScope#bbf AS R ON FIRST 2 OUTPUT r4.<0>, r4.<1>, r4.<2>
722      ~1%     {3} r6 = JOIN r5 WITH m#DataFlowPublic::Node::getCallableScope#bbf AS R ON FIRST 2 OUTPUT r5.<0>, r5.<2>, r5.<1>
722      ~1%     {3} r7 = r6 AND NOT project#DataFlowPrivate::DataFlowCallable::getScope_dispred#ff AS R(r6.<2>)
722      ~1%     {3} r8 = SCAN r7 OUTPUT r7.<0>, r7.<2>, r7.<1>
                 return r8
```

In this case, the join went away by simply moving the helper predicate
out of the class it was situated in (and since it doesn't mention
`this`, it didn't really belong there in the first place).

Result:
```
DataFlowPublic.qll-8:DataFlowPublic::getCallableScope#ff ........................... 26ms
    (4 evaluations with max 15ms in DataFlowPublic::getCallableScope#ff/2@i3#709a9e)
```
2020-10-21 14:45:05 +02:00
yoff
9bd808c924 Merge pull request #4528 from tausbn/python-remove-cartesian-product-in-tkwoverflownode
Python: Prevent early join on `argName` in `getArg`
2020-10-21 14:43:39 +02:00
Rasmus Lerchedahl Petersen
53ff1a32c1 Merge branch 'main' of github.com:github/codeql into python-port-sql-injection 2020-10-21 14:38:02 +02:00
Rasmus Lerchedahl Petersen
77d4cbc0df Python: Only allow unsafe positional args to extra 2020-10-21 14:21:36 +02:00
Rasmus Lerchedahl Petersen
3a416bce2d Python: Move test annotation 2020-10-21 14:18:16 +02:00
Rasmus Lerchedahl Petersen
4571b3188c Python: Fix false negative 2020-10-21 14:16:35 +02:00
Rasmus Lerchedahl Petersen
03c62fd267 Python: Fix typo in test case 2020-10-21 14:03:46 +02:00
yoff
75357727c4 Merge pull request #4490 from RasmusWL/python-model-django-sources
Python: model Django HttpRequest as RemoteFlowSource
2020-10-21 13:46:51 +02:00
Rasmus Lerchedahl Petersen
7087522ed9 Python: Move doc reference 2020-10-21 13:30:38 +02:00
Rasmus Lerchedahl Petersen
e49c7d64bd Python: test for keyword arguments to extra 2020-10-21 13:28:12 +02:00
Rasmus Lerchedahl Petersen
d249b51a5e Python: Add test-case for indirect RawSQL 2020-10-21 13:23:19 +02:00
Taus Brock-Nannestad
eb3333c0ce Python: Prevent early join on argName in getArg 2020-10-21 13:23:09 +02:00
Rasmus Lerchedahl Petersen
e51543ea79 Python: allow any positional argument in annotate 2020-10-21 13:17:20 +02:00