Commit Graph

2502 Commits

Author SHA1 Message Date
Rasmus Wriedt Larsen
ed0fe29d7d Python: Fix grammar
Co-authored-by: Taus <tausbn@github.com>
2020-10-23 13:53:16 +02:00
Rasmus Wriedt Larsen
be166d9c02 Python: Expand Django 2/3 routing tests with 1.x way
Added it to the `testapp` so it's easy to run the server to SEE that it works.

Added it to `routing_test` so it's obvious this is supported by our modeling
when we _know_ it's running Django 2/3.
2020-10-23 13:43:27 +02:00
yoff
462e839a83 Update python/ql/src/experimental/Security-new-dataflow/CWE-022/PathInjection.ql
Co-authored-by: Rasmus Wriedt Larsen <rasmuswriedtlarsen@gmail.com>
2020-10-23 13:35:13 +02:00
Rasmus Wriedt Larsen
ae60ac211b Python: Annotate django v1 routing tests
Again need to remove trailing $, since inline-expectation tests still don't
handle $
2020-10-23 12:05:05 +02:00
Rasmus Wriedt Larsen
78ab637b54 Python: Port django v1 tests 2020-10-23 12:00:27 +02:00
Rasmus Lerchedahl Petersen
f88cc3c98e Python: Use custom PathGraph 2020-10-23 01:10:21 +02:00
Rasmus Wriedt Larsen
41ec4f8166 Python: Use FlaskModel as workaround name
As suggested by Taus 👍
2020-10-22 19:13:26 +02:00
Rasmus Lerchedahl Petersen
8ce5f41366 Python: Fix source of second part of path 2020-10-22 12:20:50 +02:00
Rasmus Lerchedahl Petersen
8549c9cfde Python: Rewrite logic to split on nomalization 2020-10-22 11:35:55 +02:00
Rasmus Lerchedahl Petersen
391925532d Python: PathCheck StartswithCall
Should this use `Value::named`?
2020-10-22 11:34:07 +02:00
Rasmus Lerchedahl Petersen
6e361c7793 Python: Make PathCheck a BarrierGuard 2020-10-22 11:32:18 +02:00
Erik Krogh Kristensen
e89e99deaa Merge pull request #4461 from erik-krogh/pyPrint
Python: implement printAst for Python
2020-10-22 09:37:10 +02:00
Rasmus Lerchedahl Petersen
f8dba85e0f Python: PathNormalization os.path.normpath 2020-10-21 22:21:40 +02:00
Rasmus Lerchedahl Petersen
17273dd27e Python: Add FileSystemAccess open 2020-10-21 22:01:49 +02:00
Rasmus Lerchedahl Petersen
4570c29a11 Python: port query 2020-10-21 21:40:42 +02:00
Rasmus Lerchedahl Petersen
eb5ed23354 Python: Add TaintTracking2 2020-10-21 21:39:50 +02:00
Rasmus Lerchedahl Petersen
da77cbb3d0 Python: concepts PathCheck and PathNormalization
Should they be in a module?
2020-10-21 21:37:43 +02:00
Rasmus Wriedt Larsen
278c071fe6 Python: Modernise werkzeug FileStorage modeling 2020-10-21 20:36:40 +02:00
Rasmus Wriedt Larsen
d0fdb542e5 Python: Modernise werkzeug MultiDict modeling 2020-10-21 20:30:20 +02:00
Rasmus Wriedt Larsen
b6bd70a5da Python: Modernise flask library modeling
Two interesting things happened while doing this:

1. I found out that you can't use the same name to define a submodule as any
parent module. So we need give unique names to the top-level module, and the
module for modeling the `flask.Flask` class. I randomly choose a new name for
the top-level module to get things moving (and not be stuck in bikeshedding
forever).

2. With this new setup, I wanted to expose the `route` and `add_url_rule`
methods on instances of `flask.Flask`. It wasn't quite obvious how to do so. I
simply lumped them next to `classRef()` and `instance()`, without too much
care. I did consider putting them inside a `instance` module, which would allow
you to access them by `flask::Flask::instance::route()`, but I wasn't quite
sure, and just did something easy to get moving.
2020-10-21 20:30:14 +02:00
Rasmus Wriedt Larsen
62d665ecb3 Python: Fix shared QLDoc for InstanceSource 2020-10-21 19:55:40 +02:00
Rasmus Wriedt Larsen
047a326183 Python: Remove reference to old PR
These have been added to internal tracking issue instead
2020-10-21 19:53:25 +02:00
Rasmus Lerchedahl Petersen
2e8cbbd866 Python: Add concept FileSystemAccess 2020-10-21 17:35:21 +02:00
Rasmus Wriedt Larsen
a6abee9b3a Merge pull request #4476 from yoff/python-port-sql-injection
Python: Port SqlInjection
2020-10-21 15:55:19 +02:00
yoff
ea4ea6b3e6 Merge pull request #4529 from tausbn/python-remove-cartesian-product-in-tkwoverflownode
Python: Remove bad join in `getCallableScope`
2020-10-21 15:36:38 +02:00
Rasmus Lerchedahl Petersen
060481053a Python: Add note about incompleteness
I was going to do this in an issue, but it makes sense
to have it in the code. We could still add an issue as well.
2020-10-21 15:15:19 +02:00
Rasmus Lerchedahl Petersen
c57c798bfa Python: Add TODO 2020-10-21 15:10:40 +02:00
yoff
ee5221abb4 Apply suggestions from code review
Co-authored-by: Rasmus Wriedt Larsen <rasmuswriedtlarsen@gmail.com>
2020-10-21 15:08:16 +02:00
Taus Brock-Nannestad
31c169daac Python: Remove bad join in getCallableScope
Clause timing report had this suspicious entry

```
CommandInjection.ql-12:DataFlowPublic::Node::getCallableScope#bbf .................. 7.2s
    (4 evaluations with max 6.4s in DataFlowPublic::Node::getCallableScope#bbf/3@i3#119d7b)
```

which indeed was a bad join:
```
Tuple counts for DataFlowPublic::Node::getCallableScope#bbf:
293509   ~2%     {3} r1 = JOIN DataFlowPublic::Node::getCallableScope#bbf#prev_delta AS L WITH DataFlowPublic::TNode#f AS R ON FIRST 1 OUTPUT L.<1>, L.<0>, L.<2>
22337162 ~0%     {3} r2 = JOIN r1 WITH Scope::Scope::getEnclosingScope_dispred#ff_10#join_rhs AS R ON FIRST 1 OUTPUT r1.<1>, r1.<2>, R.<1>
22337162 ~0%     {3} r3 = r2 AND NOT DataFlowPublic::Node::getCallableScope#bbf#prev AS R(r2.<0>, r2.<2>, r2.<1>)
22337162 ~0%     {3} r4 = SCAN r3 OUTPUT r3.<0>, r3.<2>, r3.<1>
722      ~1%     {3} r5 = JOIN r4 WITH m#DataFlowPublic::Node::getCallableScope#bbf AS R ON FIRST 2 OUTPUT r4.<0>, r4.<1>, r4.<2>
722      ~1%     {3} r6 = JOIN r5 WITH m#DataFlowPublic::Node::getCallableScope#bbf AS R ON FIRST 2 OUTPUT r5.<0>, r5.<2>, r5.<1>
722      ~1%     {3} r7 = r6 AND NOT project#DataFlowPrivate::DataFlowCallable::getScope_dispred#ff AS R(r6.<2>)
722      ~1%     {3} r8 = SCAN r7 OUTPUT r7.<0>, r7.<2>, r7.<1>
                 return r8
```

In this case, the join went away by simply moving the helper predicate
out of the class it was situated in (and since it doesn't mention
`this`, it didn't really belong there in the first place).

Result:
```
DataFlowPublic.qll-8:DataFlowPublic::getCallableScope#ff ........................... 26ms
    (4 evaluations with max 15ms in DataFlowPublic::getCallableScope#ff/2@i3#709a9e)
```
2020-10-21 14:45:05 +02:00
yoff
9bd808c924 Merge pull request #4528 from tausbn/python-remove-cartesian-product-in-tkwoverflownode
Python: Prevent early join on `argName` in `getArg`
2020-10-21 14:43:39 +02:00
Rasmus Lerchedahl Petersen
53ff1a32c1 Merge branch 'main' of github.com:github/codeql into python-port-sql-injection 2020-10-21 14:38:02 +02:00
Rasmus Lerchedahl Petersen
77d4cbc0df Python: Only allow unsafe positional args to extra 2020-10-21 14:21:36 +02:00
Rasmus Lerchedahl Petersen
3a416bce2d Python: Move test annotation 2020-10-21 14:18:16 +02:00
Rasmus Lerchedahl Petersen
4571b3188c Python: Fix false negative 2020-10-21 14:16:35 +02:00
Rasmus Lerchedahl Petersen
03c62fd267 Python: Fix typo in test case 2020-10-21 14:03:46 +02:00
yoff
75357727c4 Merge pull request #4490 from RasmusWL/python-model-django-sources
Python: model Django HttpRequest as RemoteFlowSource
2020-10-21 13:46:51 +02:00
Rasmus Lerchedahl Petersen
7087522ed9 Python: Move doc reference 2020-10-21 13:30:38 +02:00
Rasmus Lerchedahl Petersen
e49c7d64bd Python: test for keyword arguments to extra 2020-10-21 13:28:12 +02:00
Rasmus Lerchedahl Petersen
d249b51a5e Python: Add test-case for indirect RawSQL 2020-10-21 13:23:19 +02:00
Taus Brock-Nannestad
eb3333c0ce Python: Prevent early join on argName in getArg 2020-10-21 13:23:09 +02:00
Rasmus Lerchedahl Petersen
e51543ea79 Python: allow any positional argument in annotate 2020-10-21 13:17:20 +02:00
Rasmus Lerchedahl Petersen
2c855c739b Python: Ignore relatively safe arguments of extra 2020-10-21 13:15:46 +02:00
Rasmus Lerchedahl Petersen
6805fb63cc Python: Use modern pattern for RawSQL class 2020-10-21 13:14:32 +02:00
Rasmus Lerchedahl Petersen
e44247bb00 Python: Add links to function docs 2020-10-21 13:03:14 +02:00
yoff
8e7e107365 Apply suggestions from code review
Co-authored-by: Rasmus Wriedt Larsen <rasmuswriedtlarsen@gmail.com>
2020-10-21 12:55:38 +02:00
Rasmus Wriedt Larsen
5874a7b422 Merge pull request #4488 from yoff/SharedDataflow_ArgumentPassingTests
Python: Shared dataflow, argument passing tests
2020-10-21 12:01:37 +02:00
CodeQL CI
eaed93fa7d Merge pull request #4513 from RasmusWL/python-model-fabric
Approved by yoff
2020-10-21 01:58:19 -07:00
Rasmus Lerchedahl Petersen
9ee5a01d7e Python: Reword comment on isBarrierIn 2020-10-21 10:30:40 +02:00
Rasmus Wriedt Larsen
b0af0b94d0 Python: Fix grammar
Co-authored-by: yoff <lerchedahl@gmail.com>
2020-10-21 09:58:37 +02:00
Erik Krogh Kristensen
e18cf08d99 documentation changes based on review 2020-10-21 09:45:16 +02:00