Commit Graph

2229 Commits

Author SHA1 Message Date
Rasmus Wriedt Larsen
88158e7414 Python: Add basic model setup for aiohttp.web.Request 2021-06-03 10:55:34 +02:00
Rasmus Wriedt Larsen
2b992a635a Python: Add aiohttp taint tests 2021-06-03 10:55:34 +02:00
Rasmus Wriedt Larsen
3cbb909a3a Python: Add modeling of coroutine routes in aiohttp.web 2021-06-03 10:55:33 +02:00
Rasmus Wriedt Larsen
85d9483c7b Python: Add basic aiohttp tests 2021-06-03 10:55:33 +02:00
Taus
d9911a016e Merge pull request #5933 from RasmusWL/expand-use-of-input-test
Python: Expand test of py/use-of-input
2021-05-31 11:39:33 +02:00
Rasmus Wriedt Larsen
d5f2846394 Merge branch 'main' into jorgectf/python/ldapInjection 2021-05-26 11:01:48 +02:00
Rasmus Wriedt Larsen
1b3f857a2f Python: Promote ClickHouse SQL models 2021-05-25 16:27:23 +02:00
Rasmus Wriedt Larsen
eb1da152a0 Python: Rewrite ClickHouse SQL lib modeling
This did turn into a few changes, that maybe could have been split into
separate PRs 🤷

* Rename `ClickHouseDriver` => `ClickhouseDriver`, to better follow
  import name in `.qll` name
* Rewrote modeling to use API graphs
* Split modeling of `aioch` into separate `.qll` file, which does re-use
  the `getExecuteMethodName` predicate. I feel that sharing code between
  the modeling like this was the best approach, and stuck the
  `INTERNAL: Do not use.` labels on both modules.
* I also added handling of keyword arguments (see change in .py files)
2021-05-25 16:13:31 +02:00
Rasmus Wriedt Larsen
c9a9535dbc Python: Use ConceptsTests for ClickHouse SQL libs
This did reveal a few places where we do not detect the incoming SQL
2021-05-25 16:10:06 +02:00
Rasmus Wriedt Larsen
35793a10bb Merge pull request #5889 from japroc/python-clickhouse-driver
Python: Implement module ClickHouseDriver.qll
2021-05-25 14:25:28 +02:00
jorgectf
37d6ff76a3 Update tests and .expected 2021-05-21 17:47:53 +02:00
Rasmus Wriedt Larsen
c4e244eb80 Python: Add getAwaited to API::Node
I _really_ wanted to call this `.await()`, but that did not fit in with
the convention, or the corresponding `getPromised` in JS.

54f191cfe3/javascript/ql/src/semmle/javascript/ApiGraphs.qll (L184)
2021-05-21 17:11:20 +02:00
Rasmus Wriedt Larsen
2408573a0a Python: Add API graph test for calling coroutines 2021-05-21 16:08:15 +02:00
Rasmus Wriedt Larsen
7a5fd02442 Python: API graph tests: add --max-import-depth=1
Before this, I ended up extracting 454 modules locally 😱
2021-05-21 15:58:15 +02:00
Rasmus Wriedt Larsen
9a4709c134 Python: API graph tests: Disallow results outside project
Running the tests locally would result in thousands of results before
this 😱
2021-05-21 15:57:10 +02:00
Evgenii Protsenko
1e40213abb use <class> instead of <class>::Range 2021-05-20 22:56:08 +03:00
Rasmus Wriedt Larsen
f17fe442a2 Python: Expand test of py/use-of-input 2021-05-20 14:52:10 +02:00
Rasmus Wriedt Larsen
0292ca6b67 Merge pull request #5880 from tausbn/python-limit-builtins
Python: Limit set of globals that may be built-ins
2021-05-20 14:47:22 +02:00
CodeQL CI
17afbdf258 Merge pull request #5635 from RasmusWL/port-weak-crypto-algorithm
Approved by yoff
2021-05-20 01:22:32 -07:00
Rasmus Wriedt Larsen
61ad5d0673 Python: Allow printing PostUpdateNode in ConceptsTest.qll
See how this works in `test_json.py`
2021-05-19 17:10:33 +02:00
Rasmus Wriedt Larsen
9dbb364cca Python: Move json tests to be part of stdlib
This is better, since the modeling is also part of Stdlib.qll
2021-05-19 17:10:33 +02:00
Rasmus Wriedt Larsen
51a25e45fe Python: Use shared prettyExpr in ConceptsTest.qll
This required quite some changes in the expected output. I think it's much more
clear what the selected nodes are now 👍 (but it was a bit boring work to fix
this up)
2021-05-19 17:10:33 +02:00
Rasmus Wriedt Larsen
1af6d97c51 Python: Remove straggling f-: annotations 2021-05-19 17:10:33 +02:00
Rasmus Wriedt Larsen
f66dccafda Python: Rename prettyExp => prettyExpr
So we're consistenly using `expr` and not leaving our the `r`.
2021-05-19 17:10:33 +02:00
Rasmus Wriedt Larsen
c4987e94e0 Python: Re-introduce syntactic handling of str/bytes/unicode
I don't want to loose results on this, so until type-tracking/API graphs
can handle this, I want to keep our syntactic handling.
2021-05-19 13:00:11 +02:00
Rasmus Wriedt Larsen
aa8b7306a3 Python: Use more API graphs in TaintTrackingPrivate
But now we suddenly don't handle the call to `unicode` :O -- at least
not when I run the test locally (using Python 3).
2021-05-19 12:59:58 +02:00
CodeQL CI
23e8092452 Merge pull request #5864 from RasmusWL/some-framework-modeling
Approved by tausbn
2021-05-19 02:31:06 -07:00
Evgenii Protsenko
af75d85b2e ClickHouseSQLInjection.qll : add tests 2021-05-18 22:49:11 +03:00
Rasmus Wriedt Larsen
97fadd9970 Merge branch 'main' into port-weak-crypto-algorithm 2021-05-18 14:04:18 +02:00
Rasmus Wriedt Larsen
0ade23ab2a Python: Apply suggestions from code review
Co-authored-by: yoff <lerchedahl@gmail.com>
2021-05-18 11:49:59 +02:00
Taus
fe12e620dd Python: Avoid clobbering range in test
This was an unwanted interaction between two unrelated tests, so I
switched to a different built-in in the second test. I also added a test
case that shows an unfortunate side effect of this more restricted
handling of built-ins.
2021-05-12 18:42:10 +00:00
thank_you
3e25b14a68 Update NoSQLInjection.expected 2021-05-11 20:07:09 -04:00
yoff
0e5a2c4573 Merge pull request #5442 from jorgectf/jorgectf/python/redos
Python: Add Regular Expression Injection query
2021-05-11 12:11:35 +02:00
thank_you
3ace49549a Add tests for SqlAlchemy modeling library
After researching SqlAlchemy and it's various query methods, I discovered several types of SQL injection possibilities.

The SQLExecution.py file contains these examples and can be broken up into two types of injections. Injections requiring the text() taint-step and injections NOT requiring the text() taint step.
2021-05-10 16:12:15 -04:00
Rasmus Wriedt Larsen
1b0d5053e7 Python: simplejson load/dump only works with lib installed
Which I had done locally. Problem is the same about not having PostUpdateNode
when points-to is not able to resolve the call, so I'm happy to just make CI
happy right now, and hopefully we'll get a fix to the underlying problem soon 😊
2021-05-10 16:21:29 +02:00
Rasmus Wriedt Larsen
c2a6b811fc Python: Add modeling of ujson PyPI package
The problem with `tainted_filelike` not having taint, is that in the call

`ujson.dump(tainted_obj, tainted_filelike)`

there is no PostUpdateNote for `tainted_filelike` :( The reason is that
points-to is not able to resolve the call, so none of the clauses in
`argumentPreUpdateNode` matches

See 08731fc6cf/python/ql/src/semmle/python/dataflow/new/internal/DataFlowPrivate.qll (L101-L111)

Let's deal with that issue in an other PR though
2021-05-10 15:10:31 +02:00
Rasmus Wriedt Larsen
72d08f4d6e Python: Model json load/dump 2021-05-10 15:10:30 +02:00
Rasmus Wriedt Larsen
63f28d7d9b Python: Model keyword args to json loads/dumps 2021-05-10 15:10:29 +02:00
Rasmus Wriedt Larsen
784e0cdb96 Python: Improve tests of json module
Inspired by the work on previous commit
2021-05-10 15:10:28 +02:00
Rasmus Wriedt Larsen
3fe9a3d933 Python: Add modeling of simplejson PyPI package
I noticed that we don't handle PostUpdateNote very well in the concept tests,
for exmaple for `json.dump(...)` there _should_ have been an `encodeOutput` as
part of the inline expectations.

I'll work on fixing that up in a separate PR, to keep things clean.
2021-05-10 15:10:27 +02:00
Rasmus Wriedt Larsen
8afdf26540 Python: Add modeling of idna PyPI package 2021-05-10 11:47:11 +02:00
thank_you
07c3e22428 Fix method name to match flask_mongoengine library 2021-05-09 19:23:52 -04:00
jorgectf
e7bdc73420 Update .expected 2021-05-07 23:00:21 +02:00
jorgectf
65c6f1976a Rename mongoengine-flask-db-document-subclass 2021-05-07 23:00:08 +02:00
Jorge
ae806cd445 Merge branch 'github:main' into jorgectf/python/ldapimproperauth 2021-05-07 20:46:09 +02:00
thank_you
7693d696cc Add additional query tests
To ensure that this query works against numerous usages of libraries such as PyMongo, Flask PyMongo, Mongoengine, and Flask Mongoengine, I've added a variety of query tests to test against. These tests deal with scenarious such as:

- Subscript expressions
- Mongoengine instances and Document subclasses
- Mongoengine connection usage
- And more...
2021-05-07 14:36:02 -04:00
Rasmus Wriedt Larsen
d50f22504e Python: Fix .expected 2021-05-05 14:07:15 +02:00
Rasmus Wriedt Larsen
668bfd3a41 Python: Support EC keygen without class-instance for cryptography
I also added a new test to show off how what the origin ends up looking
like... I think it looks ok
2021-05-05 12:29:55 +02:00
Rasmus Wriedt Larsen
3ceb8bbcc6 Python: Add cryptography test for EC
Apparently, passing in the class (without instantiating it) is allowed
2021-05-05 10:52:57 +02:00
CodeQL CI
95f26aadd3 Merge pull request #5681 from yoff/python-support-pathlib
Approved by tausbn
2021-05-04 09:20:24 -07:00