Commit Graph

4560 Commits

Author SHA1 Message Date
Rasmus Wriedt Larsen
925e67d734 Python: Model sensitive data from subscripts 2021-06-04 11:28:07 +02:00
Rasmus Wriedt Larsen
d6532e280a Python: minor cleanup in SensitiveDataSources 2021-06-04 11:28:07 +02:00
Rasmus Wriedt Larsen
00a71a1c41 Python: Port sensitive data modeling
No longer using points-to 🎉
2021-06-04 11:28:07 +02:00
Rasmus Wriedt Larsen
3b68c87b6c Python: Add sensitive data test-cases 2021-06-03 13:38:29 +02:00
Rasmus Wriedt Larsen
79bef11cf7 Python: Use "new" SensitiveDataHeuristics 2021-06-03 12:10:29 +02:00
Rasmus Wriedt Larsen
e9acea8643 Python: Improve multidict modeling 2021-06-03 11:50:49 +02:00
Rasmus Wriedt Larsen
2e851cd5f0 Python: Improve yarl.URL modeling 2021-06-03 11:38:15 +02:00
Rasmus Wriedt Larsen
9372e3b284 Python: Add aiohttp.web change-note 2021-06-03 11:23:28 +02:00
Rasmus Wriedt Larsen
3c47e583d8 Python: Add test for missing data-flow step in aiohttp.web 2021-06-03 10:55:34 +02:00
Rasmus Wriedt Larsen
2dbbf52903 Python: Model HTTP responses in aiohttp.web 2021-06-03 10:55:34 +02:00
Rasmus Wriedt Larsen
735df4597f Python: Aiohttp add response tests 2021-06-03 10:55:34 +02:00
Rasmus Wriedt Larsen
5d4140d3e2 Python: Handle more complicated route-setup in aiohttp
Since we want to be able to easy select request-handlers that are not
set up as part of a view-class, we need to easily be able to identify
those. To handle cases like the one below, we _can't_ just define these
to be all the async functions that are not methods on a class :(

```py
    # see https://docs.aiohttp.org/en/stable/web_quickstart.html#organizing-handlers-in-classes

    class MyCustomHandlerClass:

        async def foo_handler(self, request):  # $ MISSING: requestHandler
            return web.Response(text="MyCustomHandlerClass.foo")

    my_custom_handler = MyCustomHandlerClass()
    app.router.add_get("/MyCustomHandlerClass/foo", my_custom_handler.foo_handler)   # $ routeSetup="/MyCustomHandlerClass/foo"
```

So it seemed easiest to narrow down the route-setups, but that means we
want both refinement and extensibility... so `::Range` pattern to the
rescue 🎉

The important piece of code that still works after this commit, but
which hasn't been changed, is the one below:

```codeql
  /**
   * A parameter that will receive a `aiohttp.web.Request` instance when a request
   * handler is invoked.
   */
  class AiohttpRequestHandlerRequestParam extends Request::InstanceSource, RemoteFlowSource::Range,
    DataFlow::ParameterNode {
    AiohttpRequestHandlerRequestParam() {
      exists(Function requestHandler |
        requestHandler = any(AiohttpCoroutineRouteSetup setup).getARequestHandler() and
```
2021-06-03 10:55:34 +02:00
Rasmus Wriedt Larsen
919a0b6b84 Python: aiohttp route setup is more complicated than expected 2021-06-03 10:55:34 +02:00
Rasmus Wriedt Larsen
c69b857662 Python: Add self.request as RemoteFlowSource for aiohttp View
Just like we do for Django in
7393443f8c/python/ql/src/semmle/python/frameworks/Django.qll (L1786-L1804)
2021-06-03 10:55:34 +02:00
Rasmus Wriedt Larsen
c4b618dcf5 Python: Model view-classes in aiohttp.web
No taint modeling of them yet though
2021-06-03 10:55:34 +02:00
Rasmus Wriedt Larsen
8c039d5688 Python: Add more aiohttp view routing tests 2021-06-03 10:55:34 +02:00
Rasmus Wriedt Larsen
1aa222d7cc Python: Add taint-test for class-based view 2021-06-03 10:55:34 +02:00
Rasmus Wriedt Larsen
fb21bc04fa Python: Add taint-steps for yarl.URL 2021-06-03 10:55:34 +02:00
Rasmus Wriedt Larsen
72e6a1489c Python: Add taint-steps for MultiDictProxy 2021-06-03 10:55:34 +02:00
Rasmus Wriedt Larsen
e76f02b016 Python: Minor refactor to use LocalSourceNode
This just more correctly reflects the reality, since the type-tracking
predicate just below only holds for LocalSourceNode anyway.
2021-06-03 10:55:34 +02:00
Rasmus Wriedt Larsen
dd131e6bf7 Python: Add taint-step for methods on aiohttp.web.Request 2021-06-03 10:55:34 +02:00
Rasmus Wriedt Larsen
63c7fa0c2c Python: aiohttp match_info should be tainted
Whoops
2021-06-03 10:55:34 +02:00
Rasmus Wriedt Larsen
597a9dfc80 Python: Don't consider has_body tainted
Although it technically is, I think it belong in the section of things
that are unlikely to be exploitable
2021-06-03 10:55:34 +02:00
Rasmus Wriedt Larsen
d953ea47d4 Python: Basic handling of tainted attributes in aiohttp 2021-06-03 10:55:34 +02:00
Rasmus Wriedt Larsen
88158e7414 Python: Add basic model setup for aiohttp.web.Request 2021-06-03 10:55:34 +02:00
Rasmus Wriedt Larsen
2b992a635a Python: Add aiohttp taint tests 2021-06-03 10:55:34 +02:00
Rasmus Wriedt Larsen
3cbb909a3a Python: Add modeling of coroutine routes in aiohttp.web 2021-06-03 10:55:33 +02:00
Rasmus Wriedt Larsen
fa1d4e6de7 Python: Extract poor mans function resolution (from django)
Since I also want to use this for aiohttp.web modeling
2021-06-03 10:55:33 +02:00
Rasmus Wriedt Larsen
85d9483c7b Python: Add basic aiohttp tests 2021-06-03 10:55:33 +02:00
Anders Schack-Mulligen
ce509eb7e1 Merge pull request #5927 from aschackmull/dataflow/flowthrough-dispatch-perf
Dataflow: Improve performance in flow-through pruning
2021-06-01 11:46:22 +02:00
Anders Schack-Mulligen
5d21c64247 Dataflow: qldoc fix. 2021-06-01 10:49:47 +02:00
Anders Schack-Mulligen
4f9a6c151b Dataflow: Code review fixes. 2021-06-01 10:29:17 +02:00
Anders Schack-Mulligen
683f853fa5 Dataflow: Fix another bad join order. 2021-05-31 15:14:13 +02:00
Taus
bae3728e3c Merge pull request #5945 from RasmusWL/minor-qldoc-cleanup
Python: Minor QLDoc cleanup
2021-05-31 11:40:44 +02:00
Taus
d9911a016e Merge pull request #5933 from RasmusWL/expand-use-of-input-test
Python: Expand test of py/use-of-input
2021-05-31 11:39:33 +02:00
Aditya Sharad
b41a06a15c Python: Treat py/summary/lines-of-user-code as the primary summary metric
Move the `lines-of-code` tag from `py/summary/lines-of-code`.
Code Scanning will eventually look for this tag.

The intent is to treat the number of lines of user code for Python as the summary of
how much code was analysed, ignoring both external libraries and generated code.
This matches the current baseline metric the CodeQL Action computes for Python.
We'll revisit this decision, and the baseline, if necessary.
2021-05-27 13:20:24 -07:00
Ian Lynagh
f0bec74ce3 python: Correct the ReturnValueIgnored.qhelp docs 2021-05-26 17:40:57 +01:00
Rasmus Wriedt Larsen
f807c2f52b Python: autoformat 2021-05-26 11:07:48 +02:00
Rasmus Wriedt Larsen
d5f2846394 Merge branch 'main' into jorgectf/python/ldapInjection 2021-05-26 11:01:48 +02:00
Rasmus Wriedt Larsen
1b3f857a2f Python: Promote ClickHouse SQL models 2021-05-25 16:27:23 +02:00
Rasmus Wriedt Larsen
eb1da152a0 Python: Rewrite ClickHouse SQL lib modeling
This did turn into a few changes, that maybe could have been split into
separate PRs 🤷

* Rename `ClickHouseDriver` => `ClickhouseDriver`, to better follow
  import name in `.qll` name
* Rewrote modeling to use API graphs
* Split modeling of `aioch` into separate `.qll` file, which does re-use
  the `getExecuteMethodName` predicate. I feel that sharing code between
  the modeling like this was the best approach, and stuck the
  `INTERNAL: Do not use.` labels on both modules.
* I also added handling of keyword arguments (see change in .py files)
2021-05-25 16:13:31 +02:00
Rasmus Wriedt Larsen
c9a9535dbc Python: Use ConceptsTests for ClickHouse SQL libs
This did reveal a few places where we do not detect the incoming SQL
2021-05-25 16:10:06 +02:00
Rasmus Wriedt Larsen
ee3477c20a Python: Remove dummy clickhouse SQL injection query 2021-05-25 14:27:29 +02:00
Rasmus Wriedt Larsen
35793a10bb Merge pull request #5889 from japroc/python-clickhouse-driver
Python: Implement module ClickHouseDriver.qll
2021-05-25 14:25:28 +02:00
Anders Schack-Mulligen
017bf68906 Dataflow: Fix bad join order. 2021-05-25 11:40:53 +02:00
jorgectf
37d6ff76a3 Update tests and .expected 2021-05-21 17:47:53 +02:00
Rasmus Wriedt Larsen
71a93ad311 Python: Fix QLDoc for PyYAML to follow convention 2021-05-21 17:17:23 +02:00
Rasmus Wriedt Larsen
6f50b12333 Python: Fix QLDoc for Werkzeug 2021-05-21 17:16:09 +02:00
Rasmus Wriedt Larsen
c4e244eb80 Python: Add getAwaited to API::Node
I _really_ wanted to call this `.await()`, but that did not fit in with
the convention, or the corresponding `getPromised` in JS.

54f191cfe3/javascript/ql/src/semmle/javascript/ApiGraphs.qll (L184)
2021-05-21 17:11:20 +02:00
Jorge
9e9678b3ca Apply documentation suggestions
Co-authored-by: Rasmus Wriedt Larsen <rasmuswriedtlarsen@gmail.com>
2021-05-21 16:17:39 +02:00