Commit Graph

51 Commits

Author SHA1 Message Date
yoff
1bcaa56b10 Python: rebless second round after shared-CFG dataflow migration
Second batch of test reblessings, capturing changes in result content
(not just toString labels):

- Framework taint/concept tests (fastapi, sqlalchemy, aiohttp, lxml,
  stdlib, django-orm): mostly gained MISSING-tainted annotations where
  the new dataflow no longer reaches sinks. Some are real taint
  regressions; left as documented failures for follow-up.

- Exception-handler tests (CWE-209-StackTraceExposure, EmptyExcept,
  CatchingBaseException, IncorrectExceptOrder, FileNotAlwaysClosed,
  FindSubclass/Find, Statements/exit/UseOfExit): the no-raise shared CFG
  abstraction does not emit ExceptionSuccessor abrupt-completion edges
  from arbitrary expressions, so except-handler bodies (and their
  exception target Names) are statically dead. Tracked separately under
  cfg-modelling-exceptions.

- Dataflow-path / control-flow node toString polish across the security
  query suite (PathInjection, CodeInjection, UnsafeUnpacking,
  UnsafeUsageOfClientSideEncryptionVersion, RequestWithoutValidation,
  ReflectedXss, CallGraph): simple-leaf nodes now stringify as their
  AST text instead of 'After X'.

- SSA / call-graph improvements (CmpTest, CallGraph/InlineCallGraphTest):
  fewer SSA mismatches between new and old; two previously-MISSING tt=
  annotations resolved.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-28 21:09:49 +00:00
yoff
04b130f57f Python: fix library-test compile errors and rebless after CFG migration
Library-test compile fixes after the shared-CFG migration:
- PointsTo/global, PointsTo/local: use `f.getNode() = s.getValue()`
  instead of `s.getValue().getAFlowNode() = f` (the new CFG does not
  surface getAFlowNode on AST nodes).
- PointsTo/new/ImpliesDataflow: bridge new Cfg::ControlFlowNode to the
  legacy ControlFlowNodeWithPointsTo via AST identity.
- frameworks/aiohttp + frameworks/modeling-example: qualify CallNode /
  NameNode / AttrNode casts with Cfg:: now that those names live in
  the new CFG facade.

Rebless 4 expected files for toString-only differences (renamed CFG
positions like 'CFG node for foo' vs 'foo' — no semantic change):
ImpliesDataflow, EnclosingCallable, NaiveModel, ProperModel.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-28 21:09:49 +00:00
Jeroen Ketema
c3ea883b11 Python: Update expected test results 2024-12-03 19:18:57 +01:00
Joe Farebrother
b28d79960b Update ConceptsTests and make a fix 2024-07-23 10:15:09 +01:00
Joe Farebrother
c404f00a9b Add additional header write models for aiohttp and tornado + added qldoc 2024-06-24 17:27:25 +01:00
Rasmus Wriedt Larsen
55f5b26ba6 Python: Accept new ordering of query predicates in .expected 2023-11-15 10:09:54 +01:00
Rasmus Wriedt Larsen
4f47461f60 Python: Add requested test 2023-08-08 10:44:48 +02:00
Rasmus Wriedt Larsen
991d5cc54b Python: Fix test of HttpResponse.getBody() 2023-07-13 13:57:08 +02:00
Rasmus Wriedt Larsen
64a7206f3e Python: Improve aiohttp FileResponse/StreamResponse modeling
However, notice that the concepts tests use the HttpResponse location
for the `responseBody` tag, which seems a little odd in this situation,
where they are actually separate. Will fix in next commit.
2023-07-13 13:57:08 +02:00
Rasmus Wriedt Larsen
15269c9166 Python: Add StreamResponse test 2023-07-13 13:57:08 +02:00
Rasmus Wriedt Larsen
0f9ab8f53e Python: Fixup tests
But notice that keyword argument is not handled yet
2023-07-13 13:57:08 +02:00
Alvaro Muñoz
ee1ba71e5d add tests 2023-07-13 13:07:12 +02:00
Jeroen Ketema
8cee4f37a4 Merge branch 'main' into inline-5 2023-07-11 10:30:11 +02:00
Rasmus Wriedt Larsen
1f93e5b58d Python: Relax restriction of flow through async with 2023-07-06 11:51:58 +02:00
Rasmus Wriedt Larsen
43af8d7ac5 Python: Fix test to use async with
It doesn't work if just using plain `with`
2023-07-06 11:34:05 +02:00
Rasmus Wriedt Larsen
79039dc7b8 Python: Wrap aiohttp client request in async def
And I added `await` before all the `resp` assignments
2023-07-06 11:29:14 +02:00
Jeroen Ketema
abe06e5b95 Python: Update remaining inline expectation tests to use the paramterized module 2023-07-03 10:22:35 +02:00
Jeroen Ketema
dba4460526 Python: Update more inline expectation tests to use the paramterized module 2023-06-20 10:16:15 +02:00
Rasmus Lerchedahl Petersen
4b4b9bf9da python: add missing summaries
For append/add:
The new results in the experimental tar slip query
show that we do not recognize the sanitisers.
2023-06-13 20:22:21 +02:00
Rasmus Lerchedahl Petersen
b72c93ff4f python: remove remaining explicit taint steps 2023-06-13 20:22:20 +02:00
Rasmus Wriedt Larsen
cfd640b1b2 Python: Apply suggestions from code review
Co-authored-by: yoff <lerchedahl@gmail.com>
2022-06-14 16:47:24 +02:00
Rasmus Wriedt Larsen
f72a1d98bb Python: Model certificate disabling in aiohttp.client 2022-06-08 17:41:45 +02:00
haby0
be40b54b9f add test 2022-02-28 20:34:58 +08:00
Rasmus Wriedt Larsen
38875ca0c7 Python: Improve handling of async methods 2021-07-22 14:17:07 +02:00
Rasmus Wriedt Larsen
6e9d9fcbbd Python: Improve taint steps in for & iterable unpacking
These were written way before the ones in DataFlowPrivate, but
apparently didn't cover quite as much :|
2021-07-22 14:16:17 +02:00
Rasmus Wriedt Larsen
226425e831 Python: Model CookieWrite for aiohttp 2021-06-24 17:34:43 +02:00
Rasmus Wriedt Larsen
e1af1f11ee Python: Add HTTP::Server::CookieWrite concept
along with tests, but no implementations (to ease reviewing).

---

I've put quite some thinking into what to call our concept for this.

[JS has `CookieDefinition`](581f4ed757/javascript/ql/src/semmle/javascript/frameworks/HTTP.qll (L148-L187)), but I couldn't find a matching concept in any other languages.

We used to call this [`CookieSet`](f07a7bf8cf/python/ql/src/semmle/python/web/Http.qll (L76)) (and had a corresponding `CookieGet`).

But for headers, [Go calls this `HeaderWrite`](cd1e14ed09/ql/src/semmle/go/concepts/HTTP.qll (L97-L131)) and [JS calls this `HeaderDefinition`](581f4ed757/javascript/ql/src/semmle/javascript/frameworks/HTTP.qll (L23-L46))

I think it would be really cool if we have a naming scheme that means the name for getting the value of a header on a incoming request is obvious. I think `HeaderWrite`/`HeaderRead` fulfils this best. We could go with `HeaderSet`/`HeaderGet`, but they feel a bit too vague to me. For me, I'm so used to talking about def-use, that I would immediately go for `HeaderDefinition` and `HeaderUse`, which could work, but is kinda strange.

So in the end that means I went with `CookieWrite`, since that allows using a consistent naming scheme for the future :)
2021-06-24 17:34:43 +02:00
Rasmus Wriedt Larsen
53f7633662 Python: Model await request.post() as MultiDictProxy
as highlight as being quite easy to do by @yoff 👍
2021-06-11 14:53:30 +02:00
Rasmus Wriedt Larsen
df67028a1d Python: Model aiohttp.StreamReader 2021-06-11 12:06:53 +02:00
Rasmus Wriedt Larsen
2d31ef7016 Python: Fix last TODOs in aiohttp tests 2021-06-11 12:00:02 +02:00
Rasmus Wriedt Larsen
e9acea8643 Python: Improve multidict modeling 2021-06-03 11:50:49 +02:00
Rasmus Wriedt Larsen
2e851cd5f0 Python: Improve yarl.URL modeling 2021-06-03 11:38:15 +02:00
Rasmus Wriedt Larsen
3c47e583d8 Python: Add test for missing data-flow step in aiohttp.web 2021-06-03 10:55:34 +02:00
Rasmus Wriedt Larsen
2dbbf52903 Python: Model HTTP responses in aiohttp.web 2021-06-03 10:55:34 +02:00
Rasmus Wriedt Larsen
735df4597f Python: Aiohttp add response tests 2021-06-03 10:55:34 +02:00
Rasmus Wriedt Larsen
5d4140d3e2 Python: Handle more complicated route-setup in aiohttp
Since we want to be able to easy select request-handlers that are not
set up as part of a view-class, we need to easily be able to identify
those. To handle cases like the one below, we _can't_ just define these
to be all the async functions that are not methods on a class :(

```py
    # see https://docs.aiohttp.org/en/stable/web_quickstart.html#organizing-handlers-in-classes

    class MyCustomHandlerClass:

        async def foo_handler(self, request):  # $ MISSING: requestHandler
            return web.Response(text="MyCustomHandlerClass.foo")

    my_custom_handler = MyCustomHandlerClass()
    app.router.add_get("/MyCustomHandlerClass/foo", my_custom_handler.foo_handler)   # $ routeSetup="/MyCustomHandlerClass/foo"
```

So it seemed easiest to narrow down the route-setups, but that means we
want both refinement and extensibility... so `::Range` pattern to the
rescue 🎉

The important piece of code that still works after this commit, but
which hasn't been changed, is the one below:

```codeql
  /**
   * A parameter that will receive a `aiohttp.web.Request` instance when a request
   * handler is invoked.
   */
  class AiohttpRequestHandlerRequestParam extends Request::InstanceSource, RemoteFlowSource::Range,
    DataFlow::ParameterNode {
    AiohttpRequestHandlerRequestParam() {
      exists(Function requestHandler |
        requestHandler = any(AiohttpCoroutineRouteSetup setup).getARequestHandler() and
```
2021-06-03 10:55:34 +02:00
Rasmus Wriedt Larsen
919a0b6b84 Python: aiohttp route setup is more complicated than expected 2021-06-03 10:55:34 +02:00
Rasmus Wriedt Larsen
c69b857662 Python: Add self.request as RemoteFlowSource for aiohttp View
Just like we do for Django in
7393443f8c/python/ql/src/semmle/python/frameworks/Django.qll (L1786-L1804)
2021-06-03 10:55:34 +02:00
Rasmus Wriedt Larsen
c4b618dcf5 Python: Model view-classes in aiohttp.web
No taint modeling of them yet though
2021-06-03 10:55:34 +02:00
Rasmus Wriedt Larsen
8c039d5688 Python: Add more aiohttp view routing tests 2021-06-03 10:55:34 +02:00
Rasmus Wriedt Larsen
1aa222d7cc Python: Add taint-test for class-based view 2021-06-03 10:55:34 +02:00
Rasmus Wriedt Larsen
fb21bc04fa Python: Add taint-steps for yarl.URL 2021-06-03 10:55:34 +02:00
Rasmus Wriedt Larsen
72e6a1489c Python: Add taint-steps for MultiDictProxy 2021-06-03 10:55:34 +02:00
Rasmus Wriedt Larsen
dd131e6bf7 Python: Add taint-step for methods on aiohttp.web.Request 2021-06-03 10:55:34 +02:00
Rasmus Wriedt Larsen
63c7fa0c2c Python: aiohttp match_info should be tainted
Whoops
2021-06-03 10:55:34 +02:00
Rasmus Wriedt Larsen
597a9dfc80 Python: Don't consider has_body tainted
Although it technically is, I think it belong in the section of things
that are unlikely to be exploitable
2021-06-03 10:55:34 +02:00
Rasmus Wriedt Larsen
d953ea47d4 Python: Basic handling of tainted attributes in aiohttp 2021-06-03 10:55:34 +02:00
Rasmus Wriedt Larsen
88158e7414 Python: Add basic model setup for aiohttp.web.Request 2021-06-03 10:55:34 +02:00
Rasmus Wriedt Larsen
2b992a635a Python: Add aiohttp taint tests 2021-06-03 10:55:34 +02:00
Rasmus Wriedt Larsen
3cbb909a3a Python: Add modeling of coroutine routes in aiohttp.web 2021-06-03 10:55:33 +02:00