Commit Graph

7907 Commits

Author SHA1 Message Date
Rasmus Wriedt Larsen
a78f13cb2e Python: Ignore known subclass models 2023-12-19 17:07:02 +01:00
Rasmus Wriedt Larsen
24a3a23c9c Python: Regenerate rest_framework models 2023-12-19 17:07:02 +01:00
Rasmus Wriedt Larsen
3e878f5a0b Python: Model django response subclass relationship 2023-12-19 17:07:02 +01:00
Rasmus Wriedt Larsen
abe6f1639a Python: Add example of models subclassing problem
In reality, we only want to model this as a `rest_framework.response.Response`, since our .qll modeling is more precise for rest-framework responses than if we also modeled it as a basic django http response. (specifically, that default mime-type handling is way different).
2023-12-19 17:07:02 +01:00
Rasmus Wriedt Larsen
5c89c38c92 Python: Add the rest_framework models for demonstration purposes
Although it might be hidden by github UI by default, it could be
interesting for a reviewer to notice the effect changes in the modeling
query has to the results in this file.
2023-12-19 17:07:02 +01:00
Rasmus Wriedt Larsen
ee3319b7b0 Python: Make split/join executable (chmod +x) 2023-12-19 17:07:02 +01:00
Rasmus Wriedt Larsen
cfd3f8938e Python: Highlight split/join subclass files usage 2023-12-19 17:07:02 +01:00
Rasmus Wriedt Larsen
933938d926 Python: Make rest_framework tests runnable again 2023-12-19 17:07:01 +01:00
Rasmus Wriedt Larsen
3e6423a13c Python: Add ability to split and join autogenerated yml files
Verified by joining all files, splitting again, and observing no diff in
git.

(these operations only take a few seconds on my local machine, so
shouldn't be too much of an issue)
2023-12-19 17:07:01 +01:00
Rasmus Wriedt Larsen
f30a3b0aba Python: Script: Improve performance by using C++ impl
these changes took performance for loading and writing all files locally
29.60s to 3.17s

(that is, using `gather_from_existing`)
2023-12-19 17:07:01 +01:00
Rasmus Wriedt Larsen
13c2378b58 Python: Update a few QLdocs 2023-12-19 17:07:01 +01:00
Rasmus Wriedt Larsen
937af906fd Apply suggestions from code review
Co-authored-by: Taus <tausbn@github.com>
2023-12-19 17:07:01 +01:00
Rasmus Wriedt Larsen
0fe29b6a86 Python: Recover subclass finder .expected after cherry picking commits from https://github.com/github/codeql/pull/15030 2023-12-19 17:07:01 +01:00
Rasmus Lerchedahl Petersen
75f9eeb4e9 Python: adjust test expectations
mostly removing of nodes from the graph.
One result lost:
```
check("submodule.submodule_attr", submodule.submodule_attr, "submodule_attr", globals()) #$ MISSING:prints=submodule_attr
```
2023-12-19 17:07:01 +01:00
Rasmus Lerchedahl Petersen
c563c7fbe4 Python: remove control flow nodes
for module entry definitions from the dataflow graph.
2023-12-19 17:07:01 +01:00
Rasmus Wriedt Larsen
e050f2e998 Python: Adjust subclass finder to no ESSA nodes
But the new test results looks very strange indeed!
2023-12-19 17:07:01 +01:00
Rasmus Wriedt Larsen
60b784a919 Python: Don't filter subclass tests away 2023-12-19 17:07:01 +01:00
Rasmus Wriedt Larsen
a9a0216c43 Python: Add change-note 2023-12-19 17:07:01 +01:00
Rasmus Wriedt Larsen
fa3e16adea Python: Refactor taint-sinks meta queries 2023-12-19 17:07:01 +01:00
Rasmus Wriedt Larsen
dc90411809 Python: Don't include docs/ folder 2023-12-08 11:27:53 +01:00
Rasmus Wriedt Larsen
004bb50ef2 Python: Disallow invalid path component 2023-12-08 11:27:53 +01:00
Rasmus Wriedt Larsen
6ce8cd38d8 Python: Disallow examples 2023-12-08 11:27:53 +01:00
Rasmus Wriedt Larsen
6db3b37ed7 Python: Use separate directory for subclass capture models 2023-12-08 11:27:53 +01:00
Rasmus Wriedt Larsen
c4abffe77e Python: Ignore any captured info with tests in it 2023-12-08 11:27:53 +01:00
Rasmus Wriedt Larsen
de41f199fc Python: Allow single file processing 2023-12-08 11:27:53 +01:00
Rasmus Wriedt Larsen
2c0bed174f Python: Remove suspicious packages 2023-12-08 11:27:53 +01:00
Rasmus Wriedt Larsen
0b5c73ae1e Python: Allow any .bqrs file 2023-12-08 11:27:53 +01:00
Rasmus Wriedt Larsen
b24e565128 SubclassFinder: don't include site-packages 2023-12-08 11:27:53 +01:00
Rasmus Wriedt Larsen
aa5eee1eac Python: Revert manual pickle modeling
This reverts commit 62910f0cab525ca4d4901c4c27f6e6b22c3375fc.
This reverts commit 75a8197879ec47094d9b18f3dab7bcc1c1cdba28.

We don't find `kombu.serialization.pickle_load` since we respect
`__all__`. I think that was an attempt to not flood the captured
modeling with useless re-exports, but I think we've ended up doing that
anyway... we should consider to remove that restriction!

see 21d7df29c7/kombu/serialization.py (L29)
2023-12-08 11:27:53 +01:00
Rasmus Wriedt Larsen
f74581ad09 Revert "Python: Model owslib.etree.etree directly"
This reverts commit 1213e786519a11142746fd3a725c874181f3a42b.

By fixing a few bugs in the SubclassFinder + manually running Find.ql on the geonode DB from DCA, I found that the installed version of owslib had both: https://github.com/geopython/OWSLib/blob/0.27.2/owslib/etree.py
2023-12-08 11:27:53 +01:00
Rasmus Wriedt Larsen
6ef9a2b11e Python: Fix problem if import is used
I fixed it in both predicates... I think we might still be able to remove
`newDirectAlias` -- but with it being better, it will allow us to better test if `newImportAlias` actually cover everything we need!
2023-12-08 11:27:52 +01:00
Rasmus Wriedt Larsen
fcdc8102e2 Python: Add test highlight problem is import is used :O 2023-12-08 11:27:52 +01:00
Rasmus Wriedt Larsen
f1fd9b4c7a Python: Fix underlying problem of not using Alias 2023-12-08 11:27:52 +01:00
Rasmus Wriedt Larsen
a956e1f613 Python: Use django View instead of MethodView
Due to the 'only model most specific spec' logic highlighted in previous
commit, I'm changing away from MethodView/View, and use Django view instead.

In practice this shouldn't matter at all, but for writing tests it would
have been a nice fix to only have the "same name but more specific"
logic apply when it's the same _definition_ location. We used to have
this information available, but right now we don't... so instead of
spending a lot of time rewriting the core library, I simply used a
different class :D :O :(
2023-12-08 11:27:52 +01:00
Rasmus Wriedt Larsen
03aa2e27df Python: Explain the funky logic in Find.ql 2023-12-08 11:27:52 +01:00
Rasmus Wriedt Larsen
1f8f6dd0ec Python: Ensure no deps visible in FindSubclass tests 2023-12-08 11:27:52 +01:00
Rasmus Wriedt Larsen
af2d783b38 Python: More examples of things to handle in find-subclass 2023-12-08 11:27:52 +01:00
Taus
fa6aec7ae2 Python: Model owslib.etree.etree directly
Somehow, this alias did not get picked up by the tooling.
2023-12-08 11:27:52 +01:00
Taus
6d40e7e0fc Python: Add extensible modelling for lxml.etree 2023-12-08 11:27:52 +01:00
Taus
5b9d56774b Python: Refactor references to ElementTree
This would probably be better as a module, but I wanted to verify
first that this would yield the right results.
2023-12-08 11:27:52 +01:00
Taus
d29879a844 Python: Model kombu.serialization
More `pickle` wrappers.
2023-12-08 11:27:52 +01:00
Taus
a6dc6f3e42 Python: Add model for flask.restful
Not subclass-related -- just an alias.
2023-12-08 11:27:52 +01:00
Taus
6261a94986 Python: Add cloudpickle model
This one didn't seem to fit into the subclass approach, so I just modeled
it manually.
2023-12-08 11:27:52 +01:00
Taus
5920a8fad6 Python: Add meta-query for "interesting" taint sinks 2023-12-08 11:27:52 +01:00
Taus
43fe9ca31d Python: Model rest_framework.exceptions.APIException
Only models the subclasses of `APIException` that share the same interface as
`APIException` itself with regard to the `getBody` predicate.
2023-12-08 11:27:52 +01:00
Taus
1f66659ff2 Python: Add Django FileField model 2023-12-08 11:27:52 +01:00
Taus
aa3f1cf3e1 Python: extend aiohttp model 2023-12-08 11:27:52 +01:00
Taus
b5bffb2220 Python: Add aioch model 2023-12-08 11:27:52 +01:00
Taus
875fa0b8f0 Python: Add Peewee model 2023-12-08 11:27:52 +01:00
Taus
5e3bdb8701 Python: Add Pydantic model 2023-12-08 11:27:51 +01:00