Commit Graph

3025 Commits

Author SHA1 Message Date
Calum Grant
649e313cb5 Add severity scores 2021-04-20 20:51:29 +01:00
Taus
76700d17d6 Merge pull request #5684 from RasmusWL/flask-more-taint-tests
Python: Add taint tests for .get() in flask
2021-04-20 14:08:08 +02:00
yoff
118840dad4 Merge pull request #5690 from tausbn/python-disallow-post-update-nodes-as-local-source-nodes
Python: Disallow `PostUpdateNode` as `LocalSourceNode`
2021-04-19 06:56:11 +02:00
Rasmus Wriedt Larsen
3c8ea167c4 Merge pull request #5668 from tausbn/python-use-api-graphs-in-fabric
Python: Use API graphs in Fabric model
2021-04-16 14:27:55 +02:00
Rasmus Wriedt Larsen
6ed1016bb8 Merge pull request #5669 from tausbn/python-use-api-graphs-for-invoke
Python: Use API graphs for Invoke
2021-04-16 14:27:19 +02:00
Taus
92b4eb7f02 Python: Cleanup and more explanation
Goes into some detail about the intended semantics of local source nodes
and `flowsTo`.
2021-04-16 11:54:20 +00:00
Taus
5c79ad2412 Python: Apply suggestions from code review
Co-authored-by: Rasmus Wriedt Larsen <rasmuswriedtlarsen@gmail.com>
2021-04-16 11:38:29 +02:00
Taus
af0c32c01d Python: Apply suggestions from code review
Co-authored-by: Rasmus Wriedt Larsen <rasmuswriedtlarsen@gmail.com>
2021-04-16 11:35:12 +02:00
Taus
451d36dc97 Python: Allow _some_ PostUpdateNodes
Specifically, allow the ones arising from calls, but not reads or
writes. This should fix the tests.
2021-04-15 21:26:12 +00:00
Taus
c9c8259ed0 Python: Disallow PostUpdateNode as LocalSourceNode
Previously, in cases like

```python
def foo(x):
    x.bar()
    x.baz()
    x.quux()
```

we would have flow from the first `x` to each use _and_ flow from the
post-update node for each method call to each subsequent use, and all
of these would be `LocalSourceNode`s. For large functions with the above
pattern, this would lead to a quadratic blowup in `hasLocalSource`.

With this commit, only the first of these will count as a
`LocalSourceNode`, and the blowup disappears.
2021-04-15 17:56:14 +00:00
Rasmus Wriedt Larsen
b359205d17 Python: Add taint tests for .get() in flask 2021-04-15 14:53:44 +02:00
yoff
447f339857 Merge pull request #5641 from tausbn/python-use-localsourcenode-in-typetrackers
Python: Use API graphs in PEP249 support
2021-04-14 15:39:49 +02:00
Taus
54c79bff74 Merge pull request #5666 from RasmusWL/django-refactor
Python: Refactoring and exposing of Django views/fields/forms
2021-04-14 13:07:20 +02:00
Rasmus Wriedt Larsen
44d2bf42d7 Merge pull request #5671 from tausbn/python-use-api-graphs-in-werkzeug
Python: Use API graphs in Werkzeug
2021-04-14 12:57:58 +02:00
Rasmus Wriedt Larsen
9de8085571 Merge pull request #5665 from tausbn/python-use-api-graphs-in-tornado
Python: Tornado cleanup using API graphs
2021-04-14 10:22:21 +02:00
Rasmus Wriedt Larsen
2d0c9b6bf2 Merge pull request #5670 from tausbn/python-use-api-graphs-in-dill
Python: Use API graphs in Dill model
2021-04-14 10:08:02 +02:00
Rasmus Wriedt Larsen
55723618a9 Python: Apply suggestions from code review
Co-authored-by: Taus <tausbn@github.com>
2021-04-14 10:05:50 +02:00
Taus
981c5deb57 Merge pull request #5639 from tausbn/python-api-graphs-missing-builtins
Python: Add missing builtins to `API::builtin`
2021-04-13 21:27:52 +02:00
Taus
a6bb9ebb9f Python: Re-introduce abstract toString
This seems like the easier solution in the short run.
2021-04-13 16:08:41 +00:00
Taus
079c7e089d Python: Autoformat 2021-04-13 16:05:45 +00:00
Taus
5f7d3d0d36 Python: Use API graphs in Werkzeug 2021-04-13 15:57:21 +00:00
Taus
2890fe6d61 Python: Use API graphs in Dill model
If only all rewrites were this smooth...
2021-04-13 15:26:54 +00:00
Taus
7ed09904b4 Python: Use API graphs for Invoke
A few stragglers remain, as they are modelling the use of decorators.

They will be dealt with at a later date.
2021-04-13 15:21:19 +00:00
Taus
7f131c1f35 Python: Get rid of _attr predicates 2021-04-13 14:55:44 +00:00
Taus
1008411594 Python: Use API graphs in Fabric model 2021-04-13 14:49:44 +00:00
Taus
1a4845f417 Python: Restrict types a bit
The `CallCfgNode` restrictions are familiar and useful.

Restricting `InstanceSource` to extend `LocalSourceNode` is novel, but I
think it makes sense. It will act as a good reminder to anyone extending
`InstanceSource` that the node in question is a `LocalSourceNode`, which
will be enforced by the return type of the internal type tracker anyway.
2021-04-13 12:28:38 +00:00
Taus
f93b68d4dc Python: Get rid of _attr methods 2021-04-13 12:25:38 +00:00
Taus
98d936d8b3 Python: Tornado cleanup using API graphs
I wasn't able to roll out API graphs as widely in Tornado as I had
hoped, since we're lacking the "def" part. This means most of the
`InstanceSource` machinery will have to stay.
2021-04-13 12:25:38 +00:00
Taus
fda750ef26 Merge pull request #5642 from tausbn/python-use-api-graphs-in-stdlib
Python: Use API graphs in `Stdlib.qll`
2021-04-12 18:05:38 +02:00
Taus
6d4ddc0329 Merge pull request #5614 from tausbn/python-allow-absolute-imports-from-source-directory
Python: Allow absolute imports from source directory
2021-04-12 18:02:00 +02:00
CodeQL CI
bc56d16c18 Merge pull request #5485 from RasmusWL/django-queryset-chains
Approved by tausbn
2021-04-12 08:49:31 -07:00
Tom Hvitved
7d2a60e910 Merge pull request #5640 from hvitved/dataflow/path-step-perf
Data flow: Prevent bad join-order in `pathStep`
2021-04-12 14:40:46 +02:00
Rasmus Wriedt Larsen
364d48948f Merge pull request #3810 from dilanbhalla/syntaxpython
Python: Function/Class Naming Convention (Syntax)
2021-04-12 10:42:17 +02:00
Taus
10be2735ec Python: Get rid of _attr predicates
Also changes all `CfgNode`s representing calls to `CallCfgNode`s.
2021-04-10 12:12:18 +00:00
Taus
720fbaf301 Python: Fix test error.
Somehow, having to type "Node" all day long made me turn "json" into
"node"...

Also removes some bits that weren't needed after all.
2021-04-09 19:04:49 +00:00
Taus
cc4827600b Python: Use API graphs in Stdlib.qll
Eliminates _almost_ all of the bespoke type trackers found here. The
ones that remain do not fit easily inside the framework of API graphs
(at least, not yet), and I did not see any easy ways to clean them up.
They have, however, been rewritten to use `LocalSourceNode` internally,
which was the primary goal of this exercise.

I'm sure we could also clean up many of the inner modules given the more
lean presentation we have now, but this can wait for a different PR.
2021-04-09 17:11:47 +00:00
Tom Hvitved
f130616369 Data flow: Make getLocalCc private again 2021-04-09 16:22:58 +02:00
Taus
d2b874f217 Python: Use API graphs in PEP249 support
Because the replacement extension point now extends `API::Node`, I
modified the `toString` method of the latter to have an empty body.
The alternative would be to require everyone to provide a `toString`
predicate for their extensions, but seeing as these will usually be
pointing to already existing API graph nodes, this seems silly.

(This may be the reason why the equivalent method in the JS libs has
such an implementation.)
2021-04-09 14:19:00 +00:00
Tom Hvitved
6874b8d4b3 Data flow: Prevent bad join-order in pathStep 2021-04-09 14:24:47 +02:00
Taus
affdedd840 Python: Add missing builtins to API::builtin
We were missing out on `None`, `True`, and `False` as these do not
appear as actual attributes of the `builtins` module in Python 3
(because they are elevated to the status of keywords there)

The simple solution, then, is to just always include them directly.
2021-04-09 12:02:07 +00:00
Dilan
d73ba13b28 autoformat fix 2021-04-08 11:41:58 -07:00
Rasmus Wriedt Larsen
c738f387b1 Merge pull request #5624 from tausbn/python-make-callcfgnode-a-localsourcenode
Python: Improve `CallCfgNode` interface
2021-04-08 13:38:24 +02:00
Taus
cf5f760ecd Merge pull request #5582 from RasmusWL/all-tuple
Python: Add support for `__all__` assigned to tuple
2021-04-08 13:03:27 +02:00
Rasmus Wriedt Larsen
83477439a1 Python: Make django views/fields/forms class modeling extensible
This also requires that we make this part of the modeling public, which I guess
is step we want to take eventually anyway!

I'm not quite sure whether the modules `Django::Views` and `Django::Forms` are
actually helpful, or whether we should just have their modules available as
`Django::View`, `Django::Form`, and `Django::Field`...
2021-04-08 12:45:37 +02:00
Rasmus Wriedt Larsen
b7483a5394 Python: Add modeledSubclassRef for Django views/fields/forms 2021-04-08 12:45:36 +02:00
Rasmus Wriedt Larsen
322bdcb703 Python: Port Django view modeling to API graphs 2021-04-08 12:45:35 +02:00
Rasmus Wriedt Larsen
8ce5c46e05 Python: Minor refactor
modName/clsName _is_ shorter, but also looks way worse :D
2021-04-08 12:45:34 +02:00
Dilan
675de07c3e autoformat ql 2021-04-07 15:04:18 -07:00
Taus
903f364dab Python: Improve CallCfgNode interface
Call nodes are always local sources (specifically sources of the return
value of the call), and so inheriting from `LocalSourceNode` will have
no effect on results, but _should_ make it a bit more smooth to use the
API.
2021-04-07 13:31:12 +00:00
Taus
6c69c1aeeb Python: Minor cleanup 2021-04-07 10:47:21 +00:00