Commit Graph

4384 Commits

Author SHA1 Message Date
Rasmus Lerchedahl Petersen
d23a8ad016 Python: elide test output 2021-02-21 13:12:54 +01:00
Rasmus Lerchedahl Petersen
46faba69ff Python: Fix for-iteration of tuples 2021-02-21 12:41:16 +01:00
Rasmus Lerchedahl Petersen
0aecf33fe6 Python: test iteration through overflow parameters
These are in a tuple, so the for-step does not fire
2021-02-21 12:33:04 +01:00
Rasmus Wriedt Larsen
40c592ab85 Python: Introduce DataFlowOnlyInternalUse to avoid re-evaluation 2021-02-19 15:29:23 +01:00
Rasmus Wriedt Larsen
d084261a79 Python: Ignore weak key-sizes from test-code in weak-crypto-key
From looking at old results on LGTM.com, this was quite common (and those alerts
doesn't really provide value).
2021-02-19 15:04:41 +01:00
Rasmus Wriedt Larsen
bfc8ead667 Python: Add example of test-code with weak crypto key 2021-02-19 15:04:14 +01:00
Rasmus Wriedt Larsen
dfa223ac6a Python: Better IntegerLiteral tracking for weak crypto key 2021-02-19 15:03:50 +01:00
Rasmus Wriedt Larsen
a6583345ba Python: Add weak crypto key example through function call
We used to handle this, but no more :(

Adding this example was inspired by looking at results differences
2021-02-19 15:03:49 +01:00
Rasmus Wriedt Larsen
37f0d5a28a Python: Make KeyGeneration range member overrides final
This was the result of an internal dicussion we had about this some time ago.
2021-02-19 15:03:49 +01:00
Rasmus Wriedt Larsen
2a8f720bc6 Python: Port cryptodome models to use API graphs 2021-02-19 15:03:48 +01:00
Rasmus Wriedt Larsen
1eabfbd0e4 Python: Port cryptography models to use API graphs (mostly) 2021-02-19 15:03:47 +01:00
Rasmus Wriedt Larsen
bfbaa85272 Python: Add test of public_key method with cryptodome
Added in 3.10 release https://github.com/Legrandin/pycryptodome/blob/master/Changelog.rst#3100-6-february-2021
2021-02-19 15:03:47 +01:00
Rasmus Wriedt Larsen
8d3170bcb4 Python: Fix bad join in crypto models 2021-02-19 15:03:46 +01:00
Rasmus Wriedt Larsen
32d0790500 Python: Use camelCase for RSA/DSA/ECC
after asking around, this seems to be the right approach
2021-02-19 15:03:45 +01:00
Rasmus Wriedt Larsen
0e9a54e9a9 Python: Rename WeakCrypto to WeakCryptoKey
Since WeakCrypto always makes me think that it's about all weak crypto (like
using MD5, or completely broken ciphers such as ARC4 ro DES) and not just about
weak key generation.
2021-02-19 15:03:44 +01:00
Rasmus Wriedt Larsen
46ad611d57 Python: Port py/weak-crypto-key to use type-tracking
instead of points-to.

Looking at query results also made me realize I didn't supply a very good
"origin" for ECC in cryptography package, so I improved that 👍 -- maybe that
sohuld have been split into multiple commits... too late :(
2021-02-19 15:03:43 +01:00
Rasmus Wriedt Larsen
2429c6c450 Python: Rewrite py/weak-crypto-key tests
* Removed backend arugment that is not required
* Added DSA constants (they are just accidentially the same as RSA right now)
* Removed FakeWeakEllipticCurve and used a real weak elliptic curve instead
2021-02-19 13:59:19 +01:00
Rasmus Wriedt Larsen
d5ff477644 Python: Add modeling for pycryptodome PyPI package 2021-02-19 13:59:18 +01:00
Rasmus Wriedt Larsen
6e4c627209 Python: Add modeling for pycryptodomex PyPI package 2021-02-19 13:59:17 +01:00
Rasmus Wriedt Larsen
bd40965afe Python: Add modeling for cryptography PyPI package 2021-02-19 13:59:17 +01:00
Rasmus Wriedt Larsen
1bf9f7d135 Python: Add missing annotations to new crypto tests 2021-02-19 13:59:16 +01:00
Rasmus Wriedt Larsen
11cd0dbbc0 Python: Add concepts for public-key generation
I did spend some time to figure out how to best write `minimumSecureKeySize`
predicate. I wanted to write once and for all the recommended sizes for each
cryptosystem.

I considered making the predicate such as

```codeql
int minimumSecureKeySize() {
    this.getName() = "RSA" and result = 2048
    or
    this.getName() = "DSA" and result = 2048
    or
    this.getName() = "ECC" and result = 244
}
```

but then it would be impossible to add a new model without also being able to
modify the body of this predicate -- which seems like a bad way to start off a
brand new way of modeling things.

So I considered if we could add it to the non-range class, such as

```codeql
class RSAKeyGeneration extends KeyGeneration {
  RSAKeyGeneration() { this.getName() = "RSA" }

  override int minimumSecureKeySize() { result = 2048 }
}
```

This has the major problem that when you're writing the models for a new
API (and therefore extending KeyGeneration::Range), there is no way for you to
see that you need to take this extra step :| (also problem about how we should
define `minimumSecureKeySize` on `KeyGeneration` class then, since if we make it
abstract, we effectively disable the ability to refine `KeyGeneration` since any
subclass must provide an implementation.)

So, therefore I ended up with this solution ;)
2021-02-19 13:59:16 +01:00
Rasmus Wriedt Larsen
4ab61bb088 Python: Add a few tests for crypto frameworks
Tests working can be verified by running

```
ls ql/python/ql/test/experimental/library-tests/frameworks/crypto*/*.py | xargs -L1 sh -c 'python $0 || exit 255'
```
2021-02-19 13:26:45 +01:00
Rasmus Wriedt Larsen
a19da54c9e Python: Exclude flask.request imports as RemoteFlowSource
When I changed the taint modeling in 19b7ea8d85, that obviously also means that
some of the related locations for alerts will change. So that's why all the
examples needs to be updated.

Besides this, I had to fix a minor problem with having too many alerts. If
running a query agaisnt code like in the example below, there would be 3 alerts,
2 of them originating from the import.

```
from flask import Flask, request
app = Flask(__name__)
@app.route("/route")
def route():
    SINK(request.args.get['input'])
```

The 2 import sources where:

- ControlFlowNode for ImportMember
- GSSA Variable request

I removed these from being a RemoteFlowSource, as seen in the diff.

I considered restricting `FlaskRequestSource` so it only extends
`DataFlow::CfgNode` (and make the logic a bit simpler), but I wasn't actually
sure if that was safe to do or not... If you know, please let me know :)
2021-02-19 12:22:05 +01:00
Rasmus Wriedt Larsen
9798e60d0f Merge pull request #5203 from tausbn/python-add-typebacktrackers
Python: Add `TypeBackTracker`
2021-02-19 12:02:53 +01:00
Rasmus Wriedt Larsen
cc72fc82f0 Merge branch 'main' into flask-clean-models 2021-02-18 16:08:18 +01:00
Rasmus Wriedt Larsen
9a42f2fb26 Python: Add missing QLdoc for FlaskMethodViewClass 2021-02-18 16:07:47 +01:00
Taus
e9cbdc4ad3 Update python/ql/src/semmle/python/dataflow/new/TypeTracker.qll
Co-authored-by: Rasmus Wriedt Larsen <rasmuswriedtlarsen@gmail.com>
2021-02-18 15:53:15 +01:00
Rasmus Wriedt Larsen
bb2613b02b Python: Flask model now ready to be publicly exposed
With a single call-out for a member-predicate that is only for internal use.
2021-02-18 15:36:30 +01:00
Rasmus Wriedt Larsen
35876f1939 Python: Re-introduce Response::instance() in flask model
We don't actually need it for anything right now, but I have plans for the
future where would need it.

Although it would be nice to have it as an `API::Node`, and we could re-write
implementations so we could provide it in this instance, I'm not convinced we
can do that in general right now.

For example, if <n'th> parameter of a function has to be modeled as belonging to
a certain type, I don't see any way to specify that as an API::Node.

For me, that's ok. Until we _can_ specify things like this as API::Nodes in the
future, I would like to keep things consistent, and use `DataFlow::Node` as the
result type.
2021-02-18 15:22:16 +01:00
Rasmus Wriedt Larsen
141e2665ea Python: Align ViewClass naming with django
Just as part of tyding up
2021-02-18 15:10:21 +01:00
Rasmus Wriedt Larsen
19b7ea8d85 Python: Align flask taint modeling with rest of code
This was a good time to do this, so we don't have 2 different ways of doing the
same thing.

I needed to do this to figure out if we should expose
`API::moduleImport("flask").getMember("request")` in a helper predicate or
not. I think I ended up using more refenreces to this in the end. Although it's
not unreasonable to let someone do this themselves, I also think it's reasonable
that we provide a helper predicate for this.
2021-02-18 15:04:07 +01:00
Rasmus Wriedt Larsen
ba61099172 Python: flask.make_response as InstanceSource of flask.Response 2021-02-18 12:52:59 +01:00
Rasmus Wriedt Larsen
e3d530dbbc Python: Flask: Remove more type-tracking helper predicates 2021-02-18 12:13:47 +01:00
Rasmus Wriedt Larsen
e4ea5f25dc Python: Flask: Moderize app and blueprint 2021-02-18 12:09:37 +01:00
Rasmus Wriedt Larsen
7de488b987 Python: Flask: Moderize views 2021-02-18 12:05:56 +01:00
CodeQL CI
d94f20ff2f Merge pull request #5194 from RasmusWL/type-tracking-snippets
Approved by tausbn
2021-02-18 02:13:21 -08:00
Taus Brock-Nannestad
23e9785efd Python: Add missing QLDoc 2021-02-17 21:38:48 +01:00
Taus Brock-Nannestad
99f3a61f61 Python: Add TypeBackTracker
This is a fairly straight port of the JS equivalent. Also adds
`Node::getALocalSourceNode` which seems like it might come in handy.
2021-02-17 21:14:20 +01:00
Rasmus Wriedt Larsen
4880350420 Python: Add a single missing QLDoc 2021-02-17 16:33:12 +01:00
Rasmus Wriedt Larsen
7afe3972d8 Revert "Merge pull request #5171 from RasmusWL/restructure-queries"
This reverts commit 8caafb3710, reversing
changes made to ec79094957.
2021-02-17 16:32:53 +01:00
Rasmus Wriedt Larsen
63a09fccdd Python: Use this = <...>.getACall() for DataFlow::CallCfgNode
I think this reads a bit cleaner
2021-02-17 14:43:48 +01:00
Taus
ce1d8ded22 Merge pull request #5192 from RasmusWL/framework-for-routed-params
Python: Expose framework identifier for route-setup and request handler
2021-02-17 13:19:43 +01:00
Rasmus Wriedt Larsen
a4de88d39c Python: Update type-tracking snippet
based on what I learned in https://github.com/github/codeql/pull/5184
2021-02-17 13:13:25 +01:00
Rasmus Wriedt Larsen
eee49cde85 Merge pull request #5184 from tausbn/python-move-type-tracker-tests-to-source-nodes
Python: Use `LocalSourceNode` in type tracker tests
2021-02-17 12:13:47 +01:00
Taus
8caafb3710 Merge pull request #5171 from RasmusWL/restructure-queries
Python: Restructure query file layout
2021-02-17 12:09:32 +01:00
Rasmus Wriedt Larsen
cf9ad0cdc5 Python: Move ExternalAPI queries back under Security
This was raised as a question at review, and I don't really have a good enough
argument for moving it under POI. At the end of the day, they are _security_
related enough I guess :)
2021-02-17 11:29:33 +01:00
Rasmus Wriedt Larsen
dec026a820 Python: Fix security qlref to have single empty line 2021-02-17 11:26:02 +01:00
Rasmus Wriedt Larsen
1adb510578 Python: Add a single missing QLDoc 2021-02-17 11:24:11 +01:00
Rasmus Wriedt Larsen
2927d888cf Python: Fix location of PathInjection tests 2021-02-17 11:20:00 +01:00