Commit Graph

4639 Commits

Author SHA1 Message Date
yoff
e3b3825ab0 Merge pull request #5151 from RasmusWL/django-get-redirect-url
Python: Model get_redirect_url in django
2021-02-25 23:07:33 +01:00
Rasmus Wriedt Larsen
81b29316e1 Merge pull request #4737 from yoff/python-dataflow-add-cast-nodes
Python: Force read- and store steps to add nodes.
2021-02-25 14:28:54 +01:00
Taus
01d581ecf3 Merge pull request #5250 from tausbn/python-port-re-security-queries
Python: Port URL sanitisation queries to API graphs
2021-02-25 13:13:55 +01:00
yoff
f15084254b Add comment explaining tacky nature of code 2021-02-25 11:49:57 +01:00
Rasmus Lerchedahl Petersen
5b51a3461d Python: Force read- and store steps to add nodes.
This gives muche nicer path explanations on some snapshots.
It is achieved by making stepped-to nodes `CastNode`s.
This seems somewhat reasonable as types then to change, when we move
between content and container.
We could probably refine it, though.
2021-02-25 11:49:57 +01:00
Rasmus Wriedt Larsen
4610b1b392 Pyhton: Use type back-tracking for keysize on key-generation
Internal evaluation showed that this didn't perform better than normal (forward)
type-tracking, but it feels more like the right approach.
2021-02-25 11:31:00 +01:00
Rasmus Wriedt Larsen
c195c64982 Python: Use type-tracking for integer literal tracking
Like we've done for pretty much everything else. An experiment to see what this
means for query performance.
2021-02-25 11:30:56 +01:00
Rasmus Wriedt Larsen
27987717dc Merge branch 'main' into crypto 2021-02-25 11:30:32 +01:00
Rasmus Lerchedahl Petersen
41743b6afa Python: restrict to caught exceptions
also modernise code
2021-02-25 07:53:35 +01:00
Rasmus Lerchedahl Petersen
24b51e8851 Merge branch 'main' of github.com:github/codeql into python-port-stacktrace-exosure 2021-02-25 07:24:41 +01:00
Rasmus Lerchedahl Petersen
76f080978a Python: Add missing QLDoc 2021-02-24 23:35:44 +01:00
Rasmus Lerchedahl Petersen
192988077e Python: Move <ul> outside of <p> 2021-02-24 23:28:13 +01:00
Rasmus Lerchedahl Petersen
10657160bc Python: Improve qlhelp according to review 2021-02-24 22:02:16 +01:00
CodeQL CI
bf66bdbb95 Merge pull request #5253 from RasmusWL/no-getAnArg
Approved by tausbn
2021-02-24 06:34:31 -08:00
Rasmus Wriedt Larsen
d05a8b8c46 Python: Remove getAnArg in DataFlow::CallCfgNode
Until we've had further discussion on what is the right approach to
naming (internal discussion in https://github.com/github/codeql-python-team/issues/95)
2021-02-24 14:58:48 +01:00
Rasmus Wriedt Larsen
a6e5ec2e09 Python: Port py/flask-debug 2021-02-24 11:37:25 +01:00
yoff
8262f0343b Merge pull request #5208 from RasmusWL/flask-clean-models
Python: Cleanup Flask models now that we have API graphs
2021-02-24 10:36:30 +01:00
Rasmus Wriedt Larsen
5bb4a1a45a Python: Use explicit argument specification instead of getAnArg
I've seen quite a few places where `getAnArg` leads to wrong behavior, and I
generally just don't like it.
2021-02-24 10:19:34 +01:00
Taus Brock-Nannestad
e77c1059a3 Python: Use source nodes and prevent bad join order 2021-02-24 10:18:54 +01:00
Taus Brock-Nannestad
cac6c4acc9 Python: Add deprecation notice to mode_from_mode_object 2021-02-24 10:18:21 +01:00
yoff
c3d2001e85 Merge pull request #5251 from tausbn/python-port-missing-host-key-validation-query
Python: Port missing host key validation query
2021-02-24 08:43:52 +01:00
Taus Brock-Nannestad
2942a11a69 Python: Import API graphs privately 2021-02-23 22:45:39 +01:00
Taus Brock-Nannestad
f241dbabab Python: Clean up query a bit 2021-02-23 22:33:18 +01:00
Taus Brock-Nannestad
002d0fe565 Python: Port missing host key query 2021-02-23 22:26:03 +01:00
Taus Brock-Nannestad
e812eb777d Python: Port URL sanitisation queries to API graphs
Really, this boils down to "Port `re` library model to use API graphs
instead of points-to", which is what this PR actually does.

Instead of using points-to to track flags, we use a type tracker. To
handle multiple flags at the same time, we add additional flow from

`x` to `x | y` and `y | x`

and, as an added bonus, the above with `+` instead of `|`, neatly
fixing https://github.com/github/codeql/issues/4707

I had to modify the `Qualified.ql` test slightly, as it now had a
result stemming from the standard library (in `warnings.py`) that
points-to previously ignored.

It might be possible to implement this as a type tracker on
`LocalSourceNode`s, but with the added steps for the above operations,
this was not obvious to me, and so I opted for the simpler
"`smallstep`" variant.
2021-02-23 22:02:35 +01:00
Rasmus Wriedt Larsen
358ade67e5 Merge pull request #5248 from tausbn/python-port-insecure-temporary-file
Python: Port `py/insecure-temporary-file`
2021-02-23 21:37:59 +01:00
Taus Brock-Nannestad
b8ce5e969e Python: Port py/insecure-temporary-file 2021-02-23 20:02:22 +01:00
yoff
9eed17f647 Merge pull request #5152 from RasmusWL/improve-pyyaml-support
Python: Improve pyyaml support
2021-02-23 19:58:04 +01:00
Rasmus Lerchedahl Petersen
b28544da9c Python: Port insecure default protocol
- use API graphs
- update .qlhelp-file
- limit to versions below 3.4
- move tests to its own directory to only test on old version
2021-02-23 19:41:36 +01:00
Rasmus Wriedt Larsen
a09f8c4b4a Python: Port bind-to-all-interfaces to type-tracking 2021-02-23 16:01:24 +01:00
Rasmus Wriedt Larsen
42de872bfa Python: Add INTERNAL annotation to Response::InstanceSource
Since we need to reserve the flexibility to change this setup within the next
few months, we don't want to commit to keeping this extension point around for
the 12 months that the normal API deprecation cycle requires.
2021-02-23 15:10:58 +01:00
Rasmus Wriedt Larsen
8ebedf26d2 Python: Add comment for MethodView being known subclass 2021-02-23 15:08:07 +01:00
Rasmus Wriedt Larsen
e160c855ad Merge pull request #5233 from yoff/python-for-tuple-iteration
Python: `for`-iteration of tuples
2021-02-22 15:28:13 +01:00
Rasmus Lerchedahl Petersen
46faba69ff Python: Fix for-iteration of tuples 2021-02-21 12:41:16 +01:00
Rasmus Wriedt Larsen
40c592ab85 Python: Introduce DataFlowOnlyInternalUse to avoid re-evaluation 2021-02-19 15:29:23 +01:00
Rasmus Wriedt Larsen
d084261a79 Python: Ignore weak key-sizes from test-code in weak-crypto-key
From looking at old results on LGTM.com, this was quite common (and those alerts
doesn't really provide value).
2021-02-19 15:04:41 +01:00
Rasmus Wriedt Larsen
dfa223ac6a Python: Better IntegerLiteral tracking for weak crypto key 2021-02-19 15:03:50 +01:00
Rasmus Wriedt Larsen
37f0d5a28a Python: Make KeyGeneration range member overrides final
This was the result of an internal dicussion we had about this some time ago.
2021-02-19 15:03:49 +01:00
Rasmus Wriedt Larsen
2a8f720bc6 Python: Port cryptodome models to use API graphs 2021-02-19 15:03:48 +01:00
Rasmus Wriedt Larsen
1eabfbd0e4 Python: Port cryptography models to use API graphs (mostly) 2021-02-19 15:03:47 +01:00
Rasmus Wriedt Larsen
8d3170bcb4 Python: Fix bad join in crypto models 2021-02-19 15:03:46 +01:00
Rasmus Wriedt Larsen
32d0790500 Python: Use camelCase for RSA/DSA/ECC
after asking around, this seems to be the right approach
2021-02-19 15:03:45 +01:00
Rasmus Wriedt Larsen
0e9a54e9a9 Python: Rename WeakCrypto to WeakCryptoKey
Since WeakCrypto always makes me think that it's about all weak crypto (like
using MD5, or completely broken ciphers such as ARC4 ro DES) and not just about
weak key generation.
2021-02-19 15:03:44 +01:00
Rasmus Wriedt Larsen
46ad611d57 Python: Port py/weak-crypto-key to use type-tracking
instead of points-to.

Looking at query results also made me realize I didn't supply a very good
"origin" for ECC in cryptography package, so I improved that 👍 -- maybe that
sohuld have been split into multiple commits... too late :(
2021-02-19 15:03:43 +01:00
Rasmus Wriedt Larsen
d5ff477644 Python: Add modeling for pycryptodome PyPI package 2021-02-19 13:59:18 +01:00
Rasmus Wriedt Larsen
6e4c627209 Python: Add modeling for pycryptodomex PyPI package 2021-02-19 13:59:17 +01:00
Rasmus Wriedt Larsen
bd40965afe Python: Add modeling for cryptography PyPI package 2021-02-19 13:59:17 +01:00
Rasmus Wriedt Larsen
11cd0dbbc0 Python: Add concepts for public-key generation
I did spend some time to figure out how to best write `minimumSecureKeySize`
predicate. I wanted to write once and for all the recommended sizes for each
cryptosystem.

I considered making the predicate such as

```codeql
int minimumSecureKeySize() {
    this.getName() = "RSA" and result = 2048
    or
    this.getName() = "DSA" and result = 2048
    or
    this.getName() = "ECC" and result = 244
}
```

but then it would be impossible to add a new model without also being able to
modify the body of this predicate -- which seems like a bad way to start off a
brand new way of modeling things.

So I considered if we could add it to the non-range class, such as

```codeql
class RSAKeyGeneration extends KeyGeneration {
  RSAKeyGeneration() { this.getName() = "RSA" }

  override int minimumSecureKeySize() { result = 2048 }
}
```

This has the major problem that when you're writing the models for a new
API (and therefore extending KeyGeneration::Range), there is no way for you to
see that you need to take this extra step :| (also problem about how we should
define `minimumSecureKeySize` on `KeyGeneration` class then, since if we make it
abstract, we effectively disable the ability to refine `KeyGeneration` since any
subclass must provide an implementation.)

So, therefore I ended up with this solution ;)
2021-02-19 13:59:16 +01:00
Rasmus Wriedt Larsen
a19da54c9e Python: Exclude flask.request imports as RemoteFlowSource
When I changed the taint modeling in 19b7ea8d85, that obviously also means that
some of the related locations for alerts will change. So that's why all the
examples needs to be updated.

Besides this, I had to fix a minor problem with having too many alerts. If
running a query agaisnt code like in the example below, there would be 3 alerts,
2 of them originating from the import.

```
from flask import Flask, request
app = Flask(__name__)
@app.route("/route")
def route():
    SINK(request.args.get['input'])
```

The 2 import sources where:

- ControlFlowNode for ImportMember
- GSSA Variable request

I removed these from being a RemoteFlowSource, as seen in the diff.

I considered restricting `FlaskRequestSource` so it only extends
`DataFlow::CfgNode` (and make the logic a bit simpler), but I wasn't actually
sure if that was safe to do or not... If you know, please let me know :)
2021-02-19 12:22:05 +01:00
Rasmus Wriedt Larsen
9798e60d0f Merge pull request #5203 from tausbn/python-add-typebacktrackers
Python: Add `TypeBackTracker`
2021-02-19 12:02:53 +01:00