Commit Graph

715 Commits

Author SHA1 Message Date
Rasmus Wriedt Larsen
1f5e52e822 Python: Cleanup "first" type-tracking predicate to be private
Since it's exposed nicely in the version that doesn't have a
`DataFlow::TypeTracker` parameter, these should be private.

Also found one instance where I had accidentially used DataFlow::Node instead of
LocalSourceNode
2021-03-23 16:40:56 +01:00
Rasmus Wriedt Larsen
d9079e34e3 Python: Move framework tests out of experimental
Since they are not experimental anymore 😄
2021-03-19 15:51:54 +01:00
yoff
37036b5e76 Merge pull request #5437 from RasmusWL/small-pyyaml-improvements
Python: Small PyYAML improvements
2021-03-19 11:15:49 +01:00
Rasmus Wriedt Larsen
7543f10593 Python: Reorganize PyYAML tests a bit 2021-03-19 09:53:25 +01:00
yoff
746e9948b0 Merge pull request #5075 from RasmusWL/crypto
Python: Port py/weak-crypto-key to use type-tracking
2021-03-18 20:53:28 +01:00
Rasmus Wriedt Larsen
42b2c3ed52 Python: Model C-based loaders for PyYAML
Not really that important. But easy to do while I was working on this library.
2021-03-18 11:55:01 +01:00
Rasmus Wriedt Larsen
54e6f51512 Python: Add example of C-based PyYAML loaders
```
In [6]: yaml.load("!!python/object/new:os.system [echo EXPLOIT!]", yaml.CLoader)
EXPLOIT!
Out[6]: 0
```
2021-03-18 11:50:59 +01:00
Rasmus Wriedt Larsen
14e9bda5de Python: Refactor PyYAML tests a bit 2021-03-18 11:39:47 +01:00
Rasmus Wriedt Larsen
d52d328587 Python: Use new API::builtin in stdlib modeling 2021-03-17 15:50:01 +01:00
Taus Brock-Nannestad
978200e2ad Python: Distinguish between Python 2 and 3
Also moves the filtering on `name` to before the big disjunction in
`MkModuleImport`.
2021-03-12 12:35:23 +01:00
Taus Brock-Nannestad
c7b2b719cf Python: Support builtins in API graphs 2021-03-11 23:03:18 +01:00
Taus
d2ed216670 Merge pull request #5280 from RasmusWL/highlight-tornado-uri
Python: Highlight how request.uri works in Tornado
2021-03-04 23:31:02 +01:00
Rasmus Wriedt Larsen
a387496832 Python: Highlight how request.uri works in Tornado 2021-02-26 16:23:21 +01:00
yoff
e3b3825ab0 Merge pull request #5151 from RasmusWL/django-get-redirect-url
Python: Model get_redirect_url in django
2021-02-25 23:07:33 +01:00
Rasmus Lerchedahl Petersen
5b51a3461d Python: Force read- and store steps to add nodes.
This gives muche nicer path explanations on some snapshots.
It is achieved by making stepped-to nodes `CastNode`s.
This seems somewhat reasonable as types then to change, when we move
between content and container.
We could probably refine it, though.
2021-02-25 11:49:57 +01:00
Rasmus Wriedt Larsen
27987717dc Merge branch 'main' into crypto 2021-02-25 11:30:32 +01:00
yoff
9eed17f647 Merge pull request #5152 from RasmusWL/improve-pyyaml-support
Python: Improve pyyaml support
2021-02-23 19:58:04 +01:00
Taus
53711dc82f Merge pull request #5238 from RasmusWL/no-flow-default-value
Python: Highlight missing flow from default value in functions
2021-02-23 13:27:41 +01:00
Rasmus Wriedt Larsen
e160c855ad Merge pull request #5233 from yoff/python-for-tuple-iteration
Python: `for`-iteration of tuples
2021-02-22 15:28:13 +01:00
Rasmus Wriedt Larsen
5249b54a9b Python: Highlight missing flow from default value in functions
Although it is becoming non-trivial to get an overview of what tests we have and
don't have, I didn't find any that highlighted this one

I used all 3 variants of parameters, just to be sure :)
2021-02-22 14:52:51 +01:00
Rasmus Lerchedahl Petersen
d23a8ad016 Python: elide test output 2021-02-21 13:12:54 +01:00
Rasmus Lerchedahl Petersen
46faba69ff Python: Fix for-iteration of tuples 2021-02-21 12:41:16 +01:00
Rasmus Lerchedahl Petersen
0aecf33fe6 Python: test iteration through overflow parameters
These are in a tuple, so the for-step does not fire
2021-02-21 12:33:04 +01:00
Rasmus Wriedt Larsen
bfbaa85272 Python: Add test of public_key method with cryptodome
Added in 3.10 release https://github.com/Legrandin/pycryptodome/blob/master/Changelog.rst#3100-6-february-2021
2021-02-19 15:03:47 +01:00
Rasmus Wriedt Larsen
d5ff477644 Python: Add modeling for pycryptodome PyPI package 2021-02-19 13:59:18 +01:00
Rasmus Wriedt Larsen
6e4c627209 Python: Add modeling for pycryptodomex PyPI package 2021-02-19 13:59:17 +01:00
Rasmus Wriedt Larsen
bd40965afe Python: Add modeling for cryptography PyPI package 2021-02-19 13:59:17 +01:00
Rasmus Wriedt Larsen
1bf9f7d135 Python: Add missing annotations to new crypto tests 2021-02-19 13:59:16 +01:00
Rasmus Wriedt Larsen
11cd0dbbc0 Python: Add concepts for public-key generation
I did spend some time to figure out how to best write `minimumSecureKeySize`
predicate. I wanted to write once and for all the recommended sizes for each
cryptosystem.

I considered making the predicate such as

```codeql
int minimumSecureKeySize() {
    this.getName() = "RSA" and result = 2048
    or
    this.getName() = "DSA" and result = 2048
    or
    this.getName() = "ECC" and result = 244
}
```

but then it would be impossible to add a new model without also being able to
modify the body of this predicate -- which seems like a bad way to start off a
brand new way of modeling things.

So I considered if we could add it to the non-range class, such as

```codeql
class RSAKeyGeneration extends KeyGeneration {
  RSAKeyGeneration() { this.getName() = "RSA" }

  override int minimumSecureKeySize() { result = 2048 }
}
```

This has the major problem that when you're writing the models for a new
API (and therefore extending KeyGeneration::Range), there is no way for you to
see that you need to take this extra step :| (also problem about how we should
define `minimumSecureKeySize` on `KeyGeneration` class then, since if we make it
abstract, we effectively disable the ability to refine `KeyGeneration` since any
subclass must provide an implementation.)

So, therefore I ended up with this solution ;)
2021-02-19 13:59:16 +01:00
Rasmus Wriedt Larsen
4ab61bb088 Python: Add a few tests for crypto frameworks
Tests working can be verified by running

```
ls ql/python/ql/test/experimental/library-tests/frameworks/crypto*/*.py | xargs -L1 sh -c 'python $0 || exit 255'
```
2021-02-19 13:26:45 +01:00
Taus Brock-Nannestad
04eb0c774c Python: Use LocalSourceNode in type tracker tests
One minor change to the tests results needed: there is no longer local
flow going into the `ModuleVariableNode` for `attr_ref` in the
`moduleattr.ql` test, but I think this is reasonable.
2021-02-16 18:25:54 +01:00
Rasmus Wriedt Larsen
bc8e61366b Python: Clarify comment about flask blueprint URL prefixes 2021-02-16 15:29:25 +01:00
Rasmus Wriedt Larsen
1e1cb87436 Python: Model flask blueprints 2021-02-16 15:26:51 +01:00
Rasmus Wriedt Larsen
b7ea469e26 Python: Add tests for flask blueprints 2021-02-16 15:03:00 +01:00
Rasmus Wriedt Larsen
bf401c7498 Merge pull request #5103 from tausbn/python-port-flask-to-api-graphs
Python: Port Flask models to use API graphs
2021-02-16 15:00:46 +01:00
Rasmus Wriedt Larsen
745148474a Python: Model get_redirect_url in django 2021-02-15 10:55:52 +01:00
Rasmus Wriedt Larsen
6934d5e642 Python: Add django test of RedirectView subclass 2021-02-15 10:55:51 +01:00
Rasmus Wriedt Larsen
79855157b3 Python: Move django response test to django v2/v3
That's really the django version I care about :P
2021-02-15 10:55:50 +01:00
CodeQL CI
178c54e69b Merge pull request #5139 from RasmusWL/django-improvements
Approved by yoff
2021-02-14 02:16:52 -08:00
Rasmus Wriedt Larsen
10fdc4bfb9 Python: Add support for more yaml loading functions 2021-02-12 12:30:00 +01:00
Rasmus Wriedt Larsen
2021cdbe33 Python: Add tests for more yaml loading functions 2021-02-12 12:30:00 +01:00
Rasmus Wriedt Larsen
c57a4df819 Python: Model taint of self.request on django view class 2021-02-10 17:48:48 +01:00
Rasmus Wriedt Larsen
9ca738d921 Python: Add taint test for self.request on django view class 2021-02-10 17:48:41 +01:00
Rasmus Wriedt Larsen
ca0d345987 Django: Model any class used in django route setup as view class 2021-02-10 16:26:25 +01:00
Rasmus Wriedt Larsen
78a3206fce Python: Add test with unkown view class in django 2021-02-10 15:56:33 +01:00
Rasmus Wriedt Larsen
42eceb80bd Python: Handle view functions with decorators 2021-02-10 15:47:55 +01:00
Rasmus Wriedt Larsen
1d25184b32 Python: Add test for type-tracking through decorators
In general, if there is _some_ decorator on a function, it might not be safe to
track content out of it (since the decorator could do anything), but in this
case, we can see what the decorator does, so we should be able to handle it (but
we don't right now).

By my understanding of how type-tracking works, if we track content through
`my_decorator`, then we would also track content to the result of
`unrelated_func()`, which I wanted to make sure our tests would catch.

I found out the core of the problem seems to come from our lack of being able to
track to the inner scope, and added an explicit test for that.
2021-02-09 13:43:10 +01:00
Rasmus Wriedt Larsen
eb7e30d472 Python: Add test of django view handler with decorator
Which we currently don't handle :(

Also added a bit more explanatory comments
2021-02-09 13:25:12 +01:00
Taus
c0c2aa69b3 Merge branch 'main' into python-port-flask-to-api-graphs 2021-02-08 14:17:25 +01:00
yoff
f1a0ec2dec Merge pull request #4981 from RasmusWL/port-url-redirect-query
Python: Port url redirect query
2021-02-06 00:39:10 +01:00