Commit Graph

1996 Commits

Author SHA1 Message Date
Taus
fe12e620dd Python: Avoid clobbering range in test
This was an unwanted interaction between two unrelated tests, so I
switched to a different built-in in the second test. I also added a test
case that shows an unfortunate side effect of this more restricted
handling of built-ins.
2021-05-12 18:42:10 +00:00
thank_you
3e25b14a68 Update NoSQLInjection.expected 2021-05-11 20:07:09 -04:00
yoff
0e5a2c4573 Merge pull request #5442 from jorgectf/jorgectf/python/redos
Python: Add Regular Expression Injection query
2021-05-11 12:11:35 +02:00
thank_you
3ace49549a Add tests for SqlAlchemy modeling library
After researching SqlAlchemy and it's various query methods, I discovered several types of SQL injection possibilities.

The SQLExecution.py file contains these examples and can be broken up into two types of injections. Injections requiring the text() taint-step and injections NOT requiring the text() taint step.
2021-05-10 16:12:15 -04:00
Rasmus Wriedt Larsen
1b0d5053e7 Python: simplejson load/dump only works with lib installed
Which I had done locally. Problem is the same about not having PostUpdateNode
when points-to is not able to resolve the call, so I'm happy to just make CI
happy right now, and hopefully we'll get a fix to the underlying problem soon 😊
2021-05-10 16:21:29 +02:00
Rasmus Wriedt Larsen
c2a6b811fc Python: Add modeling of ujson PyPI package
The problem with `tainted_filelike` not having taint, is that in the call

`ujson.dump(tainted_obj, tainted_filelike)`

there is no PostUpdateNote for `tainted_filelike` :( The reason is that
points-to is not able to resolve the call, so none of the clauses in
`argumentPreUpdateNode` matches

See 08731fc6cf/python/ql/src/semmle/python/dataflow/new/internal/DataFlowPrivate.qll (L101-L111)

Let's deal with that issue in an other PR though
2021-05-10 15:10:31 +02:00
Rasmus Wriedt Larsen
72d08f4d6e Python: Model json load/dump 2021-05-10 15:10:30 +02:00
Rasmus Wriedt Larsen
63f28d7d9b Python: Model keyword args to json loads/dumps 2021-05-10 15:10:29 +02:00
Rasmus Wriedt Larsen
784e0cdb96 Python: Improve tests of json module
Inspired by the work on previous commit
2021-05-10 15:10:28 +02:00
Rasmus Wriedt Larsen
3fe9a3d933 Python: Add modeling of simplejson PyPI package
I noticed that we don't handle PostUpdateNote very well in the concept tests,
for exmaple for `json.dump(...)` there _should_ have been an `encodeOutput` as
part of the inline expectations.

I'll work on fixing that up in a separate PR, to keep things clean.
2021-05-10 15:10:27 +02:00
Rasmus Wriedt Larsen
8afdf26540 Python: Add modeling of idna PyPI package 2021-05-10 11:47:11 +02:00
thank_you
07c3e22428 Fix method name to match flask_mongoengine library 2021-05-09 19:23:52 -04:00
jorgectf
e7bdc73420 Update .expected 2021-05-07 23:00:21 +02:00
jorgectf
65c6f1976a Rename mongoengine-flask-db-document-subclass 2021-05-07 23:00:08 +02:00
Jorge
ae806cd445 Merge branch 'github:main' into jorgectf/python/ldapimproperauth 2021-05-07 20:46:09 +02:00
thank_you
7693d696cc Add additional query tests
To ensure that this query works against numerous usages of libraries such as PyMongo, Flask PyMongo, Mongoengine, and Flask Mongoengine, I've added a variety of query tests to test against. These tests deal with scenarious such as:

- Subscript expressions
- Mongoengine instances and Document subclasses
- Mongoengine connection usage
- And more...
2021-05-07 14:36:02 -04:00
Rasmus Wriedt Larsen
d50f22504e Python: Fix .expected 2021-05-05 14:07:15 +02:00
Rasmus Wriedt Larsen
668bfd3a41 Python: Support EC keygen without class-instance for cryptography
I also added a new test to show off how what the origin ends up looking
like... I think it looks ok
2021-05-05 12:29:55 +02:00
Rasmus Wriedt Larsen
3ceb8bbcc6 Python: Add cryptography test for EC
Apparently, passing in the class (without instantiating it) is allowed
2021-05-05 10:52:57 +02:00
CodeQL CI
95f26aadd3 Merge pull request #5681 from yoff/python-support-pathlib
Approved by tausbn
2021-05-04 09:20:24 -07:00
CodeQL CI
84d43946de Merge pull request #5755 from RasmusWL/non-alert-data-part1
Approved by tausbn
2021-04-29 02:51:34 -07:00
Rasmus Lerchedahl Petersen
16bde2729d Python: add flow from methods to calls 2021-04-28 17:02:24 +02:00
Rasmus Wriedt Larsen
f2b4e31e7f Python: Make Diagnostics tests pass
I had comitted a bad .expected file it seems, and since the encoding for UTF-8
is named differently from Python 2 to Python 3, we're only going to run the test
for one version.
2021-04-28 10:21:59 +02:00
jorgectf
12ccd7e3b6 Update .expected 2021-04-27 19:54:39 +02:00
jorgectf
05ee853c4e Remove wrong comment 2021-04-27 19:54:39 +02:00
jorgectf
d968eea914 Move expected to /test 2021-04-27 19:54:38 +02:00
jorgectf
81d23c066c Move tests and qlref from /src to /test 2021-04-27 19:54:37 +02:00
Rasmus Wriedt Larsen
37db21d269 Merge pull request #5284 from yoff/python-port-insecure-protocol
Python: port py/insecure-protocol
2021-04-27 09:30:18 +02:00
Rasmus Wriedt Larsen
deb3db3f95 Python: Add non-alert data for extractor diagnostics
This is basically just a port of the C++/JS queries added in:

- https://github.com/github/codeql/pull/5414 (C++)
- https://github.com/github/codeql/pull/5656 (JS)

SyntaxError should capture all errors we have information about. At least in
`python/ql/src/semmlecode.python.dbscheme` the only match for `error` is
`py_syntax_error_versioned` (which `SyntaxError` is based on).
2021-04-23 13:29:44 +02:00
Rasmus Wriedt Larsen
354dee1b09 Python: Add non-alert data for lines of code
`py/summary/lines-of-code` is just a port of the C++/JS queries added in:

- https://github.com/github/codeql/pull/5271 (C++)
- https://github.com/github/codeql/pull/5304 (JS)

We are the first to implement the `lines-of-user-code` query, so nothing to
compare with in other languages -- but it makes a lot of sense to do for Python 👍
2021-04-23 13:22:18 +02:00
Rasmus Wriedt Larsen
ac83c695ad Python: Add py/weak-sensitive-data-hashing query 2021-04-22 15:23:41 +02:00
Rasmus Wriedt Larsen
499adc26a3 Python: Extend SensitiveDataSource tests
Now it contains all the sort of things we actually support 👍
2021-04-22 15:23:40 +02:00
Rasmus Wriedt Larsen
794a86a6b0 Python: Add SensitiveDataSource 2021-04-22 15:23:39 +02:00
Rasmus Wriedt Larsen
56c409737d Python: Port py/weak-cryptographic-algorithm
The other query (py/weak-sensitive-data-hashing) is added in future commit
2021-04-22 15:23:38 +02:00
Rasmus Wriedt Larsen
59edd18c34 Python: Move framework test-files out of experimental
This PR was rebased on newest main, but was written a long time ago when all the
framework test-files were still in experimental. I have not re-written my local
git-history, since there are MANY updates to those files (and I dare not risk
it).
2021-04-22 15:23:37 +02:00
Rasmus Wriedt Larsen
1616975e06 Python: Model hashlib from standard library 2021-04-22 15:23:37 +02:00
Rasmus Lerchedahl Petersen
5a4e661e60 Merge branch 'main' of github.com:github/codeql into python-support-pathlib 2021-04-22 15:04:21 +02:00
Rasmus Wriedt Larsen
7ffbfa8043 Python: Expand stdlib md5 tests with keyword-arguments 2021-04-22 14:51:20 +02:00
Rasmus Wriedt Larsen
fa88f22453 Python: Model hashing operations in cryptography package 2021-04-22 14:51:20 +02:00
Rasmus Wriedt Larsen
c5f826580b Python: Model encrypt/decrypt in cryptography package
I introduced a InternalTypeTracking module, since the type-tracking code got so
verbose, that it was impossible to get an overview of the relevant predicates.
(this means the "first" type-tracking predicate that is usually private, cannot
be marked private anymore, since it needs to be exposed in the private module.
2021-04-22 14:51:19 +02:00
Rasmus Wriedt Larsen
bf6f5074c2 Python: Port cryptodome tests to crypto
I don't know if this is really a smart test-setup... I feel a bit stupid when
doing this xD
2021-04-22 14:51:19 +02:00
Rasmus Wriedt Larsen
f8254381f3 Python: Add MISSING: CryptographicOperationAlgorithm annotations
For RSA it's unclear what the algorithm name should even be. Signatures based on
RSA private keys with PSS scheme is ok, but with pkcs#1 v1.5 they are
weak/vulnerable. So clearly just putting RSA as the algorithm name is not enough
information...

and that problem is also why I wanted to do this commit separetely (to call
extra atten to this).
2021-04-22 14:51:18 +02:00
Rasmus Wriedt Larsen
23140dfb76 Python: Add CryptographicOperation modeling for Cryptodome 2021-04-22 14:51:17 +02:00
Rasmus Wriedt Larsen
1b2ed9d99a Python: Align cryptodome tests 2021-04-22 14:51:16 +02:00
Rasmus Wriedt Larsen
2c0df8e656 Python: Add MD5 tests 2021-04-22 14:51:16 +02:00
Rasmus Wriedt Larsen
65c8d9605e Python: Add CryptographicOperation Concept
I considered using `getInput` like in JS, but things like signature verification
has multiple inputs (message and signature).

Using getAnInput also aligns better with Decoding/Encoding.
2021-04-22 14:51:14 +02:00
Rasmus Wriedt Larsen
d18fbb7f07 Python: Add working tests of AES and RC4 2021-04-22 14:51:14 +02:00
Rasmus Wriedt Larsen
cf64701bcb Python: Move weak-crypto-algorithm tests to own folder 2021-04-22 14:51:13 +02:00
Rasmus Wriedt Larsen
5a9e27c6fc Merge branch 'main' into django-3.2 2021-04-21 17:15:47 +02:00
CodeQL CI
30d7f0dc98 Merge pull request #5687 from RasmusWL/inline-taint-tests
Approved by yoff
2021-04-21 06:24:12 -07:00