Commit Graph

4508 Commits

Author SHA1 Message Date
yoff
138a7ae67f Merge pull request #6349 from RasmusWL/more-modeling
Python: Improve various library modeling
2021-09-06 17:01:45 +02:00
yoff
c7146ac10c Update python/ql/src/meta/alerts/RemoteFlowSourcesReach.ql
Co-authored-by: Rasmus Wriedt Larsen <rasmuswl@github.com>
2021-09-06 16:00:58 +02:00
Andrew Eisenberg
6a47fcaf1f Packaging: Normalize all qlpack.yml files for all languages
This commit ensures consistency among all of our qlpacks. Here are the
changes:

1. Ensure only modern references are used (codeql-{lang} is converted to
   codeql/{lang}-all or codeql/{lang}-queries where appropriate).
2. Use consistent version numbers. All languages are at 0.0.2 except
   javascript, which is 0.0.3.
3. Convert all `libraryPathDependencies` to `dependencies` with version
   constraints
4. Dependencies from query packs to other packs are always `"*"` since
   these dependencies are always from source and we should get the
   latest.
5. Dependencies from codeql/{lang}-lib to codeql/{lang}-upgrades must
   be strict since there is a tight connection between the libary
   and its relevant upgrades.
2021-09-03 11:53:28 -07:00
Rasmus Lerchedahl Petersen
913990bc62 Python: Add suggested comments and test case 2021-09-03 14:40:16 +02:00
yoff
c6eb795e76 Apply suggestions from code review
Co-authored-by: Rasmus Wriedt Larsen <rasmuswriedtlarsen@gmail.com>
2021-09-03 14:23:57 +02:00
Rasmus Wriedt Larsen
9f590dbf2d Python: Fix .expected
After we now model `db.text()` calls from Flask-SQLAlchemy
2021-09-02 16:04:25 +02:00
Rasmus Wriedt Larsen
414bf12f86 Python: Fix DefaultTextClauseConstruction 2021-09-02 16:03:25 +02:00
Rasmus Wriedt Larsen
88c6d4bb20 Python: Fix .qhelp 2021-09-02 16:02:04 +02:00
Rasmus Wriedt Larsen
065075056b Python: Highlight how await taint-step works 2021-09-02 15:45:59 +02:00
Rasmus Wriedt Larsen
ad102e2746 Python: Minor cleanup to snippets
As pointed out in review, we don't need this override any more!
2021-09-02 15:40:32 +02:00
CodeQL CI
b4963c7538 Merge pull request #6558 from erik-krogh/redosCasing
Approved by esbena, yoff
2021-09-02 12:20:08 +01:00
Taus
e4fd749a46 Merge pull request #6547 from github/RasmusWL/cwe328-weak-hash
Python: Add CWE-328 to `py/weak-sensitive-data-hashing`
2021-09-02 11:42:31 +02:00
Rasmus Wriedt Larsen
d55f18f8e3 Python: Add modeling of Flask-SQLAlchemy 2021-09-02 10:48:24 +02:00
Rasmus Wriedt Larsen
f1744890b1 Python: Add tests for Flask-SQLAlchemy 2021-09-02 10:48:15 +02:00
Rasmus Wriedt Larsen
c34d6d1162 Python: Add query to handle SQLAlchemy TextClause Injection
instead of doing this via taint-steps. See description in code/tests.
2021-09-02 10:19:57 +02:00
Rasmus Wriedt Larsen
81dbe36e99 Python: Promote SQLAlchemy modeling
Due to the split between `src/` and `lib/`, I was not really able to do
the next step without having moved the SQLAlchemy modeling over to be in
`lib/` as well.
2021-09-02 10:19:57 +02:00
Rasmus Wriedt Larsen
ba99e21875 Python: Remove modeling of sqlescapy PyPI package
I've never seen this being used in real code, and this library doesn't
have a lot of traction, so I would rather not commit to supporting it
(which includes verifying that it actually makes things safe).

Personally I don't think this is the right approach for avoiding SQL
injection either.
2021-09-02 10:19:57 +02:00
Rasmus Wriedt Larsen
91442e100c Python: Model sessionmaker().begin() 2021-09-02 10:19:57 +02:00
Rasmus Wriedt Larsen
feb2303e1f Python: Model the underlying DB-API connection 2021-09-02 10:19:57 +02:00
Rasmus Wriedt Larsen
1ab04a7276 Python: Model Connection.execution_options 2021-09-02 10:19:57 +02:00
Rasmus Wriedt Larsen
2acf518037 Python: Model exec_driver_sql 2021-09-02 10:19:57 +02:00
Rasmus Wriedt Larsen
fe143c7dfa Python: Rewrite most of SQLAlchemy modeling 2021-09-02 10:19:57 +02:00
Rasmus Wriedt Larsen
b39bb24fcf Python: Add more SQLAlchemy tests 2021-09-02 10:19:57 +02:00
Erik Krogh Kristensen
1ad204d89e make after and TState private in ReDoSUtil 2021-09-02 09:15:43 +02:00
Erik Krogh Kristensen
df04c5044c use concat instead of strictconcat in RegexTreeView.qll 2021-09-02 08:54:39 +02:00
Tom Hvitved
c3ecae503b Data flow: Sync files 2021-09-01 19:58:47 +02:00
Erik Krogh Kristensen
a3289fabe1 sync ReDoSUtil with python 2021-09-01 12:47:06 +02:00
Rasmus Lerchedahl Petersen
a01fca5d48 Merge branch 'main' of github.com:github/codeql into python-regex-parsing-consistency-checks
To fix conflicts
2021-08-30 18:40:12 +02:00
yoff
13c5857241 Update python/ql/src/semmle/python/RegexTreeView.qll
Co-authored-by: Taus <tausbn@github.com>
2021-08-30 18:38:38 +02:00
Rasmus Lerchedahl Petersen
a855074588 Python: Try to remove py2/3 differences 2021-08-30 15:41:51 +02:00
Rasmus Lerchedahl Petersen
0de621edf9 Python: Add qldoc 2021-08-30 15:03:58 +02:00
Rasmus Lerchedahl Petersen
1903cb8f82 Python: Add change note 2021-08-30 11:27:55 +02:00
Rasmus Lerchedahl Petersen
a762373ad6 Python: Implement simple barrier guard
The one found in the original test case
2021-08-30 11:04:27 +02:00
Erik Krogh Kristensen
f5a1a12435 support case insensitive regexps in the ReDoS queries 2021-08-30 09:59:33 +02:00
jorgectf
64b305cf7a Add .qhelp along with its example 2021-08-26 23:29:45 +02:00
Rasmus Lerchedahl Petersen
49ae549e89 Python: Implement modifying syntax 2021-08-26 14:29:18 +02:00
Rasmus Lerchedahl Petersen
097c23e437 Python: add inline expectations test
Consider removing the original test
2021-08-26 14:08:52 +02:00
Rasmus Wriedt Larsen
47377c7197 Merge branch 'main' into more-modeling 2021-08-26 13:40:17 +02:00
jorgectf
786edb72df Update .expected 2021-08-26 12:36:34 +02:00
Jorge
d458464e6b Apply suggestions from code review
Co-authored-by: Rasmus Wriedt Larsen <rasmuswriedtlarsen@gmail.com>
2021-08-26 12:20:09 +02:00
Rasmus Lerchedahl Petersen
d834cec9b9 Python: test simple sanitizer 2021-08-26 11:31:20 +02:00
Rasmus Lerchedahl Petersen
8614563b42 Python: More tests of syntactic constructs 2021-08-26 10:56:41 +02:00
Rasmus Lerchedahl Petersen
5bff5188ac Python: switch from negative to positive list
This should avoid potentially terrible performance.
Also noted the missing syntactic constructs,
as I went through the documnetation.
2021-08-25 23:52:42 +02:00
Erik Krogh Kristensen
0cc19d914e use toUnicode in ReDoSUtil.qll 2021-08-25 22:21:43 +02:00
Rasmus Wriedt Larsen
605bd19306 Python: Add CWE-328 to py/weak-sensitive-data-hashing
Reading over the description at https://cwe.mitre.org/data/definitions/328.html:

> The product uses a hashing algorithm that produces a hash value that can be used to determine the original input, or to find an input that can produce the same hash, more efficiently than brute force techniques.

For the data that does not require computationally expensive hashing, that will be the exactly problems that this query finds 👍 (that is, MD5, SHA1)
2021-08-25 10:19:22 +02:00
Andrew Eisenberg
3660c64328 Packaging: Rafactor Python core libraries
Extract the external facing `qll` files into the codeql/python-all
query pack.
2021-08-24 13:23:45 -07:00
yoff
2f5ed03798 Merge pull request #6323 from RasmusWL/sec-test-layout
Python: Restructure security tests to contain query name
2021-08-24 16:50:08 +02:00
Rasmus Lerchedahl Petersen
e865a290de Python: straight port of query
The old query uses `pointsTo` to limit the sinks
to methods on lists and dictionaries.
That constraint is omitted here which could hurt performance.
2021-08-24 16:35:11 +02:00
Rasmus Lerchedahl Petersen
e3765ced78 Python: Add tests for modification of defaults 2021-08-24 16:35:11 +02:00
Rasmus Wriedt Larsen
ca341bde08 Merge pull request #5612 from jty-team/jty/python/nosqlInjection
Python: CWE-943 - Add NoSQL injection query
2021-08-24 11:29:25 +02:00