Commit Graph

2894 Commits

Author SHA1 Message Date
yoff
138a7ae67f Merge pull request #6349 from RasmusWL/more-modeling
Python: Improve various library modeling
2021-09-06 17:01:45 +02:00
yoff
c7146ac10c Update python/ql/src/meta/alerts/RemoteFlowSourcesReach.ql
Co-authored-by: Rasmus Wriedt Larsen <rasmuswl@github.com>
2021-09-06 16:00:58 +02:00
Andrew Eisenberg
6a47fcaf1f Packaging: Normalize all qlpack.yml files for all languages
This commit ensures consistency among all of our qlpacks. Here are the
changes:

1. Ensure only modern references are used (codeql-{lang} is converted to
   codeql/{lang}-all or codeql/{lang}-queries where appropriate).
2. Use consistent version numbers. All languages are at 0.0.2 except
   javascript, which is 0.0.3.
3. Convert all `libraryPathDependencies` to `dependencies` with version
   constraints
4. Dependencies from query packs to other packs are always `"*"` since
   these dependencies are always from source and we should get the
   latest.
5. Dependencies from codeql/{lang}-lib to codeql/{lang}-upgrades must
   be strict since there is a tight connection between the libary
   and its relevant upgrades.
2021-09-03 11:53:28 -07:00
Taus
e4fd749a46 Merge pull request #6547 from github/RasmusWL/cwe328-weak-hash
Python: Add CWE-328 to `py/weak-sensitive-data-hashing`
2021-09-02 11:42:31 +02:00
Rasmus Wriedt Larsen
47377c7197 Merge branch 'main' into more-modeling 2021-08-26 13:40:17 +02:00
Rasmus Wriedt Larsen
605bd19306 Python: Add CWE-328 to py/weak-sensitive-data-hashing
Reading over the description at https://cwe.mitre.org/data/definitions/328.html:

> The product uses a hashing algorithm that produces a hash value that can be used to determine the original input, or to find an input that can produce the same hash, more efficiently than brute force techniques.

For the data that does not require computationally expensive hashing, that will be the exactly problems that this query finds 👍 (that is, MD5, SHA1)
2021-08-25 10:19:22 +02:00
Andrew Eisenberg
3660c64328 Packaging: Rafactor Python core libraries
Extract the external facing `qll` files into the codeql/python-all
query pack.
2021-08-24 13:23:45 -07:00
Rasmus Wriedt Larsen
ca341bde08 Merge pull request #5612 from jty-team/jty/python/nosqlInjection
Python: CWE-943 - Add NoSQL injection query
2021-08-24 11:29:25 +02:00
Erik Krogh Kristensen
38477d7d2e Merge pull request #6462 from erik-krogh/repeat
JS: support more regular expressions in js/incomplete-multi-character-sanitization
2021-08-23 15:39:31 +02:00
yoff
0c0f335b1c Merge pull request #6508 from github/RasmusWL-patch-1
Python: Update comment for RegExpTreeView isExcluded
2021-08-23 15:07:29 +02:00
yoff
467aa647da Merge pull request #6507 from tausbn/python-prevent-polynomial-redos-explosion
Python: Prevent explosion in poly-ReDoS query
2021-08-23 11:48:14 +02:00
Taus
021e5ff510 Python: Autoformat 2021-08-18 14:27:54 +00:00
Rasmus Wriedt Larsen
60eb81106a Python: Update comment for RegExpTreeView isExcluded
I noticed after reading https://github.com/github/codeql/pull/6507, but didn't want to overload that PR.
2021-08-18 16:16:26 +02:00
Taus
af91a2df00 Python: Prevent explosion in poly-ReDoS query
I consider this to be a short-term solution to the performance problems
we identified. The choice of "at most ten occurrences of `.*`" is
somewhat arbitrary, and it's possible a higher limit would work just as
well.
2021-08-18 13:21:46 +00:00
Andrew Eisenberg
03d6b15401 Merge branch 'main' into aeisenberg/pack/cpp 2021-08-17 15:28:47 -07:00
Rasmus Wriedt Larsen
3231ae77ef Python: Apply suggestions from code review
Co-authored-by: yoff <lerchedahl@gmail.com>
2021-08-17 15:17:10 +02:00
Erik Krogh Kristensen
3f7f5d2418 performance improvements in ReDoSUtil 2021-08-17 15:10:33 +02:00
Erik Krogh Kristensen
49e47641e4 sync ReDoSUtil.qll with python 2021-08-17 15:10:33 +02:00
Rasmus Wriedt Larsen
15d483d56c Python: Use TypeTrackingNode in new PEP249 modeling 2021-08-17 12:03:40 +02:00
Rasmus Wriedt Larsen
b649f5f38c Merge branch 'main' into peewee-modeling 2021-08-17 12:03:18 +02:00
Andrew Eisenberg
e566fb9c5a Packaging: Update suite-helpers qlpack
Uses new style naming scheme.
2021-08-16 17:51:33 -07:00
Erik Krogh Kristensen
46959234b7 Merge pull request #6288 from erik-krogh/emptyRedos
JS/Python: Fix FP in redos related to empty lookaheads
2021-08-16 13:48:22 +02:00
Erik Krogh Kristensen
e962a7c77c Update python/ql/src/semmle/python/RegexTreeView.qll
Co-authored-by: yoff <lerchedahl@gmail.com>
2021-08-16 11:24:05 +02:00
jorgectf
e6ce10b5c5 Merge remote-tracking branch 'origin/main' into jty/python/nosqlInjection 2021-08-10 20:01:08 +02:00
Tom Hvitved
ea6d51f123 Python: Avoid bad join in AstExtended::AstNode::containsInScope 2021-08-09 11:20:57 +02:00
Taus
74f1992aaf Merge pull request #6352 from tausbn/mergeback-rc/3.2-to-main
Mergeback `rc/3.2` to `main`
2021-07-22 19:58:29 +02:00
Rasmus Wriedt Larsen
42a997cbcb Python: Fix deprecation warning 2021-07-22 15:59:13 +02:00
Rasmus Wriedt Larsen
71e6db8a01 Merge branch 'main' into jorgectf/python/ldapimproperauth 2021-07-22 15:57:43 +02:00
Taus
6ea8ef5d16 Merge branch 'rc/3.2' into mergeback-rc/3.2-to-main 2021-07-22 13:52:56 +00:00
Rasmus Wriedt Larsen
802d9bda83 Merge pull request #5680 from mrthankyou/python-use-sqlalchemy
Python: Add SqlAlchemy model
2021-07-22 15:31:39 +02:00
Taus
badf6311c9 Python: Remove flow between globals...
... in a local scope. Or rather, remove these from the `hasLocalSource`
relation.

This prevents a quadratic blowup when the same global is mentioned
_a lot_ of times within a single function scope.
2021-07-22 13:10:40 +00:00
Taus
ed794f42b5 Python: Soft revert TypeTrackingNode
Temporarily instates `TypeTrackingNode` as an alias of `LocalSourceNode`
as having it as a separate class lead to performance regressions.

In the hopes that this will be resolved in the near future, I have left
the current `TypeTrackingNode` implementation in situ, but hidden inside
a `FutureWork` private module.
2021-07-22 13:10:07 +00:00
Mathias Vorreiter Pedersen
e34261accf Merge branch 'rc/3.2' into mergeback-2021-07-22 2021-07-22 14:40:22 +02:00
Rasmus Wriedt Larsen
38875ca0c7 Python: Improve handling of async methods 2021-07-22 14:17:07 +02:00
Rasmus Wriedt Larsen
c3f942f899 Python: Provide internal InstanceTaintStepsHelper
I realized that if you ever wanted to the way taint-steps works again,
you would have to go to all the 117 places it has been implemented, and
change EVERY ONE OF THEM :( so trying to solve that problem here.

Not super happy with the name, but that was just the best I could come up with :D
2021-07-22 14:16:50 +02:00
Rasmus Wriedt Larsen
6e9d9fcbbd Python: Improve taint steps in for & iterable unpacking
These were written way before the ones in DataFlowPrivate, but
apparently didn't cover quite as much :|
2021-07-22 14:16:17 +02:00
Taus
e9a4114c04 Python: Hotfix: Disable ReDoS queries 2021-07-22 10:58:49 +00:00
Rasmus Wriedt Larsen
f5ae5a581b Python: A bit more additional taint clean up
A few stragglers that did not have the same TODO comments as the others
2021-07-22 11:59:46 +02:00
Rasmus Wriedt Larsen
d2efe0b84d Python: Normalize additional taint steps for modeled classes
Such that it should be next to the other class-related predicates (such
as `instance()`), the class is called `AdditionalTaintStep`, and it
marked private.

I also moved any modeling of attributes as well, while I was at it.
2021-07-22 11:59:46 +02:00
Rasmus Wriedt Larsen
be1cad864b Python: Resolve all meth = obj.meth; meth() TODOs
It would probably have been easier to do this as the _first_ thing...
but that's too late now 😓
2021-07-22 11:59:46 +02:00
Rasmus Wriedt Larsen
6f63c03558 Python: Model http.cookies.Morsel and usage in Tornado 2021-07-22 10:43:18 +02:00
Rasmus Wriedt Larsen
7e09a1cbfd Python: Model tornado.httputil.HTTPHeaders 2021-07-22 10:43:18 +02:00
Rasmus Wriedt Larsen
7020e4132b Python: Model BaseHTTPRequestHandler.rfile as file-like object 2021-07-22 10:43:18 +02:00
Rasmus Wriedt Larsen
d388dd547e Python: Model HTTPMessage from Stdlib 2021-07-22 10:43:18 +02:00
Rasmus Wriedt Larsen
dac71ded9d Python: Add Authorization modeling in Flask 2021-07-22 10:43:18 +02:00
Rasmus Wriedt Larsen
133632119d Python: Model werkzeug Headers
Also removed a misleading comment link to method on wrong class :D
2021-07-22 10:43:18 +02:00
Rasmus Wriedt Larsen
4d9c86a252 Python: Model Werkzeug FileStorage.save as FileSystemAccess 2021-07-22 10:43:18 +02:00
Rasmus Wriedt Larsen
9cb4899c5c Python: Add FileStorage modeling in Flask 2021-07-22 10:43:18 +02:00
Rasmus Wriedt Larsen
09b0c300d9 Python: Rewrite werkzeug to avoid InstanceSourceApiNode
InstanceSourceApiNode is a really good idea, but it just happened too
soon. I can't do what I need if I have to supply an API-node. So to
avoid confusion between deprecating to/from InstanceSource in those
classes, I opted to do some major reorganizing as well 👍

Due to aliasing restrictions, I had to use a little trick with the
`WerkzeugOld` module.
2021-07-22 10:43:18 +02:00
Rasmus Wriedt Larsen
04190ea308 Python: Add file-like modeling to werkzeug FileStorage 2021-07-22 10:43:18 +02:00