hohn/codeql
1
0
Fork 0
mirror of https://github.com/github/codeql.git synced 2026-03-31 04:38:18 +02:00
Code Issues Packages Projects Releases Wiki Activity
29,605 Commits 1,723 Branches 164 Tags
fd23fa94a5f5e86e14ee87b8ade2b45a70440fbb
Commit Graph

4498 Commits

Author SHA1 Message Date
Rasmus Wriedt Larsen
fd23fa94a5 Python: Remove dubious fstat* modeling 
These operate on file descriptors, and not on paths. file descriptors
doesn't fit into the rest of our modeling, so I would rather remove them
than to make it look like it's properly handled.

I also did not include any of the functions that work on file
descriptors when looking through all of `os`. So this keeps everything
consistent at least ;)
 2021-11-29 14:54:02 +01:00
Rasmus Wriedt Larsen
e79b8f3e23 Python: Treat os.exec*, os.spawn*, and os.posix_spawn* as FileSystemAccess 2021-11-29 14:54:02 +01:00
Rasmus Wriedt Larsen
d2d5cce787 Python: Recognize keyword arguments for os.*spawn* calls 2021-11-29 14:54:02 +01:00
Rasmus Wriedt Larsen
14590436f9 Python: Expand tests for os.exec*, os.spawn*, and os.posix_spawn* 2021-11-29 14:54:02 +01:00
Rasmus Wriedt Larsen
50d3592ad3 Python: Add more complete tests of os module 
I went through https://docs.python.org/3.10/library/os.html in order,
and added all the functions that works on paths.

`lstat` and `statvfs` were already modeled, but did not have any tests.
 2021-11-29 14:54:02 +01:00
Rasmus Wriedt Larsen
a91208fd2c Python: Fix kwarg modeling for os.path.isdir 2021-11-29 14:54:02 +01:00
Rasmus Wriedt Larsen
36f14b31bc Python: Add explicit tests for kwargs 
I also renamed the arguments to match what the keyword argument is
called. It doesn't matter too much for these specific tests, but for the
tests I'm about to add, it makes things a lot easier to get an overview
of.

Oh, and a test failure :O
 2021-11-29 14:54:02 +01:00
Rasmus Wriedt Larsen
82602014ad Python: Minor refactor to use os.path.<func> 
Since that's the idiomatic way to use this module
 2021-11-29 14:54:02 +01:00
Anders Schack-Mulligen
00ee34c0a0 Merge pull request #7237 from hvitved/dataflow/consistency-config 
Data flow: Introduce `ConsistencyConfiguration` class
 2021-11-26 12:49:25 +01:00
Anders Schack-Mulligen
a06642944f Merge pull request #7232 from aschackmull/dataflow/perf 
Data flow: Performance tuning
 2021-11-25 15:01:01 +01:00
Tom Hvitved
6cb00992e8 Data flow: Introduce ConsistencyConfiguration class 2021-11-25 10:01:47 +01:00
Erik Krogh Kristensen
3bab8c6d1d Merge pull request #7173 from erik-krogh/getRubyInSync 
JS/PY/RB: get ReDoSUtil in sync for ruby
 2021-11-24 15:20:23 +01:00
Anders Schack-Mulligen
7ca3407c86 Dataflow: Sync. 2021-11-24 14:43:00 +01:00
Rasmus Wriedt Larsen
7dde52ced2 Merge pull request #7131 from RasmusWL/wsgiref.simple_server 
Python: Model `wsgiref.simple_server` applications
 2021-11-24 14:22:23 +01:00
Rasmus Wriedt Larsen
2a5e0a3b77 Merge pull request #7145 from RasmusWL/remove-owasp-tags 
Python/Ruby: Remove owasp tags
 2021-11-24 13:56:48 +01:00
Rasmus Wriedt Larsen
e2652591a5 Python: Change perf fix PoorMansFunctionResolution 
Thanks @yoff, this leaves us with the following evaluation, which looks
very close to the one in the other fix (but with cleaner implementation)
-- both at 688k max tuples (although numbers are not exactly the same).

```
[2021-11-24 13:48:40] (14s) Tuple counts for PoorMansFunctionResolution::getSimpleMethodReferenceWithinClass#ff/2@e5f05asv after 74ms:
                      47493  ~3%     {3} r1 = JOIN Class::Class::getAMethod_dispred#ff WITH py_Classes ON FIRST 1 OUTPUT Lhs.1, 0, Lhs.0
                      47335  ~0%     {2} r2 = JOIN r1 WITH AstGenerated::Function_::getArg_dispred#fff ON FIRST 2 OUTPUT Rhs.2, Lhs.2
                      46683  ~0%     {2} r3 = JOIN r2 WITH DataFlowPublic::ParameterNode::getParameter_dispred#fb_10#join_rhs ON FIRST 1 OUTPUT Rhs.1, Lhs.1
                      259968 ~4%     {2} r4 = JOIN r3 WITH LocalSources::Cached::hasLocalSource#ff_10#join_rhs ON FIRST 1 OUTPUT Rhs.1, Lhs.1
                      161985 ~0%     {3} r5 = JOIN r4 WITH Attributes::AttrRef::accesses_dispred#bff_102#join_rhs ON FIRST 1 OUTPUT Rhs.1 'result', Lhs.1, Rhs.2
                      161985 ~2%     {3} r6 = JOIN r5 WITH Attributes::AttrRead#class#f ON FIRST 1 OUTPUT Lhs.2, Lhs.1, Lhs.0 'result'
                      688766 ~0%     {3} r7 = JOIN r6 WITH Function::Function::getName_dispred#ff_10#join_rhs ON FIRST 1 OUTPUT Lhs.1, Rhs.1 'func', Lhs.2 'result'
                      20928  ~0%     {2} r8 = JOIN r7 WITH Class::Class::getAMethod_dispred#ff ON FIRST 2 OUTPUT Lhs.1 'func', Lhs.2 'result'
                                     return r8
```
 2021-11-24 13:52:05 +01:00
Rasmus Wriedt Larsen
47448d9efc Python: Apply suggestions from code review 
Co-authored-by: yoff <lerchedahl@gmail.com>
 2021-11-24 12:02:12 +01:00
yoff
f9729bccef Merge pull request #7143 from RasmusWL/path-improvements 
Python: Model `posixpath` and `os.stat`
 2021-11-24 11:36:06 +01:00
Erik Krogh Kristensen
87a1ccd428 Merge branch 'main' into getRubyInSync 2021-11-23 20:20:37 +01:00
Taus
8cccee6eba Merge pull request #6972 from yoff/python/promote-redos 
Python: Promote ReDoS queries
 2021-11-23 14:02:09 +01:00
Rasmus Wriedt Larsen
eaed870b31 Python: Fix performance problem in PoorMansFunctionResolution 
Before these changes:

[2021-11-22 12:02:50] (8s) Tuple counts for PoorMansFunctionResolution::getSimpleMethodReferenceWithinClass#ff/2@cbddf257 after 8.6s:
                      387565   ~0%     {3} r1 = JOIN Attributes::AttrRead#class#f WITH Attributes::AttrRef::accesses_dispred#bff ON FIRST 1 OUTPUT Rhs.2, Lhs.0 'result', Rhs.1
                      6548632  ~0%     {3} r2 = JOIN r1 WITH Function::Function::getName_dispred#ff_10#join_rhs ON FIRST 1 OUTPUT Rhs.1 'func', Lhs.1 'result', Lhs.2
                      5640480  ~0%     {4} r3 = JOIN r2 WITH Class::Class::getAMethod_dispred#ff_10#join_rhs ON FIRST 1 OUTPUT Rhs.1, Lhs.1 'result', Lhs.2, Lhs.0 'func'
                      55660458 ~0%     {5} r4 = JOIN r3 WITH Class::Class::getAMethod_dispred#ff ON FIRST 1 OUTPUT Rhs.1, 0, Lhs.1 'result', Lhs.2, Lhs.3 'func'
                      55621412 ~0%     {4} r5 = JOIN r4 WITH AstGenerated::Function_::getArg_dispred#fff ON FIRST 2 OUTPUT Rhs.2, Lhs.2 'result', Lhs.3, Lhs.4 'func'
                      54467144 ~0%     {4} r6 = JOIN r5 WITH DataFlowPublic::ParameterNode::getParameter_dispred#fb_10#join_rhs ON FIRST 1 OUTPUT Lhs.2, Rhs.1, Lhs.1 'result', Lhs.3 'func'
                      20928    ~0%     {2} r7 = JOIN r6 WITH LocalSources::Cached::hasLocalSource#ff ON FIRST 2 OUTPUT Lhs.3 'func', Lhs.2 'result'
                                       return r7

With these changes:

[2021-11-22 11:54:25] (415s) Tuple counts for PoorMansFunctionResolution::getSimpleMethodReferenceWithinClass_helper#fff/3@14db70a8 after 75ms:
                      388306 ~0%     {2} r1 = JOIN Attributes::AttrRead#class#f WITH Attributes::AttrRef::getObject_dispred#bf ON FIRST 1 OUTPUT Rhs.1, Lhs.0 'read'
                      379420 ~4%     {2} r2 = JOIN r1 WITH LocalSources::Cached::hasLocalSource#ff ON FIRST 1 OUTPUT Rhs.1, Lhs.1 'read'
                      175082 ~0%     {2} r3 = JOIN r2 WITH DataFlowPublic::ParameterNode#class#fff ON FIRST 1 OUTPUT Rhs.2, Lhs.1 'read'
                      175082 ~2%     {3} r4 = JOIN r3 WITH Essa::ParameterDefinition::getParameter_dispred#ff ON FIRST 1 OUTPUT 0, Rhs.1, Lhs.1 'read'
                      166798 ~0%     {2} r5 = JOIN r4 WITH AstGenerated::Function_::getArg_dispred#fff_120#join_rhs ON FIRST 2 OUTPUT Rhs.2 'func', Lhs.2 'read'
                      162096 ~0%     {3} r6 = JOIN r5 WITH Class::Class::getAMethod_dispred#ff_10#join_rhs ON FIRST 1 OUTPUT Lhs.0 'func', Rhs.1 'cls', Lhs.1 'read'
                                     return r6

[2021-11-22 11:54:25] (415s) Tuple counts for PoorMansFunctionResolution::getSimpleMethodReferenceWithinClass_helper2#ffff/4@2b60f0s9 after 63ms:
                      162046 ~0%     {3} r1 = SCAN PoorMansFunctionResolution::getSimpleMethodReferenceWithinClass_helper#fff OUTPUT In.2 'read', In.0 'func', In.1 'cls'
                      162046 ~0%     {3} r2 = JOIN r1 WITH Attributes::AttrRead#class#f ON FIRST 1 OUTPUT Lhs.1 'func', Lhs.2 'cls', Lhs.0 'read'
                      162046 ~1%     {3} r3 = JOIN r2 WITH py_Functions ON FIRST 1 OUTPUT Lhs.1 'cls', Lhs.2 'read', Lhs.0 'func'
                      162046 ~0%     {3} r4 = JOIN r3 WITH py_Classes ON FIRST 1 OUTPUT Lhs.1 'read', Lhs.2 'func', Lhs.0 'cls'
                      161935 ~5%     {4} r5 = JOIN r4 WITH Attributes::AttrRef::getAttributeName_dispred#bf ON FIRST 1 OUTPUT Rhs.1, Lhs.0 'read', Lhs.1 'func', Lhs.2 'cls'
                      688526 ~1%     {4} r6 = JOIN r5 WITH Function::Function::getName_dispred#ff_10#join_rhs ON FIRST 1 OUTPUT Lhs.2 'func', Lhs.3 'cls', Lhs.1 'read', Rhs.1 'readFunction'
                                     return r6

[2021-11-22 11:54:25] (415s) Tuple counts for PoorMansFunctionResolution::getSimpleMethodReferenceWithinClass#ff/2@f73ae6dq after 58ms:
                      688526 ~0%     {4} r1 = SCAN PoorMansFunctionResolution::getSimpleMethodReferenceWithinClass_helper2#ffff OUTPUT In.1, In.0, In.3 'func', In.2 'result'
                      688526 ~0%     {3} r2 = JOIN r1 WITH Class::Class::getAMethod_dispred#ff ON FIRST 2 OUTPUT Rhs.0, Lhs.2 'func', Lhs.3 'result'
                      20913  ~0%     {2} r3 = JOIN r2 WITH Class::Class::getAMethod_dispred#ff ON FIRST 2 OUTPUT Lhs.1 'func', Lhs.2 'result'
                                     return r3

We need the `pragma[only_bind_into]` in getSimpleMethodReferenceWithinClass_helper2, otherwise the tuple counts would look like, which is needlessly big.

[2021-11-22 17:14:34] (2s) Tuple counts for PoorMansFunctionResolution::getSimpleMethodReferenceWithinClass_helper2#ffff/4@5f0505h7 after 711ms:
                      13570510 ~3%     {2} r1 = JOIN Function::Function::getName_dispred#ff_10#join_rhs WITH Attributes::AttrRef::getAttributeName_dispred#ff_10#join_rhs ON FIRST 1 OUTPUT Rhs.1 'read', Lhs.1 'readFunction'
                      688526   ~1%     {4} r2 = JOIN r1 WITH PoorMansFunctionResolution::getSimpleMethodReferenceWithinClass_helper#fff_201#join_rhs ON FIRST 1 OUTPUT Rhs.1 'func', Rhs.2 'cls', Lhs.0 'read', Lhs.1 'readFunction'
                                       return r2
 2021-11-22 17:22:39 +01:00
Rasmus Wriedt Larsen
f09f1c4c50 Python: Minor refactor in PoorMansFunctionResolution 2021-11-22 11:11:29 +01:00
Erik Krogh Kristensen
ee858d840e get ReDoSUtil in sync for ruby 2021-11-18 16:49:34 +01:00
Erik Krogh Kristensen
1cca377e7d Merge pull request #6561 from erik-krogh/htmlReg 
JS/Py/Ruby: add a bad-tag-filter query
 2021-11-18 09:39:13 +01:00
Taus
eed98bd76a Merge pull request #5588 from jorgectf/jorgectf/python/jwt-queries 
Python: Add JWT security-related queries
 2021-11-16 15:40:45 +01:00
jorgectf
9ad8a85f4d Delete redundant checks in verifiesSignature() 2021-11-16 15:08:18 +01:00
Anders Schack-Mulligen
c70d384d28 Merge pull request #7045 from aschackmull/dataflow/hidden-ret-subpaths 
Data flow: Support hidden return nodes in subpaths predicate
 2021-11-16 15:04:51 +01:00
jorgectf
3fe2a08376 Update .expected file 2021-11-16 15:03:49 +01:00
Jorge
a722631278 Apply suggestions from code review 
Co-authored-by: Taus <tausbn@github.com>
 2021-11-16 15:02:06 +01:00
Rasmus Wriedt Larsen
98e6fc8a88 Python/Ruby: Remove owasp tags 
These are no longer correct, since the A1 category changed from 2017 to
2021, see https://owasp.org/Top10/#whats-changed-in-the-top-10-for-2021

Since only a very few queries had these tags, I think we're much better
off having them removed.
 2021-11-16 12:03:50 +01:00
Rasmus Wriedt Larsen
a980f26fda Python: Model os.stat (and friends) 2021-11-16 10:45:32 +01:00
Rasmus Wriedt Larsen
9f4107d211 Python: Model posixpath, ntpath, and genericpath modules 2021-11-16 10:45:14 +01:00
Rasmus Wriedt Larsen
6b7abacc5f Merge pull request #7135 from RasmusWL/b32hexencode 
Python: Model `b32hexencode`/`b32hexdecode`
 2021-11-15 15:51:46 +01:00
Rasmus Wriedt Larsen
39927fa613 Python: Model b32hexencode/b32hexdecode 
New in Python 3.10

See
- https://devdocs.io/python~3.10/library/base64#base64.b32hexencode
- https://devdocs.io/python~3.10/library/base64#base64.b32hexdecode
 2021-11-15 15:23:49 +01:00
Rasmus Wriedt Larsen
cfdfcaa3e8 Python: Support Path.hardlink_to (new in 3.10) 
See https://docs.python.org/3.10/library/pathlib.html#pathlib.Path.hardlink_to
 2021-11-15 14:57:59 +01:00
Rasmus Wriedt Larsen
5d60975f65 Python: Support aiter and anext (new in 3.10) 
See
- https://docs.python.org/3/whatsnew/3.10.html#other-language-changes
- https://docs.python.org/3.10/library/functions.html#aiter
- https://docs.python.org/3.10/library/functions.html#anext
 2021-11-15 14:55:34 +01:00
Rasmus Wriedt Larsen
7c3b68b7f8 Merge pull request #7091 from RasmusWL/port-request-without-validation 
Python: Port `py/request-without-cert-validation` to use API graphs
 2021-11-15 13:51:57 +01:00
Rasmus Wriedt Larsen
9e097f5430 Python: Improve PoorMansFunctionResolution 2021-11-15 13:40:19 +01:00
Rasmus Wriedt Larsen
0d4cb1e6ce Python: Add test of PoorMansFunctionResolution 2021-11-15 13:34:39 +01:00
Rasmus Wriedt Larsen
6eb4525ab2 Python: Model wsgiref.simple_server applications 2021-11-15 13:34:39 +01:00
Rasmus Wriedt Larsen
e812029c03 Python: Add test for wsgiref.simple_server 2021-11-15 13:34:38 +01:00
Taus
c17560f948 Merge pull request #7096 from tausbn/python-fix-more-bad-joins 
Python: Fix a bunch of performance issues
 2021-11-15 12:10:27 +01:00
yoff
5beb681580 Merge pull request #7087 from RasmusWL/path-injection-fp 
Python: Add interesting path-injection FP
 2021-11-12 15:20:19 +01:00
yoff
9f614b1d98 Merge pull request #7016 from RasmusWL/django-rest-framework 
Python: Model Django REST framework
 2021-11-12 14:27:56 +01:00
Rasmus Wriedt Larsen
491f72bb2a Python: Adjust generated code to be more familiar 2021-11-12 13:30:03 +01:00
Rasmus Wriedt Larsen
de69e4c645 Python: Expand on SubclassFinder implementation note 2021-11-12 13:29:03 +01:00
Rasmus Wriedt Larsen
f7b53321b9 Python: Remove copy-pasted comment 2021-11-12 13:19:20 +01:00
Taus
55ea715ce9 Merge pull request #7033 from RasmusWL/flask-admin 2021-11-12 12:18:56 +01:00
Rasmus Wriedt Larsen
860b1a5cc3 Python: Other minor QLDoc adjustment 2021-11-12 11:46:45 +01:00
Rasmus Wriedt Larsen
99081ea7e0 Python: Minor adjustment in QLDoc 2021-11-12 11:42:36 +01:00