Taus
e2f79d8516
Python: Fix several bad getScope joins
...
It seems the optimiser has started getting the wrong end of the stick
whenever we write `foo.getScope() = bar.getScope()` for some expressions
`foo` and `bar`.
This lead to things like
```
(196s) Tuple counts for Definitions::ModuleVariable::global_variable_callnode#ff/2@5ab278 after 2m33s:
2952757013 ~0% {2} r1 = JOIN Definitions::ModuleVariable::global_variable_callnode#ff#shared WITH Variables::Variable::getScope_dispred#ff_10#join_rhs ON FIRST 1 OUTPUT Rhs.1 'this', Lhs.1 'result'
495693 ~0% {2} r2 = JOIN r1 WITH Variables::GlobalVariable#class#f ON FIRST 1 OUTPUT Lhs.0 'this', Lhs.1 'result'
453589 ~0% {2} r3 = JOIN r2 WITH Definitions::ModuleVariable#f ON FIRST 1 OUTPUT Lhs.0 'this', Lhs.1 'result'
return r3
```
and
```
(315s) Tuple counts for Definitions::SsaSourceVariable::getAUse_dispred#ff/2@a39328 after 1m57s:
...
1785275 ~3% {2} r24 = Definitions::ModuleVariable::global_variable_callnode#ff#shared UNION Definitions::SsaSourceVariable::getAUse_dispred#ff#shared
3008614987 ~0% {2} r25 = JOIN r24 WITH Variables::Variable::getScope_dispred#ff_10#join_rhs ON FIRST 1 OUTPUT Rhs.1 'this', Lhs.1 'result'
127 ~1% {2} r26 = JOIN r25 WITH Definitions::NonLocalVariable#class#f ON FIRST 1 OUTPUT Lhs.0 'this', Lhs.1 'result'
127 ~1% {2} r27 = JOIN r26 WITH Variables::LocalVariable#f ON FIRST 1 OUTPUT Lhs.0 'this', Lhs.1 'result'
...
```
(Note the timings: 2m33s and 1m57s.)
Now we have the much more reasonable
```
(38s) Tuple counts for Definitions::ModuleVariable::global_variable_callnode#ff/2@c53031 after 42ms:
453589 ~0% {2} r1 = JOIN Definitions::ModuleVariable::global_variable_callnode#ff#shared WITH Definitions::ModuleVariable::scope_as_global_variable#ff_10#join_rhs ON FIRST 1 OUTPUT Rhs.1 'this', Lhs.1 'result'
return r1
```
and
```
(46s) Tuple counts for Definitions::SsaSourceVariable::getAUse_dispred#ff/2@4b19de after 375ms:
...
```
2021-11-09 20:54:41 +00:00
Rasmus Wriedt Larsen
1e31416049
Merge pull request #7031 from yoff/python/taint-through-with
...
Python: Taint through `async with`
2021-11-09 14:08:07 +01:00
Rasmus Lerchedahl Petersen
a58c47b07b
Python: model aiopg.sa
2021-11-09 12:49:57 +01:00
Rasmus Lerchedahl Petersen
cd332a75fc
Python: model aiopg
2021-11-09 12:32:21 +01:00
Erik Krogh Kristensen
8727060ca7
add comment about modes of operation
...
Co-authored-by: Rasmus Wriedt Larsen <rasmuswriedtlarsen@gmail.com >
2021-11-09 11:15:12 +01:00
Rasmus Wriedt Larsen
9e2bc41648
Python: Improve hashlib.new modeling
...
By using a backwards type-tracker to find possible hashing algorithm
names.
2021-11-04 15:36:32 +01:00
Erik Krogh Kristensen
a19627c72f
optionally ignore everything after a dash
2021-11-04 13:19:44 +01:00
Erik Krogh Kristensen
02f500b9c2
Merge branch 'main' into htmlReg
2021-11-04 12:58:42 +01:00
Erik Krogh Kristensen
523c15cd72
don't include mode-of-operation into the algorithm names
2021-11-03 14:54:50 +01:00
Mathias Vorreiter Pedersen
4a2894a707
Merge pull request #7025 from MathiasVP/nomagic-parameterCand
...
Dataflow: Replace a 'noinline' pragma with a 'nomagic' pragma
2021-11-02 20:40:44 +00:00
Rasmus Wriedt Larsen
8cd9fdebf9
Python: Model flask_admin
2021-11-02 15:43:13 +01:00
Rasmus Lerchedahl Petersen
768932d7b3
Python: Add tainttracking step that was removed
...
when the correpsonding datadlow step was removed.
2021-11-02 15:01:47 +01:00
Erik Krogh Kristensen
5975e19f53
sync identical files
2021-11-02 14:45:33 +01:00
yoff
97625d7c2c
Merge pull request #7023 from RasmusWL/toml
...
Python: Add modeling of `toml`
2021-11-02 14:42:06 +01:00
Rasmus Wriedt Larsen
cb6bcada4c
Merge branch 'main' into django-rest-framework
2021-11-02 14:33:16 +01:00
yoff
0240631510
Merge pull request #6782 from RasmusWL/fastapi
...
Python: Model FastAPI
2021-11-02 14:16:12 +01:00
Rasmus Wriedt Larsen
c52e453342
Python: Minor rewrite
2021-11-02 13:37:50 +01:00
Anders Schack-Mulligen
7d0152f3c0
Merge pull request #6932 from aschackmull/dataflow/flow-features
...
Dataflow: Add support for call context restrictions on sources/sinks.
2021-11-02 13:24:17 +01:00
Mathias Vorreiter Pedersen
6f4107ff23
Dataflow: Replace a 'noinline' pragma with a 'nomagic' pragma.
2021-11-02 11:37:40 +00:00
Rasmus Wriedt Larsen
8ee804a8c2
Python: Add toml modeling
2021-11-02 11:57:15 +01:00
Tom Hvitved
302373d154
Merge pull request #6858 from hvitved/python/type-tracker-changes
...
Python: Type tracker changes
2021-11-02 11:47:01 +01:00
CodeQL CI
d5e2026a26
Merge pull request #6934 from erik-krogh/more-instanceof
...
Approved by MathiasVP, esbena, yoff
2021-11-02 03:46:23 -07:00
CodeQL CI
5d62aa5b29
Merge pull request #6994 from erik-krogh/redundant-cast
...
Approved by RasmusWL, aschackmull, esbena, geoffw0, hvitved, nickrolfe
2021-11-02 03:45:48 -07:00
Tom Hvitved
1e64893742
Update python/ql/lib/semmle/python/dataflow/new/internal/TypeTracker.qll
...
Co-authored-by: Taus <tausbn@github.com >
2021-11-02 11:16:32 +01:00
Tom Hvitved
660398aa78
Python: Introduce TypeBackTracker::getACompatibleTypeTracker()
2021-11-02 11:16:32 +01:00
Tom Hvitved
73fd66cfed
Python: Cache TypeBackTracker::prepend
2021-11-02 11:16:32 +01:00
Rasmus Wriedt Larsen
83389be8e2
Python: Add some missing QLDocs
2021-11-02 11:02:51 +01:00
Rasmus Wriedt Larsen
a7e4e5ef83
Python: Add rest_framework Response modeling
2021-11-02 10:55:44 +01:00
Rasmus Wriedt Larsen
13815fe728
Python: Model known APIView subclasses
...
Added internal helper `.qll` file as well
2021-11-02 10:55:44 +01:00
Rasmus Wriedt Larsen
62d30630aa
Python: Add rest_framework Request taint modeling
2021-11-02 10:55:44 +01:00
Rasmus Wriedt Larsen
5d77e62f3a
Python: Add basic rest_framework Request modeling
2021-11-02 10:55:44 +01:00
Rasmus Wriedt Larsen
57e13c6066
Python: rest_framework.decorators.api_view handling
...
Had to expose even more things, and had to make the `DjangoRouteHandler`
modeling more flexible so I could extend the char-pred in a different
file.
2021-11-02 10:55:44 +01:00
Rasmus Wriedt Larsen
222db37c0d
Python: Add initial rest_framework modeling
...
I had to make the Django and PrivateDjango modeling non-private :O
2021-11-02 10:55:44 +01:00
Rasmus Wriedt Larsen
b7b9120724
Python: Better handling of Pydantic models
2021-11-02 10:29:17 +01:00
Rasmus Wriedt Larsen
17da28118a
Python: Small refactor to use extends .. instanceof
2021-11-02 10:06:11 +01:00
Erik Krogh Kristensen
0897b004eb
revert removal of redundant inline casts in some python files
2021-10-29 14:40:27 +02:00
Erik Krogh Kristensen
d36c66cfca
remove redundant inline casts in arguments where the type is inferred by the call target
2021-10-29 14:37:56 +02:00
yoff
1c78c792ff
Merge pull request #6991 from RasmusWL/flask-blueprints
...
Python: Support `flask.blueprints.Blueprint`
2021-10-29 14:06:43 +02:00
Rasmus Wriedt Larsen
7e7c363e43
Python: Apply suggestions from code review
...
Co-authored-by: yoff <lerchedahl@gmail.com >
2021-10-29 13:59:36 +02:00
Rasmus Wriedt Larsen
85f00fda19
Merge pull request #6776 from yoff/python/model-asyncpg
...
Python: Model `asyncpg`
2021-10-29 13:54:44 +02:00
Anders Schack-Mulligen
3a1836c9f6
Merge pull request #7000 from aschackmull/dataflow/interface-refactor
...
Dataflow: Refactor public references to DataFlowCallable
2021-10-29 12:21:13 +02:00
Anders Schack-Mulligen
5951ae79b9
Dataflow: Add language specific predicates.
2021-10-29 11:11:35 +02:00
Anders Schack-Mulligen
00df6798b1
Dataflow: Sync
2021-10-29 11:00:23 +02:00
Erik Krogh Kristensen
6fffdf6101
Merge pull request #6855 from erik-krogh/secCookie
...
JS: Move cookie queries out of experimental.
2021-10-29 10:23:48 +02:00
yoff
8f9741ae72
Update python/ql/lib/semmle/python/internal/Awaited.qll
...
Co-authored-by: Rasmus Wriedt Larsen <rasmuswriedtlarsen@gmail.com >
2021-10-28 19:13:08 +02:00
Erik Krogh Kristensen
15c90adec5
remove redundant cast where the type is enforced by an equality comparison
2021-10-28 18:08:20 +02:00
Erik Krogh Kristensen
e75448ebb0
remove redundant inline casts
2021-10-28 16:35:53 +02:00
Rasmus Wriedt Larsen
a33a8fd518
Python: Support flask.blueprints.Blueprint
...
Thanks to @haby0 who originally proposed this as part of
https://github.com/github/codeql/pull/6977
2021-10-28 14:02:03 +02:00
Rasmus Lerchedahl Petersen
3abe3e43d0
Python: autoformat
2021-10-28 13:58:01 +02:00
Rasmus Wriedt Larsen
8c3349f40f
Python: Properly model flask.send_from_directory
...
To not include `filename` as path-injection sink.
2021-10-28 13:41:39 +02:00