Commit Graph

1729 Commits

Author SHA1 Message Date
Rasmus Wriedt Larsen
afb160fbbb Python: Add additional taint steps for for-iteration 2020-08-26 20:18:31 +02:00
Rasmus Wriedt Larsen
e2a89aa296 Python: Add additional taint steps for copy
deepcopy was already handled somehow, don't really know how :D
2020-08-26 19:39:38 +02:00
Rasmus Wriedt Larsen
b974dadca1 Python: Add additional taint steps for containers 2020-08-26 19:39:37 +02:00
Rasmus Wriedt Larsen
b6049765a8 Python: Add a few more collection taint tests 2020-08-26 19:39:36 +02:00
Rasmus Wriedt Larsen
32f9d30136 Python: Add syntactic taint steps for json methods 2020-08-26 19:39:36 +02:00
Rasmus Wriedt Larsen
41e24ae93f Python: Add non-syntactical test for taint of json methods 2020-08-26 19:39:35 +02:00
Rasmus Wriedt Larsen
5f9aa4c3b9 Python: Restructure defaultAdditionalTaintStep tests
This makes it easier to add a new test-case, and makes it easier to work with
the existing files. It does have a downside on making it a bit more annoying
looking at TestTaint.expected, and possible longer runtime, but I think it's
still worth it.
2020-08-26 19:39:33 +02:00
Rasmus Wriedt Larsen
a1ada62596 Python: Remodel taint tests for shared lib
I took the bits from ql/test/library-tests/taint/ that seemed easy to port. I
left out namedtuple for now, but it is part of internal tracking ticket, so
won't be forgotten.
2020-08-26 19:39:32 +02:00
Taus
b1946c60dd Merge pull request #4127 from RasmusWL/python-tainttracking-fstring
Python: Handle f-strings in (current) taint tracking
2020-08-26 16:06:01 +02:00
Taus
000fa33d54 Merge pull request #4013 from yoff/SharedDataflow_SequenceFlow
Python: Shared dataflow: Content flow
2020-08-25 15:38:14 +02:00
Rasmus Wriedt Larsen
2dbf83b579 Python: TaintTracking: Move tests of py3 string methods 2020-08-25 13:06:27 +02:00
Rasmus Wriedt Larsen
cf121cc4d0 Python: TaintTracking: stringMethods => stringManipualtion 2020-08-25 13:05:27 +02:00
Rasmus Wriedt Larsen
238e0845aa Python: Minor refactoring 2020-08-25 12:50:41 +02:00
Rasmus Wriedt Larsen
0439b83c60 Python: Taint when using unicode 2020-08-25 12:50:32 +02:00
Rasmus Wriedt Larsen
2a29e26687 Python: Fix grammar
Co-authored-by: yoff <lerchedahl@gmail.com>
2020-08-25 12:41:53 +02:00
Rasmus Wriedt Larsen
483bd0e863 Python: Fix shared taint tracking tests
Since there was a .ql file, qltest tried to run a test in
test/experimental/dataflow/taintracking/ which failed since there was no code.
2020-08-25 11:15:11 +02:00
Rasmus Wriedt Larsen
13148b42d3 Python: Handle taint of f-strings 2020-08-24 17:23:10 +02:00
Rasmus Wriedt Larsen
2f090df6d3 Python: Transform comments to QLDoc for security.strings.Basic 2020-08-24 17:20:04 +02:00
Rasmus Lerchedahl Petersen
2608509fa7 Merge branch 'main' of github.com:github/codeql into SharedDataflow_SequenceFlow 2020-08-24 17:16:33 +02:00
Rasmus Wriedt Larsen
be2acc00db Python: Add test for tainted f-string 2020-08-24 17:14:51 +02:00
Rasmus Wriedt Larsen
d96ef73033 Python: Handle taint for f-strings
Which we seem to not handle in the current taint tracking :O

f-strings needs to be Python 3 only, so enabled that test setup. I really liked
the idea for having the version specific tests right next to the normal tests,
so you don't have to look in
test/experimental/3/dataflow/i/will/forget/to/look/here.
2020-08-24 16:46:00 +02:00
Rasmus Wriedt Larsen
cb4b4e91ab Python: Taint for string multiplication 2020-08-24 14:54:06 +02:00
Rasmus Wriedt Larsen
b688fe68d6 Python: Add options file to shared dataflow tests
Since there isn't one in top-level of experimental, making a single import made
tests go really slow :|
2020-08-24 14:54:05 +02:00
Rasmus Wriedt Larsen
5125c7a55c Python: Add taint tests for encode/decode functions 2020-08-24 14:54:04 +02:00
Rasmus Wriedt Larsen
31b398937a Python: Handle taint from bytes(obj) 2020-08-24 14:17:59 +02:00
Rasmus Wriedt Larsen
1e447c5ca2 Python: Handle taint for % formatting 2020-08-24 14:15:27 +02:00
Rasmus Wriedt Larsen
80745e8881 Python: Model string methods in shared taint tracking library 2020-08-24 13:58:42 +02:00
Rasmus Wriedt Larsen
a77f118b62 Python: Shared taint tracking: Handle string concat + subcript 2020-08-24 13:58:41 +02:00
Rasmus Wriedt Larsen
61f89ca3c3 Python: Add tests for shared taint tracking for strings
I adopted the TestTaint testing setup that I made for the "old" taint tracking
tests. This time around we should figure out if we can use .qlref or similar so
it doesn't end up in multiple copies that are not kept up to date :|

The `repr` predicate could probably be placed somewhere better. For now I just
wanted something that could help me. I considered just expanding the `repr`
predicate in `ql/src/semmle/python/strings.qll`, but since it's currently used
by queries, I didn't want to do anything about it.

Anyway, the output it gives is much more useful than seeing this ;)

```
| test.py:20 | ok   | str_operations | test.py:20:9:20:10 | ts |
| test.py:21 | fail | str_operations | test.py:21:9:21:18 | BinaryExpr |
| test.py:22 | fail | str_operations | test.py:22:9:22:18 | BinaryExpr |
| test.py:23 | fail | str_operations | test.py:23:9:23:21 | Subscript |
| test.py:24 | fail | str_operations | test.py:24:9:24:13 | Subscript |
| test.py:25 | fail | str_operations | test.py:25:9:25:18 | Subscript |
| test.py:26 | fail | str_operations | test.py:26:9:26:13 | Subscript |
| test.py:27 | fail | str_operations | test.py:27:9:27:15 | str() |
| test.py:35 | fail | str_methods | test.py:35:9:35:23 | Attribute() |
| test.py:36 | fail | str_methods | test.py:36:9:36:21 | Attribute() |
| test.py:37 | fail | str_methods | test.py:37:9:37:22 | Attribute() |
| test.py:38 | fail | str_methods | test.py:38:9:38:23 | Attribute() |
| test.py:40 | fail | str_methods | test.py:40:9:40:19 | Attribute() |
| test.py:41 | fail | str_methods | test.py:41:9:41:23 | Attribute() |
| test.py:42 | fail | str_methods | test.py:42:9:42:36 | Attribute() |
| test.py:44 | fail | str_methods | test.py:44:9:44:25 | Attribute() |
| test.py:45 | fail | str_methods | test.py:45:9:45:45 | Attribute() |
| test.py:47 | fail | str_methods | test.py:47:9:47:21 | Attribute() |
| test.py:48 | fail | str_methods | test.py:48:9:48:19 | Attribute() |
| test.py:49 | fail | str_methods | test.py:49:9:49:18 | Attribute() |
| test.py:51 | fail | str_methods | test.py:51:9:51:32 | Attribute() |
| test.py:52 | fail | str_methods | test.py:52:9:52:34 | Attribute() |
| test.py:54 | fail | str_methods | test.py:54:9:54:21 | Attribute() |
| test.py:55 | fail | str_methods | test.py:55:9:55:19 | Attribute() |
| test.py:56 | fail | str_methods | test.py:56:9:56:18 | Attribute() |
| test.py:57 | fail | str_methods | test.py:57:9:57:21 | Attribute() |
| test.py:58 | fail | str_methods | test.py:58:9:58:18 | Attribute() |
| test.py:59 | fail | str_methods | test.py:59:9:59:18 | Attribute() |
| test.py:60 | fail | str_methods | test.py:60:9:60:21 | Attribute() |
| test.py:62 | fail | str_methods | test.py:62:9:62:26 | Attribute() |
| test.py:63 | fail | str_methods | test.py:63:9:63:42 | Attribute() |
| test.py:65 | fail | str_methods | test.py:65:9:65:26 | Attribute() |
| test.py:66 | fail | str_methods | test.py:66:9:66:42 | Attribute() |
| test.py:69 | fail | str_methods | test.py:69:9:69:25 | Attribute() |
| test.py:70 | fail | str_methods | test.py:70:9:70:26 | Attribute() |
| test.py:71 | fail | str_methods | test.py:71:9:71:22 | Attribute() |
| test.py:72 | fail | str_methods | test.py:72:9:72:21 | Attribute() |
| test.py:73 | fail | str_methods | test.py:73:9:73:23 | Attribute() |
| test.py:78 | ok   | str_methods | test.py:78:9:78:39 | Attribute() |
```
2020-08-24 13:58:39 +02:00
Taus
b8d6f76749 Merge pull request #4056 from yoff/SharedDataflow_ParameterTests
Python: Shared dataflow, parameter routing tests
2020-08-24 11:36:30 +02:00
Rasmus Lerchedahl Petersen
e1343c7f1e Python: Support set literals. 2020-08-21 11:15:04 +02:00
Rasmus Lerchedahl Petersen
ccff84d546 Python: Test flow into conprehension 2020-08-21 10:40:22 +02:00
Rasmus Lerchedahl Petersen
f9b1c5e4bd Python: Fix bug pointed out by reviewer 2020-08-21 10:04:27 +02:00
yoff
bfd9c0860f Apply suggestions from code review
Co-authored-by: Rasmus Wriedt Larsen <rasmuswriedtlarsen@gmail.com>
2020-08-21 09:43:29 +02:00
Rasmus Wriedt Larsen
7fb8e0e277 Python: Add basic shared taint tracking test 2020-08-20 14:49:17 +02:00
Rasmus Wriedt Larsen
0baac8fd54 Python: Adjust shared taint tracking skeleton
So it fits the setup from Java/Go, with AdditionalTaintStep class.
2020-08-20 14:49:09 +02:00
Rasmus Lerchedahl Petersen
bd53a711d3 Merge branch 'main' of github.com:github/codeql into SharedDataflow_SequenceFlow 2020-08-19 11:42:41 +02:00
Rasmus Lerchedahl Petersen
176aa06fad Python: Address review comments 2020-08-19 09:21:16 +02:00
yoff
5e84754f73 Update python/ql/src/experimental/dataflow/internal/DataFlowPrivate.qll
Co-authored-by: Rasmus Wriedt Larsen <rasmuswriedtlarsen@gmail.com>
2020-08-19 08:03:47 +02:00
yoff
06bd436aea Update python/ql/src/experimental/dataflow/internal/DataFlowPrivate.qll
Co-authored-by: Rasmus Wriedt Larsen <rasmuswriedtlarsen@gmail.com>
2020-08-19 08:02:53 +02:00
yoff
8fbb447f4c Update python/ql/src/experimental/dataflow/internal/DataFlowPrivate.qll
Co-authored-by: Taus <tausbn@github.com>
2020-08-19 08:02:29 +02:00
yoff
1c3b945e55 Update python/ql/src/experimental/dataflow/internal/DataFlowPublic.qll
Co-authored-by: Taus <tausbn@github.com>
2020-08-19 08:01:54 +02:00
yoff
43a5e74c65 Update python/ql/src/experimental/dataflow/internal/DataFlowPublic.qll
Co-authored-by: Taus <tausbn@github.com>
2020-08-19 08:01:42 +02:00
Anders Schack-Mulligen
f75f5ab125 Merge pull request #3838 from hvitved/dataflow/flow-fwd-ctx
Data flow: Use precise call contexts in `flowFwd()`
2020-08-18 13:06:11 +02:00
Tom Hvitved
a2fc92b9db Data flow: Address review comments 2020-08-17 15:46:43 +02:00
Rasmus Lerchedahl Petersen
2817602a97 Merge branch 'master' of github.com:github/codeql into SharedDataflow_ParameterTests 2020-08-14 14:27:57 +02:00
CodeQL CI
e9a36b2524 Merge pull request #4062 from tausbn/python-fix-unknown-import-star
Approved by yoff
2020-08-14 13:17:45 +01:00
Rasmus Lerchedahl Petersen
9556937840 Python: address review comments 2020-08-14 11:29:58 +02:00
Tom Hvitved
e518cbabd6 Python: Sync data flow files 2020-08-14 11:04:45 +02:00
yoff
8d49ad7325 Update python/ql/test/experimental/dataflow/coverage/datamodel.py
Co-authored-by: Taus <tausbn@github.com>
2020-08-14 10:53:37 +02:00