Commit Graph

246 Commits

Author SHA1 Message Date
yoff
5f6f85c998 Merge pull request #4465 from tausbn/python-remove-essa-flow
Python: Remove flow between ESSA variables
2020-10-14 15:37:39 +02:00
Rasmus Wriedt Larsen
61ecec7d17 Merge pull request #4467 from tausbn/python-fix-import-type-tracking
Python: Fix unwanted module type tracking
2020-10-14 13:08:57 +02:00
Taus Brock-Nannestad
f3c07e3849 Python: Fix up import helper tests 2020-10-14 11:58:14 +02:00
Taus Brock-Nannestad
7d86b53b71 Python: Fix unwanted module type tracking 2020-10-13 22:47:57 +02:00
Taus Brock-Nannestad
76e5b59dab Python: Add test case for unwanted module type tracking 2020-10-13 22:47:03 +02:00
Taus Brock-Nannestad
fdb489fc93 Python: Remove flow between ESSA variables
This required a minor change in the type tracker implementation, but
apart from that no other changes appear to be needed. Seems to clean
up the test output quite a bit.
2020-10-13 16:35:41 +02:00
Rasmus Wriedt Larsen
76c9b8c49f Python: Expose importNode instead of importModule/importMember
Since predicate name `import` is not allowed, I adopted `importNode` as it sort
of matches what `exprNode` does.

---

Due to only using `importMember` in `os_attr` we previously didn't handle
`import os.path as alias` :|

I did creat a hotfix for this (https://github.com/github/codeql/pull/4446), but
in doing so I realized the core of the problem: We're exposing ourselves to
making these kinds of mistakes by having BOTH importModule and importMember, and
we don't really gain anything from doing this!

We do loose the ability to easily only modeling `from mod import val` and not
`import mod.val`, but I don't think that will ever be relevant.

This change will also make us to recognize some invalid code, for example in

    import os.system as runtime_error

we would now model that `runtime_error` is a reference to the `os.system`
function (although the actual import would result in a runtime error).

Overall these are tradeoffs I'm willing to make, as it does makes things simpler
from a QL modeling point of view, and THAT sounds nice 👍
2020-10-13 15:03:22 +02:00
Rasmus Wriedt Larsen
4bfd55f1af Python: Show problem with os.path modeling
This is not a very good test for showing that we don't handle direct imports,
but it was the best I had available without inventing something new. It's very
fragile, since any of these would propagate taint (due to handling all `join`
calls as if the qualifier was a string):

    ospath_alias.join(ts)
    ospath_alias.join(ts, "foo", "bar")

But this test DOES serve the purpose of illustrating that my fix works :D
2020-10-13 14:50:00 +02:00
CodeQL CI
d3f8fb5e53 Merge pull request #4423 from tausbn/python-add-attribute-access-interface
Approved by RasmusWL
2020-10-13 02:56:21 -07:00
Anders Schack-Mulligen
091e3a2931 Dataflow: Adjust test output. 2020-10-09 16:25:14 +02:00
Taus Brock-Nannestad
d46453caaa Python: Support named imports as attribute reads
Required a small change in `DataFlow::importModule` to get the desired
behaviour (cf. the type trackers defined in `moduleattr.ql`, but this
should be harmless. The node that is added doesn't have any flow
anywhere.
2020-10-08 18:08:55 +02:00
Taus Brock-Nannestad
b905a3d5e3 Python: Attribute access API 2020-10-06 16:36:29 +02:00
CodeQL CI
5bc7e19c44 Merge pull request #4414 from yoff/SharedDataflow_Conditionals
Approved by RasmusWL
2020-10-06 05:46:24 -07:00
Rasmus Lerchedahl Petersen
f9c5b864bb Python: Fix test of parenthesized form 2020-10-06 13:12:12 +02:00
Rasmus Lerchedahl Petersen
0f077f5d7d Python: Add flow inside IfExprNodes 2020-10-06 10:54:23 +02:00
Rasmus Lerchedahl Petersen
8f13d586b7 Python: More tests of conditonals
Also use better formatter
(better because comments are close to what they comment)
2020-10-06 10:49:15 +02:00
Rasmus Wriedt Larsen
fbe115c046 Python: Show TypeTracking doesn't work for module members 2020-10-06 03:12:39 +02:00
Rasmus Wriedt Larsen
1595fed2d6 Python: Add preliminary taint tests for pathlib 2020-09-30 11:44:37 +02:00
Rasmus Wriedt Larsen
0542c3b91e Python: Model os.path.join and add taint-step 2020-09-30 11:42:36 +02:00
Rasmus Wriedt Larsen
efa2484718 Python: Add taint test for os.path.join
Surprisingly the first two just worked, due to our very general handling of any
`join` methods :D
2020-09-30 11:35:21 +02:00
Rasmus Wriedt Larsen
aa6fad558c Python: Minor cleanup in taint-step tests 2020-09-30 11:15:53 +02:00
Taus
fc84286b56 Merge pull request #3830 from yoff/SharedDataflow_FieldFlow
Python: Shared dataflow: Field flow
2020-09-25 14:53:57 +02:00
Rasmus Lerchedahl Petersen
4621e6d8c0 Python: fix QL format 2020-09-25 13:37:39 +02:00
Rasmus Lerchedahl Petersen
88bba46698 Python: Modify tests based on review
The extra hist in `test.py` seen in `globalStep.expected`
are due to the removal of manual filtering code.
(That code was from when dataflow had many strange things in it.)
2020-09-25 13:35:30 +02:00
yoff
c56ff986d4 Apply suggestions from code review
Co-authored-by: Taus <tausbn@github.com>
2020-09-25 11:56:50 +02:00
Rasmus Wriedt Larsen
0265f26301 Python: Add importModule and importMember DataFlow helpers 2020-09-22 16:28:19 +02:00
Rasmus Lerchedahl Petersen
3e2331c87f Merge branch 'main' of github.com:github/codeql into SharedDataflow_FieldFlow 2020-09-22 13:32:36 +02:00
Rasmus Lerchedahl Petersen
08b51e67c4 Python: Update test annotation 2020-09-21 17:44:36 +02:00
Rasmus Lerchedahl Petersen
73d2d9b1f8 Python: Make constructor calls post-update nodes 2020-09-21 17:32:22 +02:00
Taus
724baaf26a Merge pull request #4308 from RasmusWL/python-private-import-of-DataFlowPrivate
Python: Make import of DataFlowPrivate private
2020-09-21 17:13:48 +02:00
Rasmus Wriedt Larsen
2f9f51dbd8 Python: Fix tests that use DataFlowPrivate 2020-09-21 16:08:17 +02:00
Taus
9d7a2d2b5d Merge branch 'main' into python-add-global-flow-steps 2020-09-21 13:50:20 +02:00
Rasmus Lerchedahl Petersen
9aa0cfb35c Python: class callable -> class call
Only have one type of callable, but have an extra type of call.
A constructor call directs to an init callable
(should also handle `call` overrides at some point).
2020-09-19 22:27:11 +02:00
Rasmus Lerchedahl Petersen
b2f1c435a8 Python: update test expectations 2020-09-19 22:27:11 +02:00
Rasmus Lerchedahl Petersen
e132361736 Python: Add missing .expected file 2020-09-19 22:27:11 +02:00
Rasmus Lerchedahl Petersen
e50b66554d Python: Add explorative test 2020-09-19 22:27:10 +02:00
Rasmus Lerchedahl Petersen
aa28167177 Python: Add malloc nodes 2020-09-19 22:27:10 +02:00
Rasmus Lerchedahl Petersen
27b25565ca Python: Implement field-stores, -reads, and -content 2020-09-19 22:27:10 +02:00
Rasmus Lerchedahl Petersen
a2d006fe47 Python: Tests for field flow 2020-09-19 22:27:10 +02:00
Taus Brock-Nannestad
11c85f0fb5 Python: Clean up various jump/local data flow steps
Removes steps from `ModuleVariableNode`s from `essaFlowStep`, and
instead puts them only in `jumpStep`. This cleans up the logic a bit.

This slightly broke the type tracker implementation (as it relied on
`essaFlowStep` being fairly liberal), so I have rewritten it to
explicitly rely on just familiar predicates for local and jump steps.

Additionally, we disallow Essa-to-Essa steps where exactly one of the
two nodes corresponds to a global variable (i.e. only local-local and
global-global steps).
2020-09-18 18:14:47 +02:00
Taus Brock-Nannestad
f93c44a688 Python: Fix typo 2020-09-17 13:26:55 +02:00
Taus Brock-Nannestad
1d462ae156 Python: Fix misnamed variable. 2020-09-17 13:22:27 +02:00
Taus Brock-Nannestad
797ac23db7 Python: Clean up global flow test 2020-09-17 13:20:58 +02:00
Taus Brock-Nannestad
9458861b18 Python: Add missing global flow test 2020-09-17 12:04:30 +02:00
Taus Brock-Nannestad
ee76d9b33d Python: Clean up tests 2020-09-16 19:04:27 +02:00
Taus
e179df7c43 Python: Apply suggestions from code review
Co-authored-by: Rasmus Wriedt Larsen <rasmuswriedtlarsen@gmail.com>
2020-09-16 18:21:50 +02:00
Taus
4b423feef9 Merge pull request #4245 from RasmusWL/python-dataflow-dynamic-tuple-tests
Python: Add dataflow tests for dynamic tuple creation
2020-09-16 15:19:41 +02:00
Taus Brock-Nannestad
2d3e23ebb0 Python: Cleanup, docs, and an extra test case 2020-09-16 14:46:04 +02:00
Rasmus Lerchedahl Petersen
e46ae9b98d Python: Move some query predicates to debug 2020-09-15 21:45:47 +02:00
Taus Brock-Nannestad
7cdd290b90 Python: Disregard module-time reads. 2020-09-15 18:25:24 +02:00