Commit Graph

2409 Commits

Author SHA1 Message Date
Rasmus Wriedt Larsen
060720aae7 Python: Add tests for all SystemCommandExecution from stdlib
Overall idea is that `test/experimental/meta/ConceptsTest.qll` will set up
inline expectation tests for all the classes defined in `Concepts.qll`, so any
time you model a new instance of Concepts, you simply just import that
file. That makes the tests a little verbose, but allows us to share test-setup
between all the different frameworks we model.

Note that since the definitions of SystemCommandExecution subclasses are
scattered across multieple framework modeling qll files, it think it makes the
most sense to have the tests for each framework in one location.

I'm not 100% convinced about if this is the right choice or not (especially when
we want to write tests for sanitizers), but for now I'm going to try it out at
least.
2020-09-28 11:05:32 +02:00
Rasmus Wriedt Larsen
2acfd4cdb1 Python: Show we're able to handle example with __init__.py files 2020-09-25 18:28:31 +02:00
Taus
fc84286b56 Merge pull request #3830 from yoff/SharedDataflow_FieldFlow
Python: Shared dataflow: Field flow
2020-09-25 14:53:57 +02:00
Rasmus Lerchedahl Petersen
4621e6d8c0 Python: fix QL format 2020-09-25 13:37:39 +02:00
Rasmus Lerchedahl Petersen
88bba46698 Python: Modify tests based on review
The extra hist in `test.py` seen in `globalStep.expected`
are due to the removal of manual filtering code.
(That code was from when dataflow had many strange things in it.)
2020-09-25 13:35:30 +02:00
yoff
c56ff986d4 Apply suggestions from code review
Co-authored-by: Taus <tausbn@github.com>
2020-09-25 11:56:50 +02:00
Rasmus Wriedt Larsen
85607fe2d5 Python: Adjust location for .expected output 2020-09-25 11:56:45 +02:00
Rasmus Wriedt Larsen
3d5511221e Python: Add test for implicit __init__.py files 2020-09-25 11:48:38 +02:00
Rasmus Wriedt Larsen
120a569c6f Python: Explain how CallGraph test.py even works
Also remove options file, since it did nothing at all (and blocked
experimental/library-tests/options from taking effect)
2020-09-25 11:42:59 +02:00
CodeQL CI
4deb43f361 Merge pull request #4323 from RasmusWL/python-new-command-injection-query
Approved by tausbn
2020-09-25 02:39:46 -07:00
Rasmus Wriedt Larsen
66815c9d3d Python: Suppress unused variable warnings in DataFlowPrivate 2020-09-23 14:33:10 +02:00
Rasmus Wriedt Larsen
6aec2ec673 Python: Fix os.popen modeling
Co-authored-by: yoff <lerchedahl@gmail.com>
2020-09-23 11:18:32 +02:00
Rasmus Wriedt Larsen
624cdd339a Python: Fix grammar
Co-authored-by: yoff <lerchedahl@gmail.com>
2020-09-23 11:18:12 +02:00
Rasmus Lerchedahl Petersen
ef4461ce54 Python: Address review comments 2020-09-22 23:48:28 +02:00
yoff
aece0ff652 Apply suggestions from code review
Co-authored-by: Taus <tausbn@github.com>
2020-09-22 22:33:46 +02:00
Rasmus Wriedt Larsen
71a75ce596 Python: Handle bound methods in flask modeling 2020-09-22 16:33:35 +02:00
Rasmus Wriedt Larsen
5709189c2a Python: Expand flask test 2020-09-22 16:33:34 +02:00
Rasmus Wriedt Larsen
e614365963 Python: Adopt new approach in flask modeling
Removed all the dict-like stuff, not sure that is how we should do things.
2020-09-22 16:33:33 +02:00
Rasmus Wriedt Larsen
a82fa04d8a Python: Add worked example of taint step modeling of external libs
This can't be seen on the example, but I went through quite a lot of iterations
before arriving at this fairly simple solution.
2020-09-22 16:28:26 +02:00
Rasmus Wriedt Larsen
00ea0cebc3 Python: More Flask modeling kinda works
It "kinda" works now, but it really is not a pretty solution. Adding all these
"tracked" objects is SUPER annoying... it _would_ be possible to skip them, but
that seems like it will give the wrong edges for dataflow/taintflow queries :|

A good chunk of it should be able to be removed with access-paths like C# does
for library modeling. Some of it could be solved by better type-tracking API
like API Graphs... but it seems like we generally are just lacking the
nice-to-have features like `.getAMemberCall` and the like. See
https://github.com/github/codeql/pull/4082/files#diff-9aa94c4d713ef9d8da73918ff53db774L33
2020-09-22 16:28:25 +02:00
Rasmus Wriedt Larsen
3c08590ee4 Python: Expand flask tests a bit 2020-09-22 16:28:24 +02:00
Rasmus Wriedt Larsen
2bdd0284dc Python: Port py-command-line-injection with new dataflow 2020-09-22 16:28:23 +02:00
Rasmus Wriedt Larsen
7c205dd3fc Python: First attempt at modeling Flask 2020-09-22 16:28:21 +02:00
Rasmus Wriedt Larsen
cdc5ca7aec Python: Model os.system and os.popen 2020-09-22 16:28:20 +02:00
Rasmus Wriedt Larsen
0265f26301 Python: Add importModule and importMember DataFlow helpers 2020-09-22 16:28:19 +02:00
Rasmus Wriedt Larsen
2551173156 Python: Update example in QLDoc for TypeTracker 2020-09-22 16:28:18 +02:00
Rasmus Lerchedahl Petersen
131cf8d2ec Python: Fix compilation error 2020-09-22 15:02:31 +02:00
Rasmus Lerchedahl Petersen
b065d8724e Python: Fixup comments after merge 2020-09-22 13:52:30 +02:00
Rasmus Lerchedahl Petersen
3e2331c87f Merge branch 'main' of github.com:github/codeql into SharedDataflow_FieldFlow 2020-09-22 13:32:36 +02:00
Tom Hvitved
71da9045e5 Java/Python: Reduce size of blockPrecedesVar 2020-09-22 11:00:26 +02:00
Rasmus Lerchedahl Petersen
08b51e67c4 Python: Update test annotation 2020-09-21 17:44:36 +02:00
Rasmus Lerchedahl Petersen
73d2d9b1f8 Python: Make constructor calls post-update nodes 2020-09-21 17:32:22 +02:00
Taus
724baaf26a Merge pull request #4308 from RasmusWL/python-private-import-of-DataFlowPrivate
Python: Make import of DataFlowPrivate private
2020-09-21 17:13:48 +02:00
Rasmus Wriedt Larsen
2f9f51dbd8 Python: Fix tests that use DataFlowPrivate 2020-09-21 16:08:17 +02:00
Rasmus Wriedt Larsen
6aca82fa82 Python: Make import of DataFlowPrivate private
Otherwise you are able to use `DataFlow::isExpressionNode` where
`isExpressionNode` is defined in `DataFlowPrivate.qll`.
2020-09-21 13:52:58 +02:00
Taus
9d7a2d2b5d Merge branch 'main' into python-add-global-flow-steps 2020-09-21 13:50:20 +02:00
Taus Brock-Nannestad
1d6558b4e8 Python: Add a bit more documentation to ModuleVariableNode 2020-09-21 11:46:18 +02:00
Rasmus Lerchedahl Petersen
9aa0cfb35c Python: class callable -> class call
Only have one type of callable, but have an extra type of call.
A constructor call directs to an init callable
(should also handle `call` overrides at some point).
2020-09-19 22:27:11 +02:00
Rasmus Lerchedahl Petersen
b2f1c435a8 Python: update test expectations 2020-09-19 22:27:11 +02:00
Rasmus Lerchedahl Petersen
e132361736 Python: Add missing .expected file 2020-09-19 22:27:11 +02:00
Rasmus Lerchedahl Petersen
e50b66554d Python: Add explorative test 2020-09-19 22:27:10 +02:00
Rasmus Lerchedahl Petersen
aa28167177 Python: Add malloc nodes 2020-09-19 22:27:10 +02:00
Rasmus Lerchedahl Petersen
27b25565ca Python: Implement field-stores, -reads, and -content 2020-09-19 22:27:10 +02:00
Rasmus Lerchedahl Petersen
a2d006fe47 Python: Tests for field flow 2020-09-19 22:27:10 +02:00
Taus Brock-Nannestad
11c85f0fb5 Python: Clean up various jump/local data flow steps
Removes steps from `ModuleVariableNode`s from `essaFlowStep`, and
instead puts them only in `jumpStep`. This cleans up the logic a bit.

This slightly broke the type tracker implementation (as it relied on
`essaFlowStep` being fairly liberal), so I have rewritten it to
explicitly rely on just familiar predicates for local and jump steps.

Additionally, we disallow Essa-to-Essa steps where exactly one of the
two nodes corresponds to a global variable (i.e. only local-local and
global-global steps).
2020-09-18 18:14:47 +02:00
Taus Brock-Nannestad
f93c44a688 Python: Fix typo 2020-09-17 13:26:55 +02:00
Taus Brock-Nannestad
1d462ae156 Python: Fix misnamed variable. 2020-09-17 13:22:27 +02:00
Taus Brock-Nannestad
797ac23db7 Python: Clean up global flow test 2020-09-17 13:20:58 +02:00
Taus Brock-Nannestad
9458861b18 Python: Add missing global flow test 2020-09-17 12:04:30 +02:00
Taus Brock-Nannestad
ee76d9b33d Python: Clean up tests 2020-09-16 19:04:27 +02:00