Commit Graph

450 Commits

Author SHA1 Message Date
Rasmus Wriedt Larsen
e05d6e71b8 Merge pull request #6064 from tausbn/python-add-get-method-call
Python: Add `getAMethodCall` to `LocalSourceNode`
2021-06-22 11:16:39 +02:00
Taus
768cab3642 Python: Address review comments
- changes `getReceiver` to `getObject`
- fixes `calls` to avoid unwanted cross-talk
- adds some more documentation to highlight the above issue
2021-06-21 14:57:19 +00:00
CodeQL CI
bcafe532ac Merge pull request #5944 from RasmusWL/async-api-graph-tests
Approved by tausbn
2021-06-16 08:46:26 -07:00
Rasmus Wriedt Larsen
04db33513e Merge branch 'main' into sensitive-improvements 2021-06-10 15:11:09 +02:00
Rasmus Wriedt Larsen
ea0c1d7db3 Python: Better handling of sensitive functions
This solution was the best I could come up with, but it _is_ a bit
brittle since you need to remember to add this additional taint step
to any configuration that relies on sensitive data sources... I don't
see an easy way around this though :|
2021-06-10 15:08:21 +02:00
Rasmus Wriedt Larsen
f167143a84 Python: Use real config in TestSensitiveDataSources
This will enable better tests in just one second
2021-06-10 15:01:31 +02:00
Rasmus Wriedt Larsen
c341643ec1 Python: Add more tests for sensitive function handling 2021-06-10 14:36:05 +02:00
Rasmus Wriedt Larsen
eb4f168dd4 Python: Clarify SensitiveAttributeAccess
The comment about imports was placed wrong. I also realized we didn't
even have a single test-case for
`this.(DataFlow::AttrRead).getAttributeNameExpr() = sensitiveLookupStringConst(classification)`
so I added that (notice that this is only `getattr(foo, x)` and not
`getattr(foo, "password")`)
2021-06-10 14:09:47 +02:00
yoff
716627c1dd Merge pull request #5878 from RasmusWL/fix-concept-tests-pretty-print
Python: Fix concept tests pretty print
2021-06-10 11:21:08 +02:00
Rasmus Wriedt Larsen
aaddd36236 Python: Add another type-tracking attr test
This one just works out of the box 💪
2021-06-09 14:20:50 +02:00
Rasmus Wriedt Larsen
89cba216ca Python: Apply suggestions from code review
Co-authored-by: Taus <tausbn@github.com>
2021-06-09 14:15:59 +02:00
Rasmus Wriedt Larsen
3d2de03674 Python: Add type-tracking test for attr set in function 2021-06-08 19:03:53 +02:00
Rasmus Wriedt Larsen
dcd448b743 Python: Refactor formatting 2021-06-08 19:03:53 +02:00
Rasmus Wriedt Larsen
09de1bcf44 Python: Add tests for type-tracking attrs on instances 2021-06-08 19:03:53 +02:00
Rasmus Wriedt Larsen
3819a361b5 Python: Autoformat 2021-06-07 14:16:33 +02:00
Rasmus Wriedt Larsen
350f79e1e1 Python: Model sensitive data based on variable names 2021-06-04 11:28:07 +02:00
Rasmus Wriedt Larsen
f5fd0f8d1c Python: Model sensitive data based on parameter names 2021-06-04 11:28:07 +02:00
Rasmus Wriedt Larsen
925e67d734 Python: Model sensitive data from subscripts 2021-06-04 11:28:07 +02:00
Rasmus Wriedt Larsen
00a71a1c41 Python: Port sensitive data modeling
No longer using points-to 🎉
2021-06-04 11:28:07 +02:00
Rasmus Wriedt Larsen
3b68c87b6c Python: Add sensitive data test-cases 2021-06-03 13:38:29 +02:00
Rasmus Wriedt Larsen
c4e244eb80 Python: Add getAwaited to API::Node
I _really_ wanted to call this `.await()`, but that did not fit in with
the convention, or the corresponding `getPromised` in JS.

54f191cfe3/javascript/ql/src/semmle/javascript/ApiGraphs.qll (L184)
2021-05-21 17:11:20 +02:00
Rasmus Wriedt Larsen
2408573a0a Python: Add API graph test for calling coroutines 2021-05-21 16:08:15 +02:00
Rasmus Wriedt Larsen
7a5fd02442 Python: API graph tests: add --max-import-depth=1
Before this, I ended up extracting 454 modules locally 😱
2021-05-21 15:58:15 +02:00
Rasmus Wriedt Larsen
9a4709c134 Python: API graph tests: Disallow results outside project
Running the tests locally would result in thousands of results before
this 😱
2021-05-21 15:57:10 +02:00
Rasmus Wriedt Larsen
0292ca6b67 Merge pull request #5880 from tausbn/python-limit-builtins
Python: Limit set of globals that may be built-ins
2021-05-20 14:47:22 +02:00
CodeQL CI
17afbdf258 Merge pull request #5635 from RasmusWL/port-weak-crypto-algorithm
Approved by yoff
2021-05-20 01:22:32 -07:00
Rasmus Wriedt Larsen
61ad5d0673 Python: Allow printing PostUpdateNode in ConceptsTest.qll
See how this works in `test_json.py`
2021-05-19 17:10:33 +02:00
Rasmus Wriedt Larsen
9dbb364cca Python: Move json tests to be part of stdlib
This is better, since the modeling is also part of Stdlib.qll
2021-05-19 17:10:33 +02:00
Rasmus Wriedt Larsen
f66dccafda Python: Rename prettyExp => prettyExpr
So we're consistenly using `expr` and not leaving our the `r`.
2021-05-19 17:10:33 +02:00
Rasmus Wriedt Larsen
97fadd9970 Merge branch 'main' into port-weak-crypto-algorithm 2021-05-18 14:04:18 +02:00
Taus
fe12e620dd Python: Avoid clobbering range in test
This was an unwanted interaction between two unrelated tests, so I
switched to a different built-in in the second test. I also added a test
case that shows an unfortunate side effect of this more restricted
handling of built-ins.
2021-05-12 18:42:10 +00:00
Rasmus Wriedt Larsen
72d08f4d6e Python: Model json load/dump 2021-05-10 15:10:30 +02:00
Rasmus Wriedt Larsen
63f28d7d9b Python: Model keyword args to json loads/dumps 2021-05-10 15:10:29 +02:00
Rasmus Wriedt Larsen
784e0cdb96 Python: Improve tests of json module
Inspired by the work on previous commit
2021-05-10 15:10:28 +02:00
Rasmus Wriedt Larsen
499adc26a3 Python: Extend SensitiveDataSource tests
Now it contains all the sort of things we actually support 👍
2021-04-22 15:23:40 +02:00
Rasmus Wriedt Larsen
794a86a6b0 Python: Add SensitiveDataSource 2021-04-22 15:23:39 +02:00
Rasmus Lerchedahl Petersen
5a4e661e60 Merge branch 'main' of github.com:github/codeql into python-support-pathlib 2021-04-22 15:04:21 +02:00
CodeQL CI
30d7f0dc98 Merge pull request #5687 from RasmusWL/inline-taint-tests
Approved by yoff
2021-04-21 06:24:12 -07:00
Taus
a55b43b67e Python: Use LocalSourceNode throughout step
This commit does a lot of stuff all at once, so here are the main
highlights:

In `TypeTracker.qll`, we change `StepSummary::step` to step only between
source nodes. Because reads and writes of global variables happen in two
different (jump) steps, this requires the intermediate
`ModuleVariableNode` to _also_ be a `LocalSourceNode`, and we therefore
modify the charpred for that class accordingly. (This also means
changing a few of the tests to account for these new source nodes.)

In addition, we change `TypeTracker::step` to likewise step between
local source nodes.

Next, to enable the use of the `track` convenience method on nodes, we
add some pragmas to `TypeTracker::step` that prevent bad joins from
occurring. With this, we can eliminate all of the manual type tracker
join predicates.

Next, we observe that because `StepSummary::step` now uses `flowsTo`, it
automatically encapsulates all local-flow steps. In particular this
means we do not have to use `typePreservingStep` in `smallstep`, but can
use `jumpStep` directly. A similar observation applies to
`TypeTracker::smallstep`.

Having done this, we no longer need `typePreservingStep`, so we get rid
of it.
2021-04-20 12:59:33 +00:00
Rasmus Lerchedahl Petersen
a8280f9b12 Python: update test expectation 2021-04-16 08:25:29 +02:00
Rasmus Wriedt Larsen
3e7dc12246 Python: Port taint tests to use inline expectations
The meat of this PR is described in the new python/ql/test/experimental/meta/InlineTaintTest.qll file:

> Defines a InlineExpectationsTest for checking whether any arguments in
> `ensure_tainted` and `ensure_not_tainted` calls are tainted.
>
> Also defines query predicates to ensure that:
> - if any arguments to `ensure_not_tainted` are tainted, their annotation is marked with `SPURIOUS`.
> - if any arguments to `ensure_tainted` are not tainted, their annotation is marked with `MISSING`.
>
> The functionality of this module is tested in `ql/test/experimental/meta/inline-taint-test-demo`.
2021-04-15 18:00:33 +02:00
Rasmus Lerchedahl Petersen
3eb1813584 Python: update test expectations 2021-04-15 10:47:49 +02:00
Rasmus Wriedt Larsen
1f5e52e822 Python: Cleanup "first" type-tracking predicate to be private
Since it's exposed nicely in the version that doesn't have a
`DataFlow::TypeTracker` parameter, these should be private.

Also found one instance where I had accidentially used DataFlow::Node instead of
LocalSourceNode
2021-03-23 16:40:56 +01:00
Taus Brock-Nannestad
978200e2ad Python: Distinguish between Python 2 and 3
Also moves the filtering on `name` to before the big disjunction in
`MkModuleImport`.
2021-03-12 12:35:23 +01:00
Taus Brock-Nannestad
c7b2b719cf Python: Support builtins in API graphs 2021-03-11 23:03:18 +01:00
Taus
53711dc82f Merge pull request #5238 from RasmusWL/no-flow-default-value
Python: Highlight missing flow from default value in functions
2021-02-23 13:27:41 +01:00
Rasmus Wriedt Larsen
e160c855ad Merge pull request #5233 from yoff/python-for-tuple-iteration
Python: `for`-iteration of tuples
2021-02-22 15:28:13 +01:00
Rasmus Wriedt Larsen
5249b54a9b Python: Highlight missing flow from default value in functions
Although it is becoming non-trivial to get an overview of what tests we have and
don't have, I didn't find any that highlighted this one

I used all 3 variants of parameters, just to be sure :)
2021-02-22 14:52:51 +01:00
Rasmus Lerchedahl Petersen
d23a8ad016 Python: elide test output 2021-02-21 13:12:54 +01:00
Rasmus Lerchedahl Petersen
46faba69ff Python: Fix for-iteration of tuples 2021-02-21 12:41:16 +01:00