Commit Graph

510 Commits

Author SHA1 Message Date
Rasmus Wriedt Larsen
f35ffa5632 Python: Add support for x in ["safe", "also_safe"] (and not in) 2020-11-23 10:42:24 +01:00
Rasmus Wriedt Larsen
431aab45f7 Python: Add support for x != "safe" BarrierGuard 2020-11-23 10:36:55 +01:00
Rasmus Wriedt Larsen
18041fd059 Python: Expand string-const-compare tests
Also moved file to reflect that. Added tests of

+ `!=`
+ `in`
+ `not in`
2020-11-23 10:36:49 +01:00
Rasmus Wriedt Larsen
08bcba98e6 Python: Add BarrierGuard test with exception inside unsafe branch 2020-11-20 11:55:07 +01:00
Rasmus Wriedt Larsen
34f78d4211 Python: Add BarrierGuard test with return inside unsafe branch 2020-11-20 11:52:36 +01:00
Rasmus Wriedt Larsen
12b36b2245 Python: Highlight that safe or also_safe doesn't clear taint :( 2020-11-20 10:43:46 +01:00
Rasmus Wriedt Larsen
1a52f17da3 Python: Add StringConstCompare BarrierGuard 2020-11-20 10:40:04 +01:00
Rasmus Wriedt Larsen
a82936c904 Python: Enable test for sanitizer-guard and logic expressions
`not` is not working properly, but otherwise pretty good
2020-11-20 10:06:25 +01:00
Rasmus Wriedt Larsen
b3d3d6e142 Python: Move logical test of sanitizers
Don't know why it would ever have been under default sanitizers :D
2020-11-19 16:46:07 +01:00
Rasmus Lerchedahl Petersen
7cbbf3bbf7 Python: slightly nicer test 2020-11-19 16:20:57 +01:00
Rasmus Lerchedahl Petersen
39f134c1c1 Python: reorganized and added to tests 2020-11-19 12:06:58 +01:00
Rasmus Wriedt Larsen
ab856d6c01 Python: Show getCallableForArgument can have multiple results 2020-11-18 10:44:32 +01:00
Rasmus Lerchedahl Petersen
71830abda0 Python: remaining c# tests, except lambdas
both via nonlocal and via dict
2020-11-17 08:28:11 +01:00
Rasmus Lerchedahl Petersen
27b4c67b9f Python: Start of tests for captured variables 2020-11-16 17:25:39 +01:00
Rasmus Lerchedahl Petersen
dc91406ff0 Python: make .expected empty
still need to annotate missing results
2020-11-13 09:22:57 +01:00
Rasmus Lerchedahl Petersen
e468d49b19 Python: routing tests 3-7 and some annotations 2020-11-12 23:07:01 +01:00
Rasmus Lerchedahl Petersen
4fe2576b9a Python: start modernizing routing tests 2020-11-12 22:43:34 +01:00
Rasmus Lerchedahl Petersen
823ed447df Python: Add new-style tests
should perhaps move `LocalFlowStepTest` and `MaximalFlowStep` into where
 they are referenced (they did not seem too reusable after all).
Should also add argument tests in the same way.
2020-11-12 21:57:29 +01:00
Rasmus Wriedt Larsen
611398586d Merge pull request #4649 from yoff/python-dataflow-cfgparameters
Python: Make `ParameterNode` a `CfgNode`
2020-11-11 10:22:12 +01:00
Anders Schack-Mulligen
89ef6ea4eb C++/C#/Java/JavaScript/Python: Autoformat set literals. 2020-11-10 13:32:27 +01:00
Rasmus Lerchedahl Petersen
109d55eb25 Python: Make ParameterNode a CfgNode
Add a step from that `CfgNode` to the corresponding `EssaNode`.
The intended effect is seen in `ImpliesDataflow.expected`.
The efeect seen in other `.expected`-files is that parameter nodes
change type, that the extra steps are seen, and that flow from
`EssaVar`s is mirrored in flow from `CfgNode`s.
There is one surprise, which is the `.0` node in
`coverage/localFlow.expected`.
2020-11-10 11:35:50 +01:00
Rasmus Lerchedahl Petersen
6cecd3ba83 Python: Move and rename query 2020-11-05 11:49:39 +01:00
Rasmus Lerchedahl Petersen
38b2bb2828 Python: Add testfile with regressions 2020-11-04 15:55:59 +01:00
Rasmus Lerchedahl Petersen
6df3b8d524 Python: Update query and expectation 2020-11-04 15:17:38 +01:00
Rasmus Lerchedahl Petersen
9baa7b73da Merge branch 'main' of github.com:github/codeql into SharedDataflow_PointsToImpliesDataflow 2020-11-04 15:05:59 +01:00
Jonas Jensen
5680b2df13 Merge remote-tracking branch 'upstream/main' into better-syntax-for-false-positives-and-negatives-inline-expectation
Required fixing up semantic conflicts in tests.

Conflicts:
	python/ql/test/experimental/library-tests/frameworks/stdlib/Decoding.py
2020-11-03 09:47:26 +01:00
Taus Brock-Nannestad
5dadb0f476 Python: Fix imports in tests 2020-11-02 23:02:29 +01:00
Rasmus Wriedt Larsen
247fd4f5f3 Python: Make encoding/decoding preserve taint automatically
With the way we have set things up, there is no way to opt out of this behavior.
2020-11-02 14:53:30 +01:00
Mathias Vorreiter Pedersen
45b24a9bc8 Python: Update inline-expectation tests 2020-10-30 16:53:33 +01:00
Rasmus Wriedt Larsen
5874a7b422 Merge pull request #4488 from yoff/SharedDataflow_ArgumentPassingTests
Python: Shared dataflow, argument passing tests
2020-10-21 12:01:37 +02:00
Rasmus Lerchedahl Petersen
9ee5a01d7e Python: Reword comment on isBarrierIn 2020-10-21 10:30:40 +02:00
yoff
17155b64f5 Merge pull request #4514 from tausbn/python-add-module-boundary-flow-steps
Python: Add module boundary flow steps
2020-10-20 14:36:10 +02:00
Taus Brock-Nannestad
a21c29507c Python: Fix false negative
I'm slightly suspicious of this fix -- it seems to work, but it makes
me wonder if we're potentially missing other kinds of flow, by not
handling other kinds of definitions.

Also, I feel like this should really be attached to an appropriate
post-update node of the given argument. As it is written now, the flow
will go from the argument _before_ the call, which obviously misses a
step if the argument is modified by the call. In practice, I would
expect this to be rather rare.
2020-10-20 13:16:54 +02:00
Taus Brock-Nannestad
860cafed4d Python: Mark failing test as false negative 2020-10-20 13:11:06 +02:00
Rasmus Wriedt Larsen
045a6c3cb5 Python: Add test for tricky module member for type-tracking
Local testing shows that the `getDefinition` result for this is a `SSA filter definition`,
and not an `AssignmentDefinition`.
2020-10-20 12:20:35 +02:00
Taus Brock-Nannestad
7755993dd3 Python: Add jump steps for module attribute reads.
This is the quick-and-dirty solution, as discussed.

An even quicker-and-dirtier solution would have used
`ModuleValue::attr` and take the `getOrigin` of that as the source of
the jump step. However, this turns out to be a bad choice, since
`attr` might fail to have a value for the given attribute (for a
variety of reasons). Thus, we instead appeal to a helper predicate
that keeps track of which names are defined by which right-hand-sides
in a given module. (Observe that type tracking works correctly for `x`
in `mymodule.py`, even though `x` is never assigned a value in the
eyes of the Value API.)

This means that points-to is only used to actually figure out if the
object we're looking an attribute up on is a module or not. This is
the next thing to replace in order to eliminate the dependence on
points-to, but this will require some care to ensure that all module
lookups are handled correctly.

Only two test files needed to be changed for the tests to pass. The
first was the fixed false negative in the type tracker, and the other
was a bunch of missing flow in the regression test. I have manually
removed the `# Flow not found` annotations to make them consistent
with the output. Pay particular attention to the annotation on line
117 -- I believe it was misplaced and should have been on line 106
instead (where, indeed, we now have flow where none appeared before).
2020-10-19 19:13:32 +02:00
Taus Brock-Nannestad
60fcb5e7d5 Python: Add PEP-328 test example
Based on https://www.python.org/dev/peps/pep-0328/#guido-s-decision

Original "code" is in the Public Domain.
2020-10-16 12:03:43 +02:00
Rasmus Lerchedahl Petersen
cced335284 Python: Switch tests to path queries 2020-10-16 09:59:30 +02:00
Tom Hvitved
27fc610c0d Python: Update expected test output 2020-10-16 09:09:06 +02:00
Rasmus Lerchedahl Petersen
41b8877579 Python: Improve test configuration 2020-10-16 08:32:05 +02:00
Rasmus Lerchedahl Petersen
9c8e968cba Python: Fix bad merge 2020-10-15 11:47:34 +02:00
Rasmus Lerchedahl Petersen
0766eef49b Merge branch 'main' of github.com:github/codeql into SharedDataflow_ArgumentPassing 2020-10-15 09:49:21 +02:00
yoff
5f6f85c998 Merge pull request #4465 from tausbn/python-remove-essa-flow
Python: Remove flow between ESSA variables
2020-10-14 15:37:39 +02:00
Rasmus Wriedt Larsen
61ecec7d17 Merge pull request #4467 from tausbn/python-fix-import-type-tracking
Python: Fix unwanted module type tracking
2020-10-14 13:08:57 +02:00
Taus Brock-Nannestad
f3c07e3849 Python: Fix up import helper tests 2020-10-14 11:58:14 +02:00
Taus Brock-Nannestad
7d86b53b71 Python: Fix unwanted module type tracking 2020-10-13 22:47:57 +02:00
Taus Brock-Nannestad
76e5b59dab Python: Add test case for unwanted module type tracking 2020-10-13 22:47:03 +02:00
Rasmus Lerchedahl Petersen
5d66c485d5 Python: IPA type for arguemnt mappings
Not sure how arg2 in line 118 is achieved
2020-10-13 19:12:52 +02:00
Taus Brock-Nannestad
fdb489fc93 Python: Remove flow between ESSA variables
This required a minor change in the type tracker implementation, but
apart from that no other changes appear to be needed. Seems to clean
up the test output quite a bit.
2020-10-13 16:35:41 +02:00
Rasmus Wriedt Larsen
76c9b8c49f Python: Expose importNode instead of importModule/importMember
Since predicate name `import` is not allowed, I adopted `importNode` as it sort
of matches what `exprNode` does.

---

Due to only using `importMember` in `os_attr` we previously didn't handle
`import os.path as alias` :|

I did creat a hotfix for this (https://github.com/github/codeql/pull/4446), but
in doing so I realized the core of the problem: We're exposing ourselves to
making these kinds of mistakes by having BOTH importModule and importMember, and
we don't really gain anything from doing this!

We do loose the ability to easily only modeling `from mod import val` and not
`import mod.val`, but I don't think that will ever be relevant.

This change will also make us to recognize some invalid code, for example in

    import os.system as runtime_error

we would now model that `runtime_error` is a reference to the `os.system`
function (although the actual import would result in a runtime error).

Overall these are tradeoffs I'm willing to make, as it does makes things simpler
from a QL modeling point of view, and THAT sounds nice 👍
2020-10-13 15:03:22 +02:00