Commit Graph

3069 Commits

Author SHA1 Message Date
Rasmus Lerchedahl Petersen
36d9097d03 Python: Add test for inner return 2020-11-25 07:09:27 +01:00
Rasmus Lerchedahl Petersen
34896ae0d7 Python: Fix failing test 2020-11-25 07:09:27 +01:00
Rasmus Wriedt Larsen
d88e5bdb3a Python: Model io.open as FileSystemAccess 2020-11-24 18:27:33 +01:00
Rasmus Wriedt Larsen
e39bb56078 Python: Model builtin open function better 2020-11-24 18:27:31 +01:00
Rasmus Wriedt Larsen
5af1fdd06f Python: Expand tests of open 2020-11-24 18:27:30 +01:00
Rasmus Wriedt Larsen
caf73e4b9b Python: Wrap all Stdlib modeling consistently
Some of these predicates had fallen outside the `private module Stdlib`
2020-11-24 18:27:29 +01:00
Rasmus Wriedt Larsen
aa4345ac76 Merge pull request #4710 from yoff/python-dataflow-variable-capture
Python: Dataflow, variable capture
2020-11-24 15:04:38 +01:00
yoff
215986bce5 Apply suggestions from code review
Co-authored-by: Rasmus Wriedt Larsen <rasmuswriedtlarsen@gmail.com>
2020-11-24 14:12:23 +01:00
Rasmus Lerchedahl Petersen
05d156ba0f Python: add comments 2020-11-24 14:11:14 +01:00
CodeQL CI
2277242196 Merge pull request #4692 from yoff/python-psycopg
Approved by RasmusWL
2020-11-24 10:59:04 +00:00
Rasmus Lerchedahl Petersen
80dcb8da4a Python: annotate missing flow 2020-11-24 11:55:28 +01:00
Rasmus Lerchedahl Petersen
737f4dff09 Python: update test annotations 2020-11-24 10:57:57 +01:00
Rasmus Lerchedahl Petersen
da805f8242 Merge remote-tracking branch 'upstream/main' into python-dataflow-modernize-tests 2020-11-24 10:56:22 +01:00
Rasmus Lerchedahl Petersen
a19304a4a0 Python: Factor out prettyPrinter and update tests 2020-11-24 02:17:38 +01:00
Rasmus Lerchedahl Petersen
39c5e0d487 Python: update test expectations 2020-11-23 16:46:35 +01:00
Rasmus Lerchedahl Petersen
38bb06a207 Merge remote-tracking branch 'upstream/main' into python-dataflow-variable-capture 2020-11-23 16:40:20 +01:00
Rasmus Wriedt Larsen
f35ffa5632 Python: Add support for x in ["safe", "also_safe"] (and not in) 2020-11-23 10:42:24 +01:00
Rasmus Wriedt Larsen
431aab45f7 Python: Add support for x != "safe" BarrierGuard 2020-11-23 10:36:55 +01:00
Rasmus Wriedt Larsen
18041fd059 Python: Expand string-const-compare tests
Also moved file to reflect that. Added tests of

+ `!=`
+ `in`
+ `not in`
2020-11-23 10:36:49 +01:00
Rasmus Lerchedahl Petersen
777100f25c Python: rename file, package, and class 2020-11-23 09:17:40 +01:00
Henning Makholm
a2a4938f60 Remove unit tests for duplicate-code detection, take II
In #4689 I forgot to remove the `.expected` files too, but they are
now of course useless.
2020-11-20 21:07:42 +01:00
Rasmus Wriedt Larsen
08bcba98e6 Python: Add BarrierGuard test with exception inside unsafe branch 2020-11-20 11:55:07 +01:00
Rasmus Wriedt Larsen
34f78d4211 Python: Add BarrierGuard test with return inside unsafe branch 2020-11-20 11:52:36 +01:00
yoff
b478a51d4e Apply suggestions from code review
Thanks for doing the work for me :-)

Co-authored-by: Rasmus Wriedt Larsen <rasmuswriedtlarsen@gmail.com>
2020-11-20 10:46:04 +01:00
Rasmus Wriedt Larsen
6c8937c5a9 Python: Add StringConstCompare to new data-flow queries
In the future, I could imagine we would have something like this, but for now,
I'm just keeping it simple.

```codeql
  /**
   * A collection of common guards that ensure the checked value cannot have arbitrary
   * values.
   *
   * Currently only supports comparison with constant string value, but could also
   * include checking whether all characters are alphanumeric, or whether a regex is
   * matched against the value.
   *
   * Such guards will be useful for many taint-tracking queries, but not necessarily
   * all, which is why you need to opt into these manually.
   */
  class CommonNonArbitraryGuard extends BarrierGuard {
    CommonNonArbitraryGuard() {
      this instanceof StringConstCompare
    }

    override predicate checks(ControlFlowNode node, boolean branch) {
      this.(StringConstCompare).checks(node, branch)
    }
  }
```
2020-11-20 10:44:50 +01:00
Rasmus Wriedt Larsen
12b36b2245 Python: Highlight that safe or also_safe doesn't clear taint :( 2020-11-20 10:43:46 +01:00
Rasmus Wriedt Larsen
1a52f17da3 Python: Add StringConstCompare BarrierGuard 2020-11-20 10:40:04 +01:00
Rasmus Wriedt Larsen
a82936c904 Python: Enable test for sanitizer-guard and logic expressions
`not` is not working properly, but otherwise pretty good
2020-11-20 10:06:25 +01:00
Rasmus Wriedt Larsen
b3d3d6e142 Python: Move logical test of sanitizers
Don't know why it would ever have been under default sanitizers :D
2020-11-19 16:46:07 +01:00
Rasmus Lerchedahl Petersen
7cbbf3bbf7 Python: slightly nicer test 2020-11-19 16:20:57 +01:00
Rasmus Wriedt Larsen
4c7c940273 Python: Add example of Code Injection FP 2020-11-19 15:05:51 +01:00
Rasmus Wriedt Larsen
7e407d43d2 Python: Change (single) test to match codeql database create 2020-11-19 14:56:18 +01:00
Rasmus Wriedt Larsen
8ffcff0824 Python: Add example of top-level module shadowing stdlib
Although this test is added under the `wrong` folder, the current results from
this CodeQL test is actually correct (compared with the Python
interpreter). However, they don't match what the extractor does when invoked
with `codeql database create`.

Since I deemed it "more than an easy fix" to change the extractor behavior for
`codeql database create` to match the real python behavior, and it turned out to
be quite a challenge to change the extractor behavior for all tests, I'm just
going to make THIS ONE test-case behave like the extractor will with `codeql
database create`...

This is a first commit, to show how the extractor works with qltest by default.

Inspired by the debugging in https://github.com/github/codeql/issues/4640
2020-11-19 14:56:17 +01:00
Rasmus Lerchedahl Petersen
6cc8e5acf1 Python: support psycopg 2020-11-19 12:13:20 +01:00
Rasmus Lerchedahl Petersen
39f134c1c1 Python: reorganized and added to tests 2020-11-19 12:06:58 +01:00
Henning Makholm
a86679a377 Remove unit tests for duplicate-code detection
The old Semmle duplicate-code detection code has never been done when
extracting databases for the CodeQL CLI, except that `codeql test run`
will run it _just_ in order to support tests of the feature. With the
sunsetting of Odasa there's no need to even _test_ the feature anymore.

This commit removes those tests that fail when the duplicate-code
detector is turned off. Once it is merged and bumped, we can finally
remove it from `codeql`.
2020-11-18 16:37:29 +01:00
Rasmus Wriedt Larsen
ab856d6c01 Python: Show getCallableForArgument can have multiple results 2020-11-18 10:44:32 +01:00
Rasmus Wriedt Larsen
abf2902a69 Python: Fix QLDoc
Co-authored-by: yoff <lerchedahl@gmail.com>
2020-11-18 09:47:23 +01:00
Rasmus Wriedt Larsen
39590a39cb Python: Rename helper predicate based on review 2020-11-18 09:26:53 +01:00
Rasmus Wriedt Larsen
14136154d6 Python: Fix bad join order in TypeTracker::callStep
From a local evaluation against flask DB, after
https://github.com/github/codeql/pull/4649 was merged we would get:

```
Tuple counts for TypeTracker::callStep#ff/2@a21b71:
9876     ~0%     {3} r1 = SCAN DataFlowPrivate::DataFlowCall::getArg_dispred#fff AS I OUTPUT I.<2>, I.<0>, I.<1>
9876     ~2%     {3} r2 = JOIN r1 WITH project#DataFlowPrivate::DataFlowCall::getArg_dispred#fff AS R ON FIRST 1 OUTPUT r1.<2>, R.<0>, r1.<1>
72388997 ~0%     {4} r3 = JOIN r2 WITH DataFlowPublic::ParameterNode::isParameterOf_dispred#fff_201#join_rhs AS R ON FIRST 1 OUTPUT r2.<2>, R.<2>, r2.<1>, R.<1>
4952     ~0%     {2} r4 = JOIN r3 WITH DataFlowPrivate::DataFlowCall::getCallable_dispred#ff AS R ON FIRST 2 OUTPUT r3.<2>, r3.<3>
                     return r4
```
2020-11-18 09:17:31 +01:00
Anders Schack-Mulligen
f74fc0ff26 Dataflow: Fix bad join-orders. 2020-11-17 14:28:25 +01:00
Rasmus Lerchedahl Petersen
71830abda0 Python: remaining c# tests, except lambdas
both via nonlocal and via dict
2020-11-17 08:28:11 +01:00
Rasmus Lerchedahl Petersen
27b4c67b9f Python: Start of tests for captured variables 2020-11-16 17:25:39 +01:00
Anders Schack-Mulligen
3dbd48063c Dataflow: Add Unit type for all languages. 2020-11-16 09:02:44 +01:00
Anders Schack-Mulligen
9e45f10c5d Dataflow: Remove headUsesContent. 2020-11-13 15:12:39 +01:00
Anders Schack-Mulligen
e0a6a485df Dataflow: Sync. 2020-11-13 15:12:16 +01:00
Rasmus Wriedt Larsen
9f1d8cd1bb Python: Convert indentation to spaces for VS Code snippets 2020-11-13 13:05:23 +01:00
Rasmus Wriedt Larsen
5200af5244 Python: Add code snippets for VS Code
Notice that in this form, the filename doesn't matter, and you need to specify
`scope` to limit the snippet to only trigger for `ql`.
2020-11-13 10:57:17 +01:00
Rasmus Lerchedahl Petersen
dc91406ff0 Python: make .expected empty
still need to annotate missing results
2020-11-13 09:22:57 +01:00
Rasmus Lerchedahl Petersen
e468d49b19 Python: routing tests 3-7 and some annotations 2020-11-12 23:07:01 +01:00