Commit Graph

932 Commits

Author SHA1 Message Date
Taus
cfb84be7b1 Merge pull request #2540 from RasmusWL/python-modernise-variables-queries
Python: modernise variables queries
2020-01-10 14:45:12 +01:00
Rasmus Wriedt Larsen
81e27aab8d Python: Modernise py/unused-loop-variable 2019-12-20 15:05:49 +01:00
Taus
52d231c219 Merge pull request #2469 from RasmusWL/python-modernise-twisted-library
Python: modernise twisted library
2019-12-18 13:55:50 +01:00
Rasmus Wriedt Larsen
4e3c183676 Python: Adapt twisted tests so they pass 2019-12-18 10:42:39 +01:00
Rasmus Wriedt Larsen
e257ba40c4 Python: Make zope web tests pass 2019-12-17 17:42:03 +01:00
Rasmus Wriedt Larsen
3e5e14a14b Merge pull request #2431 from tausbn/python-cyclic-import-future-annotations
Python: Account for non-evaluation of annotations in cyclic imports.
2019-11-27 13:31:53 +01:00
Taus Brock-Nannestad
036e0f75c8 Python: Account for non-evaluation of annotations in cyclic imports.
Should fix #2426.

Essentially, we disregard expressions used inside annotations, if these
annotations occur in a file that has `from __future__ import annotations`, as
this prevents the annotations from being evaluated.
2019-11-25 15:32:52 +01:00
Rebecca Valentine
a8204385c3 Adds fix for __init_subclass__ bug. (#2390)
* Adds fix for __init_subclass__ bug.

* Adds test case.

* Move test on name.

I think it makes more sense here, alongside the other "special" method names.
2019-11-24 12:18:17 +01:00
Rasmus Wriedt Larsen
46b6e6d722 Merge pull request #2409 from tausbn/python-typing-forward-reference-fp
Python: Support forward references inside return type annotations.
2019-11-22 11:18:04 +01:00
Taus Brock-Nannestad
033524ce63 Python: Support forward references inside return type annotations.
Should fix #2407.

Also allows for the string containing the forward reference to appear inside a
subexpression of the type annotation.
2019-11-21 15:37:32 +01:00
Taus Brock-Nannestad
9fda4ab480 Python: Fix false positive in py/non-iterator-in-for-loop
Should fix #1833, #2137, and #2187.

Internally, comprehensions are (at present) elaborated into local functions and
iterators as described in [PEP-289](https://www.python.org/dev/peps/pep-0289/).
That is, something like:

```
g = (x**2 for x in range(10))
```

becomes something akin to

```
def __gen(exp):
    for x in exp:
        yield x**2
g = __gen(iter(range(10)))
```

In the context of the top-level of a class, this means `__gen` looks as if it is
a method of the class, and in particular `exp` looks like it's the `self`
argument of this method, which leads the points-to analysis to think that `exp`
is an instance of the surrounding class itself.

The fix in this case is pretty simple: we look for occurrences of `exp` (in fact
called `.0` internally -- carefully chosen to _not_ be a valid Python
identifier) and explicitly exclude this parameter from being classified as a
`self` parameter.
2019-11-21 11:49:29 +01:00
Rasmus Wriedt Larsen
b39bcde31c Merge pull request #2375 from tausbn/python-fix-mutable-value-type-coercion-fp
Python: Don't report mutable parameters that are in fact immutable.
2019-11-19 13:26:23 +01:00
Rasmus Wriedt Larsen
231414ceaf Merge pull request #2374 from tausbn/python-fix-mappingproxytype-fp
Python: Fix non-container FP relating to `MappingProxyType`.
2019-11-19 13:13:26 +01:00
Taus
4c700882b6 Merge pull request #2190 from RasmusWL/python-modernise-tornado-library
Python: modernise tornado library
2019-11-19 09:36:30 +01:00
Taus Brock-Nannestad
1385f3c018 Python: Fix non-container FP relating to MappingProxyType.
Fixes #2307.

Also modernises the query to use the `Value` API.
2019-11-18 16:50:32 +01:00
Taus Brock-Nannestad
cac261858c Python: Don't report mutable parameters that are in fact immutable.
Fixes #1832.

In the taint sink, we add an additional check that the given control-flow node
can indeed point to a value that is mutable. This takes care of the guard on the
type.

If and when we get around to adding configurations for all of the taint
analyses, we may want to implement this as a barrier instead, pruning any steps
that go through a type test where the type is not mutable.
2019-11-18 16:18:44 +01:00
Taus
78109db243 Merge pull request #2181 from RasmusWL/python-modernise-pyramid-library
Python: modernise pyramid library
2019-11-15 15:05:44 +01:00
Rasmus Wriedt Larsen
54246660c6 Python: Add test-case to password_in_cookie 2019-11-12 10:36:12 +01:00
Rasmus Wriedt Larsen
8476bc7d42 Python: correctly handle flask.make_response
Fixes https://github.com/Semmle/ql/issues/1572

Adjust mock so it's more aligned with what the flask code actually does. Tests
were passing before, even though we didn't handle the case in real code :\
2019-11-11 17:24:36 +01:00
Rasmus Wriedt Larsen
9151a7e433 Python: Always enable legacy taint tracking configuration
If the legacy configuration is only enabled if there are no other
configurations, defining a configuration in an imported library can lead to
unwanted results. For example, code that uses `any(MyTaintKind t).taints(node)`
would *stop* working, if it did not define its own configuration. (this actually
happened to us)

We performed a dist-compare to ensure there is not a performance deg ration by
doing this. Results at https://git.semmle.com/gist/rasmuswl/a1eca07f3a92f5f65ee78d733e5d260e

Tests that were affected by this:

- RockPaperScissors + Simple: new edges because no configuration was defined for
  SqlInjectionTaint or CommandInjectionTaint
- CleartextLogging + CleartextStorage: new edges because no configuration was
  defined before, AND duplicate deges.
- TestNode: new edges because no configuration was defined before

- PathInjection: Duplicate edges
- TarSlip: Duplicate edges
- CommandInjection: Duplicate edges
- ReflectedXss: Duplicate edges
- SqlInjection: Duplicate edges
- CodeInjection: Duplicate edges
- StackTraceExposure: Duplicate edges
- UnsafeDeserialization: Duplicate edges
- UrlRedirect: Duplicate edges
2019-11-11 11:17:21 +01:00
Taus
e9336fe30e Merge pull request #2129 from RasmusWL/python-update-django
Python: update django support
2019-11-05 20:51:55 +01:00
Rasmus Wriedt Larsen
ca22ec6104 Merge pull request #2042 from tausbn/python-fix-unused-import-fps
Python: Fix false positives in `py/unused-import`.
2019-11-04 14:47:30 +01:00
Taus Brock-Nannestad
5e62da7690 Python: Do not report unreachable "catch-all" cases in elif-chains.
This was brought up on the LGTM.com forums here:
https://discuss.lgtm.com/t/warn-when-always-failing-assert-is-reachable-rather-than-unreachable/2436

Essentially, in a complex chain of `elif` statements, like

```python
if x < 0:
    ...
elif x >= 0:
    ...
else:
    ...
```

the `else` clause is redundant, since the preceding conditions completely
exhaust the possible values for `x` (assuming `x` is an integer). Rather than
promoting the final `elif` clause to an `else` clause, it is common to instead
raise an explicit exception in the `else` clause. During execution, this
exception will never actually be raised, but its presence indicates that the
preceding conditions are intended to cover all possible cases.

I think it's a fair point. This is a clear instance where the alert, even if it
is technically correct, is not useful for the end user.

Also, I decided to make the exclusion fairly restrictive: it only applies if
the unreachable statement is an `assert False, ...` or `raise ...`, and only
if said statement is the first in the `else` block. Any other statements will
still be reported.
2019-10-29 15:30:32 +01:00
Rasmus Wriedt Larsen
fc851b46c3 Python: Fix Django class-based views 2019-10-29 13:58:07 +01:00
Rasmus Wriedt Larsen
fb864b7262 Python: Consolidate tests for django
The tests in 3/ was not Python 3 specific anymore
2019-10-29 13:58:07 +01:00
Rasmus Wriedt Larsen
471318369b Python: Don't quote %s in django example
This is vulnerable to SQL injection because of the quotes around %s -- added
some code that highlights this in test.py

Since our examples did this in the safe query, I ended up rewriting them
completely, causing a lot of trouble for myself :D
2019-10-29 13:58:07 +01:00
Rasmus Wriedt Larsen
f1004b10ba Merge pull request #2147 from tausbn/python-cyclic-import-package-fp
Python: Fix cyclic import FP relating to packages.
2019-10-25 11:57:55 +02:00
Rasmus Wriedt Larsen
bc50e90f5b Python: Use mock for tornado tests 2019-10-24 15:01:40 +02:00
Rasmus Wriedt Larsen
2874c54133 Python: Move pyramid tests from internal repo
Use minimal mock instead of full library
2019-10-23 16:28:46 +02:00
Rasmus Wriedt Larsen
a98466392d Python: Improve tests and docs for py/iter-returns-non-iterator 2019-10-23 10:46:07 +02:00
Taus Brock-Nannestad
32de65c0c6 Python: Add discussed test case (a false negative). 2019-10-22 15:10:40 +02:00
Taus Brock-Nannestad
83bf54c524 Python: Move false positive (now a true negative) into subfolder. 2019-10-22 15:08:29 +02:00
Taus Brock-Nannestad
70d9d1bd0e Python: Add false positive test case for cyclic import. 2019-10-18 14:03:23 +02:00
semmle-qlci
ff5a98b260 Merge pull request #2074 from taus-semmle/python-unreachable-nonlocal
Approved by RasmusWL
2019-10-07 15:45:24 +01:00
semmle-qlci
e36e16af48 Merge pull request #2079 from taus-semmle/python-unused-local-nonlocal
Approved by RasmusWL
2019-10-07 15:38:21 +01:00
Rasmus Wriedt Larsen
3f45d8614b Merge pull request #2047 from taus-semmle/python-modernise-and-fix-cyclic-import-fp
Python: modernise and fix cyclic import false positive.
2019-10-07 14:28:36 +02:00
Taus Brock-Nannestad
5946a4a066 Python: Teach py/unused-local-variable about nonlocal. 2019-10-03 17:56:29 +02:00
AlexTereshenkov
3e6f8fb6be Add bind-socket-all-network-interfaces Python query (#2048)
Add bind-socket-all-network-interfaces Python query
2019-10-03 11:23:11 +01:00
Taus Brock-Nannestad
384013e0dc Python: Add tests for reachability when using nonlocal. 2019-10-02 17:13:00 +02:00
Taus
fb20cab4c8 Merge pull request #2012 from RasmusWL/python-modernise-cls-self-checks
Python: modernise cls self argument name checks
2019-09-30 15:50:32 +02:00
Taus Brock-Nannestad
aa16d20d5a Python: Fix false positive for cyclic imports guarded by if False:. 2019-09-27 15:22:12 +02:00
Taus Brock-Nannestad
25985e901b Python: Remove a few false positives from py/unused-import. 2019-09-27 11:46:59 +02:00
Rasmus Wriedt Larsen
457794e030 Python: Consistenly use parameter instead of argument in docs
The Python 3 FAQ states that this is the right thing [0]

It sadly doesn't align 100% with PEP8, which calls them for "arguments" [1], but
after discussion with Taus, we decided to go with "parameter" everywhere to be
consistent.

[0] https://docs.python.org/3/faq/programming.html#faq-argument-vs-parameter
[1] https://www.python.org/dev/peps/pep-0008/#function-and-method-arguments
2019-09-26 16:31:09 +02:00
Rasmus Wriedt Larsen
12c49031e8 Python: Modernise bottle library 2019-09-26 15:03:47 +02:00
Rasmus Wriedt Larsen
546405a379 Python: Add more tests for cls/self argument names 2019-09-26 13:25:14 +02:00
Rasmus Wriedt Larsen
c6d9eb9254 Python: Move more tests for argument names into own file
Plus fixup of expected output from unrelated tests
2019-09-26 13:25:14 +02:00
Rasmus Wriedt Larsen
d273974045 Python: Don't flag return procedure_call() in __init__ as error
This commit fixes the results for
0d8a429b7e/files/mayaTools/cgm/lib/classes/AttrFactory.py (L90)

```
def __init__(...):
    if error_case:
        return guiFactory.warning(...)
```

that was wrongly reporting _Explicit return in __init__ method._ as an error.
2019-09-23 11:22:55 +02:00
Rasmus Wriedt Larsen
6e50a0ef84 Python: Modernise the py/explicit-return-in-init query.
Add explicit test case to show that we don't doulbe report this problem.
2019-09-23 11:22:55 +02:00
Rasmus Wriedt Larsen
3c33e863ad Python: split tests for Functions into more files
Makes it easier to see what the testcases are relevant for what queries.
2019-09-19 11:54:28 +02:00
Taus Brock-Nannestad
d336140c19 Python: Modernise the py/non-iterable-in-for-loop query.
Also adds a small test case exhibiting the same false positive seen in
ODASA-8042.
2019-09-05 12:24:51 +02:00