Commit Graph

502 Commits

Author SHA1 Message Date
semmle-qlci
ec90627a64 Merge pull request #2909 from yo-h/experimental
Approved by aschackmull, jbj, max-schaefer, tausbn
2020-02-28 03:15:58 +00:00
Taus
0da554c701 Merge pull request #2914 from RasmusWL/python-remove-optimize-true-directive
Python: Remove `--optimize: true` from options files
2020-02-27 13:16:59 +01:00
Taus
d9383d0e86 Merge pull request #2902 from RasmusWL/python-use-of-input
Python: Highlight py/use-of-input is for Python 2
2020-02-27 13:15:32 +01:00
Taus
8bd3063d2b Merge pull request #2875 from RasmusWL/python-taint-urlsplit
Python: Add taint for urlsplit
2020-02-27 13:13:47 +01:00
Taus
e09907894d Merge pull request #2817 from BekaValentine/objectapi-to-valueapi-truncateddivision
Python: ObjectAPI to ValueAPI: TruncatedDivision
2020-02-27 12:52:26 +01:00
Rebecca Valentine
b0493458d6 Combine and clean up the test files 2020-02-26 09:04:14 -08:00
Rasmus Wriedt Larsen
771dfecf6d Python: Add sanitized edges for urlsplit test 2020-02-26 14:10:30 +01:00
Rasmus Wriedt Larsen
0b31cb1716 Python: Show that we have initial taint in urlsplit test 2020-02-26 14:09:02 +01:00
Rasmus Wriedt Larsen
4330d4e289 Python: Remove unused import in test 2020-02-26 10:26:30 +01:00
Rasmus Wriedt Larsen
b213db03fd Python: Consolidate stdlib http client tests
Move the stdlib tests from test/{2,3}/library-tests/ into /test/library-tests/,
and deal with version by using sys.version_info (results should be the same for
both versions).

six tests were moved from /library-tests/web/client/stdlib => /library-tests/web/client/six
2020-02-26 10:26:30 +01:00
Rasmus Wriedt Larsen
cd5399d43e Python: Model outgoing http client requests 2020-02-26 10:26:30 +01:00
Rasmus Wriedt Larsen
f10a86d3ac Python: Remove --optimize: true from options files
Tests will be run with optimizations on by default now.
2020-02-25 15:52:00 +01:00
yo-h
43bcd5b26c Add guidelines for experimental CodeQL queries and libraries 2020-02-24 15:08:31 -05:00
Rasmus Wriedt Larsen
2b997ec94a Python: Add Python 3 Imports tests from internal repo 2020-02-24 15:36:45 +01:00
Rasmus Wriedt Larsen
9d629aef95 Python: Highlight py/use-of-input is for Python 2 2020-02-24 15:13:19 +01:00
Rebecca Valentine
14273fc677 Adds missing result to expected file 2020-02-21 11:25:03 -08:00
Rasmus Wriedt Larsen
bfa7553095 Python: urlsplit sanitizer handles in [KNOWN_VALUE] 2020-02-21 16:03:29 +01:00
Rasmus Wriedt Larsen
798db91f71 Python: Add more urlsplit tests 2020-02-21 15:51:33 +01:00
Rasmus Wriedt Larsen
31ff652cb3 Python: Make Sanitizer available for urlsplit taint
It isn't used by default, it has to *actively* be enabled.
2020-02-21 15:18:53 +01:00
Rebecca Valentine
2f3ea10cf8 Move the query and examples over to 2/query-tests 2020-02-20 16:31:58 -08:00
Rebecca Valentine
91ea46f5ee Adds test output. 2020-02-20 15:41:51 -08:00
Rebecca Valentine
115495450d Adds test cases. 2020-02-20 15:41:51 -08:00
Rebecca Valentine
96b8d78650 Adds modernized files. 2020-02-20 15:41:51 -08:00
Rasmus Wriedt Larsen
fd270cc02c Python: Add basic taint support for urlsplit/urlparse 2020-02-19 16:31:10 +01:00
Rasmus Wriedt Larsen
74345b1c05 Python: Make library-tests/taint/strings tests more transparent
Following the setup I invented for library-tests/taint/unpacking.

TestStep is still a bit annoying, since the output is not easy to eyeball; but
for now I guess we can live with it :)

I honestly didn't get the point of DistinctStringKinds.ql, other than showing we
can handle multiple taint kinds
2020-02-19 16:24:22 +01:00
Taus
ffbb5d0529 Merge pull request #2739 from RasmusWL/python-modernise-security
Python: modernise Security/ queries
2020-02-18 16:28:53 +01:00
Taus
03ae7831ad Merge pull request #2711 from RasmusWL/python-fix-import-deprecated-module
Python: fix alerts for py/import-deprecated-module
2020-02-17 11:46:12 +01:00
Taus
df3ac49c28 Merge pull request #2700 from RasmusWL/python-taint-iterable-unpacking
Python: Handle iterable unpacking in taint tracking
2020-02-17 11:44:25 +01:00
Taus
990d1c1663 Merge pull request #2802 from RasmusWL/python-fix-fp-py/import-own-module
Python: Fix FP for py/import own module
2020-02-17 11:23:11 +01:00
jack1142
e1644dd68b Python: Handle __class_getitem__ in py/not-named-self (#2825)
Fixes #2824
2020-02-13 13:38:36 +01:00
Taus
895f2f74ab Merge branch 'master' into python-clean-qltest-options 2020-02-12 13:44:41 +01:00
Taus
12113e947f Merge pull request #2603 from RasmusWL/python-fix-http-source-sink
Python: Make web libs use HttpRequestTaintSource and HttpResponseTaintSink
2020-02-12 13:42:22 +01:00
Rasmus Wriedt Larsen
efedcd26d0 Python: Django tests need --lang=3 2020-02-11 13:16:52 +01:00
Rasmus Wriedt Larsen
1f762841ec Python: In py/import-own-module handle from foo import * 2020-02-11 11:45:48 +01:00
Rasmus Wriedt Larsen
5cc2efef8e Python: Fix FPs for py/import-own-module
Before I added `--max-import-depth=2`, there was a bit of trouble, where it
would alert on `from pkg_ok import foo2` -- since all the `pkg_ok.foo<n>`
modules were missing, I guess the analysis didn't make any assumptions on
whether `foo2` is a module or a regular attribute.
2020-02-11 11:45:48 +01:00
Rasmus Wriedt Larsen
f3f9e340d3 Python: Update tests for py/import-own-module
So I've been thinking a bit about import pkg_ok.foo1 after reading the Python
references for imports of submodules
https://docs.python.org/3/reference/import.html#submodules

> When a submodule is loaded using any mechanism (...) a binding is placed in the
parent module’s namespace to the submodule object. For example, if package spam
has a submodule foo, after importing spam.foo, spam will have an attribute foo
which is bound to the submodule.

That does at least explain what is going on here.

I feel that import pkg_ok.foo1 might be a very contrived example. In principle
it should be an alert, since the module pkg_ok ends up with an import of itself,
but my gut feeling is that in practice it's not a very important piece of code
to give alerts for. if we really care about giving these import related alerts,
we could probably add a new query for this pattern, as it's kind of surprising
that it works when you're just an ordinary python programmer.
2020-02-11 11:45:48 +01:00
Rasmus Wriedt Larsen
2bffbf0734 Python: Add testcases for py/import-own-module
You can try out:

python2 -c "import pkg_ok; print(pkg_ok.foo1); print(pkg_ok.foo2); print(pkg_ok.foo3); print(pkg_ok.foo4); print(pkg_ok.foo5); print(pkg_ok.Foo3); print(pkg_ok.Foo5); print(pkg_ok.pkg_ok)"

python3 -c "import pkg_ok; print(pkg_ok.foo1); print(pkg_ok.foo2); print(pkg_ok.foo3); print(pkg_ok.foo4); print(pkg_ok.foo5); print(pkg_ok.Foo3); print(pkg_ok.Foo5); print(pkg_ok.pkg_ok)"
2020-02-10 15:16:47 +01:00
Rasmus Wriedt Larsen
c0b7dcc019 Python: Remove ignored automatic_locations in qltest options files 2020-02-06 14:28:10 +01:00
Rasmus Wriedt Larsen
397c17c4ff Python: Use --lang=3 in tests
With internal update to qltest it will not actually do something.

- also remove it from the tests that never needed it.
2020-02-06 14:20:59 +01:00
Rasmus Wriedt Larsen
cb891a1a49 Python: Clean up six tests
We can't understand the real `six.py` file, so we have some internal plumbing
that enables us to handle six anyway. While updating that, I had a hell of a lot
of trouble with these tests.

What we actually want, is to see that we can understand what the values imported
from six are (i.e., their points-to information). I added a few more, that I
think would be useful. If we can figure out all of these, I don't actually care
if we're doing it by understanding the real `six.py` file, or by some internal
trick.

I verified that we don't get results with the real `six.py` file by disabling
our internal tricks, and putting a copy of six.py just next to test.py.

We used to have an other file that would list all the properties we knew and
their value, but that turned out to be a fragile and annoying test, since the
results differed from which version of python you ran it with (3.5 vs 3.8) and
which machine you ran it on (my machien vs jenkins). I don't care about the
results in this file, and I can certainly not eyeball it to see if it's correct
or not.
2020-02-06 13:50:51 +01:00
Rasmus Wriedt Larsen
c1d073a54d Python: Add test-cases for py/hardcoded-credentials 2020-02-04 11:42:11 +01:00
Rasmus Wriedt Larsen
2837f987c5 Python: Show how pointsTo handles 0+0 == 0 (2/2) 2020-02-04 11:42:11 +01:00
Rasmus Wriedt Larsen
4231bb1bcf Python: Show how pointsTo handles 0+0 == 0 (1/2) 2020-02-04 11:42:11 +01:00
Rasmus Wriedt Larsen
6b5b28aded Python: Add Value.getABooleanValue and Value.getDefiniteBooleanValue
Replacing `Value.booleanValue`. We wanted to match `Object.booleanValue` that
only gives a result if it is either `true` or `false`, but also wanted to keep
the flexibility to see if the Value _could_ be `true`/`false`. We don't have a
motivating usecase, so let's see if we ever need it :P

+ fix modernisation regression on py/jinja2/autoescape-false
2020-02-04 11:42:11 +01:00
Rasmus Wriedt Larsen
bd1f21fb7a Python: Fix modernisation regression on py/weak-crypto-key
also fixes test code to use the right argument name
2020-02-04 11:42:11 +01:00
Rasmus Wriedt Larsen
e5abfd0196 Python: Modernise Security/ queries 2020-02-04 11:42:11 +01:00
Rasmus Wriedt Larsen
5bc592514a Python: Consistenly use "a user-provided value"
ReflectedXss was the only query that used it with the "a"
2020-02-03 14:35:09 +01:00
Rasmus Wriedt Larsen
4ca72de4cd Python: Fix recommended module for deprecated posixfile
$ python2 -W default -c 'import posixfile'
-c:1: DeprecationWarning: The posixfile module is deprecated; fcntl.lockf() provides better locking

https://docs.python.org/2.7/library/posixfile.html
2020-01-28 16:44:47 +01:00
Rasmus Wriedt Larsen
6c7cddf258 Python: py/import-deprecated-module handle backwards compatible code 2020-01-28 16:36:47 +01:00
Rasmus Wriedt Larsen
e92d6c0459 Python: Stop py/import-deprecated-module from double alerting
This changes the location from the import statement, to the actual expression
2020-01-28 16:15:46 +01:00