Commit Graph

992 Commits

Author SHA1 Message Date
Rasmus Lerchedahl Petersen
e808d3033a Python: Add magic to DataFlowCall 2020-08-14 14:19:18 +02:00
Rasmus Lerchedahl Petersen
360ddc6314 Python: better charPred 2020-08-14 13:25:17 +02:00
Rasmus Lerchedahl Petersen
5ed3107045 Python: Start scaffold for magic methods 2020-08-14 11:12:23 +02:00
Taus
df4d145490 Merge branch 'master' into python-qlformat-everything-again 2020-07-07 16:33:21 +02:00
Taus Brock-Nannestad
f07a7bf8cf Python: Autoformat everything using qlformat.
Will need subsequent PRs fixing up test failures (due to deprecated
methods moving around), but other than that everything should be
straight-forward.
2020-07-07 15:43:52 +02:00
Anders Schack-Mulligen
67db1df00c C++/C#/JavaScript/Python: Port Location qldoc update. 2020-07-07 11:39:27 +02:00
Rasmus Wriedt Larsen
513c2974bd Merge branch 'master' into python-keyword-only-args 2020-07-02 14:48:32 +02:00
Rasmus Wriedt Larsen
26b7a301d6 Merge branch 'master' into python-keyword-only-args 2020-07-02 12:27:02 +02:00
Rasmus Wriedt Larsen
67be45f045 Merge branch 'master' into python-fix-django-taint-sinks 2020-07-02 11:55:42 +02:00
Rasmus Wriedt Larsen
9a82927187 Python: Autoformat 2020-07-02 11:54:41 +02:00
Rasmus Wriedt Larsen
a947d151e5 Python: Django changes now backwards compatible deprecation 2020-07-02 11:53:25 +02:00
Rasmus Wriedt Larsen
4a7bfbe091 Python: Use .matches instead of .indexOf() = 0 2020-07-02 11:43:23 +02:00
Taus Brock-Nannestad
7e97bd1d36 Python: Address review comments. 2020-06-30 11:36:26 +02:00
Taus Brock-Nannestad
b469d55d17 Python: Fix a few things in Stmts.qll. 2020-06-29 13:32:36 +02:00
Taus Brock-Nannestad
5744356dbc Python: Add a bunch more toString docs. 2020-06-28 14:55:45 +02:00
Taus Brock-Nannestad
e72e662f68 Python: Autogenerate QLDoc for toString AST methods.
Only adds these for the methods that do not `override` other
methods (as these presumably have their own `toString` documentation).
2020-06-28 14:41:45 +02:00
Taus Brock-Nannestad
24daf2c4d1 Python: Document internal AST classes.
We already document these in the classes that override them, so I
simply added a pointer to this information.
2020-06-26 21:15:30 +02:00
Rasmus Wriedt Larsen
3f0975f5a1 Merge pull request #3770 from tausbn/python-add-a-bunch-of-documentation
Python: Add a bunch of documentation.
2020-06-26 13:30:45 +02:00
Taus Brock-Nannestad
4dbc8e515a Python: Address a few more review comments. 2020-06-25 14:19:18 +02:00
Taus
1608758219 Python: Apply suggestions from documentation review.
Co-authored-by: Felicity Chapman <felicitymay@github.com>
Co-authored-by: Rasmus Wriedt Larsen <rasmuswriedtlarsen@gmail.com>
2020-06-25 14:16:44 +02:00
Taus Brock-Nannestad
02363d76c1 Python: Document Comment.qll.
I didn't do the `toString` methods in this commit. I'm thinking
they're better to do in a separate commit. (There are 48 undocumented
instances!)
2020-06-24 22:43:59 +02:00
Taus Brock-Nannestad
fe78e68fd0 Python: Document a bunch of hasLocationInfo methods.
If only we had been _somewhat consistent in how we named the
parameters for these...
2020-06-24 22:38:03 +02:00
Taus Brock-Nannestad
682e1b6040 Python: Document Comparisons.qll. 2020-06-24 22:13:46 +02:00
Taus Brock-Nannestad
b8e744eade Python: Document Class.qll. 2020-06-24 22:07:47 +02:00
Rasmus Lerchedahl Petersen
f6c59abcd9 Merge branch 'master' of github.com:github/codeql into UnmatchableDollar
to make CodeScan happy
2020-06-24 11:04:07 +02:00
Rasmus Lerchedahl Petersen
226c295b4c Python: format 2020-06-24 10:48:51 +02:00
Taus Brock-Nannestad
1e4ec5c987 Python: Make QLDoc for TObject.qll visible. 2020-06-23 14:31:30 +02:00
Rasmus Wriedt Larsen
daa1b6fc79 Python: Fix grammar in QLDoc
Co-authored-by: Taus <tausbn@gmail.com>
2020-06-22 13:41:03 +02:00
Taus
2081d0cecc Merge pull request #3575 from RasmusWL/python-add-qldoc-FunctionValue.getQualifiedName
Python: Add QLDoc for FunctionValue.getQualifiedName
2020-06-19 16:32:23 +02:00
Rasmus Wriedt Larsen
c0043eb9db Python: Don't treat re.escape(...) as a regex
Fixes https://github.com/github/codeql/issues/3712
2020-06-15 11:54:14 +02:00
semmle-qlci
4cdb3c13df Merge pull request #3658 from RasmusWL/python-3.8-dict-ismapping
Approved by tausbn
2020-06-10 17:19:49 +01:00
Taus
5b0d92d72b Merge pull request #3464 from yoff/UnicodeEscape
Python: Handle more escapes in regexes
2020-06-10 15:47:09 +02:00
Rasmus Wriedt Larsen
bacd491875 Python: Fix isSequence() and isMapping() 2020-06-09 14:21:02 +02:00
semmle-qlci
1a7570ebbe Merge pull request #3563 from RasmusWL/python-fabric-execute
Approved by tausbn
2020-06-08 16:00:49 +01:00
Rasmus Wriedt Larsen
551420401a Python: Fix typo
Co-authored-by: Taus <tausbn@gmail.com>
2020-05-29 14:27:07 +02:00
Rasmus Wriedt Larsen
48be57c8fd Python: Improve QLDoc for ExternalStringDictKind 2020-05-29 12:06:57 +02:00
Rasmus Wriedt Larsen
b083c01520 Python: Deprecate StringDictKind
This QL

```codeql
import python
import semmle.python.dataflow.TaintTracking
import semmle.python.security.strings.Untrusted

from CollectionKind ck
where
    ck.(DictKind).getMember() instanceof StringKind
    or
    ck.getMember().(DictKind).getMember() instanceof StringKind
select ck, ck.getAQlClass(), ck.getMember().getAQlClass()
```

generates these 6 results.

```
1	{externally controlled string}          ExternalStringDictKind	UntrustedStringKind
2	{externally controlled string}	        StringDictKind	        UntrustedStringKind
3	[{externally controlled string}]	SequenceKind	        ExternalStringDictKind
4	[{externally controlled string}]	SequenceKind	        StringDictKind
5	{{externally controlled string}}	DictKind	        ExternalStringDictKind
6	{{externally controlled string}}	DictKind	        StringDictKind
```

StringDictKind was only used in *one* place in our library code. As illustrated
above, it pollutes our set of TaintKinds. Effectively, every time we make a
flow-step for dictionaries with tainted strings as values, we do it TWICE --
once for ExternalStringDictKind, and once for StringDictKind... that is just a
waste.
2020-05-29 12:06:57 +02:00
Rasmus Wriedt Larsen
87bc8ae28d Python: Don't use UntrustedStringKind in web lib
If I wanted to use my own TaintKind and not have any interaction with
`UntrustedStringKind` that wouldn't be possible today since these standard http
libraries import it directly. (also, I wouldn't get any sources of my custom
TaintKind from turbogears or bottle). I changed them to use the same pattern of
`ExternalStringKind` as everything else does.
2020-05-29 12:06:57 +02:00
Rasmus Wriedt Larsen
21d531f81e Python: Add QLDoc for FunctionValue.getQualifiedName
Matching the one for Function.getQualifiedName
2020-05-27 16:59:18 +02:00
Rasmus Wriedt Larsen
6cba2fe4f8 Python: Model Django response sinks that are not vuln to XSS
Since HttpResponse is not *only* used for XSS, it is still valuable to know the
content is send as part of the response.

The *proper* solution to this problem of not all HttpResponses being vulnerable
to XSS is probably to define a new abstract class in Http.qll called
HttpResponseXSSVulnerableSink (or similar). I would like to model a few more
libraries/frameworks before fully comitting to an approach though.
2020-05-26 16:45:46 +02:00
Jonas Jensen
5deeda0337 Merge pull request #3387 from geoffw0/tostringperf
C++: Eliminate recursion from toString().
2020-05-26 13:24:43 +02:00
Rasmus Lerchedahl Petersen
6b168de7fc Python: re, handle \Z 2020-05-26 11:42:21 +02:00
Rasmus Wriedt Larsen
c78ca2616c Merge branch 'master' into python-keyword-only-args 2020-05-26 11:20:04 +02:00
Rasmus Wriedt Larsen
9c75a39b81 Python: Extend command-injection to handle fabric.api.execute 2020-05-26 10:22:27 +02:00
Taus
7716cff3d8 Merge pull request #3551 from RasmusWL/python-fix-upcoming-deprecation
Python: Fix (upcoming) deprecation compiler-warnings
2020-05-25 16:17:57 +02:00
semmle-qlci
8146073c74 Merge pull request #3553 from RasmusWL/python-fix-tainttracking-import
Approved by tausbn
2020-05-25 14:18:54 +01:00
Rasmus Wriedt Larsen
f602f3e1c7 Python: Use proper import for semmle.python.dataflow.TaintTracking
It was moved in 637677d515, but imports were not
updated.
2020-05-25 13:45:49 +02:00
semmle-qlci
ac1a338390 Merge pull request #3407 from RasmusWL/python-add-BoundMethodValue-v2
Approved by tausbn
2020-05-25 12:00:45 +01:00
Rasmus Wriedt Larsen
32c8dd0491 Python: Fix (upcoming) deprecation compiler-warnings
In a near-future release overriding a deprecated predicate without making as
deprecated would give a compiler warning.

Not fixing the XML one. [I can see that this shouldn't be reported
anymore](https://github.com/github/codeql/pull/3520#issuecomment-631552943), and
it's not safe to remove since it was only marked as deprecated in
e6425bb4cf.
2020-05-25 11:05:30 +02:00
Rasmus Wriedt Larsen
49d7e12acd Python: Remove unnecessary restriction from getNamedArgumentForCall
As agreed in https://github.com/github/codeql/pull/3407
2020-05-25 10:17:37 +02:00