Commit Graph

3064 Commits

Author SHA1 Message Date
yoff
716e0f1404 Merge pull request #5517 from tausbn/python-prevent-potentially-bad-join-order
Python: Prevent potentially bad join order
2021-03-25 18:14:47 +01:00
Taus Brock-Nannestad
dbef36cbbb Python: Prevent bad TC and add a bit of caching
Using `simpleLocalFlowStep+` with the first argument specialised to
`CfgNode` was causing the compiler to turn this into a very slowly
converging manual TC computation.

Instead, we use `simpleLocalFlowStep*` (which is fast) and then join
that with a single step from any `CfgNode`. This should amount to the
same thing.

I also noticed that the charpred for `LocalSourceNode` was getting
recomputed a lot, so this is now cached. (The recomputation was
especially bad since it relied on `simpleLocalFlowStep+`, but anyway
it's a good idea not to recompute this.)
2021-03-25 17:28:37 +01:00
Rasmus Wriedt Larsen
9abe02f419 Python: Fix query metadata for old queries that have been ported
I'm not sure even I want to keep these around much longer. They seem to be
causing more problem than they are doing good.
2021-03-25 16:01:56 +01:00
Rasmus Wriedt Larsen
bd4934380a Python: Remove code duplication library 2021-03-25 15:27:55 +01:00
Rasmus Wriedt Larsen
09fbf480db Python: Remove precision tag from metric queries 2021-03-25 15:06:47 +01:00
Rasmus Wriedt Larsen
e3b2e0a1de Python: Delete filter queries 2021-03-25 15:06:46 +01:00
Taus Brock-Nannestad
0ae8b69102 Python: Prevent joining on scope in PointsToContext::appliesTo
One of those cases where I _wish_ `pragma[inline]` also meant "don't
join on the stuff inside this predicate -- it's inlined for a reason".

Unsurprisingly, joining on the scope first works poorly.
2021-03-24 23:12:48 +01:00
Taus Brock-Nannestad
28d6cad3d0 Python: Prevent joining on name as the first thing
Many instances of `lookup` are restricted by the presence of
`attributeRequired`, but this does not work well if we join on
`name`. A few instances of `only_bind_into` prevents this.
2021-03-24 23:11:09 +01:00
Taus Brock-Nannestad
ed8ffab356 Python: Prevent potentially bad join order
This has no effect on the current compilation (indeed,
`ssa_filter_definition_bool` is not currently inlined), but will
prevent this from ever occurring, should the heuristics for inlining
ever change...
2021-03-24 19:20:19 +01:00
yoff
8d15680af4 Merge pull request #5506 from tausbn/python-allow-absolute-imports-from-source-directory
Python: Allow absolute imports in directories with scripts
2021-03-24 14:42:14 +01:00
yoff
b023d73016 Merge pull request #5504 from RasmusWL/type-tracking-first-predicate-private
Python: Ensure first type-tracking predicate is private
2021-03-24 14:23:27 +01:00
Rasmus Wriedt Larsen
1473778bb8 Merge pull request #5493 from yoff/python-add-experimental-structure
Python: Add stub structure to `experimental` for external contributions
2021-03-24 14:11:13 +01:00
Taus Brock-Nannestad
47686a6e4c Python: Disregard all files matching .py% 2021-03-24 14:03:00 +01:00
Rasmus Wriedt Larsen
59200386a7 Python: Fix mistake in refactor 2021-03-24 13:51:29 +01:00
Rasmus Lerchedahl Petersen
a9af135d7e Python: Remove getALocalTaintSource
and `taintFlowsTo` for now..
2021-03-24 01:22:21 +01:00
yoff
61cff8faed Update python/ql/src/experimental/semmle/python/Concepts.qll
Co-authored-by: Rasmus Wriedt Larsen <rasmuswriedtlarsen@gmail.com>
2021-03-24 01:06:03 +01:00
Taus Brock-Nannestad
17d1768259 Python: Allow absolute imports in directories with scripts
Fixes the import logic to account for absolute imports.

We do this by classifying which files and folders may serve as the
entry point for execution, based on a few simple heuristics. If the
file `module.py` is in the same folder as a file `main.py` that may be
executed directly, then we allow `module` to be a valid name for
`module.py` so that `import module` will work as expected.
2021-03-23 18:32:17 +01:00
Rasmus Wriedt Larsen
deefbefffc Python: Minor refactor to use CallCfgNode 2021-03-23 16:42:41 +01:00
Rasmus Wriedt Larsen
1f5e52e822 Python: Cleanup "first" type-tracking predicate to be private
Since it's exposed nicely in the version that doesn't have a
`DataFlow::TypeTracker` parameter, these should be private.

Also found one instance where I had accidentially used DataFlow::Node instead of
LocalSourceNode
2021-03-23 16:40:56 +01:00
Rasmus Wriedt Larsen
f2bc413318 Python: remove single commented out line of code 2021-03-23 14:00:38 +01:00
Rasmus Wriedt Larsen
a4924856a2 Python: Model known form/field subclasses in Django
I used some ad-hoc QL queries to help me find all these extra instances, but not
quite ready to share that code yet :P
2021-03-23 13:57:39 +01:00
Rasmus Wriedt Larsen
8d0f6086af Python: Model django forms/fields
I'm not feeling 100% confident about `SelfRefMixin`, but since I needed it for
both DjangoViewClass and DjangoFormClass, I wanted to avoid copy-pasting this
code around. However, I'm not so opitimistic about it that I want to add it to a
sharable utility qll file :D
2021-03-23 13:57:38 +01:00
Taus
b46a3616d8 Merge pull request #5490 from RasmusWL/private-imports
Python: Make import private for better auto-complete
2021-03-23 12:00:35 +01:00
Rasmus Lerchedahl Petersen
198a4ca79b Python: Add files to experimental 2021-03-22 21:42:06 +01:00
Taus Brock-Nannestad
7cdf439b83 Python: Clean up basicStoreStep
Moves the `flowsTo` logic into the shared implementation, so that
`TypeTrackingPrivate` only has to define the shape of immediate store
steps.

Also cleans up the documentation to talk a bit more about what
`content` can represent, and what caveats there are.
2021-03-22 18:42:24 +01:00
Taus Brock-Nannestad
0e81fd2624 Python: Move Boolean into TypeTrackerPrivate
In general, this may be defined already for other languages, so moving
it in here will avoid potential clashes.
2021-03-22 18:41:22 +01:00
Rasmus Wriedt Larsen
1890e63d4c Python: Make import private for better auto-complete
With the non-private imports, auto-completing on `API::` gave ALL results
available from `import python`, as well as the ones specified in the `API`
module.

The non-private import in Attributes.qll did the same for `DataFlow::`.
2021-03-22 16:45:44 +01:00
Taus Brock-Nannestad
4a6589d0ae Python: Make API::Node::getACall return a CallCfgNode
This should eliminate the need for explicit casting to
`CallCfgNode` (which does not appear in our code as far as I can see,
but was observed in an external contribution).
2021-03-22 16:37:24 +01:00
Rasmus Wriedt Larsen
c8a6e837b5 Python: Model QuerySet chains in django 2021-03-22 14:38:54 +01:00
Rasmus Wriedt Larsen
f800bf243f Python: Better text for getSourceType in Django 2021-03-22 01:39:19 +01:00
Dilan
1385b22642 pr fixes, typo in qhelp file and helper method for queries 2021-03-19 16:43:29 -07:00
Tom Hvitved
09a49e4580 Merge pull request #5311 from hvitved/dataflow/lambda
Data flow: Move C# lambda flow logic into shared library
2021-03-19 11:44:15 +01:00
yoff
37036b5e76 Merge pull request #5437 from RasmusWL/small-pyyaml-improvements
Python: Small PyYAML improvements
2021-03-19 11:15:49 +01:00
Rasmus Lerchedahl Petersen
e0e6d5724e Merge branch 'main' of github.com:github/codeql into python-port-insecure-protocol 2021-03-18 23:34:53 +01:00
yoff
746e9948b0 Merge pull request #5075 from RasmusWL/crypto
Python: Port py/weak-crypto-key to use type-tracking
2021-03-18 20:53:28 +01:00
jorgectf
957b3e1e85 Precision warn 2021-03-18 20:39:53 +01:00
jorgectf
3ce0a9c8c0 Move to experimental folder 2021-03-18 20:20:04 +01:00
jorgectf
7de9214c99 Upload LDAP Insecure authentication query and tests 2021-03-18 17:41:34 +01:00
Rasmus Wriedt Larsen
42b2c3ed52 Python: Model C-based loaders for PyYAML
Not really that important. But easy to do while I was working on this library.
2021-03-18 11:55:01 +01:00
Rasmus Wriedt Larsen
25b15d7470 Python: Move PyYAML modeling classes within module
For now, this is how we're trying to structure things -- all in all it doesn't
matter too much, since everything is still marked as private.
2021-03-18 11:48:30 +01:00
Rasmus Wriedt Larsen
5ec8511d50 Python: Port PyYAML model to API graphs 2021-03-18 11:47:46 +01:00
Rasmus Wriedt Larsen
45a1fc6a96 Python: Add link to better PyYAML docs
I found this randomly
2021-03-18 11:20:22 +01:00
Rasmus Wriedt Larsen
7b92012edf Python: Apply suggestions from code review
Co-authored-by: yoff <lerchedahl@gmail.com>
2021-03-18 10:58:49 +01:00
Rasmus Wriedt Larsen
27032af2eb Python: Use API graphs for io.open 2021-03-17 15:50:02 +01:00
Rasmus Wriedt Larsen
d52d328587 Python: Use new API::builtin in stdlib modeling 2021-03-17 15:50:01 +01:00
Rasmus Lerchedahl Petersen
8f467003d2 Python: More review suggestions 2021-03-17 15:11:17 +01:00
yoff
63b732ce1f Apply suggestions from code review
Co-authored-by: Taus <tausbn@github.com>
2021-03-17 15:11:17 +01:00
Rasmus Lerchedahl Petersen
4d856d4461 Python: Add small api enhancements
determined useful during documentation work.
2021-03-17 15:11:17 +01:00
Rasmus Wriedt Larsen
1ecee2da0d Merge pull request #5357 from yoff/python-rework-documentation
Python: rework documentation
2021-03-17 14:25:23 +01:00
Rasmus Wriedt Larsen
fbbec5d2b9 Merge pull request #5118 from yoff/python-port-stacktrace-exosure
Python: Port stack trace exposure
2021-03-16 14:52:44 +01:00