Commit Graph

500 Commits

Author SHA1 Message Date
jorgectf
d475d52c76 Add partial modeling 2021-06-30 00:59:40 +02:00
jorgectf
c3b3bde35d Add XMLParser concept 2021-06-30 00:59:17 +02:00
jorgectf
b9fa57f518 Move tests to test/ 2021-06-30 00:58:58 +02:00
jorgectf
78deec84fc Upload main structure and initial tests 2021-06-22 16:41:08 +02:00
Arthur Baars
6693c5bdd0 Merge pull request #5395 from tausbn/python-share-typetracker
Python: Make the type tracking implementation shareable
2021-04-29 12:06:12 +02:00
Taus
71780228ae Python: Rename TypeTrackerPrivate.qll 2021-04-21 13:08:26 +00:00
Taus
489e1e94e4 Python: Prevent bad joins
Adds a few unbinds to prevent bad joins from occurring.

Firstly, we never want to join `StepSummary::step` with
`TypeTracker::append` on `summary` as the first join, as the resulting
relation is absolutely massive. So we decouple the two occurrences of
`summary` by unbinding each of them.

Secondly, in some cases the node we're stepping to (`nodeTo` for type
trackers, `nodeFrom` for type backtrackers) will get joined eagerly
with the typetracker one is defining, and again this produces an
uncomfortably large intermediate join. A bit of unbinding prevents this
as well.
2021-04-21 11:44:34 +00:00
Taus
9e95f6e7c1 Python: Remove typePreservingStep
This requires a bit of explanation, so strap in.

Firstly, because we use `LocalSourceNode`s as the start and end points
of our `StepSummary::step` relation, there's no need to include
`simpleLocalFlowStep` (via `typePreservingStep`) in `smallstep`. Indeed,
since the successor node for a `step` is a `LocalSourceNode`, and local
sources never have incoming flow, this is entirely futile -- we can find
values for `mid` and `nodeTo` that satisfy the body of `step`, but
`nodeTo` will never be a `LocalSourceNode`.

With this in mind, we can simplify `smallstep` to only refer to
`jumpStep`.

This then brings the other uses of `typePreservingStep` into question.
The only other place we use this predicate is in the `TypeTracker` and
`TypeBackTracker` `smallstep` predicates. Note, however, that here we
no longer need `jumpStep` to be part of `typeTrackingStep` (as it is
already accounted for in `StepSummary::smallstep`) so we can simplify
to `simpleLocalFlowStep`. At this point, `typePreservingStep` is unused.

Finally, because of the way `smallstep` is used in `step` (inside
`StepSummary`), `nodeTo` must always be a `LocalSourceNode`, so I have
propagated this restriction to `smallstep` as well. We can always lift
this restriction later, but for now it seems like it's likely to cause
fewer surprises to have made this explicit.
2021-04-21 11:12:06 +00:00
Rasmus Wriedt Larsen
364d48948f Merge pull request #3810 from dilanbhalla/syntaxpython
Python: Function/Class Naming Convention (Syntax)
2021-04-12 10:42:17 +02:00
Dilan
d73ba13b28 autoformat fix 2021-04-08 11:41:58 -07:00
Dilan
675de07c3e autoformat ql 2021-04-07 15:04:18 -07:00
Calum Grant
c26d05b1d5 Merge pull request #5532 from RasmusWL/python-cleanup
Python: Delete filter queries, code duplication library, and precision tag from metric queries
2021-03-29 17:16:43 +01:00
Rasmus Wriedt Larsen
96a66fa4ee Python: Apply suggestions from code review 2021-03-29 17:02:56 +02:00
Rasmus Wriedt Larsen
9abe02f419 Python: Fix query metadata for old queries that have been ported
I'm not sure even I want to keep these around much longer. They seem to be
causing more problem than they are doing good.
2021-03-25 16:01:56 +01:00
yoff
61cff8faed Update python/ql/src/experimental/semmle/python/Concepts.qll
Co-authored-by: Rasmus Wriedt Larsen <rasmuswriedtlarsen@gmail.com>
2021-03-24 01:06:03 +01:00
Rasmus Lerchedahl Petersen
198a4ca79b Python: Add files to experimental 2021-03-22 21:42:06 +01:00
Taus Brock-Nannestad
7cdf439b83 Python: Clean up basicStoreStep
Moves the `flowsTo` logic into the shared implementation, so that
`TypeTrackingPrivate` only has to define the shape of immediate store
steps.

Also cleans up the documentation to talk a bit more about what
`content` can represent, and what caveats there are.
2021-03-22 18:42:24 +01:00
Taus Brock-Nannestad
0e81fd2624 Python: Move Boolean into TypeTrackerPrivate
In general, this may be defined already for other languages, so moving
it in here will avoid potential clashes.
2021-03-22 18:41:22 +01:00
Dilan
1385b22642 pr fixes, typo in qhelp file and helper method for queries 2021-03-19 16:43:29 -07:00
yoff
746e9948b0 Merge pull request #5075 from RasmusWL/crypto
Python: Port py/weak-crypto-key to use type-tracking
2021-03-18 20:53:28 +01:00
Taus
dfc0e9b906 Merge pull request #5243 from RasmusWL/port-bind-to-all-interfaces
Python: Port py/bind-socket-all-network-interfaces query
2021-03-12 16:04:19 +01:00
Taus Brock-Nannestad
f05313435d Python: Move typePreservingStep into Private 2021-03-12 14:06:39 +01:00
Taus Brock-Nannestad
9b8056371f Python: Make the type tracking implementation shareable 2021-03-12 13:51:24 +01:00
Rasmus Wriedt Larsen
024a586a7d Python: Remove tags for old query copy
Co-authored-by: Taus <tausbn@github.com>
2021-03-05 17:08:55 +01:00
Rasmus Wriedt Larsen
b43533ce8d Python: Ensure old dataflow queries are not used
There seems to have been some cases where the old ones have been picked up
instead of the new ones. At least I spotted _one_ case where this happened, in
an internal actions run.

I'm not sure how to actual debug this, so just removing all the tags that could
make these queries to become picked up :|
2021-02-26 11:22:23 +01:00
Rasmus Wriedt Larsen
a09f8c4b4a Python: Port bind-to-all-interfaces to type-tracking 2021-02-23 16:01:24 +01:00
Rasmus Wriedt Larsen
46ad611d57 Python: Port py/weak-crypto-key to use type-tracking
instead of points-to.

Looking at query results also made me realize I didn't supply a very good
"origin" for ECC in cryptography package, so I improved that 👍 -- maybe that
sohuld have been split into multiple commits... too late :(
2021-02-19 15:03:43 +01:00
Rasmus Wriedt Larsen
d8bfa3565f Python: Simple port of URL redirect query
Still have not added sanitizer, but seems like old sanitizer was a bit too broad
(also covering %-formatting)
2021-01-20 11:35:44 +01:00
Taus Brock-Nannestad
a5121babc8 Python: The one with changes that don't look like renames anymore 2020-11-02 22:19:15 +01:00
Taus Brock-Nannestad
5156bf756d Python: Promote data-flow libraries
Step 1: Moving stuff around. Also includes a bit of import renaming.
2020-11-02 22:15:38 +01:00
Taus
25e88ed585 Merge pull request #4588 from yoff/python-pep-249
Python: Model PEP 249
2020-11-02 18:57:15 +01:00
yoff
1535ce1f5d Apply suggestions from code review
Co-authored-by: Taus <tausbn@github.com>
2020-11-02 18:26:37 +01:00
Rasmus Lerchedahl Petersen
ea74c7f12b Python: add tests 2020-11-02 17:59:51 +01:00
Rasmus Lerchedahl Petersen
cb47b57440 Python: fix typos 2020-11-02 17:22:01 +01:00
yoff
222a2f7f5d Update python/ql/src/experimental/semmle/python/frameworks/PEP249.qll
Co-authored-by: Taus <tausbn@github.com>
2020-11-02 17:19:21 +01:00
yoff
ca34219ec5 Update python/ql/src/experimental/semmle/python/frameworks/PEP249.qll
Co-authored-by: Taus <tausbn@github.com>
2020-11-02 17:19:10 +01:00
yoff
1c83d67513 Update python/ql/src/experimental/semmle/python/frameworks/PEP249.qll
Co-authored-by: Taus <tausbn@github.com>
2020-11-02 17:18:18 +01:00
Rasmus Lerchedahl Petersen
6a81987284 Python: Rename and add docs 2020-11-02 17:17:12 +01:00
Taus
2dfffdbab8 Merge pull request #4590 from RasmusWL/python-model-base64
Python: Model encoding/decoding with base64 module
2020-11-02 17:00:21 +01:00
Rasmus Lerchedahl Petersen
6d850b2e0c Merge branch 'main' of github.com:github/codeql into python-pep-249 2020-11-02 16:58:31 +01:00
yoff
ac85a77ac5 Update python/ql/src/experimental/semmle/python/Frameworks.qll
Co-authored-by: Rasmus Wriedt Larsen <rasmuswriedtlarsen@gmail.com>
2020-11-02 16:40:22 +01:00
Calum Grant
cb527cae73 Merge pull request #4583 from tausbn/python-test-2
Python: Promote experimental queries
2020-11-02 14:42:27 +00:00
Rasmus Wriedt Larsen
247fd4f5f3 Python: Make encoding/decoding preserve taint automatically
With the way we have set things up, there is no way to opt out of this behavior.
2020-11-02 14:53:30 +01:00
Rasmus Lerchedahl Petersen
36e364d6ef Python: Django use PEP 249 2020-11-02 14:49:34 +01:00
Rasmus Wriedt Larsen
66f5d0d9d5 Python: Model encoding/decoding with base64 module 2020-11-02 14:44:53 +01:00
Rasmus Wriedt Larsen
eff244db71 Python: Add Encoding concept
I wasn't able to find a good opposite of "parsing", so left that out of the list
of intended purposes.
2020-11-02 14:19:20 +01:00
Taus Brock-Nannestad
8147ad4e0b Python: Remove irrelevant files 2020-11-02 14:08:59 +01:00
Taus Brock-Nannestad
9d6c07c8df Python: Add copy of old queries 2020-11-02 13:35:20 +01:00
Rasmus Lerchedahl Petersen
d35bf8f446 Python: Update comments on PEP 249 module 2020-11-02 11:22:51 +01:00
Rasmus Lerchedahl Petersen
0240670d62 Python: import frameworks 2020-11-01 18:02:36 +01:00