Commit Graph

322 Commits

Author SHA1 Message Date
Joe Farebrother
d1cca13563 Merge pull request #17314 from joefarebrother/python-x509-cert
Python: Exclude certificate classification fo sensitive data queries
2024-09-09 10:48:36 +01:00
erik-krogh
20dfdc9661 delete some deprecated files 2024-09-03 20:30:59 +02:00
erik-krogh
0fdd06fff5 use my script to delete outdated deprecations 2024-09-03 20:30:58 +02:00
Joe Farebrother
ec7ad84cd1 Update formatting 2024-08-30 13:51:33 +01:00
Joe Farebrother
5360192a58 Apply review suggestions - change = to in
Co-authored-by: Rasmus Wriedt Larsen <rasmuswriedtlarsen@gmail.com>
2024-08-30 13:25:59 +01:00
Joe Farebrother
1cb23e7e86 Exclude certificates from being cinsidered sensitive data by cleartext-storage and cleartext-logging queries 2024-08-27 14:18:39 +01:00
Joe Farebrother
123214cb2b Promoto cookie injection query 2024-07-16 16:49:56 +01:00
Joe Farebrother
8152ec7472 Merge pull request #16696 from joefarebrother/python-cookie-write-headers
Python: Model CookieWrites from HeaderWrites
2024-07-11 14:25:54 +01:00
Joe Farebrother
b71ba7c30f Move Header Write derrived concepts to Concepts 2024-06-24 17:26:51 +01:00
Rasmus Lerchedahl Petersen
f0e68887d4 Python: autoformat 2024-06-20 10:59:39 +02:00
Rasmus Lerchedahl Petersen
5cb37f5c4c python: Document MaD format
- add a few tests reflecting the documentation
- make the mentioned sink-kinds have an effect on relevant queries
2024-06-19 17:00:15 +02:00
Joe Farebrother
93f10fcf14 Add sanitizers for compiled regexes 2024-06-11 15:44:16 +01:00
Joe Farebrother
6ac46b8436 Add additional sanitizers to SSRF for methods that restrict the contents of a string. 2024-06-03 23:23:25 +01:00
Sid Shankar
859e8db5f2 Fixes typo in deprecation notice 2024-06-03 16:31:29 +00:00
Joe Farebrother
01a6c5e82f Merge pull request #16446 from joefarebrother/shared-sensitive-heuristics
Ruby/Python/JS/Swift: Add category of Private information to shared sensitive data heuristics
2024-05-21 09:07:13 +01:00
Joe Farebrother
9aff22c664 Fix typos in sensitive data regex 2024-05-09 09:39:03 +01:00
Joe Farebrother
5f4bc4197b Add private category to sensitive data heuristics 2024-05-08 10:02:00 +01:00
Joe Farebrother
2b935e575a Add concept tests + fix typo 2024-04-24 14:05:41 +01:00
Joe Farebrother
ec4c820391 Fix deprecation 2024-04-24 14:05:41 +01:00
Joe Farebrother
eeef062f7c Implement sinks for wsgiref + allow lists in bulk header updates + local flow 2024-04-24 14:05:39 +01:00
Joe Farebrother
fa28d94363 Added a sanitizer for replacing newlines. 2024-04-24 14:05:38 +01:00
Joe Farebrother
a88ad62c00 Implemented sinks for bulk header updates, and added corresponding tests. 2024-04-24 14:05:38 +01:00
Joe Farebrother
68d90918cf Add to header write concept a specification of whether the name or value arg allows newlines.
Ported sink defenitions from Flask and Werzeug from experimental to main.
Removed experimental sink definitions for Django, as neither name nor value are vulnerable.
2024-04-24 14:05:37 +01:00
Joe Farebrother
25ffcb2fde Split into customizations file 2024-04-24 14:05:37 +01:00
Joe Farebrother
6021d9238c Move headers injection query and concept from experimental to main 2024-04-24 14:05:37 +01:00
Nick Rolfe
af72c0848e Merge pull request #16306 from github/nickrolfe/js-sensitive
JS: do fewer regexp matches in SensitiveActions
2024-04-24 09:49:44 +01:00
Nick Rolfe
003d208574 JS: do fewer regexp matches in SensitiveActions 2024-04-23 15:31:38 +01:00
Taus
d51fcd4f2a Python: Change Str to StringLiteral
As far as I can tell, this was the only occurrence of `Str` as a type
throughout the entire library.
2024-04-22 12:00:09 +00:00
Taus
b484aee39e Python: Autoformat everything
Of course, `StringLiteral` being much longer than `StrConst` meant a
bunch of files changed formatting.
2024-04-22 12:00:09 +00:00
Taus
1c68c987b0 Python: Change all remaining occurrences of StrConst
Done using
```
git grep StrConst | xargs sed -i 's/StrConst/StringLiteral/g'
```
2024-04-22 12:00:09 +00:00
James Ockers
eb5e0123d6 exclude certification from maybeCertificate() regexes 2024-01-30 13:16:18 -08:00
Erik Krogh Kristensen
f1d6f56621 Merge pull request #15393 from erik-krogh/deps-jan-2024
All: delete outdated deprecations
2024-01-23 13:52:38 +01:00
Max Schaefer
17e3a45ad7 Apply suggestions from code review
Co-authored-by: Taus <tausbn@github.com>
2024-01-22 13:36:12 +00:00
Max Schaefer
98178458d0 Python: Add support for more URL redirect sanitisers.
Since some sanitisers don't handle backslashes correctly, I updated the data-flow configuration to incorporate a flow state tracking whether or not backslashes have been eliminated or converted to forward slashes.
2024-01-22 13:24:18 +00:00
erik-krogh
8be7eadace delete outdated deprecations 2024-01-22 09:11:35 +01:00
yoff
e0c027f13c Merge pull request #14848 from hvitved/python/shared-type-tracking
Python: Adopt shared type tracking library
2023-12-18 21:14:42 +01:00
Anders Schack-Mulligen
8ef4821f63 Python: Remove references to FlowStateString. 2023-12-14 15:05:33 +01:00
Tom Hvitved
3b1146bf98 Python: Adopt shared type tracking library 2023-12-14 13:22:44 +01:00
Rasmus Lerchedahl Petersen
2539e2ec1a Python: slightly improve tarslip logic 2023-12-08 17:18:25 +01:00
Rasmus Lerchedahl Petersen
e091ae84ab Merge branch 'main' of https://github.com/github/codeql into python/remove-ssa-nodes-from-dataflow-graph 2023-12-04 14:05:40 +01:00
Taus
6e279183d9 Python: Remove unused unsafeFilter predicates 2023-11-28 13:54:17 +00:00
Taus
91643ad08f Python: Update hasUnsafeFilter to use API graph
This will probably break the tests in the short run. I'll fix the remaining issues in a follow-up commit.

Co-authored-by: Rasmus Wriedt Larsen <rasmuswl@github.com>
2023-11-28 14:48:26 +01:00
Taus
95e9284d08 Python: Add support for extraction filters
Adds support for extraction filters as defined in
https://peps.python.org/pep-0706/
and implemented in Python 3.12.

By my reading, setting the filter to `'data'` or `'tar'` is probably
safe, whereas `'fully_trusted'` or the default (which is the same as
`None`) is not.

For now, I have just added this modelling to the tarslip query. We could
also share it with the modelling of `shutil.unpack_archive` (which has also
gained a `filter` argument), but it was unclear to me where we should put
this modelling in that case. Perhaps the best solution would be to merge
the experimental `py/tarslip-extended` query into the existing query (in
which case the current location is perhaps not too bad).
2023-11-27 14:11:17 +00:00
Rasmus Lerchedahl Petersen
421d4f3497 Python: filter more sinks in stdlib
Rename variable to reflect larger scope

We had test results inside `os.py`, I suppose we have found a little extra flow.
2023-11-20 21:35:52 +01:00
Rasmus Lerchedahl Petersen
11c71fdd18 Python: remove EssaNodes
This commit removes SSA nodes from the data flow graph. Specifically, for a definition and use such as
```python
  x = expr
  y = x + 2
```
we used to have flow from `expr` to an SSA variable representing x and from that SSA variable to the use of `x` in the definition of `y`. Now we instead have flow from `expr` to the control flow node for `x` at line 1 and from there to the control flow node for `x` at line 2.

Specific changes:
- `EssaNode` from the data flow layer no longer exists.
- Several glue steps between `EssaNode`s and `CfgNode`s have been deleted.
- Entry nodes are now admitted as `CfgNodes` in the data flow layer (they were filtered out before).
- Entry nodes now have a new `toString` taking into account that the module name may be ambigous.
- Some tests have been rewritten to accomodate the changes, but only `python/ql/test/experimental/dataflow/basic/maximalFlowsConfig.qll` should have semantic changes.
- Comments have been updated
- Test output has been updated, but apart from `python/ql/test/experimental/dataflow/basic/maximalFlows.expected` only `python/ql/test/experimental/dataflow/typetracking-summaries/summaries.py` should have a semantic change. This is a bonus fix, probably meaning that something was never connected up correctly.
2023-11-20 21:35:32 +01:00
Geoffrey White
e8a466a02c Update dead link. 2023-11-07 09:26:07 +00:00
erik-krogh
194f918c0b Python: delete various outdated deprecations 2023-10-09 09:14:55 +02:00
erik-krogh
0d992a3d1f delete old deprecated aliases of various regex libraries 2023-10-09 09:14:54 +02:00
erik-krogh
4bc4e0845d delete the deprecated isBarrierGuard predicate from the shared dataflow library, and its uses 2023-10-07 21:48:49 +02:00
yoff
dbecb1bd0f Merge pull request #14070 from yoff/python/promote-nosql-query
Python: promote nosql query
2023-09-29 14:21:22 +02:00