hohn/codeql
1
0
Fork 0
mirror of https://github.com/github/codeql.git synced 2025-12-24 04:36:35 +01:00
Code Issues Packages Projects Releases Wiki Activity

Merge branch 'main' into inline-5

Browse Source
This commit is contained in:
Jeroen Ketema
2023-07-11 10:30:11 +02:00
parent c2be2c2738 93ad204abd
commit 8cee4f37a4
371 changed files with 7311 additions and 3813 deletions
Download Patch File Download Diff File Expand all files Collapse all files

23
python/ql/lib/CHANGELOG.md
View File

@@ -1,3 +1,26 @@
## 0.10.0
### New Features
* It is now possible to specify flow summaries in the format "MyPkg;Member[list_map];Argument[1].ListElement;Argument[0].Parameter[0];value"
### Minor Analysis Improvements
* Deleted many models that used the old dataflow library, the new models can be found in the `python/ql/lib/semmle/python/frameworks` folder.
* More precise modeling of several container functions (such as `sorted`, `reversed`) and methods (such as `set.add`, `list.append`).
* Added modeling of taint flow through the template argument of `flask.render_template_string` and `flask.stream_template_string`.
* Deleted many deprecated predicates and classes with uppercase `API`, `HTTP`, `XSS`, `SQL`, etc. in their names. Use the PascalCased versions instead.
* Deleted the deprecated `getName()` predicate from the `Container` class, use `getAbsolutePath()` instead.
* Deleted many deprecated module names that started with a lowercase letter, use the versions that start with an uppercase letter instead.
* Deleted many deprecated predicates in `PointsTo.qll`.
* Deleted many deprecated files from the `semmle.python.security` package.
* Deleted the deprecated `BottleRoutePointToExtension` class from `Extensions.qll`.
* Type tracking is now aware of flow summaries. This leads to a richer API graph, and may lead to more results in some queries.
## 0.9.4
No user-facing changes.
## 0.9.3
No user-facing changes.

4
python/ql/lib/change-notes/2023-05-30-typetracking-via-flow-summaries.md
View File

@@ -1,4 +0,0 @@
---
category: minorAnalysis
---
* Type tracking is now aware of flow summaries. This leads to a richer API graph, and may lead to more results in some queries.

9
python/ql/lib/change-notes/2023-06-09-delete-deps.md
View File

@@ -1,9 +0,0 @@
---
category: minorAnalysis
---
* Deleted many deprecated predicates and classes with uppercase `API`, `HTTP`, `XSS`, `SQL`, etc. in their names. Use the PascalCased versions instead.
* Deleted the deprecated `getName()` predicate from the `Container` class, use `getAbsolutePath()` instead.
* Deleted many deprecated module names that started with a lowercase letter, use the versions that start with an uppercase letter instead.
* Deleted many deprecated predicates in `PointsTo.qll`.
* Deleted many deprecated files from the `semmle.python.security` package.
* Deleted the deprecated `BottleRoutePointToExtension` class from `Extensions.qll`.

4
python/ql/lib/change-notes/2023-06-12-flask-render-template-string.md
View File

@@ -1,4 +0,0 @@
---
category: minorAnalysis
---
* Added modeling of taint flow through the template argument of `flask.render_template_string` and `flask.stream_template_string`.

4
python/ql/lib/change-notes/2023-06-13-container-store-steps.md
View File

@@ -1,4 +0,0 @@
---
category: minorAnalysis
---
* More precise modelling of several container functions (such as `sorted`, `reversed`) and methods (such as `set.add`, `list.append`).

4
python/ql/lib/change-notes/2023-06-14-delete-deps.md
View File

@@ -1,4 +0,0 @@
---
category: minorAnalysis
---
* Deleted many models that used the old dataflow library, the new models can be found in the `python/ql/lib/semmle/python/frameworks` folder.

4
python/ql/lib/change-notes/2023-06-20-summaries-from-models.md
View File

@@ -1,4 +0,0 @@
---
category: feature
---
* It is now possible to specify flow summaries in the format "MyPkg;Member[list_map];Argument[1].ListElement;Argument[0].Parameter[0];value"

18
python/ql/lib/change-notes/released/0.10.0.md Normal file
View File

@@ -0,0 +1,18 @@
## 0.10.0
### New Features
* It is now possible to specify flow summaries in the format "MyPkg;Member[list_map];Argument[1].ListElement;Argument[0].Parameter[0];value"
### Minor Analysis Improvements
* Deleted many models that used the old dataflow library, the new models can be found in the `python/ql/lib/semmle/python/frameworks` folder.
* More precise modeling of several container functions (such as `sorted`, `reversed`) and methods (such as `set.add`, `list.append`).
* Added modeling of taint flow through the template argument of `flask.render_template_string` and `flask.stream_template_string`.
* Deleted many deprecated predicates and classes with uppercase `API`, `HTTP`, `XSS`, `SQL`, etc. in their names. Use the PascalCased versions instead.
* Deleted the deprecated `getName()` predicate from the `Container` class, use `getAbsolutePath()` instead.
* Deleted many deprecated module names that started with a lowercase letter, use the versions that start with an uppercase letter instead.
* Deleted many deprecated predicates in `PointsTo.qll`.
* Deleted many deprecated files from the `semmle.python.security` package.
* Deleted the deprecated `BottleRoutePointToExtension` class from `Extensions.qll`.
* Type tracking is now aware of flow summaries. This leads to a richer API graph, and may lead to more results in some queries.

3
python/ql/lib/change-notes/released/0.9.4.md Normal file
View File

@@ -0,0 +1,3 @@
## 0.9.4
No user-facing changes.

2
python/ql/lib/codeql-pack.release.yml
View File

@@ -1,2 +1,2 @@
---
lastReleaseVersion: 0.9.3
lastReleaseVersion: 0.10.0

2
python/ql/lib/qlpack.yml
View File

@@ -1,5 +1,5 @@
name: codeql/python-all
version: 0.9.4-dev
version: 0.10.1-dev
groups: python
dbscheme: semmlecode.python.dbscheme
extractor: python

12
python/ql/lib/semmle/python/Concepts.qll
View File

@@ -680,6 +680,9 @@ module Escaping {
/** Gets the escape-kind for escaping a string so it can safely be included in HTML. */
string getHtmlKind() { result = "html" }
/** Gets the escape-kind for escaping a string so it can safely be included in XML. */
string getXmlKind() { result = "xml" }
/** Gets the escape-kind for escaping a string so it can safely be included in a regular expression. */
string getRegexKind() { result = "regex" }
@@ -710,6 +713,15 @@ class HtmlEscaping extends Escaping {
HtmlEscaping() { super.getKind() = Escaping::getHtmlKind() }
}
/**
* An escape of a string so it can be safely included in
* the body of an XML element, for example, replacing `&` and `<>` in
* `<foo>&xxe;<foo>`.
*/
class XmlEscaping extends Escaping {
XmlEscaping() { super.getKind() = Escaping::getXmlKind() }
}
/**
* An escape of a string so it can be safely included in
* the body of a regex.

1
python/ql/lib/semmle/python/dataflow/new/internal/DataFlowDispatch.qll
View File

@@ -1227,7 +1227,6 @@ predicate normalCallArg(CallNode call, Node arg, ArgumentPosition apos) {
* time the bound method is used, such that the `clear()` call would essentially be
* translated into `l.clear()`, and we can still have use-use flow.
*/
pragma[assume_small_delta]
cached
predicate getCallArg(CallNode call, Function target, CallType type, Node arg, ArgumentPosition apos) {
Stages::DataFlow::ref() and

10
python/ql/lib/semmle/python/dataflow/new/internal/DataFlowImpl.qll
View File

@@ -460,7 +460,6 @@ module Impl<FullStateConfigSig Config> {
* The Boolean `cc` records whether the node is reached through an
* argument in a call.
*/
pragma[assume_small_delta]
private predicate fwdFlow(NodeEx node, Cc cc) {
sourceNode(node, _) and
if hasSourceCallCtx() then cc = true else cc = false
@@ -570,7 +569,6 @@ module Impl<FullStateConfigSig Config> {
/**
* Holds if `c` is the target of a store in the flow covered by `fwdFlow`.
*/
pragma[assume_small_delta]
pragma[nomagic]
private predicate fwdFlowConsCand(Content c) {
exists(NodeEx mid, NodeEx node |
@@ -1216,7 +1214,6 @@ module Impl<FullStateConfigSig Config> {
fwdFlow1(_, _, _, _, _, _, t0, t, ap, _) and t0 != t
}
pragma[assume_small_delta]
pragma[nomagic]
private predicate fwdFlow0(
NodeEx node, FlowState state, Cc cc, ParamNodeOption summaryCtx, TypOption argT,
@@ -2777,7 +2774,6 @@ module Impl<FullStateConfigSig Config> {
/**
* Gets the number of `AccessPath`s that correspond to `apa`.
*/
pragma[assume_small_delta]
private int countAps(AccessPathApprox apa) {
evalUnfold(apa, false) and
result = 1 and
@@ -2796,7 +2792,6 @@ module Impl<FullStateConfigSig Config> {
* that it is expanded to a precise head-tail representation.
*/
language[monotonicAggregates]
pragma[assume_small_delta]
private int countPotentialAps(AccessPathApprox apa) {
apa instanceof AccessPathApproxNil and result = 1
or
@@ -2833,7 +2828,6 @@ module Impl<FullStateConfigSig Config> {
}
private newtype TPathNode =
pragma[assume_small_delta]
TPathNodeMid(
NodeEx node, FlowState state, CallContext cc, SummaryCtx sc, DataFlowType t, AccessPath ap
) {
@@ -2918,7 +2912,6 @@ module Impl<FullStateConfigSig Config> {
override AccessPathFrontHead getFront() { result = TFrontHead(head_) }
pragma[assume_small_delta]
override AccessPathApproxCons getApprox() {
result = TConsNil(head_, t) and tail_ = TAccessPathNil()
or
@@ -2927,7 +2920,6 @@ module Impl<FullStateConfigSig Config> {
result = TCons1(head_, this.length())
}
pragma[assume_small_delta]
override int length() { result = 1 + tail_.length() }
private string toStringImpl(boolean needsSuffix) {
@@ -3379,7 +3371,6 @@ module Impl<FullStateConfigSig Config> {
* Holds if data may flow from `mid` to `node`. The last step in or out of
* a callable is recorded by `cc`.
*/
pragma[assume_small_delta]
pragma[nomagic]
private predicate pathStep0(
PathNodeMid mid, NodeEx node, FlowState state, CallContext cc, SummaryCtx sc, DataFlowType t,
@@ -3592,7 +3583,6 @@ module Impl<FullStateConfigSig Config> {
)
}
pragma[assume_small_delta]
pragma[nomagic]
private predicate pathThroughCallable0(
DataFlowCall call, PathNodeMid mid, ReturnKindExt kind, FlowState state, CallContext cc,

2
python/ql/lib/semmle/python/dataflow/new/internal/DataFlowImplCommon.qll
View File

@@ -187,7 +187,6 @@ private module LambdaFlow {
else any()
}
pragma[assume_small_delta]
pragma[nomagic]
predicate revLambdaFlow0(
DataFlowCall lambdaCall, LambdaCallKind kind, Node node, DataFlowType t, boolean toReturn,
@@ -274,7 +273,6 @@ private module LambdaFlow {
)
}
pragma[assume_small_delta]
pragma[nomagic]
predicate revLambdaFlowOut(
DataFlowCall lambdaCall, LambdaCallKind kind, TReturnPositionSimple pos, DataFlowType t,

12
python/ql/lib/semmle/python/dataflow/new/internal/DataFlowPrivate.qll
View File

@@ -304,8 +304,12 @@ module EssaFlow {
// see `with_flow` in `python/ql/src/semmle/python/dataflow/Implementation.qll`
with.getContextExpr() = contextManager.getNode() and
with.getOptionalVars() = var.getNode() and
not with.isAsync() and
contextManager.strictlyDominates(var)
// note: we allow this for both `with` and `async with`, since some
// implementations do `async def __aenter__(self): return self`, so you can do
// both:
// * `foo = x.foo(); await foo.async_method(); foo.close()` and
// * `async with x.foo() as foo: await foo.async_method()`.
)
or
// Async with var definition
@@ -314,6 +318,12 @@ module EssaFlow {
// nodeTo is `x`, essa var
//
// This makes the cfg node the local source of the awaited value.
//
// We have this step in addition to the step above, to handle cases where the QL
// modeling of `f(42)` requires a `.getAwaited()` step (in API graphs) when not
// using `async with`, so you can do both:
// * `foo = await x.foo(); await foo.async_method(); foo.close()` and
// * `async with x.foo() as foo: await foo.async_method()`.
exists(With with, ControlFlowNode var |
nodeFrom.(CfgNode).getNode() = var and
nodeTo.(EssaNode).getVar().getDefinition().(WithDefinition).getDefiningNode() = var and

81
python/ql/lib/semmle/python/dataflow/new/internal/FlowSummaryImpl.qll
View File

@@ -23,29 +23,30 @@ module Public {
* content type, or a return kind.
*/
class SummaryComponent extends TSummaryComponent {
/** Gets a textual representation of this summary component. */
string toString() {
exists(ContentSet c | this = TContentSummaryComponent(c) and result = c.toString())
or
exists(ContentSet c | this = TWithoutContentSummaryComponent(c) and result = "without " + c)
or
exists(ContentSet c | this = TWithContentSummaryComponent(c) and result = "with " + c)
/** Gets a textual representation of this component used for MaD models. */
string getMadRepresentation() {
result = getMadRepresentationSpecific(this)
or
exists(ArgumentPosition pos |
this = TParameterSummaryComponent(pos) and result = "parameter " + pos
this = TParameterSummaryComponent(pos) and
result = "Parameter[" + getArgumentPosition(pos) + "]"
)
or
exists(ParameterPosition pos |
this = TArgumentSummaryComponent(pos) and result = "argument " + pos
this = TArgumentSummaryComponent(pos) and
result = "Argument[" + getParameterPosition(pos) + "]"
)
or
exists(ReturnKind rk | this = TReturnSummaryComponent(rk) and result = "return (" + rk + ")")
or
exists(SummaryComponent::SyntheticGlobal sg |
this = TSyntheticGlobalSummaryComponent(sg) and
result = "synthetic global (" + sg + ")"
exists(string synthetic |
this = TSyntheticGlobalSummaryComponent(synthetic) and
result = "SyntheticGlobal[" + synthetic + "]"
)
or
this = TReturnSummaryComponent(getReturnValueKind()) and result = "ReturnValue"
}
/** Gets a textual representation of this summary component. */
string toString() { result = this.getMadRepresentation() }
}
/** Provides predicates for constructing summary components. */
@@ -110,7 +111,6 @@ module Public {
}
/** Gets the stack obtained by dropping the first `i` elements, if any. */
pragma[assume_small_delta]
SummaryComponentStack drop(int i) {
i = 0 and result = this
or
@@ -125,19 +125,22 @@ module Public {
this = TSingletonSummaryComponentStack(result) or result = this.tail().bottom()
}
/** Gets a textual representation of this stack. */
string toString() {
/** Gets a textual representation of this stack used for MaD models. */
string getMadRepresentation() {
exists(SummaryComponent head, SummaryComponentStack tail |
head = this.head() and
tail = this.tail() and
result = tail + "." + head
result = tail.getMadRepresentation() + "." + head.getMadRepresentation()
)
or
exists(SummaryComponent c |
this = TSingletonSummaryComponentStack(c) and
result = c.toString()
result = c.getMadRepresentation()
)
}
/** Gets a textual representation of this stack. */
string toString() { result = this.getMadRepresentation() }
}
/** Provides predicates for constructing stacks of summary components. */
@@ -166,42 +169,6 @@ module Public {
SummaryComponentStack return(ReturnKind rk) { result = singleton(SummaryComponent::return(rk)) }
}
/** Gets a textual representation of this component used for flow summaries. */
private string getComponent(SummaryComponent sc) {
result = getComponentSpecific(sc)
or
exists(ArgumentPosition pos |
sc = TParameterSummaryComponent(pos) and
result = "Parameter[" + getArgumentPosition(pos) + "]"
)
or
exists(ParameterPosition pos |
sc = TArgumentSummaryComponent(pos) and
result = "Argument[" + getParameterPosition(pos) + "]"
)
or
exists(string synthetic |
sc = TSyntheticGlobalSummaryComponent(synthetic) and
result = "SyntheticGlobal[" + synthetic + "]"
)
or
sc = TReturnSummaryComponent(getReturnValueKind()) and result = "ReturnValue"
}
/** Gets a textual representation of this stack used for flow summaries. */
string getComponentStack(SummaryComponentStack stack) {
exists(SummaryComponent head, SummaryComponentStack tail |
head = stack.head() and
tail = stack.tail() and
result = getComponentStack(tail) + "." + getComponent(head)
)
or
exists(SummaryComponent c |
stack = TSingletonSummaryComponentStack(c) and
result = getComponent(c)
)
}
/**
* A class that exists for QL technical reasons only (the IPA type used
* to represent component stacks needs to be bounded).
@@ -1382,8 +1349,8 @@ module Private {
c.relevantSummary(input, output, preservesValue) and
csv =
c.getCallableCsv() // Callable information
+ getComponentStack(input) + ";" // input
+ getComponentStack(output) + ";" // output
+ input.getMadRepresentation() + ";" // input
+ output.getMadRepresentation() + ";" // output
+ renderKind(preservesValue) + ";" // kind
+ renderProvenance(c) // provenance
)

28
python/ql/lib/semmle/python/dataflow/new/internal/FlowSummaryImplSpecific.qll
View File

@@ -128,10 +128,30 @@ SummaryComponent interpretComponentSpecific(AccessPathToken c) {
)
}
/** Gets the textual representation of a summary component in the format used for flow summaries. */
string getComponentSpecific(SummaryComponent sc) {
sc = TContentSummaryComponent(any(ListElementContent c)) and
result = "ListElement"
private string getContentSpecific(Content cs) {
cs = TListElementContent() and result = "ListElement"
or
cs = TSetElementContent() and result = "SetElement"
or
exists(int index |
cs = TTupleElementContent(index) and result = "TupleElement[" + index.toString() + "]"
)
or
exists(string key |
cs = TDictionaryElementContent(key) and result = "DictionaryElement[" + key + "]"
)
or
cs = TDictionaryElementAnyContent() and result = "DictionaryElementAny"
or
exists(string attr | cs = TAttributeContent(attr) and result = "Attribute[" + attr + "]")
}
/** Gets the textual representation of a summary component in the format used for MaD models. */
string getMadRepresentationSpecific(SummaryComponent sc) {
exists(Content c |
sc = TContentSummaryComponent(c) and
result = getContentSpecific(c)
)
}
/** Gets the textual representation of a parameter position in the format used for flow summaries. */

45
python/ql/lib/semmle/python/dataflow/new/internal/TypeTracker.qll
View File

@@ -55,10 +55,9 @@ private module Cached {
)
}
pragma[nomagic]
private TypeTracker noContentTypeTracker(boolean hasCall) {
result = MkTypeTracker(hasCall, noContent())
}
/** Gets a type tracker with no content and the call bit set to the given value. */
cached
TypeTracker noContentTypeTracker(boolean hasCall) { result = MkTypeTracker(hasCall, noContent()) }
/** Gets the summary resulting from appending `step` to type-tracking summary `tt`. */
cached
@@ -318,6 +317,8 @@ class StepSummary extends TStepSummary {
/** Provides predicates for updating step summaries (`StepSummary`s). */
module StepSummary {
predicate append = Cached::append/2;
/**
* Gets the summary that corresponds to having taken a forwards
* inter-procedural step from `nodeFrom` to `nodeTo`.
@@ -378,6 +379,35 @@ module StepSummary {
}
deprecated predicate localSourceStoreStep = flowsToStoreStep/3;
/** Gets the step summary for a level step. */
StepSummary levelStep() { result = LevelStep() }
/** Gets the step summary for a call step. */
StepSummary callStep() { result = CallStep() }
/** Gets the step summary for a return step. */
StepSummary returnStep() { result = ReturnStep() }
/** Gets the step summary for storing into `content`. */
StepSummary storeStep(TypeTrackerContent content) { result = StoreStep(content) }
/** Gets the step summary for loading from `content`. */
StepSummary loadStep(TypeTrackerContent content) { result = LoadStep(content) }
/** Gets the step summary for loading from `load` and then storing into `store`. */
StepSummary loadStoreStep(TypeTrackerContent load, TypeTrackerContent store) {
result = LoadStoreStep(load, store)
}
/** Gets the step summary for a step that only permits contents matched by `filter`. */
StepSummary withContent(ContentFilter filter) { result = WithContent(filter) }
/** Gets the step summary for a step that blocks contents matched by `filter`. */
StepSummary withoutContent(ContentFilter filter) { result = WithoutContent(filter) }
/** Gets the step summary for a jump step. */
StepSummary jumpStep() { result = JumpStep() }
}
/**
@@ -540,6 +570,13 @@ module TypeTracker {
* Gets a valid end point of type tracking.
*/
TypeTracker end() { result.end() }
/**
* INTERNAL USE ONLY.
*
* Gets a valid end point of type tracking with the call bit set to the given value.
*/
predicate end = Cached::noContentTypeTracker/1;
}
pragma[nomagic]

8
python/ql/lib/semmle/python/frameworks/MarkupSafe.qll
View File

@@ -83,7 +83,7 @@ private module MarkupSafeModel {
}
/** Taint propagation for `markupsafe.Markup`. */
private class AddtionalTaintStep extends TaintTracking::AdditionalTaintStep {
private class AdditionalTaintStep extends TaintTracking::AdditionalTaintStep {
override predicate step(DataFlow::Node nodeFrom, DataFlow::Node nodeTo) {
nodeTo.(ClassInstantiation).getArg(0) = nodeFrom
}
@@ -92,11 +92,7 @@ private module MarkupSafeModel {
/** Any escaping performed via the `markupsafe` package. */
abstract private class MarkupSafeEscape extends Escaping::Range {
override string getKind() {
// TODO: this package claims to escape for both HTML and XML, but for now we don't
// model XML.
result = Escaping::getHtmlKind()
}
override string getKind() { result in [Escaping::getHtmlKind(), Escaping::getXmlKind()] }
}
/** A call to any of the escaping functions in `markupsafe` */

7
python/ql/lib/semmle/python/security/dataflow/XxeCustomizations.qll
View File

@@ -44,4 +44,11 @@ module Xxe {
)
}
}
/**
* An XML escaping, considered as a sanitizer.
*/
class XmlEscapingAsSanitizer extends Sanitizer {
XmlEscapingAsSanitizer() { this = any(XmlEscaping esc).getOutput() }
}
}

10
python/ql/src/CHANGELOG.md
View File

@@ -1,3 +1,13 @@
## 0.8.0
### Bug Fixes
* The query "Arbitrary file write during archive extraction ("Zip Slip")" (`py/zipslip`) has been renamed to "Arbitrary file access during archive extraction ("Zip Slip")."
## 0.7.4
No user-facing changes.
## 0.7.3
### Bug Fixes

4
python/ql/src/change-notes/2023-07-06-aiohttp-clientsession-modeling.md Normal file
View File

@@ -0,0 +1,4 @@
---
category: minorAnalysis
---
* Fixed modeling of `aiohttp.ClientSession` so we properly handle `async with` uses. This can impact results of server-side request forgery queries (`py/full-ssrf`, `py/partial-ssrf`).

3
python/ql/src/change-notes/released/0.7.4.md Normal file
View File

@@ -0,0 +1,3 @@
## 0.7.4
No user-facing changes.

7
python/ql/src/change-notes/2023-06-16-zipslip-rename.md → python/ql/src/change-notes/released/0.8.0.md
View File

@@ -1,4 +1,5 @@
---
category: fix
---
## 0.8.0
### Bug Fixes
* The query "Arbitrary file write during archive extraction ("Zip Slip")" (`py/zipslip`) has been renamed to "Arbitrary file access during archive extraction ("Zip Slip")."

2
python/ql/src/codeql-pack.release.yml
View File

@@ -1,2 +1,2 @@
---
lastReleaseVersion: 0.7.3
lastReleaseVersion: 0.8.0

2
python/ql/src/qlpack.yml
View File

@@ -1,5 +1,5 @@
name: codeql/python-queries
version: 0.7.4-dev
version: 0.8.1-dev
groups:
- python
- queries

52
python/ql/test/library-tests/frameworks/aiohttp/client_request.py
View File

@@ -1,35 +1,35 @@
import aiohttp
import asyncio
import ssl
s = aiohttp.ClientSession()
resp = s.request("method", "url") # $ clientRequestUrlPart="url"
resp = s.request("method", url="url") # $ clientRequestUrlPart="url"
async def test():
s = aiohttp.ClientSession()
resp = await s.request("method", "url") # $ clientRequestUrlPart="url"
resp = await s.request("method", url="url") # $ clientRequestUrlPart="url"
with aiohttp.ClientSession() as session:
resp = session.get("url") # $ clientRequestUrlPart="url"
resp = session.request(method="GET", url="url") # $ clientRequestUrlPart="url"
async with aiohttp.ClientSession() as session:
resp = await session.get("url") # $ clientRequestUrlPart="url"
resp = await session.request(method="GET", url="url") # $ clientRequestUrlPart="url"
# other methods than GET
s = aiohttp.ClientSession()
resp = s.post("url") # $ clientRequestUrlPart="url"
resp = s.patch("url") # $ clientRequestUrlPart="url"
resp = s.options("url") # $ clientRequestUrlPart="url"
# other methods than GET
s = aiohttp.ClientSession()
resp = await s.post("url") # $ clientRequestUrlPart="url"
resp = await s.patch("url") # $ clientRequestUrlPart="url"
resp = await s.options("url") # $ clientRequestUrlPart="url"
# disabling of SSL validation
# see https://docs.aiohttp.org/en/stable/client_reference.html#aiohttp.ClientSession.request
s.get("url", ssl=False) # $ clientRequestUrlPart="url" clientRequestCertValidationDisabled
s.get("url", ssl=0) # $ clientRequestUrlPart="url" clientRequestCertValidationDisabled
# None is treated as default and so does _not_ disable the check
s.get("url", ssl=None) # $ clientRequestUrlPart="url"
# disabling of SSL validation
# see https://docs.aiohttp.org/en/stable/client_reference.html#aiohttp.ClientSession.request
s.get("url", ssl=False) # $ clientRequestUrlPart="url" clientRequestCertValidationDisabled
s.get("url", ssl=0) # $ clientRequestUrlPart="url" clientRequestCertValidationDisabled
# None is treated as default and so does _not_ disable the check
s.get("url", ssl=None) # $ clientRequestUrlPart="url"
# deprecated since 3.0, but still supported
s.get("url", verify_ssl=False) # $ clientRequestUrlPart="url" clientRequestCertValidationDisabled
# deprecated since 3.0, but still supported
s.get("url", verify_ssl=False) # $ clientRequestUrlPart="url" clientRequestCertValidationDisabled
# A manually constructed SSLContext does not have safe defaults, so is effectively the
# same as turning off SSL validation
context = ssl.SSLContext()
assert context.check_hostname == False
assert context.verify_mode == ssl.VerifyMode.CERT_NONE
# A manually constructed SSLContext does not have safe defaults, so is effectively the
# same as turning off SSL validation
context = ssl.SSLContext()
assert context.check_hostname == False
assert context.verify_mode == ssl.VerifyMode.CERT_NONE
s.get("url", ssl=context) # $ clientRequestUrlPart="url" MISSING: clientRequestCertValidationDisabled
s.get("url", ssl=context) # $ clientRequestUrlPart="url" MISSING: clientRequestCertValidationDisabled

40
python/ql/test/library-tests/frameworks/markupsafe/taint_test.py
View File

@@ -27,40 +27,40 @@ def test():
# as tainted even after it has been escaped in some place. This _might_ not be the
# case since data-flow library has taint-steps from adjacent uses...
ensure_tainted(ts) # $ tainted
ensure_not_tainted(escape(ts)) # $ escapeInput=ts escapeKind=html escapeOutput=escape(..)
ensure_not_tainted(escape(ts)) # $ escapeInput=ts escapeKind=html escapeKind=xml escapeOutput=escape(..)
ensure_tainted(ts) # $ tainted
ensure_tainted(
ts, # $ tainted
m_unsafe, # $ tainted
m_unsafe + SAFE, # $ escapeInput=SAFE escapeKind=html escapeOutput=BinaryExpr MISSING: tainted
SAFE + m_unsafe, # $ escapeInput=SAFE escapeKind=html escapeOutput=BinaryExpr MISSING: tainted
m_unsafe.format(SAFE), # $ escapeInput=SAFE escapeKind=html escapeOutput=m_unsafe.format(..) MISSING: tainted
m_unsafe % SAFE, # $ escapeInput=SAFE escapeKind=html escapeOutput=BinaryExpr MISSING: tainted
m_unsafe + ts, # $ escapeInput=ts escapeKind=html escapeOutput=BinaryExpr MISSING: tainted
m_unsafe + SAFE, # $ escapeInput=SAFE escapeKind=html escapeKind=xml escapeOutput=BinaryExpr MISSING: tainted
SAFE + m_unsafe, # $ escapeInput=SAFE escapeKind=html escapeKind=xml escapeOutput=BinaryExpr MISSING: tainted
m_unsafe.format(SAFE), # $ escapeInput=SAFE escapeKind=html escapeKind=xml escapeOutput=m_unsafe.format(..) MISSING: tainted
m_unsafe % SAFE, # $ escapeInput=SAFE escapeKind=html escapeKind=xml escapeOutput=BinaryExpr MISSING: tainted
m_unsafe + ts, # $ escapeInput=ts escapeKind=html escapeKind=xml escapeOutput=BinaryExpr MISSING: tainted
m_safe.format(m_unsafe), # $ tainted
m_safe % m_unsafe, # $ tainted
escape(ts).unescape(), # $ escapeInput=ts escapeKind=html escapeOutput=escape(..) MISSING: tainted
escape_silent(ts).unescape(), # $ escapeInput=ts escapeKind=html escapeOutput=escape_silent(..) MISSING: tainted
escape(ts).unescape(), # $ escapeInput=ts escapeKind=html escapeKind=xml escapeOutput=escape(..) MISSING: tainted
escape_silent(ts).unescape(), # $ escapeInput=ts escapeKind=html escapeKind=xml escapeOutput=escape_silent(..) MISSING: tainted
)
ensure_not_tainted(
escape(ts), # $ escapeInput=ts escapeKind=html escapeOutput=escape(..)
escape_silent(ts), # $ escapeInput=ts escapeKind=html escapeOutput=escape_silent(..)
escape(ts), # $ escapeInput=ts escapeKind=html escapeKind=xml escapeOutput=escape(..)
escape_silent(ts), # $ escapeInput=ts escapeKind=html escapeKind=xml escapeOutput=escape_silent(..)
Markup.escape(ts), # $ escapeInput=ts escapeKind=html escapeOutput=Markup.escape(..)
Markup.escape(ts), # $ escapeInput=ts escapeKind=html escapeKind=xml escapeOutput=Markup.escape(..)
m_safe,
m_safe + ts, # $ escapeInput=ts escapeKind=html escapeOutput=BinaryExpr
ts + m_safe, # $ escapeInput=ts escapeKind=html escapeOutput=BinaryExpr
m_safe.format(ts), # $ escapeInput=ts escapeKind=html escapeOutput=m_safe.format(..)
m_safe % ts, # $ escapeInput=ts escapeKind=html escapeOutput=BinaryExpr
m_safe + ts, # $ escapeInput=ts escapeKind=html escapeKind=xml escapeOutput=BinaryExpr
ts + m_safe, # $ escapeInput=ts escapeKind=html escapeKind=xml escapeOutput=BinaryExpr
m_safe.format(ts), # $ escapeInput=ts escapeKind=html escapeKind=xml escapeOutput=m_safe.format(..)
m_safe % ts, # $ escapeInput=ts escapeKind=html escapeKind=xml escapeOutput=BinaryExpr
escape(ts) + ts, # $ escapeInput=ts escapeKind=html escapeOutput=BinaryExpr escapeOutput=escape(..)
escape_silent(ts) + ts, # $ escapeInput=ts escapeKind=html escapeOutput=BinaryExpr escapeOutput=escape_silent(..)
Markup.escape(ts) + ts, # $ escapeInput=ts escapeKind=html escapeOutput=BinaryExpr escapeOutput=Markup.escape(..)
escape(ts) + ts, # $ escapeInput=ts escapeKind=html escapeKind=xml escapeOutput=BinaryExpr escapeOutput=escape(..)
escape_silent(ts) + ts, # $ escapeInput=ts escapeKind=html escapeKind=xml escapeOutput=BinaryExpr escapeOutput=escape_silent(..)
Markup.escape(ts) + ts, # $ escapeInput=ts escapeKind=html escapeKind=xml escapeOutput=BinaryExpr escapeOutput=Markup.escape(..)
)
# flask re-exports these, as:
@@ -73,8 +73,8 @@ def test():
)
ensure_not_tainted(
flask.escape(ts), # $ escapeInput=ts escapeKind=html escapeOutput=flask.escape(..)
flask.Markup.escape(ts), # $ escapeInput=ts escapeKind=html escapeOutput=flask.Markup.escape(..)
flask.escape(ts), # $ escapeInput=ts escapeKind=html escapeKind=xml escapeOutput=flask.escape(..)
flask.Markup.escape(ts), # $ escapeInput=ts escapeKind=html escapeKind=xml escapeOutput=flask.Markup.escape(..)
)

36
python/ql/test/query-tests/Security/CWE-611-Xxe/Xxe.expected
View File

@@ -1,25 +1,25 @@
edges
| test.py:1:26:1:32 | ControlFlowNode for ImportMember | test.py:1:26:1:32 | GSSA Variable request |
| test.py:1:26:1:32 | GSSA Variable request | test.py:8:19:8:25 | ControlFlowNode for request |
| test.py:1:26:1:32 | GSSA Variable request | test.py:19:19:19:25 | ControlFlowNode for request |
| test.py:8:19:8:25 | ControlFlowNode for request | test.py:8:19:8:30 | ControlFlowNode for Attribute |
| test.py:8:19:8:30 | ControlFlowNode for Attribute | test.py:8:19:8:45 | ControlFlowNode for Subscript |
| test.py:8:19:8:45 | ControlFlowNode for Subscript | test.py:9:34:9:44 | ControlFlowNode for xml_content |
| test.py:19:19:19:25 | ControlFlowNode for request | test.py:19:19:19:30 | ControlFlowNode for Attribute |
| test.py:19:19:19:30 | ControlFlowNode for Attribute | test.py:19:19:19:45 | ControlFlowNode for Subscript |
| test.py:19:19:19:45 | ControlFlowNode for Subscript | test.py:30:34:30:44 | ControlFlowNode for xml_content |
| test.py:1:26:1:32 | GSSA Variable request | test.py:9:19:9:25 | ControlFlowNode for request |
| test.py:1:26:1:32 | GSSA Variable request | test.py:20:19:20:25 | ControlFlowNode for request |
| test.py:9:19:9:25 | ControlFlowNode for request | test.py:9:19:9:30 | ControlFlowNode for Attribute |
| test.py:9:19:9:30 | ControlFlowNode for Attribute | test.py:9:19:9:45 | ControlFlowNode for Subscript |
| test.py:9:19:9:45 | ControlFlowNode for Subscript | test.py:10:34:10:44 | ControlFlowNode for xml_content |
| test.py:20:19:20:25 | ControlFlowNode for request | test.py:20:19:20:30 | ControlFlowNode for Attribute |
| test.py:20:19:20:30 | ControlFlowNode for Attribute | test.py:20:19:20:45 | ControlFlowNode for Subscript |
| test.py:20:19:20:45 | ControlFlowNode for Subscript | test.py:31:34:31:44 | ControlFlowNode for xml_content |
nodes
| test.py:1:26:1:32 | ControlFlowNode for ImportMember | semmle.label | ControlFlowNode for ImportMember |
| test.py:1:26:1:32 | GSSA Variable request | semmle.label | GSSA Variable request |
| test.py:8:19:8:25 | ControlFlowNode for request | semmle.label | ControlFlowNode for request |
| test.py:8:19:8:30 | ControlFlowNode for Attribute | semmle.label | ControlFlowNode for Attribute |
| test.py:8:19:8:45 | ControlFlowNode for Subscript | semmle.label | ControlFlowNode for Subscript |
| test.py:9:34:9:44 | ControlFlowNode for xml_content | semmle.label | ControlFlowNode for xml_content |
| test.py:19:19:19:25 | ControlFlowNode for request | semmle.label | ControlFlowNode for request |
| test.py:19:19:19:30 | ControlFlowNode for Attribute | semmle.label | ControlFlowNode for Attribute |
| test.py:19:19:19:45 | ControlFlowNode for Subscript | semmle.label | ControlFlowNode for Subscript |
| test.py:30:34:30:44 | ControlFlowNode for xml_content | semmle.label | ControlFlowNode for xml_content |
| test.py:9:19:9:25 | ControlFlowNode for request | semmle.label | ControlFlowNode for request |
| test.py:9:19:9:30 | ControlFlowNode for Attribute | semmle.label | ControlFlowNode for Attribute |
| test.py:9:19:9:45 | ControlFlowNode for Subscript | semmle.label | ControlFlowNode for Subscript |
| test.py:10:34:10:44 | ControlFlowNode for xml_content | semmle.label | ControlFlowNode for xml_content |
| test.py:20:19:20:25 | ControlFlowNode for request | semmle.label | ControlFlowNode for request |
| test.py:20:19:20:30 | ControlFlowNode for Attribute | semmle.label | ControlFlowNode for Attribute |
| test.py:20:19:20:45 | ControlFlowNode for Subscript | semmle.label | ControlFlowNode for Subscript |
| test.py:31:34:31:44 | ControlFlowNode for xml_content | semmle.label | ControlFlowNode for xml_content |
subpaths
#select
| test.py:9:34:9:44 | ControlFlowNode for xml_content | test.py:1:26:1:32 | ControlFlowNode for ImportMember | test.py:9:34:9:44 | ControlFlowNode for xml_content | XML parsing depends on a $@ without guarding against external entity expansion. | test.py:1:26:1:32 | ControlFlowNode for ImportMember | user-provided value |
| test.py:30:34:30:44 | ControlFlowNode for xml_content | test.py:1:26:1:32 | ControlFlowNode for ImportMember | test.py:30:34:30:44 | ControlFlowNode for xml_content | XML parsing depends on a $@ without guarding against external entity expansion. | test.py:1:26:1:32 | ControlFlowNode for ImportMember | user-provided value |
| test.py:10:34:10:44 | ControlFlowNode for xml_content | test.py:1:26:1:32 | ControlFlowNode for ImportMember | test.py:10:34:10:44 | ControlFlowNode for xml_content | XML parsing depends on a $@ without guarding against external entity expansion. | test.py:1:26:1:32 | ControlFlowNode for ImportMember | user-provided value |
| test.py:31:34:31:44 | ControlFlowNode for xml_content | test.py:1:26:1:32 | ControlFlowNode for ImportMember | test.py:31:34:31:44 | ControlFlowNode for xml_content | XML parsing depends on a $@ without guarding against external entity expansion. | test.py:1:26:1:32 | ControlFlowNode for ImportMember | user-provided value |

7
python/ql/test/query-tests/Security/CWE-611-Xxe/test.py
View File

@@ -1,5 +1,6 @@
from flask import Flask, request
import lxml.etree
import markupsafe
app = Flask(__name__)
@@ -28,3 +29,9 @@ def super_vuln_handler():
huge_tree=True,
)
return lxml.etree.fromstring(xml_content, parser=parser).text
@app.route("/sanitized-handler")
def sanitized_handler():
xml_content = request.args['xml_content']
xml_content = markupsafe.escape(xml_content)
return lxml.etree.fromstring(xml_content).text