diff --git a/change-notes/1.25/analysis-java.md b/change-notes/1.25/analysis-java.md index 7cdd9e491a2..ab11e5aaaf1 100644 --- a/change-notes/1.25/analysis-java.md +++ b/change-notes/1.25/analysis-java.md @@ -4,20 +4,26 @@ The following changes in version 1.25 affect Java analysis in all applications. ## General improvements -## New queries - -| **Query** | **Tags** | **Purpose** | -|-----------------------------|-----------|--------------------------------------------------------------------| - +The Java autobuilder has been improved to detect more Gradle Java versions. ## Changes to existing queries | **Query** | **Expected impact** | **Change** | |------------------------------|------------------------|-----------------------------------| - +| Hard-coded credential in API call (`java/hardcoded-credential-api-call`) | More results | The query now recognizes the `BasicAWSCredentials` class of the Amazon client SDK library with hardcoded access key/secret key. | +| Deserialization of user-controlled data (`java/unsafe-deserialization`) | Fewer false positive results | The query no longer reports results using `org.apache.commons.io.serialization.ValidatingObjectInputStream`. | +| Use of a broken or risky cryptographic algorithm (`java/weak-cryptographic-algorithm`) | More results | The query now recognizes the `MessageDigest.getInstance` method. | +| Use of a potentially broken or risky cryptographic algorithm (`java/potentially-weak-cryptographic-algorithm`) | More results | The query now recognizes the `MessageDigest.getInstance` method. | +| Reading from a world writable file (`java/world-writable-file-read`) | More results | The query now recognizes more JDK file operations. | ## Changes to libraries +* The data-flow library has been improved with more taint flow modeling for the + Collections framework and other classes of the JDK. This affects all security + queries using data flow and can yield additional results. +* The data-flow library has been improved with more taint flow modeling for the + Spring framework. This affects all security queries using data flow and can + yield additional results on project that rely on the Spring framework. * The data-flow library has been improved, which affects most security queries by potentially adding more results. Flow through methods now takes nested field reads/writes into account. For example, the library is able to track flow from `"taint"` to `sink()` via the method @@ -39,3 +45,5 @@ The following changes in version 1.25 affect Java analysis in all applications. } } ``` +* The library has been extended with more support for Java 14 features + (`switch` expressions and pattern-matching for `instanceof`). diff --git a/change-notes/1.25/analysis-python.md b/change-notes/1.25/analysis-python.md index 5d0fc69ec80..ed3496bc734 100644 --- a/change-notes/1.25/analysis-python.md +++ b/change-notes/1.25/analysis-python.md @@ -1,22 +1,9 @@ # Improvements to Python analysis -The following changes in version 1.25 affect Python analysis in all applications. - -## General improvements - - -## New queries - -| **Query** | **Tags** | **Purpose** | -|-----------------------------|-----------|--------------------------------------------------------------------| - - -## Changes to existing queries - -| **Query** | **Expected impact** | **Change** | -|----------------------------|------------------------|------------------------------------------------------------------| - - -## Changes to libraries - * Importing `semmle.python.web.HttpRequest` will no longer import `UntrustedStringKind` transitively. `UntrustedStringKind` is the most commonly used non-abstract subclass of `ExternalStringKind`. If not imported (by one mean or another), taint-tracking queries that concern `ExternalStringKind` will not produce any results. Please ensure such queries contain an explicit import (`import semmle.python.security.strings.Untrusted`). +* Added model of taint sources for HTTP servers using `http.server`. +* Added taint modeling of routed parameters in Flask. +* Improved modeling of built-in methods on strings for taint tracking. +* Improved classification of test files. +* New class `BoundMethodValue` represents a bound method during runtime. +* The query `py/command-line-injection` now recognizes command execution with the `fabric` and `invoke` Python libraries. diff --git a/change-notes/1.26/analysis-csharp.md b/change-notes/1.26/analysis-csharp.md index 5b65481c925..3d17e00ab70 100644 --- a/change-notes/1.26/analysis-csharp.md +++ b/change-notes/1.26/analysis-csharp.md @@ -12,7 +12,7 @@ The following changes in version 1.26 affect C# analysis in all applications. | **Query** | **Expected impact** | **Change** | |------------------------------|------------------------|-----------------------------------| - +| Weak encryption: Insufficient key size (`cs/insufficient-key-size`) | More results | The required key size has been increased from 1024 to 2048. | ## Removal of old queries diff --git a/change-notes/1.26/analysis-javascript.md b/change-notes/1.26/analysis-javascript.md index b371f7384a4..f035156652f 100644 --- a/change-notes/1.26/analysis-javascript.md +++ b/change-notes/1.26/analysis-javascript.md @@ -14,12 +14,14 @@ - [json-stringify-safe](https://www.npmjs.com/package/json-stringify-safe) - [json3](https://www.npmjs.com/package/json3) - [lodash](https://www.npmjs.com/package/lodash) + - [needle](https://www.npmjs.com/package/needle) - [object-inspect](https://www.npmjs.com/package/object-inspect) - [pretty-format](https://www.npmjs.com/package/pretty-format) - [stringify-object](https://www.npmjs.com/package/stringify-object) - [underscore](https://www.npmjs.com/package/underscore) * Analyzing files with the ".cjs" extension is now supported. +* ES2021 features are now supported. ## New queries @@ -38,6 +40,7 @@ | Unsafe shell command constructed from library input (`js/shell-command-constructed-from-input`) | More results | This query now recognizes more commands where colon, dash, and underscore are used. | | Unsafe jQuery plugin (`js/unsafe-jquery-plugin`) | More results | This query now detects more unsafe uses of nested option properties. | | Client-side URL redirect (`js/client-side-unvalidated-url-redirection`) | More results | This query now recognizes some unsafe uses of `importScripts()` inside WebWorkers. | +| Missing CSRF middleware (`js/missing-token-validation`) | More results | This query now recognizes writes to cookie and session variables as potentially vulnerable to CSRF attacks. | ## Changes to libraries diff --git a/config/identical-files.json b/config/identical-files.json index 7e62497c9bf..e0a4e8dabf4 100644 --- a/config/identical-files.json +++ b/config/identical-files.json @@ -50,6 +50,18 @@ "csharp/ql/src/semmle/code/csharp/dataflow/internal/DataFlowImplConsistency.qll", "python/ql/src/experimental/dataflow/internal/DataFlowImplConsistency.qll" ], + "SsaReadPosition Java/C#": [ + "java/ql/src/semmle/code/java/dataflow/internal/rangeanalysis/SsaReadPositionCommon.qll", + "csharp/ql/src/semmle/code/csharp/dataflow/internal/rangeanalysis/SsaReadPositionCommon.qll" + ], + "Sign Java/C#": [ + "java/ql/src/semmle/code/java/dataflow/internal/rangeanalysis/Sign.qll", + "csharp/ql/src/semmle/code/csharp/dataflow/internal/rangeanalysis/Sign.qll" + ], + "SignAnalysis Java/C#": [ + "java/ql/src/semmle/code/java/dataflow/internal/rangeanalysis/SignAnalysisCommon.qll", + "csharp/ql/src/semmle/code/csharp/dataflow/internal/rangeanalysis/SignAnalysisCommon.qll" + ], "C++ SubBasicBlocks": [ "cpp/ql/src/semmle/code/cpp/controlflow/SubBasicBlocks.qll", "cpp/ql/src/semmle/code/cpp/dataflow/internal/SubBasicBlocks.qll" @@ -87,7 +99,7 @@ "cpp/ql/src/semmle/code/cpp/ir/implementation/unaliased_ssa/Operand.qll", "cpp/ql/src/semmle/code/cpp/ir/implementation/aliased_ssa/Operand.qll", "csharp/ql/src/experimental/ir/implementation/raw/Operand.qll", - "csharp/ql/src/experimental/ir/implementation/unaliased_ssa/Operand.qll" + "csharp/ql/src/experimental/ir/implementation/unaliased_ssa/Operand.qll" ], "IR IRType": [ "cpp/ql/src/semmle/code/cpp/ir/implementation/IRType.qll", @@ -109,11 +121,11 @@ "cpp/ql/src/semmle/code/cpp/ir/implementation/internal/OperandTag.qll", "csharp/ql/src/experimental/ir/implementation/internal/OperandTag.qll" ], - "IR TInstruction":[ + "IR TInstruction": [ "cpp/ql/src/semmle/code/cpp/ir/implementation/internal/TInstruction.qll", "csharp/ql/src/experimental/ir/implementation/internal/TInstruction.qll" ], - "IR TIRVariable":[ + "IR TIRVariable": [ "cpp/ql/src/semmle/code/cpp/ir/implementation/internal/TIRVariable.qll", "csharp/ql/src/experimental/ir/implementation/internal/TIRVariable.qll" ], @@ -381,4 +393,4 @@ "javascript/ql/src/Comments/CommentedOutCodeReferences.qhelp", "python/ql/src/Lexical/CommentedOutCodeReferences.qhelp" ] -} +} \ No newline at end of file diff --git a/cpp/ql/src/Security/CWE/CWE-022/TaintedPath.qhelp b/cpp/ql/src/Security/CWE/CWE-022/TaintedPath.qhelp index d7432369f0e..eba2ede58f5 100644 --- a/cpp/ql/src/Security/CWE/CWE-022/TaintedPath.qhelp +++ b/cpp/ql/src/Security/CWE/CWE-022/TaintedPath.qhelp @@ -39,7 +39,7 @@ access all the system's passwords.

  • OWASP: -Path Traversal. +Path Traversal.
    • diff --git a/cpp/ql/src/experimental/semmle/code/cpp/models/interfaces/SimpleRangeAnalysisDefinition.qll b/cpp/ql/src/experimental/semmle/code/cpp/models/interfaces/SimpleRangeAnalysisDefinition.qll new file mode 100644 index 00000000000..0cf570a72b4 --- /dev/null +++ b/cpp/ql/src/experimental/semmle/code/cpp/models/interfaces/SimpleRangeAnalysisDefinition.qll @@ -0,0 +1,65 @@ +/** + * EXPERIMENTAL: The API of this module may change without notice. + * + * Provides a class for modeling `RangeSsaDefinition`s with a restricted range. + */ + +import cpp +import semmle.code.cpp.rangeanalysis.SimpleRangeAnalysis + +/** + * EXPERIMENTAL: The API of this class may change without notice. + * + * An SSA definition for which a range can be deduced. As with + * `RangeSsaDefinition` and `SsaDefinition`, instances of this class + * correspond to points in the program where one or more variables are defined + * or have their value constrained in some way. + * + * Extend this class to add functionality to the range analysis library. + */ +abstract class SimpleRangeAnalysisDefinition extends RangeSsaDefinition { + /** + * Holds if this `SimpleRangeAnalysisDefinition` adds range information for + * `v`. Because a `SimpleRangeAnalysisDefinition` is just a point in the + * program, it's possible that more than one variable might be defined at + * this point. This predicate clarifies which variable(s) should get range + * information from `this`. + * + * This predicate **must be overridden** to hold for any `v` that can show + * up in the other members of `SimpleRangeAnalysisDefinition`. Conversely, + * the other members **must be accurate** for any `v` in this predicate. + */ + abstract predicate hasRangeInformationFor(StackVariable v); + + /** + * Holds if `(this, v)` depends on the range of the unconverted expression + * `e`. This information is used to inform the range analysis about cyclic + * dependencies. Without this information, range analysis might work for + * simple cases but will go into infinite loops on complex code. + * + * For example, when modelling the definition by reference in a call to an + * overloaded `operator=`, written as `v = e`, the definition of `(this, v)` + * depends on `e`. + */ + abstract predicate dependsOnExpr(StackVariable v, Expr e); + + /** + * Gets the lower bound of the variable `v` defined by this definition. + * + * Implementations of this predicate should use + * `getFullyConvertedLowerBounds` and `getFullyConvertedUpperBounds` for + * recursive calls to get the bounds of their dependencies. + */ + abstract float getLowerBounds(StackVariable v); + + /** + * Gets the upper bound of the variable `v` defined by this definition. + * + * Implementations of this predicate should use + * `getFullyConvertedLowerBounds` and `getFullyConvertedUpperBounds` for + * recursive calls to get the bounds of their dependencies. + */ + abstract float getUpperBounds(StackVariable v); +} + +import SimpleRangeAnalysisInternal diff --git a/cpp/ql/src/experimental/semmle/code/cpp/rangeanalysis/ExtendedRangeAnalysis.qll b/cpp/ql/src/experimental/semmle/code/cpp/rangeanalysis/ExtendedRangeAnalysis.qll new file mode 100644 index 00000000000..4c9b0c738f4 --- /dev/null +++ b/cpp/ql/src/experimental/semmle/code/cpp/rangeanalysis/ExtendedRangeAnalysis.qll @@ -0,0 +1,4 @@ +import semmle.code.cpp.rangeanalysis.SimpleRangeAnalysis +// +// Import each extension we want to enable +import extensions.SubtractSelf diff --git a/cpp/ql/src/experimental/semmle/code/cpp/rangeanalysis/extensions/SubtractSelf.qll b/cpp/ql/src/experimental/semmle/code/cpp/rangeanalysis/extensions/SubtractSelf.qll new file mode 100644 index 00000000000..ff716d02d6f --- /dev/null +++ b/cpp/ql/src/experimental/semmle/code/cpp/rangeanalysis/extensions/SubtractSelf.qll @@ -0,0 +1,15 @@ +import experimental.semmle.code.cpp.models.interfaces.SimpleRangeAnalysisExpr + +private class SelfSub extends SimpleRangeAnalysisExpr, SubExpr { + SelfSub() { + // Match `x - x` but not `myInt - (unsigned char)myInt`. + getLeftOperand().getExplicitlyConverted().(VariableAccess).getTarget() = + getRightOperand().getExplicitlyConverted().(VariableAccess).getTarget() + } + + override float getLowerBounds() { result = 0 } + + override float getUpperBounds() { result = 0 } + + override predicate dependsOnChild(Expr child) { none() } +} diff --git a/cpp/ql/src/semmle/code/cpp/Variable.qll b/cpp/ql/src/semmle/code/cpp/Variable.qll index 597facea442..527d8879b96 100644 --- a/cpp/ql/src/semmle/code/cpp/Variable.qll +++ b/cpp/ql/src/semmle/code/cpp/Variable.qll @@ -144,6 +144,11 @@ class Variable extends Declaration, @variable { */ predicate isConstexpr() { this.hasSpecifier("is_constexpr") } + /** + * Holds if this variable is declared `constinit`. + */ + predicate isConstinit() { this.hasSpecifier("declared_constinit") } + /** * Holds if this variable is `thread_local`. */ diff --git a/cpp/ql/src/semmle/code/cpp/dataflow/internal/DataFlowPrivate.qll b/cpp/ql/src/semmle/code/cpp/dataflow/internal/DataFlowPrivate.qll index 42c31bca69c..4562eb3fd19 100644 --- a/cpp/ql/src/semmle/code/cpp/dataflow/internal/DataFlowPrivate.qll +++ b/cpp/ql/src/semmle/code/cpp/dataflow/internal/DataFlowPrivate.qll @@ -1,6 +1,7 @@ private import cpp private import DataFlowUtil private import DataFlowDispatch +private import FlowVar /** Gets the instance argument of a non-static call. */ private Node getInstanceArgument(Call call) { @@ -106,7 +107,7 @@ private class ExprOutNode extends OutNode, ExprNode { override DataFlowCall getCall() { result = this.getExpr() } } -private class RefOutNode extends OutNode, DefinitionByReferenceNode { +private class RefOutNode extends OutNode, DefinitionByReferenceOrIteratorNode { /** Gets the underlying call. */ override DataFlowCall getCall() { result = this.getArgument().getParent() } } @@ -120,7 +121,7 @@ OutNode getAnOutNode(DataFlowCall call, ReturnKind kind) { kind = TNormalReturnKind() or exists(int i | - result.asDefiningArgument() = call.getArgument(i) and + result.(DefinitionByReferenceOrIteratorNode).getArgument() = call.getArgument(i) and kind = TRefReturnKind(i) ) } diff --git a/cpp/ql/src/semmle/code/cpp/dataflow/internal/DataFlowUtil.qll b/cpp/ql/src/semmle/code/cpp/dataflow/internal/DataFlowUtil.qll index 097332bee06..09409eb30f2 100644 --- a/cpp/ql/src/semmle/code/cpp/dataflow/internal/DataFlowUtil.qll +++ b/cpp/ql/src/semmle/code/cpp/dataflow/internal/DataFlowUtil.qll @@ -183,28 +183,29 @@ class ImplicitParameterNode extends ParameterNode, TInstanceParameterNode { } /** - * A node that represents the value of a variable after a function call that - * may have changed the variable because it's passed by reference. + * INTERNAL: do not use. * - * A typical example would be a call `f(&x)`. Firstly, there will be flow into - * `x` from previous definitions of `x`. Secondly, there will be a - * `DefinitionByReferenceNode` to represent the value of `x` after the call has - * returned. This node will have its `getArgument()` equal to `&x`. + * A node that represents the value of a variable after a function call that + * may have changed the variable because it's passed by reference or because an + * iterator for it was passed by value or by reference. */ -class DefinitionByReferenceNode extends PartialDefinitionNode { +class DefinitionByReferenceOrIteratorNode extends PartialDefinitionNode { Expr inner; Expr argument; - DefinitionByReferenceNode() { - this.getPartialDefinition().(DefinitionByReference).definesExpressions(inner, argument) + DefinitionByReferenceOrIteratorNode() { + this.getPartialDefinition().definesExpressions(inner, argument) and + ( + this.getPartialDefinition() instanceof DefinitionByReference + or + this.getPartialDefinition() instanceof DefinitionByIterator + ) } override Function getFunction() { result = inner.getEnclosingFunction() } override Type getType() { result = inner.getType() } - override string toString() { result = "ref arg " + argument.toString() } - override Location getLocation() { result = argument.getLocation() } override ExprNode getPreUpdateNode() { result.getExpr() = argument } @@ -221,6 +222,21 @@ class DefinitionByReferenceNode extends PartialDefinitionNode { } } +/** + * A node that represents the value of a variable after a function call that + * may have changed the variable because it's passed by reference. + * + * A typical example would be a call `f(&x)`. Firstly, there will be flow into + * `x` from previous definitions of `x`. Secondly, there will be a + * `DefinitionByReferenceNode` to represent the value of `x` after the call has + * returned. This node will have its `getArgument()` equal to `&x`. + */ +class DefinitionByReferenceNode extends DefinitionByReferenceOrIteratorNode { + override VariablePartialDefinition pd; + + override string toString() { result = "ref arg " + argument.toString() } +} + /** * The value of an uninitialized local variable, viewed as a node in a data * flow graph. @@ -284,13 +300,11 @@ abstract class PostUpdateNode extends Node { override Location getLocation() { result = getPreUpdateNode().getLocation() } } -private class PartialDefinitionNode extends PostUpdateNode, TPartialDefinitionNode { +abstract private class PartialDefinitionNode extends PostUpdateNode, TPartialDefinitionNode { PartialDefinition pd; PartialDefinitionNode() { this = TPartialDefinitionNode(pd) } - override Node getPreUpdateNode() { pd.definesExpressions(_, result.asExpr()) } - override Location getLocation() { result = pd.getActualLocation() } PartialDefinition getPartialDefinition() { result = pd } @@ -298,6 +312,24 @@ private class PartialDefinitionNode extends PostUpdateNode, TPartialDefinitionNo override string toString() { result = getPreUpdateNode().toString() + " [post update]" } } +private class VariablePartialDefinitionNode extends PartialDefinitionNode { + override VariablePartialDefinition pd; + + override Node getPreUpdateNode() { pd.definesExpressions(_, result.asExpr()) } +} + +/** + * INTERNAL: do not use. + * + * A synthetic data flow node used for flow into a collection when an iterator + * write occurs in a callee. + */ +class IteratorPartialDefinitionNode extends PartialDefinitionNode { + override IteratorPartialDefinition pd; + + override Node getPreUpdateNode() { pd.definesExpressions(_, result.asExpr()) } +} + /** * A post-update node on the `e->f` in `f(&e->f)` (and other forms). */ diff --git a/cpp/ql/src/semmle/code/cpp/dataflow/internal/FlowVar.qll b/cpp/ql/src/semmle/code/cpp/dataflow/internal/FlowVar.qll index 6dc5da2f20b..50395dbaafc 100644 --- a/cpp/ql/src/semmle/code/cpp/dataflow/internal/FlowVar.qll +++ b/cpp/ql/src/semmle/code/cpp/dataflow/internal/FlowVar.qll @@ -6,6 +6,7 @@ import cpp private import semmle.code.cpp.controlflow.SSA private import semmle.code.cpp.dataflow.internal.SubBasicBlocks private import semmle.code.cpp.dataflow.internal.AddressFlow +private import semmle.code.cpp.models.implementations.Iterator /** * A conceptual variable that is assigned only once, like an SSA variable. This @@ -108,21 +109,12 @@ class FlowVar extends TFlowVar { * ``` */ private module PartialDefinitions { - class PartialDefinition extends Expr { - Expr innerDefinedExpr; + abstract class PartialDefinition extends Expr { ControlFlowNode node; - PartialDefinition() { - exists(Expr convertedInner | - valueToUpdate(convertedInner, this.getFullyConverted(), node) and - innerDefinedExpr = convertedInner.getUnconverted() and - not this instanceof Conversion - ) - } + abstract deprecated predicate partiallyDefines(Variable v); - deprecated predicate partiallyDefines(Variable v) { innerDefinedExpr = v.getAnAccess() } - - deprecated predicate partiallyDefinesThis(ThisExpr e) { innerDefinedExpr = e } + abstract deprecated predicate partiallyDefinesThis(ThisExpr e); /** * Gets the subBasicBlock where this `PartialDefinition` is defined. @@ -133,11 +125,9 @@ private module PartialDefinitions { * Holds if this `PartialDefinition` defines variable `v` at control-flow * node `cfn`. */ + // does this work with a dispred? pragma[noinline] - predicate partiallyDefinesVariableAt(Variable v, ControlFlowNode cfn) { - innerDefinedExpr = v.getAnAccess() and - cfn = node - } + abstract predicate partiallyDefinesVariableAt(Variable v, ControlFlowNode cfn); /** * Holds if this partial definition may modify `inner` (or what it points @@ -147,10 +137,7 @@ private module PartialDefinitions { * - `inner` = `... .x`, `outer` = `&...` * - `inner` = `a`, `outer` = `*` */ - predicate definesExpressions(Expr inner, Expr outer) { - inner = innerDefinedExpr and - outer = this - } + abstract predicate definesExpressions(Expr inner, Expr outer); /** * Gets the location of this element, adjusted to avoid unknown locations @@ -166,10 +153,107 @@ private module PartialDefinitions { } } + class IteratorPartialDefinition extends PartialDefinition { + Variable collection; + Expr innerDefinedExpr; + + IteratorPartialDefinition() { + exists(Expr convertedInner | + not this instanceof Conversion and + valueToUpdate(convertedInner, this.getFullyConverted(), node) and + innerDefinedExpr = convertedInner.getUnconverted() and + ( + innerDefinedExpr.(Call).getQualifier() = getAnIteratorAccess(collection) + or + innerDefinedExpr.(Call).getQualifier() = collection.getAnAccess() and + collection instanceof IteratorParameter + ) and + innerDefinedExpr.(Call).getTarget() instanceof IteratorPointerDereferenceMemberOperator + ) + or + // iterators passed by value without a copy constructor + exists(Call call | + call = node and + call.getAnArgument() = innerDefinedExpr and + innerDefinedExpr = this and + this = getAnIteratorAccess(collection) and + not call.getTarget() instanceof IteratorPointerDereferenceMemberOperator + ) + or + // iterators passed by value with a copy constructor + exists(Call call, ConstructorCall copy | + copy.getTarget() instanceof CopyConstructor and + call = node and + call.getAnArgument() = copy and + copy.getArgument(0) = getAnIteratorAccess(collection) and + innerDefinedExpr = this and + this = copy and + not call.getTarget() instanceof IteratorPointerDereferenceMemberOperator + ) + } + + deprecated override predicate partiallyDefines(Variable v) { v = collection } + + deprecated override predicate partiallyDefinesThis(ThisExpr e) { none() } + + override predicate definesExpressions(Expr inner, Expr outer) { + inner = innerDefinedExpr and + outer = this + } + + override predicate partiallyDefinesVariableAt(Variable v, ControlFlowNode cfn) { + v = collection and + cfn = node + } + } + + class VariablePartialDefinition extends PartialDefinition { + Expr innerDefinedExpr; + + VariablePartialDefinition() { + not this instanceof Conversion and + exists(Expr convertedInner | + valueToUpdate(convertedInner, this.getFullyConverted(), node) and + innerDefinedExpr = convertedInner.getUnconverted() + ) + } + + deprecated override predicate partiallyDefines(Variable v) { + innerDefinedExpr = v.getAnAccess() + } + + deprecated override predicate partiallyDefinesThis(ThisExpr e) { innerDefinedExpr = e } + + /** + * Holds if this partial definition may modify `inner` (or what it points + * to) through `outer`. These expressions will never be `Conversion`s. + * + * For example, in `f(& (*a).x)`, there are two results: + * - `inner` = `... .x`, `outer` = `&...` + * - `inner` = `a`, `outer` = `*` + */ + override predicate definesExpressions(Expr inner, Expr outer) { + inner = innerDefinedExpr and + outer = this + } + + override predicate partiallyDefinesVariableAt(Variable v, ControlFlowNode cfn) { + innerDefinedExpr = v.getAnAccess() and + cfn = node + } + } + + /** + * A partial definition that's a definition via an output iterator. + */ + class DefinitionByIterator extends IteratorPartialDefinition { + DefinitionByIterator() { exists(Call c | this = c.getAnArgument() or this = c.getQualifier()) } + } + /** * A partial definition that's a definition by reference. */ - class DefinitionByReference extends PartialDefinition { + class DefinitionByReference extends VariablePartialDefinition { DefinitionByReference() { exists(Call c | this = c.getAnArgument() or this = c.getQualifier()) } } } @@ -211,7 +295,8 @@ module FlowVar_internal { // The SSA library has a theoretically accurate treatment of reference types, // treating them as immutable, but for data flow it gives better results in // practice to make the variable synonymous with its contents. - not v.getUnspecifiedType() instanceof ReferenceType + not v.getUnspecifiedType() instanceof ReferenceType and + not v instanceof IteratorParameter } /** @@ -240,7 +325,7 @@ module FlowVar_internal { ( initializer(v, sbb.getANode()) or - assignmentLikeOperation(sbb, v, _, _) + assignmentLikeOperation(sbb, v, _) or exists(PartialDefinition p | p.partiallyDefinesVariableAt(v, sbb)) or @@ -359,7 +444,7 @@ module FlowVar_internal { } override predicate definedByExpr(Expr e, ControlFlowNode node) { - assignmentLikeOperation(node, v, _, e) and + assignmentLikeOperation(node, v, e) and node = sbb or // We pick the defining `ControlFlowNode` of an `Initializer` to be its @@ -449,7 +534,7 @@ module FlowVar_internal { pragma[noinline] private Variable getAVariableAssignedInLoop() { exists(BasicBlock bbAssign | - assignmentLikeOperation(bbAssign.getANode(), result, _, _) and + assignmentLikeOperation(bbAssign.getANode(), result, _) and this.bbInLoop(bbAssign) ) } @@ -487,7 +572,7 @@ module FlowVar_internal { pragma[noinline] private predicate assignsToVar(BasicBlock bb, Variable v) { - assignmentLikeOperation(bb.getANode(), v, _, _) and + assignmentLikeOperation(bb.getANode(), v, _) and exists(AlwaysTrueUponEntryLoop loop | v = loop.getARelevantVariable()) } @@ -524,7 +609,7 @@ module FlowVar_internal { result = mid.getASuccessor() and variableLiveInSBB(result, v) and forall(AlwaysTrueUponEntryLoop loop | skipLoop(mid, result, v, loop) | loop.sbbInLoop(sbbDef)) and - not assignmentLikeOperation(result, v, _, _) + not assignmentLikeOperation(result, v, _) ) } @@ -560,13 +645,15 @@ module FlowVar_internal { refType = p.getUnderlyingType() and not refType.getBaseType().isConst() ) + or + p instanceof IteratorParameter } /** * Holds if liveness of `v` should stop propagating backwards from `sbb`. */ private predicate variableNotLiveBefore(SubBasicBlock sbb, Variable v) { - assignmentLikeOperation(sbb, v, _, _) + assignmentLikeOperation(sbb, v, _) or // Liveness of `v` is killed when going backwards from a block that declares it exists(DeclStmt ds | ds.getADeclaration().(LocalVariable) = v and sbb.contains(ds)) @@ -686,21 +773,17 @@ module FlowVar_internal { * `node instanceof Initializer` is covered by `initializer` instead of this * predicate. */ - predicate assignmentLikeOperation( - ControlFlowNode node, Variable v, VariableAccess va, Expr assignedExpr - ) { + predicate assignmentLikeOperation(ControlFlowNode node, Variable v, Expr assignedExpr) { // Together, the two following cases cover `Assignment` node = any(AssignExpr ae | - va = ae.getLValue() and - v = va.getTarget() and + v.getAnAccess() = ae.getLValue() and assignedExpr = ae.getRValue() ) or node = any(AssignOperation ao | - va = ao.getLValue() and - v = va.getTarget() and + v.getAnAccess() = ao.getLValue() and // Here and in the `PrefixCrementOperation` case, we say that the assigned // expression is the operation itself. For example, we say that `x += 1` // assigns `x += 1` to `x`. The justification is that after this operation, @@ -712,12 +795,24 @@ module FlowVar_internal { // `PrefixCrementOperation` is itself a source node = any(CrementOperation op | - va = op.getOperand() and - v = va.getTarget() and + v.getAnAccess() = op.getOperand() and assignedExpr = op ) } + Expr getAnIteratorAccess(Variable collection) { + exists(Call c, SsaDefinition def, Variable iterator | + c.getQualifier() = collection.getAnAccess() and + c.getTarget() instanceof BeginOrEndFunction and + def.getAnUltimateDefiningValue(iterator) = c and + result = def.getAUse(iterator) + ) + } + + class IteratorParameter extends Parameter { + IteratorParameter() { this.getUnspecifiedType() instanceof Iterator } + } + /** * Holds if `v` is initialized to have value `assignedExpr`. */ @@ -749,7 +844,7 @@ module FlowVar_internal { class DataFlowSubBasicBlockCutNode extends SubBasicBlockCutNode { DataFlowSubBasicBlockCutNode() { exists(Variable v | not fullySupportedSsaVariable(v) | - assignmentLikeOperation(this, v, _, _) + assignmentLikeOperation(this, v, _) or exists(PartialDefinition p | p.partiallyDefinesVariableAt(v, this)) // It is not necessary to cut the basic blocks at `Initializer` nodes diff --git a/cpp/ql/src/semmle/code/cpp/dataflow/internal/TaintTrackingUtil.qll b/cpp/ql/src/semmle/code/cpp/dataflow/internal/TaintTrackingUtil.qll index 9f580778c3d..1ef340c4f21 100644 --- a/cpp/ql/src/semmle/code/cpp/dataflow/internal/TaintTrackingUtil.qll +++ b/cpp/ql/src/semmle/code/cpp/dataflow/internal/TaintTrackingUtil.qll @@ -10,6 +10,7 @@ private import semmle.code.cpp.models.interfaces.DataFlow private import semmle.code.cpp.models.interfaces.Taint +private import semmle.code.cpp.models.interfaces.Iterator private module DataFlow { import semmle.code.cpp.dataflow.internal.DataFlowUtil @@ -255,4 +256,12 @@ private predicate exprToPartialDefinitionStep(Expr exprIn, Expr exprOut) { exprIn = call.getArgument(argInIndex) ) ) + or + exists(Assignment a | + iteratorDereference(exprOut) and + a.getLValue() = exprOut and + a.getRValue() = exprIn + ) } + +private predicate iteratorDereference(Call c) { c.getTarget() instanceof IteratorReferenceFunction } diff --git a/cpp/ql/src/semmle/code/cpp/ir/dataflow/internal/DataFlowPrivate.qll b/cpp/ql/src/semmle/code/cpp/ir/dataflow/internal/DataFlowPrivate.qll index 9d188351272..fb9b39c0fda 100644 --- a/cpp/ql/src/semmle/code/cpp/ir/dataflow/internal/DataFlowPrivate.qll +++ b/cpp/ql/src/semmle/code/cpp/ir/dataflow/internal/DataFlowPrivate.qll @@ -355,14 +355,16 @@ private class ArrayToPointerConvertInstruction extends ConvertInstruction { } } -private Instruction skipOneCopyValueInstruction(Instruction instr) { - not instr instanceof CopyValueInstruction and result = instr +private Instruction skipOneCopyValueInstructionRec(CopyValueInstruction copy) { + copy.getUnary() = result and not result instanceof CopyValueInstruction or - result = instr.(CopyValueInstruction).getUnary() + result = skipOneCopyValueInstructionRec(copy.getUnary()) } private Instruction skipCopyValueInstructions(Instruction instr) { - result = skipOneCopyValueInstruction*(instr) and not result instanceof CopyValueInstruction + not result instanceof CopyValueInstruction and result = instr + or + result = skipOneCopyValueInstructionRec(instr) } private predicate arrayReadStep(Node node1, ArrayContent a, Node node2) { diff --git a/cpp/ql/src/semmle/code/cpp/ir/implementation/aliased_ssa/Operand.qll b/cpp/ql/src/semmle/code/cpp/ir/implementation/aliased_ssa/Operand.qll index 14f62da51cd..e476aec60af 100644 --- a/cpp/ql/src/semmle/code/cpp/ir/implementation/aliased_ssa/Operand.qll +++ b/cpp/ql/src/semmle/code/cpp/ir/implementation/aliased_ssa/Operand.qll @@ -79,7 +79,8 @@ private PhiOperandBase phiOperand( } /** - * A source operand of an `Instruction`. The operand represents a value consumed by the instruction. + * An operand of an `Instruction`. The operand represents a use of the result of one instruction + * (the defining instruction) in another instruction (the use instruction) */ class Operand extends TOperand { /** Gets a textual representation of this element. */ diff --git a/cpp/ql/src/semmle/code/cpp/ir/implementation/aliased_ssa/internal/AliasedSSA.qll b/cpp/ql/src/semmle/code/cpp/ir/implementation/aliased_ssa/internal/AliasedSSA.qll index 8124fa438d4..1e034051f05 100644 --- a/cpp/ql/src/semmle/code/cpp/ir/implementation/aliased_ssa/internal/AliasedSSA.qll +++ b/cpp/ql/src/semmle/code/cpp/ir/implementation/aliased_ssa/internal/AliasedSSA.qll @@ -133,6 +133,12 @@ abstract class MemoryLocation extends TMemoryLocation { predicate isAlwaysAllocatedOnStack() { none() } } +/** + * Represents a set of `MemoryLocation`s that cannot overlap with + * `MemoryLocation`s outside of the set. The `VirtualVariable` will be + * represented by a `MemoryLocation` that totally overlaps all other + * `MemoryLocations` in the set. + */ abstract class VirtualVariable extends MemoryLocation { } abstract class AllocationMemoryLocation extends MemoryLocation { diff --git a/cpp/ql/src/semmle/code/cpp/ir/implementation/raw/Operand.qll b/cpp/ql/src/semmle/code/cpp/ir/implementation/raw/Operand.qll index 14f62da51cd..e476aec60af 100644 --- a/cpp/ql/src/semmle/code/cpp/ir/implementation/raw/Operand.qll +++ b/cpp/ql/src/semmle/code/cpp/ir/implementation/raw/Operand.qll @@ -79,7 +79,8 @@ private PhiOperandBase phiOperand( } /** - * A source operand of an `Instruction`. The operand represents a value consumed by the instruction. + * An operand of an `Instruction`. The operand represents a use of the result of one instruction + * (the defining instruction) in another instruction (the use instruction) */ class Operand extends TOperand { /** Gets a textual representation of this element. */ diff --git a/cpp/ql/src/semmle/code/cpp/ir/implementation/unaliased_ssa/Operand.qll b/cpp/ql/src/semmle/code/cpp/ir/implementation/unaliased_ssa/Operand.qll index 14f62da51cd..e476aec60af 100644 --- a/cpp/ql/src/semmle/code/cpp/ir/implementation/unaliased_ssa/Operand.qll +++ b/cpp/ql/src/semmle/code/cpp/ir/implementation/unaliased_ssa/Operand.qll @@ -79,7 +79,8 @@ private PhiOperandBase phiOperand( } /** - * A source operand of an `Instruction`. The operand represents a value consumed by the instruction. + * An operand of an `Instruction`. The operand represents a use of the result of one instruction + * (the defining instruction) in another instruction (the use instruction) */ class Operand extends TOperand { /** Gets a textual representation of this element. */ diff --git a/cpp/ql/src/semmle/code/cpp/ir/implementation/unaliased_ssa/internal/SimpleSSA.qll b/cpp/ql/src/semmle/code/cpp/ir/implementation/unaliased_ssa/internal/SimpleSSA.qll index 3de8f5259f8..a7b9160bdc7 100644 --- a/cpp/ql/src/semmle/code/cpp/ir/implementation/unaliased_ssa/internal/SimpleSSA.qll +++ b/cpp/ql/src/semmle/code/cpp/ir/implementation/unaliased_ssa/internal/SimpleSSA.qll @@ -59,6 +59,12 @@ class MemoryLocation extends TMemoryLocation { final string getUniqueId() { result = var.getUniqueId() } } +/** + * Represents a set of `MemoryLocation`s that cannot overlap with + * `MemoryLocation`s outside of the set. The `VirtualVariable` will be + * represented by a `MemoryLocation` that totally overlaps all other + * `MemoryLocations` in the set. + */ class VirtualVariable extends MemoryLocation { } /** A virtual variable that groups all escaped memory within a function. */ diff --git a/cpp/ql/src/semmle/code/cpp/ir/internal/Overlap.qll b/cpp/ql/src/semmle/code/cpp/ir/internal/Overlap.qll index 8ce0549b2b4..f9a0c574f8c 100644 --- a/cpp/ql/src/semmle/code/cpp/ir/internal/Overlap.qll +++ b/cpp/ql/src/semmle/code/cpp/ir/internal/Overlap.qll @@ -3,18 +3,33 @@ private newtype TOverlap = TMustTotallyOverlap() or TMustExactlyOverlap() +/** + * Represents a possible overlap between two memory ranges. + */ abstract class Overlap extends TOverlap { abstract string toString(); } +/** + * Represents a partial overlap between two memory ranges, which may or may not + * actually occur in practice. + */ class MayPartiallyOverlap extends Overlap, TMayPartiallyOverlap { final override string toString() { result = "MayPartiallyOverlap" } } +/** + * Represents an overlap in which the first memory range is known to include all + * bits of the second memory range, but may be larger or have a different type. + */ class MustTotallyOverlap extends Overlap, TMustTotallyOverlap { final override string toString() { result = "MustTotallyOverlap" } } +/** + * Represents an overlap between two memory ranges that have the same extent and + * the same type. + */ class MustExactlyOverlap extends Overlap, TMustExactlyOverlap { final override string toString() { result = "MustExactlyOverlap" } } diff --git a/cpp/ql/src/semmle/code/cpp/models/implementations/Iterator.qll b/cpp/ql/src/semmle/code/cpp/models/implementations/Iterator.qll index 2ad464d1634..ded937d5312 100644 --- a/cpp/ql/src/semmle/code/cpp/models/implementations/Iterator.qll +++ b/cpp/ql/src/semmle/code/cpp/models/implementations/Iterator.qll @@ -8,6 +8,7 @@ import cpp import semmle.code.cpp.models.interfaces.Taint import semmle.code.cpp.models.interfaces.DataFlow +import semmle.code.cpp.models.interfaces.Iterator /** * An instantiation of the `std::iterator_traits` template. @@ -80,7 +81,7 @@ private FunctionInput getIteratorArgumentInput(Operator op, int index) { /** * A non-member prefix `operator*` function for an iterator type. */ -class IteratorPointerDereferenceOperator extends Operator, TaintFunction { +class IteratorPointerDereferenceOperator extends Operator, TaintFunction, IteratorReferenceFunction { FunctionInput iteratorInput; IteratorPointerDereferenceOperator() { @@ -169,7 +170,8 @@ class IteratorAssignArithmeticOperator extends Operator, DataFlowFunction, Taint /** * A prefix `operator*` member function for an iterator type. */ -class IteratorPointerDereferenceMemberOperator extends MemberFunction, TaintFunction { +class IteratorPointerDereferenceMemberOperator extends MemberFunction, TaintFunction, + IteratorReferenceFunction { IteratorPointerDereferenceMemberOperator() { this.hasName("operator*") and this.getDeclaringType() instanceof Iterator @@ -260,7 +262,7 @@ class IteratorAssignArithmeticMemberOperator extends MemberFunction, DataFlowFun /** * An `operator[]` member function of an iterator class. */ -class IteratorArrayMemberOperator extends MemberFunction, TaintFunction { +class IteratorArrayMemberOperator extends MemberFunction, TaintFunction, IteratorReferenceFunction { IteratorArrayMemberOperator() { this.hasName("operator[]") and this.getDeclaringType() instanceof Iterator @@ -271,3 +273,19 @@ class IteratorArrayMemberOperator extends MemberFunction, TaintFunction { output.isReturnValue() } } + +/** + * A `begin` or `end` member function, or a related member function, that + * returns an iterator. + */ +class BeginOrEndFunction extends MemberFunction, TaintFunction { + BeginOrEndFunction() { + this.hasName(["begin", "cbegin", "rbegin", "crbegin", "end", "cend", "rend", "crend"]) and + this.getType().getUnspecifiedType() instanceof Iterator + } + + override predicate hasTaintFlow(FunctionInput input, FunctionOutput output) { + input.isQualifierObject() and + output.isReturnValue() + } +} diff --git a/cpp/ql/src/semmle/code/cpp/models/implementations/StdString.qll b/cpp/ql/src/semmle/code/cpp/models/implementations/StdString.qll index 5c6f6ec6189..b8a000b963e 100644 --- a/cpp/ql/src/semmle/code/cpp/models/implementations/StdString.qll +++ b/cpp/ql/src/semmle/code/cpp/models/implementations/StdString.qll @@ -216,23 +216,6 @@ class StdStringAssign extends TaintFunction { } } -/** - * The standard functions `std::string.begin` and `std::string.end` and their - * variants. - */ -class StdStringBeginEnd extends TaintFunction { - StdStringBeginEnd() { - this - .hasQualifiedName("std", "basic_string", - ["begin", "cbegin", "rbegin", "crbegin", "end", "cend", "rend", "crend"]) - } - - override predicate hasTaintFlow(FunctionInput input, FunctionOutput output) { - input.isQualifierObject() and - output.isReturnValue() - } -} - /** * The standard function `std::string.copy`. */ diff --git a/cpp/ql/src/semmle/code/cpp/models/interfaces/Iterator.qll b/cpp/ql/src/semmle/code/cpp/models/interfaces/Iterator.qll new file mode 100644 index 00000000000..de697776525 --- /dev/null +++ b/cpp/ql/src/semmle/code/cpp/models/interfaces/Iterator.qll @@ -0,0 +1,17 @@ +/** + * Provides an abstract class for accurate modeling of flow through output + * iterators. To use this QL library, create a QL class extending + * `IteratorReferenceFunction` with a characteristic predicate that selects the + * function or set of functions you are modeling. Within that class, override + * the predicates provided by `AliasFunction` to match the flow within that + * function. + */ + +import cpp +import semmle.code.cpp.models.Models + +/** + * A function which takes an iterator argument and returns a reference that + * can be used to write to the iterator's underlying collection. + */ +abstract class IteratorReferenceFunction extends Function { } diff --git a/cpp/ql/src/semmle/code/cpp/rangeanalysis/SimpleRangeAnalysis.qll b/cpp/ql/src/semmle/code/cpp/rangeanalysis/SimpleRangeAnalysis.qll index d94f4e23e1b..886c6347622 100644 --- a/cpp/ql/src/semmle/code/cpp/rangeanalysis/SimpleRangeAnalysis.qll +++ b/cpp/ql/src/semmle/code/cpp/rangeanalysis/SimpleRangeAnalysis.qll @@ -45,6 +45,7 @@ import cpp private import RangeAnalysisUtils private import experimental.semmle.code.cpp.models.interfaces.SimpleRangeAnalysisExpr +private import experimental.semmle.code.cpp.models.interfaces.SimpleRangeAnalysisDefinition import RangeSSA import SimpleRangeAnalysisCached private import NanAnalysis @@ -335,6 +336,11 @@ private predicate defDependsOnDef( or // Phi nodes. phiDependsOnDef(def, v, srcDef, srcVar) + or + // Extensions + exists(Expr expr | def.(SimpleRangeAnalysisDefinition).dependsOnExpr(v, expr) | + exprDependsOnDef(expr, srcDef, srcVar) + ) } /** @@ -492,6 +498,9 @@ private predicate analyzableDef(RangeSsaDefinition def, StackVariable v) { v = def.getAVariable() or phiDependsOnDef(def, v, _, _) + or + // A modeled def for range analysis + def.(SimpleRangeAnalysisDefinition).hasRangeInformationFor(v) } /** @@ -1215,6 +1224,9 @@ private float getDefLowerBoundsImpl(RangeSsaDefinition def, StackVariable v) { // Phi nodes. result = getPhiLowerBounds(v, def) or + // A modeled def for range analysis + result = def.(SimpleRangeAnalysisDefinition).getLowerBounds(v) + or // Unanalyzable definitions. unanalyzableDefBounds(def, v, result, _) } @@ -1248,6 +1260,9 @@ private float getDefUpperBoundsImpl(RangeSsaDefinition def, StackVariable v) { // Phi nodes. result = getPhiUpperBounds(v, def) or + // A modeled def for range analysis + result = def.(SimpleRangeAnalysisDefinition).getUpperBounds(v) + or // Unanalyzable definitions. unanalyzableDefBounds(def, v, _, result) } diff --git a/cpp/ql/test/experimental/library-tests/rangeanalysis/extended/extended.cpp b/cpp/ql/test/experimental/library-tests/rangeanalysis/extended/extended.cpp new file mode 100644 index 00000000000..0fed35bc9af --- /dev/null +++ b/cpp/ql/test/experimental/library-tests/rangeanalysis/extended/extended.cpp @@ -0,0 +1,9 @@ + + +void test_overridability_sub(int x) { + int zero = x - x; + zero; // 0 + + int nonzero = x - (unsigned char)x; + nonzero; // full range +} diff --git a/cpp/ql/test/experimental/library-tests/rangeanalysis/extended/extended.expected b/cpp/ql/test/experimental/library-tests/rangeanalysis/extended/extended.expected new file mode 100644 index 00000000000..b43601c8088 --- /dev/null +++ b/cpp/ql/test/experimental/library-tests/rangeanalysis/extended/extended.expected @@ -0,0 +1,6 @@ +| extended.cpp:4:14:4:14 | x | -2.147483648E9 | 2.147483647E9 | +| extended.cpp:4:18:4:18 | x | -2.147483648E9 | 2.147483647E9 | +| extended.cpp:5:3:5:6 | zero | 0.0 | 0.0 | +| extended.cpp:7:17:7:17 | x | -2.147483648E9 | 2.147483647E9 | +| extended.cpp:7:36:7:36 | x | -2.147483648E9 | 2.147483647E9 | +| extended.cpp:8:3:8:9 | nonzero | -2.147483648E9 | 2.147483647E9 | diff --git a/cpp/ql/test/experimental/library-tests/rangeanalysis/extended/extended.ql b/cpp/ql/test/experimental/library-tests/rangeanalysis/extended/extended.ql new file mode 100644 index 00000000000..d6344e5d062 --- /dev/null +++ b/cpp/ql/test/experimental/library-tests/rangeanalysis/extended/extended.ql @@ -0,0 +1,7 @@ +import experimental.semmle.code.cpp.rangeanalysis.ExtendedRangeAnalysis + +from VariableAccess expr, float lower, float upper +where + lower = lowerBound(expr) and + upper = upperBound(expr) +select expr, lower, upper diff --git a/cpp/ql/test/experimental/library-tests/rangeanalysis/extensibility/extensibility.c b/cpp/ql/test/experimental/library-tests/rangeanalysis/extensibility/extensibility.c index afdde0b615c..33cf54ee9eb 100644 --- a/cpp/ql/test/experimental/library-tests/rangeanalysis/extensibility/extensibility.c +++ b/cpp/ql/test/experimental/library-tests/rangeanalysis/extensibility/extensibility.c @@ -9,6 +9,11 @@ int test_extensibility_add(int x) { } int test_overridability_sub(int x) { - int result = x - x; // Returns 0 due to custom modeling in QL + int result = x - (unsigned char)x; // Returns 0 due to custom modeling for this test being deliberately wrong return result; // 0 -} \ No newline at end of file +} + +void test_parameter_override(int magic_name_at_most_10, int magic_name_at_most_20) { + magic_name_at_most_10; + magic_name_at_most_20; +} diff --git a/cpp/ql/test/experimental/library-tests/rangeanalysis/extensibility/extensibility.expected b/cpp/ql/test/experimental/library-tests/rangeanalysis/extensibility/extensibility.expected index ad97d9b2df5..e9133e19104 100644 --- a/cpp/ql/test/experimental/library-tests/rangeanalysis/extensibility/extensibility.expected +++ b/cpp/ql/test/experimental/library-tests/rangeanalysis/extensibility/extensibility.expected @@ -3,5 +3,7 @@ | extensibility.c:6:38:6:38 | x | -10.0 | 10.0 | | extensibility.c:7:12:7:17 | result | 90.0 | 110.0 | | extensibility.c:12:16:12:16 | x | -2.147483648E9 | 2.147483647E9 | -| extensibility.c:12:20:12:20 | x | -2.147483648E9 | 2.147483647E9 | +| extensibility.c:12:35:12:35 | x | -2.147483648E9 | 2.147483647E9 | | extensibility.c:13:10:13:15 | result | 0.0 | 0.0 | +| extensibility.c:17:3:17:23 | magic_name_at_most_10 | -2.147483648E9 | 10.0 | +| extensibility.c:18:3:18:23 | magic_name_at_most_20 | -2.147483648E9 | 20.0 | diff --git a/cpp/ql/test/experimental/library-tests/rangeanalysis/extensibility/extensibility.ql b/cpp/ql/test/experimental/library-tests/rangeanalysis/extensibility/extensibility.ql index a007a5d4dab..4fbfed1706d 100644 --- a/cpp/ql/test/experimental/library-tests/rangeanalysis/extensibility/extensibility.ql +++ b/cpp/ql/test/experimental/library-tests/rangeanalysis/extensibility/extensibility.ql @@ -1,5 +1,7 @@ import semmle.code.cpp.rangeanalysis.SimpleRangeAnalysis +import semmle.code.cpp.rangeanalysis.RangeAnalysisUtils import experimental.semmle.code.cpp.models.interfaces.SimpleRangeAnalysisExpr +import experimental.semmle.code.cpp.models.interfaces.SimpleRangeAnalysisDefinition class CustomAddFunctionCall extends SimpleRangeAnalysisExpr, FunctionCall { CustomAddFunctionCall() { this.getTarget().hasGlobalName("custom_add_function") } @@ -37,6 +39,40 @@ class SelfSub extends SimpleRangeAnalysisExpr, SubExpr { override predicate dependsOnChild(Expr child) { child = this.getAnOperand() } } +/** + * A definition for test purposes of a parameter `p` that starts with a + * special prefix. This class is written to exploit how QL behaves when class + * fields are not functionally determined by `this`. When multiple parameters + * of the same function have the special prefix, there is still only one + * instance of this class. + */ +class MagicParameterName extends SimpleRangeAnalysisDefinition { + Parameter p; + float value; + + MagicParameterName() { + this.definedByParameter(p) and + value = p.getName().regexpCapture("magic_name_at_most_(\\d+)", 1).toFloat() + } + + override predicate hasRangeInformationFor(StackVariable v) { v = p } + + override predicate dependsOnExpr(StackVariable v, Expr e) { + // No dependencies. This sample class yields constant values. + none() + } + + override float getLowerBounds(StackVariable var) { + var = p and + result = typeLowerBound(p.getUnspecifiedType()) + } + + override float getUpperBounds(StackVariable var) { + var = p and + result = value + } +} + from VariableAccess expr, float lower, float upper where lower = lowerBound(expr) and diff --git a/cpp/ql/test/library-tests/dataflow/taint-tests/localTaint.expected b/cpp/ql/test/library-tests/dataflow/taint-tests/localTaint.expected index 56cf29bde53..187382eb0db 100644 --- a/cpp/ql/test/library-tests/dataflow/taint-tests/localTaint.expected +++ b/cpp/ql/test/library-tests/dataflow/taint-tests/localTaint.expected @@ -1884,23 +1884,29 @@ | smart_pointer.cpp:65:28:65:46 | call to make_unique | smart_pointer.cpp:67:10:67:10 | p | | | smart_pointer.cpp:65:48:65:53 | call to source | smart_pointer.cpp:65:28:65:46 | call to make_unique | TAINT | | smart_pointer.cpp:65:58:65:58 | 0 | smart_pointer.cpp:65:28:65:46 | call to make_unique | TAINT | +| standalone_iterators.cpp:39:45:39:51 | source1 | standalone_iterators.cpp:39:45:39:51 | source1 | | | standalone_iterators.cpp:39:45:39:51 | source1 | standalone_iterators.cpp:40:11:40:17 | source1 | | | standalone_iterators.cpp:39:45:39:51 | source1 | standalone_iterators.cpp:41:12:41:18 | source1 | | | standalone_iterators.cpp:39:45:39:51 | source1 | standalone_iterators.cpp:42:14:42:20 | source1 | | | standalone_iterators.cpp:40:11:40:17 | source1 | standalone_iterators.cpp:40:10:40:10 | call to operator* | TAINT | +| standalone_iterators.cpp:41:12:41:18 | ref arg source1 | standalone_iterators.cpp:39:45:39:51 | source1 | | | standalone_iterators.cpp:41:12:41:18 | ref arg source1 | standalone_iterators.cpp:42:14:42:20 | source1 | | | standalone_iterators.cpp:41:12:41:18 | source1 | standalone_iterators.cpp:41:19:41:19 | call to operator++ | | | standalone_iterators.cpp:41:19:41:19 | call to operator++ | standalone_iterators.cpp:41:10:41:10 | call to operator* | TAINT | | standalone_iterators.cpp:42:12:42:12 | call to operator++ | standalone_iterators.cpp:42:10:42:10 | call to operator* | TAINT | +| standalone_iterators.cpp:42:14:42:20 | ref arg source1 | standalone_iterators.cpp:39:45:39:51 | source1 | | | standalone_iterators.cpp:42:14:42:20 | source1 | standalone_iterators.cpp:42:12:42:12 | call to operator++ | | +| standalone_iterators.cpp:45:39:45:45 | source1 | standalone_iterators.cpp:45:39:45:45 | source1 | | | standalone_iterators.cpp:45:39:45:45 | source1 | standalone_iterators.cpp:46:11:46:17 | source1 | | | standalone_iterators.cpp:45:39:45:45 | source1 | standalone_iterators.cpp:47:12:47:18 | source1 | | | standalone_iterators.cpp:45:39:45:45 | source1 | standalone_iterators.cpp:48:14:48:20 | source1 | | | standalone_iterators.cpp:46:11:46:17 | source1 | standalone_iterators.cpp:46:10:46:10 | call to operator* | TAINT | +| standalone_iterators.cpp:47:12:47:18 | ref arg source1 | standalone_iterators.cpp:45:39:45:45 | source1 | | | standalone_iterators.cpp:47:12:47:18 | ref arg source1 | standalone_iterators.cpp:48:14:48:20 | source1 | | | standalone_iterators.cpp:47:12:47:18 | source1 | standalone_iterators.cpp:47:19:47:19 | call to operator++ | | | standalone_iterators.cpp:47:19:47:19 | call to operator++ | standalone_iterators.cpp:47:10:47:10 | call to operator* | TAINT | | standalone_iterators.cpp:48:12:48:12 | call to operator++ | standalone_iterators.cpp:48:10:48:10 | call to operator* | TAINT | +| standalone_iterators.cpp:48:14:48:20 | ref arg source1 | standalone_iterators.cpp:45:39:45:45 | source1 | | | standalone_iterators.cpp:48:14:48:20 | source1 | standalone_iterators.cpp:48:12:48:12 | call to operator++ | | | standalone_iterators.cpp:51:37:51:43 | source1 | standalone_iterators.cpp:52:11:52:17 | source1 | | | standalone_iterators.cpp:51:37:51:43 | source1 | standalone_iterators.cpp:53:12:53:18 | source1 | | @@ -5309,11 +5315,14 @@ | vector.cpp:255:3:255:4 | ref arg v6 | vector.cpp:261:8:261:9 | v6 | | | vector.cpp:255:3:255:4 | ref arg v6 | vector.cpp:262:2:262:2 | v6 | | | vector.cpp:255:13:255:14 | call to iterator | vector.cpp:255:3:255:4 | ref arg v6 | TAINT | +| vector.cpp:255:13:255:14 | call to iterator [post update] | vector.cpp:277:1:277:1 | v3 | | | vector.cpp:255:13:255:14 | i1 | vector.cpp:255:13:255:14 | call to iterator | | +| vector.cpp:255:13:255:14 | i1 [post update] | vector.cpp:277:1:277:1 | v3 | | | vector.cpp:255:17:255:18 | call to iterator | vector.cpp:255:3:255:4 | ref arg v6 | TAINT | | vector.cpp:255:17:255:18 | i2 | vector.cpp:255:17:255:18 | call to iterator | | | vector.cpp:257:8:257:9 | ref arg v4 | vector.cpp:262:2:262:2 | v4 | | | vector.cpp:258:8:258:9 | ref arg v5 | vector.cpp:262:2:262:2 | v5 | | +| vector.cpp:259:8:259:9 | ref arg i1 | vector.cpp:277:1:277:1 | v3 | | | vector.cpp:261:8:261:9 | ref arg v6 | vector.cpp:262:2:262:2 | v6 | | | vector.cpp:265:22:265:23 | call to vector | vector.cpp:269:3:269:4 | v7 | | | vector.cpp:265:22:265:23 | call to vector | vector.cpp:273:8:273:9 | v7 | | @@ -5516,3 +5525,265 @@ | vector.cpp:324:7:324:8 | ref arg v2 | vector.cpp:327:1:327:1 | v2 | | | vector.cpp:325:7:325:8 | ref arg v3 | vector.cpp:327:1:327:1 | v3 | | | vector.cpp:326:7:326:8 | ref arg v4 | vector.cpp:327:1:327:1 | v4 | | +| vector.cpp:329:62:329:65 | iter | vector.cpp:329:62:329:65 | iter | | +| vector.cpp:329:62:329:65 | iter | vector.cpp:330:3:330:6 | iter | | +| vector.cpp:330:2:330:2 | call to operator* [post update] | vector.cpp:329:62:329:65 | iter | | +| vector.cpp:330:2:330:17 | ... = ... | vector.cpp:330:2:330:2 | call to operator* [post update] | | +| vector.cpp:330:3:330:6 | iter | vector.cpp:330:2:330:2 | call to operator* | TAINT | +| vector.cpp:330:10:330:15 | call to source | vector.cpp:330:2:330:2 | call to operator* [post update] | TAINT | +| vector.cpp:330:10:330:15 | call to source | vector.cpp:330:2:330:17 | ... = ... | | +| vector.cpp:333:64:333:67 | iter | vector.cpp:333:64:333:67 | iter | | +| vector.cpp:333:64:333:67 | iter | vector.cpp:334:3:334:6 | iter | | +| vector.cpp:333:74:333:74 | i | vector.cpp:334:10:334:10 | i | | +| vector.cpp:334:2:334:2 | call to operator* [post update] | vector.cpp:333:64:333:67 | iter | | +| vector.cpp:334:2:334:10 | ... = ... | vector.cpp:334:2:334:2 | call to operator* [post update] | | +| vector.cpp:334:3:334:6 | iter | vector.cpp:334:2:334:2 | call to operator* | TAINT | +| vector.cpp:334:10:334:10 | i | vector.cpp:334:2:334:2 | call to operator* [post update] | TAINT | +| vector.cpp:334:10:334:10 | i | vector.cpp:334:2:334:10 | ... = ... | | +| vector.cpp:337:38:337:38 | b | vector.cpp:372:5:372:5 | b | | +| vector.cpp:338:22:338:24 | call to vector | vector.cpp:340:34:340:35 | v1 | | +| vector.cpp:338:22:338:24 | call to vector | vector.cpp:342:7:342:8 | v1 | | +| vector.cpp:338:22:338:24 | call to vector | vector.cpp:401:1:401:1 | v1 | | +| vector.cpp:338:30:338:32 | call to vector | vector.cpp:344:38:344:39 | v2 | | +| vector.cpp:338:30:338:32 | call to vector | vector.cpp:344:56:344:57 | v2 | | +| vector.cpp:338:30:338:32 | call to vector | vector.cpp:347:7:347:8 | v2 | | +| vector.cpp:338:30:338:32 | call to vector | vector.cpp:401:1:401:1 | v2 | | +| vector.cpp:338:38:338:40 | call to vector | vector.cpp:349:15:349:16 | v3 | | +| vector.cpp:338:38:338:40 | call to vector | vector.cpp:352:7:352:8 | v3 | | +| vector.cpp:338:38:338:40 | call to vector | vector.cpp:401:1:401:1 | v3 | | +| vector.cpp:338:46:338:48 | call to vector | vector.cpp:354:38:354:39 | v4 | | +| vector.cpp:338:46:338:48 | call to vector | vector.cpp:354:56:354:57 | v4 | | +| vector.cpp:338:46:338:48 | call to vector | vector.cpp:357:7:357:8 | v4 | | +| vector.cpp:338:46:338:48 | call to vector | vector.cpp:401:1:401:1 | v4 | | +| vector.cpp:338:54:338:56 | call to vector | vector.cpp:359:34:359:35 | v5 | | +| vector.cpp:338:54:338:56 | call to vector | vector.cpp:361:7:361:8 | v5 | | +| vector.cpp:338:54:338:56 | call to vector | vector.cpp:363:7:363:8 | v5 | | +| vector.cpp:338:54:338:56 | call to vector | vector.cpp:401:1:401:1 | v5 | | +| vector.cpp:338:62:338:64 | call to vector | vector.cpp:365:34:365:35 | v6 | | +| vector.cpp:338:62:338:64 | call to vector | vector.cpp:367:7:367:8 | v6 | | +| vector.cpp:338:62:338:64 | call to vector | vector.cpp:368:2:368:3 | v6 | | +| vector.cpp:338:62:338:64 | call to vector | vector.cpp:369:7:369:8 | v6 | | +| vector.cpp:338:62:338:64 | call to vector | vector.cpp:401:1:401:1 | v6 | | +| vector.cpp:338:70:338:72 | call to vector | vector.cpp:371:34:371:35 | v7 | | +| vector.cpp:338:70:338:72 | call to vector | vector.cpp:374:8:374:9 | v7 | | +| vector.cpp:338:70:338:72 | call to vector | vector.cpp:377:8:377:9 | v7 | | +| vector.cpp:338:70:338:72 | call to vector | vector.cpp:379:7:379:8 | v7 | | +| vector.cpp:338:70:338:72 | call to vector | vector.cpp:401:1:401:1 | v7 | | +| vector.cpp:338:78:338:80 | call to vector | vector.cpp:381:34:381:35 | v8 | | +| vector.cpp:338:78:338:80 | call to vector | vector.cpp:383:7:383:8 | v8 | | +| vector.cpp:338:78:338:80 | call to vector | vector.cpp:385:7:385:8 | v8 | | +| vector.cpp:338:78:338:80 | call to vector | vector.cpp:401:1:401:1 | v8 | | +| vector.cpp:338:86:338:88 | call to vector | vector.cpp:387:34:387:35 | v9 | | +| vector.cpp:338:86:338:88 | call to vector | vector.cpp:392:7:392:8 | v9 | | +| vector.cpp:338:86:338:88 | call to vector | vector.cpp:401:1:401:1 | v9 | | +| vector.cpp:338:95:338:97 | call to vector | vector.cpp:394:35:394:37 | v10 | | +| vector.cpp:338:95:338:97 | call to vector | vector.cpp:396:7:396:9 | v10 | | +| vector.cpp:338:95:338:97 | call to vector | vector.cpp:401:1:401:1 | v10 | | +| vector.cpp:338:104:338:106 | call to vector | vector.cpp:398:35:398:37 | v11 | | +| vector.cpp:338:104:338:106 | call to vector | vector.cpp:400:7:400:9 | v11 | | +| vector.cpp:338:104:338:106 | call to vector | vector.cpp:401:1:401:1 | v11 | | +| vector.cpp:340:34:340:35 | ref arg v1 | vector.cpp:342:7:342:8 | v1 | | +| vector.cpp:340:34:340:35 | ref arg v1 | vector.cpp:401:1:401:1 | v1 | | +| vector.cpp:340:34:340:35 | v1 | vector.cpp:340:37:340:41 | call to begin | TAINT | +| vector.cpp:340:37:340:41 | call to begin | vector.cpp:341:3:341:4 | i1 | | +| vector.cpp:341:2:341:2 | call to operator* [post update] | vector.cpp:342:7:342:8 | v1 | | +| vector.cpp:341:2:341:2 | call to operator* [post update] | vector.cpp:401:1:401:1 | v1 | | +| vector.cpp:341:2:341:15 | ... = ... | vector.cpp:341:2:341:2 | call to operator* [post update] | | +| vector.cpp:341:3:341:4 | i1 | vector.cpp:341:2:341:2 | call to operator* | TAINT | +| vector.cpp:341:8:341:13 | call to source | vector.cpp:341:2:341:2 | call to operator* [post update] | TAINT | +| vector.cpp:341:8:341:13 | call to source | vector.cpp:341:2:341:15 | ... = ... | | +| vector.cpp:342:7:342:8 | ref arg v1 | vector.cpp:401:1:401:1 | v1 | | +| vector.cpp:344:38:344:39 | ref arg v2 | vector.cpp:344:56:344:57 | v2 | | +| vector.cpp:344:38:344:39 | ref arg v2 | vector.cpp:347:7:347:8 | v2 | | +| vector.cpp:344:38:344:39 | ref arg v2 | vector.cpp:401:1:401:1 | v2 | | +| vector.cpp:344:38:344:39 | v2 | vector.cpp:344:41:344:45 | call to begin | TAINT | +| vector.cpp:344:41:344:45 | call to begin | vector.cpp:344:50:344:51 | it | | +| vector.cpp:344:41:344:45 | call to begin | vector.cpp:344:68:344:69 | it | | +| vector.cpp:344:41:344:45 | call to begin | vector.cpp:345:4:345:5 | it | | +| vector.cpp:344:56:344:57 | ref arg v2 | vector.cpp:344:56:344:57 | v2 | | +| vector.cpp:344:56:344:57 | ref arg v2 | vector.cpp:347:7:347:8 | v2 | | +| vector.cpp:344:56:344:57 | ref arg v2 | vector.cpp:401:1:401:1 | v2 | | +| vector.cpp:344:56:344:57 | v2 | vector.cpp:344:59:344:61 | call to end | TAINT | +| vector.cpp:344:68:344:69 | it | vector.cpp:344:66:344:66 | call to operator++ | | +| vector.cpp:344:68:344:69 | ref arg it | vector.cpp:344:50:344:51 | it | | +| vector.cpp:344:68:344:69 | ref arg it | vector.cpp:344:68:344:69 | it | | +| vector.cpp:344:68:344:69 | ref arg it | vector.cpp:345:4:345:5 | it | | +| vector.cpp:345:3:345:3 | call to operator* [post update] | vector.cpp:344:56:344:57 | v2 | | +| vector.cpp:345:3:345:3 | call to operator* [post update] | vector.cpp:347:7:347:8 | v2 | | +| vector.cpp:345:3:345:3 | call to operator* [post update] | vector.cpp:401:1:401:1 | v2 | | +| vector.cpp:345:3:345:16 | ... = ... | vector.cpp:345:3:345:3 | call to operator* [post update] | | +| vector.cpp:345:4:345:5 | it | vector.cpp:345:3:345:3 | call to operator* | TAINT | +| vector.cpp:345:9:345:14 | call to source | vector.cpp:345:3:345:3 | call to operator* [post update] | TAINT | +| vector.cpp:345:9:345:14 | call to source | vector.cpp:345:3:345:16 | ... = ... | | +| vector.cpp:347:7:347:8 | ref arg v2 | vector.cpp:401:1:401:1 | v2 | | +| vector.cpp:349:15:349:15 | (__begin) | vector.cpp:349:15:349:15 | call to operator* | TAINT | +| vector.cpp:349:15:349:15 | (__begin) | vector.cpp:349:15:349:15 | call to operator++ | | +| vector.cpp:349:15:349:15 | (__end) | vector.cpp:349:15:349:15 | call to iterator | | +| vector.cpp:349:15:349:15 | (__range) | vector.cpp:349:15:349:15 | call to begin | TAINT | +| vector.cpp:349:15:349:15 | (__range) | vector.cpp:349:15:349:15 | call to end | TAINT | +| vector.cpp:349:15:349:15 | call to begin | vector.cpp:349:15:349:15 | (__begin) | | +| vector.cpp:349:15:349:15 | call to begin | vector.cpp:349:15:349:15 | (__begin) | | +| vector.cpp:349:15:349:15 | call to begin | vector.cpp:349:15:349:15 | (__begin) | | +| vector.cpp:349:15:349:15 | call to end | vector.cpp:349:15:349:15 | (__end) | | +| vector.cpp:349:15:349:15 | ref arg (__begin) | vector.cpp:349:15:349:15 | (__begin) | | +| vector.cpp:349:15:349:15 | ref arg (__begin) | vector.cpp:349:15:349:15 | (__begin) | | +| vector.cpp:349:15:349:15 | ref arg (__begin) | vector.cpp:349:15:349:15 | (__begin) | | +| vector.cpp:349:15:349:15 | ref arg (__range) | vector.cpp:349:15:349:15 | (__range) | | +| vector.cpp:349:15:349:16 | v3 | vector.cpp:349:15:349:15 | (__range) | | +| vector.cpp:349:15:349:16 | v3 | vector.cpp:349:15:349:15 | (__range) | | +| vector.cpp:349:15:349:16 | v3 | vector.cpp:349:15:349:15 | call to operator* | TAINT | +| vector.cpp:350:3:350:14 | ... = ... | vector.cpp:350:3:350:3 | x [post update] | | +| vector.cpp:350:7:350:12 | call to source | vector.cpp:350:3:350:14 | ... = ... | | +| vector.cpp:352:7:352:8 | ref arg v3 | vector.cpp:401:1:401:1 | v3 | | +| vector.cpp:354:38:354:39 | ref arg v4 | vector.cpp:354:56:354:57 | v4 | | +| vector.cpp:354:38:354:39 | ref arg v4 | vector.cpp:357:7:357:8 | v4 | | +| vector.cpp:354:38:354:39 | ref arg v4 | vector.cpp:401:1:401:1 | v4 | | +| vector.cpp:354:38:354:39 | v4 | vector.cpp:354:41:354:45 | call to begin | TAINT | +| vector.cpp:354:41:354:45 | call to begin | vector.cpp:354:50:354:51 | it | | +| vector.cpp:354:41:354:45 | call to begin | vector.cpp:354:68:354:69 | it | | +| vector.cpp:354:41:354:45 | call to begin | vector.cpp:355:32:355:33 | it | | +| vector.cpp:354:56:354:57 | ref arg v4 | vector.cpp:354:56:354:57 | v4 | | +| vector.cpp:354:56:354:57 | ref arg v4 | vector.cpp:357:7:357:8 | v4 | | +| vector.cpp:354:56:354:57 | ref arg v4 | vector.cpp:401:1:401:1 | v4 | | +| vector.cpp:354:56:354:57 | v4 | vector.cpp:354:59:354:61 | call to end | TAINT | +| vector.cpp:354:68:354:69 | it | vector.cpp:354:66:354:66 | call to operator++ | | +| vector.cpp:354:68:354:69 | ref arg it | vector.cpp:354:50:354:51 | it | | +| vector.cpp:354:68:354:69 | ref arg it | vector.cpp:354:68:354:69 | it | | +| vector.cpp:354:68:354:69 | ref arg it | vector.cpp:355:32:355:33 | it | | +| vector.cpp:355:32:355:33 | call to iterator [post update] | vector.cpp:354:56:354:57 | v4 | | +| vector.cpp:355:32:355:33 | call to iterator [post update] | vector.cpp:357:7:357:8 | v4 | | +| vector.cpp:355:32:355:33 | call to iterator [post update] | vector.cpp:401:1:401:1 | v4 | | +| vector.cpp:355:32:355:33 | it | vector.cpp:355:32:355:33 | call to iterator | | +| vector.cpp:355:32:355:33 | it [post update] | vector.cpp:354:56:354:57 | v4 | | +| vector.cpp:355:32:355:33 | it [post update] | vector.cpp:357:7:357:8 | v4 | | +| vector.cpp:355:32:355:33 | it [post update] | vector.cpp:401:1:401:1 | v4 | | +| vector.cpp:357:7:357:8 | ref arg v4 | vector.cpp:401:1:401:1 | v4 | | +| vector.cpp:359:34:359:35 | ref arg v5 | vector.cpp:361:7:361:8 | v5 | | +| vector.cpp:359:34:359:35 | ref arg v5 | vector.cpp:363:7:363:8 | v5 | | +| vector.cpp:359:34:359:35 | ref arg v5 | vector.cpp:401:1:401:1 | v5 | | +| vector.cpp:359:34:359:35 | v5 | vector.cpp:359:37:359:41 | call to begin | TAINT | +| vector.cpp:359:37:359:41 | call to begin | vector.cpp:360:3:360:4 | i5 | | +| vector.cpp:359:37:359:41 | call to begin | vector.cpp:362:3:362:4 | i5 | | +| vector.cpp:360:2:360:2 | call to operator* [post update] | vector.cpp:361:7:361:8 | v5 | | +| vector.cpp:360:2:360:2 | call to operator* [post update] | vector.cpp:363:7:363:8 | v5 | | +| vector.cpp:360:2:360:2 | call to operator* [post update] | vector.cpp:401:1:401:1 | v5 | | +| vector.cpp:360:2:360:15 | ... = ... | vector.cpp:360:2:360:2 | call to operator* [post update] | | +| vector.cpp:360:3:360:4 | i5 | vector.cpp:360:2:360:2 | call to operator* | TAINT | +| vector.cpp:360:8:360:13 | call to source | vector.cpp:360:2:360:2 | call to operator* [post update] | TAINT | +| vector.cpp:360:8:360:13 | call to source | vector.cpp:360:2:360:15 | ... = ... | | +| vector.cpp:361:7:361:8 | ref arg v5 | vector.cpp:363:7:363:8 | v5 | | +| vector.cpp:361:7:361:8 | ref arg v5 | vector.cpp:401:1:401:1 | v5 | | +| vector.cpp:362:2:362:2 | call to operator* [post update] | vector.cpp:363:7:363:8 | v5 | | +| vector.cpp:362:2:362:2 | call to operator* [post update] | vector.cpp:401:1:401:1 | v5 | | +| vector.cpp:362:2:362:8 | ... = ... | vector.cpp:362:2:362:2 | call to operator* [post update] | | +| vector.cpp:362:3:362:4 | i5 | vector.cpp:362:2:362:2 | call to operator* | TAINT | +| vector.cpp:362:8:362:8 | 1 | vector.cpp:362:2:362:2 | call to operator* [post update] | TAINT | +| vector.cpp:362:8:362:8 | 1 | vector.cpp:362:2:362:8 | ... = ... | | +| vector.cpp:363:7:363:8 | ref arg v5 | vector.cpp:401:1:401:1 | v5 | | +| vector.cpp:365:34:365:35 | ref arg v6 | vector.cpp:367:7:367:8 | v6 | | +| vector.cpp:365:34:365:35 | ref arg v6 | vector.cpp:368:2:368:3 | v6 | | +| vector.cpp:365:34:365:35 | ref arg v6 | vector.cpp:369:7:369:8 | v6 | | +| vector.cpp:365:34:365:35 | ref arg v6 | vector.cpp:401:1:401:1 | v6 | | +| vector.cpp:365:34:365:35 | v6 | vector.cpp:365:37:365:41 | call to begin | TAINT | +| vector.cpp:365:37:365:41 | call to begin | vector.cpp:366:3:366:4 | i6 | | +| vector.cpp:366:2:366:2 | call to operator* [post update] | vector.cpp:367:7:367:8 | v6 | | +| vector.cpp:366:2:366:2 | call to operator* [post update] | vector.cpp:368:2:368:3 | v6 | | +| vector.cpp:366:2:366:2 | call to operator* [post update] | vector.cpp:369:7:369:8 | v6 | | +| vector.cpp:366:2:366:2 | call to operator* [post update] | vector.cpp:401:1:401:1 | v6 | | +| vector.cpp:366:2:366:15 | ... = ... | vector.cpp:366:2:366:2 | call to operator* [post update] | | +| vector.cpp:366:3:366:4 | i6 | vector.cpp:366:2:366:2 | call to operator* | TAINT | +| vector.cpp:366:8:366:13 | call to source | vector.cpp:366:2:366:2 | call to operator* [post update] | TAINT | +| vector.cpp:366:8:366:13 | call to source | vector.cpp:366:2:366:15 | ... = ... | | +| vector.cpp:367:7:367:8 | ref arg v6 | vector.cpp:368:2:368:3 | v6 | | +| vector.cpp:367:7:367:8 | ref arg v6 | vector.cpp:369:7:369:8 | v6 | | +| vector.cpp:367:7:367:8 | ref arg v6 | vector.cpp:401:1:401:1 | v6 | | +| vector.cpp:368:2:368:3 | ref arg v6 | vector.cpp:369:7:369:8 | v6 | | +| vector.cpp:368:2:368:3 | ref arg v6 | vector.cpp:401:1:401:1 | v6 | | +| vector.cpp:368:7:368:26 | call to vector | vector.cpp:368:2:368:3 | ref arg v6 | TAINT | +| vector.cpp:368:7:368:26 | call to vector | vector.cpp:368:5:368:5 | call to operator= | TAINT | +| vector.cpp:369:7:369:8 | ref arg v6 | vector.cpp:401:1:401:1 | v6 | | +| vector.cpp:371:34:371:35 | ref arg v7 | vector.cpp:374:8:374:9 | v7 | | +| vector.cpp:371:34:371:35 | ref arg v7 | vector.cpp:377:8:377:9 | v7 | | +| vector.cpp:371:34:371:35 | ref arg v7 | vector.cpp:379:7:379:8 | v7 | | +| vector.cpp:371:34:371:35 | ref arg v7 | vector.cpp:401:1:401:1 | v7 | | +| vector.cpp:371:34:371:35 | v7 | vector.cpp:371:37:371:41 | call to begin | TAINT | +| vector.cpp:371:37:371:41 | call to begin | vector.cpp:373:4:373:5 | i7 | | +| vector.cpp:371:37:371:41 | call to begin | vector.cpp:376:4:376:5 | i7 | | +| vector.cpp:373:3:373:3 | call to operator* [post update] | vector.cpp:374:8:374:9 | v7 | | +| vector.cpp:373:3:373:3 | call to operator* [post update] | vector.cpp:379:7:379:8 | v7 | | +| vector.cpp:373:3:373:3 | call to operator* [post update] | vector.cpp:401:1:401:1 | v7 | | +| vector.cpp:373:3:373:16 | ... = ... | vector.cpp:373:3:373:3 | call to operator* [post update] | | +| vector.cpp:373:4:373:5 | i7 | vector.cpp:373:3:373:3 | call to operator* | TAINT | +| vector.cpp:373:9:373:14 | call to source | vector.cpp:373:3:373:3 | call to operator* [post update] | TAINT | +| vector.cpp:373:9:373:14 | call to source | vector.cpp:373:3:373:16 | ... = ... | | +| vector.cpp:374:8:374:9 | ref arg v7 | vector.cpp:379:7:379:8 | v7 | | +| vector.cpp:374:8:374:9 | ref arg v7 | vector.cpp:401:1:401:1 | v7 | | +| vector.cpp:376:3:376:3 | call to operator* [post update] | vector.cpp:377:8:377:9 | v7 | | +| vector.cpp:376:3:376:3 | call to operator* [post update] | vector.cpp:379:7:379:8 | v7 | | +| vector.cpp:376:3:376:3 | call to operator* [post update] | vector.cpp:401:1:401:1 | v7 | | +| vector.cpp:376:3:376:9 | ... = ... | vector.cpp:376:3:376:3 | call to operator* [post update] | | +| vector.cpp:376:4:376:5 | i7 | vector.cpp:376:3:376:3 | call to operator* | TAINT | +| vector.cpp:376:9:376:9 | 1 | vector.cpp:376:3:376:3 | call to operator* [post update] | TAINT | +| vector.cpp:376:9:376:9 | 1 | vector.cpp:376:3:376:9 | ... = ... | | +| vector.cpp:377:8:377:9 | ref arg v7 | vector.cpp:379:7:379:8 | v7 | | +| vector.cpp:377:8:377:9 | ref arg v7 | vector.cpp:401:1:401:1 | v7 | | +| vector.cpp:379:7:379:8 | ref arg v7 | vector.cpp:401:1:401:1 | v7 | | +| vector.cpp:381:34:381:35 | ref arg v8 | vector.cpp:383:7:383:8 | v8 | | +| vector.cpp:381:34:381:35 | ref arg v8 | vector.cpp:385:7:385:8 | v8 | | +| vector.cpp:381:34:381:35 | ref arg v8 | vector.cpp:401:1:401:1 | v8 | | +| vector.cpp:381:34:381:35 | v8 | vector.cpp:381:37:381:41 | call to begin | TAINT | +| vector.cpp:381:37:381:41 | call to begin | vector.cpp:382:3:382:4 | i8 | | +| vector.cpp:381:37:381:41 | call to begin | vector.cpp:384:3:384:4 | i8 | | +| vector.cpp:382:2:382:2 | call to operator* [post update] | vector.cpp:383:7:383:8 | v8 | | +| vector.cpp:382:2:382:2 | call to operator* [post update] | vector.cpp:385:7:385:8 | v8 | | +| vector.cpp:382:2:382:2 | call to operator* [post update] | vector.cpp:401:1:401:1 | v8 | | +| vector.cpp:382:2:382:15 | ... = ... | vector.cpp:382:2:382:2 | call to operator* [post update] | | +| vector.cpp:382:3:382:4 | i8 | vector.cpp:382:2:382:2 | call to operator* | TAINT | +| vector.cpp:382:8:382:13 | call to source | vector.cpp:382:2:382:2 | call to operator* [post update] | TAINT | +| vector.cpp:382:8:382:13 | call to source | vector.cpp:382:2:382:15 | ... = ... | | +| vector.cpp:383:7:383:8 | ref arg v8 | vector.cpp:385:7:385:8 | v8 | | +| vector.cpp:383:7:383:8 | ref arg v8 | vector.cpp:401:1:401:1 | v8 | | +| vector.cpp:384:2:384:2 | call to operator* [post update] | vector.cpp:385:7:385:8 | v8 | | +| vector.cpp:384:2:384:2 | call to operator* [post update] | vector.cpp:401:1:401:1 | v8 | | +| vector.cpp:384:2:384:8 | ... = ... | vector.cpp:384:2:384:2 | call to operator* [post update] | | +| vector.cpp:384:3:384:4 | i8 | vector.cpp:384:2:384:2 | call to operator* | TAINT | +| vector.cpp:384:8:384:8 | 1 | vector.cpp:384:2:384:2 | call to operator* [post update] | TAINT | +| vector.cpp:384:8:384:8 | 1 | vector.cpp:384:2:384:8 | ... = ... | | +| vector.cpp:385:7:385:8 | ref arg v8 | vector.cpp:401:1:401:1 | v8 | | +| vector.cpp:387:34:387:35 | ref arg v9 | vector.cpp:392:7:392:8 | v9 | | +| vector.cpp:387:34:387:35 | ref arg v9 | vector.cpp:401:1:401:1 | v9 | | +| vector.cpp:387:34:387:35 | v9 | vector.cpp:387:37:387:41 | call to begin | TAINT | +| vector.cpp:387:37:387:41 | call to begin | vector.cpp:389:3:389:4 | i9 | | +| vector.cpp:387:37:387:41 | call to begin | vector.cpp:390:31:390:32 | i9 | | +| vector.cpp:389:2:389:2 | call to operator* [post update] | vector.cpp:392:7:392:8 | v9 | | +| vector.cpp:389:2:389:2 | call to operator* [post update] | vector.cpp:401:1:401:1 | v9 | | +| vector.cpp:389:2:389:15 | ... = ... | vector.cpp:389:2:389:2 | call to operator* [post update] | | +| vector.cpp:389:3:389:4 | i9 | vector.cpp:389:2:389:2 | call to operator* | TAINT | +| vector.cpp:389:8:389:13 | call to source | vector.cpp:389:2:389:2 | call to operator* [post update] | TAINT | +| vector.cpp:389:8:389:13 | call to source | vector.cpp:389:2:389:15 | ... = ... | | +| vector.cpp:390:31:390:32 | call to iterator [post update] | vector.cpp:392:7:392:8 | v9 | | +| vector.cpp:390:31:390:32 | call to iterator [post update] | vector.cpp:401:1:401:1 | v9 | | +| vector.cpp:390:31:390:32 | i9 | vector.cpp:390:31:390:32 | call to iterator | | +| vector.cpp:390:31:390:32 | i9 [post update] | vector.cpp:392:7:392:8 | v9 | | +| vector.cpp:390:31:390:32 | i9 [post update] | vector.cpp:401:1:401:1 | v9 | | +| vector.cpp:392:7:392:8 | ref arg v9 | vector.cpp:401:1:401:1 | v9 | | +| vector.cpp:394:35:394:37 | ref arg v10 | vector.cpp:396:7:396:9 | v10 | | +| vector.cpp:394:35:394:37 | ref arg v10 | vector.cpp:401:1:401:1 | v10 | | +| vector.cpp:394:35:394:37 | v10 | vector.cpp:394:39:394:43 | call to begin | TAINT | +| vector.cpp:394:39:394:43 | call to begin | vector.cpp:395:33:395:35 | i10 | | +| vector.cpp:395:33:395:35 | call to iterator [post update] | vector.cpp:396:7:396:9 | v10 | | +| vector.cpp:395:33:395:35 | call to iterator [post update] | vector.cpp:401:1:401:1 | v10 | | +| vector.cpp:395:33:395:35 | i10 | vector.cpp:395:33:395:35 | call to iterator | | +| vector.cpp:395:33:395:35 | i10 [post update] | vector.cpp:396:7:396:9 | v10 | | +| vector.cpp:395:33:395:35 | i10 [post update] | vector.cpp:401:1:401:1 | v10 | | +| vector.cpp:396:7:396:9 | ref arg v10 | vector.cpp:401:1:401:1 | v10 | | +| vector.cpp:398:35:398:37 | ref arg v11 | vector.cpp:400:7:400:9 | v11 | | +| vector.cpp:398:35:398:37 | ref arg v11 | vector.cpp:401:1:401:1 | v11 | | +| vector.cpp:398:35:398:37 | v11 | vector.cpp:398:39:398:43 | call to begin | TAINT | +| vector.cpp:398:39:398:43 | call to begin | vector.cpp:399:33:399:35 | i11 | | +| vector.cpp:399:33:399:35 | call to iterator [post update] | vector.cpp:400:7:400:9 | v11 | | +| vector.cpp:399:33:399:35 | call to iterator [post update] | vector.cpp:401:1:401:1 | v11 | | +| vector.cpp:399:33:399:35 | i11 | vector.cpp:399:33:399:35 | call to iterator | | +| vector.cpp:399:33:399:35 | i11 [post update] | vector.cpp:400:7:400:9 | v11 | | +| vector.cpp:399:33:399:35 | i11 [post update] | vector.cpp:401:1:401:1 | v11 | | +| vector.cpp:400:7:400:9 | ref arg v11 | vector.cpp:401:1:401:1 | v11 | | diff --git a/cpp/ql/test/library-tests/dataflow/taint-tests/taint.expected b/cpp/ql/test/library-tests/dataflow/taint-tests/taint.expected index 3fcbfb6fbfa..3d648347451 100644 --- a/cpp/ql/test/library-tests/dataflow/taint-tests/taint.expected +++ b/cpp/ql/test/library-tests/dataflow/taint-tests/taint.expected @@ -531,3 +531,17 @@ | vector.cpp:312:7:312:7 | d | vector.cpp:303:14:303:19 | call to source | | vector.cpp:324:7:324:8 | v2 | vector.cpp:318:15:318:20 | call to source | | vector.cpp:326:7:326:8 | v4 | vector.cpp:318:15:318:20 | call to source | +| vector.cpp:342:7:342:8 | v1 | vector.cpp:341:8:341:13 | call to source | +| vector.cpp:347:7:347:8 | v2 | vector.cpp:345:9:345:14 | call to source | +| vector.cpp:357:7:357:8 | v4 | vector.cpp:330:10:330:15 | call to source | +| vector.cpp:361:7:361:8 | v5 | vector.cpp:360:8:360:13 | call to source | +| vector.cpp:363:7:363:8 | v5 | vector.cpp:360:8:360:13 | call to source | +| vector.cpp:367:7:367:8 | v6 | vector.cpp:366:8:366:13 | call to source | +| vector.cpp:369:7:369:8 | v6 | vector.cpp:366:8:366:13 | call to source | +| vector.cpp:374:8:374:9 | v7 | vector.cpp:373:9:373:14 | call to source | +| vector.cpp:379:7:379:8 | v7 | vector.cpp:373:9:373:14 | call to source | +| vector.cpp:383:7:383:8 | v8 | vector.cpp:382:8:382:13 | call to source | +| vector.cpp:385:7:385:8 | v8 | vector.cpp:382:8:382:13 | call to source | +| vector.cpp:392:7:392:8 | v9 | vector.cpp:330:10:330:15 | call to source | +| vector.cpp:392:7:392:8 | v9 | vector.cpp:389:8:389:13 | call to source | +| vector.cpp:400:7:400:9 | v11 | vector.cpp:399:38:399:43 | call to source | diff --git a/cpp/ql/test/library-tests/dataflow/taint-tests/test_diff.expected b/cpp/ql/test/library-tests/dataflow/taint-tests/test_diff.expected index e0a02761830..95b16152f58 100644 --- a/cpp/ql/test/library-tests/dataflow/taint-tests/test_diff.expected +++ b/cpp/ql/test/library-tests/dataflow/taint-tests/test_diff.expected @@ -267,3 +267,17 @@ | vector.cpp:292:7:292:18 | vector.cpp:289:17:289:30 | AST only | | vector.cpp:308:9:308:14 | vector.cpp:303:14:303:19 | AST only | | vector.cpp:311:9:311:14 | vector.cpp:303:14:303:19 | AST only | +| vector.cpp:342:7:342:8 | vector.cpp:341:8:341:13 | AST only | +| vector.cpp:347:7:347:8 | vector.cpp:345:9:345:14 | AST only | +| vector.cpp:357:7:357:8 | vector.cpp:330:10:330:15 | AST only | +| vector.cpp:361:7:361:8 | vector.cpp:360:8:360:13 | AST only | +| vector.cpp:363:7:363:8 | vector.cpp:360:8:360:13 | AST only | +| vector.cpp:367:7:367:8 | vector.cpp:366:8:366:13 | AST only | +| vector.cpp:369:7:369:8 | vector.cpp:366:8:366:13 | AST only | +| vector.cpp:374:8:374:9 | vector.cpp:373:9:373:14 | AST only | +| vector.cpp:379:7:379:8 | vector.cpp:373:9:373:14 | AST only | +| vector.cpp:383:7:383:8 | vector.cpp:382:8:382:13 | AST only | +| vector.cpp:385:7:385:8 | vector.cpp:382:8:382:13 | AST only | +| vector.cpp:392:7:392:8 | vector.cpp:330:10:330:15 | AST only | +| vector.cpp:392:7:392:8 | vector.cpp:389:8:389:13 | AST only | +| vector.cpp:400:7:400:9 | vector.cpp:399:38:399:43 | AST only | diff --git a/cpp/ql/test/library-tests/dataflow/taint-tests/vector.cpp b/cpp/ql/test/library-tests/dataflow/taint-tests/vector.cpp index a718f2a69e2..b5dad811998 100644 --- a/cpp/ql/test/library-tests/dataflow/taint-tests/vector.cpp +++ b/cpp/ql/test/library-tests/dataflow/taint-tests/vector.cpp @@ -21,7 +21,7 @@ void test_range_based_for_loop_vector(int source1) { } for(std::vector::iterator it = v.begin(); it != v.end(); ++it) { - sink(*it); // tainted + sink(*it); // tainted [NOT DETECTED by IR] } for(int& x : v) { @@ -325,3 +325,77 @@ void test_constructors_more() { sink(v3); sink(v4); // tainted } + +void taint_vector_output_iterator(std::vector::iterator iter) { + *iter = source(); +} + +void vector_iterator_assign_wrapper(std::vector::iterator iter, int i) { + *iter = i; +} + +void test_vector_output_iterator(int b) { + std::vector v1(10), v2(10), v3(10), v4(10), v5(10), v6(10), v7(10), v8(10), v9(10), v10(10), v11(10); + + std::vector::iterator i1 = v1.begin(); + *i1 = source(); + sink(v1); // tainted [NOT DETECTED by IR] + + for(std::vector::iterator it = v2.begin(); it != v2.end(); ++it) { + *it = source(); + } + sink(v2); // tainted [NOT DETECTED by IR] + + for(int& x : v3) { + x = source(); + } + sink(v3); // tainted [NOT DETECTED] + + for(std::vector::iterator it = v4.begin(); it != v4.end(); ++it) { + taint_vector_output_iterator(it); + } + sink(v4); // tainted [NOT DETECTED by IR] + + std::vector::iterator i5 = v5.begin(); + *i5 = source(); + sink(v5); // tainted [NOT DETECTED by IR] + *i5 = 1; + sink(v5); // tainted [NOT DETECTED by IR] + + std::vector::iterator i6 = v6.begin(); + *i6 = source(); + sink(v6); // tainted [NOT DETECTED by IR] + v6 = std::vector(10); + sink(v6); // [FALSE POSITIVE in AST] + + std::vector::iterator i7 = v7.begin(); + if(b) { + *i7 = source(); + sink(v7); // tainted [NOT DETECTED by IR] + } else { + *i7 = 1; + sink(v7); + } + sink(v7); // tainted [NOT DETECTED by IR] + + std::vector::iterator i8 = v8.begin(); + *i8 = source(); + sink(v8); // tainted [NOT DETECTED by IR] + *i8 = 1; + sink(v8); + + std::vector::iterator i9 = v9.begin(); + + *i9 = source(); + taint_vector_output_iterator(i9); + + sink(v9); + + std::vector::iterator i10 = v10.begin(); + vector_iterator_assign_wrapper(i10, 10); + sink(v10); + + std::vector::iterator i11 = v11.begin(); + vector_iterator_assign_wrapper(i11, source()); + sink(v11); // tainted [NOT DETECTED by IR] +} diff --git a/cpp/ql/test/library-tests/lambdas/captures/elements.expected b/cpp/ql/test/library-tests/lambdas/captures/elements.expected index 25bb695ea97..8ba9cad4009 100644 --- a/cpp/ql/test/library-tests/lambdas/captures/elements.expected +++ b/cpp/ql/test/library-tests/lambdas/captures/elements.expected @@ -13,9 +13,9 @@ | captures.cpp:3:5:3:5 | (constructor) | | captures.cpp:3:5:3:5 | (constructor) | | captures.cpp:3:5:3:5 | (constructor) | -| captures.cpp:3:5:3:5 | declaration of (null) | -| captures.cpp:3:5:3:5 | declaration of (null) | -| captures.cpp:3:5:3:5 | definition of (null) | +| captures.cpp:3:5:3:5 | declaration of (constructor) | +| captures.cpp:3:5:3:5 | declaration of (constructor) | +| captures.cpp:3:5:3:5 | definition of (constructor) | | captures.cpp:3:5:3:5 | definition of operator= | | captures.cpp:3:5:3:5 | operator= | | captures.cpp:3:5:5:5 | [...](...){...} | @@ -50,9 +50,9 @@ | captures.cpp:9:5:9:5 | (constructor) | | captures.cpp:9:5:9:5 | (constructor) | | captures.cpp:9:5:9:5 | (constructor) | -| captures.cpp:9:5:9:5 | declaration of (null) | -| captures.cpp:9:5:9:5 | declaration of (null) | -| captures.cpp:9:5:9:5 | definition of (null) | +| captures.cpp:9:5:9:5 | declaration of (constructor) | +| captures.cpp:9:5:9:5 | declaration of (constructor) | +| captures.cpp:9:5:9:5 | definition of (constructor) | | captures.cpp:9:5:9:5 | definition of operator= | | captures.cpp:9:5:9:5 | operator= | | captures.cpp:9:5:11:5 | [...](...){...} | @@ -87,9 +87,9 @@ | captures.cpp:15:5:15:5 | (constructor) | | captures.cpp:15:5:15:5 | (constructor) | | captures.cpp:15:5:15:5 | (constructor) | -| captures.cpp:15:5:15:5 | declaration of (null) | -| captures.cpp:15:5:15:5 | declaration of (null) | -| captures.cpp:15:5:15:5 | definition of (null) | +| captures.cpp:15:5:15:5 | declaration of (constructor) | +| captures.cpp:15:5:15:5 | declaration of (constructor) | +| captures.cpp:15:5:15:5 | definition of (constructor) | | captures.cpp:15:5:15:5 | definition of operator= | | captures.cpp:15:5:15:5 | operator= | | captures.cpp:15:5:17:5 | [...](...){...} | @@ -129,9 +129,9 @@ | captures.cpp:22:19:22:19 | Unknown literal | | captures.cpp:22:19:22:19 | constructor init of field x | | captures.cpp:22:19:22:19 | constructor init of field y | -| captures.cpp:22:19:22:19 | declaration of (null) | -| captures.cpp:22:19:22:19 | definition of (null) | -| captures.cpp:22:19:22:19 | definition of (null) | +| captures.cpp:22:19:22:19 | declaration of (constructor) | +| captures.cpp:22:19:22:19 | definition of (constructor) | +| captures.cpp:22:19:22:19 | definition of (constructor) | | captures.cpp:22:19:22:19 | definition of operator= | | captures.cpp:22:19:22:19 | operator= | | captures.cpp:22:19:22:19 | return ... | @@ -187,9 +187,9 @@ | end_pos.cpp:9:15:9:15 | (constructor) | | end_pos.cpp:9:15:9:15 | Unknown literal | | end_pos.cpp:9:15:9:15 | constructor init of field ii | -| end_pos.cpp:9:15:9:15 | declaration of (null) | -| end_pos.cpp:9:15:9:15 | definition of (null) | -| end_pos.cpp:9:15:9:15 | definition of (null) | +| end_pos.cpp:9:15:9:15 | declaration of (constructor) | +| end_pos.cpp:9:15:9:15 | definition of (constructor) | +| end_pos.cpp:9:15:9:15 | definition of (constructor) | | end_pos.cpp:9:15:9:15 | definition of operator= | | end_pos.cpp:9:15:9:15 | operator= | | end_pos.cpp:9:15:9:15 | return ... | diff --git a/cpp/ql/test/library-tests/noexcept/copy_from_prototype/copy_from_prototype.expected b/cpp/ql/test/library-tests/noexcept/copy_from_prototype/copy_from_prototype.expected index f5ef2d77c98..5488cf803e6 100644 --- a/cpp/ql/test/library-tests/noexcept/copy_from_prototype/copy_from_prototype.expected +++ b/cpp/ql/test/library-tests/noexcept/copy_from_prototype/copy_from_prototype.expected @@ -2,8 +2,8 @@ | copy_from_prototype.cpp:3:7:3:7 | a | a::a(const a &) -> void | copy_from_prototype.cpp:3:7:3:7 | a | | | copy_from_prototype.cpp:3:7:3:7 | operator= | a::operator=(a &&) -> a & | copy_from_prototype.cpp:3:7:3:7 | a | | | copy_from_prototype.cpp:3:7:3:7 | operator= | a::operator=(const a &) -> a & | copy_from_prototype.cpp:3:7:3:7 | a | | -| copy_from_prototype.cpp:4:26:4:26 | a | a<>::a<(unnamed)>() -> void | copy_from_prototype.cpp:3:7:3:7 | a<> | 123 | -| copy_from_prototype.cpp:4:26:4:26 | a | a::a<(unnamed)>() -> void | copy_from_prototype.cpp:3:7:3:7 | a | | +| copy_from_prototype.cpp:4:26:4:26 | a | a<>::a<(unnamed template parameter)>() -> void | copy_from_prototype.cpp:3:7:3:7 | a<> | 123 | +| copy_from_prototype.cpp:4:26:4:26 | a | a::a<(unnamed template parameter)>() -> void | copy_from_prototype.cpp:3:7:3:7 | a | | | copy_from_prototype.cpp:7:7:7:7 | b | b::b() -> void | copy_from_prototype.cpp:7:7:7:7 | b | | | copy_from_prototype.cpp:7:7:7:7 | b | b::b(b &&) -> void | copy_from_prototype.cpp:7:7:7:7 | b | | | copy_from_prototype.cpp:7:7:7:7 | b | b::b(const b &) -> void | copy_from_prototype.cpp:7:7:7:7 | b | | @@ -13,8 +13,8 @@ | copy_from_prototype.cpp:13:7:13:7 | c | c::c(const c &) -> void | copy_from_prototype.cpp:13:7:13:7 | c | | | copy_from_prototype.cpp:13:7:13:7 | operator= | c::operator=(c &&) -> c & | copy_from_prototype.cpp:13:7:13:7 | c | | | copy_from_prototype.cpp:13:7:13:7 | operator= | c::operator=(const c &) -> c & | copy_from_prototype.cpp:13:7:13:7 | c | | -| copy_from_prototype.cpp:14:26:14:26 | c | c::c<(unnamed)>() -> void | copy_from_prototype.cpp:13:7:13:7 | c | X | -| copy_from_prototype.cpp:14:26:14:26 | c | c::c<(unnamed)>() -> void | copy_from_prototype.cpp:13:7:13:7 | c | | +| copy_from_prototype.cpp:14:26:14:26 | c | c::c<(unnamed template parameter)>() -> void | copy_from_prototype.cpp:13:7:13:7 | c | X | +| copy_from_prototype.cpp:14:26:14:26 | c | c::c<(unnamed template parameter)>() -> void | copy_from_prototype.cpp:13:7:13:7 | c | | | copy_from_prototype.cpp:17:7:17:7 | d | d::d() -> void | copy_from_prototype.cpp:17:7:17:7 | d | | | copy_from_prototype.cpp:17:7:17:7 | d | d::d(const d &) -> void | copy_from_prototype.cpp:17:7:17:7 | d | | | copy_from_prototype.cpp:17:7:17:7 | d | d::d(d &&) -> void | copy_from_prototype.cpp:17:7:17:7 | d | | @@ -24,7 +24,7 @@ | copy_from_prototype.cpp:22:8:22:8 | e | e::e(e &&) -> void | copy_from_prototype.cpp:22:8:22:8 | e | | | copy_from_prototype.cpp:22:8:22:8 | operator= | e::operator=(const e &) -> e & | copy_from_prototype.cpp:22:8:22:8 | e | | | copy_from_prototype.cpp:22:8:22:8 | operator= | e::operator=(e &&) -> e & | copy_from_prototype.cpp:22:8:22:8 | e | | -| copy_from_prototype.cpp:23:26:23:26 | e | e::e<(unnamed)>() -> void | copy_from_prototype.cpp:22:8:22:8 | e | 456 | -| copy_from_prototype.cpp:26:35:26:43 | e | e::e<(unnamed)>() -> void | copy_from_prototype.cpp:22:8:22:8 | e | 456 | +| copy_from_prototype.cpp:23:26:23:26 | e | e::e<(unnamed template parameter)>() -> void | copy_from_prototype.cpp:22:8:22:8 | e | 456 | +| copy_from_prototype.cpp:26:35:26:43 | e | e::e<(unnamed template parameter)>() -> void | copy_from_prototype.cpp:22:8:22:8 | e | 456 | | file://:0:0:0:0 | operator= | __va_list_tag::operator=(__va_list_tag &&) -> __va_list_tag & | file://:0:0:0:0 | __va_list_tag | | | file://:0:0:0:0 | operator= | __va_list_tag::operator=(const __va_list_tag &) -> __va_list_tag & | file://:0:0:0:0 | __va_list_tag | | diff --git a/cpp/ql/test/library-tests/special_members/generated_copy/functions.expected b/cpp/ql/test/library-tests/special_members/generated_copy/functions.expected index 2c85514d592..22282641b37 100644 --- a/cpp/ql/test/library-tests/special_members/generated_copy/functions.expected +++ b/cpp/ql/test/library-tests/special_members/generated_copy/functions.expected @@ -78,8 +78,6 @@ | copy.cpp:111:9:111:9 | MoveAssign | deleted | | | copy.cpp:111:9:111:9 | operator= | deleted | | | copy.cpp:113:17:113:25 | operator= | | | -| copy.cpp:120:9:120:9 | OnlyCtor | | | -| copy.cpp:120:9:120:9 | OnlyCtor | | | | copy.cpp:120:9:120:9 | OnlyCtor | deleted | | | copy.cpp:120:9:120:9 | operator= | deleted | | | copy.cpp:126:11:126:19 | operator= | | | diff --git a/cpp/ql/test/library-tests/syntax-zoo/dataflow-ir-consistency.expected b/cpp/ql/test/library-tests/syntax-zoo/dataflow-ir-consistency.expected index ee50bd638f9..b4175c9e499 100644 --- a/cpp/ql/test/library-tests/syntax-zoo/dataflow-ir-consistency.expected +++ b/cpp/ql/test/library-tests/syntax-zoo/dataflow-ir-consistency.expected @@ -539,8 +539,6 @@ uniqueNodeLocation | file://:0:0:0:0 | p#0 | Node should have one location but has 0. | | file://:0:0:0:0 | p#0 | Node should have one location but has 0. | | file://:0:0:0:0 | p#0 | Node should have one location but has 0. | -| file://:0:0:0:0 | p#0 | Node should have one location but has 0. | -| file://:0:0:0:0 | p#0 | Node should have one location but has 0. | | file://:0:0:0:0 | p#1 | Node should have one location but has 0. | | file://:0:0:0:0 | p#1 | Node should have one location but has 0. | | file://:0:0:0:0 | p#1 | Node should have one location but has 0. | @@ -1418,7 +1416,7 @@ uniqueNodeLocation | whilestmt.c:39:6:39:11 | ReturnVoid | Node should have one location but has 4. | | whilestmt.c:39:6:39:11 | SideEffect | Node should have one location but has 4. | missingLocation -| Nodes without location: 36 | +| Nodes without location: 34 | uniqueNodeToString | break_labels.c:2:11:2:11 | i | Node should have one toString but has 2. | | break_labels.c:2:11:2:11 | i | Node should have one toString but has 2. | diff --git a/cpp/ql/test/library-tests/templates/CPP-202/template_args.expected b/cpp/ql/test/library-tests/templates/CPP-202/template_args.expected index f38eef610e4..c1d6e7fe9fa 100644 --- a/cpp/ql/test/library-tests/templates/CPP-202/template_args.expected +++ b/cpp/ql/test/library-tests/templates/CPP-202/template_args.expected @@ -1,7 +1,7 @@ | file://:0:0:0:0 | __va_list_tag | | | test.cpp:3:8:3:9 | s1<> | {...} | -| test.cpp:3:8:3:9 | s1<> | (null) | +| test.cpp:3:8:3:9 | s1<> | (unnamed template parameter constant) | | test.cpp:5:8:5:9 | s2 | T | | test.cpp:5:8:5:9 | s2 | T | -| test.cpp:7:8:7:9 | s3> | (unnamed) | +| test.cpp:7:8:7:9 | s3> | (unnamed template parameter) | | test.cpp:7:8:7:9 | s3> | T | diff --git a/cpp/ql/test/library-tests/templates/decls/decls.expected b/cpp/ql/test/library-tests/templates/decls/decls.expected index 43a691e53c9..cfb16b63828 100644 --- a/cpp/ql/test/library-tests/templates/decls/decls.expected +++ b/cpp/ql/test/library-tests/templates/decls/decls.expected @@ -7,7 +7,7 @@ | decls.cpp:4:30:4:34 | p#0 | | decls.cpp:4:30:4:34 | p#0 | | decls.cpp:6:17:6:17 | f | -| decls.cpp:8:18:8:18 | (unnamed) | +| decls.cpp:8:18:8:18 | (unnamed template parameter) | | decls.cpp:8:25:8:25 | g | | file://:0:0:0:0 | __va_list_tag | | file://:0:0:0:0 | auto | diff --git a/cpp/ql/test/library-tests/unnamed/elements.expected b/cpp/ql/test/library-tests/unnamed/elements.expected index 8df7d4578a2..52176aa66df 100644 --- a/cpp/ql/test/library-tests/unnamed/elements.expected +++ b/cpp/ql/test/library-tests/unnamed/elements.expected @@ -1,6 +1,6 @@ | file://:0:0:0:0 | | Other | | file://:0:0:0:0 | (global namespace) | Other | -| file://:0:0:0:0 | | Other | +| file://:0:0:0:0 | (unnamed global/namespace variable) | Other | | file://:0:0:0:0 | _Complex __float128 | Other | | file://:0:0:0:0 | _Complex double | Other | | file://:0:0:0:0 | _Complex float | Other | @@ -111,8 +111,8 @@ | test.c:0:0:0:0 | test.c | Other | | test.c:2:6:2:6 | a | Other | | test.c:2:6:2:6 | definition of a | Other | -| test.c:2:10:2:18 | | Variable access | +| test.c:2:10:2:18 | (unnamed global/namespace variable) | Variable access | | test.c:2:10:2:18 | array to pointer conversion | Other | | test.c:2:10:2:18 | initializer for a | Other | -| test.c:2:17:2:18 | initializer for | Other | +| test.c:2:17:2:18 | initializer for (unnamed global/namespace variable) | Other | | test.c:2:17:2:18 | {...} | Other | diff --git a/csharp/autobuilder/Semmle.Autobuild.CSharp.Tests/BuildScripts.cs b/csharp/autobuilder/Semmle.Autobuild.CSharp.Tests/BuildScripts.cs index 51d56e6cda1..5934fb51c67 100644 --- a/csharp/autobuilder/Semmle.Autobuild.CSharp.Tests/BuildScripts.cs +++ b/csharp/autobuilder/Semmle.Autobuild.CSharp.Tests/BuildScripts.cs @@ -363,7 +363,6 @@ namespace Semmle.Autobuild.CSharp.Tests string cwd = @"C:\Project") { string codeqlUpperLanguage = Language.CSharp.UpperCaseName; - Actions.GetEnvironmentVariable[$"CODEQL_AUTOBUILDER_{codeqlUpperLanguage}_NO_INDEXING"] = "false"; Actions.GetEnvironmentVariable[$"CODEQL_EXTRACTOR_{codeqlUpperLanguage}_TRAP_DIR"] = ""; Actions.GetEnvironmentVariable[$"CODEQL_EXTRACTOR_{codeqlUpperLanguage}_SOURCE_ARCHIVE_DIR"] = ""; Actions.GetEnvironmentVariable[$"CODEQL_EXTRACTOR_{codeqlUpperLanguage}_ROOT"] = $@"C:\codeql\{codeqlUpperLanguage.ToLowerInvariant()}"; @@ -400,8 +399,6 @@ namespace Semmle.Autobuild.CSharp.Tests Actions.RunProcess[@"cmd.exe /C dotnet clean C:\Project\test.csproj"] = 0; Actions.RunProcess[@"cmd.exe /C dotnet restore C:\Project\test.csproj"] = 0; Actions.RunProcess[@"cmd.exe /C C:\odasa\tools\odasa index --auto dotnet build --no-incremental C:\Project\test.csproj"] = 0; - Actions.RunProcess[@"cmd.exe /C C:\codeql\tools\java\bin\java -jar C:\codeql\csharp\tools\extractor-asp.jar ."] = 0; - Actions.RunProcess[@"cmd.exe /C C:\odasa\tools\odasa index --xml --extensions config csproj props xml"] = 0; Actions.FileExists["csharp.log"] = true; Actions.FileExists[@"C:\Project\test.csproj"] = true; Actions.GetEnvironmentVariable["CODEQL_EXTRACTOR_CSHARP_TRAP_DIR"] = ""; @@ -419,7 +416,7 @@ namespace Semmle.Autobuild.CSharp.Tests Actions.LoadXml[@"C:\Project\test.csproj"] = xml; var autobuilder = CreateAutoBuilder(true); - TestAutobuilderScript(autobuilder, 0, 6); + TestAutobuilderScript(autobuilder, 0, 4); } [Fact] @@ -432,8 +429,6 @@ Microsoft.NETCore.App 2.2.5 [/usr/local/share/dotnet/shared/Microsoft.NETCore.Ap Actions.RunProcess[@"dotnet clean C:\Project/test.csproj"] = 0; Actions.RunProcess[@"dotnet restore C:\Project/test.csproj"] = 0; Actions.RunProcess[@"C:\odasa/tools/odasa index --auto dotnet build --no-incremental /p:UseSharedCompilation=false C:\Project/test.csproj"] = 0; - Actions.RunProcess[@"C:\codeql\tools\java/bin/java -jar C:\codeql\csharp/tools/extractor-asp.jar ."] = 0; - Actions.RunProcess[@"C:\odasa/tools/odasa index --xml --extensions config csproj props xml"] = 0; Actions.FileExists["csharp.log"] = true; Actions.FileExists[@"C:\Project/test.csproj"] = true; Actions.GetEnvironmentVariable["CODEQL_EXTRACTOR_CSHARP_TRAP_DIR"] = ""; @@ -451,7 +446,7 @@ Microsoft.NETCore.App 2.2.5 [/usr/local/share/dotnet/shared/Microsoft.NETCore.Ap Actions.LoadXml[@"C:\Project/test.csproj"] = xml; var autobuilder = CreateAutoBuilder(false); - TestAutobuilderScript(autobuilder, 0, 7); + TestAutobuilderScript(autobuilder, 0, 5); } [Fact] @@ -522,8 +517,6 @@ Microsoft.NETCore.App 2.2.5 [/usr/local/share/dotnet/shared/Microsoft.NETCore.Ap public void TestLinuxBuildlessExtractionSuccess() { Actions.RunProcess[@"C:\codeql\csharp/tools/linux64/Semmle.Extraction.CSharp.Standalone --references:."] = 0; - Actions.RunProcess[@"C:\codeql\tools\java/bin/java -jar C:\codeql\csharp/tools/extractor-asp.jar ."] = 0; - Actions.RunProcess[@"C:\odasa/tools/odasa index --xml --extensions config csproj props xml"] = 0; Actions.FileExists["csharp.log"] = true; Actions.GetEnvironmentVariable["CODEQL_EXTRACTOR_CSHARP_TRAP_DIR"] = ""; Actions.GetEnvironmentVariable["CODEQL_EXTRACTOR_CSHARP_SOURCE_ARCHIVE_DIR"] = ""; @@ -531,7 +524,7 @@ Microsoft.NETCore.App 2.2.5 [/usr/local/share/dotnet/shared/Microsoft.NETCore.Ap Actions.EnumerateDirectories[@"C:\Project"] = ""; var autobuilder = CreateAutoBuilder(false, buildless: "true"); - TestAutobuilderScript(autobuilder, 0, 3); + TestAutobuilderScript(autobuilder, 0, 1); } [Fact] @@ -552,8 +545,6 @@ Microsoft.NETCore.App 2.2.5 [/usr/local/share/dotnet/shared/Microsoft.NETCore.Ap public void TestLinuxBuildlessExtractionSolution() { Actions.RunProcess[@"C:\codeql\csharp/tools/linux64/Semmle.Extraction.CSharp.Standalone foo.sln --references:."] = 0; - Actions.RunProcess[@"C:\codeql\tools\java/bin/java -jar C:\codeql\csharp/tools/extractor-asp.jar ."] = 0; - Actions.RunProcess[@"C:\odasa/tools/odasa index --xml --extensions config csproj props xml"] = 0; Actions.FileExists["csharp.log"] = true; Actions.GetEnvironmentVariable["CODEQL_EXTRACTOR_CSHARP_TRAP_DIR"] = ""; Actions.GetEnvironmentVariable["CODEQL_EXTRACTOR_CSHARP_SOURCE_ARCHIVE_DIR"] = ""; @@ -561,7 +552,7 @@ Microsoft.NETCore.App 2.2.5 [/usr/local/share/dotnet/shared/Microsoft.NETCore.Ap Actions.EnumerateDirectories[@"C:\Project"] = ""; var autobuilder = CreateAutoBuilder(false, buildless: "true", solution: "foo.sln"); - TestAutobuilderScript(autobuilder, 0, 3); + TestAutobuilderScript(autobuilder, 0, 1); } void SkipVsWhere() @@ -598,8 +589,6 @@ Microsoft.NETCore.App 2.2.5 [/usr/local/share/dotnet/shared/Microsoft.NETCore.Ap Actions.RunProcess["dotnet --list-runtimes"] = 1; Actions.RunProcessOut["dotnet --list-runtimes"] = ""; Actions.RunProcess[@"C:\odasa/tools/odasa index --auto ""./build.sh --skip-tests"""] = 0; - Actions.RunProcess[@"C:\codeql\tools\java/bin/java -jar C:\codeql\csharp/tools/extractor-asp.jar ."] = 0; - Actions.RunProcess[@"C:\odasa/tools/odasa index --xml --extensions config csproj props xml"] = 0; Actions.FileExists["csharp.log"] = true; Actions.GetEnvironmentVariable["CODEQL_EXTRACTOR_CSHARP_TRAP_DIR"] = ""; Actions.GetEnvironmentVariable["CODEQL_EXTRACTOR_CSHARP_SOURCE_ARCHIVE_DIR"] = ""; @@ -609,7 +598,7 @@ Microsoft.NETCore.App 2.2.5 [/usr/local/share/dotnet/shared/Microsoft.NETCore.Ap SkipVsWhere(); var autobuilder = CreateAutoBuilder(false, buildCommand: "./build.sh --skip-tests"); - TestAutobuilderScript(autobuilder, 0, 4); + TestAutobuilderScript(autobuilder, 0, 2); } [Fact] @@ -624,12 +613,10 @@ Microsoft.NETCore.App 2.2.5 [/usr/local/share/dotnet/shared/Microsoft.NETCore.Ap Actions.RunProcessOut["dotnet --list-runtimes"] = ""; Actions.RunProcess[@"C:\odasa/tools/odasa index --auto C:\Project/build/build.sh"] = 0; Actions.RunProcessWorkingDirectory[@"C:\odasa/tools/odasa index --auto C:\Project/build/build.sh"] = @"C:\Project/build"; - Actions.RunProcess[@"C:\codeql\tools\java/bin/java -jar C:\codeql\csharp/tools/extractor-asp.jar ."] = 0; - Actions.RunProcess[@"C:\odasa/tools/odasa index --xml --extensions config csproj props xml"] = 0; Actions.FileExists["csharp.log"] = true; var autobuilder = CreateAutoBuilder(false); - TestAutobuilderScript(autobuilder, 0, 5); + TestAutobuilderScript(autobuilder, 0, 3); } [Fact] @@ -679,12 +666,10 @@ Microsoft.NETCore.App 2.2.5 [/usr/local/share/dotnet/shared/Microsoft.NETCore.Ap Actions.GetEnvironmentVariable["CODEQL_EXTRACTOR_CSHARP_SOURCE_ARCHIVE_DIR"] = ""; Actions.RunProcess[@"cmd.exe /C C:\odasa\tools\odasa index --auto C:\Project\build.bat"] = 0; Actions.RunProcessWorkingDirectory[@"cmd.exe /C C:\odasa\tools\odasa index --auto C:\Project\build.bat"] = @"C:\Project"; - Actions.RunProcess[@"cmd.exe /C C:\codeql\tools\java\bin\java -jar C:\codeql\csharp\tools\extractor-asp.jar ."] = 0; - Actions.RunProcess[@"cmd.exe /C C:\odasa\tools\odasa index --xml --extensions config csproj props xml"] = 0; Actions.FileExists["csharp.log"] = true; var autobuilder = CreateAutoBuilder(true); - TestAutobuilderScript(autobuilder, 0, 3); + TestAutobuilderScript(autobuilder, 0, 1); } [Fact] @@ -729,8 +714,6 @@ Microsoft.NETCore.App 2.2.5 [/usr/local/share/dotnet/shared/Microsoft.NETCore.Ap Actions.RunProcess["cmd.exe /C CALL ^\"C:\\Program Files ^(x86^)\\Microsoft Visual Studio 12.0\\VC\\vcvarsall.bat^\" && set Platform=&& type NUL && C:\\odasa\\tools\\odasa index --auto msbuild C:\\Project\\test1.sln /p:UseSharedCompilation=false /t:Windows /p:Platform=\"x86\" /p:Configuration=\"Debug\" /p:MvcBuildViews=true /P:Fu=Bar"] = 0; Actions.RunProcess[@"cmd.exe /C C:\Project\.nuget\nuget.exe restore C:\Project\test2.sln"] = 0; Actions.RunProcess["cmd.exe /C CALL ^\"C:\\Program Files ^(x86^)\\Microsoft Visual Studio 12.0\\VC\\vcvarsall.bat^\" && set Platform=&& type NUL && C:\\odasa\\tools\\odasa index --auto msbuild C:\\Project\\test2.sln /p:UseSharedCompilation=false /t:Windows /p:Platform=\"x86\" /p:Configuration=\"Debug\" /p:MvcBuildViews=true /P:Fu=Bar"] = 0; - Actions.RunProcess[@"cmd.exe /C C:\codeql\tools\java\bin\java -jar C:\codeql\csharp\tools\extractor-asp.jar ."] = 0; - Actions.RunProcess[@"cmd.exe /C C:\odasa\tools\odasa index --xml --extensions config csproj props xml"] = 0; Actions.FileExists["csharp.log"] = true; Actions.FileExists[@"C:\Program Files (x86)\Microsoft Visual Studio\Installer\vswhere.exe"] = false; Actions.FileExists[@"C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\vcvarsall.bat"] = false; @@ -752,7 +735,7 @@ Microsoft.NETCore.App 2.2.5 [/usr/local/share/dotnet/shared/Microsoft.NETCore.Ap autobuilder.ProjectsOrSolutionsToBuild.Add(testSolution1); autobuilder.ProjectsOrSolutionsToBuild.Add(testSolution2); - TestAutobuilderScript(autobuilder, 0, 6); + TestAutobuilderScript(autobuilder, 0, 4); } [Fact] @@ -762,8 +745,6 @@ Microsoft.NETCore.App 2.2.5 [/usr/local/share/dotnet/shared/Microsoft.NETCore.Ap Actions.RunProcess["cmd.exe /C CALL ^\"C:\\Program Files ^(x86^)\\Microsoft Visual Studio 12.0\\VC\\vcvarsall.bat^\" && set Platform=&& type NUL && C:\\odasa\\tools\\odasa index --auto msbuild C:\\Project\\test1.csproj /p:UseSharedCompilation=false /t:Windows /p:Platform=\"x86\" /p:Configuration=\"Debug\" /p:MvcBuildViews=true /P:Fu=Bar"] = 0; Actions.RunProcess[@"cmd.exe /C nuget restore C:\Project\test2.csproj"] = 0; Actions.RunProcess["cmd.exe /C CALL ^\"C:\\Program Files ^(x86^)\\Microsoft Visual Studio 12.0\\VC\\vcvarsall.bat^\" && set Platform=&& type NUL && C:\\odasa\\tools\\odasa index --auto msbuild C:\\Project\\test2.csproj /p:UseSharedCompilation=false /t:Windows /p:Platform=\"x86\" /p:Configuration=\"Debug\" /p:MvcBuildViews=true /P:Fu=Bar"] = 0; - Actions.RunProcess[@"cmd.exe /C C:\codeql\tools\java\bin\java -jar C:\codeql\csharp\tools\extractor-asp.jar ."] = 0; - Actions.RunProcess[@"cmd.exe /C C:\odasa\tools\odasa index --xml --extensions config csproj props xml"] = 0; Actions.FileExists["csharp.log"] = true; Actions.FileExists[@"C:\Project\test1.csproj"] = true; Actions.FileExists[@"C:\Project\test2.csproj"] = true; @@ -799,7 +780,7 @@ Microsoft.NETCore.App 2.2.5 [/usr/local/share/dotnet/shared/Microsoft.NETCore.Ap var autobuilder = CreateAutoBuilder(true, msBuildArguments: "/P:Fu=Bar", msBuildTarget: "Windows", msBuildPlatform: "x86", msBuildConfiguration: "Debug", vsToolsVersion: "12"); - TestAutobuilderScript(autobuilder, 0, 6); + TestAutobuilderScript(autobuilder, 0, 4); } [Fact] @@ -834,8 +815,6 @@ Microsoft.NETCore.App 2.2.5 [/usr/local/share/dotnet/shared/Microsoft.NETCore.Ap { Actions.RunProcess["cmd.exe /C CALL ^\"C:\\Program Files ^(x86^)\\Microsoft Visual Studio 12.0\\VC\\vcvarsall.bat^\" && set Platform=&& type NUL && C:\\odasa\\tools\\odasa index --auto msbuild C:\\Project\\test1.sln /p:UseSharedCompilation=false /t:Windows /p:Platform=\"x86\" /p:Configuration=\"Debug\" /p:MvcBuildViews=true /P:Fu=Bar"] = 0; Actions.RunProcess["cmd.exe /C CALL ^\"C:\\Program Files ^(x86^)\\Microsoft Visual Studio 12.0\\VC\\vcvarsall.bat^\" && set Platform=&& type NUL && C:\\odasa\\tools\\odasa index --auto msbuild C:\\Project\\test2.sln /p:UseSharedCompilation=false /t:Windows /p:Platform=\"x86\" /p:Configuration=\"Debug\" /p:MvcBuildViews=true /P:Fu=Bar"] = 0; - Actions.RunProcess[@"cmd.exe /C C:\codeql\tools\java\bin\java -jar C:\codeql\csharp\tools\extractor-asp.jar ."] = 0; - Actions.RunProcess[@"cmd.exe /C C:\odasa\tools\odasa index --xml --extensions config csproj props xml"] = 0; Actions.FileExists["csharp.log"] = true; Actions.FileExists[@"C:\Program Files (x86)\Microsoft Visual Studio\Installer\vswhere.exe"] = false; Actions.FileExists[@"C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\vcvarsall.bat"] = false; @@ -855,15 +834,13 @@ Microsoft.NETCore.App 2.2.5 [/usr/local/share/dotnet/shared/Microsoft.NETCore.Ap autobuilder.ProjectsOrSolutionsToBuild.Add(testSolution1); autobuilder.ProjectsOrSolutionsToBuild.Add(testSolution2); - TestAutobuilderScript(autobuilder, 0, 4); + TestAutobuilderScript(autobuilder, 0, 2); } [Fact] public void TestSkipNugetBuildless() { Actions.RunProcess[@"C:\codeql\csharp/tools/linux64/Semmle.Extraction.CSharp.Standalone foo.sln --references:. --skip-nuget"] = 0; - Actions.RunProcess[@"C:\codeql\tools\java/bin/java -jar C:\codeql\csharp/tools/extractor-asp.jar ."] = 0; - Actions.RunProcess[@"C:\odasa/tools/odasa index --xml --extensions config csproj props xml"] = 0; Actions.FileExists["csharp.log"] = true; Actions.GetEnvironmentVariable["CODEQL_EXTRACTOR_CSHARP_TRAP_DIR"] = ""; Actions.GetEnvironmentVariable["CODEQL_EXTRACTOR_CSHARP_SOURCE_ARCHIVE_DIR"] = ""; @@ -871,7 +848,7 @@ Microsoft.NETCore.App 2.2.5 [/usr/local/share/dotnet/shared/Microsoft.NETCore.Ap Actions.EnumerateDirectories[@"C:\Project"] = ""; var autobuilder = CreateAutoBuilder(false, buildless: "true", solution: "foo.sln", nugetRestore: "false"); - TestAutobuilderScript(autobuilder, 0, 3); + TestAutobuilderScript(autobuilder, 0, 1); } @@ -885,8 +862,6 @@ Microsoft.NETCore.App 2.1.3 [/usr/local/share/dotnet/shared/Microsoft.NETCore.Ap Actions.RunProcess[@"dotnet clean C:\Project/test.csproj"] = 0; Actions.RunProcess[@"dotnet restore C:\Project/test.csproj"] = 0; Actions.RunProcess[@"C:\odasa/tools/odasa index --auto dotnet build --no-incremental /p:UseSharedCompilation=false --no-restore C:\Project/test.csproj"] = 0; - Actions.RunProcess[@"C:\codeql\tools\java/bin/java -jar C:\codeql\csharp/tools/extractor-asp.jar ."] = 0; - Actions.RunProcess[@"C:\odasa/tools/odasa index --xml --extensions config csproj props xml"] = 0; Actions.FileExists["csharp.log"] = true; Actions.FileExists[@"C:\Project/test.csproj"] = true; Actions.GetEnvironmentVariable["CODEQL_EXTRACTOR_CSHARP_TRAP_DIR"] = ""; @@ -904,7 +879,7 @@ Microsoft.NETCore.App 2.1.3 [/usr/local/share/dotnet/shared/Microsoft.NETCore.Ap Actions.LoadXml[@"C:\Project/test.csproj"] = xml; var autobuilder = CreateAutoBuilder(false, dotnetArguments: "--no-restore"); // nugetRestore=false does not work for now. - TestAutobuilderScript(autobuilder, 0, 7); + TestAutobuilderScript(autobuilder, 0, 5); } [Fact] @@ -922,8 +897,6 @@ Microsoft.NETCore.App 3.0.0 [/usr/local/share/dotnet/shared/Microsoft.NETCore.Ap Actions.RunProcess[@"C:\Project/.dotnet/dotnet clean C:\Project/test.csproj"] = 0; Actions.RunProcess[@"C:\Project/.dotnet/dotnet restore C:\Project/test.csproj"] = 0; Actions.RunProcess[@"C:\odasa/tools/odasa index --auto C:\Project/.dotnet/dotnet build --no-incremental C:\Project/test.csproj"] = 0; - Actions.RunProcess[@"C:\codeql\tools\java/bin/java -jar C:\codeql\csharp/tools/extractor-asp.jar ."] = 0; - Actions.RunProcess[@"C:\odasa/tools/odasa index --xml --extensions config csproj props xml"] = 0; Actions.FileExists["csharp.log"] = true; Actions.FileExists["test.csproj"] = true; Actions.GetEnvironmentVariable["CODEQL_EXTRACTOR_CSHARP_TRAP_DIR"] = ""; @@ -943,7 +916,7 @@ Microsoft.NETCore.App 3.0.0 [/usr/local/share/dotnet/shared/Microsoft.NETCore.Ap Actions.DownloadFiles.Add(("https://dot.net/v1/dotnet-install.sh", "dotnet-install.sh")); var autobuilder = CreateAutoBuilder(false, dotnetVersion: "2.1.3"); - TestAutobuilderScript(autobuilder, 0, 11); + TestAutobuilderScript(autobuilder, 0, 9); } [Fact] @@ -964,8 +937,6 @@ Microsoft.NETCore.App 2.1.4 [/usr/local/share/dotnet/shared/Microsoft.NETCore.Ap Actions.RunProcess[@"C:\Project/.dotnet/dotnet clean C:\Project/test.csproj"] = 0; Actions.RunProcess[@"C:\Project/.dotnet/dotnet restore C:\Project/test.csproj"] = 0; Actions.RunProcess[@"C:\odasa/tools/odasa index --auto C:\Project/.dotnet/dotnet build --no-incremental /p:UseSharedCompilation=false C:\Project/test.csproj"] = 0; - Actions.RunProcess[@"C:\codeql\tools\java/bin/java -jar C:\codeql\csharp/tools/extractor-asp.jar ."] = 0; - Actions.RunProcess[@"C:\odasa/tools/odasa index --xml --extensions config csproj props xml"] = 0; Actions.FileExists["csharp.log"] = true; Actions.FileExists["test.csproj"] = true; Actions.GetEnvironmentVariable["CODEQL_EXTRACTOR_CSHARP_TRAP_DIR"] = ""; @@ -985,7 +956,7 @@ Microsoft.NETCore.App 2.1.4 [/usr/local/share/dotnet/shared/Microsoft.NETCore.Ap Actions.DownloadFiles.Add(("https://dot.net/v1/dotnet-install.sh", "dotnet-install.sh")); var autobuilder = CreateAutoBuilder(false, dotnetVersion: "2.1.3"); - TestAutobuilderScript(autobuilder, 0, 11); + TestAutobuilderScript(autobuilder, 0, 9); } [Fact] @@ -999,8 +970,6 @@ Microsoft.NETCore.App 2.1.4 [/usr/local/share/dotnet/shared/Microsoft.NETCore.Ap Actions.RunProcess[@"cmd.exe /C C:\Project\.dotnet\dotnet clean C:\Project\test.csproj"] = 0; Actions.RunProcess[@"cmd.exe /C C:\Project\.dotnet\dotnet restore C:\Project\test.csproj"] = 0; Actions.RunProcess[@"cmd.exe /C C:\odasa\tools\odasa index --auto C:\Project\.dotnet\dotnet build --no-incremental C:\Project\test.csproj"] = 0; - Actions.RunProcess[@"cmd.exe /C C:\codeql\tools\java\bin\java -jar C:\codeql\csharp\tools\extractor-asp.jar ."] = 0; - Actions.RunProcess[@"cmd.exe /C C:\odasa\tools\odasa index --xml --extensions config csproj props xml"] = 0; Actions.FileExists["csharp.log"] = true; Actions.FileExists[@"C:\Project\test.csproj"] = true; Actions.GetEnvironmentVariable["CODEQL_EXTRACTOR_CSHARP_TRAP_DIR"] = ""; @@ -1019,7 +988,7 @@ Microsoft.NETCore.App 2.1.4 [/usr/local/share/dotnet/shared/Microsoft.NETCore.Ap Actions.LoadXml[@"C:\Project\test.csproj"] = xml; var autobuilder = CreateAutoBuilder(true, dotnetVersion: "2.1.3"); - TestAutobuilderScript(autobuilder, 0, 9); + TestAutobuilderScript(autobuilder, 0, 7); } [Fact] @@ -1028,8 +997,6 @@ Microsoft.NETCore.App 2.1.4 [/usr/local/share/dotnet/shared/Microsoft.NETCore.Ap Actions.RunProcess[@"cmd.exe /C nuget restore C:\Project\dirs.proj"] = 1; Actions.RunProcess[@"cmd.exe /C C:\Project\.nuget\nuget.exe restore C:\Project\dirs.proj"] = 0; Actions.RunProcess["cmd.exe /C CALL ^\"C:\\Program Files ^(x86^)\\Microsoft Visual Studio 12.0\\VC\\vcvarsall.bat^\" && set Platform=&& type NUL && C:\\odasa\\tools\\odasa index --auto msbuild C:\\Project\\dirs.proj /p:UseSharedCompilation=false /t:Windows /p:Platform=\"x86\" /p:Configuration=\"Debug\" /p:MvcBuildViews=true /P:Fu=Bar"] = 0; - Actions.RunProcess[@"cmd.exe /C C:\codeql\tools\java\bin\java -jar C:\codeql\csharp\tools\extractor-asp.jar ."] = 0; - Actions.RunProcess[@"cmd.exe /C C:\odasa\tools\odasa index --xml --extensions config csproj props xml"] = 0; Actions.FileExists["csharp.log"] = true; Actions.FileExists[@"C:\Project\a\test.csproj"] = true; Actions.FileExists[@"C:\Project\dirs.proj"] = true; @@ -1065,7 +1032,7 @@ Microsoft.NETCore.App 2.1.4 [/usr/local/share/dotnet/shared/Microsoft.NETCore.Ap var autobuilder = CreateAutoBuilder(true, msBuildArguments: "/P:Fu=Bar", msBuildTarget: "Windows", msBuildPlatform: "x86", msBuildConfiguration: "Debug", vsToolsVersion: "12", allSolutions: "true"); - TestAutobuilderScript(autobuilder, 0, 5); + TestAutobuilderScript(autobuilder, 0, 3); } [Fact] @@ -1074,8 +1041,6 @@ Microsoft.NETCore.App 2.1.4 [/usr/local/share/dotnet/shared/Microsoft.NETCore.Ap Actions.RunProcess[@"nuget restore C:\Project/dirs.proj"] = 1; Actions.RunProcess[@"mono C:\Project/.nuget/nuget.exe restore C:\Project/dirs.proj"] = 0; Actions.RunProcess[@"C:\odasa/tools/odasa index --auto msbuild C:\Project/dirs.proj /p:UseSharedCompilation=false /t:rebuild /p:MvcBuildViews=true"] = 0; - Actions.RunProcess[@"C:\codeql\tools\java/bin/java -jar C:\codeql\csharp/tools/extractor-asp.jar ."] = 0; - Actions.RunProcess[@"C:\odasa/tools/odasa index --xml --extensions config csproj props xml"] = 0; Actions.FileExists["csharp.log"] = true; Actions.FileExists[@"C:\Project/a/test.csproj"] = true; Actions.FileExists[@"C:\Project/dirs.proj"] = true; @@ -1104,7 +1069,7 @@ Microsoft.NETCore.App 2.1.4 [/usr/local/share/dotnet/shared/Microsoft.NETCore.Ap Actions.LoadXml[@"C:\Project/dirs.proj"] = dirsproj; var autobuilder = CreateAutoBuilder(false); - TestAutobuilderScript(autobuilder, 0, 5); + TestAutobuilderScript(autobuilder, 0, 3); } [Fact] diff --git a/csharp/autobuilder/Semmle.Autobuild.CSharp/AspBuildRule.cs b/csharp/autobuilder/Semmle.Autobuild.CSharp/AspBuildRule.cs deleted file mode 100644 index 2f69faeafde..00000000000 --- a/csharp/autobuilder/Semmle.Autobuild.CSharp/AspBuildRule.cs +++ /dev/null @@ -1,23 +0,0 @@ -using Semmle.Autobuild.Shared; - -namespace Semmle.Autobuild.CSharp -{ - /// - /// ASP extraction. - /// - class AspBuildRule : IBuildRule - { - public BuildScript Analyse(Autobuilder builder, bool auto) - { - var javaHome = builder.JavaHome; - var dist = builder.Distribution; - - var command = new CommandBuilder(builder.Actions). - RunCommand(builder.Actions.PathCombine(javaHome, "bin", "java")). - Argument("-jar"). - QuoteArgument(builder.Actions.PathCombine(dist, "tools", "extractor-asp.jar")). - Argument("."); - return command.Script; - } - } -} diff --git a/csharp/autobuilder/Semmle.Autobuild.CSharp/CSharpAutobuilder.cs b/csharp/autobuilder/Semmle.Autobuild.CSharp/CSharpAutobuilder.cs index 647a3ad2b4d..cf52b77b239 100644 --- a/csharp/autobuilder/Semmle.Autobuild.CSharp/CSharpAutobuilder.cs +++ b/csharp/autobuilder/Semmle.Autobuild.CSharp/CSharpAutobuilder.cs @@ -85,10 +85,7 @@ namespace Semmle.Autobuild.CSharp break; } - return - attempt & - (() => new AspBuildRule().Analyse(this, false)) & - (() => new XmlBuildRule().Analyse(this, false)); + return attempt; } /// diff --git a/csharp/autobuilder/Semmle.Autobuild.CSharp/XmlBuildRule.cs b/csharp/autobuilder/Semmle.Autobuild.CSharp/XmlBuildRule.cs deleted file mode 100644 index d262ec1f20b..00000000000 --- a/csharp/autobuilder/Semmle.Autobuild.CSharp/XmlBuildRule.cs +++ /dev/null @@ -1,21 +0,0 @@ -using Semmle.Autobuild.Shared; - -namespace Semmle.Autobuild.CSharp -{ - /// - /// XML extraction. - /// - class XmlBuildRule : IBuildRule - { - public BuildScript Analyse(Autobuilder builder, bool auto) - { - if (!builder.Options.Indexing || builder.Odasa is null) - return BuildScript.Success; - - var command = new CommandBuilder(builder.Actions). - RunCommand(builder.Odasa). - Argument("index --xml --extensions config csproj props xml"); - return command.Script; - } - } -} diff --git a/csharp/autobuilder/Semmle.Autobuild.Shared/AutobuildOptions.cs b/csharp/autobuilder/Semmle.Autobuild.Shared/AutobuildOptions.cs index 28c14be13ff..92bad4d6c7f 100644 --- a/csharp/autobuilder/Semmle.Autobuild.Shared/AutobuildOptions.cs +++ b/csharp/autobuilder/Semmle.Autobuild.Shared/AutobuildOptions.cs @@ -29,7 +29,6 @@ namespace Semmle.Autobuild.Shared public readonly bool NugetRestore; public readonly Language Language; - public readonly bool Indexing; /// /// Reads options from environment variables. @@ -54,8 +53,6 @@ namespace Semmle.Autobuild.Shared NugetRestore = actions.GetEnvironmentVariable(prefix + "NUGET_RESTORE").AsBool("nuget_restore", true); Language = language; - - Indexing = !actions.GetEnvironmentVariable($"CODEQL_AUTOBUILDER_{Language.UpperCaseName}_NO_INDEXING").AsBool("no_indexing", false); } } diff --git a/csharp/autobuilder/Semmle.Autobuild.Shared/Autobuilder.cs b/csharp/autobuilder/Semmle.Autobuild.Shared/Autobuilder.cs index 16ba41bfe34..1e8f58c45b7 100644 --- a/csharp/autobuilder/Semmle.Autobuild.Shared/Autobuilder.cs +++ b/csharp/autobuilder/Semmle.Autobuild.Shared/Autobuilder.cs @@ -1,4 +1,4 @@ -using Semmle.Util.Logging; +using Semmle.Util.Logging; using System; using System.Collections.Generic; using System.IO; @@ -190,21 +190,10 @@ namespace Semmle.Autobuild.Shared }); CodeQLExtractorLangRoot = Actions.GetEnvironmentVariable($"CODEQL_EXTRACTOR_{this.Options.Language.UpperCaseName}_ROOT"); - SemmleDist = Actions.GetEnvironmentVariable("SEMMLE_DIST"); SemmlePlatformTools = Actions.GetEnvironmentVariable("SEMMLE_PLATFORM_TOOLS"); CodeQlPlatform = Actions.GetEnvironmentVariable("CODEQL_PLATFORM"); - JavaHome = - Actions.GetEnvironmentVariable("CODEQL_JAVA_HOME") ?? - Actions.GetEnvironmentVariable("SEMMLE_JAVA_HOME") ?? - throw new InvalidEnvironmentException("The environment variable CODEQL_JAVA_HOME or SEMMLE_JAVA_HOME has not been set."); - - Distribution = - CodeQLExtractorLangRoot ?? - SemmleDist ?? - throw new InvalidEnvironmentException($"The environment variable CODEQL_EXTRACTOR_{this.Options.Language.UpperCaseName}_ROOT or SEMMLE_DIST has not been set."); - TrapDir = Actions.GetEnvironmentVariable($"CODEQL_EXTRACTOR_{this.Options.Language.UpperCaseName}_TRAP_DIR") ?? Actions.GetEnvironmentVariable("TRAP_FOLDER") ?? @@ -275,15 +264,6 @@ namespace Semmle.Autobuild.Shared /// public string? CodeQLExtractorLangRoot { get; } - /// - /// Value of SEMMLE_DIST environment variable. - /// - private string? SemmleDist { get; } - - public string Distribution { get; } - - public string JavaHome { get; } - /// /// Value of SEMMLE_PLATFORM_TOOLS environment variable. /// @@ -298,13 +278,20 @@ namespace Semmle.Autobuild.Shared /// The absolute path of the odasa executable. /// null if we are running in CodeQL. /// - public string? Odasa => SemmleDist is null ? null : Actions.PathCombine(SemmleDist, "tools", "odasa"); + public string? Odasa + { + get + { + var semmleDist = Actions.GetEnvironmentVariable("SEMMLE_DIST"); + return semmleDist is null ? null : Actions.PathCombine(semmleDist, "tools", "odasa"); + } + } /// /// Construct a command that executed the given wrapped in /// an odasa --index, unless indexing has been disabled, in which case /// is run directly. /// - public CommandBuilder MaybeIndex(CommandBuilder builder, string cmd) => Options.Indexing && !(Odasa is null) ? builder.IndexCommand(Odasa, cmd) : builder.RunCommand(cmd); + public CommandBuilder MaybeIndex(CommandBuilder builder, string cmd) => Odasa is null ? builder.RunCommand(cmd) : builder.IndexCommand(Odasa, cmd); } } diff --git a/csharp/extractor/Semmle.Extraction.CIL/Context.cs b/csharp/extractor/Semmle.Extraction.CIL/Context.cs index 618293553f0..e3fee31c57e 100644 --- a/csharp/extractor/Semmle.Extraction.CIL/Context.cs +++ b/csharp/extractor/Semmle.Extraction.CIL/Context.cs @@ -43,7 +43,7 @@ namespace Semmle.Extraction.CIL namespaceFactory = new CachedFunction(n => CreateNamespace(mdReader.GetString(n))); namespaceDefinitionFactory = new CachedFunction(CreateNamespace); sourceFiles = new CachedFunction(path => new Entities.PdbSourceFile(this, path)); - folders = new CachedFunction(path => new Entities.Folder(this, path)); + folders = new CachedFunction(path => new Entities.Folder(this, path)); sourceLocations = new CachedFunction(location => new Entities.PdbSourceLocation(this, location)); defaultGenericContext = new EmptyContext(this); diff --git a/csharp/extractor/Semmle.Extraction.CIL/Entities/Assembly.cs b/csharp/extractor/Semmle.Extraction.CIL/Entities/Assembly.cs index f633af96db7..a38575d2f61 100644 --- a/csharp/extractor/Semmle.Extraction.CIL/Entities/Assembly.cs +++ b/csharp/extractor/Semmle.Extraction.CIL/Entities/Assembly.cs @@ -5,6 +5,7 @@ using Semmle.Util.Logging; using System; using Semmle.Extraction.Entities; using System.IO; +using Semmle.Util; namespace Semmle.Extraction.CIL.Entities { @@ -134,9 +135,12 @@ namespace Semmle.Extraction.CIL.Entities extracted = false; try { - var extractor = new Extractor(false, assemblyPath, logger); - var project = layout.LookupProjectOrDefault(assemblyPath); - using (var trapWriter = project.CreateTrapWriter(logger, assemblyPath + ".cil", true, trapCompression)) + var canonicalPathCache = CanonicalPathCache.Create(logger, 1000); + var pathTransformer = new PathTransformer(canonicalPathCache); + var extractor = new Extractor(false, assemblyPath, logger, pathTransformer); + var transformedAssemblyPath = pathTransformer.Transform(assemblyPath); + var project = layout.LookupProjectOrDefault(transformedAssemblyPath); + using (var trapWriter = project.CreateTrapWriter(logger, transformedAssemblyPath.WithSuffix(".cil"), true, trapCompression)) { trapFile = trapWriter.TrapFile; if (nocache || !System.IO.File.Exists(trapFile)) diff --git a/csharp/extractor/Semmle.Extraction.CIL/Entities/File.cs b/csharp/extractor/Semmle.Extraction.CIL/Entities/File.cs index 2506ea6ee00..175af111615 100644 --- a/csharp/extractor/Semmle.Extraction.CIL/Entities/File.cs +++ b/csharp/extractor/Semmle.Extraction.CIL/Entities/File.cs @@ -1,4 +1,4 @@ -using System.Collections.Generic; +using System.Collections.Generic; using System.IO; namespace Semmle.Extraction.CIL.Entities @@ -13,37 +13,38 @@ namespace Semmle.Extraction.CIL.Entities public class File : LabelledEntity, IFile { - protected readonly string path; + protected readonly string OriginalPath; + protected readonly PathTransformer.ITransformedPath TransformedPath; public File(Context cx, string path) : base(cx) { - this.path = Semmle.Extraction.Entities.File.PathAsDatabaseString(path); + this.OriginalPath = path; + TransformedPath = cx.cx.Extractor.PathTransformer.Transform(OriginalPath); } public override void WriteId(TextWriter trapFile) { - trapFile.Write(Semmle.Extraction.Entities.File.PathAsDatabaseId(path)); + trapFile.Write(TransformedPath.DatabaseId); } public override bool Equals(object? obj) { - return GetType() == obj?.GetType() && path == ((File)obj).path; + return GetType() == obj?.GetType() && OriginalPath == ((File)obj).OriginalPath; } - public override int GetHashCode() => 11 * path.GetHashCode(); + public override int GetHashCode() => 11 * OriginalPath.GetHashCode(); public override IEnumerable Contents { get { - var directoryName = System.IO.Path.GetDirectoryName(path); - if (directoryName is null) - throw new InternalError($"Directory name for path '{path}' is null."); - - var parent = cx.CreateFolder(directoryName); - yield return parent; - yield return Tuples.containerparent(parent, this); - yield return Tuples.files(this, path, System.IO.Path.GetFileNameWithoutExtension(path), System.IO.Path.GetExtension(path).Substring(1)); + if (TransformedPath.ParentDirectory is PathTransformer.ITransformedPath dir) + { + var parent = cx.CreateFolder(dir); + yield return parent; + yield return Tuples.containerparent(parent, this); + } + yield return Tuples.files(this, TransformedPath.Value, TransformedPath.NameWithoutExtension, TransformedPath.Extension); } } @@ -69,9 +70,9 @@ namespace Semmle.Extraction.CIL.Entities var text = file.Contents; if (text == null) - cx.cx.Extractor.Logger.Log(Util.Logging.Severity.Warning, string.Format("PDB source file {0} could not be found", path)); + cx.cx.Extractor.Logger.Log(Util.Logging.Severity.Warning, string.Format("PDB source file {0} could not be found", OriginalPath)); else - cx.cx.TrapWriter.Archive(path, text); + cx.cx.TrapWriter.Archive(TransformedPath, text); yield return Tuples.file_extraction_mode(this, 2); } diff --git a/csharp/extractor/Semmle.Extraction.CIL/Entities/Folder.cs b/csharp/extractor/Semmle.Extraction.CIL/Entities/Folder.cs index b6597a3eba9..0769ac48106 100644 --- a/csharp/extractor/Semmle.Extraction.CIL/Entities/Folder.cs +++ b/csharp/extractor/Semmle.Extraction.CIL/Entities/Folder.cs @@ -9,16 +9,16 @@ namespace Semmle.Extraction.CIL.Entities public sealed class Folder : LabelledEntity, IFolder { - readonly string path; + readonly PathTransformer.ITransformedPath TransformedPath; - public Folder(Context cx, string path) : base(cx) + public Folder(Context cx, PathTransformer.ITransformedPath path) : base(cx) { - this.path = path; + this.TransformedPath = path; } public override void WriteId(TextWriter trapFile) { - trapFile.Write(Semmle.Extraction.Entities.File.PathAsDatabaseId(path)); + trapFile.Write(TransformedPath.DatabaseId); } public override string IdSuffix => ";folder"; @@ -27,25 +27,21 @@ namespace Semmle.Extraction.CIL.Entities { get { - // On Posix, we could get a Windows directory of the form "C:" - bool windowsDriveLetter = path.Length == 2 && char.IsLetter(path[0]) && path[1] == ':'; - - var parent = Path.GetDirectoryName(path); - if (parent != null && !windowsDriveLetter) + if (TransformedPath.ParentDirectory is PathTransformer.ITransformedPath parent) { var parentFolder = cx.CreateFolder(parent); yield return parentFolder; yield return Tuples.containerparent(parentFolder, this); } - yield return Tuples.folders(this, Semmle.Extraction.Entities.File.PathAsDatabaseString(path), Path.GetFileName(path)); + yield return Tuples.folders(this, TransformedPath.Value, TransformedPath.NameWithoutExtension); } } public override bool Equals(object? obj) { - return obj is Folder folder && path == folder.path; + return obj is Folder folder && TransformedPath == folder.TransformedPath; } - public override int GetHashCode() => path.GetHashCode(); + public override int GetHashCode() => TransformedPath.GetHashCode(); } } diff --git a/csharp/extractor/Semmle.Extraction.CIL/Factories.cs b/csharp/extractor/Semmle.Extraction.CIL/Factories.cs index f2f98b64d17..d43eaf780a9 100644 --- a/csharp/extractor/Semmle.Extraction.CIL/Factories.cs +++ b/csharp/extractor/Semmle.Extraction.CIL/Factories.cs @@ -201,7 +201,7 @@ namespace Semmle.Extraction.CIL #region Locations readonly CachedFunction sourceFiles; - readonly CachedFunction folders; + readonly CachedFunction folders; readonly CachedFunction sourceLocations; /// @@ -216,7 +216,7 @@ namespace Semmle.Extraction.CIL /// /// The path of the folder. /// A folder entity. - public Folder CreateFolder(string path) => folders[path]; + public Folder CreateFolder(PathTransformer.ITransformedPath path) => folders[path]; /// /// Creates a source location. diff --git a/csharp/extractor/Semmle.Extraction.CSharp/Analyser.cs b/csharp/extractor/Semmle.Extraction.CSharp/Analyser.cs index 34dece8e160..cc888326dfe 100644 --- a/csharp/extractor/Semmle.Extraction.CSharp/Analyser.cs +++ b/csharp/extractor/Semmle.Extraction.CSharp/Analyser.cs @@ -27,13 +27,16 @@ namespace Semmle.Extraction.CSharp public readonly bool AddAssemblyTrapPrefix; - public Analyser(IProgressMonitor pm, ILogger logger, bool addAssemblyTrapPrefix) + public readonly PathTransformer PathTransformer; + + public Analyser(IProgressMonitor pm, ILogger logger, bool addAssemblyTrapPrefix, PathTransformer pathTransformer) { Logger = logger; AddAssemblyTrapPrefix = addAssemblyTrapPrefix; Logger.Log(Severity.Info, "EXTRACTION STARTING at {0}", DateTime.Now); stopWatch.Start(); progressMonitor = pm; + PathTransformer = pathTransformer; } CSharpCompilation compilation; @@ -67,7 +70,7 @@ namespace Semmle.Extraction.CSharp layout = new Layout(); this.options = options; this.compilation = compilation; - extractor = new Extraction.Extractor(false, GetOutputName(compilation, commandLineArguments), Logger); + extractor = new Extraction.Extractor(false, GetOutputName(compilation, commandLineArguments), Logger, PathTransformer); LogDiagnostics(); SetReferencePaths(); @@ -117,7 +120,7 @@ namespace Semmle.Extraction.CSharp { compilation = compilationIn; layout = new Layout(); - extractor = new Extraction.Extractor(true, null, Logger); + extractor = new Extraction.Extractor(true, null, Logger, PathTransformer); this.options = options; LogExtractorInfo(Extraction.Extractor.Version); SetReferencePaths(); @@ -230,9 +233,10 @@ namespace Semmle.Extraction.CSharp try { var assemblyPath = extractor.OutputPath; + var transformedAssemblyPath = PathTransformer.Transform(assemblyPath); var assembly = compilation.Assembly; - var projectLayout = layout.LookupProjectOrDefault(assemblyPath); - var trapWriter = projectLayout.CreateTrapWriter(Logger, assemblyPath, true, options.TrapCompression); + var projectLayout = layout.LookupProjectOrDefault(transformedAssemblyPath); + var trapWriter = projectLayout.CreateTrapWriter(Logger, transformedAssemblyPath, true, options.TrapCompression); compilationTrapFile = trapWriter; // Dispose later var cx = extractor.CreateContext(compilation.Clone(), trapWriter, new AssemblyScope(assembly, assemblyPath, true), AddAssemblyTrapPrefix); @@ -260,8 +264,9 @@ namespace Semmle.Extraction.CSharp stopwatch.Start(); var assemblyPath = r.FilePath; - var projectLayout = layout.LookupProjectOrDefault(assemblyPath); - using (var trapWriter = projectLayout.CreateTrapWriter(Logger, assemblyPath, true, options.TrapCompression)) + var transformedAssemblyPath = PathTransformer.Transform(assemblyPath); + var projectLayout = layout.LookupProjectOrDefault(transformedAssemblyPath); + using (var trapWriter = projectLayout.CreateTrapWriter(Logger, transformedAssemblyPath, true, options.TrapCompression)) { var skipExtraction = options.Cache && File.Exists(trapWriter.TrapFile); @@ -360,16 +365,17 @@ namespace Semmle.Extraction.CSharp var stopwatch = new Stopwatch(); stopwatch.Start(); var sourcePath = tree.FilePath; + var transformedSourcePath = PathTransformer.Transform(sourcePath); - var projectLayout = layout.LookupProjectOrNull(sourcePath); + var projectLayout = layout.LookupProjectOrNull(transformedSourcePath); bool excluded = projectLayout == null; - string trapPath = excluded ? "" : projectLayout.GetTrapPath(Logger, sourcePath, options.TrapCompression); + string trapPath = excluded ? "" : projectLayout.GetTrapPath(Logger, transformedSourcePath, options.TrapCompression); bool upToDate = false; if (!excluded) { // compilation.Clone() is used to allow symbols to be garbage collected. - using (var trapWriter = projectLayout.CreateTrapWriter(Logger, sourcePath, false, options.TrapCompression)) + using (var trapWriter = projectLayout.CreateTrapWriter(Logger, transformedSourcePath, false, options.TrapCompression)) { upToDate = options.Fast && FileIsUpToDate(sourcePath, trapWriter.TrapFile); diff --git a/csharp/extractor/Semmle.Extraction.CSharp/Entities/Compilation.cs b/csharp/extractor/Semmle.Extraction.CSharp/Entities/Compilation.cs index 95c568047c5..9f96b03f9f3 100644 --- a/csharp/extractor/Semmle.Extraction.CSharp/Entities/Compilation.cs +++ b/csharp/extractor/Semmle.Extraction.CSharp/Entities/Compilation.cs @@ -3,6 +3,7 @@ using System; using System.Collections.Generic; using System.IO; using System.Linq; +using Semmle.Util; namespace Semmle.Extraction.CSharp.Entities { @@ -22,7 +23,7 @@ namespace Semmle.Extraction.CSharp.Entities { Extraction.Entities.Assembly.CreateOutputAssembly(cx); - trapFile.compilations(this, Extraction.Entities.File.PathAsDatabaseString(cwd)); + trapFile.compilations(this, FileUtils.ConvertToUnix(cwd)); // Arguments int index = 0; diff --git a/csharp/extractor/Semmle.Extraction.CSharp/Entities/Expressions/ImplicitCast.cs b/csharp/extractor/Semmle.Extraction.CSharp/Entities/Expressions/ImplicitCast.cs index d8d8c80cf75..299c3aa8f72 100644 --- a/csharp/extractor/Semmle.Extraction.CSharp/Entities/Expressions/ImplicitCast.cs +++ b/csharp/extractor/Semmle.Extraction.CSharp/Entities/Expressions/ImplicitCast.cs @@ -86,6 +86,15 @@ namespace Semmle.Extraction.CSharp.Entities.Expressions return new ImplicitCast(info); } + if (conversion.IsIdentity && conversion.IsImplicit && + convertedType.Symbol is IPointerTypeSymbol && + !(resolvedType.Symbol is IPointerTypeSymbol)) + { + // int[] -> int* + // string -> char* + return new ImplicitCast(info); + } + // Default: Just create the expression without a conversion. return Factory.Create(info); } diff --git a/csharp/extractor/Semmle.Extraction.CSharp/Entities/Expressions/Switch.cs b/csharp/extractor/Semmle.Extraction.CSharp/Entities/Expressions/Switch.cs index 5316998a38f..0dfd33751fc 100644 --- a/csharp/extractor/Semmle.Extraction.CSharp/Entities/Expressions/Switch.cs +++ b/csharp/extractor/Semmle.Extraction.CSharp/Entities/Expressions/Switch.cs @@ -18,10 +18,9 @@ namespace Semmle.Extraction.CSharp.Entities.Expressions protected override void PopulateExpression(TextWriter trapFile) { SwitchedExpr = Expression.Create(cx, Syntax.GoverningExpression, this, -1); - int child = 0; - foreach (var arm in Syntax.Arms) + for (var i = 0; i < Syntax.Arms.Count; i++) { - new SwitchCase(cx, arm, this, child++); + new SwitchCase(cx, Syntax.Arms[i], this, i); } } } @@ -29,7 +28,9 @@ namespace Semmle.Extraction.CSharp.Entities.Expressions class SwitchCase : Expression { internal SwitchCase(Context cx, SwitchExpressionArmSyntax arm, Switch parent, int child) : - base(new ExpressionInfo(cx, parent.SwitchedExpr.Type, cx.Create(arm.GetLocation()), ExprKind.SWITCH_CASE, parent, child, false, null)) + base(new ExpressionInfo( + cx, Entities.Type.Create(cx, cx.GetType(arm.Expression)), cx.Create(arm.GetLocation()), + ExprKind.SWITCH_CASE, parent, child, false, null)) { cx.CreatePattern(arm.Pattern, this, 0); if (arm.WhenClause is WhenClauseSyntax when) diff --git a/csharp/extractor/Semmle.Extraction.CSharp/Entities/Field.cs b/csharp/extractor/Semmle.Extraction.CSharp/Entities/Field.cs index 5f881338200..896614840ab 100644 --- a/csharp/extractor/Semmle.Extraction.CSharp/Entities/Field.cs +++ b/csharp/extractor/Semmle.Extraction.CSharp/Entities/Field.cs @@ -68,13 +68,12 @@ namespace Semmle.Extraction.CSharp.Entities Context.PopulateLater(() => { var loc = Context.Create(initializer.GetLocation()); - var simpleAssignExpr = new Expression(new ExpressionInfo(Context, Type, loc, ExprKind.SIMPLE_ASSIGN, this, child++, false, null)); - Expression.CreateFromNode(new ExpressionNodeInfo(Context, initializer.Initializer.Value, simpleAssignExpr, 0)); - var access = new Expression(new ExpressionInfo(Context, Type, Location, ExprKind.FIELD_ACCESS, simpleAssignExpr, 1, false, null)); - trapFile.expr_access(access, this); + + var fieldAccess = AddInitializerAssignment(trapFile, initializer.Initializer.Value, loc, null, ref child); + if (!symbol.IsStatic) { - This.CreateImplicit(Context, Entities.Type.Create(Context, symbol.ContainingType), Location, access, -1); + This.CreateImplicit(Context, Entities.Type.Create(Context, symbol.ContainingType), Location, fieldAccess, -1); } }); } @@ -85,8 +84,13 @@ namespace Semmle.Extraction.CSharp.Entities Where(n => n.EqualsValue != null)) { // Mark fields that have explicit initializers. - var expr = new Expression(new ExpressionInfo(Context, Type, Context.Create(initializer.EqualsValue.Value.FixedLocation()), Kinds.ExprKind.FIELD_ACCESS, this, child++, false, null)); - trapFile.expr_access(expr, this); + var constValue = symbol.HasConstantValue + ? Expression.ValueAsString(symbol.ConstantValue) + : null; + + var loc = Context.Create(initializer.GetLocation()); + + AddInitializerAssignment(trapFile, initializer.EqualsValue.Value, loc, constValue, ref child); } if (IsSourceDeclaration) @@ -96,6 +100,16 @@ namespace Semmle.Extraction.CSharp.Entities TypeMention.Create(Context, syntax.Type, this, Type); } + private Expression AddInitializerAssignment(TextWriter trapFile, ExpressionSyntax initializer, Extraction.Entities.Location loc, + string constValue, ref int child) + { + var simpleAssignExpr = new Expression(new ExpressionInfo(Context, Type, loc, ExprKind.SIMPLE_ASSIGN, this, child++, false, constValue)); + Expression.CreateFromNode(new ExpressionNodeInfo(Context, initializer, simpleAssignExpr, 0)); + var access = new Expression(new ExpressionInfo(Context, Type, Location, ExprKind.FIELD_ACCESS, simpleAssignExpr, 1, false, constValue)); + trapFile.expr_access(access, this); + return access; + } + readonly Lazy type; public AnnotatedType Type => type.Value; diff --git a/csharp/extractor/Semmle.Extraction.CSharp/Entities/LocalFunction.cs b/csharp/extractor/Semmle.Extraction.CSharp/Entities/LocalFunction.cs index 8b90f6905dd..edce7b66ce2 100644 --- a/csharp/extractor/Semmle.Extraction.CSharp/Entities/LocalFunction.cs +++ b/csharp/extractor/Semmle.Extraction.CSharp/Entities/LocalFunction.cs @@ -51,5 +51,7 @@ namespace Semmle.Extraction.CSharp.Entities trapFile.local_functions(this, symbol.Name, returnType, originalDefinition); ExtractRefReturn(trapFile); } + + public override TrapStackBehaviour TrapStackBehaviour => TrapStackBehaviour.NeedsLabel; } } diff --git a/csharp/extractor/Semmle.Extraction.CSharp/Extractor.cs b/csharp/extractor/Semmle.Extraction.CSharp/Extractor.cs index 1b5bf15d187..261ce7f9e43 100644 --- a/csharp/extractor/Semmle.Extraction.CSharp/Extractor.cs +++ b/csharp/extractor/Semmle.Extraction.CSharp/Extractor.cs @@ -76,16 +76,16 @@ namespace Semmle.Extraction.CSharp return ExitCode.Ok; } - using (var analyser = new Analyser(new LogProgressMonitor(logger), logger, commandLineArguments.AssemblySensitiveTrap)) + var canonicalPathCache = CanonicalPathCache.Create(logger, 1000); + var pathTransformer = new PathTransformer(canonicalPathCache); + + using (var analyser = new Analyser(new LogProgressMonitor(logger), logger, commandLineArguments.AssemblySensitiveTrap, pathTransformer)) using (var references = new BlockingCollection()) { try { var compilerVersion = new CompilerVersion(commandLineArguments); - bool preserveSymlinks = Environment.GetEnvironmentVariable("SEMMLE_PRESERVE_SYMLINKS") == "true"; - var canonicalPathCache = CanonicalPathCache.Create(logger, 1000, preserveSymlinks ? CanonicalPathCache.Symlinks.Preserve : CanonicalPathCache.Symlinks.Follow); - if (compilerVersion.SkipExtraction) { logger.Log(Severity.Warning, " Unrecognized compiler '{0}' because {1}", compilerVersion.SpecifiedCompiler, compilerVersion.SkipReason); @@ -318,7 +318,10 @@ namespace Semmle.Extraction.CSharp ILogger logger, CommonOptions options) { - using (var analyser = new Analyser(pm, logger, false)) + var canonicalPathCache = CanonicalPathCache.Create(logger, 1000); + var pathTransformer = new PathTransformer(canonicalPathCache); + + using (var analyser = new Analyser(pm, logger, false, pathTransformer)) using (var references = new BlockingCollection()) { try diff --git a/csharp/extractor/Semmle.Extraction.Tests/FilePattern.cs b/csharp/extractor/Semmle.Extraction.Tests/FilePattern.cs new file mode 100644 index 00000000000..dfff75ea18b --- /dev/null +++ b/csharp/extractor/Semmle.Extraction.Tests/FilePattern.cs @@ -0,0 +1,48 @@ +using Xunit; + +namespace Semmle.Extraction.Tests +{ + public class FilePatternTests + { + [Fact] + public void TestRegexCompilation() + { + var fp = new FilePattern("/hadoop*"); + Assert.Equal("^hadoop[^/]*.*", fp.RegexPattern); + fp = new FilePattern("**/org/apache/hadoop"); + Assert.Equal("^.*/org/apache/hadoop.*", fp.RegexPattern); + fp = new FilePattern("hadoop-common/**/test// "); + Assert.Equal("^hadoop-common/.*/test(?/).*", fp.RegexPattern); + fp = new FilePattern(@"-C:\agent\root\asdf//"); + Assert.Equal("^C:/agent/root/asdf(?/).*", fp.RegexPattern); + fp = new FilePattern(@"-C:\agent+\[root]\asdf//"); + Assert.Equal(@"^C:/agent\+/\[root]/asdf(?/).*", fp.RegexPattern); + } + + [Fact] + public void TestMatching() + { + var fp1 = new FilePattern(@"C:\agent\root\abc//"); + var fp2 = new FilePattern(@"C:\agent\root\def//ghi"); + var patterns = new[] { fp1, fp2 }; + + var success = FilePattern.Matches(patterns, @"C:\agent\root\abc\file.cs", out var s); + Assert.True(success); + Assert.Equal("/file.cs", s); + + success = FilePattern.Matches(patterns, @"C:\agent\root\def\ghi\file.cs", out s); + Assert.True(success); + Assert.Equal("/ghi/file.cs", s); + + success = FilePattern.Matches(patterns, @"C:\agent\root\def\file.cs", out s); + Assert.False(success); + } + + [Fact] + public void TestInvalidPatterns() + { + Assert.Throws(() => new FilePattern("/abc//def//ghi")); + Assert.Throws(() => new FilePattern("/abc**def")); + } + } +} diff --git a/csharp/extractor/Semmle.Extraction.Tests/Layout.cs b/csharp/extractor/Semmle.Extraction.Tests/Layout.cs index c66b804f3a7..a5f36f0c19d 100644 --- a/csharp/extractor/Semmle.Extraction.Tests/Layout.cs +++ b/csharp/extractor/Semmle.Extraction.Tests/Layout.cs @@ -1,10 +1,30 @@ -using System.IO; +using System.IO; using Xunit; using Semmle.Util.Logging; using System.Runtime.InteropServices; namespace Semmle.Extraction.Tests { + struct TransformedPathStub : PathTransformer.ITransformedPath + { + readonly string value; + public TransformedPathStub(string value) => this.value = value; + public string Value => value; + + public string Extension => throw new System.NotImplementedException(); + + public string NameWithoutExtension => throw new System.NotImplementedException(); + + public PathTransformer.ITransformedPath ParentDirectory => throw new System.NotImplementedException(); + + public string DatabaseId => throw new System.NotImplementedException(); + + public PathTransformer.ITransformedPath WithSuffix(string suffix) + { + throw new System.NotImplementedException(); + } + } + public class Layout { readonly ILogger Logger = new LoggerMock(); @@ -13,12 +33,12 @@ namespace Semmle.Extraction.Tests public void TestDefaultLayout() { var layout = new Semmle.Extraction.Layout(null, null, null); - var project = layout.LookupProjectOrNull("foo.cs"); + var project = layout.LookupProjectOrNull(new TransformedPathStub("foo.cs")); Assert.NotNull(project); // All files are mapped when there's no layout file. - Assert.True(layout.FileInLayout("foo.cs")); + Assert.True(layout.FileInLayout(new TransformedPathStub("foo.cs"))); // Test trap filename var tmpDir = Path.GetTempPath(); @@ -30,13 +50,13 @@ namespace Semmle.Extraction.Tests Assert.NotEqual(Directory.GetCurrentDirectory(), tmpDir); return; } - var f1 = project!.GetTrapPath(Logger, "foo.cs", TrapWriter.CompressionMode.Gzip); - var g1 = TrapWriter.NestPaths(Logger, tmpDir, "foo.cs.trap.gz", TrapWriter.InnerPathComputation.ABSOLUTE); + var f1 = project!.GetTrapPath(Logger, new TransformedPathStub("foo.cs"), TrapWriter.CompressionMode.Gzip); + var g1 = TrapWriter.NestPaths(Logger, tmpDir, "foo.cs.trap.gz"); Assert.Equal(f1, g1); // Test trap file generation - var trapwriterFilename = project.GetTrapPath(Logger, "foo.cs", TrapWriter.CompressionMode.Gzip); - using (var trapwriter = project.CreateTrapWriter(Logger, "foo.cs", false, TrapWriter.CompressionMode.Gzip)) + var trapwriterFilename = project.GetTrapPath(Logger, new TransformedPathStub("foo.cs"), TrapWriter.CompressionMode.Gzip); + using (var trapwriter = project.CreateTrapWriter(Logger, new TransformedPathStub("foo.cs"), false, TrapWriter.CompressionMode.Gzip)) { trapwriter.Emit("1=*"); Assert.False(File.Exists(trapwriterFilename)); @@ -65,25 +85,24 @@ namespace Semmle.Extraction.Tests var layout = new Semmle.Extraction.Layout(null, null, "layout.txt"); // Test general pattern matching - Assert.True(layout.FileInLayout("bar.cs")); - Assert.False(layout.FileInLayout("foo.cs")); - Assert.False(layout.FileInLayout("goo.cs")); - Assert.False(layout.FileInLayout("excluded/bar.cs")); - Assert.True(layout.FileInLayout("excluded/foo.cs")); - Assert.True(layout.FileInLayout("included/foo.cs")); + Assert.True(layout.FileInLayout(new TransformedPathStub("bar.cs"))); + Assert.False(layout.FileInLayout(new TransformedPathStub("foo.cs"))); + Assert.False(layout.FileInLayout(new TransformedPathStub("goo.cs"))); + Assert.False(layout.FileInLayout(new TransformedPathStub("excluded/bar.cs"))); + Assert.True(layout.FileInLayout(new TransformedPathStub("excluded/foo.cs"))); + Assert.True(layout.FileInLayout(new TransformedPathStub("included/foo.cs"))); // Test the trap file - var project = layout.LookupProjectOrNull("bar.cs"); + var project = layout.LookupProjectOrNull(new TransformedPathStub("bar.cs")); Assert.NotNull(project); - - var trapwriterFilename = project!.GetTrapPath(Logger, "bar.cs", TrapWriter.CompressionMode.Gzip); - Assert.Equal(TrapWriter.NestPaths(Logger, Path.GetFullPath("snapshot\\trap"), "bar.cs.trap.gz", TrapWriter.InnerPathComputation.ABSOLUTE), + var trapwriterFilename = project!.GetTrapPath(Logger, new TransformedPathStub("bar.cs"), TrapWriter.CompressionMode.Gzip); + Assert.Equal(TrapWriter.NestPaths(Logger, Path.GetFullPath("snapshot\\trap"), "bar.cs.trap.gz"), trapwriterFilename); // Test the source archive - var trapWriter = project.CreateTrapWriter(Logger, "bar.cs", false, TrapWriter.CompressionMode.Gzip); - trapWriter.Archive("layout.txt", System.Text.Encoding.ASCII); - var writtenFile = TrapWriter.NestPaths(Logger, Path.GetFullPath("snapshot\\archive"), "layout.txt", TrapWriter.InnerPathComputation.ABSOLUTE); + var trapWriter = project.CreateTrapWriter(Logger, new TransformedPathStub("bar.cs"), false, TrapWriter.CompressionMode.Gzip); + trapWriter.Archive("layout.txt", new TransformedPathStub("layout.txt"), System.Text.Encoding.ASCII); + var writtenFile = TrapWriter.NestPaths(Logger, Path.GetFullPath("snapshot\\archive"), "layout.txt"); Assert.True(File.Exists(writtenFile)); File.Delete("layout.txt"); } @@ -93,11 +112,11 @@ namespace Semmle.Extraction.Tests { // When you specify both a trap file and a layout, use the trap file. var layout = new Semmle.Extraction.Layout(Path.GetFullPath("snapshot\\trap"), null, "something.txt"); - Assert.True(layout.FileInLayout("bar.cs")); - var subProject = layout.LookupProjectOrNull("foo.cs"); + Assert.True(layout.FileInLayout(new TransformedPathStub("bar.cs"))); + var subProject = layout.LookupProjectOrNull(new TransformedPathStub("foo.cs")); Assert.NotNull(subProject); - var f1 = subProject!.GetTrapPath(Logger, "foo.cs", TrapWriter.CompressionMode.Gzip); - var g1 = TrapWriter.NestPaths(Logger, Path.GetFullPath("snapshot\\trap"), "foo.cs.trap.gz", TrapWriter.InnerPathComputation.ABSOLUTE); + var f1 = subProject!.GetTrapPath(Logger, new TransformedPathStub("foo.cs"), TrapWriter.CompressionMode.Gzip); + var g1 = TrapWriter.NestPaths(Logger, Path.GetFullPath("snapshot\\trap"), "foo.cs.trap.gz"); Assert.Equal(f1, g1); } @@ -123,30 +142,30 @@ namespace Semmle.Extraction.Tests var layout = new Semmle.Extraction.Layout(null, null, "layout.txt"); // Use Section 2 - Assert.True(layout.FileInLayout("bar.cs")); - var subProject = layout.LookupProjectOrNull("bar.cs"); + Assert.True(layout.FileInLayout(new TransformedPathStub("bar.cs"))); + var subProject = layout.LookupProjectOrNull(new TransformedPathStub("bar.cs")); Assert.NotNull(subProject); - var f1 = subProject!.GetTrapPath(Logger, "bar.cs", TrapWriter.CompressionMode.Gzip); - var g1 = TrapWriter.NestPaths(Logger, Path.GetFullPath("snapshot\\trap2"), "bar.cs.trap.gz", TrapWriter.InnerPathComputation.ABSOLUTE); + var f1 = subProject!.GetTrapPath(Logger, new TransformedPathStub("bar.cs"), TrapWriter.CompressionMode.Gzip); + var g1 = TrapWriter.NestPaths(Logger, Path.GetFullPath("snapshot\\trap2"), "bar.cs.trap.gz"); Assert.Equal(f1, g1); // Use Section 1 - Assert.True(layout.FileInLayout("foo.cs")); - subProject = layout.LookupProjectOrNull("foo.cs"); + Assert.True(layout.FileInLayout(new TransformedPathStub("foo.cs"))); + subProject = layout.LookupProjectOrNull(new TransformedPathStub("foo.cs")); Assert.NotNull(subProject); - var f2 = subProject!.GetTrapPath(Logger, "foo.cs", TrapWriter.CompressionMode.Gzip); - var g2 = TrapWriter.NestPaths(Logger, Path.GetFullPath("snapshot\\trap1"), "foo.cs.trap.gz", TrapWriter.InnerPathComputation.ABSOLUTE); + var f2 = subProject!.GetTrapPath(Logger, new TransformedPathStub("foo.cs"), TrapWriter.CompressionMode.Gzip); + var g2 = TrapWriter.NestPaths(Logger, Path.GetFullPath("snapshot\\trap1"), "foo.cs.trap.gz"); Assert.Equal(f2, g2); // boo.dll is not in the layout, so use layout from first section. - Assert.False(layout.FileInLayout("boo.dll")); - var f3 = layout.LookupProjectOrDefault("boo.dll").GetTrapPath(Logger, "boo.dll", TrapWriter.CompressionMode.Gzip); - var g3 = TrapWriter.NestPaths(Logger, Path.GetFullPath("snapshot\\trap1"), "boo.dll.trap.gz", TrapWriter.InnerPathComputation.ABSOLUTE); + Assert.False(layout.FileInLayout(new TransformedPathStub("boo.dll"))); + var f3 = layout.LookupProjectOrDefault(new TransformedPathStub("boo.dll")).GetTrapPath(Logger, new TransformedPathStub("boo.dll"), TrapWriter.CompressionMode.Gzip); + var g3 = TrapWriter.NestPaths(Logger, Path.GetFullPath("snapshot\\trap1"), "boo.dll.trap.gz"); Assert.Equal(f3, g3); // boo.cs is not in the layout, so return null - Assert.False(layout.FileInLayout("boo.cs")); - Assert.Null(layout.LookupProjectOrNull("boo.cs")); + Assert.False(layout.FileInLayout(new TransformedPathStub("boo.cs"))); + Assert.Null(layout.LookupProjectOrNull(new TransformedPathStub("boo.cs"))); } [Fact] diff --git a/csharp/extractor/Semmle.Extraction.Tests/PathTransformer.cs b/csharp/extractor/Semmle.Extraction.Tests/PathTransformer.cs new file mode 100644 index 00000000000..b0f0ba8c51f --- /dev/null +++ b/csharp/extractor/Semmle.Extraction.Tests/PathTransformer.cs @@ -0,0 +1,45 @@ +using Semmle.Util; +using Xunit; + +namespace Semmle.Extraction.Tests +{ + class PathCacheStub : IPathCache + { + public string GetCanonicalPath(string path) => path; + } + + public class PathTransformerTests + { + [Fact] + public void TestTransformerFile() + { + var spec = new string[] + { + @"#D:\src", + @"C:\agent*\src//", + @"-C:\agent*\src\external", + @"", + @"#empty", + @"", + @"#src2", + @"/agent*//src", + @"", + @"#optsrc", + @"opt/src//" + }; + + var pathTransformer = new PathTransformer(new PathCacheStub(), spec); + + // Windows-style matching + Assert.Equal(@"C:/bar.cs", pathTransformer.Transform(@"C:\bar.cs").Value); + Assert.Equal("D:/src/file.cs", pathTransformer.Transform(@"C:\agent42\src\file.cs").Value); + Assert.Equal("D:/src/file.cs", pathTransformer.Transform(@"C:\agent43\src\file.cs").Value); + Assert.Equal(@"C:/agent43/src/external/file.cs", pathTransformer.Transform(@"C:\agent43\src\external\file.cs").Value); + + // Linux-style matching + Assert.Equal(@"src2/src/file.cs", pathTransformer.Transform(@"/agent/src/file.cs").Value); + Assert.Equal(@"src2/src/file.cs", pathTransformer.Transform(@"/agent42/src/file.cs").Value); + Assert.Equal(@"optsrc/file.cs", pathTransformer.Transform(@"/opt/src/file.cs").Value); + } + } +} diff --git a/csharp/extractor/Semmle.Extraction.Tests/TrapWriter.cs b/csharp/extractor/Semmle.Extraction.Tests/TrapWriter.cs index fd7f77f427b..54da865689b 100644 --- a/csharp/extractor/Semmle.Extraction.Tests/TrapWriter.cs +++ b/csharp/extractor/Semmle.Extraction.Tests/TrapWriter.cs @@ -14,7 +14,7 @@ namespace Semmle.Extraction.Tests string tempDir = System.IO.Path.GetTempPath(); string root1, root2, root3; - if(Win32.IsWindows()) + if (Win32.IsWindows()) { root1 = "E:"; root2 = "e:"; @@ -27,32 +27,21 @@ namespace Semmle.Extraction.Tests root3 = "/"; } - string formattedTempDir = tempDir.Replace('/', '\\').Replace(':', '_').Trim('\\'); - var logger = new LoggerMock(); - System.IO.Directory.SetCurrentDirectory(tempDir); - if (RuntimeInformation.IsOSPlatform(OSPlatform.OSX)) - { - // `Directory.SetCurrentDirectory()` doesn't seem to work on macOS, - // so disable this test on macOS, for now - Assert.NotEqual(Directory.GetCurrentDirectory(), tempDir); - return; - } + Assert.Equal($@"C:\Temp\source_archive\def.cs", TrapWriter.NestPaths(logger, @"C:\Temp\source_archive", "def.cs").Replace('/', '\\')); - Assert.Equal($@"C:\Temp\source_archive\{formattedTempDir}\def.cs", TrapWriter.NestPaths(logger, @"C:\Temp\source_archive", "def.cs", TrapWriter.InnerPathComputation.ABSOLUTE).Replace('/','\\')); + Assert.Equal(@"C:\Temp\source_archive\def.cs", TrapWriter.NestPaths(logger, @"C:\Temp\source_archive", "def.cs").Replace('/', '\\')); - Assert.Equal(@"C:\Temp\source_archive\def.cs", TrapWriter.NestPaths(logger, @"C:\Temp\source_archive", "def.cs", TrapWriter.InnerPathComputation.RELATIVE).Replace('/', '\\')); + Assert.Equal(@"C:\Temp\source_archive\E_\source\def.cs", TrapWriter.NestPaths(logger, @"C:\Temp\source_archive", $@"{root1}\source\def.cs").Replace('/', '\\')); - Assert.Equal(@"C:\Temp\source_archive\E_\source\def.cs", TrapWriter.NestPaths(logger, @"C:\Temp\source_archive", $@"{root1}\source\def.cs", TrapWriter.InnerPathComputation.ABSOLUTE).Replace('/', '\\')); + Assert.Equal(@"C:\Temp\source_archive\e_\source\def.cs", TrapWriter.NestPaths(logger, @"C:\Temp\source_archive", $@"{root2}\source\def.cs").Replace('/', '\\')); - Assert.Equal(@"C:\Temp\source_archive\e_\source\def.cs", TrapWriter.NestPaths(logger, @"C:\Temp\source_archive", $@"{root2}\source\def.cs", TrapWriter.InnerPathComputation.RELATIVE).Replace('/', '\\')); + Assert.Equal(@"C:\Temp\source_archive\source\def.cs", TrapWriter.NestPaths(logger, @"C:\Temp\source_archive", $@"{root3}source\def.cs").Replace('/', '\\')); - Assert.Equal(@"C:\Temp\source_archive\source\def.cs", TrapWriter.NestPaths(logger, @"C:\Temp\source_archive", $@"{root3}source\def.cs", TrapWriter.InnerPathComputation.ABSOLUTE).Replace('/', '\\')); + Assert.Equal(@"C:\Temp\source_archive\source\def.cs", TrapWriter.NestPaths(logger, @"C:\Temp\source_archive", $@"{root3}source\def.cs").Replace('/', '\\')); - Assert.Equal(@"C:\Temp\source_archive\source\def.cs", TrapWriter.NestPaths(logger, @"C:\Temp\source_archive", $@"{root3}source\def.cs", TrapWriter.InnerPathComputation.RELATIVE).Replace('/', '\\')); - - Assert.Equal(@"C:\Temp\source_archive\diskstation\share\source\def.cs", TrapWriter.NestPaths(logger, @"C:\Temp\source_archive", $@"{root3}{root3}diskstation\share\source\def.cs", TrapWriter.InnerPathComputation.ABSOLUTE).Replace('/', '\\')); + Assert.Equal(@"C:\Temp\source_archive\diskstation\share\source\def.cs", TrapWriter.NestPaths(logger, @"C:\Temp\source_archive", $@"{root3}{root3}diskstation\share\source\def.cs").Replace('/', '\\')); } class LoggerMock : ILogger diff --git a/csharp/extractor/Semmle.Extraction/Entities/File.cs b/csharp/extractor/Semmle.Extraction/Entities/File.cs index eba6fab83b7..5b3f79acdd3 100644 --- a/csharp/extractor/Semmle.Extraction/Entities/File.cs +++ b/csharp/extractor/Semmle.Extraction/Entities/File.cs @@ -10,93 +10,55 @@ namespace Semmle.Extraction.Entities File(Context cx, string path) : base(cx, path) { - Path = path; + OriginalPath = path; + TransformedPathLazy = new Lazy(() => Context.Extractor.PathTransformer.Transform(OriginalPath)); } - public string Path - { - get; - private set; - } + readonly string OriginalPath; + readonly Lazy TransformedPathLazy; + PathTransformer.ITransformedPath TransformedPath => TransformedPathLazy.Value; - public string DatabasePath => PathAsDatabaseId(Path); - - public override bool NeedsPopulation => Context.DefinesFile(Path) || Path == Context.Extractor.OutputPath; + public override bool NeedsPopulation => Context.DefinesFile(OriginalPath) || OriginalPath == Context.Extractor.OutputPath; public override void Populate(TextWriter trapFile) { - if (Path == null) + trapFile.files(this, TransformedPath.Value, TransformedPath.NameWithoutExtension, TransformedPath.Extension); + + if (TransformedPath.ParentDirectory is PathTransformer.ITransformedPath dir) + trapFile.containerparent(Folder.Create(Context, dir), this); + + var fromSource = TransformedPath.Extension.ToLowerInvariant().Equals("cs"); + if (fromSource) { - trapFile.files(this, "", "", ""); - } - else - { - var fi = new FileInfo(Path); - - string extension = fi.Extension ?? ""; - string name = fi.Name; - name = name.Substring(0, name.Length - extension.Length); - int fromSource = extension.ToLowerInvariant().Equals(".cs") ? 1 : 2; - - // remove the dot from the extension - if (extension.Length > 0) - extension = extension.Substring(1); - trapFile.files(this, PathAsDatabaseString(Path), name, extension); - - trapFile.containerparent(Folder.Create(Context, fi.Directory), this); - if (fromSource == 1) + foreach (var text in Context.Compilation.SyntaxTrees. + Where(t => t.FilePath == OriginalPath). + Select(tree => tree.GetText())) { - foreach (var text in Context.Compilation.SyntaxTrees. - Where(t => t.FilePath == Path). - Select(tree => tree.GetText())) - { - var rawText = text.ToString() ?? ""; - var lineCounts = LineCounter.ComputeLineCounts(rawText); - if (rawText.Length > 0 && rawText[rawText.Length - 1] != '\n') lineCounts.Total++; + var rawText = text.ToString() ?? ""; + var lineCounts = LineCounter.ComputeLineCounts(rawText); + if (rawText.Length > 0 && rawText[rawText.Length - 1] != '\n') lineCounts.Total++; - trapFile.numlines(this, lineCounts); - Context.TrapWriter.Archive(fi.FullName, text.Encoding ?? System.Text.Encoding.Default); - } + trapFile.numlines(this, lineCounts); + Context.TrapWriter.Archive(OriginalPath, TransformedPath, text.Encoding ?? System.Text.Encoding.Default); } - - trapFile.file_extraction_mode(this, Context.Extractor.Standalone ? 1 : 0); } + + trapFile.file_extraction_mode(this, Context.Extractor.Standalone ? 1 : 0); } public override void WriteId(System.IO.TextWriter trapFile) { - if (Path is null) - trapFile.Write("GENERATED;sourcefile"); - else - { - trapFile.Write(DatabasePath); - trapFile.Write(";sourcefile"); - } + trapFile.Write(TransformedPath.DatabaseId); + trapFile.Write(";sourcefile"); } - /// - /// Converts a path string into a string to use as an ID - /// in the QL database. - /// - /// An absolute path. - /// The database ID. - public static string PathAsDatabaseId(string path) - { - if (path.Length >= 2 && path[1] == ':' && Char.IsLower(path[0])) - path = Char.ToUpper(path[0]) + "_" + path.Substring(2); - return path.Replace('\\', '/').Replace(":", "_"); - } - - public static string PathAsDatabaseString(string path) => path.Replace('\\', '/'); - public static File Create(Context cx, string path) => FileFactory.Instance.CreateEntity(cx, (typeof(File), path), path); public static File CreateGenerated(Context cx) => GeneratedFile.Create(cx); class GeneratedFile : File { - GeneratedFile(Context cx) - : base(cx, "") { } + GeneratedFile(Context cx) : base(cx, "") { } public override bool NeedsPopulation => true; diff --git a/csharp/extractor/Semmle.Extraction/Entities/Folder.cs b/csharp/extractor/Semmle.Extraction/Entities/Folder.cs index 86653e19244..05d552f0352 100644 --- a/csharp/extractor/Semmle.Extraction/Entities/Folder.cs +++ b/csharp/extractor/Semmle.Extraction/Entities/Folder.cs @@ -2,65 +2,44 @@ using System.IO; namespace Semmle.Extraction.Entities { - sealed class Folder : CachedEntity + sealed class Folder : CachedEntity { - Folder(Context cx, DirectoryInfo init) - : base(cx, init) - { - Path = init.FullName; - } - - public string Path - { - get; - private set; - } - - public string DatabasePath => File.PathAsDatabaseId(Path); + Folder(Context cx, PathTransformer.ITransformedPath init) : base(cx, init) { } public override void Populate(TextWriter trapFile) { - // Ensure that the name of the root directory is consistent - // with the XmlTrapWriter. - // Linux/Windows: java.io.File.getName() returns "" - // On Linux: System.IO.DirectoryInfo.Name returns "/" - // On Windows: System.IO.DirectoryInfo.Name returns "L:\" - string shortName = symbol.Parent == null ? "" : symbol.Name; - - trapFile.folders(this, File.PathAsDatabaseString(Path), shortName); - if (symbol.Parent != null) - { - trapFile.containerparent(Create(Context, symbol.Parent), this); - } + trapFile.folders(this, symbol.Value, symbol.NameWithoutExtension); + if (symbol.ParentDirectory is PathTransformer.ITransformedPath parent) + trapFile.containerparent(Create(Context, parent), this); } public override bool NeedsPopulation => true; public override void WriteId(System.IO.TextWriter trapFile) { - trapFile.Write(DatabasePath); + trapFile.Write(symbol.DatabaseId); trapFile.Write(";folder"); } - public static Folder Create(Context cx, DirectoryInfo folder) => + public static Folder Create(Context cx, PathTransformer.ITransformedPath folder) => FolderFactory.Instance.CreateEntity(cx, folder, folder); public override Microsoft.CodeAnalysis.Location? ReportingLocation => null; - class FolderFactory : ICachedEntityFactory + class FolderFactory : ICachedEntityFactory { public static readonly FolderFactory Instance = new FolderFactory(); - public Folder Create(Context cx, DirectoryInfo init) => new Folder(cx, init); + public Folder Create(Context cx, PathTransformer.ITransformedPath init) => new Folder(cx, init); } public override TrapStackBehaviour TrapStackBehaviour => TrapStackBehaviour.NoLabel; - public override int GetHashCode() => Path.GetHashCode(); + public override int GetHashCode() => symbol.GetHashCode(); public override bool Equals(object? obj) { - return obj is Folder folder && folder.Path == Path; + return obj is Folder folder && Equals(folder.symbol, symbol); } } } diff --git a/csharp/extractor/Semmle.Extraction/Extractor.cs b/csharp/extractor/Semmle.Extraction/Extractor.cs index c08dd726abf..02dfc3bb11a 100644 --- a/csharp/extractor/Semmle.Extraction/Extractor.cs +++ b/csharp/extractor/Semmle.Extraction/Extractor.cs @@ -81,6 +81,11 @@ namespace Semmle.Extraction /// ILogger Logger { get; } + /// + /// The path transformer to apply. + /// + PathTransformer PathTransformer { get; } + /// /// Creates a new context. /// @@ -112,11 +117,14 @@ namespace Semmle.Extraction /// /// If the extraction is standalone. /// The name of the output DLL/EXE, or null if not specified (standalone extraction). - public Extractor(bool standalone, string outputPath, ILogger logger) + /// The object used for logging. + /// The object used for path transformations. + public Extractor(bool standalone, string outputPath, ILogger logger, PathTransformer pathTransformer) { Standalone = standalone; OutputPath = outputPath; Logger = logger; + PathTransformer = pathTransformer; } // Limit the number of error messages in the log file @@ -206,5 +214,7 @@ namespace Semmle.Extraction public ILogger Logger { get; private set; } public static string Version => $"{ThisAssembly.Git.BaseTag} ({ThisAssembly.Git.Sha})"; + + public PathTransformer PathTransformer { get; } } } diff --git a/csharp/extractor/Semmle.Extraction/FilePattern.cs b/csharp/extractor/Semmle.Extraction/FilePattern.cs new file mode 100644 index 00000000000..2d61badd2f2 --- /dev/null +++ b/csharp/extractor/Semmle.Extraction/FilePattern.cs @@ -0,0 +1,131 @@ +using System; +using System.Collections.Generic; +using System.Linq; +using System.Text; +using System.Text.RegularExpressions; +using System.Diagnostics.CodeAnalysis; +using Semmle.Util; + +namespace Semmle.Extraction +{ + public sealed class InvalidFilePatternException : Exception + { + public InvalidFilePatternException(string pattern, string message) : + base($"Invalid file pattern '{pattern}': {message}") + { } + } + + /// + /// A file pattern, as used in either an extractor layout file or + /// a path transformer file. + /// + public sealed class FilePattern + { + /// + /// Whether this is an inclusion pattern. + /// + public bool Include { get; } + + public FilePattern(string pattern) + { + Include = true; + if (pattern.StartsWith("-")) + { + pattern = pattern.Substring(1); + Include = false; + } + pattern = FileUtils.ConvertToUnix(pattern.Trim()).TrimStart('/'); + RegexPattern = BuildRegex(pattern).ToString(); + } + + /// + /// Constructs a regex string from a file pattern. Throws + /// `InvalidFilePatternException` for invalid patterns. + /// + static StringBuilder BuildRegex(string pattern) + { + bool HasCharAt(int i, Predicate p) => + i >= 0 && i < pattern.Length && p(pattern[i]); + var sb = new StringBuilder(); + var i = 0; + var seenDoubleSlash = false; + sb.Append('^'); + while (i < pattern.Length) + { + if (pattern[i] == '/') + { + if (HasCharAt(i + 1, c => c == '/')) + { + if (seenDoubleSlash) + throw new InvalidFilePatternException(pattern, "'//' is allowed at most once."); + sb.Append("(?/)"); + i += 2; + seenDoubleSlash = true; + } + else + { + sb.Append('/'); + i++; + } + } + else if (pattern[i] == '*') + { + if (HasCharAt(i + 1, c => c == '*')) + { + if (HasCharAt(i - 1, c => c != '/')) + throw new InvalidFilePatternException(pattern, "'**' preceeded by non-`/` character."); + if (HasCharAt(i + 2, c => c != '/')) + throw new InvalidFilePatternException(pattern, "'**' succeeded by non-`/` character"); + sb.Append(".*"); + i += 2; + } + else + { + sb.Append("[^/]*"); + i++; + } + } + else + sb.Append(Regex.Escape(pattern[i++].ToString())); + } + return sb.Append(".*"); + } + + + /// + /// The regex pattern compiled from this file pattern. + /// + public string RegexPattern { get; } + + /// + /// Returns `true` if the set of file patterns `patterns` match the path `path`. + /// If so, `transformerSuffix` will contain the part of `path` that needs to be + /// suffixed when using path transformers. + /// + public static bool Matches(IEnumerable patterns, string path, [NotNullWhen(true)] out string? transformerSuffix) + { + path = FileUtils.ConvertToUnix(path).TrimStart('/'); + + foreach (var pattern in patterns.Reverse()) + { + var m = new Regex(pattern.RegexPattern).Match(path); + if (m.Success) + { + if (pattern.Include) + { + transformerSuffix = m.Groups.TryGetValue("doubleslash", out var group) + ? path.Substring(group.Index) + : path; + return true; + } + + transformerSuffix = null; + return false; + } + } + + transformerSuffix = null; + return false; + } + } +} \ No newline at end of file diff --git a/csharp/extractor/Semmle.Extraction/Layout.cs b/csharp/extractor/Semmle.Extraction/Layout.cs index 82b266318c1..d740d2c05b9 100644 --- a/csharp/extractor/Semmle.Extraction/Layout.cs +++ b/csharp/extractor/Semmle.Extraction/Layout.cs @@ -54,14 +54,15 @@ namespace Semmle.Extraction /// /// The source file. /// The full filepath of the trap file. - public string GetTrapPath(ILogger logger, string srcFile, TrapWriter.CompressionMode trapCompression) => TrapWriter.TrapPath(logger, TRAP_FOLDER, srcFile, trapCompression); + public string GetTrapPath(ILogger logger, PathTransformer.ITransformedPath srcFile, TrapWriter.CompressionMode trapCompression) => + TrapWriter.TrapPath(logger, TRAP_FOLDER, srcFile, trapCompression); /// /// Creates a trap writer for a given source/assembly file. /// /// The source file. /// A newly created TrapWriter. - public TrapWriter CreateTrapWriter(ILogger logger, string srcFile, bool discardDuplicates, TrapWriter.CompressionMode trapCompression) => + public TrapWriter CreateTrapWriter(ILogger logger, PathTransformer.ITransformedPath srcFile, bool discardDuplicates, TrapWriter.CompressionMode trapCompression) => new TrapWriter(logger, srcFile, TRAP_FOLDER, SOURCE_ARCHIVE, discardDuplicates, trapCompression); } @@ -73,7 +74,7 @@ namespace Semmle.Extraction /// /// The file to look up. /// The relevant subproject, or null if not found. - public SubProject? LookupProjectOrNull(string sourceFile) + public SubProject? LookupProjectOrNull(PathTransformer.ITransformedPath sourceFile) { if (!useLayoutFile) return DefaultProject; @@ -89,7 +90,7 @@ namespace Semmle.Extraction /// /// The file to look up. /// The relevant subproject, or DefaultProject if not found. - public SubProject LookupProjectOrDefault(string sourceFile) + public SubProject LookupProjectOrDefault(PathTransformer.ITransformedPath sourceFile) { return LookupProjectOrNull(sourceFile) ?? DefaultProject; } @@ -134,7 +135,7 @@ namespace Semmle.Extraction /// /// The absolute path of the file to query. /// True iff there is no layout file or the layout file specifies the file. - public bool FileInLayout(string path) => LookupProjectOrNull(path) != null; + public bool FileInLayout(PathTransformer.ITransformedPath path) => LookupProjectOrNull(path) != null; void ReadLayoutFile(string layout) { @@ -167,33 +168,7 @@ namespace Semmle.Extraction sealed class LayoutBlock { - struct Condition - { - private readonly bool include; - private readonly string prefix; - - public bool Include => include; - - public string Prefix => prefix; - - public Condition(string line) - { - include = false; - if (line.StartsWith("-")) - line = line.Substring(1); - else - include = true; - prefix = Normalise(line.Trim()); - } - - static public string Normalise(string path) - { - path = Path.GetFullPath(path); - return path.Replace('\\', '/'); - } - } - - private readonly List conditions = new List(); + private readonly List filePatterns = new List(); public readonly Layout.SubProject Directories; @@ -219,22 +194,10 @@ namespace Semmle.Extraction ReadVariable("ODASA_BUILD_ERROR_DIR", lines[i++]); while (i < lines.Length && !lines[i].StartsWith("#")) { - conditions.Add(new Condition(lines[i++])); + filePatterns.Add(new FilePattern(lines[i++])); } } - public bool Matches(string path) - { - bool matches = false; - path = Condition.Normalise(path); - foreach (Condition condition in conditions) - { - if (condition.Include) - matches |= path.StartsWith(condition.Prefix); - else - matches &= !path.StartsWith(condition.Prefix); - } - return matches; - } + public bool Matches(PathTransformer.ITransformedPath path) => FilePattern.Matches(filePatterns, path.Value, out var _); } } diff --git a/csharp/extractor/Semmle.Extraction/PathTransformer.cs b/csharp/extractor/Semmle.Extraction/PathTransformer.cs new file mode 100644 index 00000000000..2c9770e790e --- /dev/null +++ b/csharp/extractor/Semmle.Extraction/PathTransformer.cs @@ -0,0 +1,177 @@ +using System; +using System.Collections.Generic; +using System.IO; +using System.Diagnostics.CodeAnalysis; +using Semmle.Util; + +namespace Semmle.Extraction +{ + /// + /// A class for interpreting path transformers specified using the environment + /// variable `CODEQL_PATH_TRANSFORMER`. + /// + public sealed class PathTransformer + { + public class InvalidPathTransformerException : Exception + { + public InvalidPathTransformerException(string message) : + base($"Invalid path transformer specification: {message}") + { } + } + + /// + /// A transformed path. + /// + public interface ITransformedPath + { + string Value { get; } + + string Extension { get; } + + string NameWithoutExtension { get; } + + ITransformedPath? ParentDirectory { get; } + + ITransformedPath WithSuffix(string suffix); + + string DatabaseId { get; } + } + + struct TransformedPath : ITransformedPath + { + public TransformedPath(string value) { this.value = value; } + readonly string value; + + public string Value => value; + + public string Extension => Path.GetExtension(value)?.Substring(1) ?? ""; + + public string NameWithoutExtension => Path.GetFileNameWithoutExtension(value); + + public ITransformedPath? ParentDirectory + { + get + { + var dir = Path.GetDirectoryName(value); + if (dir is null) + return null; + var isWindowsDriveLetter = dir.Length == 2 && char.IsLetter(dir[0]) && dir[1] == ':'; + if (isWindowsDriveLetter) + return null; + return new TransformedPath(FileUtils.ConvertToUnix(dir)); + } + } + + public ITransformedPath WithSuffix(string suffix) => new TransformedPath(value + suffix); + + public string DatabaseId + { + get + { + var ret = value; + if (ret.Length >= 2 && ret[1] == ':' && Char.IsLower(ret[0])) + ret = Char.ToUpper(ret[0]) + "_" + ret.Substring(2); + return ret.Replace('\\', '/').Replace(":", "_"); + } + } + + public override int GetHashCode() => 11 * value.GetHashCode(); + + public override bool Equals(object? obj) => obj is TransformedPath tp && tp.value == value; + + public override string ToString() => value; + } + + readonly Func transform; + + /// + /// Returns the path obtained by transforming `path`. + /// + public ITransformedPath Transform(string path) => new TransformedPath(transform(path)); + + /// + /// Default constructor reads parameters from the environment. + /// + public PathTransformer(IPathCache pathCache) : + this(pathCache, Environment.GetEnvironmentVariable("CODEQL_PATH_TRANSFORMER") is string file ? File.ReadAllLines(file) : null) + { + } + + /// + /// Creates a path transformer based on the specification in `lines`. + /// Throws `InvalidPathTransformerException` for invalid specifications. + /// + public PathTransformer(IPathCache pathCache, string[]? lines) + { + if (lines is null) + { + transform = path => FileUtils.ConvertToUnix(pathCache.GetCanonicalPath(path)); + return; + } + + var sections = ParsePathTransformerSpec(lines); + transform = path => + { + path = FileUtils.ConvertToUnix(pathCache.GetCanonicalPath(path)); + foreach (var section in sections) + { + if (section.Matches(path, out var transformed)) + return transformed; + } + return path; + }; + } + + static IEnumerable ParsePathTransformerSpec(string[] lines) + { + var sections = new List(); + try + { + int i = 0; + while (i < lines.Length && !lines[i].StartsWith("#")) + i++; + while (i < lines.Length) + { + var section = new TransformerSection(lines, ref i); + sections.Add(section); + } + + if (sections.Count == 0) + throw new InvalidPathTransformerException("contains no sections."); + } + catch (InvalidFilePatternException ex) + { + throw new InvalidPathTransformerException(ex.Message); + } + return sections; + } + } + + sealed class TransformerSection + { + readonly string name; + readonly List filePatterns = new List(); + + public TransformerSection(string[] lines, ref int i) + { + name = lines[i++].Substring(1); // skip the '#' + for (; i < lines.Length && !lines[i].StartsWith("#"); i++) + { + var line = lines[i]; + if (!string.IsNullOrWhiteSpace(line)) + filePatterns.Add(new FilePattern(line)); + } + } + + public bool Matches(string path, [NotNullWhen(true)] out string? transformed) + { + if (FilePattern.Matches(filePatterns, path, out var suffix)) + { + transformed = FileUtils.ConvertToUnix(name) + suffix; + return true; + } + transformed = null; + return false; + } + } +} diff --git a/csharp/extractor/Semmle.Extraction/TrapWriter.cs b/csharp/extractor/Semmle.Extraction/TrapWriter.cs index 7ea08eafc1c..8082567c825 100644 --- a/csharp/extractor/Semmle.Extraction/TrapWriter.cs +++ b/csharp/extractor/Semmle.Extraction/TrapWriter.cs @@ -14,12 +14,6 @@ namespace Semmle.Extraction public sealed class TrapWriter : IDisposable { - public enum InnerPathComputation - { - ABSOLUTE, - RELATIVE - } - public enum CompressionMode { None, @@ -45,7 +39,7 @@ namespace Semmle.Extraction readonly CompressionMode TrapCompression; - public TrapWriter(ILogger logger, string outputfile, string? trap, string? archive, bool discardDuplicates, CompressionMode trapCompression) + public TrapWriter(ILogger logger, PathTransformer.ITransformedPath outputfile, string? trap, string? archive, bool discardDuplicates, CompressionMode trapCompression) { Logger = logger; TrapCompression = trapCompression; @@ -107,16 +101,17 @@ namespace Semmle.Extraction /// Adds the specified input file to the source archive. It may end up in either the normal or long path area /// of the source archive, depending on the length of its full path. /// - /// The path to the input file. + /// The path to the input file. + /// The transformed path to the input file. /// The encoding used by the input file. - public void Archive(string inputPath, Encoding inputEncoding) + public void Archive(string originalPath, PathTransformer.ITransformedPath transformedPath, Encoding inputEncoding) { if (string.IsNullOrEmpty(archive)) return; // Calling GetFullPath makes this use the canonical capitalisation, if the file exists. - string fullInputPath = Path.GetFullPath(inputPath); + string fullInputPath = Path.GetFullPath(originalPath); - ArchivePath(fullInputPath, inputEncoding); + ArchivePath(fullInputPath, transformedPath, inputEncoding); } /// @@ -124,14 +119,11 @@ namespace Semmle.Extraction /// /// The path of the file. /// The contents of the file. - public void Archive(string inputPath, string contents) + public void Archive(PathTransformer.ITransformedPath inputPath, string contents) { if (string.IsNullOrEmpty(archive)) return; - // Calling GetFullPath makes this use the canonical capitalisation, if the file exists. - string fullInputPath = Path.GetFullPath(inputPath); - - ArchiveContents(fullInputPath, contents); + ArchiveContents(inputPath, contents); } /// @@ -210,18 +202,19 @@ namespace Semmle.Extraction /// source archive less than the system path limit of 260 characters. /// /// The full path to the input file. + /// The transformed path to the input file. /// The encoding used by the input file. /// If the output path in the source archive would /// exceed the system path limit of 260 characters. - private void ArchivePath(string fullInputPath, Encoding inputEncoding) + private void ArchivePath(string fullInputPath, PathTransformer.ITransformedPath transformedPath, Encoding inputEncoding) { string contents = File.ReadAllText(fullInputPath, inputEncoding); - ArchiveContents(fullInputPath, contents); + ArchiveContents(transformedPath, contents); } - private void ArchiveContents(string fullInputPath, string contents) + private void ArchiveContents(PathTransformer.ITransformedPath transformedPath, string contents) { - string dest = NestPaths(Logger, archive, fullInputPath, InnerPathComputation.ABSOLUTE); + string dest = NestPaths(Logger, archive, transformedPath.Value); string tmpSrcFile = Path.GetTempFileName(); File.WriteAllText(tmpSrcFile, contents, UTF8); try @@ -236,14 +229,11 @@ namespace Semmle.Extraction } } - public static string NestPaths(ILogger logger, string? outerpath, string innerpath, InnerPathComputation innerPathComputation) + public static string NestPaths(ILogger logger, string? outerpath, string innerpath) { string nested = innerpath; if (!string.IsNullOrEmpty(outerpath)) { - if (!Path.IsPathRooted(innerpath) && innerPathComputation == InnerPathComputation.ABSOLUTE) - innerpath = Path.GetFullPath(innerpath); - // Remove all leading path separators / or \ // For example, UNC paths have two leading \\ innerpath = innerpath.TrimStart(Path.DirectorySeparatorChar, Path.AltDirectorySeparatorChar); @@ -276,13 +266,13 @@ namespace Semmle.Extraction } } - public static string TrapPath(ILogger logger, string? folder, string filename, TrapWriter.CompressionMode trapCompression) + public static string TrapPath(ILogger logger, string? folder, PathTransformer.ITransformedPath path, TrapWriter.CompressionMode trapCompression) { - filename = $"{Path.GetFullPath(filename)}.trap{TrapExtension(trapCompression)}"; + var filename = $"{path.Value}.trap{TrapExtension(trapCompression)}"; if (string.IsNullOrEmpty(folder)) folder = Directory.GetCurrentDirectory(); - return NestPaths(logger, folder, filename, InnerPathComputation.ABSOLUTE); ; + return NestPaths(logger, folder, filename); } } } diff --git a/csharp/extractor/Semmle.Util/CanonicalPathCache.cs b/csharp/extractor/Semmle.Util/CanonicalPathCache.cs index bbc8ab995b4..339641ecb35 100644 --- a/csharp/extractor/Semmle.Util/CanonicalPathCache.cs +++ b/csharp/extractor/Semmle.Util/CanonicalPathCache.cs @@ -222,6 +222,29 @@ namespace Semmle.Util this.pathStrategy = pathStrategy; } + + /// + /// Create a CanonicalPathCache. + /// + /// + /// + /// Creates the appropriate PathStrategy object which encapsulates + /// the correct algorithm. Falls back to different implementations + /// depending on platform. + /// + /// + /// Size of the cache. + /// Policy for following symlinks. + /// A new CanonicalPathCache. + public static CanonicalPathCache Create(ILogger logger, int maxCapacity) + { + var preserveSymlinks = + Environment.GetEnvironmentVariable("CODEQL_PRESERVE_SYMLINKS") == "true" || + Environment.GetEnvironmentVariable("SEMMLE_PRESERVE_SYMLINKS") == "true"; + return Create(logger, maxCapacity, preserveSymlinks ? CanonicalPathCache.Symlinks.Preserve : CanonicalPathCache.Symlinks.Follow); + + } + /// /// Create a CanonicalPathCache. /// diff --git a/csharp/ql/src/Likely Bugs/SelfAssignment.ql b/csharp/ql/src/Likely Bugs/SelfAssignment.ql index 0f0e23e8629..01c5d904cd4 100644 --- a/csharp/ql/src/Likely Bugs/SelfAssignment.ql +++ b/csharp/ql/src/Likely Bugs/SelfAssignment.ql @@ -20,7 +20,9 @@ class StructuralComparisonConfig extends StructuralComparisonConfiguration { exists(AssignExpr ae | // Member initializers are never self-assignments, in particular // not initializers such as `new C { F = F };` - not ae instanceof MemberInitializer + not ae instanceof MemberInitializer and + // Enum field initializers are never self assignments. `enum E { A = 42 }` + not ae.getParent().(Field).getDeclaringType() instanceof Enum | ae.getLValue() = x and ae.getRValue() = y diff --git a/csharp/ql/src/Linq/Helpers.qll b/csharp/ql/src/Linq/Helpers.qll index 41d1e305ca6..e1b1beb1222 100644 --- a/csharp/ql/src/Linq/Helpers.qll +++ b/csharp/ql/src/Linq/Helpers.qll @@ -132,6 +132,17 @@ class AnyCall extends MethodCall { } } +/** A LINQ Count(...) call. */ +class CountCall extends MethodCall { + CountCall() { + exists(Method m | + m = getTarget() and + isEnumerableType(m.getDeclaringType()) and + m.hasName("Count") + ) + } +} + /** A variable of type IEnumerable<T>, for some T. */ class IEnumerableSequence extends Variable { IEnumerableSequence() { isIEnumerableType(getType()) } diff --git a/csharp/ql/src/Security Features/CWE-022/TaintedPath.qhelp b/csharp/ql/src/Security Features/CWE-022/TaintedPath.qhelp index fec61771992..e838d8c56a4 100644 --- a/csharp/ql/src/Security Features/CWE-022/TaintedPath.qhelp +++ b/csharp/ql/src/Security Features/CWE-022/TaintedPath.qhelp @@ -41,7 +41,7 @@ sent back to the user, giving them access to all the system's passwords.

  • OWASP: -Path Traversal. +Path Traversal.
    • diff --git a/csharp/ql/src/Security Features/CWE-022/ZipSlip.qhelp b/csharp/ql/src/Security Features/CWE-022/ZipSlip.qhelp index 9ee9ce6d0a1..bee5d819836 100644 --- a/csharp/ql/src/Security Features/CWE-022/ZipSlip.qhelp +++ b/csharp/ql/src/Security Features/CWE-022/ZipSlip.qhelp @@ -71,7 +71,7 @@ Snyk:
  • OWASP: -Path Traversal. +Path Traversal.
    • diff --git a/csharp/ql/src/Security Features/CWE-643/XPathInjection.qhelp b/csharp/ql/src/Security Features/CWE-643/XPathInjection.qhelp index 7241f6f97c4..cf50b4ee174 100644 --- a/csharp/ql/src/Security Features/CWE-643/XPathInjection.qhelp +++ b/csharp/ql/src/Security Features/CWE-643/XPathInjection.qhelp @@ -42,7 +42,7 @@ variables in an XsltArgumentList. -
  • OWASP: Testing for XPath Injection.
    • +
  • OWASP: Testing for XPath Injection.
  • OWASP: XPath Injection.
  • MSDN: User Defined Functions and Variables.
    •  diff --git a/csharp/ql/src/Security Features/InsufficientKeySize.cs b/csharp/ql/src/Security Features/InsufficientKeySize.cs index 5a12d01c1a1..9d12299dfb0 100644 --- a/csharp/ql/src/Security Features/InsufficientKeySize.cs +++ b/csharp/ql/src/Security Features/InsufficientKeySize.cs @@ -11,7 +11,7 @@ namespace InsufficientKeySize { try { - RSACryptoServiceProvider rsa = new RSACryptoServiceProvider(512); // BAD + RSACryptoServiceProvider rsa = new RSACryptoServiceProvider(1024); // BAD rsa.ImportParameters(key); return rsa.Encrypt(plaintext, true); } @@ -27,7 +27,7 @@ namespace InsufficientKeySize try { RSACryptoServiceProvider rsa = new RSACryptoServiceProvider(); // BAD - rsa = new RSACryptoServiceProvider(1024); // GOOD + rsa = new RSACryptoServiceProvider(2048); // GOOD rsa.ImportParameters(key); return rsa.Encrypt(plaintext, true); } @@ -58,7 +58,7 @@ namespace InsufficientKeySize try { DSACryptoServiceProvider dsa = new DSACryptoServiceProvider(); // BAD - dsa = new DSACryptoServiceProvider(1024); // GOOD + dsa = new DSACryptoServiceProvider(2048); // GOOD dsa.ImportParameters(key); return dsa.SignData(plaintext); } @@ -121,7 +121,7 @@ namespace InsufficientKeySize try { // Create a new instance of DSACryptoServiceProvider. - using (DSACryptoServiceProvider DSA = new DSACryptoServiceProvider(1024)) // GOOD + using (DSACryptoServiceProvider DSA = new DSACryptoServiceProvider(2048)) // GOOD { // Import the key information. DSA.ImportParameters(DSAKeyInfo); diff --git a/csharp/ql/src/Security Features/InsufficientKeySize.qhelp b/csharp/ql/src/Security Features/InsufficientKeySize.qhelp index 2b9ee39c610..906881cf0c2 100644 --- a/csharp/ql/src/Security Features/InsufficientKeySize.qhelp +++ b/csharp/ql/src/Security Features/InsufficientKeySize.qhelp @@ -8,7 +8,7 @@ are vulnerable to brute force attack when too small a key size is used.

    -

    The key should be at least 1024-bit long when using RSA encryption, and 128-bit long when using +

    The key should be at least 2048-bit long when using RSA encryption, and 128-bit long when using symmetric encryption.

     diff --git a/csharp/ql/src/Security Features/InsufficientKeySize.ql b/csharp/ql/src/Security Features/InsufficientKeySize.ql index 08bdcfd6724..04623b1d4b0 100644 --- a/csharp/ql/src/Security Features/InsufficientKeySize.ql +++ b/csharp/ql/src/Security Features/InsufficientKeySize.ql @@ -29,8 +29,8 @@ predicate incorrectUseOfDSA(ObjectCreation e, string msg) { .getTarget() .getDeclaringType() .hasQualifiedName("System.Security.Cryptography", "DSACryptoServiceProvider") and - exists(Expr i | e.getArgument(0) = i and i.getValue().toInt() < 1024) and - msg = "Key size should be at least 1024 bits for DSA encryption." + exists(Expr i | e.getArgument(0) = i and i.getValue().toInt() < 2048) and + msg = "Key size should be at least 2048 bits for DSA encryption." } predicate incorrectUseOfRSA(ObjectCreation e, string msg) { @@ -38,8 +38,8 @@ predicate incorrectUseOfRSA(ObjectCreation e, string msg) { .getTarget() .getDeclaringType() .hasQualifiedName("System.Security.Cryptography", "RSACryptoServiceProvider") and - exists(Expr i | e.getArgument(0) = i and i.getValue().toInt() < 1024) and - msg = "Key size should be at least 1024 bits for RSA encryption." + exists(Expr i | e.getArgument(0) = i and i.getValue().toInt() < 2048) and + msg = "Key size should be at least 2048 bits for RSA encryption." } from Expr e, string msg diff --git a/csharp/ql/src/experimental/CWE-099/TaintedWebClient.qhelp b/csharp/ql/src/experimental/CWE-099/TaintedWebClient.qhelp index 77721307eda..d7f195905a8 100644 --- a/csharp/ql/src/experimental/CWE-099/TaintedWebClient.qhelp +++ b/csharp/ql/src/experimental/CWE-099/TaintedWebClient.qhelp @@ -51,7 +51,7 @@ system's passwords.

  • OWASP: -Path Traversal. +Path Traversal.
    • diff --git a/csharp/ql/src/experimental/ir/implementation/raw/Operand.qll b/csharp/ql/src/experimental/ir/implementation/raw/Operand.qll index 14f62da51cd..e476aec60af 100644 --- a/csharp/ql/src/experimental/ir/implementation/raw/Operand.qll +++ b/csharp/ql/src/experimental/ir/implementation/raw/Operand.qll @@ -79,7 +79,8 @@ private PhiOperandBase phiOperand( } /** - * A source operand of an `Instruction`. The operand represents a value consumed by the instruction. + * An operand of an `Instruction`. The operand represents a use of the result of one instruction + * (the defining instruction) in another instruction (the use instruction) */ class Operand extends TOperand { /** Gets a textual representation of this element. */ diff --git a/csharp/ql/src/experimental/ir/implementation/unaliased_ssa/Operand.qll b/csharp/ql/src/experimental/ir/implementation/unaliased_ssa/Operand.qll index 14f62da51cd..e476aec60af 100644 --- a/csharp/ql/src/experimental/ir/implementation/unaliased_ssa/Operand.qll +++ b/csharp/ql/src/experimental/ir/implementation/unaliased_ssa/Operand.qll @@ -79,7 +79,8 @@ private PhiOperandBase phiOperand( } /** - * A source operand of an `Instruction`. The operand represents a value consumed by the instruction. + * An operand of an `Instruction`. The operand represents a use of the result of one instruction + * (the defining instruction) in another instruction (the use instruction) */ class Operand extends TOperand { /** Gets a textual representation of this element. */ diff --git a/csharp/ql/src/experimental/ir/implementation/unaliased_ssa/internal/SimpleSSA.qll b/csharp/ql/src/experimental/ir/implementation/unaliased_ssa/internal/SimpleSSA.qll index 3de8f5259f8..a7b9160bdc7 100644 --- a/csharp/ql/src/experimental/ir/implementation/unaliased_ssa/internal/SimpleSSA.qll +++ b/csharp/ql/src/experimental/ir/implementation/unaliased_ssa/internal/SimpleSSA.qll @@ -59,6 +59,12 @@ class MemoryLocation extends TMemoryLocation { final string getUniqueId() { result = var.getUniqueId() } } +/** + * Represents a set of `MemoryLocation`s that cannot overlap with + * `MemoryLocation`s outside of the set. The `VirtualVariable` will be + * represented by a `MemoryLocation` that totally overlaps all other + * `MemoryLocations` in the set. + */ class VirtualVariable extends MemoryLocation { } /** A virtual variable that groups all escaped memory within a function. */ diff --git a/csharp/ql/src/experimental/ir/internal/Overlap.qll b/csharp/ql/src/experimental/ir/internal/Overlap.qll index 8ce0549b2b4..f9a0c574f8c 100644 --- a/csharp/ql/src/experimental/ir/internal/Overlap.qll +++ b/csharp/ql/src/experimental/ir/internal/Overlap.qll @@ -3,18 +3,33 @@ private newtype TOverlap = TMustTotallyOverlap() or TMustExactlyOverlap() +/** + * Represents a possible overlap between two memory ranges. + */ abstract class Overlap extends TOverlap { abstract string toString(); } +/** + * Represents a partial overlap between two memory ranges, which may or may not + * actually occur in practice. + */ class MayPartiallyOverlap extends Overlap, TMayPartiallyOverlap { final override string toString() { result = "MayPartiallyOverlap" } } +/** + * Represents an overlap in which the first memory range is known to include all + * bits of the second memory range, but may be larger or have a different type. + */ class MustTotallyOverlap extends Overlap, TMustTotallyOverlap { final override string toString() { result = "MustTotallyOverlap" } } +/** + * Represents an overlap between two memory ranges that have the same extent and + * the same type. + */ class MustExactlyOverlap extends Overlap, TMustExactlyOverlap { final override string toString() { result = "MustExactlyOverlap" } } diff --git a/csharp/ql/src/external/examples/filters/BumpMetricBy10.ql b/csharp/ql/src/external/examples/filters/BumpMetricBy10.ql deleted file mode 100644 index 209d580bc19..00000000000 --- a/csharp/ql/src/external/examples/filters/BumpMetricBy10.ql +++ /dev/null @@ -1,11 +0,0 @@ -/** - * @name Edit the value of a metric - * @description Add 10 to a metric's value - * @deprecated - */ - -import csharp -import external.MetricFilter - -from MetricResult res -select res, res.getValue() + 10 diff --git a/csharp/ql/src/external/examples/filters/EditDefectMessage.ql b/csharp/ql/src/external/examples/filters/EditDefectMessage.ql deleted file mode 100644 index 4fbf4146550..00000000000 --- a/csharp/ql/src/external/examples/filters/EditDefectMessage.ql +++ /dev/null @@ -1,11 +0,0 @@ -/** - * @name Edit the message of a query - * @description Change the string in the select to edit the message - * @deprecated - */ - -import csharp -import external.DefectFilter - -from DefectResult res -select res, "Filtered query result: " + res.getMessage() diff --git a/csharp/ql/src/external/examples/filters/ExcludeGeneratedCode.ql b/csharp/ql/src/external/examples/filters/ExcludeGeneratedCode.ql deleted file mode 100644 index 2a1257427b2..00000000000 --- a/csharp/ql/src/external/examples/filters/ExcludeGeneratedCode.ql +++ /dev/null @@ -1,14 +0,0 @@ -/** - * @name Filter: removed results from generated code - * @description Shows how to exclude certain files or folders from results. - * @deprecated - */ - -import csharp -import external.DefectFilter - -predicate generatedFile(File f) { f.getAbsolutePath().matches("%generated%") } - -from DefectResult res -where not generatedFile(res.getFile()) -select res, res.getMessage() diff --git a/csharp/ql/src/external/examples/filters/FromSource.ql b/csharp/ql/src/external/examples/filters/FromSource.ql deleted file mode 100644 index 2f714ec2225..00000000000 --- a/csharp/ql/src/external/examples/filters/FromSource.ql +++ /dev/null @@ -1,12 +0,0 @@ -/** - * @name Filter: only keep results from source - * @description Shows how to filter for only certain files - * @deprecated - */ - -import csharp -import external.DefectFilter - -from DefectResult res -where res.getFile().fromSource() -select res, res.getMessage() diff --git a/csharp/ql/src/external/tests/DefectFromExternalData.ql b/csharp/ql/src/external/tests/DefectFromExternalData.ql deleted file mode 100644 index 70557f357a3..00000000000 --- a/csharp/ql/src/external/tests/DefectFromExternalData.ql +++ /dev/null @@ -1,19 +0,0 @@ -/** - * @name Defect from external data - * @description Insert description here... - * @kind problem - * @problem.severity warning - * @deprecated - */ - -import csharp -import external.ExternalArtifact - -// custom://[FileUtil][2011-01-02][false][1.1][6][Message 2] -from ExternalData d, File u -where - d.getQueryPath() = "external-data.ql" and - u.getStem() = d.getField(0) -select u, - d.getField(5) + ", " + d.getFieldAsDate(1) + ", " + d.getField(2) + ", " + d.getFieldAsFloat(3) + - ", " + d.getFieldAsInt(4) + ": " + d.getNumFields() diff --git a/csharp/ql/src/external/tests/DefectFromExternalDefect.ql b/csharp/ql/src/external/tests/DefectFromExternalDefect.ql deleted file mode 100644 index 57138a6879f..00000000000 --- a/csharp/ql/src/external/tests/DefectFromExternalDefect.ql +++ /dev/null @@ -1,17 +0,0 @@ -/** - * @name Defect from external defect - * @description Create a defect from external data - * @kind problem - * @problem.severity warning - * @deprecated - */ - -import csharp -import external.ExternalArtifact - -class DuplicateCode extends ExternalDefect { - DuplicateCode() { getQueryPath() = "duplicate-code/duplicateCode.ql" } -} - -from DuplicateCode d -select d, "External Defect " + d.getMessage() diff --git a/csharp/ql/src/external/tests/DefectFromExternalMetric.ql b/csharp/ql/src/external/tests/DefectFromExternalMetric.ql deleted file mode 100644 index 88d38d5f00b..00000000000 --- a/csharp/ql/src/external/tests/DefectFromExternalMetric.ql +++ /dev/null @@ -1,17 +0,0 @@ -/** - * @name Defect from external metric - * @description Create a defect from external data - * @kind problem - * @problem.severity warning - * @deprecated - */ - -import csharp -import external.ExternalArtifact - -from ExternalMetric m, File f -where - m.getQueryPath() = "filesBuilt.ql" and - m.getValue() = 1.0 and - m.getFile() = f -select f, "File is built" diff --git a/csharp/ql/src/external/tests/MetricFilter.ql b/csharp/ql/src/external/tests/MetricFilter.ql deleted file mode 100644 index 9f79465be75..00000000000 --- a/csharp/ql/src/external/tests/MetricFilter.ql +++ /dev/null @@ -1,13 +0,0 @@ -/** - * @name Metric filter - * @description Only include results in large files (200) lines of code. - * @kind treemap - * @deprecated - */ - -import csharp -import external.MetricFilter - -from MetricResult res -where res.getFile().getNumberOfLinesOfCode() > 200 -select res, res.getValue() diff --git a/csharp/ql/src/external/tests/MetricFromExternalDefect.ql b/csharp/ql/src/external/tests/MetricFromExternalDefect.ql deleted file mode 100644 index 4894de28761..00000000000 --- a/csharp/ql/src/external/tests/MetricFromExternalDefect.ql +++ /dev/null @@ -1,21 +0,0 @@ -/** - * @name Metric from external defect - * @description Find number of duplicate code entries in a file - * @treemap.warnOn lowValues - * @metricType file - * @kind treemap - * @deprecated - */ - -import csharp -import external.ExternalArtifact - -class DuplicateCode extends ExternalDefect { - DuplicateCode() { getQueryPath() = "duplicate-code/duplicateCode.ql" } -} - -predicate numDuplicateEntries(File f, int i) { i = count(DuplicateCode d | d.getFile() = f) } - -from File f, int i -where numDuplicateEntries(f, i) -select f, i diff --git a/csharp/ql/src/external/tests/MetricFromExternalMetric.ql b/csharp/ql/src/external/tests/MetricFromExternalMetric.ql deleted file mode 100644 index 4d2ab4088ae..00000000000 --- a/csharp/ql/src/external/tests/MetricFromExternalMetric.ql +++ /dev/null @@ -1,27 +0,0 @@ -/** - * @name Metric from external metric - * @description Each file in a folder gets as metric value the number of files built in that folder - * @treemap.warnOn lowValues - * @metricType file - * @kind treemap - * @deprecated - */ - -import csharp -import external.ExternalArtifact - -predicate numBuiltFiles(Folder fold, int i) { - i = - count(File f | - exists(ExternalMetric m | - m.getQueryPath() = "filesBuilt.ql" and - m.getValue() = 1.0 and - m.getFile() = f - ) and - f.getParentContainer() = fold - ) -} - -from File f, int i -where numBuiltFiles(f.getParentContainer(), i) -select f, i diff --git a/csharp/ql/src/semmle/code/csharp/commons/ComparisonTest.qll b/csharp/ql/src/semmle/code/csharp/commons/ComparisonTest.qll index 5d083679f9d..ea7d250abb0 100644 --- a/csharp/ql/src/semmle/code/csharp/commons/ComparisonTest.qll +++ b/csharp/ql/src/semmle/code/csharp/commons/ComparisonTest.qll @@ -2,7 +2,7 @@ * Provides classes for capturing various ways of performing comparison tests. */ -import csharp +private import csharp private import semmle.code.csharp.frameworks.System private import semmle.code.csharp.frameworks.system.Collections private import semmle.code.csharp.frameworks.system.collections.Generic diff --git a/csharp/ql/src/semmle/code/csharp/controlflow/Guards.qll b/csharp/ql/src/semmle/code/csharp/controlflow/Guards.qll index 89042f7de07..65a3bfe7213 100644 --- a/csharp/ql/src/semmle/code/csharp/controlflow/Guards.qll +++ b/csharp/ql/src/semmle/code/csharp/controlflow/Guards.qll @@ -31,6 +31,25 @@ class Guard extends Expr { predicate controlsNode(ControlFlow::Nodes::ElementNode cfn, AccessOrCallExpr sub, AbstractValue v) { isGuardedByNode(cfn, this, sub, v) } + + /** + * Holds if basic block `bb` is guarded by this expression having value `v`. + */ + predicate controlsBasicBlock(BasicBlock bb, AbstractValue v) { + Internal::guardControls(this, bb, v) + } + + /** + * Holds if this guard is an equality test between `e1` and `e2`. If the test is + * negated, that is `!=`, then `polarity` is false, otherwise `polarity` is + * true. + */ + predicate isEquality(Expr e1, Expr e2, boolean polarity) { + exists(BooleanValue v | + this = Internal::getAnEqualityCheck(e1, v, e2) and + polarity = v.getValue() + ) + } } /** An abstract value. */ @@ -943,13 +962,14 @@ module Internal { e = any(BinaryArithmeticOperation bao | result = bao.getAnOperand()) } - /** Holds if basic block `bb` only is reached when guard `g` has abstract value `v`. */ - private predicate guardControls(Guard g, BasicBlock bb, AbstractValue v) { - exists(ControlFlowElement cfe, ConditionalSuccessor s, AbstractValue v0, Guard g0 | - cfe.controlsBlock(bb, s) - | - v0.branch(cfe, s, g0) and - impliesSteps(g0, v0, g, v) + pragma[noinline] + private predicate assertionControlsNodeInSameBasicBlock0( + Guard g, AbstractValue v, BasicBlock bb, int i + ) { + exists(Assertion a, Guard g0, AbstractValue v0 | + asserts(a, g0, v0) and + impliesSteps(g0, v0, g, v) and + bb.getNode(i) = a.getAControlFlowNode() ) } @@ -957,17 +977,13 @@ module Internal { * Holds if control flow node `cfn` only is reached when guard `g` evaluates to `v`, * because of an assertion. */ - private predicate guardAssertionControlsNode(Guard g, ControlFlow::Node cfn, AbstractValue v) { - exists(Assertion a, Guard g0, AbstractValue v0 | - asserts(a, g0, v0) and - impliesSteps(g0, v0, g, v) - | - a.strictlyDominates(cfn.getBasicBlock()) - or - exists(BasicBlock bb, int i, int j | bb.getNode(i) = a.getAControlFlowNode() | - bb.getNode(j) = cfn and - j > i - ) + private predicate assertionControlsNodeInSameBasicBlock( + Guard g, ControlFlow::Node cfn, AbstractValue v + ) { + exists(BasicBlock bb, int i, int j | + assertionControlsNodeInSameBasicBlock0(g, v, bb, i) and + bb.getNode(j) = cfn and + j > i ) } @@ -977,7 +993,7 @@ module Internal { */ private predicate guardAssertionControlsElement(Guard g, ControlFlowElement cfe, AbstractValue v) { forex(ControlFlow::Node cfn | cfn = cfe.getAControlFlowNode() | - guardAssertionControlsNode(g, cfn, v) + assertionControlsNodeInSameBasicBlock(g, cfn, v) ) } @@ -1291,24 +1307,6 @@ module Internal { ) } - /** - * Gets an expression that tests whether expression `e1` is equal to - * expression `e2`. - * - * If the returned expression has abstract value `v`, then expression `e1` is - * guaranteed to be equal to `e2`, and if the returned expression has abstract - * value `v.getDualValue()`, then this expression is guaranteed to be - * non-equal to `e`. - * - * For example, if the expression `x != ""` evaluates to `false` then the - * expression `x` is guaranteed to be equal to `""`. - */ - Expr getAnEqualityCheck(Expr e1, AbstractValue v, Expr e2) { - result = getABooleanEqualityCheck(e1, v, e2) - or - result = getAMatchingEqualityCheck(e1, v, e2) - } - private Expr getAnEqualityCheckVal(Expr e, AbstractValue v, AbstractValue vExpr) { result = getAnEqualityCheck(e, v, vExpr.getAnExpr()) } @@ -1464,6 +1462,29 @@ module Internal { not e = any(LocalVariableDeclStmt s).getAVariableDeclExpr() } + /** + * Gets an expression that tests whether expression `e1` is equal to + * expression `e2`. + * + * If the returned expression has abstract value `v`, then expression `e1` is + * guaranteed to be equal to `e2`, and if the returned expression has abstract + * value `v.getDualValue()`, then this expression is guaranteed to be + * non-equal to `e`. + * + * For example, if the expression `x != ""` evaluates to `false` then the + * expression `x` is guaranteed to be equal to `""`. + */ + cached + Expr getAnEqualityCheck(Expr e1, AbstractValue v, Expr e2) { + result = getABooleanEqualityCheck(e1, v, e2) + or + result = getABooleanEqualityCheck(e2, v, e1) + or + result = getAMatchingEqualityCheck(e1, v, e2) + or + result = getAMatchingEqualityCheck(e2, v, e1) + } + cached predicate isCustomNullCheck(Call call, Expr arg, BooleanValue v, boolean isNull) { exists(Callable callable, Parameter p | @@ -1749,7 +1770,7 @@ module Internal { exists(Guard g | e = getAChildExprStar(g) | guardControls(g, bb, _) or - guardAssertionControlsNode(g, bb.getANode(), _) + assertionControlsNodeInSameBasicBlock(g, bb.getANode(), _) ) } } @@ -1758,6 +1779,21 @@ module Internal { private module Cached { private import semmle.code.csharp.Caching + /** Holds if basic block `bb` only is reached when guard `g` has abstract value `v`. */ + cached + predicate guardControls(Guard g, BasicBlock bb, AbstractValue v) { + exists(AbstractValue v0, Guard g0 | impliesSteps(g0, v0, g, v) | + exists(ControlFlowElement cfe, ConditionalSuccessor s | + v0.branch(cfe, s, g0) and cfe.controlsBlock(bb, s) + ) + or + exists(Assertion a | + asserts(a, g0, v0) and + a.strictlyDominates(bb) + ) + ) + } + pragma[noinline] private predicate isGuardedByNode0( ControlFlow::Node cfn, AccessOrCallExpr guarded, Guard g, AccessOrCallExpr sub, @@ -1813,7 +1849,7 @@ module Internal { ) { isGuardedByNode0(guarded, _, g, sub, v) or - guardAssertionControlsNode(g, guarded, v) and + assertionControlsNodeInSameBasicBlock(g, guarded, v) and exists(ConditionOnExprComparisonConfig c | c.same(sub, guarded.getElement())) } diff --git a/csharp/ql/src/semmle/code/csharp/dataflow/Nullness.qll b/csharp/ql/src/semmle/code/csharp/dataflow/Nullness.qll index a537d08aabc..ed10e700966 100644 --- a/csharp/ql/src/semmle/code/csharp/dataflow/Nullness.qll +++ b/csharp/ql/src/semmle/code/csharp/dataflow/Nullness.qll @@ -21,10 +21,8 @@ import csharp private import ControlFlow private import internal.CallableReturns private import semmle.code.csharp.commons.Assertions -private import semmle.code.csharp.commons.ComparisonTest private import semmle.code.csharp.controlflow.Guards as G private import semmle.code.csharp.controlflow.Guards::AbstractValues -private import semmle.code.csharp.dataflow.SSA private import semmle.code.csharp.frameworks.System private import semmle.code.csharp.frameworks.Test diff --git a/csharp/ql/src/semmle/code/csharp/dataflow/SSA.qll b/csharp/ql/src/semmle/code/csharp/dataflow/SSA.qll index adcfdfaaca7..f7a018c2d70 100644 --- a/csharp/ql/src/semmle/code/csharp/dataflow/SSA.qll +++ b/csharp/ql/src/semmle/code/csharp/dataflow/SSA.qll @@ -706,9 +706,9 @@ module Ssa { /** * Holds if `def` is accessed in basic block `bb1` (either a read or a write), - * `bb2` is a transitive successor of `bb1`, and `def` is *maybe* read in `bb2` - * or one of its transitive successors, but not in any block on the path between - * `bb1` and `bb2`. + * `bb2` is a transitive successor of `bb1`, `def` is live at the end of `bb1`, + * and the underlying variable for `def` is neither read nor written in any block + * on the path between `bb1` and `bb2`. */ private predicate varBlockReaches(TrackedDefinition def, BasicBlock bb1, BasicBlock bb2) { varOccursInBlock(def, bb1, _) and @@ -2537,6 +2537,13 @@ module Ssa { ) } + /** Holds if `inp` is an input to the phi node along the edge originating in `bb`. */ + predicate hasInputFromBlock(Definition inp, BasicBlock bb) { + this.getAnInput() = inp and + this.getBasicBlock().getAPredecessor() = bb and + inp.isLiveAtEndOfBlock(bb) + } + override string toString() { result = getToStringPrefix(this) + "SSA phi(" + getSourceVariable() + ")" } diff --git a/csharp/ql/src/semmle/code/csharp/dataflow/SignAnalysis.qll b/csharp/ql/src/semmle/code/csharp/dataflow/SignAnalysis.qll new file mode 100644 index 00000000000..3742f87de25 --- /dev/null +++ b/csharp/ql/src/semmle/code/csharp/dataflow/SignAnalysis.qll @@ -0,0 +1,9 @@ +/** + * Provides sign analysis to determine whether expression are always positive + * or negative. + * + * The analysis is implemented as an abstract interpretation over the + * three-valued domain `{negative, zero, positive}`. + */ + +import semmle.code.csharp.dataflow.internal.rangeanalysis.SignAnalysisCommon diff --git a/csharp/ql/src/semmle/code/csharp/dataflow/internal/rangeanalysis/ConstantUtils.qll b/csharp/ql/src/semmle/code/csharp/dataflow/internal/rangeanalysis/ConstantUtils.qll new file mode 100644 index 00000000000..e283909155c --- /dev/null +++ b/csharp/ql/src/semmle/code/csharp/dataflow/internal/rangeanalysis/ConstantUtils.qll @@ -0,0 +1,64 @@ +/** + * Provides classes and predicates to represent constant integer expressions. + */ + +private import csharp +private import Ssa + +/** + * Holds if property `p` matches `property` in `baseClass` or any overrides. + */ +predicate propertyOverrides(Property p, string baseClass, string property) { + exists(Property p2 | + p2.getSourceDeclaration().getDeclaringType().hasQualifiedName(baseClass) and + p2.hasName(property) + | + p.overridesOrImplementsOrEquals(p2) + ) +} + +/** + * Holds if expression `e` is either + * - a compile time constant with integer value `val`, or + * - a read of a compile time constant with integer value `val`, or + * - a read of the `Length` of an array with `val` lengths. + */ +private predicate constantIntegerExpr(Expr e, int val) { + e.getValue().toInt() = val + or + exists(ExplicitDefinition v, Expr src | + e = v.getARead() and + src = v.getADefinition().getSource() and + constantIntegerExpr(src, val) + ) + or + isArrayLengthAccess(e, val) +} + +private int getArrayLength(ArrayCreation arrCreation, int index) { + constantIntegerExpr(arrCreation.getLengthArgument(index), result) +} + +private int getArrayLengthRec(ArrayCreation arrCreation, int index) { + index = 0 and result = getArrayLength(arrCreation, 0) + or + index > 0 and + result = getArrayLength(arrCreation, index) * getArrayLengthRec(arrCreation, index - 1) +} + +private predicate isArrayLengthAccess(PropertyAccess pa, int length) { + propertyOverrides(pa.getTarget(), "System.Array", "Length") and + exists(ExplicitDefinition arr, ArrayCreation arrCreation | + getArrayLengthRec(arrCreation, arrCreation.getNumberOfLengthArguments() - 1) = length and + arrCreation = arr.getADefinition().getSource() and + pa.getQualifier() = arr.getARead() + ) +} + +/** An expression that always has the same integer value. */ +class ConstantIntegerExpr extends Expr { + ConstantIntegerExpr() { constantIntegerExpr(this, _) } + + /** Gets the integer value of this expression. */ + int getIntValue() { constantIntegerExpr(this, result) } +} diff --git a/csharp/ql/src/semmle/code/csharp/dataflow/internal/rangeanalysis/Sign.qll b/csharp/ql/src/semmle/code/csharp/dataflow/internal/rangeanalysis/Sign.qll new file mode 100644 index 00000000000..10ca946a044 --- /dev/null +++ b/csharp/ql/src/semmle/code/csharp/dataflow/internal/rangeanalysis/Sign.qll @@ -0,0 +1,219 @@ +newtype TSign = + TNeg() or + TZero() or + TPos() + +/** Class representing expression signs (+, -, 0). */ +class Sign extends TSign { + /** Gets the string representation of this sign. */ + string toString() { + result = "-" and this = TNeg() + or + result = "0" and this = TZero() + or + result = "+" and this = TPos() + } + + /** Gets a possible sign after incrementing an expression that has this sign. */ + Sign inc() { + this = TNeg() and result = TNeg() + or + this = TNeg() and result = TZero() + or + this = TZero() and result = TPos() + or + this = TPos() and result = TPos() + } + + /** Gets a possible sign after decrementing an expression that has this sign. */ + Sign dec() { result.inc() = this } + + /** Gets a possible sign after negating an expression that has this sign. */ + Sign neg() { + this = TNeg() and result = TPos() + or + this = TZero() and result = TZero() + or + this = TPos() and result = TNeg() + } + + /** + * Gets a possible sign after bitwise complementing an expression that has this + * sign. + */ + Sign bitnot() { + this = TNeg() and result = TPos() + or + this = TNeg() and result = TZero() + or + this = TZero() and result = TNeg() + or + this = TPos() and result = TNeg() + } + + /** + * Gets a possible sign after adding an expression with sign `s` to an expression + * that has this sign. + */ + Sign add(Sign s) { + this = TZero() and result = s + or + s = TZero() and result = this + or + this = s and this = result + or + this = TPos() and s = TNeg() + or + this = TNeg() and s = TPos() + } + + /** + * Gets a possible sign after multiplying an expression with sign `s` to an expression + * that has this sign. + */ + Sign mul(Sign s) { + result = TZero() and this = TZero() + or + result = TZero() and s = TZero() + or + result = TNeg() and this = TPos() and s = TNeg() + or + result = TNeg() and this = TNeg() and s = TPos() + or + result = TPos() and this = TPos() and s = TPos() + or + result = TPos() and this = TNeg() and s = TNeg() + } + + /** + * Gets a possible sign after integer dividing an expression that has this sign + * by an expression with sign `s`. + */ + Sign div(Sign s) { + result = TZero() and s = TNeg() // ex: 3 / -5 = 0 + or + result = TZero() and s = TPos() // ex: 3 / 5 = 0 + or + result = TNeg() and this = TPos() and s = TNeg() + or + result = TNeg() and this = TNeg() and s = TPos() + or + result = TPos() and this = TPos() and s = TPos() + or + result = TPos() and this = TNeg() and s = TNeg() + } + + /** + * Gets a possible sign after modulo dividing an expression that has this sign + * by an expression with sign `s`. + */ + Sign rem(Sign s) { + result = TZero() and s = TNeg() + or + result = TZero() and s = TPos() + or + result = this and s = TNeg() + or + result = this and s = TPos() + } + + /** + * Gets a possible sign after bitwise `and` of an expression that has this sign + * and an expression with sign `s`. + */ + Sign bitand(Sign s) { + result = TZero() and this = TZero() + or + result = TZero() and s = TZero() + or + result = TZero() and this = TPos() + or + result = TZero() and s = TPos() + or + result = TNeg() and this = TNeg() and s = TNeg() + or + result = TPos() and this = TNeg() and s = TPos() + or + result = TPos() and this = TPos() and s = TNeg() + or + result = TPos() and this = TPos() and s = TPos() + } + + /** + * Gets a possible sign after bitwise `or` of an expression that has this sign + * and an expression with sign `s`. + */ + Sign bitor(Sign s) { + result = TZero() and this = TZero() and s = TZero() + or + result = TNeg() and this = TNeg() + or + result = TNeg() and s = TNeg() + or + result = TPos() and this = TPos() and s = TZero() + or + result = TPos() and this = TZero() and s = TPos() + or + result = TPos() and this = TPos() and s = TPos() + } + + /** + * Gets a possible sign after bitwise `xor` of an expression that has this sign + * and an expression with sign `s`. + */ + Sign bitxor(Sign s) { + result = TZero() and this = s + or + result = this and s = TZero() + or + result = s and this = TZero() + or + result = TPos() and this = TPos() and s = TPos() + or + result = TNeg() and this = TNeg() and s = TPos() + or + result = TNeg() and this = TPos() and s = TNeg() + or + result = TPos() and this = TNeg() and s = TNeg() + } + + /** + * Gets a possible sign after left shift of an expression that has this sign + * by an expression with sign `s`. + */ + Sign lshift(Sign s) { + result = TZero() and this = TZero() + or + result = this and s = TZero() + or + this != TZero() and s != TZero() + } + + /** + * Gets a possible sign after right shift of an expression that has this sign + * by an expression with sign `s`. + */ + Sign rshift(Sign s) { + result = TZero() and this = TZero() + or + result = this and s = TZero() + or + result = TNeg() and this = TNeg() + or + result != TNeg() and this = TPos() and s != TZero() + } + + /** + * Gets a possible sign after unsigned right shift of an expression that has + * this sign by an expression with sign `s`. + */ + Sign urshift(Sign s) { + result = TZero() and this = TZero() + or + result = this and s = TZero() + or + result != TZero() and this = TNeg() and s != TZero() + or + result != TNeg() and this = TPos() and s != TZero() + } +} diff --git a/csharp/ql/src/semmle/code/csharp/dataflow/internal/rangeanalysis/SignAnalysisCommon.qll b/csharp/ql/src/semmle/code/csharp/dataflow/internal/rangeanalysis/SignAnalysisCommon.qll new file mode 100644 index 00000000000..c734e6acb0e --- /dev/null +++ b/csharp/ql/src/semmle/code/csharp/dataflow/internal/rangeanalysis/SignAnalysisCommon.qll @@ -0,0 +1,299 @@ +/** + * Provides sign analysis to determine whether expression are always positive + * or negative. + * + * The analysis is implemented as an abstract interpretation over the + * three-valued domain `{negative, zero, positive}`. + */ + +private import SignAnalysisSpecific::Private +private import SsaReadPositionCommon +private import Sign + +/** Gets the sign of `e` if this can be directly determined. */ +Sign certainExprSign(Expr e) { + exists(int i | e.(ConstantIntegerExpr).getIntValue() = i | + i < 0 and result = TNeg() + or + i = 0 and result = TZero() + or + i > 0 and result = TPos() + ) + or + not exists(e.(ConstantIntegerExpr).getIntValue()) and + ( + exists(float f | f = getNonIntegerValue(e) | + f < 0 and result = TNeg() + or + f = 0 and result = TZero() + or + f > 0 and result = TPos() + ) + or + exists(string charlit | charlit = getCharValue(e) | + if charlit.regexpMatch("\\u0000") then result = TZero() else result = TPos() + ) + or + containerSizeAccess(e) and + (result = TPos() or result = TZero()) + or + positiveExpression(e) and result = TPos() + ) +} + +/** Holds if the sign of `e` is too complicated to determine. */ +predicate unknownSign(Expr e) { + not exists(certainExprSign(e)) and + ( + exists(IntegerLiteral lit | lit = e and not exists(lit.getValue().toInt())) + or + exists(LongLiteral lit | lit = e and not exists(lit.getValue().toFloat())) + or + exists(CastExpr cast, Type fromtyp | + cast = e and + fromtyp = cast.getExpr().getType() and + not fromtyp instanceof NumericOrCharType + ) + or + unknownIntegerAccess(e) + ) +} + +/** + * Holds if `lowerbound` is a lower bound for `v` at `pos`. This is restricted + * to only include bounds for which we might determine a sign. + */ +private predicate lowerBound(Expr lowerbound, SsaVariable v, SsaReadPosition pos, boolean isStrict) { + exists(boolean testIsTrue, ComparisonExpr comp | + pos.hasReadOfVar(v) and + guardControlsSsaRead(getComparisonGuard(comp), pos, testIsTrue) and + not unknownSign(lowerbound) + | + testIsTrue = true and + comp.getLesserOperand() = lowerbound and + comp.getGreaterOperand() = ssaRead(v, 0) and + (if comp.isStrict() then isStrict = true else isStrict = false) + or + testIsTrue = false and + comp.getGreaterOperand() = lowerbound and + comp.getLesserOperand() = ssaRead(v, 0) and + (if comp.isStrict() then isStrict = false else isStrict = true) + ) +} + +/** + * Holds if `upperbound` is an upper bound for `v` at `pos`. This is restricted + * to only include bounds for which we might determine a sign. + */ +private predicate upperBound(Expr upperbound, SsaVariable v, SsaReadPosition pos, boolean isStrict) { + exists(boolean testIsTrue, ComparisonExpr comp | + pos.hasReadOfVar(v) and + guardControlsSsaRead(getComparisonGuard(comp), pos, testIsTrue) and + not unknownSign(upperbound) + | + testIsTrue = true and + comp.getGreaterOperand() = upperbound and + comp.getLesserOperand() = ssaRead(v, 0) and + (if comp.isStrict() then isStrict = true else isStrict = false) + or + testIsTrue = false and + comp.getLesserOperand() = upperbound and + comp.getGreaterOperand() = ssaRead(v, 0) and + (if comp.isStrict() then isStrict = false else isStrict = true) + ) +} + +/** + * Holds if `eqbound` is an equality/inequality for `v` at `pos`. This is + * restricted to only include bounds for which we might determine a sign. The + * boolean `isEq` gives the polarity: + * - `isEq = true` : `v = eqbound` + * - `isEq = false` : `v != eqbound` + */ +private predicate eqBound(Expr eqbound, SsaVariable v, SsaReadPosition pos, boolean isEq) { + exists(Guard guard, boolean testIsTrue, boolean polarity | + pos.hasReadOfVar(v) and + guardControlsSsaRead(guard, pos, testIsTrue) and + guard.isEquality(eqbound, ssaRead(v, 0), polarity) and + isEq = polarity.booleanXor(testIsTrue).booleanNot() and + not unknownSign(eqbound) + ) +} + +/** + * Holds if `bound` is a bound for `v` at `pos` that needs to be positive in + * order for `v` to be positive. + */ +private predicate posBound(Expr bound, SsaVariable v, SsaReadPosition pos) { + upperBound(bound, v, pos, _) or + eqBound(bound, v, pos, true) +} + +/** + * Holds if `bound` is a bound for `v` at `pos` that needs to be negative in + * order for `v` to be negative. + */ +private predicate negBound(Expr bound, SsaVariable v, SsaReadPosition pos) { + lowerBound(bound, v, pos, _) or + eqBound(bound, v, pos, true) +} + +/** + * Holds if `bound` is a bound for `v` at `pos` that can restrict whether `v` + * can be zero. + */ +private predicate zeroBound(Expr bound, SsaVariable v, SsaReadPosition pos) { + lowerBound(bound, v, pos, _) or + upperBound(bound, v, pos, _) or + eqBound(bound, v, pos, _) +} + +/** Holds if `bound` allows `v` to be positive at `pos`. */ +private predicate posBoundOk(Expr bound, SsaVariable v, SsaReadPosition pos) { + posBound(bound, v, pos) and TPos() = exprSign(bound) +} + +/** Holds if `bound` allows `v` to be negative at `pos`. */ +private predicate negBoundOk(Expr bound, SsaVariable v, SsaReadPosition pos) { + negBound(bound, v, pos) and TNeg() = exprSign(bound) +} + +/** Holds if `bound` allows `v` to be zero at `pos`. */ +private predicate zeroBoundOk(Expr bound, SsaVariable v, SsaReadPosition pos) { + lowerBound(bound, v, pos, _) and TNeg() = exprSign(bound) + or + lowerBound(bound, v, pos, false) and TZero() = exprSign(bound) + or + upperBound(bound, v, pos, _) and TPos() = exprSign(bound) + or + upperBound(bound, v, pos, false) and TZero() = exprSign(bound) + or + eqBound(bound, v, pos, true) and TZero() = exprSign(bound) + or + eqBound(bound, v, pos, false) and TZero() != exprSign(bound) +} + +/** + * Holds if there is a bound that might restrict whether `v` has the sign `s` + * at `pos`. + */ +private predicate hasGuard(SsaVariable v, SsaReadPosition pos, Sign s) { + s = TPos() and posBound(_, v, pos) + or + s = TNeg() and negBound(_, v, pos) + or + s = TZero() and zeroBound(_, v, pos) +} + +pragma[noinline] +private Sign guardedSsaSign(SsaVariable v, SsaReadPosition pos) { + // SSA variable can have sign `result` + result = ssaDefSign(v) and + pos.hasReadOfVar(v) and + // there are guards at this position on `v` that might restrict it to be sign `result`. + // (So we need to check if they are satisfied) + hasGuard(v, pos, result) +} + +pragma[noinline] +private Sign unguardedSsaSign(SsaVariable v, SsaReadPosition pos) { + // SSA variable can have sign `result` + result = ssaDefSign(v) and + pos.hasReadOfVar(v) and + // there's no guard at this position on `v` that might restrict it to be sign `result`. + not hasGuard(v, pos, result) +} + +/** + * Gets the sign of `v` at read position `pos`, when there's at least one guard + * on `v` at position `pos`. Each bound corresponding to a given sign must be met + * in order for `v` to be of that sign. + */ +private Sign guardedSsaSignOk(SsaVariable v, SsaReadPosition pos) { + result = TPos() and + forex(Expr bound | posBound(bound, v, pos) | posBoundOk(bound, v, pos)) + or + result = TNeg() and + forex(Expr bound | negBound(bound, v, pos) | negBoundOk(bound, v, pos)) + or + result = TZero() and + forex(Expr bound | zeroBound(bound, v, pos) | zeroBoundOk(bound, v, pos)) +} + +/** Gets a possible sign for `v` at `pos`. */ +Sign ssaSign(SsaVariable v, SsaReadPosition pos) { + result = unguardedSsaSign(v, pos) + or + result = guardedSsaSign(v, pos) and + result = guardedSsaSignOk(v, pos) +} + +/** Gets a possible sign for `v`. */ +pragma[nomagic] +Sign ssaDefSign(SsaVariable v) { + result = explicitSsaDefSign(v) + or + result = implicitSsaDefSign(v) + or + exists(SsaPhiNode phi, SsaVariable inp, SsaReadPositionPhiInputEdge edge | + v = phi and + edge.phiInput(phi, inp) and + result = ssaSign(inp, edge) + ) +} + +/** Gets a possible sign for `e`. */ +cached +Sign exprSign(Expr e) { + exists(Sign s | + s = certainExprSign(e) + or + not exists(certainExprSign(e)) and + ( + unknownSign(e) + or + exists(SsaVariable v | getARead(v) = e | s = ssaVariableSign(v, e)) + or + e = + any(VarAccess access | + not exists(SsaVariable v | getARead(v) = access) and + ( + s = fieldSign(getField(access.(FieldAccess))) or + not access instanceof FieldAccess + ) + ) + or + s = specificSubExprSign(e) + ) + | + if e.getType() instanceof UnsignedNumericType and s = TNeg() + then result = TPos() + else result = s + ) +} + +/** Holds if `e` can be positive and cannot be negative. */ +predicate positive(Expr e) { + exprSign(e) = TPos() and + not exprSign(e) = TNeg() +} + +/** Holds if `e` can be negative and cannot be positive. */ +predicate negative(Expr e) { + exprSign(e) = TNeg() and + not exprSign(e) = TPos() +} + +/** Holds if `e` is strictly positive. */ +predicate strictlyPositive(Expr e) { + exprSign(e) = TPos() and + not exprSign(e) = TNeg() and + not exprSign(e) = TZero() +} + +/** Holds if `e` is strictly negative. */ +predicate strictlyNegative(Expr e) { + exprSign(e) = TNeg() and + not exprSign(e) = TPos() and + not exprSign(e) = TZero() +} diff --git a/csharp/ql/src/semmle/code/csharp/dataflow/internal/rangeanalysis/SignAnalysisSpecific.qll b/csharp/ql/src/semmle/code/csharp/dataflow/internal/rangeanalysis/SignAnalysisSpecific.qll new file mode 100644 index 00000000000..813834c3fd8 --- /dev/null +++ b/csharp/ql/src/semmle/code/csharp/dataflow/internal/rangeanalysis/SignAnalysisSpecific.qll @@ -0,0 +1,299 @@ +/** + * Provides C#-specific definitions for use in sign analysis. + */ +module Private { + private import SsaUtils as SU + private import csharp as CS + private import ConstantUtils as CU + private import semmle.code.csharp.controlflow.Guards as G + import Impl + + class Guard = G::Guard; + + class ConstantIntegerExpr = CU::ConstantIntegerExpr; + + class SsaVariable = CS::Ssa::Definition; + + class SsaPhiNode = CS::Ssa::PhiNode; + + class VarAccess = CS::AssignableAccess; + + class FieldAccess = CS::FieldAccess; + + class CharacterLiteral = CS::CharLiteral; + + class IntegerLiteral = CS::IntegerLiteral; + + class LongLiteral = CS::LongLiteral; + + class CastExpr = CS::CastExpr; + + class Type = CS::Type; + + class Expr = CS::Expr; + + predicate ssaRead = SU::ssaRead/2; +} + +private module Impl { + private import csharp + private import SsaUtils + private import ConstantUtils + private import semmle.code.csharp.controlflow.Guards + private import Linq.Helpers + private import Sign + private import SignAnalysisCommon + private import SsaReadPositionCommon + private import semmle.code.csharp.commons.ComparisonTest + + private class BooleanValue = AbstractValues::BooleanValue; + + float getNonIntegerValue(Expr e) { + exists(string s | + s = e.getValue() and + result = s.toFloat() and + not exists(s.toInt()) + ) + } + + string getCharValue(Expr e) { result = e.getValue() and e.getType() instanceof CharType } + + predicate containerSizeAccess(Expr e) { + exists(Property p | p = e.(PropertyAccess).getTarget() | + propertyOverrides(p, "System.Collections.Generic.IEnumerable<>", "Count") or + propertyOverrides(p, "System.Collections.ICollection", "Count") or + propertyOverrides(p, "System.String", "Length") or + propertyOverrides(p, "System.Array", "Length") + ) + or + e instanceof CountCall + } + + predicate positiveExpression(Expr e) { e instanceof SizeofExpr } + + abstract class NumericOrCharType extends Type { } + + class UnsignedNumericType extends NumericOrCharType { + UnsignedNumericType() { + this instanceof CharType + or + this instanceof UnsignedIntegralType + or + this instanceof PointerType + or + this instanceof Enum and this.(Enum).getUnderlyingType() instanceof UnsignedIntegralType + } + } + + class SignedNumericType extends NumericOrCharType { + SignedNumericType() { + this instanceof SignedIntegralType + or + this instanceof FloatingPointType + or + this instanceof DecimalType + or + this instanceof Enum and this.(Enum).getUnderlyingType() instanceof SignedIntegralType + } + } + + Sign explicitSsaDefSign(Ssa::ExplicitDefinition v) { + exists(AssignableDefinition def | def = v.getADefinition() | + result = exprSign(def.getSource()) + or + not exists(def.getSource()) and + not def.getElement() instanceof MutatorOperation + or + result = exprSign(def.getElement().(IncrementOperation).getOperand()).inc() + or + result = exprSign(def.getElement().(DecrementOperation).getOperand()).dec() + ) + } + + Sign implicitSsaDefSign(Ssa::ImplicitDefinition v) { + result = fieldSign(v.getSourceVariable().getAssignable()) or + not v.getSourceVariable().getAssignable() instanceof Field + } + + pragma[inline] + Sign ssaVariableSign(Ssa::Definition v, Expr e) { + result = ssaSign(v, any(SsaReadPositionBlock bb | getAnExpression(bb) = e)) + } + + /** Gets a possible sign for `f`. */ + Sign fieldSign(Field f) { + if f.fromSource() and f.isEffectivelyPrivate() + then + result = exprSign(f.getAnAssignedValue()) + or + any(IncrementOperation inc).getOperand() = f.getAnAccess() and result = fieldSign(f).inc() + or + any(DecrementOperation dec).getOperand() = f.getAnAccess() and result = fieldSign(f).dec() + or + exists(AssignOperation a | a.getLValue() = f.getAnAccess() | result = exprSign(a)) + or + not exists(f.getInitializer()) and result = TZero() + else any() + } + + predicate unknownIntegerAccess(Expr e) { + e.getType() instanceof NumericOrCharType and + not e = getARead(_) and + not e instanceof FieldAccess and + not e instanceof TypeAccess and + // The expression types that are listed here are the ones handled in `specificSubExprSign`. + // Keep them in sync. + not e instanceof AssignExpr and + not e instanceof AssignOperation and + not e instanceof UnaryPlusExpr and + not e instanceof PostIncrExpr and + not e instanceof PostDecrExpr and + not e instanceof PreIncrExpr and + not e instanceof PreDecrExpr and + not e instanceof UnaryMinusExpr and + not e instanceof ComplementExpr and + not e instanceof AddExpr and + not e instanceof SubExpr and + not e instanceof MulExpr and + not e instanceof DivExpr and + not e instanceof RemExpr and + not e instanceof BitwiseAndExpr and + not e instanceof BitwiseOrExpr and + not e instanceof BitwiseXorExpr and + not e instanceof LShiftExpr and + not e instanceof RShiftExpr and + not e instanceof ConditionalExpr and + not e instanceof RefExpr and + not e instanceof LocalVariableDeclAndInitExpr and + not e instanceof SwitchCaseExpr and + not e instanceof CastExpr and + not e instanceof SwitchExpr and + not e instanceof NullCoalescingExpr + } + + Sign specificSubExprSign(Expr e) { + // The expression types that are handled here should be excluded in `unknownIntegerAccess`. + // Keep them in sync. + result = exprSign(e.(AssignExpr).getRValue()) + or + result = exprSign(e.(AssignOperation).getExpandedAssignment()) + or + result = exprSign(e.(UnaryPlusExpr).getOperand()) + or + result = exprSign(e.(PostIncrExpr).getOperand()) + or + result = exprSign(e.(PostDecrExpr).getOperand()) + or + result = exprSign(e.(PreIncrExpr).getOperand()).inc() + or + result = exprSign(e.(PreDecrExpr).getOperand()).dec() + or + result = exprSign(e.(UnaryMinusExpr).getOperand()).neg() + or + result = exprSign(e.(ComplementExpr).getOperand()).bitnot() + or + e = + any(DivExpr div | + result = exprSign(div.getLeftOperand()) and + result != TZero() and + div.getRightOperand().(RealLiteral).getValue().toFloat() = 0 + ) + or + exists(Sign s1, Sign s2 | binaryOpSigns(e, s1, s2) | + e instanceof AddExpr and result = s1.add(s2) + or + e instanceof SubExpr and result = s1.add(s2.neg()) + or + e instanceof MulExpr and result = s1.mul(s2) + or + e instanceof DivExpr and result = s1.div(s2) + or + e instanceof RemExpr and result = s1.rem(s2) + or + e instanceof BitwiseAndExpr and result = s1.bitand(s2) + or + e instanceof BitwiseOrExpr and result = s1.bitor(s2) + or + e instanceof BitwiseXorExpr and result = s1.bitxor(s2) + or + e instanceof LShiftExpr and result = s1.lshift(s2) + or + e instanceof RShiftExpr and result = s1.rshift(s2) + ) + or + result = exprSign(e.(ConditionalExpr).getAChild()) + or + result = exprSign(e.(NullCoalescingExpr).getAChild()) + or + result = exprSign(e.(SwitchExpr).getACase().getBody()) + or + result = exprSign(e.(CastExpr).getExpr()) + or + result = exprSign(e.(SwitchCaseExpr).getBody()) + or + result = exprSign(e.(LocalVariableDeclAndInitExpr).getInitializer()) + or + result = exprSign(e.(RefExpr).getExpr()) + } + + private Sign binaryOpLhsSign(BinaryOperation e) { result = exprSign(e.getLeftOperand()) } + + private Sign binaryOpRhsSign(BinaryOperation e) { result = exprSign(e.getRightOperand()) } + + pragma[noinline] + private predicate binaryOpSigns(Expr e, Sign lhs, Sign rhs) { + lhs = binaryOpLhsSign(e) and + rhs = binaryOpRhsSign(e) + } + + Expr getARead(Ssa::Definition v) { result = v.getARead() } + + Field getField(FieldAccess fa) { result = fa.getTarget() } + + Expr getAnExpression(SsaReadPositionBlock bb) { result = bb.getBlock().getANode().getElement() } + + Guard getComparisonGuard(ComparisonExpr ce) { result = ce.getExpr() } + + /** + * Holds if `guard` controls the position `controlled` with the value `testIsTrue`. + */ + predicate guardControlsSsaRead(Guard guard, SsaReadPosition controlled, boolean testIsTrue) { + exists(BooleanValue b | b.getValue() = testIsTrue | + guard.controlsBasicBlock(controlled.(SsaReadPositionBlock).getBlock(), b) + ) + } + + /** A relational comparison */ + class ComparisonExpr extends ComparisonTest { + private boolean strict; + + ComparisonExpr() { + this.getComparisonKind() = + any(ComparisonKind ck | + ck.isLessThan() and strict = true + or + ck.isLessThanEquals() and + strict = false + ) + } + + /** + * Gets the operand on the "greater" (or "greater-or-equal") side + * of this relational expression, that is, the side that is larger + * if the overall expression evaluates to `true`; for example on + * `x <= 20` this is the `20`, and on `y > 0` it is `y`. + */ + Expr getGreaterOperand() { result = this.getSecondArgument() } + + /** + * Gets the operand on the "lesser" (or "lesser-or-equal") side + * of this relational expression, that is, the side that is smaller + * if the overall expression evaluates to `true`; for example on + * `x <= 20` this is `x`, and on `y > 0` it is the `0`. + */ + Expr getLesserOperand() { result = this.getFirstArgument() } + + /** Holds if this comparison is strict, i.e. `<` or `>`. */ + predicate isStrict() { strict = true } + } +} diff --git a/csharp/ql/src/semmle/code/csharp/dataflow/internal/rangeanalysis/SsaReadPositionCommon.qll b/csharp/ql/src/semmle/code/csharp/dataflow/internal/rangeanalysis/SsaReadPositionCommon.qll new file mode 100644 index 00000000000..558ecd1b88b --- /dev/null +++ b/csharp/ql/src/semmle/code/csharp/dataflow/internal/rangeanalysis/SsaReadPositionCommon.qll @@ -0,0 +1,57 @@ +/** + * Provides classes for representing a position at which an SSA variable is read. + */ + +private import SsaReadPositionSpecific + +private newtype TSsaReadPosition = + TSsaReadPositionBlock(BasicBlock bb) { bb = getAReadBasicBlock(_) } or + TSsaReadPositionPhiInputEdge(BasicBlock bbOrig, BasicBlock bbPhi) { + exists(SsaPhiNode phi | phi.hasInputFromBlock(_, bbOrig) and bbPhi = phi.getBasicBlock()) + } + +/** + * A position at which an SSA variable is read. This includes both ordinary + * reads occurring in basic blocks and input to phi nodes occurring along an + * edge between two basic blocks. + */ +class SsaReadPosition extends TSsaReadPosition { + /** Holds if `v` is read at this position. */ + abstract predicate hasReadOfVar(SsaVariable v); + + /** Gets a textual representation of this SSA read position. */ + abstract string toString(); +} + +/** A basic block in which an SSA variable is read. */ +class SsaReadPositionBlock extends SsaReadPosition, TSsaReadPositionBlock { + /** Gets the basic block corresponding to this position. */ + BasicBlock getBlock() { this = TSsaReadPositionBlock(result) } + + override predicate hasReadOfVar(SsaVariable v) { getBlock() = getAReadBasicBlock(v) } + + override string toString() { result = "block" } +} + +/** + * An edge between two basic blocks where the latter block has an SSA phi + * definition. The edge therefore has a read of an SSA variable serving as the + * input to the phi node. + */ +class SsaReadPositionPhiInputEdge extends SsaReadPosition, TSsaReadPositionPhiInputEdge { + /** Gets the source of the edge. */ + BasicBlock getOrigBlock() { this = TSsaReadPositionPhiInputEdge(result, _) } + + /** Gets the target of the edge. */ + BasicBlock getPhiBlock() { this = TSsaReadPositionPhiInputEdge(_, result) } + + override predicate hasReadOfVar(SsaVariable v) { this.phiInput(_, v) } + + /** Holds if `inp` is an input to `phi` along this edge. */ + predicate phiInput(SsaPhiNode phi, SsaVariable inp) { + phi.hasInputFromBlock(inp, getOrigBlock()) and + getPhiBlock() = phi.getBasicBlock() + } + + override string toString() { result = "edge" } +} diff --git a/csharp/ql/src/semmle/code/csharp/dataflow/internal/rangeanalysis/SsaReadPositionSpecific.qll b/csharp/ql/src/semmle/code/csharp/dataflow/internal/rangeanalysis/SsaReadPositionSpecific.qll new file mode 100644 index 00000000000..d7df9781b2a --- /dev/null +++ b/csharp/ql/src/semmle/code/csharp/dataflow/internal/rangeanalysis/SsaReadPositionSpecific.qll @@ -0,0 +1,16 @@ +/** + * Provides C#-specific definitions for use in the `SsaReadPosition`. + */ + +private import csharp + +class SsaVariable = Ssa::Definition; + +class SsaPhiNode = Ssa::PhiNode; + +class BasicBlock = Ssa::BasicBlock; + +/** Gets a basic block in which SSA variable `v` is read. */ +BasicBlock getAReadBasicBlock(SsaVariable v) { + result = v.getARead().getAControlFlowNode().getBasicBlock() +} diff --git a/csharp/ql/src/semmle/code/csharp/dataflow/internal/rangeanalysis/SsaUtils.qll b/csharp/ql/src/semmle/code/csharp/dataflow/internal/rangeanalysis/SsaUtils.qll new file mode 100644 index 00000000000..7f1811c1f37 --- /dev/null +++ b/csharp/ql/src/semmle/code/csharp/dataflow/internal/rangeanalysis/SsaUtils.qll @@ -0,0 +1,42 @@ +/** + * Provides utility predicates to extend the core SSA functionality. + */ + +private import csharp +private import Ssa +private import ConstantUtils + +/** + * Gets an expression that equals `v - delta`. + */ +Expr ssaRead(Definition v, int delta) { + result = v.getARead() and delta = 0 + or + exists(AddExpr add, int d1, ConstantIntegerExpr c | + result = add and + delta = d1 - c.getIntValue() + | + add.getLeftOperand() = ssaRead(v, d1) and add.getRightOperand() = c + or + add.getRightOperand() = ssaRead(v, d1) and add.getLeftOperand() = c + ) + or + exists(SubExpr sub, int d1, ConstantIntegerExpr c | + result = sub and + sub.getLeftOperand() = ssaRead(v, d1) and + sub.getRightOperand() = c and + delta = d1 + c.getIntValue() + ) + or + v.(ExplicitDefinition).getADefinition().getExpr().(PreIncrExpr) = result and delta = 0 + or + v.(ExplicitDefinition).getADefinition().getExpr().(PreDecrExpr) = result and delta = 0 + or + v.(ExplicitDefinition).getADefinition().getExpr().(PostIncrExpr) = result and delta = 1 // x++ === ++x - 1 + or + v.(ExplicitDefinition).getADefinition().getExpr().(PostDecrExpr) = result and delta = -1 // x-- === --x + 1 + or + v.(ExplicitDefinition).getADefinition().getExpr().(Assignment) = result and delta = 0 + or + result.(AssignExpr).getRValue() = ssaRead(v, delta) +} diff --git a/csharp/ql/src/semmle/code/csharp/exprs/ComparisonOperation.qll b/csharp/ql/src/semmle/code/csharp/exprs/ComparisonOperation.qll index e48aabd76a7..8b94ef5b4d7 100644 --- a/csharp/ql/src/semmle/code/csharp/exprs/ComparisonOperation.qll +++ b/csharp/ql/src/semmle/code/csharp/exprs/ComparisonOperation.qll @@ -57,6 +57,9 @@ class RelationalOperation extends ComparisonOperation, @rel_op_expr { * `x <= 20` this is `x`, and on `y > 0` it is the `0`. */ Expr getLesserOperand() { none() } + + /** Holds if this comparison is strict, i.e. `<` or `>`. */ + predicate isStrict() { this instanceof LTExpr or this instanceof GTExpr } } /** diff --git a/csharp/ql/test/experimental/ir/ir/PrintAst.expected b/csharp/ql/test/experimental/ir/ir/PrintAst.expected index 0ba7be8a584..25ea10bc476 100644 --- a/csharp/ql/test/experimental/ir/ir/PrintAst.expected +++ b/csharp/ql/test/experimental/ir/ir/PrintAst.expected @@ -762,7 +762,8 @@ pointers.cs: # 5| 1: [LocalVariableAccess] access to local variable length # 6| 1: [FixedStmt] fixed(...) { ... } # 6| -1: [LocalVariableDeclAndInitExpr] Int32* b = ... -# 6| 0: [ParameterAccess] access to parameter arr +# 6| 0: [CastExpr] (...) ... +# 6| 0: [ParameterAccess] access to parameter arr # 6| 1: [LocalVariableAccess] access to local variable b # 7| 0: [BlockStmt] {...} # 8| 0: [LocalVariableDeclStmt] ... ...; diff --git a/csharp/ql/test/experimental/ir/ir/raw_ir.expected b/csharp/ql/test/experimental/ir/ir/raw_ir.expected index 785f3872787..e426955f9da 100644 --- a/csharp/ql/test/experimental/ir/ir/raw_ir.expected +++ b/csharp/ql/test/experimental/ir/ir/raw_ir.expected @@ -1327,8 +1327,8 @@ pointers.cs: # 6| r6_1(glval) = VariableAddress[b] : # 6| r6_2(glval) = VariableAddress[arr] : # 6| r6_3(Int32[]) = Load : &:r6_2, ~m? -# 6| r6_4(Int32*) = Convert : r6_3 -# 6| mu6_5(Int32*) = Store : &:r6_1, r6_3 +# 6| r6_4(Int32*) = CheckedConvertOrThrow : r6_3 +# 6| mu6_5(Int32*) = Store : &:r6_1, r6_4 # 8| r8_1(glval) = VariableAddress[p] : # 8| r8_2(glval) = VariableAddress[b] : # 8| r8_3(Int32*) = Load : &:r8_2, ~m? diff --git a/csharp/ql/test/library-tests/controlflow/graph/BasicBlock.expected b/csharp/ql/test/library-tests/controlflow/graph/BasicBlock.expected index cf6a34b5ba7..d7ca5bdd073 100644 --- a/csharp/ql/test/library-tests/controlflow/graph/BasicBlock.expected +++ b/csharp/ql/test/library-tests/controlflow/graph/BasicBlock.expected @@ -725,7 +725,7 @@ | TypeAccesses.cs:3:10:3:10 | enter M | TypeAccesses.cs:7:13:7:22 | ... is ... | 14 | | TypeAccesses.cs:7:25:7:25 | ; | TypeAccesses.cs:7:25:7:25 | ; | 1 | | TypeAccesses.cs:8:9:8:28 | ... ...; | TypeAccesses.cs:3:10:3:10 | exit M | 4 | -| VarDecls.cs:5:18:5:19 | enter M1 | VarDecls.cs:5:18:5:19 | exit M1 | 16 | +| VarDecls.cs:5:18:5:19 | enter M1 | VarDecls.cs:5:18:5:19 | exit M1 | 18 | | VarDecls.cs:13:12:13:13 | enter M2 | VarDecls.cs:13:12:13:13 | exit M2 | 12 | | VarDecls.cs:19:7:19:8 | enter M3 | VarDecls.cs:25:20:25:20 | access to parameter b | 12 | | VarDecls.cs:25:13:25:29 | return ...; | VarDecls.cs:19:7:19:8 | exit M3 | 2 | diff --git a/csharp/ql/test/library-tests/controlflow/graph/Dominance.expected b/csharp/ql/test/library-tests/controlflow/graph/Dominance.expected index 9faece057e1..773e3cd0204 100644 --- a/csharp/ql/test/library-tests/controlflow/graph/Dominance.expected +++ b/csharp/ql/test/library-tests/controlflow/graph/Dominance.expected @@ -2568,11 +2568,13 @@ dominance | VarDecls.cs:7:9:10:9 | fixed(...) { ... } | VarDecls.cs:7:27:7:33 | access to parameter strings | | VarDecls.cs:7:22:7:36 | Char* c1 = ... | VarDecls.cs:7:44:7:50 | access to parameter strings | | VarDecls.cs:7:27:7:33 | access to parameter strings | VarDecls.cs:7:35:7:35 | 0 | -| VarDecls.cs:7:27:7:36 | access to array element | VarDecls.cs:7:22:7:36 | Char* c1 = ... | +| VarDecls.cs:7:27:7:36 | (...) ... | VarDecls.cs:7:22:7:36 | Char* c1 = ... | +| VarDecls.cs:7:27:7:36 | access to array element | VarDecls.cs:7:27:7:36 | (...) ... | | VarDecls.cs:7:35:7:35 | 0 | VarDecls.cs:7:27:7:36 | access to array element | | VarDecls.cs:7:39:7:53 | Char* c2 = ... | VarDecls.cs:8:9:10:9 | {...} | | VarDecls.cs:7:44:7:50 | access to parameter strings | VarDecls.cs:7:52:7:52 | 1 | -| VarDecls.cs:7:44:7:53 | access to array element | VarDecls.cs:7:39:7:53 | Char* c2 = ... | +| VarDecls.cs:7:44:7:53 | (...) ... | VarDecls.cs:7:39:7:53 | Char* c2 = ... | +| VarDecls.cs:7:44:7:53 | access to array element | VarDecls.cs:7:44:7:53 | (...) ... | | VarDecls.cs:7:52:7:52 | 1 | VarDecls.cs:7:44:7:53 | access to array element | | VarDecls.cs:8:9:10:9 | {...} | VarDecls.cs:9:27:9:28 | access to local variable c1 | | VarDecls.cs:9:13:9:29 | return ...; | VarDecls.cs:5:18:5:19 | exit M1 | @@ -5703,12 +5705,14 @@ postDominance | VarDecls.cs:5:18:5:19 | exit M1 | VarDecls.cs:9:13:9:29 | return ...; | | VarDecls.cs:6:5:11:5 | {...} | VarDecls.cs:5:18:5:19 | enter M1 | | VarDecls.cs:7:9:10:9 | fixed(...) { ... } | VarDecls.cs:6:5:11:5 | {...} | -| VarDecls.cs:7:22:7:36 | Char* c1 = ... | VarDecls.cs:7:27:7:36 | access to array element | +| VarDecls.cs:7:22:7:36 | Char* c1 = ... | VarDecls.cs:7:27:7:36 | (...) ... | | VarDecls.cs:7:27:7:33 | access to parameter strings | VarDecls.cs:7:9:10:9 | fixed(...) { ... } | +| VarDecls.cs:7:27:7:36 | (...) ... | VarDecls.cs:7:27:7:36 | access to array element | | VarDecls.cs:7:27:7:36 | access to array element | VarDecls.cs:7:35:7:35 | 0 | | VarDecls.cs:7:35:7:35 | 0 | VarDecls.cs:7:27:7:33 | access to parameter strings | -| VarDecls.cs:7:39:7:53 | Char* c2 = ... | VarDecls.cs:7:44:7:53 | access to array element | +| VarDecls.cs:7:39:7:53 | Char* c2 = ... | VarDecls.cs:7:44:7:53 | (...) ... | | VarDecls.cs:7:44:7:50 | access to parameter strings | VarDecls.cs:7:22:7:36 | Char* c1 = ... | +| VarDecls.cs:7:44:7:53 | (...) ... | VarDecls.cs:7:44:7:53 | access to array element | | VarDecls.cs:7:44:7:53 | access to array element | VarDecls.cs:7:52:7:52 | 1 | | VarDecls.cs:7:52:7:52 | 1 | VarDecls.cs:7:44:7:50 | access to parameter strings | | VarDecls.cs:8:9:10:9 | {...} | VarDecls.cs:7:39:7:53 | Char* c2 = ... | diff --git a/csharp/ql/test/library-tests/controlflow/graph/EnclosingCallable.expected b/csharp/ql/test/library-tests/controlflow/graph/EnclosingCallable.expected index 6dbf1e568b0..10afd142066 100644 --- a/csharp/ql/test/library-tests/controlflow/graph/EnclosingCallable.expected +++ b/csharp/ql/test/library-tests/controlflow/graph/EnclosingCallable.expected @@ -2931,10 +2931,12 @@ nodeEnclosing | VarDecls.cs:7:9:10:9 | fixed(...) { ... } | VarDecls.cs:5:18:5:19 | M1 | | VarDecls.cs:7:22:7:36 | Char* c1 = ... | VarDecls.cs:5:18:5:19 | M1 | | VarDecls.cs:7:27:7:33 | access to parameter strings | VarDecls.cs:5:18:5:19 | M1 | +| VarDecls.cs:7:27:7:36 | (...) ... | VarDecls.cs:5:18:5:19 | M1 | | VarDecls.cs:7:27:7:36 | access to array element | VarDecls.cs:5:18:5:19 | M1 | | VarDecls.cs:7:35:7:35 | 0 | VarDecls.cs:5:18:5:19 | M1 | | VarDecls.cs:7:39:7:53 | Char* c2 = ... | VarDecls.cs:5:18:5:19 | M1 | | VarDecls.cs:7:44:7:50 | access to parameter strings | VarDecls.cs:5:18:5:19 | M1 | +| VarDecls.cs:7:44:7:53 | (...) ... | VarDecls.cs:5:18:5:19 | M1 | | VarDecls.cs:7:44:7:53 | access to array element | VarDecls.cs:5:18:5:19 | M1 | | VarDecls.cs:7:52:7:52 | 1 | VarDecls.cs:5:18:5:19 | M1 | | VarDecls.cs:8:9:10:9 | {...} | VarDecls.cs:5:18:5:19 | M1 | diff --git a/csharp/ql/test/library-tests/controlflow/graph/EntryElement.expected b/csharp/ql/test/library-tests/controlflow/graph/EntryElement.expected index 8751b4d35f7..cf3a8c054eb 100644 --- a/csharp/ql/test/library-tests/controlflow/graph/EntryElement.expected +++ b/csharp/ql/test/library-tests/controlflow/graph/EntryElement.expected @@ -2064,10 +2064,12 @@ | VarDecls.cs:7:9:10:9 | fixed(...) { ... } | VarDecls.cs:7:9:10:9 | fixed(...) { ... } | | VarDecls.cs:7:22:7:36 | Char* c1 = ... | VarDecls.cs:7:27:7:33 | access to parameter strings | | VarDecls.cs:7:27:7:33 | access to parameter strings | VarDecls.cs:7:27:7:33 | access to parameter strings | +| VarDecls.cs:7:27:7:36 | (...) ... | VarDecls.cs:7:27:7:33 | access to parameter strings | | VarDecls.cs:7:27:7:36 | access to array element | VarDecls.cs:7:27:7:33 | access to parameter strings | | VarDecls.cs:7:35:7:35 | 0 | VarDecls.cs:7:35:7:35 | 0 | | VarDecls.cs:7:39:7:53 | Char* c2 = ... | VarDecls.cs:7:44:7:50 | access to parameter strings | | VarDecls.cs:7:44:7:50 | access to parameter strings | VarDecls.cs:7:44:7:50 | access to parameter strings | +| VarDecls.cs:7:44:7:53 | (...) ... | VarDecls.cs:7:44:7:50 | access to parameter strings | | VarDecls.cs:7:44:7:53 | access to array element | VarDecls.cs:7:44:7:50 | access to parameter strings | | VarDecls.cs:7:52:7:52 | 1 | VarDecls.cs:7:52:7:52 | 1 | | VarDecls.cs:8:9:10:9 | {...} | VarDecls.cs:8:9:10:9 | {...} | diff --git a/csharp/ql/test/library-tests/controlflow/graph/ExitElement.expected b/csharp/ql/test/library-tests/controlflow/graph/ExitElement.expected index 96165e8fe19..428ddfeca24 100644 --- a/csharp/ql/test/library-tests/controlflow/graph/ExitElement.expected +++ b/csharp/ql/test/library-tests/controlflow/graph/ExitElement.expected @@ -2770,10 +2770,12 @@ | VarDecls.cs:7:9:10:9 | fixed(...) { ... } | VarDecls.cs:9:13:9:29 | return ...; | return | | VarDecls.cs:7:22:7:36 | Char* c1 = ... | VarDecls.cs:7:22:7:36 | Char* c1 = ... | normal | | VarDecls.cs:7:27:7:33 | access to parameter strings | VarDecls.cs:7:27:7:33 | access to parameter strings | normal | +| VarDecls.cs:7:27:7:36 | (...) ... | VarDecls.cs:7:27:7:36 | (...) ... | normal | | VarDecls.cs:7:27:7:36 | access to array element | VarDecls.cs:7:27:7:36 | access to array element | normal | | VarDecls.cs:7:35:7:35 | 0 | VarDecls.cs:7:35:7:35 | 0 | normal | | VarDecls.cs:7:39:7:53 | Char* c2 = ... | VarDecls.cs:7:39:7:53 | Char* c2 = ... | normal | | VarDecls.cs:7:44:7:50 | access to parameter strings | VarDecls.cs:7:44:7:50 | access to parameter strings | normal | +| VarDecls.cs:7:44:7:53 | (...) ... | VarDecls.cs:7:44:7:53 | (...) ... | normal | | VarDecls.cs:7:44:7:53 | access to array element | VarDecls.cs:7:44:7:53 | access to array element | normal | | VarDecls.cs:7:52:7:52 | 1 | VarDecls.cs:7:52:7:52 | 1 | normal | | VarDecls.cs:8:9:10:9 | {...} | VarDecls.cs:9:13:9:29 | return ...; | return | diff --git a/csharp/ql/test/library-tests/controlflow/graph/NodeGraph.expected b/csharp/ql/test/library-tests/controlflow/graph/NodeGraph.expected index 03cbc257d26..7feee30c18b 100644 --- a/csharp/ql/test/library-tests/controlflow/graph/NodeGraph.expected +++ b/csharp/ql/test/library-tests/controlflow/graph/NodeGraph.expected @@ -2937,11 +2937,13 @@ | VarDecls.cs:7:9:10:9 | fixed(...) { ... } | VarDecls.cs:7:27:7:33 | access to parameter strings | semmle.label | successor | | VarDecls.cs:7:22:7:36 | Char* c1 = ... | VarDecls.cs:7:44:7:50 | access to parameter strings | semmle.label | successor | | VarDecls.cs:7:27:7:33 | access to parameter strings | VarDecls.cs:7:35:7:35 | 0 | semmle.label | successor | -| VarDecls.cs:7:27:7:36 | access to array element | VarDecls.cs:7:22:7:36 | Char* c1 = ... | semmle.label | successor | +| VarDecls.cs:7:27:7:36 | (...) ... | VarDecls.cs:7:22:7:36 | Char* c1 = ... | semmle.label | successor | +| VarDecls.cs:7:27:7:36 | access to array element | VarDecls.cs:7:27:7:36 | (...) ... | semmle.label | successor | | VarDecls.cs:7:35:7:35 | 0 | VarDecls.cs:7:27:7:36 | access to array element | semmle.label | successor | | VarDecls.cs:7:39:7:53 | Char* c2 = ... | VarDecls.cs:8:9:10:9 | {...} | semmle.label | successor | | VarDecls.cs:7:44:7:50 | access to parameter strings | VarDecls.cs:7:52:7:52 | 1 | semmle.label | successor | -| VarDecls.cs:7:44:7:53 | access to array element | VarDecls.cs:7:39:7:53 | Char* c2 = ... | semmle.label | successor | +| VarDecls.cs:7:44:7:53 | (...) ... | VarDecls.cs:7:39:7:53 | Char* c2 = ... | semmle.label | successor | +| VarDecls.cs:7:44:7:53 | access to array element | VarDecls.cs:7:44:7:53 | (...) ... | semmle.label | successor | | VarDecls.cs:7:52:7:52 | 1 | VarDecls.cs:7:44:7:53 | access to array element | semmle.label | successor | | VarDecls.cs:8:9:10:9 | {...} | VarDecls.cs:9:27:9:28 | access to local variable c1 | semmle.label | successor | | VarDecls.cs:9:13:9:29 | return ...; | VarDecls.cs:5:18:5:19 | exit M1 | semmle.label | return | diff --git a/csharp/ql/test/library-tests/controlflow/guards/Implications.expected b/csharp/ql/test/library-tests/controlflow/guards/Implications.expected index 59f89c30800..ec6c60ff9e8 100644 --- a/csharp/ql/test/library-tests/controlflow/guards/Implications.expected +++ b/csharp/ql/test/library-tests/controlflow/guards/Implications.expected @@ -397,8 +397,14 @@ | Guards.cs:276:16:276:16 | access to parameter o | match access to type Action | Guards.cs:276:16:276:16 | access to parameter o | non-null | | Guards.cs:276:16:276:16 | access to parameter o | match null | Guards.cs:276:16:276:16 | access to parameter o | null | | Guards.cs:276:16:276:16 | access to parameter o | non-match null | Guards.cs:276:16:276:16 | access to parameter o | non-null | +| Guards.cs:278:13:279:28 | ... => ... | true | Guards.cs:276:16:276:16 | access to parameter o | non-null | +| Guards.cs:280:13:281:28 | ... => ... | true | Guards.cs:276:16:276:16 | access to parameter o | non-null | | Guards.cs:281:17:281:17 | access to local variable a | non-null | Guards.cs:276:16:276:16 | access to parameter o | non-null | | Guards.cs:281:17:281:17 | access to local variable a | null | Guards.cs:276:16:276:16 | access to parameter o | null | +| Guards.cs:282:13:283:28 | ... => ... | true | Guards.cs:276:16:276:16 | access to parameter o | non-null | +| Guards.cs:284:13:285:28 | ... => ... | false | Guards.cs:276:16:276:16 | access to parameter o | non-null | +| Guards.cs:284:13:285:28 | ... => ... | true | Guards.cs:276:16:276:16 | access to parameter o | null | +| Guards.cs:286:13:287:28 | ... => ... | true | Guards.cs:276:16:276:16 | access to parameter o | non-null | | Guards.cs:296:16:296:17 | access to local variable b2 | match true | Guards.cs:294:13:294:14 | access to parameter b1 | false | | Guards.cs:296:16:296:17 | access to local variable b2 | match true | Guards.cs:296:16:296:17 | access to local variable b2 | true | | Guards.cs:308:16:308:17 | access to local variable b2 | match true | Guards.cs:306:13:306:14 | access to parameter b1 | true | diff --git a/csharp/ql/test/library-tests/conversion/pointer/Pointer.cs b/csharp/ql/test/library-tests/conversion/pointer/Pointer.cs new file mode 100644 index 00000000000..c7e12ac39b4 --- /dev/null +++ b/csharp/ql/test/library-tests/conversion/pointer/Pointer.cs @@ -0,0 +1,28 @@ +using System; + +class C +{ + unsafe static void M1(int[] arr) + { + fixed (int* i1 = arr) + { + } + + fixed (int* i2 = &arr[0]) + { + int* i3 = i2; + i3 = i3 + 1; + *i2 = *i2 + 1; + void* v2 = i2; + } + + int* i4 = null; + + int number = 1024; + byte* p = (byte*)&number; + + var s = "some string"; + fixed (char* c1 = s) + { } + } +} diff --git a/csharp/ql/test/library-tests/conversion/pointer/Pointer.expected b/csharp/ql/test/library-tests/conversion/pointer/Pointer.expected new file mode 100644 index 00000000000..5aca5582d14 --- /dev/null +++ b/csharp/ql/test/library-tests/conversion/pointer/Pointer.expected @@ -0,0 +1,11 @@ +| Pointer.cs:7:21:7:28 | Pointer.cs:7:21:7:28 | Int32* | Int32* | (...) ... | +| Pointer.cs:11:21:11:32 | Pointer.cs:11:21:11:32 | Int32* | Int32* | &... | +| Pointer.cs:13:18:13:24 | Pointer.cs:13:18:13:24 | Int32* | Int32* | access to local variable i2 | +| Pointer.cs:14:13:14:23 | Pointer.cs:14:13:14:23 | Int32* | Int32* | ... + ... | +| Pointer.cs:15:13:15:25 | Pointer.cs:15:13:15:25 | Int32 | Int32 | ... + ... | +| Pointer.cs:16:19:16:25 | Pointer.cs:16:19:16:25 | Void* | Void* | (...) ... | +| Pointer.cs:19:14:19:22 | Pointer.cs:19:14:19:22 | Int32* | null | null | +| Pointer.cs:21:13:21:25 | Pointer.cs:21:13:21:25 | Int32 | Int32 | 1024 | +| Pointer.cs:22:15:22:32 | Pointer.cs:22:15:22:32 | Byte* | Byte* | (...) ... | +| Pointer.cs:24:13:24:29 | Pointer.cs:24:13:24:29 | String | String | "some string" | +| Pointer.cs:25:22:25:27 | Pointer.cs:25:22:25:27 | Char* | Char* | (...) ... | diff --git a/csharp/ql/test/library-tests/conversion/pointer/Pointer.ql b/csharp/ql/test/library-tests/conversion/pointer/Pointer.ql new file mode 100644 index 00000000000..69e7db8c1cf --- /dev/null +++ b/csharp/ql/test/library-tests/conversion/pointer/Pointer.ql @@ -0,0 +1,5 @@ +import csharp + +from Assignment a +select a.getLocation(), a.getLValue().getType().toString(), a.getRValue().getType().toString(), + a.getRValue().toString() diff --git a/csharp/ql/test/library-tests/csharp8/switchCaseExprTypes.expected b/csharp/ql/test/library-tests/csharp8/switchCaseExprTypes.expected new file mode 100644 index 00000000000..06e2fd994c5 --- /dev/null +++ b/csharp/ql/test/library-tests/csharp8/switchCaseExprTypes.expected @@ -0,0 +1,16 @@ +| patterns.cs:101:34:101:40 | "large" | String | +| patterns.cs:102:18:102:24 | "small" | String | +| patterns.cs:110:22:110:26 | (..., ...) | (Int32,Int32) | +| patterns.cs:111:22:111:26 | (..., ...) | (Int32,Int32) | +| patterns.cs:117:27:117:33 | (..., ...) | (Int32,Int32) | +| patterns.cs:118:28:118:34 | (..., ...) | (Int32,Int32) | +| patterns.cs:119:33:119:38 | (..., ...) | (Int32,Int32) | +| patterns.cs:128:49:128:49 | 0 | Int32 | +| patterns.cs:129:38:129:38 | 1 | Int32 | +| patterns.cs:130:23:130:23 | 2 | Int32 | +| patterns.cs:131:27:131:27 | 3 | Int32 | +| patterns.cs:138:22:138:50 | throw ... | null | +| patterns.cs:139:22:139:22 | 3 | Int32 | +| patterns.cs:140:42:140:42 | 4 | Int32 | +| patterns.cs:141:29:141:29 | 5 | Int32 | +| patterns.cs:142:41:142:41 | 6 | Int32 | diff --git a/csharp/ql/test/library-tests/csharp8/switchCaseExprTypes.ql b/csharp/ql/test/library-tests/csharp8/switchCaseExprTypes.ql new file mode 100644 index 00000000000..a9ad01da2f1 --- /dev/null +++ b/csharp/ql/test/library-tests/csharp8/switchCaseExprTypes.ql @@ -0,0 +1,4 @@ +import csharp + +from SwitchCaseExpr case +select case.getBody(), case.getType().toString() diff --git a/csharp/ql/test/library-tests/dataflow/signanalysis/MissingSign.expected b/csharp/ql/test/library-tests/dataflow/signanalysis/MissingSign.expected new file mode 100644 index 00000000000..e69de29bb2d diff --git a/csharp/ql/test/library-tests/dataflow/signanalysis/MissingSign.ql b/csharp/ql/test/library-tests/dataflow/signanalysis/MissingSign.ql new file mode 100644 index 00000000000..836980829a3 --- /dev/null +++ b/csharp/ql/test/library-tests/dataflow/signanalysis/MissingSign.ql @@ -0,0 +1,16 @@ +import csharp +import semmle.code.csharp.dataflow.SignAnalysis + +from Expr e +where + not exists(exprSign(e)) and + not e instanceof TypeAccess and + ( + e.getType() instanceof CharType or + e.getType() instanceof IntegralType or + e.getType() instanceof FloatingPointType or + e.getType() instanceof DecimalType or + e.getType() instanceof Enum or + e.getType() instanceof PointerType + ) +select e diff --git a/csharp/ql/test/library-tests/dataflow/signanalysis/SignAnalysis.cs b/csharp/ql/test/library-tests/dataflow/signanalysis/SignAnalysis.cs new file mode 100644 index 00000000000..e4f3781b10d --- /dev/null +++ b/csharp/ql/test/library-tests/dataflow/signanalysis/SignAnalysis.cs @@ -0,0 +1,469 @@ +using System; +using System.Linq; +using System.Diagnostics; + +class SignAnalysis +{ + static int GetRandomValue() { return (new System.Random()).Next(0, 1000) - 500; } + + int RandomValue { get => GetRandomValue(); } + + int random = GetRandomValue(); + + int SsaSources(int p, int[] values) + { + var v = GetRandomValue(); + if (v < 0) + { + return v; + } + + v = RandomValue; + if (v < 0) + { + return v; + } + + v = p; + if (v < 0) + { + return v; + } + + v = random; + if (v < 0) + { + return v; + } + + v = values[0]; + if (v < 0) + { + return v; + } + + int x = values[1]; + v = x; + if (v < 0) + { + return v; + } + + return 0; + } + + void Operations(int i, int j, bool b) + { + if (i < 0 && j < 0) + { + var x = i + j; + System.Console.WriteLine(x); // strictly neg + x = i * j; + System.Console.WriteLine(x); // strictly pos + x = i / j; + System.Console.WriteLine(x); // pos + x = i - j; + System.Console.WriteLine(x); // no clue + x = i % j; + System.Console.WriteLine(x); // neg + x = i++; + System.Console.WriteLine(x); // strictly neg + x = i--; + System.Console.WriteLine(x); // neg + x = -i; + System.Console.WriteLine(x); // strictly pos + x = +i; + System.Console.WriteLine(x); // strictly neg + var l = (long)i; + System.Console.WriteLine(l); // strictly neg + + x = i; + x += i; + System.Console.WriteLine(x); // strictly neg + } + + if (i < 0 && j > 0) + { + var x = i + j; + System.Console.WriteLine(x); + x = i * j; + System.Console.WriteLine(x); // strictly neg + x = i / j; + System.Console.WriteLine(x); // neg + x = i - j; + System.Console.WriteLine(x); // strictly neg + x = i % j; + System.Console.WriteLine(x); // neg + x = b ? i : j; + System.Console.WriteLine(x); // any (except 0) + } + } + + void NumericalTypes() + { + var f = 4.2f; + System.Console.WriteLine(f); + var d = 4.2; + System.Console.WriteLine(d); + var de = 4.2m; + System.Console.WriteLine(de); + var c = 'a'; + System.Console.WriteLine(c); + } + + int f0; + + int f1; + + void Field0() + { + f0++; + System.Console.WriteLine(f0); // strictly positive + f0 = 0; + } + + void Field1() + { + f1++; + System.Console.WriteLine(f1); // no clue + f1 = -10; + } + + void Field2() + { + System.Console.WriteLine(f1); // no clue + } + + void Ctor() + { + var i = new Int32(); // const 0 value + i++; + System.Console.WriteLine(i); // strictly pos + } + + int Guards(int x, int y) + { + if (x < 0) + { + return x; // strictly negative + } + + if (y == 1) + { + return y; // strictly positive + } + + if (y is -1) + { + return y; // strictly negative + } + + if (x < y) + { + return y; // strictly positive + } + + var b = y == 1; + if (b) + { + return y; // strictly positive + } + + return 0; + } + + void Inconsistent() + { + var i = 1; + if (i < 0) + { + System.Console.WriteLine(i); // reported as strictly pos, although unreachable + } + } + + void SpecialValues(int[] ints) + { + System.Console.WriteLine(ints.Length); // positive + ints = new int[] { 1, 2, 3 }; + System.Console.WriteLine(ints.Length); // 3, so strictly positive + System.Console.WriteLine(ints.Count()); // positive + System.Console.WriteLine(ints.Count(i => i > 1)); // positive + + var s = "abc"; + System.Console.WriteLine(s.Length); // positive, could be strictly positive + + var enumerable = Enumerable.Empty(); + System.Console.WriteLine(enumerable.Count()); // positive + + var i = new int[,] { { 1, 1 }, { 1, 2 }, { 1, 3 } }; + System.Console.WriteLine(i.Length); // 6, so strictly positive + } + + void Phi1(int i) + { + if (i > 0) + { + System.Console.WriteLine(i); // strictly positive + } + else + { + System.Console.WriteLine(i); // negative + } + System.Console.WriteLine(i); // any + } + + void Phi2(int i) + { + if (i > 0) + { + System.Console.WriteLine(i); // strictly positive + } + else + { + if (i < 0) // negative + { + System.Console.WriteLine(i); // strictly negative + return; + } + } + System.Console.WriteLine(i); // positive, not found + } + + void Phi3(int i) + { + if (i > 0) + { + System.Console.WriteLine(i); // strictly positive + } + else + { + if (i < 0) // negative + { + System.Console.WriteLine(i); // strictly negative + } + else + { + System.Console.WriteLine(i); // zero, nothing is reported + } + } + } + + void Loop(int i, int j, int k) + { + if (i > 0) + { + while (i >= 0) // any + { + i--; // positive + System.Console.WriteLine(i); // any + } + System.Console.WriteLine(i); // strictly neg + } + + if (j > 0) + { + while (j > 0) + { + j--; // strictly pos + System.Console.WriteLine(j); // positive + } + System.Console.WriteLine(j); // reported negative, can only be 0 + } + + if (k > 0) + { + while (k > 0) + { + k--; // strictly pos + System.Console.WriteLine(k); // positive + + if (k == 5) // positive + { + break; + } + } + System.Console.WriteLine(k); // any + } + } + + void Assert(int i, bool b) + { + Debug.Assert(i > 0); + System.Console.WriteLine(i); // strictly positive [MISSING] + + if (b) + System.Console.WriteLine(i); // strictly positive + } + + void CheckedUnchecked(int i) + { + var x = unchecked(-1 * i * i); + if (x < 0) + { + System.Console.WriteLine(x); // strictly negative + } + + x = checked(-1 * i * i); + if (x < 0) + { + System.Console.WriteLine(x); // strictly negative + } + } + + void CharMinMax() + { + var min = char.MinValue; + var max = char.MaxValue; + var c = min + 1; + System.Console.WriteLine(c); // strictly positive + c = min - 1; + System.Console.WriteLine(c); // strictly negative + c = max + 1; + System.Console.WriteLine(c); // strictly positive + } + + void NullCoalesce(int? v) + { + if (v > 0) + { + var x = v ?? 1; + System.Console.WriteLine(x); // strictly positive + } + + if (v == null) + { + var x = v ?? 1; + System.Console.WriteLine(x); // strictly positive + } + + if (v < 0) + { + var x = v ?? 0; + System.Console.WriteLine(x); // negative + } + } + + async System.Threading.Tasks.Task Await() + { + var i = await System.Threading.Tasks.Task.FromResult(5); + if (i < 0) + { + System.Console.WriteLine(i); // strictly negative + } + } + + void Unsigned(uint i) + { + if (i != 0) // positive + { + System.Console.WriteLine(i); // strictly positive + } + } + + public int MyField = 0; + + void FieldAccess() + { + var x = new SignAnalysis(); + var y = x.MyField; + if (y < 0) + { + System.Console.WriteLine(y); // strictly negative + } + } + + private static unsafe void Pointer(float d) + { + float* dp = &d; + var x = *dp; + if (x < 0) + { + System.Console.WriteLine(x); // strictly negative + } + } + + [System.Runtime.InteropServices.StructLayout(System.Runtime.InteropServices.LayoutKind.Explicit, Size = 15)] + struct MyStruct { } + + unsafe void Sizeof() + { + var x = sizeof(MyStruct); + System.Console.WriteLine(x); // strictly positive + } + + void SwitchCase(string s) + { + var x = s switch + { + "x" => 0, + _ => 2 + }; + System.Console.WriteLine(x); // positive + } + + void Capture() + { + var i = 1; + void Capture() + { + if (i > 0) + Console.WriteLine(i); // strictly positive + } + Capture(); + + if (i > 0) + Console.WriteLine(i); // strictly positive + } + + public struct MyStruct2 { public int F; } + void RefExpression(MyStruct2 s) + { + ref var x = ref s.F; + if (x < 0) + { + Console.WriteLine(x); // strictly negative + } + } + + enum MyEnum { A = 12, B, C } + void EnumOp(MyEnum x, MyEnum y) + { + var i = x - y; + if (i < 0) + { + System.Console.WriteLine(i); // strictly negative + } + } + + unsafe void PointerCast(byte* src, byte* dst) + { + var x = (int)(src - dst); + if (x < 0) + { + System.Console.WriteLine(x); // strictly negative + } + + byte[] buf = new byte[10]; + + fixed (byte* to = buf) + { + System.Console.WriteLine((int)to); + } + } + + uint Unsigned() { return 1; } + void UnsignedCheck(int i) + { + long l = Unsigned(); + if (l != 0) + { + System.Console.WriteLine(l); // strictly positive + } + + uint x = (uint)i; + x++; + System.Console.WriteLine(x); // strictly positive + } +} + +// semmle-extractor-options: /r:System.Linq.dll \ No newline at end of file diff --git a/csharp/ql/test/library-tests/dataflow/signanalysis/SignAnalysis.expected b/csharp/ql/test/library-tests/dataflow/signanalysis/SignAnalysis.expected new file mode 100644 index 00000000000..ec16a427dcf --- /dev/null +++ b/csharp/ql/test/library-tests/dataflow/signanalysis/SignAnalysis.expected @@ -0,0 +1,248 @@ +| SignAnalysis.cs:7:72:7:75 | 1000 | strictlyPositive | +| SignAnalysis.cs:7:80:7:82 | 500 | strictlyPositive | +| SignAnalysis.cs:18:20:18:20 | access to local variable v | strictlyNegative | +| SignAnalysis.cs:24:20:24:20 | access to local variable v | strictlyNegative | +| SignAnalysis.cs:30:20:30:20 | access to local variable v | strictlyNegative | +| SignAnalysis.cs:36:20:36:20 | access to local variable v | strictlyNegative | +| SignAnalysis.cs:42:20:42:20 | access to local variable v | strictlyNegative | +| SignAnalysis.cs:45:24:45:24 | 1 | strictlyPositive | +| SignAnalysis.cs:49:20:49:20 | access to local variable v | strictlyNegative | +| SignAnalysis.cs:59:17:59:25 | Int32 x = ... | strictlyNegative | +| SignAnalysis.cs:59:21:59:21 | access to parameter i | strictlyNegative | +| SignAnalysis.cs:59:21:59:25 | ... + ... | strictlyNegative | +| SignAnalysis.cs:59:25:59:25 | access to parameter j | strictlyNegative | +| SignAnalysis.cs:60:38:60:38 | access to local variable x | strictlyNegative | +| SignAnalysis.cs:61:13:61:21 | ... = ... | strictlyPositive | +| SignAnalysis.cs:61:17:61:17 | access to parameter i | strictlyNegative | +| SignAnalysis.cs:61:17:61:21 | ... * ... | strictlyPositive | +| SignAnalysis.cs:61:21:61:21 | access to parameter j | strictlyNegative | +| SignAnalysis.cs:62:38:62:38 | access to local variable x | strictlyPositive | +| SignAnalysis.cs:63:13:63:21 | ... = ... | positive | +| SignAnalysis.cs:63:17:63:17 | access to parameter i | strictlyNegative | +| SignAnalysis.cs:63:17:63:21 | ... / ... | positive | +| SignAnalysis.cs:63:21:63:21 | access to parameter j | strictlyNegative | +| SignAnalysis.cs:64:38:64:38 | access to local variable x | positive | +| SignAnalysis.cs:65:17:65:17 | access to parameter i | strictlyNegative | +| SignAnalysis.cs:65:21:65:21 | access to parameter j | strictlyNegative | +| SignAnalysis.cs:67:13:67:21 | ... = ... | negative | +| SignAnalysis.cs:67:17:67:17 | access to parameter i | strictlyNegative | +| SignAnalysis.cs:67:17:67:21 | ... % ... | negative | +| SignAnalysis.cs:67:21:67:21 | access to parameter j | strictlyNegative | +| SignAnalysis.cs:68:38:68:38 | access to local variable x | negative | +| SignAnalysis.cs:69:13:69:19 | ... = ... | strictlyNegative | +| SignAnalysis.cs:69:17:69:17 | access to parameter i | strictlyNegative | +| SignAnalysis.cs:69:17:69:19 | ...++ | strictlyNegative | +| SignAnalysis.cs:70:38:70:38 | access to local variable x | strictlyNegative | +| SignAnalysis.cs:71:13:71:19 | ... = ... | negative | +| SignAnalysis.cs:71:17:71:17 | access to parameter i | negative | +| SignAnalysis.cs:71:17:71:19 | ...-- | negative | +| SignAnalysis.cs:72:38:72:38 | access to local variable x | negative | +| SignAnalysis.cs:73:13:73:18 | ... = ... | strictlyPositive | +| SignAnalysis.cs:73:17:73:18 | -... | strictlyPositive | +| SignAnalysis.cs:73:18:73:18 | access to parameter i | strictlyNegative | +| SignAnalysis.cs:74:38:74:38 | access to local variable x | strictlyPositive | +| SignAnalysis.cs:75:13:75:18 | ... = ... | strictlyNegative | +| SignAnalysis.cs:75:17:75:18 | +... | strictlyNegative | +| SignAnalysis.cs:75:18:75:18 | access to parameter i | strictlyNegative | +| SignAnalysis.cs:76:38:76:38 | access to local variable x | strictlyNegative | +| SignAnalysis.cs:77:17:77:27 | Int64 l = ... | strictlyNegative | +| SignAnalysis.cs:77:21:77:27 | (...) ... | strictlyNegative | +| SignAnalysis.cs:77:27:77:27 | access to parameter i | strictlyNegative | +| SignAnalysis.cs:78:38:78:38 | access to local variable l | strictlyNegative | +| SignAnalysis.cs:80:13:80:17 | ... = ... | strictlyNegative | +| SignAnalysis.cs:80:17:80:17 | access to parameter i | strictlyNegative | +| SignAnalysis.cs:81:13:81:13 | access to local variable x | strictlyNegative | +| SignAnalysis.cs:81:13:81:18 | ... + ... | strictlyNegative | +| SignAnalysis.cs:81:13:81:18 | ... += ... | strictlyNegative | +| SignAnalysis.cs:81:13:81:18 | ... = ... | strictlyNegative | +| SignAnalysis.cs:81:18:81:18 | access to parameter i | strictlyNegative | +| SignAnalysis.cs:82:38:82:38 | access to local variable x | strictlyNegative | +| SignAnalysis.cs:87:21:87:21 | access to parameter i | strictlyNegative | +| SignAnalysis.cs:87:25:87:25 | access to parameter j | strictlyPositive | +| SignAnalysis.cs:89:13:89:21 | ... = ... | strictlyNegative | +| SignAnalysis.cs:89:17:89:17 | access to parameter i | strictlyNegative | +| SignAnalysis.cs:89:17:89:21 | ... * ... | strictlyNegative | +| SignAnalysis.cs:89:21:89:21 | access to parameter j | strictlyPositive | +| SignAnalysis.cs:90:38:90:38 | access to local variable x | strictlyNegative | +| SignAnalysis.cs:91:13:91:21 | ... = ... | negative | +| SignAnalysis.cs:91:17:91:17 | access to parameter i | strictlyNegative | +| SignAnalysis.cs:91:17:91:21 | ... / ... | negative | +| SignAnalysis.cs:91:21:91:21 | access to parameter j | strictlyPositive | +| SignAnalysis.cs:92:38:92:38 | access to local variable x | negative | +| SignAnalysis.cs:93:13:93:21 | ... = ... | strictlyNegative | +| SignAnalysis.cs:93:17:93:17 | access to parameter i | strictlyNegative | +| SignAnalysis.cs:93:17:93:21 | ... - ... | strictlyNegative | +| SignAnalysis.cs:93:21:93:21 | access to parameter j | strictlyPositive | +| SignAnalysis.cs:94:38:94:38 | access to local variable x | strictlyNegative | +| SignAnalysis.cs:95:13:95:21 | ... = ... | negative | +| SignAnalysis.cs:95:17:95:17 | access to parameter i | strictlyNegative | +| SignAnalysis.cs:95:17:95:21 | ... % ... | negative | +| SignAnalysis.cs:95:21:95:21 | access to parameter j | strictlyPositive | +| SignAnalysis.cs:96:38:96:38 | access to local variable x | negative | +| SignAnalysis.cs:97:21:97:21 | access to parameter i | strictlyNegative | +| SignAnalysis.cs:97:25:97:25 | access to parameter j | strictlyPositive | +| SignAnalysis.cs:104:13:104:20 | Single f = ... | strictlyPositive | +| SignAnalysis.cs:104:17:104:20 | 4.2 | strictlyPositive | +| SignAnalysis.cs:105:34:105:34 | access to local variable f | strictlyPositive | +| SignAnalysis.cs:106:13:106:19 | Double d = ... | strictlyPositive | +| SignAnalysis.cs:106:17:106:19 | 4.2 | strictlyPositive | +| SignAnalysis.cs:107:34:107:34 | access to local variable d | strictlyPositive | +| SignAnalysis.cs:108:13:108:21 | Decimal de = ... | strictlyPositive | +| SignAnalysis.cs:108:18:108:21 | 4.2 | strictlyPositive | +| SignAnalysis.cs:109:34:109:35 | access to local variable de | strictlyPositive | +| SignAnalysis.cs:110:13:110:13 | access to local variable c | positive | +| SignAnalysis.cs:110:13:110:19 | Char c = ... | strictlyPositive | +| SignAnalysis.cs:110:17:110:19 | a | strictlyPositive | +| SignAnalysis.cs:111:34:111:34 | access to local variable c | strictlyPositive | +| SignAnalysis.cs:120:9:120:10 | access to field f0 | positive | +| SignAnalysis.cs:120:9:120:12 | ...++ | positive | +| SignAnalysis.cs:121:34:121:35 | access to field f0 | strictlyPositive | +| SignAnalysis.cs:122:9:122:10 | access to field f0 | positive | +| SignAnalysis.cs:129:9:129:16 | ... = ... | strictlyNegative | +| SignAnalysis.cs:129:14:129:16 | -... | strictlyNegative | +| SignAnalysis.cs:129:15:129:16 | 10 | strictlyPositive | +| SignAnalysis.cs:141:34:141:34 | access to local variable i | strictlyPositive | +| SignAnalysis.cs:148:20:148:20 | access to parameter x | strictlyNegative | +| SignAnalysis.cs:151:18:151:18 | 1 | strictlyPositive | +| SignAnalysis.cs:153:20:153:20 | access to parameter y | strictlyPositive | +| SignAnalysis.cs:156:18:156:19 | -... | strictlyNegative | +| SignAnalysis.cs:156:19:156:19 | 1 | strictlyPositive | +| SignAnalysis.cs:158:20:158:20 | access to parameter y | strictlyNegative | +| SignAnalysis.cs:161:13:161:13 | access to parameter x | positive | +| SignAnalysis.cs:163:20:163:20 | access to parameter y | strictlyPositive | +| SignAnalysis.cs:166:22:166:22 | 1 | strictlyPositive | +| SignAnalysis.cs:169:20:169:20 | access to parameter y | strictlyPositive | +| SignAnalysis.cs:177:13:177:17 | Int32 i = ... | strictlyPositive | +| SignAnalysis.cs:177:17:177:17 | 1 | strictlyPositive | +| SignAnalysis.cs:178:13:178:13 | access to local variable i | strictlyPositive | +| SignAnalysis.cs:180:38:180:38 | access to local variable i | strictlyPositive | +| SignAnalysis.cs:186:34:186:44 | access to property Length | positive | +| SignAnalysis.cs:187:16:187:36 | 3 | strictlyPositive | +| SignAnalysis.cs:187:28:187:28 | 1 | strictlyPositive | +| SignAnalysis.cs:187:31:187:31 | 2 | strictlyPositive | +| SignAnalysis.cs:187:34:187:34 | 3 | strictlyPositive | +| SignAnalysis.cs:188:34:188:44 | access to property Length | strictlyPositive | +| SignAnalysis.cs:189:34:189:45 | call to method Count | positive | +| SignAnalysis.cs:190:34:190:55 | call to method Count | positive | +| SignAnalysis.cs:190:54:190:54 | 1 | strictlyPositive | +| SignAnalysis.cs:193:34:193:41 | access to property Length | positive | +| SignAnalysis.cs:196:34:196:51 | call to method Count | positive | +| SignAnalysis.cs:198:17:198:59 | 2 | strictlyPositive | +| SignAnalysis.cs:198:17:198:59 | 3 | strictlyPositive | +| SignAnalysis.cs:198:32:198:32 | 1 | strictlyPositive | +| SignAnalysis.cs:198:35:198:35 | 1 | strictlyPositive | +| SignAnalysis.cs:198:42:198:42 | 1 | strictlyPositive | +| SignAnalysis.cs:198:45:198:45 | 2 | strictlyPositive | +| SignAnalysis.cs:198:52:198:52 | 1 | strictlyPositive | +| SignAnalysis.cs:198:55:198:55 | 3 | strictlyPositive | +| SignAnalysis.cs:199:34:199:41 | access to property Length | strictlyPositive | +| SignAnalysis.cs:206:38:206:38 | access to parameter i | strictlyPositive | +| SignAnalysis.cs:210:38:210:38 | access to parameter i | negative | +| SignAnalysis.cs:219:38:219:38 | access to parameter i | strictlyPositive | +| SignAnalysis.cs:223:17:223:17 | access to parameter i | negative | +| SignAnalysis.cs:225:42:225:42 | access to parameter i | strictlyNegative | +| SignAnalysis.cs:236:38:236:38 | access to parameter i | strictlyPositive | +| SignAnalysis.cs:240:17:240:17 | access to parameter i | negative | +| SignAnalysis.cs:242:42:242:42 | access to parameter i | strictlyNegative | +| SignAnalysis.cs:257:17:257:17 | access to parameter i | positive | +| SignAnalysis.cs:257:17:257:19 | ...-- | positive | +| SignAnalysis.cs:260:38:260:38 | access to parameter i | strictlyNegative | +| SignAnalysis.cs:267:17:267:17 | access to parameter j | strictlyPositive | +| SignAnalysis.cs:267:17:267:19 | ...-- | strictlyPositive | +| SignAnalysis.cs:268:42:268:42 | access to parameter j | positive | +| SignAnalysis.cs:270:38:270:38 | access to parameter j | negative | +| SignAnalysis.cs:277:17:277:17 | access to parameter k | strictlyPositive | +| SignAnalysis.cs:277:17:277:19 | ...-- | strictlyPositive | +| SignAnalysis.cs:278:42:278:42 | access to parameter k | positive | +| SignAnalysis.cs:280:21:280:21 | access to parameter k | positive | +| SignAnalysis.cs:280:26:280:26 | 5 | strictlyPositive | +| SignAnalysis.cs:295:38:295:38 | access to parameter i | strictlyPositive | +| SignAnalysis.cs:300:27:300:28 | -... | strictlyNegative | +| SignAnalysis.cs:300:28:300:28 | 1 | strictlyPositive | +| SignAnalysis.cs:303:38:303:38 | access to local variable x | strictlyNegative | +| SignAnalysis.cs:306:21:306:22 | -... | strictlyNegative | +| SignAnalysis.cs:306:22:306:22 | 1 | strictlyPositive | +| SignAnalysis.cs:309:38:309:38 | access to local variable x | strictlyNegative | +| SignAnalysis.cs:315:13:315:15 | access to local variable min | positive | +| SignAnalysis.cs:316:13:316:15 | access to local variable max | positive | +| SignAnalysis.cs:316:13:316:31 | Char max = ... | strictlyPositive | +| SignAnalysis.cs:316:19:316:31 | access to constant MaxValue | strictlyPositive | +| SignAnalysis.cs:317:13:317:23 | Int32 c = ... | strictlyPositive | +| SignAnalysis.cs:317:17:317:23 | ... + ... | strictlyPositive | +| SignAnalysis.cs:317:23:317:23 | 1 | strictlyPositive | +| SignAnalysis.cs:318:34:318:34 | access to local variable c | strictlyPositive | +| SignAnalysis.cs:319:9:319:19 | ... = ... | strictlyNegative | +| SignAnalysis.cs:319:13:319:19 | ... - ... | strictlyNegative | +| SignAnalysis.cs:319:19:319:19 | 1 | strictlyPositive | +| SignAnalysis.cs:320:34:320:34 | access to local variable c | strictlyNegative | +| SignAnalysis.cs:321:9:321:19 | ... = ... | strictlyPositive | +| SignAnalysis.cs:321:13:321:15 | (...) ... | strictlyPositive | +| SignAnalysis.cs:321:13:321:15 | access to local variable max | strictlyPositive | +| SignAnalysis.cs:321:13:321:19 | ... + ... | strictlyPositive | +| SignAnalysis.cs:321:19:321:19 | 1 | strictlyPositive | +| SignAnalysis.cs:322:34:322:34 | access to local variable c | strictlyPositive | +| SignAnalysis.cs:329:17:329:26 | Int32 x = ... | strictlyPositive | +| SignAnalysis.cs:329:21:329:21 | access to parameter v | strictlyPositive | +| SignAnalysis.cs:329:21:329:26 | ... ?? ... | strictlyPositive | +| SignAnalysis.cs:329:26:329:26 | 1 | strictlyPositive | +| SignAnalysis.cs:330:38:330:38 | access to local variable x | strictlyPositive | +| SignAnalysis.cs:335:17:335:26 | Int32 x = ... | strictlyPositive | +| SignAnalysis.cs:335:21:335:26 | ... ?? ... | strictlyPositive | +| SignAnalysis.cs:335:26:335:26 | 1 | strictlyPositive | +| SignAnalysis.cs:336:38:336:38 | access to local variable x | strictlyPositive | +| SignAnalysis.cs:341:17:341:26 | Int32 x = ... | negative | +| SignAnalysis.cs:341:21:341:21 | access to parameter v | strictlyNegative | +| SignAnalysis.cs:341:21:341:26 | ... ?? ... | negative | +| SignAnalysis.cs:342:38:342:38 | access to local variable x | negative | +| SignAnalysis.cs:348:62:348:62 | 5 | strictlyPositive | +| SignAnalysis.cs:351:38:351:38 | access to local variable i | strictlyNegative | +| SignAnalysis.cs:357:13:357:13 | access to parameter i | positive | +| SignAnalysis.cs:359:38:359:38 | access to parameter i | strictlyPositive | +| SignAnalysis.cs:371:38:371:38 | access to local variable y | strictlyNegative | +| SignAnalysis.cs:377:16:377:17 | access to local variable dp | positive | +| SignAnalysis.cs:377:16:377:22 | Single* dp = ... | positive | +| SignAnalysis.cs:377:21:377:22 | &... | positive | +| SignAnalysis.cs:378:18:378:19 | access to local variable dp | positive | +| SignAnalysis.cs:381:38:381:38 | access to local variable x | strictlyNegative | +| SignAnalysis.cs:385:50:385:99 | access to constant Explicit | strictlyPositive | +| SignAnalysis.cs:385:109:385:110 | 15 | strictlyPositive | +| SignAnalysis.cs:390:13:390:32 | Int32 x = ... | strictlyPositive | +| SignAnalysis.cs:390:17:390:32 | sizeof(..) | strictlyPositive | +| SignAnalysis.cs:391:34:391:34 | access to local variable x | strictlyPositive | +| SignAnalysis.cs:396:13:400:9 | Int32 x = ... | positive | +| SignAnalysis.cs:396:17:400:9 | ... switch { ... } | positive | +| SignAnalysis.cs:399:13:399:18 | ... => ... | strictlyPositive | +| SignAnalysis.cs:399:18:399:18 | 2 | strictlyPositive | +| SignAnalysis.cs:401:34:401:34 | access to local variable x | positive | +| SignAnalysis.cs:406:13:406:17 | Int32 i = ... | strictlyPositive | +| SignAnalysis.cs:406:17:406:17 | 1 | strictlyPositive | +| SignAnalysis.cs:410:35:410:35 | access to local variable i | strictlyPositive | +| SignAnalysis.cs:414:13:414:13 | access to local variable i | strictlyPositive | +| SignAnalysis.cs:415:31:415:31 | access to local variable i | strictlyPositive | +| SignAnalysis.cs:424:31:424:31 | access to local variable x | strictlyNegative | +| SignAnalysis.cs:428:19:428:19 | access to constant A | strictlyPositive | +| SignAnalysis.cs:428:19:428:24 | ... = ... | strictlyPositive | +| SignAnalysis.cs:428:23:428:24 | 12 | strictlyPositive | +| SignAnalysis.cs:434:38:434:38 | access to local variable i | strictlyNegative | +| SignAnalysis.cs:440:23:440:25 | access to parameter src | positive | +| SignAnalysis.cs:440:29:440:31 | access to parameter dst | positive | +| SignAnalysis.cs:443:38:443:38 | access to local variable x | strictlyNegative | +| SignAnalysis.cs:446:31:446:32 | 10 | strictlyPositive | +| SignAnalysis.cs:448:22:448:23 | access to local variable to | positive | +| SignAnalysis.cs:448:22:448:29 | Byte* to = ... | positive | +| SignAnalysis.cs:448:27:448:29 | (...) ... | positive | +| SignAnalysis.cs:450:38:450:44 | (...) ... | positive | +| SignAnalysis.cs:450:43:450:44 | access to local variable to | positive | +| SignAnalysis.cs:454:30:454:30 | 1 | strictlyPositive | +| SignAnalysis.cs:454:30:454:30 | (...) ... | strictlyPositive | +| SignAnalysis.cs:457:14:457:27 | Int64 l = ... | positive | +| SignAnalysis.cs:457:18:457:27 | (...) ... | positive | +| SignAnalysis.cs:457:18:457:27 | call to method Unsigned | positive | +| SignAnalysis.cs:458:13:458:13 | access to local variable l | positive | +| SignAnalysis.cs:460:38:460:38 | access to local variable l | strictlyPositive | +| SignAnalysis.cs:463:14:463:14 | access to local variable x | positive | +| SignAnalysis.cs:463:14:463:24 | UInt32 x = ... | positive | +| SignAnalysis.cs:463:18:463:24 | (...) ... | positive | +| SignAnalysis.cs:464:9:464:9 | access to local variable x | positive | +| SignAnalysis.cs:464:9:464:11 | ...++ | positive | +| SignAnalysis.cs:465:34:465:34 | access to local variable x | strictlyPositive | diff --git a/csharp/ql/test/library-tests/dataflow/signanalysis/SignAnalysis.ql b/csharp/ql/test/library-tests/dataflow/signanalysis/SignAnalysis.ql new file mode 100644 index 00000000000..4350e8f1742 --- /dev/null +++ b/csharp/ql/test/library-tests/dataflow/signanalysis/SignAnalysis.ql @@ -0,0 +1,21 @@ +import csharp +import semmle.code.csharp.dataflow.SignAnalysis + +string getASignString(Expr e) { + positive(e) and + not strictlyPositive(e) and + result = "positive" + or + negative(e) and + not strictlyNegative(e) and + result = "negative" + or + strictlyPositive(e) and + result = "strictlyPositive" + or + strictlyNegative(e) and + result = "strictlyNegative" +} + +from Expr e +select e, strictconcat(string s | s = getASignString(e) | s, " ") diff --git a/csharp/ql/test/library-tests/definitions/PrintAst.expected b/csharp/ql/test/library-tests/definitions/PrintAst.expected index 18c13fe61bd..5543a3363d5 100644 --- a/csharp/ql/test/library-tests/definitions/PrintAst.expected +++ b/csharp/ql/test/library-tests/definitions/PrintAst.expected @@ -8,9 +8,13 @@ definitions.cs: # 9| 4: [BlockStmt] {...} # 13| 2: [Enum] Enumeration # 15| 5: [Field] e1 -# 15| 1: [MemberConstantAccess] access to constant e1 +# 15| 1: [AssignExpr] ... = ... +# 15| 0: [IntLiteral] 1 +# 15| 1: [MemberConstantAccess] access to constant e1 # 15| 6: [Field] e2 -# 15| 1: [MemberConstantAccess] access to constant e2 +# 15| 1: [AssignExpr] ... = ... +# 15| 0: [IntLiteral] 2 +# 15| 1: [MemberConstantAccess] access to constant e2 # 15| 7: [Field] e3 # 18| 3: [Class] C1 # 20| 4: [InstanceConstructor] C1 diff --git a/csharp/ql/test/library-tests/enums/Enums11.expected b/csharp/ql/test/library-tests/enums/Enums11.expected new file mode 100644 index 00000000000..715beb400ff --- /dev/null +++ b/csharp/ql/test/library-tests/enums/Enums11.expected @@ -0,0 +1,5 @@ +| enums.cs:28:18:28:18 | (...) ... | 1 | +| enums.cs:29:20:29:20 | (...) ... | 2 | +| enums.cs:30:20:30:20 | (...) ... | 4 | +| enums.cs:38:17:38:18 | 10 | 10 | +| enums.cs:40:23:40:32 | ... + ... | 11 | diff --git a/csharp/ql/test/library-tests/enums/Enums11.ql b/csharp/ql/test/library-tests/enums/Enums11.ql new file mode 100644 index 00000000000..36b2c005a21 --- /dev/null +++ b/csharp/ql/test/library-tests/enums/Enums11.ql @@ -0,0 +1,12 @@ +/** + * @name Test for enums + */ + +import csharp + +from Expr e +where + exists(Assignment a | a.getRValue() = e | + a.getParent().(Field).getDeclaringType() instanceof Enum + ) +select e, e.getValue() diff --git a/csharp/ql/test/library-tests/enums/PrintAst.expected b/csharp/ql/test/library-tests/enums/PrintAst.expected index 05101b6ca19..446b4c01856 100644 --- a/csharp/ql/test/library-tests/enums/PrintAst.expected +++ b/csharp/ql/test/library-tests/enums/PrintAst.expected @@ -11,18 +11,35 @@ enums.cs: # 23| 3: [Enum] E # 25| 4: [Enum] ValueColor # 28| 5: [Field] OneRed -# 28| 1: [MemberConstantAccess] access to constant OneRed +# 28| 1: [AssignExpr] ... = ... +# 28| 0: [CastExpr] (...) ... +# 28| 0: [IntLiteral] 1 +# 28| 1: [MemberConstantAccess] access to constant OneRed # 29| 6: [Field] TwoGreen -# 29| 1: [MemberConstantAccess] access to constant TwoGreen +# 29| 1: [AssignExpr] ... = ... +# 29| 0: [CastExpr] (...) ... +# 29| 0: [IntLiteral] 2 +# 29| 1: [MemberConstantAccess] access to constant TwoGreen # 30| 7: [Field] FourBlue -# 30| 1: [MemberConstantAccess] access to constant FourBlue +# 30| 1: [AssignExpr] ... = ... +# 30| 0: [CastExpr] (...) ... +# 30| 0: [IntLiteral] 4 +# 30| 1: [MemberConstantAccess] access to constant FourBlue # 34| 5: [Enum] SparseColor # 37| 5: [Field] Red # 38| 6: [Field] Green -# 38| 1: [MemberConstantAccess] access to constant Green +# 38| 1: [AssignExpr] ... = ... +# 38| 0: [IntLiteral] 10 +# 38| 1: [MemberConstantAccess] access to constant Green # 39| 7: [Field] Blue # 40| 8: [Field] AnotherBlue -# 40| 1: [MemberConstantAccess] access to constant AnotherBlue +# 40| 1: [AssignExpr] ... = ... +# 40| 0: [AddExpr] ... + ... +# 40| 0: [CastExpr] (...) ... +# 40| 0: [MemberConstantAccess] access to constant Blue +# 40| 1: [CastExpr] (...) ... +# 40| 0: [MemberConstantAccess] access to constant Red +# 40| 1: [MemberConstantAccess] access to constant AnotherBlue # 44| 6: [Class] Test # 47| 5: [Method] Main # 48| 4: [BlockStmt] {...} diff --git a/csharp/ql/test/library-tests/enums/enums.cs b/csharp/ql/test/library-tests/enums/enums.cs index df7df79c066..784d4f64114 100644 --- a/csharp/ql/test/library-tests/enums/enums.cs +++ b/csharp/ql/test/library-tests/enums/enums.cs @@ -37,7 +37,7 @@ namespace Enums Red, Green = 10, Blue, - AnotherBlue = Blue + AnotherBlue = Blue + Red } diff --git a/csharp/ql/test/library-tests/unsafe/PrintAst.expected b/csharp/ql/test/library-tests/unsafe/PrintAst.expected index b4e9ce56c64..079719907c7 100644 --- a/csharp/ql/test/library-tests/unsafe/PrintAst.expected +++ b/csharp/ql/test/library-tests/unsafe/PrintAst.expected @@ -109,7 +109,8 @@ unsafe.cs: # 36| 1: [LocalVariableAccess] access to local variable data # 37| 1: [FixedStmt] fixed(...) { ... } # 37| -1: [LocalVariableDeclAndInitExpr] Int32* p = ... -# 37| 0: [LocalVariableAccess] access to local variable data +# 37| 0: [CastExpr] (...) ... +# 37| 0: [LocalVariableAccess] access to local variable data # 37| 1: [LocalVariableAccess] access to local variable p # 38| 0: [BlockStmt] {...} # 44| 2: [Class] SafeClass diff --git a/csharp/ql/test/query-tests/Language Abuse/UselessCastToSelf/UselessCastToSelf.cs b/csharp/ql/test/query-tests/Language Abuse/UselessCastToSelf/UselessCastToSelf.cs index 47df3468b4a..e36143d6b40 100644 --- a/csharp/ql/test/query-tests/Language Abuse/UselessCastToSelf/UselessCastToSelf.cs +++ b/csharp/ql/test/query-tests/Language Abuse/UselessCastToSelf/UselessCastToSelf.cs @@ -18,4 +18,13 @@ class Test var good6 = (Action)(delegate (int x) { }); var good7 = (Action)((int x) => { }); } + + enum Enum + { + A = 2, + B = 1 | A, + C = 1 | (int)A, // BAD + D = 9 | (32 << A), + E = 9 | (32 << (int)A) // BAD + } } diff --git a/csharp/ql/test/query-tests/Language Abuse/UselessCastToSelf/UselessCastToSelf.expected b/csharp/ql/test/query-tests/Language Abuse/UselessCastToSelf/UselessCastToSelf.expected index 28dcbbb9f6e..f5c4b708253 100644 --- a/csharp/ql/test/query-tests/Language Abuse/UselessCastToSelf/UselessCastToSelf.expected +++ b/csharp/ql/test/query-tests/Language Abuse/UselessCastToSelf/UselessCastToSelf.expected @@ -1,3 +1,5 @@ | UselessCastToSelf.cs:8:20:8:25 | (...) ... | This cast is redundant because the expression already has type Int32. | | UselessCastToSelf.cs:9:20:9:29 | (...) ... | This cast is redundant because the expression already has type Test. | | UselessCastToSelf.cs:10:20:10:31 | ... as ... | This cast is redundant because the expression already has type Test. | +| UselessCastToSelf.cs:26:17:26:22 | (...) ... | This cast is redundant because the expression already has type Int32. | +| UselessCastToSelf.cs:28:24:28:29 | (...) ... | This cast is redundant because the expression already has type Int32. | diff --git a/csharp/ql/test/query-tests/Likely Bugs/SelfAssignment/selfassigns.cs b/csharp/ql/test/query-tests/Likely Bugs/SelfAssignment/selfassigns.cs index 3553606275f..6d1387a5e1d 100644 --- a/csharp/ql/test/query-tests/Likely Bugs/SelfAssignment/selfassigns.cs +++ b/csharp/ql/test/query-tests/Likely Bugs/SelfAssignment/selfassigns.cs @@ -82,4 +82,11 @@ class SelfAssigns : Super this.Self.Self.Self.StringProp = Self.Self.Self.StringProp; intArray[1] = this.intArray[1 + 0]; } + + enum Enum + { + X = 42, + Y = 100, + Z + } } diff --git a/csharp/ql/test/query-tests/Nullness/EqualityCheck.expected b/csharp/ql/test/query-tests/Nullness/EqualityCheck.expected index 5f739e93c2b..dc70346eea5 100644 --- a/csharp/ql/test/query-tests/Nullness/EqualityCheck.expected +++ b/csharp/ql/test/query-tests/Nullness/EqualityCheck.expected @@ -191,8 +191,11 @@ | E.cs:85:18:85:29 | ... != ... | false | E.cs:85:18:85:21 | access to parameter vals | E.cs:85:26:85:29 | null | | E.cs:85:18:85:29 | ... != ... | false | E.cs:85:26:85:29 | null | E.cs:85:18:85:21 | access to parameter vals | | E.cs:90:17:90:27 | access to local variable switchguard | match access to constant MY_CONST_A | E.cs:90:17:90:27 | access to local variable switchguard | E.cs:92:18:92:27 | access to constant MY_CONST_A | +| E.cs:90:17:90:27 | access to local variable switchguard | match access to constant MY_CONST_A | E.cs:92:18:92:27 | access to constant MY_CONST_A | E.cs:90:17:90:27 | access to local variable switchguard | | E.cs:90:17:90:27 | access to local variable switchguard | match access to constant MY_CONST_B | E.cs:90:17:90:27 | access to local variable switchguard | E.cs:97:18:97:27 | access to constant MY_CONST_B | +| E.cs:90:17:90:27 | access to local variable switchguard | match access to constant MY_CONST_B | E.cs:97:18:97:27 | access to constant MY_CONST_B | E.cs:90:17:90:27 | access to local variable switchguard | | E.cs:90:17:90:27 | access to local variable switchguard | match access to constant MY_CONST_C | E.cs:90:17:90:27 | access to local variable switchguard | E.cs:95:18:95:27 | access to constant MY_CONST_C | +| E.cs:90:17:90:27 | access to local variable switchguard | match access to constant MY_CONST_C | E.cs:95:18:95:27 | access to constant MY_CONST_C | E.cs:90:17:90:27 | access to local variable switchguard | | E.cs:126:21:126:29 | ... == ... | true | E.cs:126:21:126:24 | access to local variable step | E.cs:126:29:126:29 | 0 | | E.cs:126:21:126:29 | ... == ... | true | E.cs:126:29:126:29 | 0 | E.cs:126:21:126:24 | access to local variable step | | E.cs:153:13:153:24 | ... != ... | false | E.cs:153:13:153:16 | access to local variable obj2 | E.cs:153:21:153:24 | null | @@ -216,6 +219,7 @@ | E.cs:293:13:293:24 | ... == ... | true | E.cs:293:15:293:19 | call to method M2 | E.cs:293:24:293:24 | (...) ... | | E.cs:293:13:293:24 | ... == ... | true | E.cs:293:24:293:24 | (...) ... | E.cs:293:15:293:19 | call to method M2 | | E.cs:321:13:321:30 | ... is ... | true | E.cs:321:14:321:21 | ... ?? ... | E.cs:321:27:321:30 | null | +| E.cs:321:13:321:30 | ... is ... | true | E.cs:321:27:321:30 | null | E.cs:321:14:321:21 | ... ?? ... | | E.cs:355:13:355:21 | dynamic call to operator != | false | E.cs:355:13:355:13 | access to local variable x | E.cs:355:18:355:21 | null | | E.cs:355:13:355:21 | dynamic call to operator != | false | E.cs:355:18:355:21 | null | E.cs:355:13:355:13 | access to local variable x | | E.cs:362:13:362:29 | ... != ... | false | E.cs:362:13:362:13 | access to local variable x | E.cs:362:18:362:29 | (...) ... | diff --git a/csharp/ql/test/query-tests/Security Features/CWE-327/InsufficientKeySize/InsufficientKeySize.cs b/csharp/ql/test/query-tests/Security Features/CWE-327/InsufficientKeySize/InsufficientKeySize.cs index 4ec825ddd63..90be31d0a07 100644 --- a/csharp/ql/test/query-tests/Security Features/CWE-327/InsufficientKeySize/InsufficientKeySize.cs +++ b/csharp/ql/test/query-tests/Security Features/CWE-327/InsufficientKeySize/InsufficientKeySize.cs @@ -13,18 +13,18 @@ public class InsufficientKeySize // GOOD: Key size is greater than 128 new RC2CryptoServiceProvider().EffectiveKeySize = 256; - // BAD: Key size is less than 1024. + // BAD: Key size is less than 2048. DSACryptoServiceProvider dsaBad = new DSACryptoServiceProvider(512); - // GOOD: Key size defaults to 1024. + // GOOD: Key size defaults to 2048. DSACryptoServiceProvider dsaGood1 = new DSACryptoServiceProvider(); - // GOOD: Key size is greater than 1024. + // GOOD: Key size is greater than 2048. DSACryptoServiceProvider dsaGood2 = new DSACryptoServiceProvider(2048); - // BAD: Key size is less than 1024. + // BAD: Key size is less than 2048. RSACryptoServiceProvider rsaBad = new RSACryptoServiceProvider(512); - // GOOD: Key size defaults to 1024. + // GOOD: Key size defaults to 2048. RSACryptoServiceProvider rsaGood1 = new RSACryptoServiceProvider(); - // GOOD: Key size is greater than 1024. + // GOOD: Key size is greater than 2048. RSACryptoServiceProvider rsaGood2 = new RSACryptoServiceProvider(2048); } } diff --git a/csharp/ql/test/query-tests/Security Features/CWE-327/InsufficientKeySize/InsufficientKeySize.expected b/csharp/ql/test/query-tests/Security Features/CWE-327/InsufficientKeySize/InsufficientKeySize.expected index dc03302c7f3..feb87da77d2 100644 --- a/csharp/ql/test/query-tests/Security Features/CWE-327/InsufficientKeySize/InsufficientKeySize.expected +++ b/csharp/ql/test/query-tests/Security Features/CWE-327/InsufficientKeySize/InsufficientKeySize.expected @@ -1,3 +1,3 @@ | InsufficientKeySize.cs:10:9:10:60 | ... = ... | Key size should be at least 128 bits for RC2 encryption. | -| InsufficientKeySize.cs:17:43:17:75 | object creation of type DSACryptoServiceProvider | Key size should be at least 1024 bits for DSA encryption. | -| InsufficientKeySize.cs:24:43:24:75 | object creation of type RSACryptoServiceProvider | Key size should be at least 1024 bits for RSA encryption. | +| InsufficientKeySize.cs:17:43:17:75 | object creation of type DSACryptoServiceProvider | Key size should be at least 2048 bits for DSA encryption. | +| InsufficientKeySize.cs:24:43:24:75 | object creation of type RSACryptoServiceProvider | Key size should be at least 2048 bits for RSA encryption. | diff --git a/csharp/tools/autobuild.cmd b/csharp/tools/autobuild.cmd index 418ba37a972..7d24060ff46 100644 --- a/csharp/tools/autobuild.cmd +++ b/csharp/tools/autobuild.cmd @@ -1,7 +1,4 @@ @echo off -rem The autobuilder is already being traced -set CODEQL_AUTOBUILDER_CSHARP_NO_INDEXING=true - type NUL && "%CODEQL_EXTRACTOR_CSHARP_ROOT%/tools/%CODEQL_PLATFORM%/Semmle.Autobuild.CSharp.exe" exit /b %ERRORLEVEL% diff --git a/csharp/tools/autobuild.sh b/csharp/tools/autobuild.sh index 341f64af50b..e8e007a10d7 100755 --- a/csharp/tools/autobuild.sh +++ b/csharp/tools/autobuild.sh @@ -7,8 +7,4 @@ if [ "$CODEQL_PLATFORM" != "linux64" ] && [ "$CODEQL_PLATFORM" != "osx64" ] ; th exit 1 fi -# The autobuilder is already being traced -CODEQL_AUTOBUILDER_CSHARP_NO_INDEXING="true" -export CODEQL_AUTOBUILDER_CSHARP_NO_INDEXING - "$CODEQL_EXTRACTOR_CSHARP_ROOT/tools/$CODEQL_PLATFORM/Semmle.Autobuild.CSharp" || exit $? diff --git a/csharp/tools/pre-finalize.cmd b/csharp/tools/pre-finalize.cmd index a8a5a6bc68a..b6e7abab41b 100644 --- a/csharp/tools/pre-finalize.cmd +++ b/csharp/tools/pre-finalize.cmd @@ -9,4 +9,7 @@ type NUL && "%CODEQL_DIST%\codeql" database index-files ^ --language xml ^ -- ^ "%CODEQL_EXTRACTOR_CSHARP_WIP_DATABASE%" +IF %ERRORLEVEL% NEQ 0 exit /b %ERRORLEVEL% + +type NUL && "%CODEQL_JAVA_HOME%\bin\java.exe" -jar "%CODEQL_EXTRACTOR_CSHARP_ROOT%\tools\extractor-asp.jar" . exit /b %ERRORLEVEL% diff --git a/csharp/tools/pre-finalize.sh b/csharp/tools/pre-finalize.sh index ed12b3b1d72..0e8ab78c9fd 100755 --- a/csharp/tools/pre-finalize.sh +++ b/csharp/tools/pre-finalize.sh @@ -10,4 +10,6 @@ set -eu --size-limit 10m \ --language xml \ -- \ - "$CODEQL_EXTRACTOR_CSHARP_WIP_DATABASE" \ No newline at end of file + "$CODEQL_EXTRACTOR_CSHARP_WIP_DATABASE" + +"$CODEQL_JAVA_HOME/bin/java" -jar "$CODEQL_EXTRACTOR_CSHARP_ROOT/tools/extractor-asp.jar" . diff --git a/java/change-notes/2020-09-22-hibernate-sql-sinks.md b/java/change-notes/2020-09-22-hibernate-sql-sinks.md new file mode 100644 index 00000000000..1d469825f8e --- /dev/null +++ b/java/change-notes/2020-09-22-hibernate-sql-sinks.md @@ -0,0 +1,3 @@ +lgtm,codescanning +* Support for the [Hibernate ORM](https://hibernate.org/orm/) library (specifically, its Query + creation methods) has been improved, which may lead to more results from the security queries. diff --git a/java/ql/src/Security/CWE/CWE-022/TaintedPath.qhelp b/java/ql/src/Security/CWE/CWE-022/TaintedPath.qhelp index 29bab0e42d2..b32d3c65039 100644 --- a/java/ql/src/Security/CWE/CWE-022/TaintedPath.qhelp +++ b/java/ql/src/Security/CWE/CWE-022/TaintedPath.qhelp @@ -39,7 +39,7 @@ giving them access to all the system's passwords.

  • OWASP: -Path Traversal. +Path Traversal.
    • diff --git a/java/ql/src/Security/CWE/CWE-022/ZipSlip.qhelp b/java/ql/src/Security/CWE/CWE-022/ZipSlip.qhelp index 97c389508b3..adea1b89c49 100644 --- a/java/ql/src/Security/CWE/CWE-022/ZipSlip.qhelp +++ b/java/ql/src/Security/CWE/CWE-022/ZipSlip.qhelp @@ -63,7 +63,7 @@ Snyk:
  • OWASP: -Path Traversal. +Path Traversal.
    • diff --git a/java/ql/src/Security/CWE/CWE-209/StackTraceExposure.qhelp b/java/ql/src/Security/CWE/CWE-209/StackTraceExposure.qhelp index b772b7e3c57..d0d7e1b0e50 100644 --- a/java/ql/src/Security/CWE/CWE-209/StackTraceExposure.qhelp +++ b/java/ql/src/Security/CWE/CWE-209/StackTraceExposure.qhelp @@ -38,7 +38,7 @@ information.

    -
  • OWASP: Information Leak.
    • +
  • OWASP: Improper Error Handling.
  • CERT Java Coding Standard: ERR01-J. diff --git a/java/ql/src/experimental/Security/CWE/CWE-643/XPathInjection.qhelp b/java/ql/src/experimental/Security/CWE/CWE-643/XPathInjection.qhelp index 555a6f7fb95..91d110b80aa 100644 --- a/java/ql/src/experimental/Security/CWE/CWE-643/XPathInjection.qhelp +++ b/java/ql/src/experimental/Security/CWE/CWE-643/XPathInjection.qhelp @@ -38,7 +38,7 @@ The fifth example is a dom4j XPath injection example. -
  • OWASP: Testing for XPath Injection.
    • +
  • OWASP: Testing for XPath Injection.
  • OWASP: XPath Injection.
    •  diff --git a/java/ql/src/semmle/code/java/Expr.qll b/java/ql/src/semmle/code/java/Expr.qll index d79a82acbad..50f4a210d60 100755 --- a/java/ql/src/semmle/code/java/Expr.qll +++ b/java/ql/src/semmle/code/java/Expr.qll @@ -1912,14 +1912,10 @@ private module Qualifier { } /** An expression that assigns a value to a field. */ -class FieldWrite extends FieldAccess { - FieldWrite() { exists(Field f | f = getVariable() and isLValue()) } -} +class FieldWrite extends FieldAccess, LValue { } /** An expression that reads a field. */ -class FieldRead extends FieldAccess { - FieldRead() { exists(Field f | f = getVariable() and isRValue()) } -} +class FieldRead extends FieldAccess, RValue { } private predicate hasInstantiation(RefType t) { t instanceof TypeVariable or diff --git a/java/ql/src/semmle/code/java/PrintAst.qll b/java/ql/src/semmle/code/java/PrintAst.qll index a3f4807661d..4e219603024 100644 --- a/java/ql/src/semmle/code/java/PrintAst.qll +++ b/java/ql/src/semmle/code/java/PrintAst.qll @@ -113,6 +113,10 @@ private predicate locationSortKeys(Element ast, string file, int line, int colum */ private newtype TPrintAstNode = TElementNode(Element el) { shouldPrint(el, _) } or + TForInitNode(ForStmt fs) { shouldPrint(fs, _) and exists(fs.getAnInit()) } or + TLocalVarDeclNode(LocalVariableDeclExpr lvde) { + shouldPrint(lvde, _) and lvde.getParent() instanceof SingleLocalVarDeclParent + } or TAnnotationsNode(Annotatable ann) { shouldPrint(ann, _) and ann.hasAnnotation() and not partOfAnnotation(ann) } or @@ -221,6 +225,24 @@ abstract class ElementNode extends PrintAstNode, TElementNode { final Element getElement() { result = element } } +/** + * A node representing an `Expr` or a `Stmt`. + */ +class ExprStmtNode extends ElementNode { + ExprStmtNode() { element instanceof ExprOrStmt } + + override PrintAstNode getChild(int childIndex) { + exists(Element el | result.(ElementNode).getElement() = el | + el.(Expr).isNthChildOf(element, childIndex) + or + el.(Stmt).isNthChildOf(element, childIndex) + ) + } +} + +/** + * Holds if the given expression is part of an annotation. + */ private predicate partOfAnnotation(Expr e) { e instanceof Annotation or @@ -228,49 +250,123 @@ private predicate partOfAnnotation(Expr e) { partOfAnnotation(e.getParent()) } -private Expr getAnAnnotationChild(Expr e) { - partOfAnnotation(e) and - ( - result = e.(Annotation).getValue(_) +/** + * A node representing an `Expr` that is part of an annotation. + */ +final class AnnotationPartNode extends ExprStmtNode { + AnnotationPartNode() { partOfAnnotation(element) } + + override ElementNode getChild(int childIndex) { + result.getElement() = + rank[childIndex](Element ch, string file, int line, int column | + ch = getAnAnnotationChild() and locationSortKeys(ch, file, line, column) + | + ch order by file, line, column + ) + } + + private Expr getAnAnnotationChild() { + result = element.(Annotation).getValue(_) or - result = e.(ArrayInit).getAnInit() + result = element.(ArrayInit).getAnInit() or - result = e.(ArrayInit).(Annotatable).getAnAnnotation() - ) + result = element.(ArrayInit).(Annotatable).getAnAnnotation() + } } /** - * An node representing an `Expr` or a `Stmt`. + * A node representing a `LocalVariableDeclExpr`. */ -final class ExprStmtNode extends ElementNode { - ExprStmtNode() { element instanceof ExprOrStmt } +final class LocalVarDeclExprNode extends ExprStmtNode { + LocalVarDeclExprNode() { element instanceof LocalVariableDeclExpr } override PrintAstNode getChild(int childIndex) { - exists(Element el | result.(ElementNode).getElement() = el | - el.(Expr).isNthChildOf(element, childIndex) and - not partOfAnnotation(element) - or - el.(Stmt).isNthChildOf(element, childIndex) - or - childIndex = -4 and - el = element.(ClassInstanceExpr).getAnonymousClass() - or - childIndex = 0 and - el = element.(LocalClassDeclStmt).getLocalClass() - or - partOfAnnotation(element) and - el = - rank[childIndex](Element ch, string file, int line, int column | - ch = getAnAnnotationChild(element) and locationSortKeys(ch, file, line, column) - | - ch order by file, line, column - ) - ) + result = super.getChild(childIndex) or - exists(Element el | result.(AnnotationsNode).getAnnotated() = el | - childIndex = -2 and - el = element.(LocalVariableDeclExpr).getVariable() - ) + childIndex = -2 and + result.(AnnotationsNode).getAnnotated() = element.(LocalVariableDeclExpr).getVariable() + } +} + +/** + * A node representing a `ClassInstanceExpr`. + */ +final class ClassInstanceExprNode extends ExprStmtNode { + ClassInstanceExprNode() { element instanceof ClassInstanceExpr } + + override ElementNode getChild(int childIndex) { + result = super.getChild(childIndex) + or + childIndex = -4 and + result.getElement() = element.(ClassInstanceExpr).getAnonymousClass() + } +} + +/** + * A node representing a `LocalClassDeclStmt`. + */ +final class LocalClassDeclStmtNode extends ExprStmtNode { + LocalClassDeclStmtNode() { element instanceof LocalClassDeclStmt } + + override ElementNode getChild(int childIndex) { + result = super.getChild(childIndex) + or + childIndex = 0 and + result.getElement() = element.(LocalClassDeclStmt).getLocalClass() + } +} + +/** + * A node representing a `ForStmt`. + */ +final class ForStmtNode extends ExprStmtNode { + ForStmtNode() { element instanceof ForStmt } + + override PrintAstNode getChild(int childIndex) { + childIndex >= 1 and + result = super.getChild(childIndex) + or + childIndex = 0 and + result.(ForInitNode).getForStmt() = element + } +} + +/** + * An element that can be the parent of up to one `LocalVariableDeclExpr` for which we want + * to use a synthetic node to hold the variable declaration and its `TypeAccess`. + */ +private class SingleLocalVarDeclParent extends ExprOrStmt { + SingleLocalVarDeclParent() { + this instanceof EnhancedForStmt or + this instanceof CatchClause or + this.(InstanceOfExpr).isPattern() + } + + /** Gets the variable declaration that this element contains */ + LocalVariableDeclExpr getVariable() { result.getParent() = this } + + /** Gets the type access of the variable */ + Expr getTypeAccess() { result = getVariable().getTypeAccess() } +} + +/** + * A node representing an element that can be the parent of up to one `LocalVariableDeclExpr` for which we + * want to use a synthetic node to variable declaration and its type access. + * + * Excludes `LocalVariableDeclStmt` and `ForStmt`, as they can hold multiple declarations. + * For these cases, either a synthetic node is not necassary or a different synthetic node is used. + */ +final class SingleLocalVarDeclParentNode extends ExprStmtNode { + SingleLocalVarDeclParent lvdp; + + SingleLocalVarDeclParentNode() { lvdp = element } + + override PrintAstNode getChild(int childIndex) { + result = super.getChild(childIndex) and + not result.(ElementNode).getElement() = [lvdp.getVariable(), lvdp.getTypeAccess()] + or + childIndex = lvdp.getVariable().getIndex() and + result.(LocalVarDeclSynthNode).getVariable() = lvdp.getVariable() } } @@ -432,6 +528,51 @@ final class TypeVariableNode extends ElementNode { } } +/** + * A node representing the initializers of a `ForStmt`. + */ +final class ForInitNode extends PrintAstNode, TForInitNode { + ForStmt fs; + + ForInitNode() { this = TForInitNode(fs) } + + override string toString() { result = "(For Initializers) " } + + override ElementNode getChild(int childIndex) { + childIndex >= 0 and + result.getElement().(Expr).isNthChildOf(fs, -childIndex) + } + + /** + * Gets the underlying `ForStmt`. + */ + ForStmt getForStmt() { result = fs } +} + +/** + * A synthetic node holding a `LocalVariableDeclExpr` and its type access. + */ +final class LocalVarDeclSynthNode extends PrintAstNode, TLocalVarDeclNode { + LocalVariableDeclExpr lvde; + + LocalVarDeclSynthNode() { this = TLocalVarDeclNode(lvde) } + + override string toString() { result = "(Single Local Variable Declaration)" } + + override ElementNode getChild(int childIndex) { + childIndex = 0 and + result.getElement() = lvde.getTypeAccess() + or + childIndex = 1 and + result.getElement() = lvde + } + + /** + * Gets the underlying `LocalVariableDeclExpr` + */ + LocalVariableDeclExpr getVariable() { result = lvde } +} + /** * A node representing the annotations of an `Annotatable`. * Only rendered if there is at least one annotation. diff --git a/java/ql/src/semmle/code/java/dataflow/ModulusAnalysis.qll b/java/ql/src/semmle/code/java/dataflow/ModulusAnalysis.qll index 1fb3eb201a8..b153e9fbd04 100644 --- a/java/ql/src/semmle/code/java/dataflow/ModulusAnalysis.qll +++ b/java/ql/src/semmle/code/java/dataflow/ModulusAnalysis.qll @@ -6,6 +6,7 @@ import java private import SSA +private import semmle.code.java.dataflow.internal.rangeanalysis.SsaReadPositionCommon private import RangeUtils private import semmle.code.java.controlflow.Guards import Bound diff --git a/java/ql/src/semmle/code/java/dataflow/RangeAnalysis.qll b/java/ql/src/semmle/code/java/dataflow/RangeAnalysis.qll index cbd59f8302a..07860c76ee4 100644 --- a/java/ql/src/semmle/code/java/dataflow/RangeAnalysis.qll +++ b/java/ql/src/semmle/code/java/dataflow/RangeAnalysis.qll @@ -66,6 +66,7 @@ import java private import SSA private import RangeUtils +private import semmle.code.java.dataflow.internal.rangeanalysis.SsaReadPositionCommon private import semmle.code.java.controlflow.internal.GuardsLogic private import SignAnalysis private import ModulusAnalysis diff --git a/java/ql/src/semmle/code/java/dataflow/RangeUtils.qll b/java/ql/src/semmle/code/java/dataflow/RangeUtils.qll index e7da9891b47..631145a2aa2 100644 --- a/java/ql/src/semmle/code/java/dataflow/RangeUtils.qll +++ b/java/ql/src/semmle/code/java/dataflow/RangeUtils.qll @@ -5,6 +5,7 @@ import java private import SSA private import semmle.code.java.controlflow.internal.GuardsLogic +private import semmle.code.java.dataflow.internal.rangeanalysis.SsaReadPositionCommon /** * Holds if `v` is an input to `phi` that is not along a back edge, and the @@ -127,63 +128,6 @@ Expr ssaRead(SsaVariable v, int delta) { result.(AssignExpr).getSource() = ssaRead(v, delta) } -private newtype TSsaReadPosition = - TSsaReadPositionBlock(BasicBlock bb) { exists(SsaVariable v | bb = v.getAUse().getBasicBlock()) } or - TSsaReadPositionPhiInputEdge(BasicBlock bbOrig, BasicBlock bbPhi) { - exists(SsaPhiNode phi | phi.hasInputFromBlock(_, bbOrig) and bbPhi = phi.getBasicBlock()) - } - -/** - * A position at which an SSA variable is read. This includes both ordinary - * reads occurring in basic blocks and input to phi nodes occurring along an - * edge between two basic blocks. - */ -class SsaReadPosition extends TSsaReadPosition { - /** Holds if `v` is read at this position. */ - abstract predicate hasReadOfVar(SsaVariable v); - - /** Gets a textual representation of this SSA read position. */ - abstract string toString(); -} - -/** A basic block in which an SSA variable is read. */ -class SsaReadPositionBlock extends SsaReadPosition, TSsaReadPositionBlock { - /** Gets the basic block corresponding to this position. */ - BasicBlock getBlock() { this = TSsaReadPositionBlock(result) } - - override predicate hasReadOfVar(SsaVariable v) { getBlock() = v.getAUse().getBasicBlock() } - - override string toString() { result = "block" } -} - -/** - * An edge between two basic blocks where the latter block has an SSA phi - * definition. The edge therefore has a read of an SSA variable serving as the - * input to the phi node. - */ -class SsaReadPositionPhiInputEdge extends SsaReadPosition, TSsaReadPositionPhiInputEdge { - /** Gets the head of the edge. */ - BasicBlock getOrigBlock() { this = TSsaReadPositionPhiInputEdge(result, _) } - - /** Gets the tail of the edge. */ - BasicBlock getPhiBlock() { this = TSsaReadPositionPhiInputEdge(_, result) } - - override predicate hasReadOfVar(SsaVariable v) { - exists(SsaPhiNode phi | - phi.hasInputFromBlock(v, getOrigBlock()) and - getPhiBlock() = phi.getBasicBlock() - ) - } - - /** Holds if `inp` is an input to `phi` along this edge. */ - predicate phiInput(SsaPhiNode phi, SsaVariable inp) { - phi.hasInputFromBlock(inp, getOrigBlock()) and - getPhiBlock() = phi.getBasicBlock() - } - - override string toString() { result = "edge" } -} - /** * Holds if `inp` is an input to `phi` along a back edge. */ diff --git a/java/ql/src/semmle/code/java/dataflow/SSA.qll b/java/ql/src/semmle/code/java/dataflow/SSA.qll index 84d6f179efb..851427c4b26 100644 --- a/java/ql/src/semmle/code/java/dataflow/SSA.qll +++ b/java/ql/src/semmle/code/java/dataflow/SSA.qll @@ -770,7 +770,9 @@ private module SsaImpl { /** Holds if `v` occurs in `b` or one of `b`'s transitive successors. */ private predicate blockPrecedesVar(TrackedVar v, BasicBlock b) { - varOccursInBlock(v, b.getABBSuccessor*()) + varOccursInBlock(v, b) + or + ssaDefReachesEndOfBlock(v, _, b) } /** diff --git a/java/ql/src/semmle/code/java/dataflow/SignAnalysis.qll b/java/ql/src/semmle/code/java/dataflow/SignAnalysis.qll index 595f9c623f2..9cd629f4ef9 100644 --- a/java/ql/src/semmle/code/java/dataflow/SignAnalysis.qll +++ b/java/ql/src/semmle/code/java/dataflow/SignAnalysis.qll @@ -6,596 +6,4 @@ * three-valued domain `{negative, zero, positive}`. */ -import java -private import SSA -private import RangeUtils -private import semmle.code.java.controlflow.Guards -private import semmle.code.java.Reflection -private import semmle.code.java.Collections -private import semmle.code.java.Maps - -private newtype TSign = - TNeg() or - TZero() or - TPos() - -private class Sign extends TSign { - string toString() { - result = "-" and this = TNeg() - or - result = "0" and this = TZero() - or - result = "+" and this = TPos() - } - - Sign inc() { - this = TNeg() and result = TNeg() - or - this = TNeg() and result = TZero() - or - this = TZero() and result = TPos() - or - this = TPos() and result = TPos() - } - - Sign dec() { result.inc() = this } - - Sign neg() { - this = TNeg() and result = TPos() - or - this = TZero() and result = TZero() - or - this = TPos() and result = TNeg() - } - - Sign bitnot() { - this = TNeg() and result = TPos() - or - this = TNeg() and result = TZero() - or - this = TZero() and result = TNeg() - or - this = TPos() and result = TNeg() - } - - Sign add(Sign s) { - this = TZero() and result = s - or - s = TZero() and result = this - or - this = s and this = result - or - this = TPos() and s = TNeg() - or - this = TNeg() and s = TPos() - } - - Sign mul(Sign s) { - result = TZero() and this = TZero() - or - result = TZero() and s = TZero() - or - result = TNeg() and this = TPos() and s = TNeg() - or - result = TNeg() and this = TNeg() and s = TPos() - or - result = TPos() and this = TPos() and s = TPos() - or - result = TPos() and this = TNeg() and s = TNeg() - } - - Sign div(Sign s) { - result = TZero() and s = TNeg() - or - result = TZero() and s = TPos() - or - result = TNeg() and this = TPos() and s = TNeg() - or - result = TNeg() and this = TNeg() and s = TPos() - or - result = TPos() and this = TPos() and s = TPos() - or - result = TPos() and this = TNeg() and s = TNeg() - } - - Sign rem(Sign s) { - result = TZero() and s = TNeg() - or - result = TZero() and s = TPos() - or - result = this and s = TNeg() - or - result = this and s = TPos() - } - - Sign bitand(Sign s) { - result = TZero() and this = TZero() - or - result = TZero() and s = TZero() - or - result = TZero() and this = TPos() - or - result = TZero() and s = TPos() - or - result = TNeg() and this = TNeg() and s = TNeg() - or - result = TPos() and this = TNeg() and s = TPos() - or - result = TPos() and this = TPos() and s = TNeg() - or - result = TPos() and this = TPos() and s = TPos() - } - - Sign bitor(Sign s) { - result = TZero() and this = TZero() and s = TZero() - or - result = TNeg() and this = TNeg() - or - result = TNeg() and s = TNeg() - or - result = TPos() and this = TPos() and s = TZero() - or - result = TPos() and this = TZero() and s = TPos() - or - result = TPos() and this = TPos() and s = TPos() - } - - Sign bitxor(Sign s) { - result = TZero() and this = s - or - result = this and s = TZero() - or - result = s and this = TZero() - or - result = TPos() and this = TPos() and s = TPos() - or - result = TNeg() and this = TNeg() and s = TPos() - or - result = TNeg() and this = TPos() and s = TNeg() - or - result = TPos() and this = TNeg() and s = TNeg() - } - - Sign lshift(Sign s) { - result = TZero() and this = TZero() - or - result = this and s = TZero() - or - this != TZero() and s != TZero() - } - - Sign rshift(Sign s) { - result = TZero() and this = TZero() - or - result = this and s = TZero() - or - result = TNeg() and this = TNeg() - or - result != TNeg() and this = TPos() and s != TZero() - } - - Sign urshift(Sign s) { - result = TZero() and this = TZero() - or - result = this and s = TZero() - or - result != TZero() and this = TNeg() and s != TZero() - or - result != TNeg() and this = TPos() and s != TZero() - } -} - -/** Gets the sign of `e` if this can be directly determined. */ -private Sign certainExprSign(Expr e) { - exists(int i | e.(ConstantIntegerExpr).getIntValue() = i | - i < 0 and result = TNeg() - or - i = 0 and result = TZero() - or - i > 0 and result = TPos() - ) - or - not exists(e.(ConstantIntegerExpr).getIntValue()) and - ( - exists(float f | - f = e.(LongLiteral).getValue().toFloat() or - f = e.(FloatingPointLiteral).getValue().toFloat() or - f = e.(DoubleLiteral).getValue().toFloat() - | - f < 0 and result = TNeg() - or - f = 0 and result = TZero() - or - f > 0 and result = TPos() - ) - or - exists(string charlit | charlit = e.(CharacterLiteral).getValue() | - if charlit = "\\0" or charlit = "\\u0000" then result = TZero() else result = TPos() - ) - or - e.(MethodAccess).getMethod() instanceof StringLengthMethod and - (result = TPos() or result = TZero()) - or - e.(MethodAccess).getMethod() instanceof CollectionSizeMethod and - (result = TPos() or result = TZero()) - or - e.(MethodAccess).getMethod() instanceof MapSizeMethod and - (result = TPos() or result = TZero()) - ) -} - -/** Holds if the sign of `e` is too complicated to determine. */ -private predicate unknownSign(Expr e) { - not exists(e.(ConstantIntegerExpr).getIntValue()) and - ( - exists(IntegerLiteral lit | lit = e and not exists(lit.getValue().toInt())) - or - exists(LongLiteral lit | lit = e and not exists(lit.getValue().toFloat())) - or - exists(CastExpr cast, Type fromtyp | - cast = e and - fromtyp = cast.getExpr().getType() and - not fromtyp instanceof NumericOrCharType - ) - or - e instanceof ArrayAccess and e.getType() instanceof NumericOrCharType - or - e instanceof MethodAccess and e.getType() instanceof NumericOrCharType - or - e instanceof ClassInstanceExpr and e.getType() instanceof NumericOrCharType - ) -} - -/** - * Holds if `lowerbound` is a lower bound for `v` at `pos`. This is restricted - * to only include bounds for which we might determine a sign. - */ -private predicate lowerBound(Expr lowerbound, SsaVariable v, SsaReadPosition pos, boolean isStrict) { - exists(boolean testIsTrue, ComparisonExpr comp | - pos.hasReadOfVar(v) and - guardControlsSsaRead(comp, pos, testIsTrue) and - not unknownSign(lowerbound) - | - testIsTrue = true and - comp.getLesserOperand() = lowerbound and - comp.getGreaterOperand() = ssaRead(v, 0) and - (if comp.isStrict() then isStrict = true else isStrict = false) - or - testIsTrue = false and - comp.getGreaterOperand() = lowerbound and - comp.getLesserOperand() = ssaRead(v, 0) and - (if comp.isStrict() then isStrict = false else isStrict = true) - ) -} - -/** - * Holds if `upperbound` is an upper bound for `v` at `pos`. This is restricted - * to only include bounds for which we might determine a sign. - */ -private predicate upperBound(Expr upperbound, SsaVariable v, SsaReadPosition pos, boolean isStrict) { - exists(boolean testIsTrue, ComparisonExpr comp | - pos.hasReadOfVar(v) and - guardControlsSsaRead(comp, pos, testIsTrue) and - not unknownSign(upperbound) - | - testIsTrue = true and - comp.getGreaterOperand() = upperbound and - comp.getLesserOperand() = ssaRead(v, 0) and - (if comp.isStrict() then isStrict = true else isStrict = false) - or - testIsTrue = false and - comp.getLesserOperand() = upperbound and - comp.getGreaterOperand() = ssaRead(v, 0) and - (if comp.isStrict() then isStrict = false else isStrict = true) - ) -} - -/** - * Holds if `eqbound` is an equality/inequality for `v` at `pos`. This is - * restricted to only include bounds for which we might determine a sign. The - * boolean `isEq` gives the polarity: - * - `isEq = true` : `v = eqbound` - * - `isEq = false` : `v != eqbound` - */ -private predicate eqBound(Expr eqbound, SsaVariable v, SsaReadPosition pos, boolean isEq) { - exists(Guard guard, boolean testIsTrue, boolean polarity | - pos.hasReadOfVar(v) and - guardControlsSsaRead(guard, pos, testIsTrue) and - guard.isEquality(eqbound, ssaRead(v, 0), polarity) and - isEq = polarity.booleanXor(testIsTrue).booleanNot() and - not unknownSign(eqbound) - ) -} - -/** - * Holds if `bound` is a bound for `v` at `pos` that needs to be positive in - * order for `v` to be positive. - */ -private predicate posBound(Expr bound, SsaVariable v, SsaReadPosition pos) { - upperBound(bound, v, pos, _) or - eqBound(bound, v, pos, true) -} - -/** - * Holds if `bound` is a bound for `v` at `pos` that needs to be negative in - * order for `v` to be negative. - */ -private predicate negBound(Expr bound, SsaVariable v, SsaReadPosition pos) { - lowerBound(bound, v, pos, _) or - eqBound(bound, v, pos, true) -} - -/** - * Holds if `bound` is a bound for `v` at `pos` that can restrict whether `v` - * can be zero. - */ -private predicate zeroBound(Expr bound, SsaVariable v, SsaReadPosition pos) { - lowerBound(bound, v, pos, _) or - upperBound(bound, v, pos, _) or - eqBound(bound, v, pos, _) -} - -/** Holds if `bound` allows `v` to be positive at `pos`. */ -private predicate posBoundOk(Expr bound, SsaVariable v, SsaReadPosition pos) { - posBound(bound, v, pos) and TPos() = exprSign(bound) -} - -/** Holds if `bound` allows `v` to be negative at `pos`. */ -private predicate negBoundOk(Expr bound, SsaVariable v, SsaReadPosition pos) { - negBound(bound, v, pos) and TNeg() = exprSign(bound) -} - -/** Holds if `bound` allows `v` to be zero at `pos`. */ -private predicate zeroBoundOk(Expr bound, SsaVariable v, SsaReadPosition pos) { - lowerBound(bound, v, pos, _) and TNeg() = exprSign(bound) - or - lowerBound(bound, v, pos, false) and TZero() = exprSign(bound) - or - upperBound(bound, v, pos, _) and TPos() = exprSign(bound) - or - upperBound(bound, v, pos, false) and TZero() = exprSign(bound) - or - eqBound(bound, v, pos, true) and TZero() = exprSign(bound) - or - eqBound(bound, v, pos, false) and TZero() != exprSign(bound) -} - -/** - * Holds if there is a bound that might restrict whether `v` has the sign `s` - * at `pos`. - */ -private predicate hasGuard(SsaVariable v, SsaReadPosition pos, Sign s) { - s = TPos() and posBound(_, v, pos) - or - s = TNeg() and negBound(_, v, pos) - or - s = TZero() and zeroBound(_, v, pos) -} - -pragma[noinline] -private Sign guardedSsaSign(SsaVariable v, SsaReadPosition pos) { - result = ssaDefSign(v) and - pos.hasReadOfVar(v) and - hasGuard(v, pos, result) -} - -pragma[noinline] -private Sign unguardedSsaSign(SsaVariable v, SsaReadPosition pos) { - result = ssaDefSign(v) and - pos.hasReadOfVar(v) and - not hasGuard(v, pos, result) -} - -private Sign guardedSsaSignOk(SsaVariable v, SsaReadPosition pos) { - result = TPos() and - forex(Expr bound | posBound(bound, v, pos) | posBoundOk(bound, v, pos)) - or - result = TNeg() and - forex(Expr bound | negBound(bound, v, pos) | negBoundOk(bound, v, pos)) - or - result = TZero() and - forex(Expr bound | zeroBound(bound, v, pos) | zeroBoundOk(bound, v, pos)) -} - -/** Gets a possible sign for `v` at `pos`. */ -private Sign ssaSign(SsaVariable v, SsaReadPosition pos) { - result = unguardedSsaSign(v, pos) - or - result = guardedSsaSign(v, pos) and - result = guardedSsaSignOk(v, pos) -} - -/** Gets a possible sign for `v`. */ -pragma[nomagic] -private Sign ssaDefSign(SsaVariable v) { - exists(VariableUpdate def | def = v.(SsaExplicitUpdate).getDefiningExpr() | - result = exprSign(def.(VariableAssign).getSource()) - or - exists(EnhancedForStmt for | def = for.getVariable()) - or - result = exprSign(def.(PostIncExpr).getExpr()).inc() - or - result = exprSign(def.(PreIncExpr).getExpr()).inc() - or - result = exprSign(def.(PostDecExpr).getExpr()).dec() - or - result = exprSign(def.(PreDecExpr).getExpr()).dec() - or - exists(AssignOp a | a = def and result = exprSign(a)) - ) - or - result = fieldSign(v.(SsaImplicitUpdate).getSourceVariable().getVariable()) - or - result = fieldSign(v.(SsaImplicitInit).getSourceVariable().getVariable()) - or - exists(Parameter p | v.(SsaImplicitInit).isParameterDefinition(p)) - or - exists(SsaPhiNode phi, SsaVariable inp, SsaReadPositionPhiInputEdge edge | - v = phi and - edge.phiInput(phi, inp) and - result = ssaSign(inp, edge) - ) -} - -/** Gets a possible sign for `f`. */ -private Sign fieldSign(Field f) { - result = exprSign(f.getAnAssignedValue()) - or - exists(PostIncExpr inc | inc.getExpr() = f.getAnAccess() and result = fieldSign(f).inc()) - or - exists(PreIncExpr inc | inc.getExpr() = f.getAnAccess() and result = fieldSign(f).inc()) - or - exists(PostDecExpr inc | inc.getExpr() = f.getAnAccess() and result = fieldSign(f).dec()) - or - exists(PreDecExpr inc | inc.getExpr() = f.getAnAccess() and result = fieldSign(f).dec()) - or - exists(AssignOp a | a.getDest() = f.getAnAccess() | result = exprSign(a)) - or - exists(ReflectiveFieldAccess rfa | rfa.inferAccessedField() = f) - or - if f.fromSource() - then not exists(f.getInitializer()) and result = TZero() - else - if f instanceof ArrayLengthField - then result != TNeg() - else - if f.hasName("MAX_VALUE") - then result = TPos() - else - if f.hasName("MIN_VALUE") - then result = TNeg() - else any() -} - -/** Gets a possible sign for `e`. */ -cached -private Sign exprSign(Expr e) { - result = certainExprSign(e) - or - not exists(certainExprSign(e)) and - ( - unknownSign(e) - or - exists(SsaVariable v | v.getAUse() = e | - result = ssaSign(v, any(SsaReadPositionBlock bb | bb.getBlock() = e.getBasicBlock())) - or - not exists(e.getBasicBlock()) and result = ssaDefSign(v) - ) - or - exists(FieldAccess fa | fa = e | - not exists(SsaVariable v | v.getAUse() = fa) and - result = fieldSign(fa.getField()) - ) - or - exists(VarAccess va | va = e | - not exists(SsaVariable v | v.getAUse() = va) and - not va instanceof FieldAccess - ) - or - result = exprSign(e.(AssignExpr).getSource()) - or - result = exprSign(e.(PlusExpr).getExpr()) - or - result = exprSign(e.(PostIncExpr).getExpr()) - or - result = exprSign(e.(PostDecExpr).getExpr()) - or - result = exprSign(e.(PreIncExpr).getExpr()).inc() - or - result = exprSign(e.(PreDecExpr).getExpr()).dec() - or - result = exprSign(e.(MinusExpr).getExpr()).neg() - or - result = exprSign(e.(BitNotExpr).getExpr()).bitnot() - or - exists(DivExpr div | - div = e and - result = exprSign(div.getLeftOperand()) and - result != TZero() - | - div.getRightOperand().(FloatingPointLiteral).getValue().toFloat() = 0 or - div.getRightOperand().(DoubleLiteral).getValue().toFloat() = 0 - ) - or - exists(Sign s1, Sign s2 | binaryOpSigns(e, s1, s2) | - (e instanceof AssignAddExpr or e instanceof AddExpr) and - result = s1.add(s2) - or - (e instanceof AssignSubExpr or e instanceof SubExpr) and - result = s1.add(s2.neg()) - or - (e instanceof AssignMulExpr or e instanceof MulExpr) and - result = s1.mul(s2) - or - (e instanceof AssignDivExpr or e instanceof DivExpr) and - result = s1.div(s2) - or - (e instanceof AssignRemExpr or e instanceof RemExpr) and - result = s1.rem(s2) - or - (e instanceof AssignAndExpr or e instanceof AndBitwiseExpr) and - result = s1.bitand(s2) - or - (e instanceof AssignOrExpr or e instanceof OrBitwiseExpr) and - result = s1.bitor(s2) - or - (e instanceof AssignXorExpr or e instanceof XorBitwiseExpr) and - result = s1.bitxor(s2) - or - (e instanceof AssignLShiftExpr or e instanceof LShiftExpr) and - result = s1.lshift(s2) - or - (e instanceof AssignRShiftExpr or e instanceof RShiftExpr) and - result = s1.rshift(s2) - or - (e instanceof AssignURShiftExpr or e instanceof URShiftExpr) and - result = s1.urshift(s2) - ) - or - result = exprSign(e.(ChooseExpr).getAResultExpr()) - or - result = exprSign(e.(CastExpr).getExpr()) - ) -} - -private Sign binaryOpLhsSign(Expr e) { - result = exprSign(e.(BinaryExpr).getLeftOperand()) or - result = exprSign(e.(AssignOp).getDest()) -} - -private Sign binaryOpRhsSign(Expr e) { - result = exprSign(e.(BinaryExpr).getRightOperand()) or - result = exprSign(e.(AssignOp).getRhs()) -} - -pragma[noinline] -private predicate binaryOpSigns(Expr e, Sign lhs, Sign rhs) { - lhs = binaryOpLhsSign(e) and - rhs = binaryOpRhsSign(e) -} - -/** Holds if `e` can be positive and cannot be negative. */ -predicate positive(Expr e) { - exprSign(e) = TPos() and - not exprSign(e) = TNeg() -} - -/** Holds if `e` can be negative and cannot be positive. */ -predicate negative(Expr e) { - exprSign(e) = TNeg() and - not exprSign(e) = TPos() -} - -/** Holds if `e` is strictly positive. */ -predicate strictlyPositive(Expr e) { - exprSign(e) = TPos() and - not exprSign(e) = TNeg() and - not exprSign(e) = TZero() -} - -/** Holds if `e` is strictly negative. */ -predicate strictlyNegative(Expr e) { - exprSign(e) = TNeg() and - not exprSign(e) = TPos() and - not exprSign(e) = TZero() -} +import semmle.code.java.dataflow.internal.rangeanalysis.SignAnalysisCommon diff --git a/java/ql/src/semmle/code/java/dataflow/internal/BaseSSA.qll b/java/ql/src/semmle/code/java/dataflow/internal/BaseSSA.qll index 9c911a2782c..58021120b5c 100644 --- a/java/ql/src/semmle/code/java/dataflow/internal/BaseSSA.qll +++ b/java/ql/src/semmle/code/java/dataflow/internal/BaseSSA.qll @@ -321,7 +321,9 @@ private module SsaImpl { /** Holds if `v` occurs in `b` or one of `b`'s transitive successors. */ private predicate blockPrecedesVar(BaseSsaSourceVariable v, BasicBlock b) { - varOccursInBlock(v, b.getABBSuccessor*()) + varOccursInBlock(v, b) + or + ssaDefReachesEndOfBlock(v, _, b) } /** diff --git a/java/ql/src/semmle/code/java/dataflow/internal/rangeanalysis/Sign.qll b/java/ql/src/semmle/code/java/dataflow/internal/rangeanalysis/Sign.qll new file mode 100644 index 00000000000..10ca946a044 --- /dev/null +++ b/java/ql/src/semmle/code/java/dataflow/internal/rangeanalysis/Sign.qll @@ -0,0 +1,219 @@ +newtype TSign = + TNeg() or + TZero() or + TPos() + +/** Class representing expression signs (+, -, 0). */ +class Sign extends TSign { + /** Gets the string representation of this sign. */ + string toString() { + result = "-" and this = TNeg() + or + result = "0" and this = TZero() + or + result = "+" and this = TPos() + } + + /** Gets a possible sign after incrementing an expression that has this sign. */ + Sign inc() { + this = TNeg() and result = TNeg() + or + this = TNeg() and result = TZero() + or + this = TZero() and result = TPos() + or + this = TPos() and result = TPos() + } + + /** Gets a possible sign after decrementing an expression that has this sign. */ + Sign dec() { result.inc() = this } + + /** Gets a possible sign after negating an expression that has this sign. */ + Sign neg() { + this = TNeg() and result = TPos() + or + this = TZero() and result = TZero() + or + this = TPos() and result = TNeg() + } + + /** + * Gets a possible sign after bitwise complementing an expression that has this + * sign. + */ + Sign bitnot() { + this = TNeg() and result = TPos() + or + this = TNeg() and result = TZero() + or + this = TZero() and result = TNeg() + or + this = TPos() and result = TNeg() + } + + /** + * Gets a possible sign after adding an expression with sign `s` to an expression + * that has this sign. + */ + Sign add(Sign s) { + this = TZero() and result = s + or + s = TZero() and result = this + or + this = s and this = result + or + this = TPos() and s = TNeg() + or + this = TNeg() and s = TPos() + } + + /** + * Gets a possible sign after multiplying an expression with sign `s` to an expression + * that has this sign. + */ + Sign mul(Sign s) { + result = TZero() and this = TZero() + or + result = TZero() and s = TZero() + or + result = TNeg() and this = TPos() and s = TNeg() + or + result = TNeg() and this = TNeg() and s = TPos() + or + result = TPos() and this = TPos() and s = TPos() + or + result = TPos() and this = TNeg() and s = TNeg() + } + + /** + * Gets a possible sign after integer dividing an expression that has this sign + * by an expression with sign `s`. + */ + Sign div(Sign s) { + result = TZero() and s = TNeg() // ex: 3 / -5 = 0 + or + result = TZero() and s = TPos() // ex: 3 / 5 = 0 + or + result = TNeg() and this = TPos() and s = TNeg() + or + result = TNeg() and this = TNeg() and s = TPos() + or + result = TPos() and this = TPos() and s = TPos() + or + result = TPos() and this = TNeg() and s = TNeg() + } + + /** + * Gets a possible sign after modulo dividing an expression that has this sign + * by an expression with sign `s`. + */ + Sign rem(Sign s) { + result = TZero() and s = TNeg() + or + result = TZero() and s = TPos() + or + result = this and s = TNeg() + or + result = this and s = TPos() + } + + /** + * Gets a possible sign after bitwise `and` of an expression that has this sign + * and an expression with sign `s`. + */ + Sign bitand(Sign s) { + result = TZero() and this = TZero() + or + result = TZero() and s = TZero() + or + result = TZero() and this = TPos() + or + result = TZero() and s = TPos() + or + result = TNeg() and this = TNeg() and s = TNeg() + or + result = TPos() and this = TNeg() and s = TPos() + or + result = TPos() and this = TPos() and s = TNeg() + or + result = TPos() and this = TPos() and s = TPos() + } + + /** + * Gets a possible sign after bitwise `or` of an expression that has this sign + * and an expression with sign `s`. + */ + Sign bitor(Sign s) { + result = TZero() and this = TZero() and s = TZero() + or + result = TNeg() and this = TNeg() + or + result = TNeg() and s = TNeg() + or + result = TPos() and this = TPos() and s = TZero() + or + result = TPos() and this = TZero() and s = TPos() + or + result = TPos() and this = TPos() and s = TPos() + } + + /** + * Gets a possible sign after bitwise `xor` of an expression that has this sign + * and an expression with sign `s`. + */ + Sign bitxor(Sign s) { + result = TZero() and this = s + or + result = this and s = TZero() + or + result = s and this = TZero() + or + result = TPos() and this = TPos() and s = TPos() + or + result = TNeg() and this = TNeg() and s = TPos() + or + result = TNeg() and this = TPos() and s = TNeg() + or + result = TPos() and this = TNeg() and s = TNeg() + } + + /** + * Gets a possible sign after left shift of an expression that has this sign + * by an expression with sign `s`. + */ + Sign lshift(Sign s) { + result = TZero() and this = TZero() + or + result = this and s = TZero() + or + this != TZero() and s != TZero() + } + + /** + * Gets a possible sign after right shift of an expression that has this sign + * by an expression with sign `s`. + */ + Sign rshift(Sign s) { + result = TZero() and this = TZero() + or + result = this and s = TZero() + or + result = TNeg() and this = TNeg() + or + result != TNeg() and this = TPos() and s != TZero() + } + + /** + * Gets a possible sign after unsigned right shift of an expression that has + * this sign by an expression with sign `s`. + */ + Sign urshift(Sign s) { + result = TZero() and this = TZero() + or + result = this and s = TZero() + or + result != TZero() and this = TNeg() and s != TZero() + or + result != TNeg() and this = TPos() and s != TZero() + } +} diff --git a/java/ql/src/semmle/code/java/dataflow/internal/rangeanalysis/SignAnalysisCommon.qll b/java/ql/src/semmle/code/java/dataflow/internal/rangeanalysis/SignAnalysisCommon.qll new file mode 100644 index 00000000000..c734e6acb0e --- /dev/null +++ b/java/ql/src/semmle/code/java/dataflow/internal/rangeanalysis/SignAnalysisCommon.qll @@ -0,0 +1,299 @@ +/** + * Provides sign analysis to determine whether expression are always positive + * or negative. + * + * The analysis is implemented as an abstract interpretation over the + * three-valued domain `{negative, zero, positive}`. + */ + +private import SignAnalysisSpecific::Private +private import SsaReadPositionCommon +private import Sign + +/** Gets the sign of `e` if this can be directly determined. */ +Sign certainExprSign(Expr e) { + exists(int i | e.(ConstantIntegerExpr).getIntValue() = i | + i < 0 and result = TNeg() + or + i = 0 and result = TZero() + or + i > 0 and result = TPos() + ) + or + not exists(e.(ConstantIntegerExpr).getIntValue()) and + ( + exists(float f | f = getNonIntegerValue(e) | + f < 0 and result = TNeg() + or + f = 0 and result = TZero() + or + f > 0 and result = TPos() + ) + or + exists(string charlit | charlit = getCharValue(e) | + if charlit.regexpMatch("\\u0000") then result = TZero() else result = TPos() + ) + or + containerSizeAccess(e) and + (result = TPos() or result = TZero()) + or + positiveExpression(e) and result = TPos() + ) +} + +/** Holds if the sign of `e` is too complicated to determine. */ +predicate unknownSign(Expr e) { + not exists(certainExprSign(e)) and + ( + exists(IntegerLiteral lit | lit = e and not exists(lit.getValue().toInt())) + or + exists(LongLiteral lit | lit = e and not exists(lit.getValue().toFloat())) + or + exists(CastExpr cast, Type fromtyp | + cast = e and + fromtyp = cast.getExpr().getType() and + not fromtyp instanceof NumericOrCharType + ) + or + unknownIntegerAccess(e) + ) +} + +/** + * Holds if `lowerbound` is a lower bound for `v` at `pos`. This is restricted + * to only include bounds for which we might determine a sign. + */ +private predicate lowerBound(Expr lowerbound, SsaVariable v, SsaReadPosition pos, boolean isStrict) { + exists(boolean testIsTrue, ComparisonExpr comp | + pos.hasReadOfVar(v) and + guardControlsSsaRead(getComparisonGuard(comp), pos, testIsTrue) and + not unknownSign(lowerbound) + | + testIsTrue = true and + comp.getLesserOperand() = lowerbound and + comp.getGreaterOperand() = ssaRead(v, 0) and + (if comp.isStrict() then isStrict = true else isStrict = false) + or + testIsTrue = false and + comp.getGreaterOperand() = lowerbound and + comp.getLesserOperand() = ssaRead(v, 0) and + (if comp.isStrict() then isStrict = false else isStrict = true) + ) +} + +/** + * Holds if `upperbound` is an upper bound for `v` at `pos`. This is restricted + * to only include bounds for which we might determine a sign. + */ +private predicate upperBound(Expr upperbound, SsaVariable v, SsaReadPosition pos, boolean isStrict) { + exists(boolean testIsTrue, ComparisonExpr comp | + pos.hasReadOfVar(v) and + guardControlsSsaRead(getComparisonGuard(comp), pos, testIsTrue) and + not unknownSign(upperbound) + | + testIsTrue = true and + comp.getGreaterOperand() = upperbound and + comp.getLesserOperand() = ssaRead(v, 0) and + (if comp.isStrict() then isStrict = true else isStrict = false) + or + testIsTrue = false and + comp.getLesserOperand() = upperbound and + comp.getGreaterOperand() = ssaRead(v, 0) and + (if comp.isStrict() then isStrict = false else isStrict = true) + ) +} + +/** + * Holds if `eqbound` is an equality/inequality for `v` at `pos`. This is + * restricted to only include bounds for which we might determine a sign. The + * boolean `isEq` gives the polarity: + * - `isEq = true` : `v = eqbound` + * - `isEq = false` : `v != eqbound` + */ +private predicate eqBound(Expr eqbound, SsaVariable v, SsaReadPosition pos, boolean isEq) { + exists(Guard guard, boolean testIsTrue, boolean polarity | + pos.hasReadOfVar(v) and + guardControlsSsaRead(guard, pos, testIsTrue) and + guard.isEquality(eqbound, ssaRead(v, 0), polarity) and + isEq = polarity.booleanXor(testIsTrue).booleanNot() and + not unknownSign(eqbound) + ) +} + +/** + * Holds if `bound` is a bound for `v` at `pos` that needs to be positive in + * order for `v` to be positive. + */ +private predicate posBound(Expr bound, SsaVariable v, SsaReadPosition pos) { + upperBound(bound, v, pos, _) or + eqBound(bound, v, pos, true) +} + +/** + * Holds if `bound` is a bound for `v` at `pos` that needs to be negative in + * order for `v` to be negative. + */ +private predicate negBound(Expr bound, SsaVariable v, SsaReadPosition pos) { + lowerBound(bound, v, pos, _) or + eqBound(bound, v, pos, true) +} + +/** + * Holds if `bound` is a bound for `v` at `pos` that can restrict whether `v` + * can be zero. + */ +private predicate zeroBound(Expr bound, SsaVariable v, SsaReadPosition pos) { + lowerBound(bound, v, pos, _) or + upperBound(bound, v, pos, _) or + eqBound(bound, v, pos, _) +} + +/** Holds if `bound` allows `v` to be positive at `pos`. */ +private predicate posBoundOk(Expr bound, SsaVariable v, SsaReadPosition pos) { + posBound(bound, v, pos) and TPos() = exprSign(bound) +} + +/** Holds if `bound` allows `v` to be negative at `pos`. */ +private predicate negBoundOk(Expr bound, SsaVariable v, SsaReadPosition pos) { + negBound(bound, v, pos) and TNeg() = exprSign(bound) +} + +/** Holds if `bound` allows `v` to be zero at `pos`. */ +private predicate zeroBoundOk(Expr bound, SsaVariable v, SsaReadPosition pos) { + lowerBound(bound, v, pos, _) and TNeg() = exprSign(bound) + or + lowerBound(bound, v, pos, false) and TZero() = exprSign(bound) + or + upperBound(bound, v, pos, _) and TPos() = exprSign(bound) + or + upperBound(bound, v, pos, false) and TZero() = exprSign(bound) + or + eqBound(bound, v, pos, true) and TZero() = exprSign(bound) + or + eqBound(bound, v, pos, false) and TZero() != exprSign(bound) +} + +/** + * Holds if there is a bound that might restrict whether `v` has the sign `s` + * at `pos`. + */ +private predicate hasGuard(SsaVariable v, SsaReadPosition pos, Sign s) { + s = TPos() and posBound(_, v, pos) + or + s = TNeg() and negBound(_, v, pos) + or + s = TZero() and zeroBound(_, v, pos) +} + +pragma[noinline] +private Sign guardedSsaSign(SsaVariable v, SsaReadPosition pos) { + // SSA variable can have sign `result` + result = ssaDefSign(v) and + pos.hasReadOfVar(v) and + // there are guards at this position on `v` that might restrict it to be sign `result`. + // (So we need to check if they are satisfied) + hasGuard(v, pos, result) +} + +pragma[noinline] +private Sign unguardedSsaSign(SsaVariable v, SsaReadPosition pos) { + // SSA variable can have sign `result` + result = ssaDefSign(v) and + pos.hasReadOfVar(v) and + // there's no guard at this position on `v` that might restrict it to be sign `result`. + not hasGuard(v, pos, result) +} + +/** + * Gets the sign of `v` at read position `pos`, when there's at least one guard + * on `v` at position `pos`. Each bound corresponding to a given sign must be met + * in order for `v` to be of that sign. + */ +private Sign guardedSsaSignOk(SsaVariable v, SsaReadPosition pos) { + result = TPos() and + forex(Expr bound | posBound(bound, v, pos) | posBoundOk(bound, v, pos)) + or + result = TNeg() and + forex(Expr bound | negBound(bound, v, pos) | negBoundOk(bound, v, pos)) + or + result = TZero() and + forex(Expr bound | zeroBound(bound, v, pos) | zeroBoundOk(bound, v, pos)) +} + +/** Gets a possible sign for `v` at `pos`. */ +Sign ssaSign(SsaVariable v, SsaReadPosition pos) { + result = unguardedSsaSign(v, pos) + or + result = guardedSsaSign(v, pos) and + result = guardedSsaSignOk(v, pos) +} + +/** Gets a possible sign for `v`. */ +pragma[nomagic] +Sign ssaDefSign(SsaVariable v) { + result = explicitSsaDefSign(v) + or + result = implicitSsaDefSign(v) + or + exists(SsaPhiNode phi, SsaVariable inp, SsaReadPositionPhiInputEdge edge | + v = phi and + edge.phiInput(phi, inp) and + result = ssaSign(inp, edge) + ) +} + +/** Gets a possible sign for `e`. */ +cached +Sign exprSign(Expr e) { + exists(Sign s | + s = certainExprSign(e) + or + not exists(certainExprSign(e)) and + ( + unknownSign(e) + or + exists(SsaVariable v | getARead(v) = e | s = ssaVariableSign(v, e)) + or + e = + any(VarAccess access | + not exists(SsaVariable v | getARead(v) = access) and + ( + s = fieldSign(getField(access.(FieldAccess))) or + not access instanceof FieldAccess + ) + ) + or + s = specificSubExprSign(e) + ) + | + if e.getType() instanceof UnsignedNumericType and s = TNeg() + then result = TPos() + else result = s + ) +} + +/** Holds if `e` can be positive and cannot be negative. */ +predicate positive(Expr e) { + exprSign(e) = TPos() and + not exprSign(e) = TNeg() +} + +/** Holds if `e` can be negative and cannot be positive. */ +predicate negative(Expr e) { + exprSign(e) = TNeg() and + not exprSign(e) = TPos() +} + +/** Holds if `e` is strictly positive. */ +predicate strictlyPositive(Expr e) { + exprSign(e) = TPos() and + not exprSign(e) = TNeg() and + not exprSign(e) = TZero() +} + +/** Holds if `e` is strictly negative. */ +predicate strictlyNegative(Expr e) { + exprSign(e) = TNeg() and + not exprSign(e) = TPos() and + not exprSign(e) = TZero() +} diff --git a/java/ql/src/semmle/code/java/dataflow/internal/rangeanalysis/SignAnalysisSpecific.qll b/java/ql/src/semmle/code/java/dataflow/internal/rangeanalysis/SignAnalysisSpecific.qll new file mode 100644 index 00000000000..8e030c9a53f --- /dev/null +++ b/java/ql/src/semmle/code/java/dataflow/internal/rangeanalysis/SignAnalysisSpecific.qll @@ -0,0 +1,237 @@ +/** + * Provides Java-specific definitions for use in sign analysis. + */ +module Private { + import semmle.code.java.dataflow.RangeUtils as RU + private import semmle.code.java.dataflow.SSA as Ssa + private import semmle.code.java.controlflow.Guards as G + private import java as J + import Impl + + class ConstantIntegerExpr = RU::ConstantIntegerExpr; + + class Guard = G::Guard; + + class SsaVariable = Ssa::SsaVariable; + + class SsaPhiNode = Ssa::SsaPhiNode; + + class VarAccess = J::VarAccess; + + class FieldAccess = J::FieldAccess; + + class CharacterLiteral = J::CharacterLiteral; + + class IntegerLiteral = J::IntegerLiteral; + + class LongLiteral = J::LongLiteral; + + class CastExpr = J::CastExpr; + + class Type = J::Type; + + class Expr = J::Expr; + + class ComparisonExpr = J::ComparisonExpr; + + class NumericOrCharType = J::NumericOrCharType; + + predicate ssaRead = RU::ssaRead/2; + + predicate guardControlsSsaRead = RU::guardControlsSsaRead/3; +} + +private module Impl { + private import java + private import semmle.code.java.dataflow.RangeUtils + private import semmle.code.java.dataflow.SSA + private import semmle.code.java.controlflow.Guards + private import semmle.code.java.Reflection + private import semmle.code.java.Collections + private import semmle.code.java.Maps + private import Sign + private import SignAnalysisCommon + private import SsaReadPositionCommon + + class UnsignedNumericType = CharacterType; + + float getNonIntegerValue(Expr e) { + result = e.(LongLiteral).getValue().toFloat() or + result = e.(FloatingPointLiteral).getValue().toFloat() or + result = e.(DoubleLiteral).getValue().toFloat() + } + + string getCharValue(Expr e) { result = e.(CharacterLiteral).getValue() } + + predicate containerSizeAccess(Expr e) { + e.(MethodAccess).getMethod() instanceof StringLengthMethod + or + e.(MethodAccess).getMethod() instanceof CollectionSizeMethod + or + e.(MethodAccess).getMethod() instanceof MapSizeMethod + } + + predicate positiveExpression(Expr e) { none() } + + predicate unknownIntegerAccess(Expr e) { + e instanceof ArrayAccess and e.getType() instanceof NumericOrCharType + or + e instanceof MethodAccess and e.getType() instanceof NumericOrCharType + or + e instanceof ClassInstanceExpr and e.getType() instanceof NumericOrCharType + } + + Sign explicitSsaDefSign(SsaVariable v) { + exists(VariableUpdate def | def = v.(SsaExplicitUpdate).getDefiningExpr() | + result = exprSign(def.(VariableAssign).getSource()) + or + exists(EnhancedForStmt for | def = for.getVariable()) + or + result = exprSign(def.(PostIncExpr).getExpr()).inc() + or + result = exprSign(def.(PreIncExpr).getExpr()).inc() + or + result = exprSign(def.(PostDecExpr).getExpr()).dec() + or + result = exprSign(def.(PreDecExpr).getExpr()).dec() + or + exists(AssignOp a | a = def and result = exprSign(a)) + ) + } + + Sign implicitSsaDefSign(SsaVariable v) { + result = fieldSign(v.(SsaImplicitUpdate).getSourceVariable().getVariable()) + or + result = fieldSign(v.(SsaImplicitInit).getSourceVariable().getVariable()) + or + exists(Parameter p | v.(SsaImplicitInit).isParameterDefinition(p)) + } + + pragma[inline] + Sign ssaVariableSign(SsaVariable v, Expr e) { + result = ssaSign(v, any(SsaReadPositionBlock bb | getAnExpression(bb) = e)) + or + not exists(SsaReadPositionBlock bb | getAnExpression(bb) = e) and + result = ssaDefSign(v) + } + + /** Gets a possible sign for `f`. */ + Sign fieldSign(Field f) { + result = exprSign(f.getAnAssignedValue()) + or + exists(PostIncExpr inc | inc.getExpr() = f.getAnAccess() and result = fieldSign(f).inc()) + or + exists(PreIncExpr inc | inc.getExpr() = f.getAnAccess() and result = fieldSign(f).inc()) + or + exists(PostDecExpr inc | inc.getExpr() = f.getAnAccess() and result = fieldSign(f).dec()) + or + exists(PreDecExpr inc | inc.getExpr() = f.getAnAccess() and result = fieldSign(f).dec()) + or + exists(AssignOp a | a.getDest() = f.getAnAccess() | result = exprSign(a)) + or + exists(ReflectiveFieldAccess rfa | rfa.inferAccessedField() = f) + or + if f.fromSource() + then not exists(f.getInitializer()) and result = TZero() + else + if f instanceof ArrayLengthField + then result != TNeg() + else + if f.hasName("MAX_VALUE") + then result = TPos() + else + if f.hasName("MIN_VALUE") + then result = TNeg() + else any() + } + + Sign specificSubExprSign(Expr e) { + result = exprSign(e.(AssignExpr).getSource()) + or + result = exprSign(e.(PlusExpr).getExpr()) + or + result = exprSign(e.(PostIncExpr).getExpr()) + or + result = exprSign(e.(PostDecExpr).getExpr()) + or + result = exprSign(e.(PreIncExpr).getExpr()).inc() + or + result = exprSign(e.(PreDecExpr).getExpr()).dec() + or + result = exprSign(e.(MinusExpr).getExpr()).neg() + or + result = exprSign(e.(BitNotExpr).getExpr()).bitnot() + or + exists(DivExpr div | + div = e and + result = exprSign(div.getLeftOperand()) and + result != TZero() + | + div.getRightOperand().(FloatingPointLiteral).getValue().toFloat() = 0 or + div.getRightOperand().(DoubleLiteral).getValue().toFloat() = 0 + ) + or + exists(Sign s1, Sign s2 | binaryOpSigns(e, s1, s2) | + (e instanceof AssignAddExpr or e instanceof AddExpr) and + result = s1.add(s2) + or + (e instanceof AssignSubExpr or e instanceof SubExpr) and + result = s1.add(s2.neg()) + or + (e instanceof AssignMulExpr or e instanceof MulExpr) and + result = s1.mul(s2) + or + (e instanceof AssignDivExpr or e instanceof DivExpr) and + result = s1.div(s2) + or + (e instanceof AssignRemExpr or e instanceof RemExpr) and + result = s1.rem(s2) + or + (e instanceof AssignAndExpr or e instanceof AndBitwiseExpr) and + result = s1.bitand(s2) + or + (e instanceof AssignOrExpr or e instanceof OrBitwiseExpr) and + result = s1.bitor(s2) + or + (e instanceof AssignXorExpr or e instanceof XorBitwiseExpr) and + result = s1.bitxor(s2) + or + (e instanceof AssignLShiftExpr or e instanceof LShiftExpr) and + result = s1.lshift(s2) + or + (e instanceof AssignRShiftExpr or e instanceof RShiftExpr) and + result = s1.rshift(s2) + or + (e instanceof AssignURShiftExpr or e instanceof URShiftExpr) and + result = s1.urshift(s2) + ) + or + result = exprSign(e.(ChooseExpr).getAResultExpr()) + or + result = exprSign(e.(CastExpr).getExpr()) + } + + private Sign binaryOpLhsSign(Expr e) { + result = exprSign(e.(BinaryExpr).getLeftOperand()) or + result = exprSign(e.(AssignOp).getDest()) + } + + private Sign binaryOpRhsSign(Expr e) { + result = exprSign(e.(BinaryExpr).getRightOperand()) or + result = exprSign(e.(AssignOp).getRhs()) + } + + pragma[noinline] + private predicate binaryOpSigns(Expr e, Sign lhs, Sign rhs) { + lhs = binaryOpLhsSign(e) and + rhs = binaryOpRhsSign(e) + } + + Expr getARead(SsaVariable v) { result = v.getAUse() } + + Field getField(FieldAccess fa) { result = fa.getField() } + + Expr getAnExpression(SsaReadPositionBlock bb) { result = bb.getBlock().getANode() } + + Guard getComparisonGuard(ComparisonExpr ce) { result = ce } +} diff --git a/java/ql/src/semmle/code/java/dataflow/internal/rangeanalysis/SsaReadPositionCommon.qll b/java/ql/src/semmle/code/java/dataflow/internal/rangeanalysis/SsaReadPositionCommon.qll new file mode 100644 index 00000000000..558ecd1b88b --- /dev/null +++ b/java/ql/src/semmle/code/java/dataflow/internal/rangeanalysis/SsaReadPositionCommon.qll @@ -0,0 +1,57 @@ +/** + * Provides classes for representing a position at which an SSA variable is read. + */ + +private import SsaReadPositionSpecific + +private newtype TSsaReadPosition = + TSsaReadPositionBlock(BasicBlock bb) { bb = getAReadBasicBlock(_) } or + TSsaReadPositionPhiInputEdge(BasicBlock bbOrig, BasicBlock bbPhi) { + exists(SsaPhiNode phi | phi.hasInputFromBlock(_, bbOrig) and bbPhi = phi.getBasicBlock()) + } + +/** + * A position at which an SSA variable is read. This includes both ordinary + * reads occurring in basic blocks and input to phi nodes occurring along an + * edge between two basic blocks. + */ +class SsaReadPosition extends TSsaReadPosition { + /** Holds if `v` is read at this position. */ + abstract predicate hasReadOfVar(SsaVariable v); + + /** Gets a textual representation of this SSA read position. */ + abstract string toString(); +} + +/** A basic block in which an SSA variable is read. */ +class SsaReadPositionBlock extends SsaReadPosition, TSsaReadPositionBlock { + /** Gets the basic block corresponding to this position. */ + BasicBlock getBlock() { this = TSsaReadPositionBlock(result) } + + override predicate hasReadOfVar(SsaVariable v) { getBlock() = getAReadBasicBlock(v) } + + override string toString() { result = "block" } +} + +/** + * An edge between two basic blocks where the latter block has an SSA phi + * definition. The edge therefore has a read of an SSA variable serving as the + * input to the phi node. + */ +class SsaReadPositionPhiInputEdge extends SsaReadPosition, TSsaReadPositionPhiInputEdge { + /** Gets the source of the edge. */ + BasicBlock getOrigBlock() { this = TSsaReadPositionPhiInputEdge(result, _) } + + /** Gets the target of the edge. */ + BasicBlock getPhiBlock() { this = TSsaReadPositionPhiInputEdge(_, result) } + + override predicate hasReadOfVar(SsaVariable v) { this.phiInput(_, v) } + + /** Holds if `inp` is an input to `phi` along this edge. */ + predicate phiInput(SsaPhiNode phi, SsaVariable inp) { + phi.hasInputFromBlock(inp, getOrigBlock()) and + getPhiBlock() = phi.getBasicBlock() + } + + override string toString() { result = "edge" } +} diff --git a/java/ql/src/semmle/code/java/dataflow/internal/rangeanalysis/SsaReadPositionSpecific.qll b/java/ql/src/semmle/code/java/dataflow/internal/rangeanalysis/SsaReadPositionSpecific.qll new file mode 100644 index 00000000000..dcfc3d69e32 --- /dev/null +++ b/java/ql/src/semmle/code/java/dataflow/internal/rangeanalysis/SsaReadPositionSpecific.qll @@ -0,0 +1,15 @@ +/** + * Provides Java-specific definitions for use in the `SsaReadPosition`. + */ + +private import semmle.code.java.dataflow.SSA as Ssa +private import semmle.code.java.controlflow.BasicBlocks as BB + +class SsaVariable = Ssa::SsaVariable; + +class SsaPhiNode = Ssa::SsaPhiNode; + +class BasicBlock = BB::BasicBlock; + +/** Gets a basic block in which SSA variable `v` is read. */ +BasicBlock getAReadBasicBlock(SsaVariable v) { result = v.getAUse().getBasicBlock() } diff --git a/java/ql/src/semmle/code/java/frameworks/Hibernate.qll b/java/ql/src/semmle/code/java/frameworks/Hibernate.qll index fdc237aac8a..5ad0448249f 100644 --- a/java/ql/src/semmle/code/java/frameworks/Hibernate.qll +++ b/java/ql/src/semmle/code/java/frameworks/Hibernate.qll @@ -4,20 +4,36 @@ import java +/** The interface `org.hibernate.query.QueryProducer`. */ +class HibernateQueryProducer extends RefType { + HibernateQueryProducer() { this.hasQualifiedName("org.hibernate.query", "QueryProducer") } +} + +/** The interface `org.hibernate.SharedSessionContract`. */ +class HibernateSharedSessionContract extends RefType { + HibernateSharedSessionContract() { + this.hasQualifiedName("org.hibernate", "SharedSessionContract") + } +} + /** The interface `org.hibernate.Session`. */ class HibernateSession extends RefType { HibernateSession() { this.hasQualifiedName("org.hibernate", "Session") } } /** - * Holds if `m` is a method on `HibernateSession` taking an SQL string as its - * first argument. + * Holds if `m` is a method on `HibernateQueryProducer`, or `HibernateSharedSessionContract` + * or `HibernateSession`, or a subclass, taking an SQL string as its first argument. */ predicate hibernateSqlMethod(Method m) { - m.getDeclaringType() instanceof HibernateSession and + exists(RefType t | + t = m.getDeclaringType().getASourceSupertype*() and + ( + t instanceof HibernateQueryProducer or + t instanceof HibernateSharedSessionContract or + t instanceof HibernateSession + ) + ) and m.getParameterType(0) instanceof TypeString and - ( - m.hasName("createQuery") or - m.hasName("createSQLQuery") - ) + m.hasName(["createQuery", "createNativeQuery", "createSQLQuery"]) } diff --git a/java/ql/test/library-tests/dataflow/sign-analysis/A.java b/java/ql/test/library-tests/dataflow/sign-analysis/A.java new file mode 100644 index 00000000000..59d7d950334 --- /dev/null +++ b/java/ql/test/library-tests/dataflow/sign-analysis/A.java @@ -0,0 +1,17 @@ +public class A { + int f1(int x, int y) { + if (x < 0) { + return x; // strictly negative + } + if (x < y) { + return y; // y is strictly positive because of the bound on x above + } + + return 0; + } + + void unsigned(int x) { + char c = (char)x; + System.out.println(c); // positive + } +} diff --git a/java/ql/test/library-tests/dataflow/sign-analysis/SignAnalysis.expected b/java/ql/test/library-tests/dataflow/sign-analysis/SignAnalysis.expected new file mode 100644 index 00000000000..4b38711aaa8 --- /dev/null +++ b/java/ql/test/library-tests/dataflow/sign-analysis/SignAnalysis.expected @@ -0,0 +1,5 @@ +| A.java:4:14:4:14 | x | strictlyNegative | +| A.java:6:9:6:9 | x | positive | +| A.java:7:14:7:14 | y | strictlyPositive | +| A.java:14:14:14:20 | (...)... | positive | +| A.java:15:24:15:24 | c | positive | diff --git a/java/ql/test/library-tests/dataflow/sign-analysis/SignAnalysis.ql b/java/ql/test/library-tests/dataflow/sign-analysis/SignAnalysis.ql new file mode 100644 index 00000000000..bc4fed33dc8 --- /dev/null +++ b/java/ql/test/library-tests/dataflow/sign-analysis/SignAnalysis.ql @@ -0,0 +1,21 @@ +import java +import semmle.code.java.dataflow.SignAnalysis + +string getASignString(Expr e) { + positive(e) and + not strictlyPositive(e) and + result = "positive" + or + negative(e) and + not strictlyNegative(e) and + result = "negative" + or + strictlyPositive(e) and + result = "strictlyPositive" + or + strictlyNegative(e) and + result = "strictlyNegative" +} + +from Expr e +select e, strictconcat(string s | s = getASignString(e) | s, " ") diff --git a/java/ql/test/library-tests/java7/MultiCatch/PrintAst.expected b/java/ql/test/library-tests/java7/MultiCatch/PrintAst.expected index 8b8fe959628..bba3b450e3e 100644 --- a/java/ql/test/library-tests/java7/MultiCatch/PrintAst.expected +++ b/java/ql/test/library-tests/java7/MultiCatch/PrintAst.expected @@ -21,10 +21,11 @@ MultiCatch.java: # 14| 0: [ClassInstanceExpr] new SQLException(...) # 14| -3: [TypeAccess] SQLException # 15| 0: [CatchClause] stmt -# 15| -1: [UnionTypeAccess] ...|... -# 15| 0: [TypeAccess] IOException -# 15| 1: [TypeAccess] SQLException -# 15| 0: [LocalVariableDeclExpr] e +#-----| 0: (Single Local Variable Declaration) +# 15| 0: [UnionTypeAccess] ...|... +# 15| 0: [TypeAccess] IOException +# 15| 1: [TypeAccess] SQLException +# 15| 1: [LocalVariableDeclExpr] e # 16| 1: [BlockStmt] stmt # 17| 0: [ExprStmt] stmt # 17| 0: [MethodAccess] printStackTrace(...) @@ -55,10 +56,11 @@ MultiCatch.java: # 30| 0: [ClassInstanceExpr] new Exception(...) # 30| -3: [TypeAccess] Exception # 31| 0: [CatchClause] stmt -# 31| -1: [UnionTypeAccess] ...|... -# 31| 0: [TypeAccess] IOException -# 31| 1: [TypeAccess] SQLException -# 31| 0: [LocalVariableDeclExpr] e +#-----| 0: (Single Local Variable Declaration) +# 31| 0: [UnionTypeAccess] ...|... +# 31| 0: [TypeAccess] IOException +# 31| 1: [TypeAccess] SQLException +# 31| 1: [LocalVariableDeclExpr] e # 32| 1: [BlockStmt] stmt # 35| 4: [Method] ordinaryCatch # 35| 3: [TypeAccess] void @@ -69,6 +71,7 @@ MultiCatch.java: # 39| 0: [ClassInstanceExpr] new IOException(...) # 39| -3: [TypeAccess] IOException # 40| 0: [CatchClause] stmt -# 40| -1: [TypeAccess] Exception -# 40| 0: [LocalVariableDeclExpr] e +#-----| 0: (Single Local Variable Declaration) +# 40| 0: [TypeAccess] Exception +# 40| 1: [LocalVariableDeclExpr] e # 41| 1: [BlockStmt] stmt diff --git a/java/ql/test/library-tests/printAst/A.java b/java/ql/test/library-tests/printAst/A.java index b0eeb9276a1..38319c64706 100644 --- a/java/ql/test/library-tests/printAst/A.java +++ b/java/ql/test/library-tests/printAst/A.java @@ -18,6 +18,12 @@ class A { /** Does something */ @Deprecated static int doSomething(@SuppressWarnings("all") String text) { + int i=0, j=1; + + for(i=0, j=1; i<3; i++) {} + + for(int m=0, n=1; m<3; m++) {} + return 0; } @@ -34,4 +40,20 @@ class A { @Ann2(7) }) String doSomethingElse() { return "c"; } + + void varDecls(Object[] things) { + try { + for(Object thing : things) { + if (thing instanceof Integer) { + return; + } + if (thing instanceof String s) { + throw new RuntimeException(s); + } + } + } + catch (RuntimeException rte) { + return; + } + } } \ No newline at end of file diff --git a/java/ql/test/library-tests/printAst/PrintAst.expected b/java/ql/test/library-tests/printAst/PrintAst.expected index b23ed34ea82..922acc4011a 100644 --- a/java/ql/test/library-tests/printAst/PrintAst.expected +++ b/java/ql/test/library-tests/printAst/PrintAst.expected @@ -31,26 +31,97 @@ A.java: # 20| 1: [StringLiteral] "all" # 20| 0: [TypeAccess] String # 20| 5: [BlockStmt] stmt -# 21| 0: [ReturnStmt] stmt -# 21| 0: [IntegerLiteral] 0 -# 24| 6: [FieldDeclaration] int counter, ...; -# 24| -1: [TypeAccess] int -# 24| 0: [IntegerLiteral] 1 -# 26| 7: [BlockStmt] stmt -# 27| 0: [ExprStmt] stmt -# 27| 0: [AssignExpr] ...=... -# 27| 0: [VarAccess] counter -# 27| 1: [MethodAccess] doSomething(...) -# 27| 0: [StringLiteral] "hi" -# 36| 8: [Method] doSomethingElse +# 21| 0: [LocalVariableDeclStmt] stmt +# 21| 0: [TypeAccess] int +# 21| 1: [LocalVariableDeclExpr] i +# 21| 0: [IntegerLiteral] 0 +# 21| 2: [LocalVariableDeclExpr] j +# 21| 0: [IntegerLiteral] 1 +# 23| 1: [ForStmt] stmt +#-----| 0: (For Initializers) +# 23| 1: [AssignExpr] ...=... +# 23| 0: [VarAccess] i +# 23| 1: [IntegerLiteral] 0 +# 23| 2: [AssignExpr] ...=... +# 23| 0: [VarAccess] j +# 23| 1: [IntegerLiteral] 1 +# 23| 1: [LTExpr] ... < ... +# 23| 0: [VarAccess] i +# 23| 1: [IntegerLiteral] 3 +# 23| 2: [BlockStmt] stmt +# 23| 3: [PostIncExpr] ...++ +# 23| 0: [VarAccess] i +# 25| 2: [ForStmt] stmt +#-----| 0: (For Initializers) +# 25| 0: [TypeAccess] int +# 25| 1: [LocalVariableDeclExpr] m +# 25| 0: [IntegerLiteral] 0 +# 25| 2: [LocalVariableDeclExpr] n +# 25| 0: [IntegerLiteral] 1 +# 25| 1: [LTExpr] ... < ... +# 25| 0: [VarAccess] m +# 25| 1: [IntegerLiteral] 3 +# 25| 2: [BlockStmt] stmt +# 25| 3: [PostIncExpr] ...++ +# 25| 0: [VarAccess] m +# 27| 3: [ReturnStmt] stmt +# 27| 0: [IntegerLiteral] 0 +# 30| 6: [FieldDeclaration] int counter, ...; +# 30| -1: [TypeAccess] int +# 30| 0: [IntegerLiteral] 1 +# 32| 7: [BlockStmt] stmt +# 33| 0: [ExprStmt] stmt +# 33| 0: [AssignExpr] ...=... +# 33| 0: [VarAccess] counter +# 33| 1: [MethodAccess] doSomething(...) +# 33| 0: [StringLiteral] "hi" +# 42| 8: [Method] doSomethingElse #-----| 1: (Annotations) -# 30| 1: [Annotation] Ann1 -# 31| 1: [StringLiteral] "a" -# 32| 2: [ArrayInit] {...} -# 33| 1: [Annotation] Ann2 -# 34| 2: [Annotation] Ann2 -# 34| 1: [IntegerLiteral] 7 -# 36| 3: [TypeAccess] String -# 36| 5: [BlockStmt] stmt -# 36| 0: [ReturnStmt] stmt -# 36| 0: [StringLiteral] "c" +# 36| 1: [Annotation] Ann1 +# 37| 1: [StringLiteral] "a" +# 38| 2: [ArrayInit] {...} +# 39| 1: [Annotation] Ann2 +# 40| 2: [Annotation] Ann2 +# 40| 1: [IntegerLiteral] 7 +# 42| 3: [TypeAccess] String +# 42| 5: [BlockStmt] stmt +# 42| 0: [ReturnStmt] stmt +# 42| 0: [StringLiteral] "c" +# 44| 9: [Method] varDecls +# 44| 3: [TypeAccess] void +#-----| 4: (Parameters) +# 44| 0: [Parameter] things +# 44| 0: [ArrayTypeAccess] ...[] +# 44| 0: [TypeAccess] Object +# 44| 5: [BlockStmt] stmt +# 45| 0: [TryStmt] stmt +# 45| -1: [BlockStmt] stmt +# 46| 0: [EnhancedForStmt] stmt +#-----| 0: (Single Local Variable Declaration) +# 46| 0: [TypeAccess] Object +# 46| 1: [LocalVariableDeclExpr] thing +# 46| 1: [VarAccess] things +# 46| 2: [BlockStmt] stmt +# 47| 0: [IfStmt] stmt +# 47| 0: [InstanceOfExpr] ...instanceof... +# 47| 0: [VarAccess] thing +# 47| 1: [TypeAccess] Integer +# 47| 1: [BlockStmt] stmt +# 48| 0: [ReturnStmt] stmt +# 50| 1: [IfStmt] stmt +# 50| 0: [InstanceOfExpr] ...instanceof... +#-----| 0: (Single Local Variable Declaration) +# 50| 0: [TypeAccess] String +# 50| 1: [LocalVariableDeclExpr] s +# 50| 0: [VarAccess] thing +# 50| 1: [BlockStmt] stmt +# 51| 0: [ThrowStmt] stmt +# 51| 0: [ClassInstanceExpr] new RuntimeException(...) +# 51| -3: [TypeAccess] RuntimeException +# 51| 0: [VarAccess] s +# 55| 0: [CatchClause] stmt +#-----| 0: (Single Local Variable Declaration) +# 55| 0: [TypeAccess] RuntimeException +# 55| 1: [LocalVariableDeclExpr] rte +# 55| 1: [BlockStmt] stmt +# 56| 0: [ReturnStmt] stmt diff --git a/java/ql/test/library-tests/printAst/options b/java/ql/test/library-tests/printAst/options new file mode 100644 index 00000000000..266b0eadc5e --- /dev/null +++ b/java/ql/test/library-tests/printAst/options @@ -0,0 +1 @@ +//semmle-extractor-options: --javac-args --enable-preview -source 14 -target 14 diff --git a/javascript/extractor/src/com/semmle/jcorn/Parser.java b/javascript/extractor/src/com/semmle/jcorn/Parser.java index a8348c3c307..7c018b6a456 100644 --- a/javascript/extractor/src/com/semmle/jcorn/Parser.java +++ b/javascript/extractor/src/com/semmle/jcorn/Parser.java @@ -531,9 +531,14 @@ public class Parser { int next2 = charAt(this.pos + 2); if (this.options.esnext()) { if (next == '.' && !('0' <= next2 && next2 <= '9')) // '?.', but not '?.X' where X is a digit - return this.finishOp(TokenType.questiondot, 2); - if (next == '?') // '??' - return this.finishOp(TokenType.questionquestion, 2); + return this.finishOp(TokenType.questiondot, 2); + if (next == '?') { // '??' + if (next2 == '=') { // ??= + return this.finishOp(TokenType.assign, 3); + } + return this.finishOp(TokenType.questionquestion, 2); + } + } return this.finishOp(TokenType.question, 1); } @@ -566,8 +571,11 @@ public class Parser { private Token readToken_pipe_amp(int code) { // '|&' int next = charAt(this.pos + 1); - if (next == code) + int next2 = charAt(this.pos + 2); + if (next == code) { // && || + if (next2 == 61) return this.finishOp(TokenType.assign, 3); // &&= ||= return this.finishOp(code == 124 ? TokenType.logicalOR : TokenType.logicalAND, 2); + } if (next == 61) return this.finishOp(TokenType.assign, 2); return this.finishOp(code == 124 ? TokenType.bitwiseOR : TokenType.bitwiseAND, 1); } @@ -709,8 +717,8 @@ public class Parser { case 42: // '%*' return this.readToken_mult_modulo_exp(code); - case 124: - case 38: // '|&' + case 124: // '|' + case 38: // '&' return this.readToken_pipe_amp(code); case 94: // '^' diff --git a/javascript/extractor/src/com/semmle/js/extractor/AutoBuild.java b/javascript/extractor/src/com/semmle/js/extractor/AutoBuild.java index 28c96f806f7..edff3c8a857 100644 --- a/javascript/extractor/src/com/semmle/js/extractor/AutoBuild.java +++ b/javascript/extractor/src/com/semmle/js/extractor/AutoBuild.java @@ -60,6 +60,7 @@ import com.semmle.util.io.WholeIO; import com.semmle.util.io.csv.CSVReader; import com.semmle.util.language.LegacyLanguage; import com.semmle.util.process.Env; +import com.semmle.util.process.Env.OS; import com.semmle.util.projectstructure.ProjectLayout; import com.semmle.util.trap.TrapWriter; @@ -1239,11 +1240,29 @@ protected DependencyInstallationResult preparePackagesAndDependencies(Set protected void extractXml() throws IOException { if (xmlExtensions.isEmpty()) return; List cmd = new ArrayList<>(); - cmd.add("odasa"); - cmd.add("index"); - cmd.add("--xml"); - cmd.add("--extensions"); - cmd.addAll(xmlExtensions); + if (EnvironmentVariables.getCodeQLDist() == null) { + // Use the legacy odasa XML extractor + cmd.add("odasa"); + cmd.add("index"); + cmd.add("--xml"); + cmd.add("--extensions"); + cmd.addAll(xmlExtensions); + } else { + String command = Env.getOS() == OS.WINDOWS ? "codeql.exe" : "codeql"; + cmd.add(Paths.get(EnvironmentVariables.getCodeQLDist(), command).toString()); + cmd.add("database"); + cmd.add("index-files"); + cmd.add("--language"); + cmd.add("xml"); + cmd.add("--size-limit"); + cmd.add("10m"); + for (String extension : xmlExtensions) { + cmd.add("--include-extension"); + cmd.add(extension); + } + cmd.add("--"); + cmd.add(EnvironmentVariables.getWipDatabase()); + } ProcessBuilder pb = new ProcessBuilder(cmd); try { pb.redirectError(Redirect.INHERIT); diff --git a/javascript/extractor/src/com/semmle/js/extractor/EnvironmentVariables.java b/javascript/extractor/src/com/semmle/js/extractor/EnvironmentVariables.java index 6f8e7124b3c..8ffcb65831c 100644 --- a/javascript/extractor/src/com/semmle/js/extractor/EnvironmentVariables.java +++ b/javascript/extractor/src/com/semmle/js/extractor/EnvironmentVariables.java @@ -14,6 +14,11 @@ public class EnvironmentVariables { public static final String LGTM_WORKSPACE_ENV_VAR = "LGTM_WORKSPACE"; + public static final String CODEQL_EXTRACTOR_JAVASCRIPT_WIP_DATABASE_ENV_VAR = + "CODEQL_EXTRACTOR_JAVASCRIPT_WIP_DATABASE"; + + public static final String CODEQL_DIST_ENV_VAR = "CODEQL_DIST"; + /** * Gets the extractor root based on the CODEQL_EXTRACTOR_JAVASCRIPT_ROOT or * SEMMLE_DIST or environment variable, or null if neither is set. @@ -49,4 +54,13 @@ public class EnvironmentVariables { throw new UserError(CODEQL_EXTRACTOR_JAVASCRIPT_SCRATCH_DIR_ENV_VAR + " or " + LGTM_WORKSPACE_ENV_VAR + " must be set"); } + + public static String getCodeQLDist() { + return Env.systemEnv().getNonEmpty(CODEQL_DIST_ENV_VAR); + } + + /** Gets the output database directory. */ + public static String getWipDatabase() { + return Env.systemEnv().getNonEmpty(CODEQL_EXTRACTOR_JAVASCRIPT_WIP_DATABASE_ENV_VAR); + } } diff --git a/javascript/extractor/src/com/semmle/js/extractor/Main.java b/javascript/extractor/src/com/semmle/js/extractor/Main.java index 359e07d3194..a7b209bc7ce 100644 --- a/javascript/extractor/src/com/semmle/js/extractor/Main.java +++ b/javascript/extractor/src/com/semmle/js/extractor/Main.java @@ -43,7 +43,7 @@ public class Main { * A version identifier that should be updated every time the extractor changes in such a way that * it may produce different tuples for the same file under the same {@link ExtractorConfig}. */ - public static final String EXTRACTOR_VERSION = "2020-09-12"; + public static final String EXTRACTOR_VERSION = "2020-09-17"; public static final Pattern NEWLINE = Pattern.compile("\n"); diff --git a/javascript/extractor/src/com/semmle/js/extractor/ScriptExtractor.java b/javascript/extractor/src/com/semmle/js/extractor/ScriptExtractor.java index b0ce54d02aa..7d8ccc6b3b1 100644 --- a/javascript/extractor/src/com/semmle/js/extractor/ScriptExtractor.java +++ b/javascript/extractor/src/com/semmle/js/extractor/ScriptExtractor.java @@ -126,7 +126,11 @@ public class ScriptExtractor implements IExtractor { } try { BufferedReader reader = new BufferedReader(new FileReader(file)); - String result = new Gson().fromJson(reader, PackageJSON.class).type; + PackageJSON pkgjson = new Gson().fromJson(reader, PackageJSON.class); + if (pkgjson == null) { + return null; + } + String result = pkgjson.type; packageTypeCache.put(folder, Optional.ofNullable(result)); return result; } catch (IOException | JsonSyntaxException e) { diff --git a/javascript/extractor/src/com/semmle/js/parser/TypeScriptASTConverter.java b/javascript/extractor/src/com/semmle/js/parser/TypeScriptASTConverter.java index 0e7031c4f9a..d2e5a0448c3 100644 --- a/javascript/extractor/src/com/semmle/js/parser/TypeScriptASTConverter.java +++ b/javascript/extractor/src/com/semmle/js/parser/TypeScriptASTConverter.java @@ -1421,8 +1421,19 @@ public class TypeScriptASTConverter { importStart = advance(importStart, m.group(0)); } } + + Node rawPath = convertChild(node, "argument"); + ITypeExpression path; + if (rawPath instanceof ITypeExpression) { + path = (ITypeExpression)rawPath; + } else if (rawPath instanceof TemplateLiteral) { + // this is a type-error, so we just fall back to some behavior that does not crash the extractor. + path = new Literal(rawPath.getLoc(), TokenType.string, ((TemplateLiteral)rawPath).getQuasis().stream().map(q -> q.getRaw()).collect(Collectors.joining(""))); + } else { + throw new ParseError("Unsupported syntax in import", getSourceLocation(node).getStart()); + } + // Find the ending parenthesis in `import(path)` by skipping whitespace after `path`. - ITypeExpression path = convertChild(node, "argument"); String endSrc = loc.getSource().substring(path.getLoc().getEnd().getOffset() - loc.getStart().getOffset()); Matcher m = WHITESPACE_END_PAREN.matcher(endSrc); diff --git a/javascript/extractor/src/com/semmle/js/parser/TypeScriptParser.java b/javascript/extractor/src/com/semmle/js/parser/TypeScriptParser.java index 2dea7826879..c091e39af55 100644 --- a/javascript/extractor/src/com/semmle/js/parser/TypeScriptParser.java +++ b/javascript/extractor/src/com/semmle/js/parser/TypeScriptParser.java @@ -299,6 +299,8 @@ public class TypeScriptParser { : getMegabyteCountFromPrefixedEnv(TYPESCRIPT_RAM_SUFFIX, 2000); int reserveMemoryMb = getMegabyteCountFromPrefixedEnv(TYPESCRIPT_RAM_RESERVE_SUFFIX, 400); + System.out.println("Memory for TypeScript process: " + mainMemoryMb + " MB, and " + reserveMemoryMb + " MB reserve"); + File parserWrapper = getParserWrapper(); String debugFlagString = Env.systemEnv().getNonEmpty(TYPESCRIPT_NODE_FLAGS); diff --git a/javascript/extractor/tests/es2021/input/assign.js b/javascript/extractor/tests/es2021/input/assign.js new file mode 100644 index 00000000000..7075534fe3b --- /dev/null +++ b/javascript/extractor/tests/es2021/input/assign.js @@ -0,0 +1,9 @@ +var x = 1; +var y = 2; +var foo = x && y; +var bar = x &&= y; +console.log(x); // 2 + +x &&= y; +x ||= y; +x ??= y; diff --git a/javascript/extractor/tests/es2021/input/numeric.js b/javascript/extractor/tests/es2021/input/numeric.js new file mode 100644 index 00000000000..e7e9d031f50 --- /dev/null +++ b/javascript/extractor/tests/es2021/input/numeric.js @@ -0,0 +1,8 @@ +1_000_000_000 // Ah, so a billion +101_475_938.38 // And this is hundreds of millions + +let fee = 123_00; // $123 (12300 cents, apparently) +let fee = 12_300; // $12,300 (woah, that fee!) +let amount = 12345_00; // 12,345 (1234500 cents, apparently) +let amount = 123_4500; // 123.45 (4-fixed financial) +let amount = 1_234_500; // 1,234,500 diff --git a/javascript/extractor/tests/es2021/options.json b/javascript/extractor/tests/es2021/options.json new file mode 100644 index 00000000000..075583ca1f6 --- /dev/null +++ b/javascript/extractor/tests/es2021/options.json @@ -0,0 +1,3 @@ +{ + "experimental": true +} diff --git a/javascript/extractor/tests/es2021/output/trap/assign.js.trap b/javascript/extractor/tests/es2021/output/trap/assign.js.trap new file mode 100644 index 00000000000..1fb2b771f15 --- /dev/null +++ b/javascript/extractor/tests/es2021/output/trap/assign.js.trap @@ -0,0 +1,633 @@ +#10000=@"/assign.js;sourcefile" +files(#10000,"/assign.js","assign","js",0) +#10001=@"/;folder" +folders(#10001,"/","") +containerparent(#10001,#10000) +#10002=@"loc,{#10000},0,0,0,0" +locations_default(#10002,#10000,0,0,0,0) +hasLocation(#10000,#10002) +#20000=@"global_scope" +scopes(#20000,0) +#20001=@"script;{#10000},1,1" +#20002=* +comments(#20002,0,#20001," 2","// 2") +#20003=@"loc,{#10000},5,17,5,20" +locations_default(#20003,#10000,5,17,5,20) +hasLocation(#20002,#20003) +#20004=* +lines(#20004,#20001,"var x = 1;"," +") +#20005=@"loc,{#10000},1,1,1,10" +locations_default(#20005,#10000,1,1,1,10) +hasLocation(#20004,#20005) +#20006=* +lines(#20006,#20001,"var y = 2;"," +") +#20007=@"loc,{#10000},2,1,2,10" +locations_default(#20007,#10000,2,1,2,10) +hasLocation(#20006,#20007) +#20008=* +lines(#20008,#20001,"var foo = x && y;"," +") +#20009=@"loc,{#10000},3,1,3,17" +locations_default(#20009,#10000,3,1,3,17) +hasLocation(#20008,#20009) +#20010=* +lines(#20010,#20001,"var bar = x &&= y;"," +") +#20011=@"loc,{#10000},4,1,4,18" +locations_default(#20011,#10000,4,1,4,18) +hasLocation(#20010,#20011) +#20012=* +lines(#20012,#20001,"console.log(x); // 2"," +") +#20013=@"loc,{#10000},5,1,5,20" +locations_default(#20013,#10000,5,1,5,20) +hasLocation(#20012,#20013) +#20014=* +lines(#20014,#20001,""," +") +#20015=@"loc,{#10000},6,1,6,0" +locations_default(#20015,#10000,6,1,6,0) +hasLocation(#20014,#20015) +#20016=* +lines(#20016,#20001,"x &&= y;"," +") +#20017=@"loc,{#10000},7,1,7,8" +locations_default(#20017,#10000,7,1,7,8) +hasLocation(#20016,#20017) +#20018=* +lines(#20018,#20001,"x ||= y;"," +") +#20019=@"loc,{#10000},8,1,8,8" +locations_default(#20019,#10000,8,1,8,8) +hasLocation(#20018,#20019) +#20020=* +lines(#20020,#20001,"x ??= y;"," +") +#20021=@"loc,{#10000},9,1,9,8" +locations_default(#20021,#10000,9,1,9,8) +hasLocation(#20020,#20021) +numlines(#20001,9,8,1) +#20022=* +tokeninfo(#20022,7,#20001,0,"var") +#20023=@"loc,{#10000},1,1,1,3" +locations_default(#20023,#10000,1,1,1,3) +hasLocation(#20022,#20023) +#20024=* +tokeninfo(#20024,6,#20001,1,"x") +#20025=@"loc,{#10000},1,5,1,5" +locations_default(#20025,#10000,1,5,1,5) +hasLocation(#20024,#20025) +#20026=* +tokeninfo(#20026,8,#20001,2,"=") +#20027=@"loc,{#10000},1,7,1,7" +locations_default(#20027,#10000,1,7,1,7) +hasLocation(#20026,#20027) +#20028=* +tokeninfo(#20028,3,#20001,3,"1") +#20029=@"loc,{#10000},1,9,1,9" +locations_default(#20029,#10000,1,9,1,9) +hasLocation(#20028,#20029) +#20030=* +tokeninfo(#20030,8,#20001,4,";") +#20031=@"loc,{#10000},1,10,1,10" +locations_default(#20031,#10000,1,10,1,10) +hasLocation(#20030,#20031) +#20032=* +tokeninfo(#20032,7,#20001,5,"var") +#20033=@"loc,{#10000},2,1,2,3" +locations_default(#20033,#10000,2,1,2,3) +hasLocation(#20032,#20033) +#20034=* +tokeninfo(#20034,6,#20001,6,"y") +#20035=@"loc,{#10000},2,5,2,5" +locations_default(#20035,#10000,2,5,2,5) +hasLocation(#20034,#20035) +#20036=* +tokeninfo(#20036,8,#20001,7,"=") +#20037=@"loc,{#10000},2,7,2,7" +locations_default(#20037,#10000,2,7,2,7) +hasLocation(#20036,#20037) +#20038=* +tokeninfo(#20038,3,#20001,8,"2") +#20039=@"loc,{#10000},2,9,2,9" +locations_default(#20039,#10000,2,9,2,9) +hasLocation(#20038,#20039) +#20040=* +tokeninfo(#20040,8,#20001,9,";") +#20041=@"loc,{#10000},2,10,2,10" +locations_default(#20041,#10000,2,10,2,10) +hasLocation(#20040,#20041) +#20042=* +tokeninfo(#20042,7,#20001,10,"var") +#20043=@"loc,{#10000},3,1,3,3" +locations_default(#20043,#10000,3,1,3,3) +hasLocation(#20042,#20043) +#20044=* +tokeninfo(#20044,6,#20001,11,"foo") +#20045=@"loc,{#10000},3,5,3,7" +locations_default(#20045,#10000,3,5,3,7) +hasLocation(#20044,#20045) +#20046=* +tokeninfo(#20046,8,#20001,12,"=") +#20047=@"loc,{#10000},3,9,3,9" +locations_default(#20047,#10000,3,9,3,9) +hasLocation(#20046,#20047) +#20048=* +tokeninfo(#20048,6,#20001,13,"x") +#20049=@"loc,{#10000},3,11,3,11" +locations_default(#20049,#10000,3,11,3,11) +hasLocation(#20048,#20049) +#20050=* +tokeninfo(#20050,8,#20001,14,"&&") +#20051=@"loc,{#10000},3,13,3,14" +locations_default(#20051,#10000,3,13,3,14) +hasLocation(#20050,#20051) +#20052=* +tokeninfo(#20052,6,#20001,15,"y") +#20053=@"loc,{#10000},3,16,3,16" +locations_default(#20053,#10000,3,16,3,16) +hasLocation(#20052,#20053) +#20054=* +tokeninfo(#20054,8,#20001,16,";") +#20055=@"loc,{#10000},3,17,3,17" +locations_default(#20055,#10000,3,17,3,17) +hasLocation(#20054,#20055) +#20056=* +tokeninfo(#20056,7,#20001,17,"var") +#20057=@"loc,{#10000},4,1,4,3" +locations_default(#20057,#10000,4,1,4,3) +hasLocation(#20056,#20057) +#20058=* +tokeninfo(#20058,6,#20001,18,"bar") +#20059=@"loc,{#10000},4,5,4,7" +locations_default(#20059,#10000,4,5,4,7) +hasLocation(#20058,#20059) +#20060=* +tokeninfo(#20060,8,#20001,19,"=") +#20061=@"loc,{#10000},4,9,4,9" +locations_default(#20061,#10000,4,9,4,9) +hasLocation(#20060,#20061) +#20062=* +tokeninfo(#20062,6,#20001,20,"x") +#20063=@"loc,{#10000},4,11,4,11" +locations_default(#20063,#10000,4,11,4,11) +hasLocation(#20062,#20063) +#20064=* +tokeninfo(#20064,8,#20001,21,"&&=") +#20065=@"loc,{#10000},4,13,4,15" +locations_default(#20065,#10000,4,13,4,15) +hasLocation(#20064,#20065) +#20066=* +tokeninfo(#20066,6,#20001,22,"y") +#20067=@"loc,{#10000},4,17,4,17" +locations_default(#20067,#10000,4,17,4,17) +hasLocation(#20066,#20067) +#20068=* +tokeninfo(#20068,8,#20001,23,";") +#20069=@"loc,{#10000},4,18,4,18" +locations_default(#20069,#10000,4,18,4,18) +hasLocation(#20068,#20069) +#20070=* +tokeninfo(#20070,6,#20001,24,"console") +#20071=@"loc,{#10000},5,1,5,7" +locations_default(#20071,#10000,5,1,5,7) +hasLocation(#20070,#20071) +#20072=* +tokeninfo(#20072,8,#20001,25,".") +#20073=@"loc,{#10000},5,8,5,8" +locations_default(#20073,#10000,5,8,5,8) +hasLocation(#20072,#20073) +#20074=* +tokeninfo(#20074,6,#20001,26,"log") +#20075=@"loc,{#10000},5,9,5,11" +locations_default(#20075,#10000,5,9,5,11) +hasLocation(#20074,#20075) +#20076=* +tokeninfo(#20076,8,#20001,27,"(") +#20077=@"loc,{#10000},5,12,5,12" +locations_default(#20077,#10000,5,12,5,12) +hasLocation(#20076,#20077) +#20078=* +tokeninfo(#20078,6,#20001,28,"x") +#20079=@"loc,{#10000},5,13,5,13" +locations_default(#20079,#10000,5,13,5,13) +hasLocation(#20078,#20079) +#20080=* +tokeninfo(#20080,8,#20001,29,")") +#20081=@"loc,{#10000},5,14,5,14" +locations_default(#20081,#10000,5,14,5,14) +hasLocation(#20080,#20081) +#20082=* +tokeninfo(#20082,8,#20001,30,";") +#20083=@"loc,{#10000},5,15,5,15" +locations_default(#20083,#10000,5,15,5,15) +hasLocation(#20082,#20083) +#20084=* +tokeninfo(#20084,6,#20001,31,"x") +#20085=@"loc,{#10000},7,1,7,1" +locations_default(#20085,#10000,7,1,7,1) +hasLocation(#20084,#20085) +next_token(#20002,#20084) +#20086=* +tokeninfo(#20086,8,#20001,32,"&&=") +#20087=@"loc,{#10000},7,3,7,5" +locations_default(#20087,#10000,7,3,7,5) +hasLocation(#20086,#20087) +#20088=* +tokeninfo(#20088,6,#20001,33,"y") +#20089=@"loc,{#10000},7,7,7,7" +locations_default(#20089,#10000,7,7,7,7) +hasLocation(#20088,#20089) +#20090=* +tokeninfo(#20090,8,#20001,34,";") +#20091=@"loc,{#10000},7,8,7,8" +locations_default(#20091,#10000,7,8,7,8) +hasLocation(#20090,#20091) +#20092=* +tokeninfo(#20092,6,#20001,35,"x") +#20093=@"loc,{#10000},8,1,8,1" +locations_default(#20093,#10000,8,1,8,1) +hasLocation(#20092,#20093) +#20094=* +tokeninfo(#20094,8,#20001,36,"||=") +#20095=@"loc,{#10000},8,3,8,5" +locations_default(#20095,#10000,8,3,8,5) +hasLocation(#20094,#20095) +#20096=* +tokeninfo(#20096,6,#20001,37,"y") +#20097=@"loc,{#10000},8,7,8,7" +locations_default(#20097,#10000,8,7,8,7) +hasLocation(#20096,#20097) +#20098=* +tokeninfo(#20098,8,#20001,38,";") +#20099=@"loc,{#10000},8,8,8,8" +locations_default(#20099,#10000,8,8,8,8) +hasLocation(#20098,#20099) +#20100=* +tokeninfo(#20100,6,#20001,39,"x") +#20101=@"loc,{#10000},9,1,9,1" +locations_default(#20101,#10000,9,1,9,1) +hasLocation(#20100,#20101) +#20102=* +tokeninfo(#20102,8,#20001,40,"??=") +#20103=@"loc,{#10000},9,3,9,5" +locations_default(#20103,#10000,9,3,9,5) +hasLocation(#20102,#20103) +#20104=* +tokeninfo(#20104,6,#20001,41,"y") +#20105=@"loc,{#10000},9,7,9,7" +locations_default(#20105,#10000,9,7,9,7) +hasLocation(#20104,#20105) +#20106=* +tokeninfo(#20106,8,#20001,42,";") +#20107=@"loc,{#10000},9,8,9,8" +locations_default(#20107,#10000,9,8,9,8) +hasLocation(#20106,#20107) +#20108=* +tokeninfo(#20108,0,#20001,43,"") +#20109=@"loc,{#10000},10,1,10,0" +locations_default(#20109,#10000,10,1,10,0) +hasLocation(#20108,#20109) +toplevels(#20001,0) +#20110=@"loc,{#10000},1,1,10,0" +locations_default(#20110,#10000,1,1,10,0) +hasLocation(#20001,#20110) +#20111=@"var;{x};{#20000}" +variables(#20111,"x",#20000) +#20112=@"var;{y};{#20000}" +variables(#20112,"y",#20000) +#20113=@"var;{foo};{#20000}" +variables(#20113,"foo",#20000) +#20114=@"var;{bar};{#20000}" +variables(#20114,"bar",#20000) +#20115=* +stmts(#20115,18,#20001,0,"var x = 1;") +hasLocation(#20115,#20005) +stmt_containers(#20115,#20001) +#20116=* +exprs(#20116,64,#20115,0,"x = 1") +#20117=@"loc,{#10000},1,5,1,9" +locations_default(#20117,#10000,1,5,1,9) +hasLocation(#20116,#20117) +enclosing_stmt(#20116,#20115) +expr_containers(#20116,#20001) +#20118=* +exprs(#20118,78,#20116,0,"x") +hasLocation(#20118,#20025) +enclosing_stmt(#20118,#20115) +expr_containers(#20118,#20001) +literals("x","x",#20118) +decl(#20118,#20111) +#20119=* +exprs(#20119,3,#20116,1,"1") +hasLocation(#20119,#20029) +enclosing_stmt(#20119,#20115) +expr_containers(#20119,#20001) +literals("1","1",#20119) +#20120=* +stmts(#20120,18,#20001,1,"var y = 2;") +hasLocation(#20120,#20007) +stmt_containers(#20120,#20001) +#20121=* +exprs(#20121,64,#20120,0,"y = 2") +#20122=@"loc,{#10000},2,5,2,9" +locations_default(#20122,#10000,2,5,2,9) +hasLocation(#20121,#20122) +enclosing_stmt(#20121,#20120) +expr_containers(#20121,#20001) +#20123=* +exprs(#20123,78,#20121,0,"y") +hasLocation(#20123,#20035) +enclosing_stmt(#20123,#20120) +expr_containers(#20123,#20001) +literals("y","y",#20123) +decl(#20123,#20112) +#20124=* +exprs(#20124,3,#20121,1,"2") +hasLocation(#20124,#20039) +enclosing_stmt(#20124,#20120) +expr_containers(#20124,#20001) +literals("2","2",#20124) +#20125=* +stmts(#20125,18,#20001,2,"var foo = x && y;") +hasLocation(#20125,#20009) +stmt_containers(#20125,#20001) +#20126=* +exprs(#20126,64,#20125,0,"foo = x && y") +#20127=@"loc,{#10000},3,5,3,16" +locations_default(#20127,#10000,3,5,3,16) +hasLocation(#20126,#20127) +enclosing_stmt(#20126,#20125) +expr_containers(#20126,#20001) +#20128=* +exprs(#20128,78,#20126,0,"foo") +hasLocation(#20128,#20045) +enclosing_stmt(#20128,#20125) +expr_containers(#20128,#20001) +literals("foo","foo",#20128) +decl(#20128,#20113) +#20129=* +exprs(#20129,44,#20126,1,"x && y") +#20130=@"loc,{#10000},3,11,3,16" +locations_default(#20130,#10000,3,11,3,16) +hasLocation(#20129,#20130) +enclosing_stmt(#20129,#20125) +expr_containers(#20129,#20001) +#20131=* +exprs(#20131,79,#20129,0,"x") +hasLocation(#20131,#20049) +enclosing_stmt(#20131,#20125) +expr_containers(#20131,#20001) +literals("x","x",#20131) +bind(#20131,#20111) +#20132=* +exprs(#20132,79,#20129,1,"y") +hasLocation(#20132,#20053) +enclosing_stmt(#20132,#20125) +expr_containers(#20132,#20001) +literals("y","y",#20132) +bind(#20132,#20112) +#20133=* +stmts(#20133,18,#20001,3,"var bar = x &&= y;") +hasLocation(#20133,#20011) +stmt_containers(#20133,#20001) +#20134=* +exprs(#20134,64,#20133,0,"bar = x &&= y") +#20135=@"loc,{#10000},4,5,4,17" +locations_default(#20135,#10000,4,5,4,17) +hasLocation(#20134,#20135) +enclosing_stmt(#20134,#20133) +expr_containers(#20134,#20001) +#20136=* +exprs(#20136,78,#20134,0,"bar") +hasLocation(#20136,#20059) +enclosing_stmt(#20136,#20133) +expr_containers(#20136,#20001) +literals("bar","bar",#20136) +decl(#20136,#20114) +#20137=* +exprs(#20137,116,#20134,1,"x &&= y") +#20138=@"loc,{#10000},4,11,4,17" +locations_default(#20138,#10000,4,11,4,17) +hasLocation(#20137,#20138) +enclosing_stmt(#20137,#20133) +expr_containers(#20137,#20001) +#20139=* +exprs(#20139,79,#20137,0,"x") +hasLocation(#20139,#20063) +enclosing_stmt(#20139,#20133) +expr_containers(#20139,#20001) +literals("x","x",#20139) +bind(#20139,#20111) +#20140=* +exprs(#20140,79,#20137,1,"y") +hasLocation(#20140,#20067) +enclosing_stmt(#20140,#20133) +expr_containers(#20140,#20001) +literals("y","y",#20140) +bind(#20140,#20112) +#20141=* +stmts(#20141,2,#20001,4,"console.log(x);") +#20142=@"loc,{#10000},5,1,5,15" +locations_default(#20142,#10000,5,1,5,15) +hasLocation(#20141,#20142) +stmt_containers(#20141,#20001) +#20143=* +exprs(#20143,13,#20141,0,"console.log(x)") +#20144=@"loc,{#10000},5,1,5,14" +locations_default(#20144,#10000,5,1,5,14) +hasLocation(#20143,#20144) +enclosing_stmt(#20143,#20141) +expr_containers(#20143,#20001) +#20145=* +exprs(#20145,14,#20143,-1,"console.log") +#20146=@"loc,{#10000},5,1,5,11" +locations_default(#20146,#10000,5,1,5,11) +hasLocation(#20145,#20146) +enclosing_stmt(#20145,#20141) +expr_containers(#20145,#20001) +#20147=* +exprs(#20147,79,#20145,0,"console") +hasLocation(#20147,#20071) +enclosing_stmt(#20147,#20141) +expr_containers(#20147,#20001) +literals("console","console",#20147) +#20148=@"var;{console};{#20000}" +variables(#20148,"console",#20000) +bind(#20147,#20148) +#20149=* +exprs(#20149,0,#20145,1,"log") +hasLocation(#20149,#20075) +enclosing_stmt(#20149,#20141) +expr_containers(#20149,#20001) +literals("log","log",#20149) +#20150=* +exprs(#20150,79,#20143,0,"x") +hasLocation(#20150,#20079) +enclosing_stmt(#20150,#20141) +expr_containers(#20150,#20001) +literals("x","x",#20150) +bind(#20150,#20111) +#20151=* +stmts(#20151,2,#20001,5,"x &&= y;") +hasLocation(#20151,#20017) +stmt_containers(#20151,#20001) +#20152=* +exprs(#20152,116,#20151,0,"x &&= y") +#20153=@"loc,{#10000},7,1,7,7" +locations_default(#20153,#10000,7,1,7,7) +hasLocation(#20152,#20153) +enclosing_stmt(#20152,#20151) +expr_containers(#20152,#20001) +#20154=* +exprs(#20154,79,#20152,0,"x") +hasLocation(#20154,#20085) +enclosing_stmt(#20154,#20151) +expr_containers(#20154,#20001) +literals("x","x",#20154) +bind(#20154,#20111) +#20155=* +exprs(#20155,79,#20152,1,"y") +hasLocation(#20155,#20089) +enclosing_stmt(#20155,#20151) +expr_containers(#20155,#20001) +literals("y","y",#20155) +bind(#20155,#20112) +#20156=* +stmts(#20156,2,#20001,6,"x ||= y;") +hasLocation(#20156,#20019) +stmt_containers(#20156,#20001) +#20157=* +exprs(#20157,117,#20156,0,"x ||= y") +#20158=@"loc,{#10000},8,1,8,7" +locations_default(#20158,#10000,8,1,8,7) +hasLocation(#20157,#20158) +enclosing_stmt(#20157,#20156) +expr_containers(#20157,#20001) +#20159=* +exprs(#20159,79,#20157,0,"x") +hasLocation(#20159,#20093) +enclosing_stmt(#20159,#20156) +expr_containers(#20159,#20001) +literals("x","x",#20159) +bind(#20159,#20111) +#20160=* +exprs(#20160,79,#20157,1,"y") +hasLocation(#20160,#20097) +enclosing_stmt(#20160,#20156) +expr_containers(#20160,#20001) +literals("y","y",#20160) +bind(#20160,#20112) +#20161=* +stmts(#20161,2,#20001,7,"x ??= y;") +hasLocation(#20161,#20021) +stmt_containers(#20161,#20001) +#20162=* +exprs(#20162,118,#20161,0,"x ??= y") +#20163=@"loc,{#10000},9,1,9,7" +locations_default(#20163,#10000,9,1,9,7) +hasLocation(#20162,#20163) +enclosing_stmt(#20162,#20161) +expr_containers(#20162,#20001) +#20164=* +exprs(#20164,79,#20162,0,"x") +hasLocation(#20164,#20101) +enclosing_stmt(#20164,#20161) +expr_containers(#20164,#20001) +literals("x","x",#20164) +bind(#20164,#20111) +#20165=* +exprs(#20165,79,#20162,1,"y") +hasLocation(#20165,#20105) +enclosing_stmt(#20165,#20161) +expr_containers(#20165,#20001) +literals("y","y",#20165) +bind(#20165,#20112) +#20166=* +entry_cfg_node(#20166,#20001) +#20167=@"loc,{#10000},1,1,1,0" +locations_default(#20167,#10000,1,1,1,0) +hasLocation(#20166,#20167) +#20168=* +exit_cfg_node(#20168,#20001) +hasLocation(#20168,#20109) +successor(#20161,#20164) +successor(#20164,#20165) +successor(#20164,#20168) +successor(#20165,#20164) +successor(#20162,#20168) +successor(#20156,#20159) +#20169=* +guard_node(#20169,1,#20159) +hasLocation(#20169,#20093) +successor(#20169,#20161) +#20170=* +guard_node(#20170,0,#20159) +hasLocation(#20170,#20093) +successor(#20170,#20160) +successor(#20159,#20169) +successor(#20159,#20170) +successor(#20160,#20159) +successor(#20157,#20161) +successor(#20151,#20154) +#20171=* +guard_node(#20171,1,#20154) +hasLocation(#20171,#20085) +successor(#20171,#20155) +#20172=* +guard_node(#20172,0,#20154) +hasLocation(#20172,#20085) +successor(#20172,#20156) +successor(#20154,#20171) +successor(#20154,#20172) +successor(#20155,#20154) +successor(#20152,#20156) +successor(#20141,#20147) +successor(#20150,#20143) +successor(#20149,#20145) +successor(#20147,#20149) +successor(#20145,#20150) +successor(#20143,#20151) +successor(#20133,#20136) +#20173=* +guard_node(#20173,1,#20139) +hasLocation(#20173,#20063) +successor(#20173,#20140) +#20174=* +guard_node(#20174,0,#20139) +hasLocation(#20174,#20063) +successor(#20174,#20134) +successor(#20139,#20173) +successor(#20139,#20174) +successor(#20140,#20139) +successor(#20137,#20134) +successor(#20136,#20139) +successor(#20134,#20141) +successor(#20125,#20128) +successor(#20129,#20131) +#20175=* +guard_node(#20175,1,#20131) +hasLocation(#20175,#20049) +successor(#20175,#20132) +#20176=* +guard_node(#20176,0,#20131) +hasLocation(#20176,#20049) +successor(#20176,#20126) +successor(#20131,#20175) +successor(#20131,#20176) +successor(#20132,#20126) +successor(#20128,#20129) +successor(#20126,#20133) +successor(#20120,#20123) +successor(#20124,#20121) +successor(#20123,#20124) +successor(#20121,#20125) +successor(#20115,#20118) +successor(#20119,#20116) +successor(#20118,#20119) +successor(#20116,#20120) +successor(#20166,#20115) +numlines(#10000,9,8,1) +filetype(#10000,"javascript") diff --git a/javascript/extractor/tests/es2021/output/trap/numeric.js.trap b/javascript/extractor/tests/es2021/output/trap/numeric.js.trap new file mode 100644 index 00000000000..afcf1627f2d --- /dev/null +++ b/javascript/extractor/tests/es2021/output/trap/numeric.js.trap @@ -0,0 +1,435 @@ +#10000=@"/numeric.js;sourcefile" +files(#10000,"/numeric.js","numeric","js",0) +#10001=@"/;folder" +folders(#10001,"/","") +containerparent(#10001,#10000) +#10002=@"loc,{#10000},0,0,0,0" +locations_default(#10002,#10000,0,0,0,0) +hasLocation(#10000,#10002) +#20000=@"global_scope" +scopes(#20000,0) +#20001=@"script;{#10000},1,1" +#20002=* +comments(#20002,0,#20001," Ah, so a billion","// Ah, so a billion") +#20003=@"loc,{#10000},1,25,1,43" +locations_default(#20003,#10000,1,25,1,43) +hasLocation(#20002,#20003) +#20004=* +comments(#20004,0,#20001," And this is hundreds of millions","// And ... illions") +#20005=@"loc,{#10000},2,25,2,59" +locations_default(#20005,#10000,2,25,2,59) +hasLocation(#20004,#20005) +#20006=* +comments(#20006,0,#20001," $123 (12300 cents, apparently)","// $123 ... rently)") +#20007=@"loc,{#10000},4,25,4,57" +locations_default(#20007,#10000,4,25,4,57) +hasLocation(#20006,#20007) +#20008=* +comments(#20008,0,#20001," $12,300 (woah, that fee!)","// $12, ... t fee!)") +#20009=@"loc,{#10000},5,25,5,52" +locations_default(#20009,#10000,5,25,5,52) +hasLocation(#20008,#20009) +#20010=* +comments(#20010,0,#20001," 12,345 (1234500 cents, apparently)","// 12,3 ... rently)") +#20011=@"loc,{#10000},6,25,6,61" +locations_default(#20011,#10000,6,25,6,61) +hasLocation(#20010,#20011) +#20012=* +comments(#20012,0,#20001," 123.45 (4-fixed financial)","// 123. ... ancial)") +#20013=@"loc,{#10000},7,25,7,53" +locations_default(#20013,#10000,7,25,7,53) +hasLocation(#20012,#20013) +#20014=* +comments(#20014,0,#20001," 1,234,500","// 1,234,500") +#20015=@"loc,{#10000},8,25,8,36" +locations_default(#20015,#10000,8,25,8,36) +hasLocation(#20014,#20015) +#20016=* +lines(#20016,#20001,"1_000_000_000 // Ah, so a billion"," +") +#20017=@"loc,{#10000},1,1,1,43" +locations_default(#20017,#10000,1,1,1,43) +hasLocation(#20016,#20017) +#20018=* +lines(#20018,#20001,"101_475_938.38 // And this is hundreds of millions"," +") +#20019=@"loc,{#10000},2,1,2,59" +locations_default(#20019,#10000,2,1,2,59) +hasLocation(#20018,#20019) +#20020=* +lines(#20020,#20001,""," +") +#20021=@"loc,{#10000},3,1,3,0" +locations_default(#20021,#10000,3,1,3,0) +hasLocation(#20020,#20021) +#20022=* +lines(#20022,#20001,"let fee = 123_00; // $123 (12300 cents, apparently)"," +") +#20023=@"loc,{#10000},4,1,4,57" +locations_default(#20023,#10000,4,1,4,57) +hasLocation(#20022,#20023) +#20024=* +lines(#20024,#20001,"let fee = 12_300; // $12,300 (woah, that fee!)"," +") +#20025=@"loc,{#10000},5,1,5,52" +locations_default(#20025,#10000,5,1,5,52) +hasLocation(#20024,#20025) +#20026=* +lines(#20026,#20001,"let amount = 12345_00; // 12,345 (1234500 cents, apparently)"," +") +#20027=@"loc,{#10000},6,1,6,61" +locations_default(#20027,#10000,6,1,6,61) +hasLocation(#20026,#20027) +#20028=* +lines(#20028,#20001,"let amount = 123_4500; // 123.45 (4-fixed financial)"," +") +#20029=@"loc,{#10000},7,1,7,53" +locations_default(#20029,#10000,7,1,7,53) +hasLocation(#20028,#20029) +#20030=* +lines(#20030,#20001,"let amount = 1_234_500; // 1,234,500"," +") +#20031=@"loc,{#10000},8,1,8,36" +locations_default(#20031,#10000,8,1,8,36) +hasLocation(#20030,#20031) +numlines(#20001,8,7,7) +#20032=* +tokeninfo(#20032,3,#20001,0,"1_000_000_000") +#20033=@"loc,{#10000},1,1,1,13" +locations_default(#20033,#10000,1,1,1,13) +hasLocation(#20032,#20033) +#20034=* +tokeninfo(#20034,3,#20001,1,"101_475_938.38") +#20035=@"loc,{#10000},2,1,2,14" +locations_default(#20035,#10000,2,1,2,14) +hasLocation(#20034,#20035) +next_token(#20002,#20034) +#20036=* +tokeninfo(#20036,7,#20001,2,"let") +#20037=@"loc,{#10000},4,1,4,3" +locations_default(#20037,#10000,4,1,4,3) +hasLocation(#20036,#20037) +next_token(#20004,#20036) +#20038=* +tokeninfo(#20038,6,#20001,3,"fee") +#20039=@"loc,{#10000},4,5,4,7" +locations_default(#20039,#10000,4,5,4,7) +hasLocation(#20038,#20039) +#20040=* +tokeninfo(#20040,8,#20001,4,"=") +#20041=@"loc,{#10000},4,9,4,9" +locations_default(#20041,#10000,4,9,4,9) +hasLocation(#20040,#20041) +#20042=* +tokeninfo(#20042,3,#20001,5,"123_00") +#20043=@"loc,{#10000},4,11,4,16" +locations_default(#20043,#10000,4,11,4,16) +hasLocation(#20042,#20043) +#20044=* +tokeninfo(#20044,8,#20001,6,";") +#20045=@"loc,{#10000},4,17,4,17" +locations_default(#20045,#10000,4,17,4,17) +hasLocation(#20044,#20045) +#20046=* +tokeninfo(#20046,7,#20001,7,"let") +#20047=@"loc,{#10000},5,1,5,3" +locations_default(#20047,#10000,5,1,5,3) +hasLocation(#20046,#20047) +next_token(#20006,#20046) +#20048=* +tokeninfo(#20048,6,#20001,8,"fee") +#20049=@"loc,{#10000},5,5,5,7" +locations_default(#20049,#10000,5,5,5,7) +hasLocation(#20048,#20049) +#20050=* +tokeninfo(#20050,8,#20001,9,"=") +#20051=@"loc,{#10000},5,9,5,9" +locations_default(#20051,#10000,5,9,5,9) +hasLocation(#20050,#20051) +#20052=* +tokeninfo(#20052,3,#20001,10,"12_300") +#20053=@"loc,{#10000},5,11,5,16" +locations_default(#20053,#10000,5,11,5,16) +hasLocation(#20052,#20053) +#20054=* +tokeninfo(#20054,8,#20001,11,";") +#20055=@"loc,{#10000},5,17,5,17" +locations_default(#20055,#10000,5,17,5,17) +hasLocation(#20054,#20055) +#20056=* +tokeninfo(#20056,7,#20001,12,"let") +#20057=@"loc,{#10000},6,1,6,3" +locations_default(#20057,#10000,6,1,6,3) +hasLocation(#20056,#20057) +next_token(#20008,#20056) +#20058=* +tokeninfo(#20058,6,#20001,13,"amount") +#20059=@"loc,{#10000},6,5,6,10" +locations_default(#20059,#10000,6,5,6,10) +hasLocation(#20058,#20059) +#20060=* +tokeninfo(#20060,8,#20001,14,"=") +#20061=@"loc,{#10000},6,12,6,12" +locations_default(#20061,#10000,6,12,6,12) +hasLocation(#20060,#20061) +#20062=* +tokeninfo(#20062,3,#20001,15,"12345_00") +#20063=@"loc,{#10000},6,14,6,21" +locations_default(#20063,#10000,6,14,6,21) +hasLocation(#20062,#20063) +#20064=* +tokeninfo(#20064,8,#20001,16,";") +#20065=@"loc,{#10000},6,22,6,22" +locations_default(#20065,#10000,6,22,6,22) +hasLocation(#20064,#20065) +#20066=* +tokeninfo(#20066,7,#20001,17,"let") +#20067=@"loc,{#10000},7,1,7,3" +locations_default(#20067,#10000,7,1,7,3) +hasLocation(#20066,#20067) +next_token(#20010,#20066) +#20068=* +tokeninfo(#20068,6,#20001,18,"amount") +#20069=@"loc,{#10000},7,5,7,10" +locations_default(#20069,#10000,7,5,7,10) +hasLocation(#20068,#20069) +#20070=* +tokeninfo(#20070,8,#20001,19,"=") +#20071=@"loc,{#10000},7,12,7,12" +locations_default(#20071,#10000,7,12,7,12) +hasLocation(#20070,#20071) +#20072=* +tokeninfo(#20072,3,#20001,20,"123_4500") +#20073=@"loc,{#10000},7,14,7,21" +locations_default(#20073,#10000,7,14,7,21) +hasLocation(#20072,#20073) +#20074=* +tokeninfo(#20074,8,#20001,21,";") +#20075=@"loc,{#10000},7,22,7,22" +locations_default(#20075,#10000,7,22,7,22) +hasLocation(#20074,#20075) +#20076=* +tokeninfo(#20076,7,#20001,22,"let") +#20077=@"loc,{#10000},8,1,8,3" +locations_default(#20077,#10000,8,1,8,3) +hasLocation(#20076,#20077) +next_token(#20012,#20076) +#20078=* +tokeninfo(#20078,6,#20001,23,"amount") +#20079=@"loc,{#10000},8,5,8,10" +locations_default(#20079,#10000,8,5,8,10) +hasLocation(#20078,#20079) +#20080=* +tokeninfo(#20080,8,#20001,24,"=") +#20081=@"loc,{#10000},8,12,8,12" +locations_default(#20081,#10000,8,12,8,12) +hasLocation(#20080,#20081) +#20082=* +tokeninfo(#20082,3,#20001,25,"1_234_500") +#20083=@"loc,{#10000},8,14,8,22" +locations_default(#20083,#10000,8,14,8,22) +hasLocation(#20082,#20083) +#20084=* +tokeninfo(#20084,8,#20001,26,";") +#20085=@"loc,{#10000},8,23,8,23" +locations_default(#20085,#10000,8,23,8,23) +hasLocation(#20084,#20085) +#20086=* +tokeninfo(#20086,0,#20001,27,"") +#20087=@"loc,{#10000},9,1,9,0" +locations_default(#20087,#10000,9,1,9,0) +hasLocation(#20086,#20087) +next_token(#20014,#20086) +toplevels(#20001,0) +#20088=@"loc,{#10000},1,1,9,0" +locations_default(#20088,#10000,1,1,9,0) +hasLocation(#20001,#20088) +#20089=@"var;{fee};{#20000}" +variables(#20089,"fee",#20000) +#20090=@"var;{amount};{#20000}" +variables(#20090,"amount",#20000) +#20091=* +stmts(#20091,2,#20001,0,"1_000_000_000") +hasLocation(#20091,#20033) +stmt_containers(#20091,#20001) +#20092=* +exprs(#20092,3,#20091,0,"1_000_000_000") +hasLocation(#20092,#20033) +enclosing_stmt(#20092,#20091) +expr_containers(#20092,#20001) +literals("1000000000","1_000_000_000",#20092) +#20093=* +stmts(#20093,2,#20001,1,"101_475_938.38") +hasLocation(#20093,#20035) +stmt_containers(#20093,#20001) +#20094=* +exprs(#20094,3,#20093,0,"101_475_938.38") +hasLocation(#20094,#20035) +enclosing_stmt(#20094,#20093) +expr_containers(#20094,#20001) +literals("1.0147593838E8","101_475_938.38",#20094) +#20095=* +stmts(#20095,23,#20001,2,"let fee = 123_00;") +#20096=@"loc,{#10000},4,1,4,17" +locations_default(#20096,#10000,4,1,4,17) +hasLocation(#20095,#20096) +stmt_containers(#20095,#20001) +#20097=* +exprs(#20097,64,#20095,0,"fee = 123_00") +#20098=@"loc,{#10000},4,5,4,16" +locations_default(#20098,#10000,4,5,4,16) +hasLocation(#20097,#20098) +enclosing_stmt(#20097,#20095) +expr_containers(#20097,#20001) +#20099=* +exprs(#20099,78,#20097,0,"fee") +hasLocation(#20099,#20039) +enclosing_stmt(#20099,#20095) +expr_containers(#20099,#20001) +literals("fee","fee",#20099) +decl(#20099,#20089) +#20100=* +exprs(#20100,3,#20097,1,"123_00") +hasLocation(#20100,#20043) +enclosing_stmt(#20100,#20095) +expr_containers(#20100,#20001) +literals("12300","123_00",#20100) +#20101=* +stmts(#20101,23,#20001,3,"let fee = 12_300;") +#20102=@"loc,{#10000},5,1,5,17" +locations_default(#20102,#10000,5,1,5,17) +hasLocation(#20101,#20102) +stmt_containers(#20101,#20001) +#20103=* +exprs(#20103,64,#20101,0,"fee = 12_300") +#20104=@"loc,{#10000},5,5,5,16" +locations_default(#20104,#10000,5,5,5,16) +hasLocation(#20103,#20104) +enclosing_stmt(#20103,#20101) +expr_containers(#20103,#20001) +#20105=* +exprs(#20105,78,#20103,0,"fee") +hasLocation(#20105,#20049) +enclosing_stmt(#20105,#20101) +expr_containers(#20105,#20001) +literals("fee","fee",#20105) +decl(#20105,#20089) +#20106=* +exprs(#20106,3,#20103,1,"12_300") +hasLocation(#20106,#20053) +enclosing_stmt(#20106,#20101) +expr_containers(#20106,#20001) +literals("12300","12_300",#20106) +#20107=* +stmts(#20107,23,#20001,4,"let amo ... 345_00;") +#20108=@"loc,{#10000},6,1,6,22" +locations_default(#20108,#10000,6,1,6,22) +hasLocation(#20107,#20108) +stmt_containers(#20107,#20001) +#20109=* +exprs(#20109,64,#20107,0,"amount = 12345_00") +#20110=@"loc,{#10000},6,5,6,21" +locations_default(#20110,#10000,6,5,6,21) +hasLocation(#20109,#20110) +enclosing_stmt(#20109,#20107) +expr_containers(#20109,#20001) +#20111=* +exprs(#20111,78,#20109,0,"amount") +hasLocation(#20111,#20059) +enclosing_stmt(#20111,#20107) +expr_containers(#20111,#20001) +literals("amount","amount",#20111) +decl(#20111,#20090) +#20112=* +exprs(#20112,3,#20109,1,"12345_00") +hasLocation(#20112,#20063) +enclosing_stmt(#20112,#20107) +expr_containers(#20112,#20001) +literals("1234500","12345_00",#20112) +#20113=* +stmts(#20113,23,#20001,5,"let amo ... 3_4500;") +#20114=@"loc,{#10000},7,1,7,22" +locations_default(#20114,#10000,7,1,7,22) +hasLocation(#20113,#20114) +stmt_containers(#20113,#20001) +#20115=* +exprs(#20115,64,#20113,0,"amount = 123_4500") +#20116=@"loc,{#10000},7,5,7,21" +locations_default(#20116,#10000,7,5,7,21) +hasLocation(#20115,#20116) +enclosing_stmt(#20115,#20113) +expr_containers(#20115,#20001) +#20117=* +exprs(#20117,78,#20115,0,"amount") +hasLocation(#20117,#20069) +enclosing_stmt(#20117,#20113) +expr_containers(#20117,#20001) +literals("amount","amount",#20117) +decl(#20117,#20090) +#20118=* +exprs(#20118,3,#20115,1,"123_4500") +hasLocation(#20118,#20073) +enclosing_stmt(#20118,#20113) +expr_containers(#20118,#20001) +literals("1234500","123_4500",#20118) +#20119=* +stmts(#20119,23,#20001,6,"let amo ... 34_500;") +#20120=@"loc,{#10000},8,1,8,23" +locations_default(#20120,#10000,8,1,8,23) +hasLocation(#20119,#20120) +stmt_containers(#20119,#20001) +#20121=* +exprs(#20121,64,#20119,0,"amount = 1_234_500") +#20122=@"loc,{#10000},8,5,8,22" +locations_default(#20122,#10000,8,5,8,22) +hasLocation(#20121,#20122) +enclosing_stmt(#20121,#20119) +expr_containers(#20121,#20001) +#20123=* +exprs(#20123,78,#20121,0,"amount") +hasLocation(#20123,#20079) +enclosing_stmt(#20123,#20119) +expr_containers(#20123,#20001) +literals("amount","amount",#20123) +decl(#20123,#20090) +#20124=* +exprs(#20124,3,#20121,1,"1_234_500") +hasLocation(#20124,#20083) +enclosing_stmt(#20124,#20119) +expr_containers(#20124,#20001) +literals("1234500","1_234_500",#20124) +#20125=* +entry_cfg_node(#20125,#20001) +#20126=@"loc,{#10000},1,1,1,0" +locations_default(#20126,#10000,1,1,1,0) +hasLocation(#20125,#20126) +#20127=* +exit_cfg_node(#20127,#20001) +hasLocation(#20127,#20087) +successor(#20119,#20123) +successor(#20124,#20121) +successor(#20123,#20124) +successor(#20121,#20127) +successor(#20113,#20117) +successor(#20118,#20115) +successor(#20117,#20118) +successor(#20115,#20119) +successor(#20107,#20111) +successor(#20112,#20109) +successor(#20111,#20112) +successor(#20109,#20113) +successor(#20101,#20105) +successor(#20106,#20103) +successor(#20105,#20106) +successor(#20103,#20107) +successor(#20095,#20099) +successor(#20100,#20097) +successor(#20099,#20100) +successor(#20097,#20101) +successor(#20093,#20094) +successor(#20094,#20095) +successor(#20091,#20092) +successor(#20092,#20093) +successor(#20125,#20091) +numlines(#10000,8,7,7) +filetype(#10000,"javascript") diff --git a/javascript/extractor/tests/moduleTypes3/input/package.json b/javascript/extractor/tests/moduleTypes3/input/package.json new file mode 100644 index 00000000000..e69de29bb2d diff --git a/javascript/extractor/tests/moduleTypes3/input/tst.js b/javascript/extractor/tests/moduleTypes3/input/tst.js new file mode 100644 index 00000000000..e69de29bb2d diff --git a/javascript/extractor/tests/moduleTypes3/output/trap/package.json.trap b/javascript/extractor/tests/moduleTypes3/output/trap/package.json.trap new file mode 100644 index 00000000000..eed0d0523d5 --- /dev/null +++ b/javascript/extractor/tests/moduleTypes3/output/trap/package.json.trap @@ -0,0 +1,15 @@ +#10000=@"/package.json;sourcefile" +files(#10000,"/package.json","package","json",0) +#10001=@"/;folder" +folders(#10001,"/","") +containerparent(#10001,#10000) +#10002=@"loc,{#10000},0,0,0,0" +locations_default(#10002,#10000,0,0,0,0) +hasLocation(#10000,#10002) +#20000=* +json_errors(#20000,"Error: Unexpected token") +#20001=@"loc,{#10000},1,1,1,1" +locations_default(#20001,#10000,1,1,1,1) +hasLocation(#20000,#20001) +numlines(#10000,0,0,0) +filetype(#10000,"json") diff --git a/javascript/extractor/tests/moduleTypes3/output/trap/tst.js.trap b/javascript/extractor/tests/moduleTypes3/output/trap/tst.js.trap new file mode 100644 index 00000000000..cedeb436b92 --- /dev/null +++ b/javascript/extractor/tests/moduleTypes3/output/trap/tst.js.trap @@ -0,0 +1,28 @@ +#10000=@"/tst.js;sourcefile" +files(#10000,"/tst.js","tst","js",0) +#10001=@"/;folder" +folders(#10001,"/","") +containerparent(#10001,#10000) +#10002=@"loc,{#10000},0,0,0,0" +locations_default(#10002,#10000,0,0,0,0) +hasLocation(#10000,#10002) +#20000=@"global_scope" +scopes(#20000,0) +#20001=@"script;{#10000},1,1" +numlines(#20001,0,0,0) +#20002=* +tokeninfo(#20002,0,#20001,0,"") +#20003=@"loc,{#10000},1,1,1,0" +locations_default(#20003,#10000,1,1,1,0) +hasLocation(#20002,#20003) +toplevels(#20001,0) +hasLocation(#20001,#20003) +#20004=* +entry_cfg_node(#20004,#20001) +hasLocation(#20004,#20003) +#20005=* +exit_cfg_node(#20005,#20001) +hasLocation(#20005,#20003) +successor(#20004,#20005) +numlines(#10000,0,0,0) +filetype(#10000,"javascript") diff --git a/javascript/extractor/tests/ts/input/importNonStrings.ts b/javascript/extractor/tests/ts/input/importNonStrings.ts new file mode 100644 index 00000000000..6ab15963636 --- /dev/null +++ b/javascript/extractor/tests/ts/input/importNonStrings.ts @@ -0,0 +1,3 @@ +type X = import(3); +type Y = import(`Foo`); +type Z = import(Y); diff --git a/javascript/extractor/tests/ts/output/trap/importNonStrings.ts.trap b/javascript/extractor/tests/ts/output/trap/importNonStrings.ts.trap new file mode 100644 index 00000000000..fdcbc0d00a1 --- /dev/null +++ b/javascript/extractor/tests/ts/output/trap/importNonStrings.ts.trap @@ -0,0 +1,252 @@ +#10000=@"/importNonStrings.ts;sourcefile" +files(#10000,"/importNonStrings.ts","importNonStrings","ts",0) +#10001=@"/;folder" +folders(#10001,"/","") +containerparent(#10001,#10000) +#10002=@"loc,{#10000},0,0,0,0" +locations_default(#10002,#10000,0,0,0,0) +hasLocation(#10000,#10002) +#20000=@"global_scope" +scopes(#20000,0) +#20001=@"script;{#10000},1,1" +#20002=* +lines(#20002,#20001,"type X = import(3);"," +") +#20003=@"loc,{#10000},1,1,1,19" +locations_default(#20003,#10000,1,1,1,19) +hasLocation(#20002,#20003) +#20004=* +lines(#20004,#20001,"type Y = import(`Foo`);"," +") +#20005=@"loc,{#10000},2,1,2,23" +locations_default(#20005,#10000,2,1,2,23) +hasLocation(#20004,#20005) +#20006=* +lines(#20006,#20001,"type Z = import(Y);"," +") +#20007=@"loc,{#10000},3,1,3,19" +locations_default(#20007,#10000,3,1,3,19) +hasLocation(#20006,#20007) +numlines(#20001,3,3,0) +#20008=* +tokeninfo(#20008,7,#20001,0,"type") +#20009=@"loc,{#10000},1,1,1,4" +locations_default(#20009,#10000,1,1,1,4) +hasLocation(#20008,#20009) +#20010=* +tokeninfo(#20010,6,#20001,1,"X") +#20011=@"loc,{#10000},1,6,1,6" +locations_default(#20011,#10000,1,6,1,6) +hasLocation(#20010,#20011) +#20012=* +tokeninfo(#20012,8,#20001,2,"=") +#20013=@"loc,{#10000},1,8,1,8" +locations_default(#20013,#10000,1,8,1,8) +hasLocation(#20012,#20013) +#20014=* +tokeninfo(#20014,7,#20001,3,"import") +#20015=@"loc,{#10000},1,10,1,15" +locations_default(#20015,#10000,1,10,1,15) +hasLocation(#20014,#20015) +#20016=* +tokeninfo(#20016,8,#20001,4,"(") +#20017=@"loc,{#10000},1,16,1,16" +locations_default(#20017,#10000,1,16,1,16) +hasLocation(#20016,#20017) +#20018=* +tokeninfo(#20018,3,#20001,5,"3") +#20019=@"loc,{#10000},1,17,1,17" +locations_default(#20019,#10000,1,17,1,17) +hasLocation(#20018,#20019) +#20020=* +tokeninfo(#20020,8,#20001,6,")") +#20021=@"loc,{#10000},1,18,1,18" +locations_default(#20021,#10000,1,18,1,18) +hasLocation(#20020,#20021) +#20022=* +tokeninfo(#20022,8,#20001,7,";") +#20023=@"loc,{#10000},1,19,1,19" +locations_default(#20023,#10000,1,19,1,19) +hasLocation(#20022,#20023) +#20024=* +tokeninfo(#20024,7,#20001,8,"type") +#20025=@"loc,{#10000},2,1,2,4" +locations_default(#20025,#10000,2,1,2,4) +hasLocation(#20024,#20025) +#20026=* +tokeninfo(#20026,6,#20001,9,"Y") +#20027=@"loc,{#10000},2,6,2,6" +locations_default(#20027,#10000,2,6,2,6) +hasLocation(#20026,#20027) +#20028=* +tokeninfo(#20028,8,#20001,10,"=") +#20029=@"loc,{#10000},2,8,2,8" +locations_default(#20029,#10000,2,8,2,8) +hasLocation(#20028,#20029) +#20030=* +tokeninfo(#20030,7,#20001,11,"import") +#20031=@"loc,{#10000},2,10,2,15" +locations_default(#20031,#10000,2,10,2,15) +hasLocation(#20030,#20031) +#20032=* +tokeninfo(#20032,8,#20001,12,"(") +#20033=@"loc,{#10000},2,16,2,16" +locations_default(#20033,#10000,2,16,2,16) +hasLocation(#20032,#20033) +#20034=* +tokeninfo(#20034,4,#20001,13,"`Foo`") +#20035=@"loc,{#10000},2,17,2,21" +locations_default(#20035,#10000,2,17,2,21) +hasLocation(#20034,#20035) +#20036=* +tokeninfo(#20036,8,#20001,14,")") +#20037=@"loc,{#10000},2,22,2,22" +locations_default(#20037,#10000,2,22,2,22) +hasLocation(#20036,#20037) +#20038=* +tokeninfo(#20038,8,#20001,15,";") +#20039=@"loc,{#10000},2,23,2,23" +locations_default(#20039,#10000,2,23,2,23) +hasLocation(#20038,#20039) +#20040=* +tokeninfo(#20040,7,#20001,16,"type") +#20041=@"loc,{#10000},3,1,3,4" +locations_default(#20041,#10000,3,1,3,4) +hasLocation(#20040,#20041) +#20042=* +tokeninfo(#20042,6,#20001,17,"Z") +#20043=@"loc,{#10000},3,6,3,6" +locations_default(#20043,#10000,3,6,3,6) +hasLocation(#20042,#20043) +#20044=* +tokeninfo(#20044,8,#20001,18,"=") +#20045=@"loc,{#10000},3,8,3,8" +locations_default(#20045,#10000,3,8,3,8) +hasLocation(#20044,#20045) +#20046=* +tokeninfo(#20046,7,#20001,19,"import") +#20047=@"loc,{#10000},3,10,3,15" +locations_default(#20047,#10000,3,10,3,15) +hasLocation(#20046,#20047) +#20048=* +tokeninfo(#20048,8,#20001,20,"(") +#20049=@"loc,{#10000},3,16,3,16" +locations_default(#20049,#10000,3,16,3,16) +hasLocation(#20048,#20049) +#20050=* +tokeninfo(#20050,6,#20001,21,"Y") +#20051=@"loc,{#10000},3,17,3,17" +locations_default(#20051,#10000,3,17,3,17) +hasLocation(#20050,#20051) +#20052=* +tokeninfo(#20052,8,#20001,22,")") +#20053=@"loc,{#10000},3,18,3,18" +locations_default(#20053,#10000,3,18,3,18) +hasLocation(#20052,#20053) +#20054=* +tokeninfo(#20054,8,#20001,23,";") +#20055=@"loc,{#10000},3,19,3,19" +locations_default(#20055,#10000,3,19,3,19) +hasLocation(#20054,#20055) +#20056=* +tokeninfo(#20056,0,#20001,24,"") +#20057=@"loc,{#10000},4,1,4,0" +locations_default(#20057,#10000,4,1,4,0) +hasLocation(#20056,#20057) +toplevels(#20001,0) +#20058=@"loc,{#10000},1,1,4,0" +locations_default(#20058,#10000,1,1,4,0) +hasLocation(#20001,#20058) +#20059=@"local_type_name;{X};{#20000}" +local_type_names(#20059,"X",#20000) +#20060=@"local_type_name;{Y};{#20000}" +local_type_names(#20060,"Y",#20000) +#20061=@"local_type_name;{Z};{#20000}" +local_type_names(#20061,"Z",#20000) +#20062=* +stmts(#20062,35,#20001,0,"type X = import(3);") +hasLocation(#20062,#20003) +stmt_containers(#20062,#20001) +#20063=* +typeexprs(#20063,1,#20062,0,"X") +hasLocation(#20063,#20011) +enclosing_stmt(#20063,#20062) +expr_containers(#20063,#20001) +literals("X","X",#20063) +typedecl(#20063,#20059) +#20064=* +typeexprs(#20064,30,#20062,1,"import(3)") +#20065=@"loc,{#10000},1,10,1,18" +locations_default(#20065,#10000,1,10,1,18) +hasLocation(#20064,#20065) +enclosing_stmt(#20064,#20062) +expr_containers(#20064,#20001) +#20066=* +typeexprs(#20066,4,#20064,0,"3") +hasLocation(#20066,#20019) +enclosing_stmt(#20066,#20062) +expr_containers(#20066,#20001) +literals("3","3",#20066) +#20067=* +stmts(#20067,35,#20001,1,"type Y ... `Foo`);") +hasLocation(#20067,#20005) +stmt_containers(#20067,#20001) +#20068=* +typeexprs(#20068,1,#20067,0,"Y") +hasLocation(#20068,#20027) +enclosing_stmt(#20068,#20067) +expr_containers(#20068,#20001) +literals("Y","Y",#20068) +typedecl(#20068,#20060) +#20069=* +typeexprs(#20069,30,#20067,1,"import(`Foo`)") +#20070=@"loc,{#10000},2,10,2,22" +locations_default(#20070,#10000,2,10,2,22) +hasLocation(#20069,#20070) +enclosing_stmt(#20069,#20067) +expr_containers(#20069,#20001) +#20071=* +typeexprs(#20071,3,#20069,0,"`Foo`") +hasLocation(#20071,#20035) +enclosing_stmt(#20071,#20067) +expr_containers(#20071,#20001) +literals("Foo","`Foo`",#20071) +#20072=* +stmts(#20072,35,#20001,2,"type Z = import(Y);") +hasLocation(#20072,#20007) +stmt_containers(#20072,#20001) +#20073=* +typeexprs(#20073,1,#20072,0,"Z") +hasLocation(#20073,#20043) +enclosing_stmt(#20073,#20072) +expr_containers(#20073,#20001) +literals("Z","Z",#20073) +typedecl(#20073,#20061) +#20074=* +typeexprs(#20074,30,#20072,1,"import(Y)") +#20075=@"loc,{#10000},3,10,3,18" +locations_default(#20075,#10000,3,10,3,18) +hasLocation(#20074,#20075) +enclosing_stmt(#20074,#20072) +expr_containers(#20074,#20001) +#20076=* +typeexprs(#20076,0,#20074,0,"Y") +hasLocation(#20076,#20051) +enclosing_stmt(#20076,#20072) +expr_containers(#20076,#20001) +literals("Y","Y",#20076) +typebind(#20076,#20060) +#20077=* +entry_cfg_node(#20077,#20001) +#20078=@"loc,{#10000},1,1,1,0" +locations_default(#20078,#10000,1,1,1,0) +hasLocation(#20077,#20078) +#20079=* +exit_cfg_node(#20079,#20001) +hasLocation(#20079,#20057) +successor(#20072,#20079) +successor(#20067,#20072) +successor(#20062,#20067) +successor(#20077,#20062) +numlines(#10000,3,3,0) +filetype(#10000,"typescript") diff --git a/javascript/ql/src/Expressions/StringInsteadOfRegex.ql b/javascript/ql/src/Expressions/StringInsteadOfRegex.ql index f81b8b1a5ae..f1efb37e330 100644 --- a/javascript/ql/src/Expressions/StringInsteadOfRegex.ql +++ b/javascript/ql/src/Expressions/StringInsteadOfRegex.ql @@ -31,7 +31,7 @@ predicate isStringSplitOrReplace(MethodCallExpr mce) { mce.getMethodName() = name and mce.getNumArgument() = arity | - name = "replace" and arity = 2 + name = ["replace", "replaceAll"] and arity = 2 or name = "split" and (arity = 1 or arity = 2) diff --git a/javascript/ql/src/RegExp/IdentityReplacement.ql b/javascript/ql/src/RegExp/IdentityReplacement.ql index f8c62a1ab3a..2a6e354cc9c 100644 --- a/javascript/ql/src/RegExp/IdentityReplacement.ql +++ b/javascript/ql/src/RegExp/IdentityReplacement.ql @@ -70,7 +70,7 @@ predicate regExpMatchesString(RegExpTerm t, string s) { from MethodCallExpr repl, string s, string friendly where - repl.getMethodName() = "replace" and + repl.getMethodName() = ["replace", "replaceAll"] and matchesString(repl.getArgument(0), s) and repl.getArgument(1).getStringValue() = s and (if s = "" then friendly = "the empty string" else friendly = "'" + s + "'") diff --git a/javascript/ql/src/Security/CWE-020/IncompleteUrlSchemeCheck.ql b/javascript/ql/src/Security/CWE-020/IncompleteUrlSchemeCheck.ql index 617692f3adb..0675b189cd1 100644 --- a/javascript/ql/src/Security/CWE-020/IncompleteUrlSchemeCheck.ql +++ b/javascript/ql/src/Security/CWE-020/IncompleteUrlSchemeCheck.ql @@ -75,7 +75,7 @@ DataFlow::Node schemeCheck(DataFlow::Node nd, DangerousScheme scheme) { exists(DataFlow::MethodCallNode stringop | stringop.getMethodName().matches("trim%") or stringop.getMethodName().matches("to%Case") or - stringop.getMethodName() = "replace" + stringop.getMethodName() = ["replace", "replaceAll"] | result = schemeCheck(stringop, scheme) and nd = stringop.getReceiver() diff --git a/javascript/ql/src/Security/CWE-022/TaintedPath.qhelp b/javascript/ql/src/Security/CWE-022/TaintedPath.qhelp index b9cb415a018..8dab2fcc4fc 100644 --- a/javascript/ql/src/Security/CWE-022/TaintedPath.qhelp +++ b/javascript/ql/src/Security/CWE-022/TaintedPath.qhelp @@ -50,7 +50,7 @@ system's passwords. -
  • OWASP: Path Traversal.
    • +
  • OWASP: Path Traversal.
  • npm: sanitize-filename package.
    •  diff --git a/javascript/ql/src/Security/CWE-022/ZipSlip.qhelp b/javascript/ql/src/Security/CWE-022/ZipSlip.qhelp index 7a361947216..d9dc0fb67a8 100644 --- a/javascript/ql/src/Security/CWE-022/ZipSlip.qhelp +++ b/javascript/ql/src/Security/CWE-022/ZipSlip.qhelp @@ -60,7 +60,7 @@ Snyk:
  • OWASP: -Path Traversal. +Path Traversal.
    • diff --git a/javascript/ql/src/Security/CWE-116/IncompleteSanitization.ql b/javascript/ql/src/Security/CWE-116/IncompleteSanitization.ql index a1b466cab9f..9494dcde0e5 100644 --- a/javascript/ql/src/Security/CWE-116/IncompleteSanitization.ql +++ b/javascript/ql/src/Security/CWE-116/IncompleteSanitization.ql @@ -79,6 +79,7 @@ predicate allBackslashesEscaped(DataFlow::Node nd) { // flow through string methods exists(DataFlow::MethodCallNode mc, string m | m = "replace" or + m = "replaceAll" or m = "slice" or m = "substr" or m = "substring" or diff --git a/javascript/ql/src/Security/CWE-209/StackTraceExposure.qhelp b/javascript/ql/src/Security/CWE-209/StackTraceExposure.qhelp index ea37581a799..20a9c65cbc7 100644 --- a/javascript/ql/src/Security/CWE-209/StackTraceExposure.qhelp +++ b/javascript/ql/src/Security/CWE-209/StackTraceExposure.qhelp @@ -52,6 +52,6 @@ will not see the information: -
  • OWASP: Information Leak.
    • +
  • OWASP: Improper Error Handling.
    •  diff --git a/javascript/ql/src/Security/CWE-352/MissingCsrfMiddleware.ql b/javascript/ql/src/Security/CWE-352/MissingCsrfMiddleware.ql index ca09a09f3a8..d63786ba08d 100644 --- a/javascript/ql/src/Security/CWE-352/MissingCsrfMiddleware.ql +++ b/javascript/ql/src/Security/CWE-352/MissingCsrfMiddleware.ql @@ -18,8 +18,8 @@ string cookieProperty() { result = "session" or result = "cookies" or result = " /** Gets a data flow node that flows to the base of an access to `cookies`, `session`, or `user`. */ private DataFlow::SourceNode nodeLeadingToCookieAccess(DataFlow::TypeBackTracker t) { t.start() and - exists(DataFlow::PropRead value | - value = result.getAPropertyRead(cookieProperty()).getAPropertyRead() and + exists(DataFlow::PropRef value | + value = result.getAPropertyRead(cookieProperty()).getAPropertyReference() and // Ignore accesses to values that are part of a CSRF or captcha check not value.getPropertyName().regexpMatch("(?i).*(csrf|xsrf|captcha).*") and // Ignore calls like `req.session.save()` diff --git a/javascript/ql/src/Security/CWE-643/XpathInjection.qhelp b/javascript/ql/src/Security/CWE-643/XpathInjection.qhelp index 346b213d180..3378b9b4078 100644 --- a/javascript/ql/src/Security/CWE-643/XpathInjection.qhelp +++ b/javascript/ql/src/Security/CWE-643/XpathInjection.qhelp @@ -33,7 +33,7 @@ by xpath: -
  • OWASP: Testing for XPath Injection.
    • +
  • OWASP: Testing for XPath Injection.
  • OWASP: XPath Injection.
  • npm: xpath.
    •  diff --git a/javascript/ql/src/meta/ApiGraphs/ApiGraphEdges.ql b/javascript/ql/src/meta/ApiGraphs/ApiGraphEdges.ql new file mode 100644 index 00000000000..16718a9d122 --- /dev/null +++ b/javascript/ql/src/meta/ApiGraphs/ApiGraphEdges.ql @@ -0,0 +1,15 @@ +/** + * @name API-graph edges + * @description The number of edges (other than points-to edges) in the API graph. + * @kind metric + * @metricType project + * @metricAggregate sum + * @tags meta + * @id js/meta/api-graph-edges + */ + +import javascript +import meta.MetaMetrics + +select projectRoot(), + count(API::Node pred, string lbl, API::Node succ | succ = pred.getASuccessor(lbl)) diff --git a/javascript/ql/src/meta/ApiGraphs/ApiGraphNodes.ql b/javascript/ql/src/meta/ApiGraphs/ApiGraphNodes.ql new file mode 100644 index 00000000000..661c1414e1d --- /dev/null +++ b/javascript/ql/src/meta/ApiGraphs/ApiGraphNodes.ql @@ -0,0 +1,14 @@ +/** + * @name API-graph nodes + * @description The number of nodes in the API graph. + * @kind metric + * @metricType project + * @metricAggregate sum + * @tags meta + * @id js/meta/api-graph-nodes + */ + +import javascript +import meta.MetaMetrics + +select projectRoot(), count(API::Node nd) diff --git a/javascript/ql/src/meta/ApiGraphs/ApiGraphPointsToEdges.ql b/javascript/ql/src/meta/ApiGraphs/ApiGraphPointsToEdges.ql new file mode 100644 index 00000000000..e410af5a4f1 --- /dev/null +++ b/javascript/ql/src/meta/ApiGraphs/ApiGraphPointsToEdges.ql @@ -0,0 +1,14 @@ +/** + * @name API-graph points-to edges + * @description The number of points-to edges in the API graph. + * @kind metric + * @metricType project + * @metricAggregate sum + * @tags meta + * @id js/meta/api-graph-points-to-edges + */ + +import javascript +import meta.MetaMetrics + +select projectRoot(), count(API::Node pred, API::Node succ | pred.refersTo(succ)) diff --git a/javascript/ql/src/meta/ApiGraphs/ApiGraphRhsNodes.ql b/javascript/ql/src/meta/ApiGraphs/ApiGraphRhsNodes.ql new file mode 100644 index 00000000000..fcb98dc57af --- /dev/null +++ b/javascript/ql/src/meta/ApiGraphs/ApiGraphRhsNodes.ql @@ -0,0 +1,15 @@ +/** + * @name API-graph right-hand-side nodes + * @description The number of data-flow nodes corresponding to a right-hand side of + * a definition of an API-graph node. + * @kind metric + * @metricType project + * @metricAggregate sum + * @tags meta + * @id js/meta/api-graph-rhs-nodes + */ + +import javascript +import meta.MetaMetrics + +select projectRoot(), count(any(API::Node nd).getARhs()) diff --git a/javascript/ql/src/meta/ApiGraphs/ApiGraphUseNodes.ql b/javascript/ql/src/meta/ApiGraphs/ApiGraphUseNodes.ql new file mode 100644 index 00000000000..446df101364 --- /dev/null +++ b/javascript/ql/src/meta/ApiGraphs/ApiGraphUseNodes.ql @@ -0,0 +1,14 @@ +/** + * @name API-graph use nodes + * @description The number of data-flow nodes corresponding to a use of an API-graph node. + * @kind metric + * @metricType project + * @metricAggregate sum + * @tags meta + * @id js/meta/api-graph-use-nodes + */ + +import javascript +import meta.MetaMetrics + +select projectRoot(), count(any(API::Node nd).getAUse()) diff --git a/javascript/ql/src/semmle/javascript/ApiGraphs.qll b/javascript/ql/src/semmle/javascript/ApiGraphs.qll index 4c92212df57..0a71f69c174 100644 --- a/javascript/ql/src/semmle/javascript/ApiGraphs.qll +++ b/javascript/ql/src/semmle/javascript/ApiGraphs.qll @@ -24,7 +24,10 @@ module API { * Gets a data-flow node corresponding to a use of the API component represented by this node. * * For example, `require('fs').readFileSync` is a use of the function `readFileSync` from the - * `fs` module, and `require('fs').readFileSync(file)` is a use of the result of that function. + * `fs` module, and `require('fs').readFileSync(file)` is a use of the return of that function. + * + * This includes indirect uses found via data flow, meaning that in + * `f(obj.foo); function f(x) {};` both `obj.foo` and `x` are uses of the `foo` member from `obj`. * * As another example, in the assignment `exports.plusOne = (x) => x+1` the two references to * `x` are uses of the first parameter of `plusOne`. @@ -35,6 +38,34 @@ module API { ) } + /** + * Gets an immediate use of the API component represented by this node. + * + * For example, `require('fs').readFileSync` is a an immediate use of the `readFileSync` member + * from the `fs` module. + * + * Unlike `getAUse()`, this predicate only gets the immediate references, not the indirect uses + * found via data flow. This means that in `const x = fs.readFile` only `fs.readFile` is a reference + * to the `readFile` member of `fs`, neither `x` nor any node that `x` flows to is a reference to + * this API component. + */ + DataFlow::SourceNode getAnImmediateUse() { Impl::use(this, result) } + + /** + * Gets a call to the function represented by this API component. + */ + DataFlow::CallNode getACall() { result = getReturn().getAnImmediateUse() } + + /** + * Gets a `new` call to the function represented by this API component. + */ + DataFlow::NewNode getAnInstantiation() { result = getInstance().getAnImmediateUse() } + + /** + * Gets an invocation (with our without `new`) to the function represented by this API component. + */ + DataFlow::InvokeNode getAnInvocation() { result = getACall() or result = getAnInstantiation() } + /** * Gets a data-flow node corresponding to the right-hand side of a definition of the API * component represented by this node. @@ -223,6 +254,9 @@ module API { ) and length in [1 .. Impl::distanceFromRoot(this)] } + + /** Gets the shortest distance from the root to this node in the API graph. */ + int getDepth() { result = Impl::distanceFromRoot(this) } } /** The root node of an API graph. */ @@ -409,16 +443,9 @@ module API { rhs = f.getAReturn() ) or - exists(DataFlow::SourceNode src, DataFlow::InvokeNode invk | - use(base, src) and invk = trackUseNode(src).getAnInvocation() - | - exists(int i | - lbl = Label::parameter(i) and - rhs = invk.getArgument(i) - ) - or - lbl = Label::receiver() and - rhs = invk.(DataFlow::CallNode).getReceiver() + exists(int i | + lbl = Label::parameter(i) and + argumentPassing(base, i, rhs) ) or exists(DataFlow::SourceNode src, DataFlow::PropWrite pw | @@ -429,6 +456,30 @@ module API { ) } + /** + * Holds if `arg` is passed as the `i`th argument to a use of `base`, either by means of a + * full invocation, or in a partial function application. + * + * The receiver is considered to be argument -1. + */ + private predicate argumentPassing(TApiNode base, int i, DataFlow::Node arg) { + exists(DataFlow::SourceNode use, DataFlow::SourceNode pred | + use(base, use) and pred = trackUseNode(use) + | + arg = pred.getAnInvocation().getArgument(i) + or + arg = pred.getACall().getReceiver() and + i = -1 + or + exists(DataFlow::PartialInvokeNode pin, DataFlow::Node callback | pred.flowsTo(callback) | + pin.isPartialArgument(callback, arg, i) + or + arg = pin.getBoundReceiver(callback) and + i = -1 + ) + ) + } + /** * Holds if `rhs` is the right-hand side of a definition of node `nd`. */ @@ -716,10 +767,14 @@ private module Label { bindingset[s] string parameterByStringIndex(string s) { result = "parameter " + s and - s.toInt() >= 0 + s.toInt() >= -1 } - /** Gets the `parameter` edge label for the `i`th parameter. */ + /** + * Gets the `parameter` edge label for the `i`th parameter. + * + * The receiver is considered to be parameter -1. + */ bindingset[i] string parameter(int i) { result = parameterByStringIndex(i.toString()) } diff --git a/javascript/ql/src/semmle/javascript/Promises.qll b/javascript/ql/src/semmle/javascript/Promises.qll index a8231d75dce..4f730b36f0a 100644 --- a/javascript/ql/src/semmle/javascript/Promises.qll +++ b/javascript/ql/src/semmle/javascript/Promises.qll @@ -117,11 +117,11 @@ class ResolvedES2015PromiseDefinition extends ResolvedPromiseDefinition { } /** - * An aggregated promise produced either by `Promise.all` or `Promise.race`. + * An aggregated promise produced either by `Promise.all`, `Promise.race`, or `Promise.any`. */ class AggregateES2015PromiseDefinition extends PromiseCreationCall { AggregateES2015PromiseDefinition() { - exists(string m | m = "all" or m = "race" | + exists(string m | m = "all" or m = "race" or m = "any" | this = DataFlow::globalVarRef("Promise").getAMemberCall(m) ) } diff --git a/javascript/ql/src/semmle/javascript/StandardLibrary.qll b/javascript/ql/src/semmle/javascript/StandardLibrary.qll index b28d1f27439..6a57bda6461 100644 --- a/javascript/ql/src/semmle/javascript/StandardLibrary.qll +++ b/javascript/ql/src/semmle/javascript/StandardLibrary.qll @@ -102,7 +102,7 @@ private class IteratorExceptionStep extends DataFlow::MethodCallNode, DataFlow:: */ class StringReplaceCall extends DataFlow::MethodCallNode { StringReplaceCall() { - getMethodName() = "replace" and + getMethodName() = ["replace", "replaceAll"] and (getNumArgument() = 2 or getReceiver().mayHaveStringValue(_)) } @@ -128,9 +128,9 @@ class StringReplaceCall extends DataFlow::MethodCallNode { /** * Holds if this is a global replacement, that is, the first argument is a regular expression - * with the `g` flag. + * with the `g` flag, or this is a call to `.replaceAll()`. */ - predicate isGlobal() { getRegExp().isGlobal() } + predicate isGlobal() { getRegExp().isGlobal() or getMethodName() = "replaceAll" } /** * Holds if this call to `replace` replaces `old` with `new`. diff --git a/javascript/ql/src/semmle/javascript/dataflow/TaintTracking.qll b/javascript/ql/src/semmle/javascript/dataflow/TaintTracking.qll index f0763c0893f..6698800918c 100644 --- a/javascript/ql/src/semmle/javascript/dataflow/TaintTracking.qll +++ b/javascript/ql/src/semmle/javascript/dataflow/TaintTracking.qll @@ -427,6 +427,7 @@ module TaintTracking { name = "padStart" or name = "repeat" or name = "replace" or + name = "replaceAll" or name = "slice" or name = "small" or name = "split" or @@ -452,7 +453,7 @@ module TaintTracking { exists(int i | pred.asExpr() = succ.getAstNode().(MethodCallExpr).getArgument(i) | name = "concat" or - name = "replace" and i = 1 + name = ["replace", "replaceAll"] and i = 1 ) ) or @@ -707,7 +708,8 @@ module TaintTracking { */ private ControlFlowNode getACaptureSetter(DataFlow::Node input) { exists(DataFlow::MethodCallNode call | result = call.asExpr() | - call.getMethodName() = ["search", "replace", "match"] and input = call.getReceiver() + call.getMethodName() = ["search", "replace", "replaceAll", "match"] and + input = call.getReceiver() or call.getMethodName() = ["test", "exec"] and input = call.getArgument(0) ) diff --git a/javascript/ql/src/semmle/javascript/frameworks/ClientRequests.qll b/javascript/ql/src/semmle/javascript/frameworks/ClientRequests.qll index 1c0ac8d6a16..830d0faeb8a 100644 --- a/javascript/ql/src/semmle/javascript/frameworks/ClientRequests.qll +++ b/javascript/ql/src/semmle/javascript/frameworks/ClientRequests.qll @@ -327,6 +327,85 @@ module ClientRequest { } } + /** + * Classes for modelling the url request library `needle`. + */ + private module Needle { + /** + * A model of a URL request made using `require("needle")(...)`. + */ + class PromisedNeedleRequest extends ClientRequest::Range { + DataFlow::Node url; + + PromisedNeedleRequest() { this = DataFlow::moduleImport("needle").getACall() } + + override DataFlow::Node getUrl() { result = getArgument(1) } + + override DataFlow::Node getHost() { none() } + + override DataFlow::Node getADataNode() { + result = getOptionArgument([2, 3], "headers") + or + result = getArgument(2) + } + + override DataFlow::Node getAResponseDataNode(string responseType, boolean promise) { + responseType = "fetch.response" and + promise = true and + result = this + } + } + + /** + * A model of a URL request made using `require("needle")[method](...)`. + * E.g. `needle.get("http://example.org", (err, resp, body) => {})`. + * + * As opposed to the calls modeled in `PromisedNeedleRequest` these calls do not return promises. + * Instead they take an optional callback as their last argument. + */ + class NeedleMethodRequest extends ClientRequest::Range { + boolean hasData; + + NeedleMethodRequest() { + exists(string method | + method = ["get", "head"] and hasData = false + or + method = ["post", "put", "patch", "delete"] and hasData = true + or + method = "request" and hasData = [true, false] + | + this = DataFlow::moduleMember("needle", method).getACall() + ) + } + + override DataFlow::Node getUrl() { result = getArgument(0) } + + override DataFlow::Node getHost() { none() } + + override DataFlow::Node getADataNode() { + hasData = true and + ( + result = getArgument(1) + or + result = getOptionArgument(2, "headers") + ) + or + hasData = false and + result = getOptionArgument(1, "headers") + } + + override DataFlow::Node getAResponseDataNode(string responseType, boolean promise) { + promise = false and + result = this.getABoundCallbackParameter(this.getNumArgument() - 1, 1) and + responseType = "fetch.response" + or + promise = false and + result = this.getABoundCallbackParameter(this.getNumArgument() - 1, 2) and + responseType = "json" + } + } + } + /** * A model of a URL request made using the `got` library. */ diff --git a/javascript/ql/src/semmle/javascript/frameworks/Express.qll b/javascript/ql/src/semmle/javascript/frameworks/Express.qll index ce47919793b..af6675d188e 100644 --- a/javascript/ql/src/semmle/javascript/frameworks/Express.qll +++ b/javascript/ql/src/semmle/javascript/frameworks/Express.qll @@ -195,7 +195,7 @@ module Express { PassportRouteHandler() { this = any(PassportRouteSetup setup).getARouteHandler() } - override SimpleParameter getRouteHandlerParameter(string kind) { + override Parameter getRouteHandlerParameter(string kind) { kind = "request" and result = astNode.getParameter(0) } @@ -329,17 +329,17 @@ module Express { * * `kind` is one of: "error", "request", "response", "next", or "parameter". */ - abstract SimpleParameter getRouteHandlerParameter(string kind); + abstract Parameter getRouteHandlerParameter(string kind); /** * Gets the parameter of the route handler that contains the request object. */ - SimpleParameter getRequestParameter() { result = getRouteHandlerParameter("request") } + Parameter getRequestParameter() { result = getRouteHandlerParameter("request") } /** * Gets the parameter of the route handler that contains the response object. */ - SimpleParameter getResponseParameter() { result = getRouteHandlerParameter("response") } + Parameter getResponseParameter() { result = getRouteHandlerParameter("response") } /** * Gets a request body access of this handler. @@ -357,7 +357,7 @@ module Express { StandardRouteHandler() { this = routeSetup.getARouteHandler() } - override SimpleParameter getRouteHandlerParameter(string kind) { + override Parameter getRouteHandlerParameter(string kind) { if routeSetup.isParameterHandler() then result = getRouteParameterHandlerParameter(astNode, kind) else result = getRouteHandlerParameter(astNode, kind) @@ -505,6 +505,10 @@ module Express { // `req.cookies` kind = "cookie" and this = request.getAPropertyRead("cookies") + or + // `req.files`, treated the same as `req.body`. + kind = "body" and + this = request.getAPropertyRead("files") ) or kind = "body" and @@ -886,7 +890,7 @@ module Express { TrackedRouteHandlerCandidateWithSetup() { this = routeSetup.getARouteHandler() } - override SimpleParameter getRouteHandlerParameter(string kind) { + override Parameter getRouteHandlerParameter(string kind) { if routeSetup.isParameterHandler() then result = getRouteParameterHandlerParameter(astNode, kind) else result = getRouteHandlerParameter(astNode, kind) diff --git a/javascript/ql/src/semmle/javascript/frameworks/HTTP.qll b/javascript/ql/src/semmle/javascript/frameworks/HTTP.qll index e552bbcb85c..044ad8b0cd7 100644 --- a/javascript/ql/src/semmle/javascript/frameworks/HTTP.qll +++ b/javascript/ql/src/semmle/javascript/frameworks/HTTP.qll @@ -237,17 +237,38 @@ module HTTP { * Holds if `call` decorates the function `pred`. * This means that `call` returns a function that forwards its arguments to `pred`. * Only holds when the decorator looks like it is decorating a route-handler. + * + * Below is a code example relating `call`, `decoratee`, `outer`, `inner`. + * ``` + * function outer(method) { + * return function inner(req, res) { + * return method.call(this, req, res); + * }; + * } + * var route = outer(function decoratee(req, res) { // <- call + * res.end("foo"); + * }); + * ``` */ private predicate isDecoratedCall(DataFlow::CallNode call, DataFlow::FunctionNode decoratee) { // indirect route-handler `result` is given to function `outer`, which returns function `inner` which calls the function `pred`. - exists(int i, Function outer, Function inner | + exists(int i, DataFlow::FunctionNode outer, HTTP::RouteHandlerCandidate inner | + inner = outer.getAReturn().getALocalSource() and decoratee = call.getArgument(i).getALocalSource() and - outer = call.getACallee() and - inner = outer.getAReturnedExpr() and - isAForwardingRouteHandlerCall(DataFlow::parameterNode(outer.getParameter(i)), inner.flow()) + outer.getFunction() = call.getACallee() and + hasForwardingHandlerParameter(i, outer, inner) ) } + /** + * Holds if the `i`th parameter of `outer` has a call that `inner` forwards its parameters to. + */ + private predicate hasForwardingHandlerParameter( + int i, DataFlow::FunctionNode outer, HTTP::RouteHandlerCandidate inner + ) { + isAForwardingRouteHandlerCall(outer.getParameter(i), inner) + } + /** * Holds if `f` looks like a route-handler and a call to `callee` inside `f` forwards all of the parameters from `f` to that call. */ @@ -449,6 +470,23 @@ module HTTP { */ abstract Expr getServer(); } + + /** + * A parameter containing data received by a NodeJS HTTP server. + * E.g. `chunk` in: `http.createServer().on('request', (req, res) => req.on("data", (chunk) => ...))`. + */ + private class ServerRequestDataEvent extends RemoteFlowSource, DataFlow::ParameterNode { + RequestSource req; + + ServerRequestDataEvent() { + exists(DataFlow::MethodCallNode mcn | mcn = req.ref().getAMethodCall(EventEmitter::on()) | + mcn.getArgument(0).mayHaveStringValue("data") and + this = mcn.getABoundCallbackParameter(1, 0) + ) + } + + override string getSourceType() { result = "NodeJS HTTP server data event" } + } } /** diff --git a/javascript/ql/src/semmle/javascript/frameworks/NoSQL.qll b/javascript/ql/src/semmle/javascript/frameworks/NoSQL.qll index 13385057b5d..2854cdaa8f4 100644 --- a/javascript/ql/src/semmle/javascript/frameworks/NoSQL.qll +++ b/javascript/ql/src/semmle/javascript/frameworks/NoSQL.qll @@ -12,11 +12,23 @@ module NoSQL { } } +/** + * Gets a reference to an object where the "$where" property has been assigned to`rhs`. + */ +DataFlow::SourceNode getADollarWherePropertyValueSource(DataFlow::TypeTracker t, DataFlow::Node rhs) { + t.start() and + rhs = result.getAPropertyWrite("$where").getRhs() + or + exists(DataFlow::TypeTracker t2 | + result = getADollarWherePropertyValueSource(t2, rhs).track(t2, t) + ) +} + /** * Gets the value of a `$where` property of an object that flows to `n`. */ private DataFlow::Node getADollarWherePropertyValue(DataFlow::Node n) { - result = n.getALocalSource().getAPropertyWrite("$where").getRhs() + getADollarWherePropertyValueSource(DataFlow::TypeTracker::end(), result).flowsTo(n) } /** diff --git a/javascript/ql/src/semmle/javascript/frameworks/SQL.qll b/javascript/ql/src/semmle/javascript/frameworks/SQL.qll index 19eaf168c08..c94750e5b3d 100644 --- a/javascript/ql/src/semmle/javascript/frameworks/SQL.qll +++ b/javascript/ql/src/semmle/javascript/frameworks/SQL.qll @@ -31,24 +31,27 @@ private module MySql { /** Gets the package name `mysql` or `mysql2`. */ API::Node mysql() { result = API::moduleImport(["mysql", "mysql2"]) } - /** Gets a call to `mysql.createConnection`. */ - API::Node createConnection() { result = mysql().getMember("createConnection").getReturn() } + /** Gets a reference to `mysql.createConnection`. */ + API::Node createConnection() { result = mysql().getMember("createConnection") } - /** Gets a call to `mysql.createPool`. */ - API::Node createPool() { result = mysql().getMember("createPool").getReturn() } + /** Gets a reference to `mysql.createPool`. */ + API::Node createPool() { result = mysql().getMember("createPool") } + + /** Gets a node that contains a MySQL pool created using `mysql.createPool()`. */ + API::Node pool() { result = createPool().getReturn() } /** Gets a data flow node that contains a freshly created MySQL connection instance. */ API::Node connection() { - result = createConnection() + result = createConnection().getReturn() or - result = createPool().getMember("getConnection").getParameter(0).getParameter(1) + result = pool().getMember("getConnection").getParameter(0).getParameter(1) } /** A call to the MySql `query` method. */ private class QueryCall extends DatabaseAccess, DataFlow::MethodCallNode { QueryCall() { - exists(API::Node recv | recv = createPool() or recv = connection() | - this = recv.getMember("query").getReturn().getAUse() + exists(API::Node recv | recv = pool() or recv = connection() | + this = recv.getMember("query").getACall() ) } @@ -63,12 +66,7 @@ private module MySql { /** A call to the `escape` or `escapeId` method that performs SQL sanitization. */ class EscapingSanitizer extends SQL::SqlSanitizer, MethodCallExpr { EscapingSanitizer() { - this = - [mysql(), createPool(), connection()] - .getMember(["escape", "escapeId"]) - .getReturn() - .getAUse() - .asExpr() and + this = [mysql(), pool(), connection()].getMember(["escape", "escapeId"]).getACall().asExpr() and input = this.getArgument(0) and output = this } @@ -79,9 +77,9 @@ private module MySql { string kind; Credentials() { - exists(API::Node call, string prop | - call in [createConnection(), createPool()] and - call.getAUse().asExpr().(CallExpr).hasOptionArgument(0, prop, this) and + exists(API::Node callee, string prop | + callee in [createConnection(), createPool()] and + this = callee.getParameter(0).getMember(prop).getARhs().asExpr() and ( prop = "user" and kind = "user name" or @@ -98,29 +96,32 @@ private module MySql { * Provides classes modelling the `pg` package. */ private module Postgres { - /** Gets an expression of the form `new require('pg').Client()`. */ - API::Node newClient() { result = API::moduleImport("pg").getMember("Client").getInstance() } + /** Gets a reference to the `Client` constructor in the `pg` package, for example `require('pg').Client`. */ + API::Node newClient() { result = API::moduleImport("pg").getMember("Client") } - /** Gets a data flow node that holds a freshly created Postgres client instance. */ + /** Gets a freshly created Postgres client instance. */ API::Node client() { - result = newClient() + result = newClient().getInstance() or // pool.connect(function(err, client) { ... }) - result = newPool().getMember("connect").getParameter(0).getParameter(1) + result = pool().getMember("connect").getParameter(0).getParameter(1) + } + + /** Gets a constructor that when invoked constructs a new connection pool. */ + API::Node newPool() { + // new require('pg').Pool() + result = API::moduleImport("pg").getMember("Pool") + or + // new require('pg-pool') + result = API::moduleImport("pg-pool") } /** Gets an expression that constructs a new connection pool. */ - API::Node newPool() { - // new require('pg').Pool() - result = API::moduleImport("pg").getMember("Pool").getInstance() - or - // new require('pg-pool') - result = API::moduleImport("pg-pool").getInstance() - } + API::Node pool() { result = newPool().getInstance() } /** A call to the Postgres `query` method. */ private class QueryCall extends DatabaseAccess, DataFlow::MethodCallNode { - QueryCall() { this = [client(), newPool()].getMember("query").getReturn().getAUse() } + QueryCall() { this = [client(), pool()].getMember("query").getACall() } override DataFlow::Node getAQueryArgument() { result = getArgument(0) } } @@ -135,9 +136,8 @@ private module Postgres { string kind; Credentials() { - exists(DataFlow::InvokeNode call, string prop | - call = [client(), newPool()].getAUse() and - this = call.getOptionArgument(0, prop).asExpr() and + exists(string prop | + this = [newClient(), newPool()].getParameter(0).getMember(prop).getARhs().asExpr() and ( prop = "user" and kind = "user name" or @@ -178,7 +178,7 @@ private module Sqlite { meth = "prepare" or meth = "run" | - this = newDb().getMember(meth).getReturn().getAUse() + this = newDb().getMember(meth).getACall() ) } @@ -222,7 +222,7 @@ private module MsSql { /** A call to a MsSql query method. */ private class QueryCall extends DatabaseAccess, DataFlow::MethodCallNode { - QueryCall() { this = request().getMember(["query", "batch"]).getReturn().getAUse() } + QueryCall() { this = request().getMember(["query", "batch"]).getACall() } override DataFlow::Node getAQueryArgument() { result = getArgument(0) } } @@ -250,13 +250,13 @@ private module MsSql { string kind; Credentials() { - exists(DataFlow::InvokeNode call, string prop | + exists(API::Node callee, string prop | ( - call = mssql().getMember("connect").getReturn().getAUse() + callee = mssql().getMember("connect") or - call = mssql().getMember("ConnectionPool").getInstance().getAUse() + callee = mssql().getMember("ConnectionPool") ) and - this = call.getOptionArgument(0, prop).asExpr() and + this = callee.getParameter(0).getMember(prop).getARhs().asExpr() and ( prop = "user" and kind = "user name" or @@ -281,7 +281,7 @@ private module Sequelize { /** A call to `Sequelize.query`. */ private class QueryCall extends DatabaseAccess, DataFlow::MethodCallNode { - QueryCall() { this = newSequelize().getMember("query").getReturn().getAUse() } + QueryCall() { this = newSequelize().getMember("query").getACall() } override DataFlow::Node getAQueryArgument() { result = getArgument(0) } } @@ -300,7 +300,7 @@ private module Sequelize { Credentials() { exists(NewExpr ne, string prop | - ne = newSequelize().getAUse().asExpr() and + ne = sequelize().getAnInstantiation().asExpr() and ( this = ne.getArgument(1) and prop = "username" or @@ -378,8 +378,7 @@ private module Spanner { */ class DatabaseRunCall extends SqlExecution { DatabaseRunCall() { - this = - database().getMember(["run", "runPartitionedUpdate", "runStream"]).getReturn().getAUse() + this = database().getMember(["run", "runPartitionedUpdate", "runStream"]).getACall() } } @@ -388,7 +387,7 @@ private module Spanner { */ class TransactionRunCall extends SqlExecution { TransactionRunCall() { - this = transaction().getMember(["run", "runStream", "runUpdate"]).getReturn().getAUse() + this = transaction().getMember(["run", "runStream", "runUpdate"]).getACall() } } @@ -397,8 +396,7 @@ private module Spanner { */ class ExecuteSqlCall extends SqlExecution { ExecuteSqlCall() { - this = - v1SpannerClient().getMember(["executeSql", "executeStreamingSql"]).getReturn().getAUse() + this = v1SpannerClient().getMember(["executeSql", "executeStreamingSql"]).getACall() } override DataFlow::Node getAQueryArgument() { diff --git a/javascript/ql/src/semmle/javascript/heuristics/AdditionalTaintSteps.qll b/javascript/ql/src/semmle/javascript/heuristics/AdditionalTaintSteps.qll index e920fb10613..5cda6f0d867 100644 --- a/javascript/ql/src/semmle/javascript/heuristics/AdditionalTaintSteps.qll +++ b/javascript/ql/src/semmle/javascript/heuristics/AdditionalTaintSteps.qll @@ -15,9 +15,7 @@ abstract class HeuristicAdditionalTaintStep extends DataFlow::ValueNode { } * A call to `tainted.replace(x, y)` that preserves taint. */ private class HeuristicStringManipulationTaintStep extends HeuristicAdditionalTaintStep, - TaintTracking::AdditionalTaintStep, DataFlow::MethodCallNode { - HeuristicStringManipulationTaintStep() { getMethodName() = "replace" } - + TaintTracking::AdditionalTaintStep, StringReplaceCall { override predicate step(DataFlow::Node pred, DataFlow::Node succ) { pred = getReceiver() and succ = this } diff --git a/javascript/ql/src/semmle/javascript/security/dataflow/CleartextLoggingCustomizations.qll b/javascript/ql/src/semmle/javascript/security/dataflow/CleartextLoggingCustomizations.qll index 6fc13631ef2..eea099a7a79 100644 --- a/javascript/ql/src/semmle/javascript/security/dataflow/CleartextLoggingCustomizations.qll +++ b/javascript/ql/src/semmle/javascript/security/dataflow/CleartextLoggingCustomizations.qll @@ -34,15 +34,11 @@ module CleartextLogging { /** * A call to `.replace()` that seems to mask sensitive information. */ - class MaskingReplacer extends Barrier, DataFlow::MethodCallNode { + class MaskingReplacer extends Barrier, StringReplaceCall { MaskingReplacer() { - this.getCalleeName() = "replace" and - exists(RegExpLiteral reg | - reg = this.getArgument(0).getALocalSource().asExpr() and - reg.isGlobal() and - any(RegExpDot term).getLiteral() = reg - ) and - exists(this.getArgument(1).getStringValue()) + this.isGlobal() and + exists(this.getRawReplacement().getStringValue()) and + any(RegExpDot term).getLiteral() = getRegExp().asExpr() } } diff --git a/javascript/ql/src/semmle/javascript/security/dataflow/ClientSideUrlRedirectCustomizations.qll b/javascript/ql/src/semmle/javascript/security/dataflow/ClientSideUrlRedirectCustomizations.qll index 64c76a84b82..a74a9f58a0f 100644 --- a/javascript/ql/src/semmle/javascript/security/dataflow/ClientSideUrlRedirectCustomizations.qll +++ b/javascript/ql/src/semmle/javascript/security/dataflow/ClientSideUrlRedirectCustomizations.qll @@ -65,7 +65,7 @@ module ClientSideUrlRedirect { or exists(MethodCallExpr mce | queryAccess.asExpr() = mce and - mce = any(RegExpLiteral re).flow().(DataFlow::SourceNode).getAMethodCall("exec").asExpr() and + mce = any(DataFlow::RegExpCreationNode re).getAMethodCall("exec").asExpr() and nd.asExpr() = mce.getArgument(0) ) } diff --git a/javascript/ql/src/semmle/javascript/security/dataflow/IndirectCommandArgument.qll b/javascript/ql/src/semmle/javascript/security/dataflow/IndirectCommandArgument.qll index 519d0938a27..a84b9701ed0 100644 --- a/javascript/ql/src/semmle/javascript/security/dataflow/IndirectCommandArgument.qll +++ b/javascript/ql/src/semmle/javascript/security/dataflow/IndirectCommandArgument.qll @@ -30,7 +30,7 @@ private DataFlow::Node commandArgument(SystemCommandExecution sys, DataFlow::Typ t.start() and result = sys.getACommandArgument() or - exists(DataFlow::TypeBackTracker t2 | t = t2.smallstep(result, commandArgument(sys, t2))) + exists(DataFlow::TypeBackTracker t2 | t2 = t.smallstep(result, commandArgument(sys, t2))) } /** @@ -63,6 +63,26 @@ private DataFlow::SourceNode argumentList(SystemCommandExecution sys) { result = argumentList(sys, DataFlow::TypeBackTracker::end()) } +/** + * Gets a data-flow node whose value ends up being interpreted as an element of the argument list + * `args` after a flow summarised by `t`. + */ +private DataFlow::Node argumentListElement(DataFlow::SourceNode args, DataFlow::TypeBackTracker t) { + t.start() and + args = argumentList(_) and + result = args.getAPropertyWrite().getRhs() + or + exists(DataFlow::TypeBackTracker t2 | t2 = t.smallstep(result, argumentListElement(args, t2))) +} + +/** + * Gets a data-flow node whose value ends up being interpreted as an element of the argument list + * `args`. + */ +private DataFlow::Node argumentListElement(DataFlow::SourceNode args) { + result = argumentListElement(args, DataFlow::TypeBackTracker::end()) +} + /** * Holds if `source` contributes to the arguments of an indirect command execution `sys`. * @@ -86,12 +106,12 @@ predicate isIndirectCommandArgument(DataFlow::Node source, SystemCommandExecutio exists(DataFlow::ArrayCreationNode args, DataFlow::Node shell, string dashC | shellCmd(shell.asExpr(), dashC) and shell = commandArgument(sys) and - args.getAPropertyWrite().getRhs().mayHaveStringValue(dashC) and args = argumentList(sys) and - ( - source = argumentList(sys) - or - source = argumentList(sys).getAPropertyWrite().getRhs() + argumentListElement(args).mayHaveStringValue(dashC) and + exists(DataFlow::SourceNode argsSource | argsSource = argumentList(sys) | + if exists(argsSource.getAPropertyWrite()) + then source = argsSource.getAPropertyWrite().getRhs() + else source = argsSource ) ) } diff --git a/javascript/ql/src/semmle/javascript/security/dataflow/TaintedPathCustomizations.qll b/javascript/ql/src/semmle/javascript/security/dataflow/TaintedPathCustomizations.qll index daf0e9324ee..76b9e350427 100644 --- a/javascript/ql/src/semmle/javascript/security/dataflow/TaintedPathCustomizations.qll +++ b/javascript/ql/src/semmle/javascript/security/dataflow/TaintedPathCustomizations.qll @@ -218,12 +218,12 @@ module TaintedPath { output = this or // non-global replace or replace of something other than /\.\./g, /[/]/g, or /[\.]/g. - this.getCalleeName() = "replace" and + this instanceof StringReplaceCall and input = getReceiver() and output = this and not exists(RegExpLiteral literal, RegExpTerm term | - getArgument(0).getALocalSource().asExpr() = literal and - literal.isGlobal() and + this.(StringReplaceCall).getRegExp().asExpr() = literal and + this.(StringReplaceCall).isGlobal() and literal.getRoot() = term | term.getAMatchedString() = "/" or @@ -247,16 +247,15 @@ module TaintedPath { /** * A call that removes all instances of "../" in the prefix of the string. */ - class DotDotSlashPrefixRemovingReplace extends DataFlow::CallNode { + class DotDotSlashPrefixRemovingReplace extends StringReplaceCall { DataFlow::Node input; DataFlow::Node output; DotDotSlashPrefixRemovingReplace() { - this.getCalleeName() = "replace" and input = getReceiver() and output = this and exists(RegExpLiteral literal, RegExpTerm term | - getArgument(0).getALocalSource().asExpr() = literal and + getRegExp().asExpr() = literal and (term instanceof RegExpStar or term instanceof RegExpPlus) and term.getChild(0) = getADotDotSlashMatcher() | @@ -298,17 +297,16 @@ module TaintedPath { /** * A call that removes all "." or ".." from a path, without also removing all forward slashes. */ - class DotRemovingReplaceCall extends DataFlow::CallNode { + class DotRemovingReplaceCall extends StringReplaceCall { DataFlow::Node input; DataFlow::Node output; DotRemovingReplaceCall() { - this.getCalleeName() = "replace" and input = getReceiver() and output = this and + isGlobal() and exists(RegExpLiteral literal, RegExpTerm term | - getArgument(0).getALocalSource().asExpr() = literal and - literal.isGlobal() and + getRegExp().asExpr() = literal and literal.getRoot() = term and not term.getAMatchedString() = "/" | diff --git a/javascript/ql/src/semmle/javascript/security/dataflow/UnsafeJQueryPlugin.qll b/javascript/ql/src/semmle/javascript/security/dataflow/UnsafeJQueryPlugin.qll index a2a7246b4a7..05c33b71330 100644 --- a/javascript/ql/src/semmle/javascript/security/dataflow/UnsafeJQueryPlugin.qll +++ b/javascript/ql/src/semmle/javascript/security/dataflow/UnsafeJQueryPlugin.qll @@ -39,10 +39,9 @@ module UnsafeJQueryPlugin { StringConcatenation::taintStep(pred, succ, _, any(int i | i >= 1)) or // prefixing through a poor-mans templating system: - exists(DataFlow::MethodCallNode replace | + exists(StringReplaceCall replace | replace = succ and - pred = replace.getArgument(1) and - replace.getMethodName() = "replace" + pred = replace.getRawReplacement() ) } diff --git a/javascript/ql/src/semmle/javascript/security/dataflow/Xss.qll b/javascript/ql/src/semmle/javascript/security/dataflow/Xss.qll index 54e7b7de47d..346ff52c862 100644 --- a/javascript/ql/src/semmle/javascript/security/dataflow/Xss.qll +++ b/javascript/ql/src/semmle/javascript/security/dataflow/Xss.qll @@ -33,11 +33,10 @@ module Shared { * The XSS queries do not attempt to reason about correctness or completeness of sanitizers, * so any such replacement stops taint propagation. */ - class MetacharEscapeSanitizer extends Sanitizer, DataFlow::MethodCallNode { + class MetacharEscapeSanitizer extends Sanitizer, StringReplaceCall { MetacharEscapeSanitizer() { - getMethodName() = "replace" and exists(RegExpConstant c | - c.getLiteral() = getArgument(0).getALocalSource().asExpr() and + c.getLiteral() = getRegExp().asExpr() and c.getValue().regexpMatch("['\"&<>]") ) } diff --git a/javascript/ql/src/semmle/javascript/security/performance/PolynomialReDoSCustomizations.qll b/javascript/ql/src/semmle/javascript/security/performance/PolynomialReDoSCustomizations.qll index 5507f9e6981..ea3d9db3ada 100644 --- a/javascript/ql/src/semmle/javascript/security/performance/PolynomialReDoSCustomizations.qll +++ b/javascript/ql/src/semmle/javascript/security/performance/PolynomialReDoSCustomizations.qll @@ -51,6 +51,7 @@ module PolynomialReDoS { name = "split" or name = "matchAll" or name = "replace" or + name = "replaceAll" or name = "search" ) or diff --git a/javascript/ql/test/ApiGraphs/partial-invoke/VerifyAssertions.expected b/javascript/ql/test/ApiGraphs/partial-invoke/VerifyAssertions.expected new file mode 100644 index 00000000000..e69de29bb2d diff --git a/javascript/ql/test/ApiGraphs/partial-invoke/VerifyAssertions.ql b/javascript/ql/test/ApiGraphs/partial-invoke/VerifyAssertions.ql new file mode 100644 index 00000000000..b9c54e26072 --- /dev/null +++ b/javascript/ql/test/ApiGraphs/partial-invoke/VerifyAssertions.ql @@ -0,0 +1 @@ +import ApiGraphs.VerifyAssertions diff --git a/javascript/ql/test/ApiGraphs/partial-invoke/index.js b/javascript/ql/test/ApiGraphs/partial-invoke/index.js new file mode 100644 index 00000000000..1642e1454f3 --- /dev/null +++ b/javascript/ql/test/ApiGraphs/partial-invoke/index.js @@ -0,0 +1,8 @@ +const cp = require('child_process'); + +module.exports = function () { + return cp.spawn.bind( + cp, // def (parameter -1 (member spawn (member exports (module child_process)))) + "cat" // def (parameter 0 (member spawn (member exports (module child_process)))) + ); +}; \ No newline at end of file diff --git a/javascript/ql/test/ApiGraphs/partial-invoke/package.json b/javascript/ql/test/ApiGraphs/partial-invoke/package.json new file mode 100644 index 00000000000..279f51b6a5e --- /dev/null +++ b/javascript/ql/test/ApiGraphs/partial-invoke/package.json @@ -0,0 +1,3 @@ +{ + "name": "partial-invoke" +} \ No newline at end of file diff --git a/javascript/ql/test/library-tests/InterProceduralFlow/TaintTracking.expected b/javascript/ql/test/library-tests/InterProceduralFlow/TaintTracking.expected index c3ea1cdb5f2..823db35f84d 100644 --- a/javascript/ql/test/library-tests/InterProceduralFlow/TaintTracking.expected +++ b/javascript/ql/test/library-tests/InterProceduralFlow/TaintTracking.expected @@ -86,6 +86,8 @@ | tst.js:2:17:2:22 | "src1" | tst.js:61:16:61:18 | o.r | | tst.js:2:17:2:22 | "src1" | tst.js:68:16:68:22 | inner() | | tst.js:2:17:2:22 | "src1" | tst.js:80:16:80:22 | outer() | +| tst.js:2:17:2:22 | "src1" | tst.js:87:16:87:43 | source1 ... /g, "") | +| tst.js:2:17:2:22 | "src1" | tst.js:88:16:88:46 | "foo".r ... ource1) | | underscore.js:2:17:2:22 | "src1" | underscore.js:3:15:3:28 | _.max(source1) | | underscore.js:5:17:5:22 | "src2" | underscore.js:6:15:6:34 | _.union([], source2) | | underscore.js:5:17:5:22 | "src2" | underscore.js:7:15:7:32 | _.zip(source2, []) | diff --git a/javascript/ql/test/library-tests/InterProceduralFlow/tst.js b/javascript/ql/test/library-tests/InterProceduralFlow/tst.js index a557538bd11..a8fc56e3d52 100644 --- a/javascript/ql/test/library-tests/InterProceduralFlow/tst.js +++ b/javascript/ql/test/library-tests/InterProceduralFlow/tst.js @@ -83,4 +83,7 @@ o.notTracked = source1; var sink22 = o.notTracked; + + var sink23 = source1.replaceAll(/f/g, ""); + var sink24 = "foo".replaceAll(/f/g, source1); })(); diff --git a/javascript/ql/test/library-tests/Promises/flow2.js b/javascript/ql/test/library-tests/Promises/flow2.js index ef91740e0d5..ccafb83fd3f 100644 --- a/javascript/ql/test/library-tests/Promises/flow2.js +++ b/javascript/ql/test/library-tests/Promises/flow2.js @@ -18,4 +18,10 @@ var [clean3, tainted3] = await Promise.all(["clean", Promise.resolve(source)]); sink(clean3); // OK sink(tainted3); // NOT OK - but only flagged by taint-tracking + + var tainted4 = await Promise.race(["clean", Promise.resolve(source)]); + sink(tainted4); // NOT OK - but only flagged by taint-tracking + + var tainted5 = await Promise.any(["clean", Promise.resolve(source)]); + sink(tainted5); // NOT OK - but only flagged by taint-tracking }); \ No newline at end of file diff --git a/javascript/ql/test/library-tests/Promises/tests.expected b/javascript/ql/test/library-tests/Promises/tests.expected index 21c685e0c40..00320cc83fc 100644 --- a/javascript/ql/test/library-tests/Promises/tests.expected +++ b/javascript/ql/test/library-tests/Promises/tests.expected @@ -9,6 +9,12 @@ test_ResolvedPromiseDefinition | flow2.js:18:33:18:79 | Promise ... urce)]) | flow2.js:18:46:18:52 | "clean" | | flow2.js:18:33:18:79 | Promise ... urce)]) | flow2.js:18:55:18:77 | Promise ... source) | | flow2.js:18:55:18:77 | Promise ... source) | flow2.js:18:71:18:76 | source | +| flow2.js:22:23:22:70 | Promise ... urce)]) | flow2.js:22:37:22:43 | "clean" | +| flow2.js:22:23:22:70 | Promise ... urce)]) | flow2.js:22:46:22:68 | Promise ... source) | +| flow2.js:22:46:22:68 | Promise ... source) | flow2.js:22:62:22:67 | source | +| flow2.js:25:23:25:69 | Promise ... urce)]) | flow2.js:25:36:25:42 | "clean" | +| flow2.js:25:23:25:69 | Promise ... urce)]) | flow2.js:25:45:25:67 | Promise ... source) | +| flow2.js:25:45:25:67 | Promise ... source) | flow2.js:25:61:25:66 | source | | flow.js:4:11:4:33 | Promise ... source) | flow.js:4:27:4:32 | source | | flow.js:20:2:20:24 | Promise ... source) | flow.js:20:18:20:23 | source | | flow.js:22:2:22:24 | Promise ... source) | flow.js:22:18:22:23 | source | @@ -201,6 +207,8 @@ flow | flow2.js:2:15:2:22 | "source" | flow2.js:6:8:6:13 | arr[0] | | flow2.js:2:15:2:22 | "source" | flow2.js:12:7:12:13 | tainted | | flow2.js:2:15:2:22 | "source" | flow2.js:16:7:16:14 | tainted2 | +| flow2.js:2:15:2:22 | "source" | flow2.js:23:7:23:14 | tainted4 | +| flow2.js:2:15:2:22 | "source" | flow2.js:26:7:26:14 | tainted5 | | flow.js:2:15:2:22 | "source" | flow.js:5:7:5:14 | await p1 | | flow.js:2:15:2:22 | "source" | flow.js:8:7:8:14 | await p2 | | flow.js:2:15:2:22 | "source" | flow.js:17:8:17:8 | e | @@ -255,6 +263,12 @@ typetrack | flow2.js:18:27:18:79 | await P ... urce)]) | flow2.js:18:33:18:79 | Promise ... urce)]) | load $PromiseResolveField$ | | flow2.js:18:33:18:79 | Promise ... urce)]) | flow2.js:18:45:18:78 | ["clean ... ource)] | copy $PromiseResolveField$ | | flow2.js:18:33:18:79 | Promise ... urce)]) | flow2.js:18:45:18:78 | ["clean ... ource)] | store $PromiseResolveField$ | +| flow2.js:22:17:22:70 | await P ... urce)]) | flow2.js:22:23:22:70 | Promise ... urce)]) | load $PromiseResolveField$ | +| flow2.js:22:23:22:70 | Promise ... urce)]) | flow2.js:22:46:22:68 | Promise ... source) | copy $PromiseResolveField$ | +| flow2.js:22:23:22:70 | Promise ... urce)]) | flow2.js:22:46:22:68 | Promise ... source) | store $PromiseResolveField$ | +| flow2.js:25:17:25:69 | await P ... urce)]) | flow2.js:25:23:25:69 | Promise ... urce)]) | load $PromiseResolveField$ | +| flow2.js:25:23:25:69 | Promise ... urce)]) | flow2.js:25:45:25:67 | Promise ... source) | copy $PromiseResolveField$ | +| flow2.js:25:23:25:69 | Promise ... urce)]) | flow2.js:25:45:25:67 | Promise ... source) | store $PromiseResolveField$ | | flow.js:20:2:20:43 | Promise ... ink(x)) | flow.js:20:36:20:42 | sink(x) | copy $PromiseResolveField$ | | flow.js:20:2:20:43 | Promise ... ink(x)) | flow.js:20:36:20:42 | sink(x) | store $PromiseResolveField$ | | flow.js:20:31:20:31 | x | flow.js:20:2:20:24 | Promise ... source) | load $PromiseResolveField$ | diff --git a/javascript/ql/test/library-tests/frameworks/ClientRequests/ClientRequests.expected b/javascript/ql/test/library-tests/frameworks/ClientRequests/ClientRequests.expected index 1c91826ac8d..5ba426c968f 100644 --- a/javascript/ql/test/library-tests/frameworks/ClientRequests/ClientRequests.expected +++ b/javascript/ql/test/library-tests/frameworks/ClientRequests/ClientRequests.expected @@ -67,6 +67,9 @@ test_ClientRequest | tst.js:202:5:208:7 | $.ajax( ... }}) | | tst.js:210:2:210:21 | $.get("example.php") | | tst.js:219:5:219:41 | data.so ... Host"}) | +| tst.js:229:5:229:67 | needle( ... ptions) | +| tst.js:231:5:233:6 | needle. ... \\n }) | +| tst.js:235:5:237:6 | needle. ... \\n }) | test_getADataNode | tst.js:53:5:53:23 | axios({data: data}) | tst.js:53:18:53:21 | data | | tst.js:57:5:57:39 | axios.p ... data2}) | tst.js:57:19:57:23 | data1 | @@ -97,6 +100,10 @@ test_getADataNode | tst.js:183:2:183:60 | $.post( ... ) { }) | tst.js:183:28:183:37 | "PostData" | | tst.js:187:2:193:3 | $.ajax( ... on"\\n\\t}) | tst.js:190:11:190:20 | "AjaxData" | | tst.js:219:5:219:41 | data.so ... Host"}) | tst.js:223:23:223:30 | "foobar" | +| tst.js:229:5:229:67 | needle( ... ptions) | tst.js:228:32:228:70 | { 'X-Cu ... tuna' } | +| tst.js:229:5:229:67 | needle( ... ptions) | tst.js:229:50:229:57 | "MyData" | +| tst.js:235:5:237:6 | needle. ... \\n }) | tst.js:228:32:228:70 | { 'X-Cu ... tuna' } | +| tst.js:235:5:237:6 | needle. ... \\n }) | tst.js:235:44:235:49 | "data" | test_getHost | tst.js:87:5:87:39 | http.ge ... host}) | tst.js:87:34:87:37 | host | | tst.js:89:5:89:23 | axios({host: host}) | tst.js:89:18:89:21 | host | @@ -177,6 +184,9 @@ test_getUrl | tst.js:202:5:208:7 | $.ajax( ... }}) | tst.js:203:10:203:22 | "example.php" | | tst.js:210:2:210:21 | $.get("example.php") | tst.js:210:8:210:20 | "example.php" | | tst.js:219:5:219:41 | data.so ... Host"}) | tst.js:219:25:219:40 | {host: "myHost"} | +| tst.js:229:5:229:67 | needle( ... ptions) | tst.js:229:20:229:47 | "http:/ ... oo/bar" | +| tst.js:231:5:233:6 | needle. ... \\n }) | tst.js:231:16:231:35 | "http://example.org" | +| tst.js:235:5:237:6 | needle. ... \\n }) | tst.js:235:17:235:41 | "http:/ ... g/post" | test_getAResponseDataNode | tst.js:19:5:19:23 | requestPromise(url) | tst.js:19:5:19:23 | requestPromise(url) | text | true | | tst.js:21:5:21:23 | superagent.get(url) | tst.js:21:5:21:23 | superagent.get(url) | stream | true | @@ -238,3 +248,8 @@ test_getAResponseDataNode | tst.js:202:5:208:7 | $.ajax( ... }}) | tst.js:207:21:207:36 | err.responseText | json | false | | tst.js:210:2:210:21 | $.get("example.php") | tst.js:210:55:210:70 | xhr.responseText | | false | | tst.js:219:5:219:41 | data.so ... Host"}) | tst.js:221:29:221:32 | data | text | false | +| tst.js:229:5:229:67 | needle( ... ptions) | tst.js:229:5:229:67 | needle( ... ptions) | fetch.response | true | +| tst.js:231:5:233:6 | needle. ... \\n }) | tst.js:231:44:231:47 | resp | fetch.response | false | +| tst.js:231:5:233:6 | needle. ... \\n }) | tst.js:231:50:231:53 | body | json | false | +| tst.js:235:5:237:6 | needle. ... \\n }) | tst.js:235:67:235:70 | resp | fetch.response | false | +| tst.js:235:5:237:6 | needle. ... \\n }) | tst.js:235:73:235:76 | body | json | false | diff --git a/javascript/ql/test/library-tests/frameworks/ClientRequests/tst.js b/javascript/ql/test/library-tests/frameworks/ClientRequests/tst.js index e828423f91c..a24a0dca791 100644 --- a/javascript/ql/test/library-tests/frameworks/ClientRequests/tst.js +++ b/javascript/ql/test/library-tests/frameworks/ClientRequests/tst.js @@ -221,4 +221,18 @@ const net = require("net"); data.socket.on("data", (data) => {}); data.socket.write("foobar"); +})(); + +const needle = require("needle"); +(function () { + const options = { headers: { 'X-Custom-Header': 'Bumbaway atuna' } }; + needle("POST", "http://example.org/foo/bar", "MyData", options).then(function(resp) { console.log(resp.body) }); + + needle.get("http://example.org", (err, resp, body) => { + + }); + + needle.post("http://example.org/post", "data", options, (err, resp, body) => { + + }); })(); \ No newline at end of file diff --git a/javascript/ql/test/library-tests/frameworks/NodeJSLib/src/http.js b/javascript/ql/test/library-tests/frameworks/NodeJSLib/src/http.js index 11a52f701f9..12dc11efcaf 100644 --- a/javascript/ql/test/library-tests/frameworks/NodeJSLib/src/http.js +++ b/javascript/ql/test/library-tests/frameworks/NodeJSLib/src/http.js @@ -68,3 +68,9 @@ function getArrowHandler() { return (req,res) => f(); } http.createServer(getArrowHandler()); + +http.createServer(function (req, res) { + req.on("data", chunk => { // RemoteFlowSource + res.send(chunk); + }) +}); \ No newline at end of file diff --git a/javascript/ql/test/library-tests/frameworks/NodeJSLib/tests.expected b/javascript/ql/test/library-tests/frameworks/NodeJSLib/tests.expected index 6234075f5f1..2e048dc43b8 100644 --- a/javascript/ql/test/library-tests/frameworks/NodeJSLib/tests.expected +++ b/javascript/ql/test/library-tests/frameworks/NodeJSLib/tests.expected @@ -8,6 +8,7 @@ test_isCreateServer | src/http.js:60:1:60:33 | createS ... res){}) | | src/http.js:62:1:65:2 | http.cr ... 2");\\n}) | | src/http.js:70:1:70:36 | http.cr ... dler()) | +| src/http.js:72:1:76:2 | http.cr ... })\\n}) | | src/https.js:4:14:10:2 | https.c ... foo;\\n}) | | src/https.js:12:1:16:2 | https.c ... r");\\n}) | | src/indirect2.js:18:14:18:35 | http.cr ... er(get) | @@ -53,6 +54,8 @@ test_ResponseExpr | src/http.js:63:3:63:5 | res | src/http.js:62:19:65:1 | functio ... r2");\\n} | | src/http.js:64:3:64:5 | res | src/http.js:62:19:65:1 | functio ... r2");\\n} | | src/http.js:68:17:68:19 | res | src/http.js:68:12:68:27 | (req,res) => f() | +| src/http.js:72:34:72:36 | res | src/http.js:72:19:76:1 | functio ... \\n })\\n} | +| src/http.js:74:5:74:7 | res | src/http.js:72:19:76:1 | functio ... \\n })\\n} | | src/https.js:4:47:4:49 | res | src/https.js:4:33:10:1 | functio ... .foo;\\n} | | src/https.js:7:3:7:5 | res | src/https.js:4:33:10:1 | functio ... .foo;\\n} | | src/https.js:12:34:12:36 | res | src/https.js:12:20:16:1 | functio ... ar");\\n} | @@ -87,6 +90,7 @@ test_RouteSetup_getServer | src/http.js:60:1:60:33 | createS ... res){}) | src/http.js:60:1:60:33 | createS ... res){}) | | src/http.js:62:1:65:2 | http.cr ... 2");\\n}) | src/http.js:62:1:65:2 | http.cr ... 2");\\n}) | | src/http.js:70:1:70:36 | http.cr ... dler()) | src/http.js:70:1:70:36 | http.cr ... dler()) | +| src/http.js:72:1:76:2 | http.cr ... })\\n}) | src/http.js:72:1:76:2 | http.cr ... })\\n}) | | src/https.js:4:14:10:2 | https.c ... foo;\\n}) | src/https.js:4:14:10:2 | https.c ... foo;\\n}) | | src/https.js:12:1:16:2 | https.c ... r");\\n}) | src/https.js:12:1:16:2 | https.c ... r");\\n}) | | src/indirect2.js:18:14:18:35 | http.cr ... er(get) | src/indirect2.js:18:14:18:35 | http.cr ... er(get) | @@ -112,6 +116,7 @@ test_ServerDefinition | src/http.js:60:1:60:33 | createS ... res){}) | | src/http.js:62:1:65:2 | http.cr ... 2");\\n}) | | src/http.js:70:1:70:36 | http.cr ... dler()) | +| src/http.js:72:1:76:2 | http.cr ... })\\n}) | | src/https.js:4:14:10:2 | https.c ... foo;\\n}) | | src/https.js:12:1:16:2 | https.c ... r");\\n}) | | src/indirect2.js:18:14:18:35 | http.cr ... er(get) | @@ -142,6 +147,8 @@ test_RouteHandler_getAResponseExpr | src/http.js:62:19:65:1 | functio ... r2");\\n} | src/http.js:63:3:63:5 | res | | src/http.js:62:19:65:1 | functio ... r2");\\n} | src/http.js:64:3:64:5 | res | | src/http.js:68:12:68:27 | (req,res) => f() | src/http.js:68:17:68:19 | res | +| src/http.js:72:19:76:1 | functio ... \\n })\\n} | src/http.js:72:34:72:36 | res | +| src/http.js:72:19:76:1 | functio ... \\n })\\n} | src/http.js:74:5:74:7 | res | | src/https.js:4:33:10:1 | functio ... .foo;\\n} | src/https.js:4:47:4:49 | res | | src/https.js:4:33:10:1 | functio ... .foo;\\n} | src/https.js:7:3:7:5 | res | | src/https.js:12:20:16:1 | functio ... ar");\\n} | src/https.js:12:34:12:36 | res | @@ -169,6 +176,7 @@ test_ServerDefinition_getARouteHandler | src/http.js:60:1:60:33 | createS ... res){}) | src/http.js:60:14:60:32 | function(req,res){} | | src/http.js:62:1:65:2 | http.cr ... 2");\\n}) | src/http.js:62:19:65:1 | functio ... r2");\\n} | | src/http.js:70:1:70:36 | http.cr ... dler()) | src/http.js:68:12:68:27 | (req,res) => f() | +| src/http.js:72:1:76:2 | http.cr ... })\\n}) | src/http.js:72:19:76:1 | functio ... \\n })\\n} | | src/https.js:4:14:10:2 | https.c ... foo;\\n}) | src/https.js:4:33:10:1 | functio ... .foo;\\n} | | src/https.js:12:1:16:2 | https.c ... r");\\n}) | src/https.js:12:20:16:1 | functio ... ar");\\n} | | src/indirect2.js:18:14:18:35 | http.cr ... er(get) | src/indirect2.js:9:1:11:1 | functio ... res);\\n} | @@ -198,6 +206,7 @@ test_RouteSetup_getARouteHandler | src/http.js:70:1:70:36 | http.cr ... dler()) | src/http.js:67:1:69:1 | return of function getArrowHandler | | src/http.js:70:1:70:36 | http.cr ... dler()) | src/http.js:68:12:68:27 | (req,res) => f() | | src/http.js:70:1:70:36 | http.cr ... dler()) | src/http.js:70:19:70:35 | getArrowHandler() | +| src/http.js:72:1:76:2 | http.cr ... })\\n}) | src/http.js:72:19:76:1 | functio ... \\n })\\n} | | src/https.js:4:14:10:2 | https.c ... foo;\\n}) | src/https.js:4:33:10:1 | functio ... .foo;\\n} | | src/https.js:12:1:16:2 | https.c ... r");\\n}) | src/https.js:12:20:16:1 | functio ... ar");\\n} | | src/indirect2.js:18:14:18:35 | http.cr ... er(get) | src/indirect2.js:9:1:11:1 | functio ... res);\\n} | @@ -224,6 +233,7 @@ test_RemoteFlowSources | src/http.js:30:28:30:32 | chunk | | src/http.js:40:23:40:30 | authInfo | | src/http.js:45:23:45:27 | error | +| src/http.js:73:18:73:22 | chunk | | src/https.js:6:26:6:32 | req.url | | src/https.js:8:3:8:20 | req.headers.cookie | | src/https.js:9:3:9:17 | req.headers.foo | @@ -238,6 +248,7 @@ test_RouteHandler | src/http.js:60:14:60:32 | function(req,res){} | src/http.js:60:1:60:33 | createS ... res){}) | | src/http.js:62:19:65:1 | functio ... r2");\\n} | src/http.js:62:1:65:2 | http.cr ... 2");\\n}) | | src/http.js:68:12:68:27 | (req,res) => f() | src/http.js:70:1:70:36 | http.cr ... dler()) | +| src/http.js:72:19:76:1 | functio ... \\n })\\n} | src/http.js:72:1:76:2 | http.cr ... })\\n}) | | src/https.js:4:33:10:1 | functio ... .foo;\\n} | src/https.js:4:14:10:2 | https.c ... foo;\\n}) | | src/https.js:12:20:16:1 | functio ... ar");\\n} | src/https.js:12:1:16:2 | https.c ... r");\\n}) | | src/indirect2.js:9:1:11:1 | functio ... res);\\n} | src/indirect2.js:18:14:18:35 | http.cr ... er(get) | @@ -259,6 +270,8 @@ test_RequestExpr | src/http.js:62:28:62:30 | req | src/http.js:62:19:65:1 | functio ... r2");\\n} | | src/http.js:63:17:63:19 | req | src/http.js:62:19:65:1 | functio ... r2");\\n} | | src/http.js:68:13:68:15 | req | src/http.js:68:12:68:27 | (req,res) => f() | +| src/http.js:72:29:72:31 | req | src/http.js:72:19:76:1 | functio ... \\n })\\n} | +| src/http.js:73:3:73:5 | req | src/http.js:72:19:76:1 | functio ... \\n })\\n} | | src/https.js:4:42:4:44 | req | src/https.js:4:33:10:1 | functio ... .foo;\\n} | | src/https.js:6:26:6:28 | req | src/https.js:4:33:10:1 | functio ... .foo;\\n} | | src/https.js:8:3:8:5 | req | src/https.js:4:33:10:1 | functio ... .foo;\\n} | @@ -296,6 +309,8 @@ test_RouteHandler_getARequestExpr | src/http.js:62:19:65:1 | functio ... r2");\\n} | src/http.js:62:28:62:30 | req | | src/http.js:62:19:65:1 | functio ... r2");\\n} | src/http.js:63:17:63:19 | req | | src/http.js:68:12:68:27 | (req,res) => f() | src/http.js:68:13:68:15 | req | +| src/http.js:72:19:76:1 | functio ... \\n })\\n} | src/http.js:72:29:72:31 | req | +| src/http.js:72:19:76:1 | functio ... \\n })\\n} | src/http.js:73:3:73:5 | req | | src/https.js:4:33:10:1 | functio ... .foo;\\n} | src/https.js:4:42:4:44 | req | | src/https.js:4:33:10:1 | functio ... .foo;\\n} | src/https.js:6:26:6:28 | req | | src/https.js:4:33:10:1 | functio ... .foo;\\n} | src/https.js:8:3:8:5 | req | diff --git a/javascript/ql/test/library-tests/frameworks/SQL/SqlString.expected b/javascript/ql/test/library-tests/frameworks/SQL/SqlString.expected index b3e9e240944..b2e0cc4b046 100644 --- a/javascript/ql/test/library-tests/frameworks/SQL/SqlString.expected +++ b/javascript/ql/test/library-tests/frameworks/SQL/SqlString.expected @@ -3,6 +3,7 @@ | mssql2.js:5:15:5:34 | 'select 1 as number' | | mssql2.js:13:15:13:66 | 'create ... table' | | mssql2.js:22:24:22:43 | 'select 1 as number' | +| mssql2.js:29:30:29:81 | 'create ... table' | | mysql1.js:13:18:13:43 | 'SELECT ... lution' | | mysql1.js:18:18:22:1 | {\\n s ... vid']\\n} | | mysql1a.js:17:18:17:43 | 'SELECT ... lution' | diff --git a/javascript/ql/test/library-tests/frameworks/SQL/mssql2.js b/javascript/ql/test/library-tests/frameworks/SQL/mssql2.js index 9b64f06068a..23eb7148a2c 100644 --- a/javascript/ql/test/library-tests/frameworks/SQL/mssql2.js +++ b/javascript/ql/test/library-tests/frameworks/SQL/mssql2.js @@ -1,6 +1,6 @@ // Adapted from https://github.com/tediousjs/node-mssql#readme const sql = require('mssql') - + const request = new sql.Request() request.query('select 1 as number', (err, result) => { // ... error checks @@ -19,7 +19,16 @@ class C { this.req = req; } send() { - this.req.query('select 1 as number', (err, result) => {}) + this.req.query('select 1 as number', (err, result) => { }) } } new C(new sql.Request()); + +var obj = { + foo: function () { + return request.batch('create procedure #temporary as select * from table', (err, result) => { + // ... error checks + }) + } +} +obj.foo("foo", "bar", "baz"); // An API-graphs gotcha: "baz" should not be considered a `SqlString` diff --git a/javascript/ql/test/query-tests/Performance/ReDoS/PolynomialBackTracking.expected b/javascript/ql/test/query-tests/Performance/ReDoS/PolynomialBackTracking.expected index 98f5cb00ed4..1d78b51e8bf 100644 --- a/javascript/ql/test/query-tests/Performance/ReDoS/PolynomialBackTracking.expected +++ b/javascript/ql/test/query-tests/Performance/ReDoS/PolynomialBackTracking.expected @@ -17,6 +17,7 @@ | polynomial-redos.js:31:42:31:43 | -+ | it can start matching anywhere | | polynomial-redos.js:32:45:32:47 | \\n* | it can start matching anywhere | | polynomial-redos.js:33:17:33:20 | (.)* | it can start matching anywhere | +| polynomial-redos.js:48:22:48:24 | \\s* | it can start matching anywhere | | regexplib/address.js:18:26:18:31 | [ \\w]* | it can start matching anywhere after the start of the preceeding '[ \\w]{3,}' | | regexplib/address.js:20:144:20:147 | [ ]+ | it can start matching anywhere after the start of the preceeding '[a-zA-Z0-9 \\-.]{6,}' | | regexplib/address.js:24:26:24:31 | [ \\w]* | it can start matching anywhere after the start of the preceeding '[ \\w]{3,}' | diff --git a/javascript/ql/test/query-tests/Performance/ReDoS/PolynomialReDoS.expected b/javascript/ql/test/query-tests/Performance/ReDoS/PolynomialReDoS.expected index 4d3a9a12da6..e8ba1a56d67 100644 --- a/javascript/ql/test/query-tests/Performance/ReDoS/PolynomialReDoS.expected +++ b/javascript/ql/test/query-tests/Performance/ReDoS/PolynomialReDoS.expected @@ -28,6 +28,8 @@ nodes | polynomial-redos.js:30:2:30:8 | tainted | | polynomial-redos.js:33:2:33:8 | tainted | | polynomial-redos.js:33:2:33:8 | tainted | +| polynomial-redos.js:48:2:48:8 | tainted | +| polynomial-redos.js:48:2:48:8 | tainted | edges | polynomial-redos.js:5:6:5:32 | tainted | polynomial-redos.js:7:2:7:8 | tainted | | polynomial-redos.js:5:6:5:32 | tainted | polynomial-redos.js:7:2:7:8 | tainted | @@ -55,6 +57,8 @@ edges | polynomial-redos.js:5:6:5:32 | tainted | polynomial-redos.js:30:2:30:8 | tainted | | polynomial-redos.js:5:6:5:32 | tainted | polynomial-redos.js:33:2:33:8 | tainted | | polynomial-redos.js:5:6:5:32 | tainted | polynomial-redos.js:33:2:33:8 | tainted | +| polynomial-redos.js:5:6:5:32 | tainted | polynomial-redos.js:48:2:48:8 | tainted | +| polynomial-redos.js:5:6:5:32 | tainted | polynomial-redos.js:48:2:48:8 | tainted | | polynomial-redos.js:5:16:5:32 | req.query.tainted | polynomial-redos.js:5:6:5:32 | tainted | | polynomial-redos.js:5:16:5:32 | req.query.tainted | polynomial-redos.js:5:6:5:32 | tainted | #select @@ -72,3 +76,4 @@ edges | polynomial-redos.js:27:77:27:83 | tainted | polynomial-redos.js:5:16:5:32 | req.query.tainted | polynomial-redos.js:27:77:27:83 | tainted | This expensive $@ use depends on $@. | polynomial-redos.js:27:14:27:22 | [A-Z]{2,} | regular expression | polynomial-redos.js:5:16:5:32 | req.query.tainted | a user-provided value | | polynomial-redos.js:30:2:30:8 | tainted | polynomial-redos.js:5:16:5:32 | req.query.tainted | polynomial-redos.js:30:2:30:8 | tainted | This expensive $@ use depends on $@. | polynomial-redos.js:30:19:30:22 | [?]+ | regular expression | polynomial-redos.js:5:16:5:32 | req.query.tainted | a user-provided value | | polynomial-redos.js:33:2:33:8 | tainted | polynomial-redos.js:5:16:5:32 | req.query.tainted | polynomial-redos.js:33:2:33:8 | tainted | This expensive $@ use depends on $@. | polynomial-redos.js:33:17:33:20 | (.)* | regular expression | polynomial-redos.js:5:16:5:32 | req.query.tainted | a user-provided value | +| polynomial-redos.js:48:2:48:8 | tainted | polynomial-redos.js:5:16:5:32 | req.query.tainted | polynomial-redos.js:48:2:48:8 | tainted | This expensive $@ use depends on $@. | polynomial-redos.js:48:22:48:24 | \\s* | regular expression | polynomial-redos.js:5:16:5:32 | req.query.tainted | a user-provided value | diff --git a/javascript/ql/test/query-tests/Performance/ReDoS/polynomial-redos.js b/javascript/ql/test/query-tests/Performance/ReDoS/polynomial-redos.js index 4175bd48a50..1dcce05f3c7 100644 --- a/javascript/ql/test/query-tests/Performance/ReDoS/polynomial-redos.js +++ b/javascript/ql/test/query-tests/Performance/ReDoS/polynomial-redos.js @@ -45,5 +45,5 @@ app.use(function(req, res) { tainted.match(/^(?:\.?[a-zA-Z_][a-zA-Z_0-9]*)+$/); // NOT OK - but not flagged tainted.match(/^(?:\.?[a-zA-Z_][a-zA-Z_0-9]*)(?:\.[a-zA-Z_][a-zA-Z_0-9]*)*$/); // OK - + tainted.replaceAll(/\s*\n\s*/g, ' '); // NOT OK }); diff --git a/javascript/ql/test/query-tests/Security/CWE-078/CommandInjection.expected b/javascript/ql/test/query-tests/Security/CWE-078/CommandInjection.expected index 2ebaeec1c24..5d9c4fcdb61 100644 --- a/javascript/ql/test/query-tests/Security/CWE-078/CommandInjection.expected +++ b/javascript/ql/test/query-tests/Security/CWE-078/CommandInjection.expected @@ -23,8 +23,6 @@ nodes | child_process-test.js:25:13:25:31 | "foo" + cmd + "bar" | | child_process-test.js:25:13:25:31 | "foo" + cmd + "bar" | | child_process-test.js:25:21:25:23 | cmd | -| child_process-test.js:39:18:39:30 | [ flag, cmd ] | -| child_process-test.js:39:18:39:30 | [ flag, cmd ] | | child_process-test.js:39:26:39:28 | cmd | | child_process-test.js:39:26:39:28 | cmd | | child_process-test.js:43:15:43:17 | cmd | @@ -36,7 +34,6 @@ nodes | child_process-test.js:56:25:56:58 | ['/C', ... , cmd]) | | child_process-test.js:56:25:56:58 | ['/C', ... , cmd]) | | child_process-test.js:56:46:56:57 | ["bar", cmd] | -| child_process-test.js:56:46:56:57 | ["bar", cmd] | | child_process-test.js:56:54:56:56 | cmd | | child_process-test.js:56:54:56:56 | cmd | | child_process-test.js:57:25:57:49 | ['/C', ... at(cmd) | @@ -55,6 +52,26 @@ nodes | child_process-test.js:83:19:83:36 | req.query.fileName | | child_process-test.js:85:37:85:54 | req.query.fileName | | child_process-test.js:85:37:85:54 | req.query.fileName | +| exec-sh2.js:9:17:9:23 | command | +| exec-sh2.js:10:40:10:46 | command | +| exec-sh2.js:10:40:10:46 | command | +| exec-sh2.js:14:9:14:49 | cmd | +| exec-sh2.js:14:15:14:38 | url.par ... , true) | +| exec-sh2.js:14:15:14:44 | url.par ... ).query | +| exec-sh2.js:14:15:14:49 | url.par ... ry.path | +| exec-sh2.js:14:25:14:31 | req.url | +| exec-sh2.js:14:25:14:31 | req.url | +| exec-sh2.js:15:12:15:14 | cmd | +| exec-sh.js:13:17:13:23 | command | +| exec-sh.js:15:44:15:50 | command | +| exec-sh.js:15:44:15:50 | command | +| exec-sh.js:19:9:19:49 | cmd | +| exec-sh.js:19:15:19:38 | url.par ... , true) | +| exec-sh.js:19:15:19:44 | url.par ... ).query | +| exec-sh.js:19:15:19:49 | url.par ... ry.path | +| exec-sh.js:19:25:19:31 | req.url | +| exec-sh.js:19:25:19:31 | req.url | +| exec-sh.js:20:12:20:14 | cmd | | execSeries.js:3:20:3:22 | arr | | execSeries.js:6:14:6:16 | arr | | execSeries.js:6:14:6:21 | arr[i++] | @@ -156,12 +173,9 @@ edges | child_process-test.js:6:25:6:31 | req.url | child_process-test.js:6:15:6:38 | url.par ... , true) | | child_process-test.js:25:21:25:23 | cmd | child_process-test.js:25:13:25:31 | "foo" + cmd + "bar" | | child_process-test.js:25:21:25:23 | cmd | child_process-test.js:25:13:25:31 | "foo" + cmd + "bar" | -| child_process-test.js:39:26:39:28 | cmd | child_process-test.js:39:18:39:30 | [ flag, cmd ] | -| child_process-test.js:39:26:39:28 | cmd | child_process-test.js:39:18:39:30 | [ flag, cmd ] | | child_process-test.js:56:46:56:57 | ["bar", cmd] | child_process-test.js:56:25:56:58 | ['/C', ... , cmd]) | | child_process-test.js:56:46:56:57 | ["bar", cmd] | child_process-test.js:56:25:56:58 | ['/C', ... , cmd]) | | child_process-test.js:56:54:56:56 | cmd | child_process-test.js:56:46:56:57 | ["bar", cmd] | -| child_process-test.js:56:54:56:56 | cmd | child_process-test.js:56:46:56:57 | ["bar", cmd] | | child_process-test.js:57:46:57:48 | cmd | child_process-test.js:57:25:57:49 | ['/C', ... at(cmd) | | child_process-test.js:57:46:57:48 | cmd | child_process-test.js:57:25:57:49 | ['/C', ... at(cmd) | | child_process-test.js:73:9:73:49 | cmd | child_process-test.js:75:29:75:31 | cmd | @@ -174,6 +188,24 @@ edges | child_process-test.js:83:19:83:36 | req.query.fileName | child_process-test.js:83:19:83:36 | req.query.fileName | | child_process-test.js:85:37:85:54 | req.query.fileName | lib/subLib/index.js:7:32:7:35 | name | | child_process-test.js:85:37:85:54 | req.query.fileName | lib/subLib/index.js:7:32:7:35 | name | +| exec-sh2.js:9:17:9:23 | command | exec-sh2.js:10:40:10:46 | command | +| exec-sh2.js:9:17:9:23 | command | exec-sh2.js:10:40:10:46 | command | +| exec-sh2.js:14:9:14:49 | cmd | exec-sh2.js:15:12:15:14 | cmd | +| exec-sh2.js:14:15:14:38 | url.par ... , true) | exec-sh2.js:14:15:14:44 | url.par ... ).query | +| exec-sh2.js:14:15:14:44 | url.par ... ).query | exec-sh2.js:14:15:14:49 | url.par ... ry.path | +| exec-sh2.js:14:15:14:49 | url.par ... ry.path | exec-sh2.js:14:9:14:49 | cmd | +| exec-sh2.js:14:25:14:31 | req.url | exec-sh2.js:14:15:14:38 | url.par ... , true) | +| exec-sh2.js:14:25:14:31 | req.url | exec-sh2.js:14:15:14:38 | url.par ... , true) | +| exec-sh2.js:15:12:15:14 | cmd | exec-sh2.js:9:17:9:23 | command | +| exec-sh.js:13:17:13:23 | command | exec-sh.js:15:44:15:50 | command | +| exec-sh.js:13:17:13:23 | command | exec-sh.js:15:44:15:50 | command | +| exec-sh.js:19:9:19:49 | cmd | exec-sh.js:20:12:20:14 | cmd | +| exec-sh.js:19:15:19:38 | url.par ... , true) | exec-sh.js:19:15:19:44 | url.par ... ).query | +| exec-sh.js:19:15:19:44 | url.par ... ).query | exec-sh.js:19:15:19:49 | url.par ... ry.path | +| exec-sh.js:19:15:19:49 | url.par ... ry.path | exec-sh.js:19:9:19:49 | cmd | +| exec-sh.js:19:25:19:31 | req.url | exec-sh.js:19:15:19:38 | url.par ... , true) | +| exec-sh.js:19:25:19:31 | req.url | exec-sh.js:19:15:19:38 | url.par ... , true) | +| exec-sh.js:20:12:20:14 | cmd | exec-sh.js:13:17:13:23 | command | | execSeries.js:3:20:3:22 | arr | execSeries.js:6:14:6:16 | arr | | execSeries.js:6:14:6:16 | arr | execSeries.js:6:14:6:21 | arr[i++] | | execSeries.js:6:14:6:21 | arr[i++] | execSeries.js:14:24:14:30 | command | @@ -247,12 +279,10 @@ edges | child_process-test.js:22:18:22:20 | cmd | child_process-test.js:6:25:6:31 | req.url | child_process-test.js:22:18:22:20 | cmd | This command depends on $@. | child_process-test.js:6:25:6:31 | req.url | a user-provided value | | child_process-test.js:23:13:23:15 | cmd | child_process-test.js:6:25:6:31 | req.url | child_process-test.js:23:13:23:15 | cmd | This command depends on $@. | child_process-test.js:6:25:6:31 | req.url | a user-provided value | | child_process-test.js:25:13:25:31 | "foo" + cmd + "bar" | child_process-test.js:6:25:6:31 | req.url | child_process-test.js:25:13:25:31 | "foo" + cmd + "bar" | This command depends on $@. | child_process-test.js:6:25:6:31 | req.url | a user-provided value | -| child_process-test.js:39:5:39:31 | cp.spaw ... cmd ]) | child_process-test.js:6:25:6:31 | req.url | child_process-test.js:39:18:39:30 | [ flag, cmd ] | This command depends on $@. | child_process-test.js:6:25:6:31 | req.url | a user-provided value | | child_process-test.js:39:5:39:31 | cp.spaw ... cmd ]) | child_process-test.js:6:25:6:31 | req.url | child_process-test.js:39:26:39:28 | cmd | This command depends on $@. | child_process-test.js:6:25:6:31 | req.url | a user-provided value | | child_process-test.js:44:5:44:34 | cp.exec ... , args) | child_process-test.js:6:25:6:31 | req.url | child_process-test.js:43:15:43:17 | cmd | This command depends on $@. | child_process-test.js:6:25:6:31 | req.url | a user-provided value | | child_process-test.js:54:5:54:39 | cp.exec ... , args) | child_process-test.js:6:25:6:31 | req.url | child_process-test.js:53:15:53:17 | cmd | This command depends on $@. | child_process-test.js:6:25:6:31 | req.url | a user-provided value | | child_process-test.js:56:5:56:59 | cp.spaw ... cmd])) | child_process-test.js:6:25:6:31 | req.url | child_process-test.js:56:25:56:58 | ['/C', ... , cmd]) | This command depends on $@. | child_process-test.js:6:25:6:31 | req.url | a user-provided value | -| child_process-test.js:56:5:56:59 | cp.spaw ... cmd])) | child_process-test.js:6:25:6:31 | req.url | child_process-test.js:56:46:56:57 | ["bar", cmd] | This command depends on $@. | child_process-test.js:6:25:6:31 | req.url | a user-provided value | | child_process-test.js:56:5:56:59 | cp.spaw ... cmd])) | child_process-test.js:6:25:6:31 | req.url | child_process-test.js:56:54:56:56 | cmd | This command depends on $@. | child_process-test.js:6:25:6:31 | req.url | a user-provided value | | child_process-test.js:57:5:57:50 | cp.spaw ... t(cmd)) | child_process-test.js:6:25:6:31 | req.url | child_process-test.js:6:15:6:49 | url.par ... ry.path | This command depends on $@. | child_process-test.js:6:25:6:31 | req.url | a user-provided value | | child_process-test.js:57:5:57:50 | cp.spaw ... t(cmd)) | child_process-test.js:6:25:6:31 | req.url | child_process-test.js:57:25:57:49 | ['/C', ... at(cmd) | This command depends on $@. | child_process-test.js:6:25:6:31 | req.url | a user-provided value | @@ -260,6 +290,8 @@ edges | child_process-test.js:67:3:67:21 | cp.spawn(cmd, args) | child_process-test.js:6:25:6:31 | req.url | child_process-test.js:48:15:48:17 | cmd | This command depends on $@. | child_process-test.js:6:25:6:31 | req.url | a user-provided value | | child_process-test.js:75:29:75:31 | cmd | child_process-test.js:73:25:73:31 | req.url | child_process-test.js:75:29:75:31 | cmd | This command depends on $@. | child_process-test.js:73:25:73:31 | req.url | a user-provided value | | child_process-test.js:83:19:83:36 | req.query.fileName | child_process-test.js:83:19:83:36 | req.query.fileName | child_process-test.js:83:19:83:36 | req.query.fileName | This command depends on $@. | child_process-test.js:83:19:83:36 | req.query.fileName | a user-provided value | +| exec-sh2.js:10:12:10:57 | cp.spaw ... ptions) | exec-sh2.js:14:25:14:31 | req.url | exec-sh2.js:10:40:10:46 | command | This command depends on $@. | exec-sh2.js:14:25:14:31 | req.url | a user-provided value | +| exec-sh.js:15:12:15:61 | cp.spaw ... ptions) | exec-sh.js:19:25:19:31 | req.url | exec-sh.js:15:44:15:50 | command | This command depends on $@. | exec-sh.js:19:25:19:31 | req.url | a user-provided value | | execSeries.js:14:41:14:47 | command | execSeries.js:18:34:18:40 | req.url | execSeries.js:14:41:14:47 | command | This command depends on $@. | execSeries.js:18:34:18:40 | req.url | a user-provided value | | lib/subLib/index.js:8:10:8:25 | "rm -rf " + name | child_process-test.js:85:37:85:54 | req.query.fileName | lib/subLib/index.js:8:10:8:25 | "rm -rf " + name | This command depends on $@. | child_process-test.js:85:37:85:54 | req.query.fileName | a user-provided value | | other.js:7:33:7:35 | cmd | other.js:5:25:5:31 | req.url | other.js:7:33:7:35 | cmd | This command depends on $@. | other.js:5:25:5:31 | req.url | a user-provided value | diff --git a/javascript/ql/test/query-tests/Security/CWE-078/UselessUseOfCat.expected b/javascript/ql/test/query-tests/Security/CWE-078/UselessUseOfCat.expected index 720292c10c2..0480391fd82 100644 --- a/javascript/ql/test/query-tests/Security/CWE-078/UselessUseOfCat.expected +++ b/javascript/ql/test/query-tests/Security/CWE-078/UselessUseOfCat.expected @@ -94,6 +94,8 @@ options | child_process-test.js:56:5:56:59 | cp.spaw ... cmd])) | child_process-test.js:56:25:56:58 | ['/C', ... , cmd]) | | child_process-test.js:57:5:57:50 | cp.spaw ... t(cmd)) | child_process-test.js:57:25:57:49 | ['/C', ... at(cmd) | | child_process-test.js:67:3:67:21 | cp.spawn(cmd, args) | child_process-test.js:67:17:67:20 | args | +| exec-sh2.js:10:12:10:57 | cp.spaw ... ptions) | exec-sh2.js:10:50:10:56 | options | +| exec-sh.js:15:12:15:61 | cp.spaw ... ptions) | exec-sh.js:15:54:15:60 | options | | lib/lib.js:152:2:152:23 | cp.spaw ... gs, cb) | lib/lib.js:152:21:152:22 | cb | | lib/lib.js:159:2:159:23 | cp.spaw ... gs, cb) | lib/lib.js:159:21:159:22 | cb | | lib/lib.js:163:2:167:2 | cp.spaw ... t' }\\n\\t) | lib/lib.js:166:3:166:22 | { stdio: 'inherit' } | diff --git a/javascript/ql/test/query-tests/Security/CWE-078/exec-sh.js b/javascript/ql/test/query-tests/Security/CWE-078/exec-sh.js new file mode 100644 index 00000000000..b5b8fc602bd --- /dev/null +++ b/javascript/ql/test/query-tests/Security/CWE-078/exec-sh.js @@ -0,0 +1,21 @@ +const cp = require('child_process'), + http = require('http'), + url = require('url'); + +function getShell() { + if (process.platform === 'win32') { + return { cmd: 'cmd', arg: '/C' } + } else { + return { cmd: 'sh', arg: '-c' } + } +} + +function execSh(command, options) { + var shell = getShell() + return cp.spawn(shell.cmd, [shell.arg, command], options) // BAD +} + +http.createServer(function (req, res) { + let cmd = url.parse(req.url, true).query.path; + execSh(cmd); +}); diff --git a/javascript/ql/test/query-tests/Security/CWE-078/exec-sh2.js b/javascript/ql/test/query-tests/Security/CWE-078/exec-sh2.js new file mode 100644 index 00000000000..ad91b66f534 --- /dev/null +++ b/javascript/ql/test/query-tests/Security/CWE-078/exec-sh2.js @@ -0,0 +1,16 @@ +const cp = require('child_process'), + http = require('http'), + url = require('url'); + +function getShell() { + return "sh"; +} + +function execSh(command, options) { + return cp.spawn(getShell(), ["-c", command], options) // BAD +}; + +http.createServer(function (req, res) { + let cmd = url.parse(req.url, true).query.path; + execSh(cmd); +}); diff --git a/javascript/ql/test/query-tests/Security/CWE-094/CodeInjection/CodeInjection.expected b/javascript/ql/test/query-tests/Security/CWE-094/CodeInjection/CodeInjection.expected index 5b23332d205..dec023d72f3 100644 --- a/javascript/ql/test/query-tests/Security/CWE-094/CodeInjection/CodeInjection.expected +++ b/javascript/ql/test/query-tests/Security/CWE-094/CodeInjection/CodeInjection.expected @@ -8,6 +8,11 @@ nodes | NoSQLCodeInjection.js:19:36:19:43 | req.body | | NoSQLCodeInjection.js:19:36:19:43 | req.body | | NoSQLCodeInjection.js:19:36:19:48 | req.body.name | +| NoSQLCodeInjection.js:22:24:22:48 | "name = ... dy.name | +| NoSQLCodeInjection.js:22:24:22:48 | "name = ... dy.name | +| NoSQLCodeInjection.js:22:36:22:43 | req.body | +| NoSQLCodeInjection.js:22:36:22:43 | req.body | +| NoSQLCodeInjection.js:22:36:22:48 | req.body.name | | angularjs.js:10:22:10:29 | location | | angularjs.js:10:22:10:29 | location | | angularjs.js:10:22:10:36 | location.search | @@ -152,6 +157,10 @@ edges | NoSQLCodeInjection.js:19:36:19:43 | req.body | NoSQLCodeInjection.js:19:36:19:48 | req.body.name | | NoSQLCodeInjection.js:19:36:19:48 | req.body.name | NoSQLCodeInjection.js:19:24:19:48 | "name = ... dy.name | | NoSQLCodeInjection.js:19:36:19:48 | req.body.name | NoSQLCodeInjection.js:19:24:19:48 | "name = ... dy.name | +| NoSQLCodeInjection.js:22:36:22:43 | req.body | NoSQLCodeInjection.js:22:36:22:48 | req.body.name | +| NoSQLCodeInjection.js:22:36:22:43 | req.body | NoSQLCodeInjection.js:22:36:22:48 | req.body.name | +| NoSQLCodeInjection.js:22:36:22:48 | req.body.name | NoSQLCodeInjection.js:22:24:22:48 | "name = ... dy.name | +| NoSQLCodeInjection.js:22:36:22:48 | req.body.name | NoSQLCodeInjection.js:22:24:22:48 | "name = ... dy.name | | angularjs.js:10:22:10:29 | location | angularjs.js:10:22:10:36 | location.search | | angularjs.js:10:22:10:29 | location | angularjs.js:10:22:10:36 | location.search | | angularjs.js:10:22:10:29 | location | angularjs.js:10:22:10:36 | location.search | @@ -275,6 +284,7 @@ edges #select | NoSQLCodeInjection.js:18:24:18:37 | req.body.query | NoSQLCodeInjection.js:18:24:18:31 | req.body | NoSQLCodeInjection.js:18:24:18:37 | req.body.query | $@ flows to here and is interpreted as code. | NoSQLCodeInjection.js:18:24:18:31 | req.body | User-provided value | | NoSQLCodeInjection.js:19:24:19:48 | "name = ... dy.name | NoSQLCodeInjection.js:19:36:19:43 | req.body | NoSQLCodeInjection.js:19:24:19:48 | "name = ... dy.name | $@ flows to here and is interpreted as code. | NoSQLCodeInjection.js:19:36:19:43 | req.body | User-provided value | +| NoSQLCodeInjection.js:22:24:22:48 | "name = ... dy.name | NoSQLCodeInjection.js:22:36:22:43 | req.body | NoSQLCodeInjection.js:22:24:22:48 | "name = ... dy.name | $@ flows to here and is interpreted as code. | NoSQLCodeInjection.js:22:36:22:43 | req.body | User-provided value | | angularjs.js:10:22:10:36 | location.search | angularjs.js:10:22:10:29 | location | angularjs.js:10:22:10:36 | location.search | $@ flows to here and is interpreted as code. | angularjs.js:10:22:10:29 | location | User-provided value | | angularjs.js:13:23:13:37 | location.search | angularjs.js:13:23:13:30 | location | angularjs.js:13:23:13:37 | location.search | $@ flows to here and is interpreted as code. | angularjs.js:13:23:13:30 | location | User-provided value | | angularjs.js:16:28:16:42 | location.search | angularjs.js:16:28:16:35 | location | angularjs.js:16:28:16:42 | location.search | $@ flows to here and is interpreted as code. | angularjs.js:16:28:16:35 | location | User-provided value | diff --git a/javascript/ql/test/query-tests/Security/CWE-094/CodeInjection/HeuristicSourceCodeInjection.expected b/javascript/ql/test/query-tests/Security/CWE-094/CodeInjection/HeuristicSourceCodeInjection.expected index 50bf4e455a2..fa34f829b92 100644 --- a/javascript/ql/test/query-tests/Security/CWE-094/CodeInjection/HeuristicSourceCodeInjection.expected +++ b/javascript/ql/test/query-tests/Security/CWE-094/CodeInjection/HeuristicSourceCodeInjection.expected @@ -8,6 +8,11 @@ nodes | NoSQLCodeInjection.js:19:36:19:43 | req.body | | NoSQLCodeInjection.js:19:36:19:43 | req.body | | NoSQLCodeInjection.js:19:36:19:48 | req.body.name | +| NoSQLCodeInjection.js:22:24:22:48 | "name = ... dy.name | +| NoSQLCodeInjection.js:22:24:22:48 | "name = ... dy.name | +| NoSQLCodeInjection.js:22:36:22:43 | req.body | +| NoSQLCodeInjection.js:22:36:22:43 | req.body | +| NoSQLCodeInjection.js:22:36:22:48 | req.body.name | | angularjs.js:10:22:10:29 | location | | angularjs.js:10:22:10:29 | location | | angularjs.js:10:22:10:36 | location.search | @@ -156,6 +161,10 @@ edges | NoSQLCodeInjection.js:19:36:19:43 | req.body | NoSQLCodeInjection.js:19:36:19:48 | req.body.name | | NoSQLCodeInjection.js:19:36:19:48 | req.body.name | NoSQLCodeInjection.js:19:24:19:48 | "name = ... dy.name | | NoSQLCodeInjection.js:19:36:19:48 | req.body.name | NoSQLCodeInjection.js:19:24:19:48 | "name = ... dy.name | +| NoSQLCodeInjection.js:22:36:22:43 | req.body | NoSQLCodeInjection.js:22:36:22:48 | req.body.name | +| NoSQLCodeInjection.js:22:36:22:43 | req.body | NoSQLCodeInjection.js:22:36:22:48 | req.body.name | +| NoSQLCodeInjection.js:22:36:22:48 | req.body.name | NoSQLCodeInjection.js:22:24:22:48 | "name = ... dy.name | +| NoSQLCodeInjection.js:22:36:22:48 | req.body.name | NoSQLCodeInjection.js:22:24:22:48 | "name = ... dy.name | | angularjs.js:10:22:10:29 | location | angularjs.js:10:22:10:36 | location.search | | angularjs.js:10:22:10:29 | location | angularjs.js:10:22:10:36 | location.search | | angularjs.js:10:22:10:29 | location | angularjs.js:10:22:10:36 | location.search | diff --git a/javascript/ql/test/query-tests/Security/CWE-094/CodeInjection/NoSQLCodeInjection.js b/javascript/ql/test/query-tests/Security/CWE-094/CodeInjection/NoSQLCodeInjection.js index 3e7025a8e6c..6facf5ec75a 100644 --- a/javascript/ql/test/query-tests/Security/CWE-094/CodeInjection/NoSQLCodeInjection.js +++ b/javascript/ql/test/query-tests/Security/CWE-094/CodeInjection/NoSQLCodeInjection.js @@ -17,5 +17,11 @@ app.post("/documents/find", (req, res) => { doc.find(query); // NOT OK, but that is flagged by js/sql-injection [INCONSISTENCY] doc.find({ $where: req.body.query }); // NOT OK doc.find({ $where: "name = " + req.body.name }); // NOT OK + + function mkWhereObj() { + return { $where: "name = " + req.body.name }; // NOT OK + } + + doc.find(mkWhereObj()); // the alert location is in mkWhereObj. }); }); diff --git a/javascript/ql/test/query-tests/Security/CWE-352/MissingCsrfMiddleware.expected b/javascript/ql/test/query-tests/Security/CWE-352/MissingCsrfMiddleware.expected index 6cf61160de1..bc6087f4354 100644 --- a/javascript/ql/test/query-tests/Security/CWE-352/MissingCsrfMiddleware.expected +++ b/javascript/ql/test/query-tests/Security/CWE-352/MissingCsrfMiddleware.expected @@ -1,6 +1,7 @@ | MissingCsrfMiddlewareBad.js:7:9:7:22 | cookieParser() | This cookie middleware is serving a request handler $@ without CSRF protection. | MissingCsrfMiddlewareBad.js:10:26:12:1 | functio ... il"];\\n} | here | | MissingCsrfMiddlewareBad.js:17:13:17:26 | cookieParser() | This cookie middleware is serving a request handler $@ without CSRF protection. | MissingCsrfMiddlewareBad.js:25:30:27:6 | errorCa ... \\n }) | here | | MissingCsrfMiddlewareBad.js:33:13:33:26 | cookieParser() | This cookie middleware is serving a request handler $@ without CSRF protection. | MissingCsrfMiddlewareBad.js:41:30:43:6 | errorCa ... \\n }) | here | +| MissingCsrfMiddlewareBad.js:33:13:33:26 | cookieParser() | This cookie middleware is serving a request handler $@ without CSRF protection. | MissingCsrfMiddlewareBad.js:45:31:47:6 | errorCa ... \\n }) | here | | csurf_api_example.js:42:37:42:50 | cookieParser() | This cookie middleware is serving a request handler $@ without CSRF protection. | csurf_api_example.js:42:53:45:3 | functio ... e')\\n } | here | | csurf_example.js:18:9:18:22 | cookieParser() | This cookie middleware is serving a request handler $@ without CSRF protection. | csurf_example.js:31:40:34:1 | functio ... sed')\\n} | here | | lusca_example.js:9:9:9:22 | cookieParser() | This cookie middleware is serving a request handler $@ without CSRF protection. | lusca_example.js:26:42:29:1 | functio ... sed')\\n} | here | diff --git a/javascript/ql/test/query-tests/Security/CWE-352/MissingCsrfMiddlewareBad.js b/javascript/ql/test/query-tests/Security/CWE-352/MissingCsrfMiddlewareBad.js index fdcaf739451..d2d9b8bd4a1 100644 --- a/javascript/ql/test/query-tests/Security/CWE-352/MissingCsrfMiddlewareBad.js +++ b/javascript/ql/test/query-tests/Security/CWE-352/MissingCsrfMiddlewareBad.js @@ -41,4 +41,8 @@ app.post('/changeEmail', function (req, res) { app.post('/changeEmail', errorCatch(async function (req, res) { let newEmail = req.cookies["newEmail"]; })); + + app.post('/doLoginStuff', errorCatch(async function (req, res) { + req.session.user = loginStuff(req); + })); }) diff --git a/javascript/ql/test/query-tests/Security/CWE-601/ClientSideUrlRedirect/ClientSideUrlRedirect.expected b/javascript/ql/test/query-tests/Security/CWE-601/ClientSideUrlRedirect/ClientSideUrlRedirect.expected index b7dca317a88..b5621666d73 100644 --- a/javascript/ql/test/query-tests/Security/CWE-601/ClientSideUrlRedirect/ClientSideUrlRedirect.expected +++ b/javascript/ql/test/query-tests/Security/CWE-601/ClientSideUrlRedirect/ClientSideUrlRedirect.expected @@ -133,6 +133,30 @@ nodes | tst.js:6:34:6:50 | document.location | | tst.js:6:34:6:50 | document.location | | tst.js:6:34:6:55 | documen ... on.href | +| tst.js:10:19:10:81 | new Reg ... n.href) | +| tst.js:10:19:10:84 | new Reg ... ref)[1] | +| tst.js:10:19:10:84 | new Reg ... ref)[1] | +| tst.js:10:59:10:75 | document.location | +| tst.js:10:59:10:75 | document.location | +| tst.js:10:59:10:80 | documen ... on.href | +| tst.js:14:20:14:56 | indirec ... n.href) | +| tst.js:14:20:14:59 | indirec ... ref)[1] | +| tst.js:14:20:14:59 | indirec ... ref)[1] | +| tst.js:14:34:14:50 | document.location | +| tst.js:14:34:14:50 | document.location | +| tst.js:14:34:14:55 | documen ... on.href | +| tst.js:18:19:18:81 | new Reg ... n.href) | +| tst.js:18:19:18:84 | new Reg ... ref)[1] | +| tst.js:18:19:18:84 | new Reg ... ref)[1] | +| tst.js:18:59:18:75 | document.location | +| tst.js:18:59:18:75 | document.location | +| tst.js:18:59:18:80 | documen ... on.href | +| tst.js:22:20:22:56 | indirec ... n.href) | +| tst.js:22:20:22:59 | indirec ... ref)[1] | +| tst.js:22:20:22:59 | indirec ... ref)[1] | +| tst.js:22:34:22:50 | document.location | +| tst.js:22:34:22:50 | document.location | +| tst.js:22:34:22:55 | documen ... on.href | edges | sanitizer.js:2:9:2:25 | url | sanitizer.js:4:27:4:29 | url | | sanitizer.js:2:9:2:25 | url | sanitizer.js:4:27:4:29 | url | @@ -260,6 +284,26 @@ edges | tst.js:6:34:6:50 | document.location | tst.js:6:34:6:55 | documen ... on.href | | tst.js:6:34:6:50 | document.location | tst.js:6:34:6:55 | documen ... on.href | | tst.js:6:34:6:55 | documen ... on.href | tst.js:6:20:6:56 | indirec ... n.href) | +| tst.js:10:19:10:81 | new Reg ... n.href) | tst.js:10:19:10:84 | new Reg ... ref)[1] | +| tst.js:10:19:10:81 | new Reg ... n.href) | tst.js:10:19:10:84 | new Reg ... ref)[1] | +| tst.js:10:59:10:75 | document.location | tst.js:10:59:10:80 | documen ... on.href | +| tst.js:10:59:10:75 | document.location | tst.js:10:59:10:80 | documen ... on.href | +| tst.js:10:59:10:80 | documen ... on.href | tst.js:10:19:10:81 | new Reg ... n.href) | +| tst.js:14:20:14:56 | indirec ... n.href) | tst.js:14:20:14:59 | indirec ... ref)[1] | +| tst.js:14:20:14:56 | indirec ... n.href) | tst.js:14:20:14:59 | indirec ... ref)[1] | +| tst.js:14:34:14:50 | document.location | tst.js:14:34:14:55 | documen ... on.href | +| tst.js:14:34:14:50 | document.location | tst.js:14:34:14:55 | documen ... on.href | +| tst.js:14:34:14:55 | documen ... on.href | tst.js:14:20:14:56 | indirec ... n.href) | +| tst.js:18:19:18:81 | new Reg ... n.href) | tst.js:18:19:18:84 | new Reg ... ref)[1] | +| tst.js:18:19:18:81 | new Reg ... n.href) | tst.js:18:19:18:84 | new Reg ... ref)[1] | +| tst.js:18:59:18:75 | document.location | tst.js:18:59:18:80 | documen ... on.href | +| tst.js:18:59:18:75 | document.location | tst.js:18:59:18:80 | documen ... on.href | +| tst.js:18:59:18:80 | documen ... on.href | tst.js:18:19:18:81 | new Reg ... n.href) | +| tst.js:22:20:22:56 | indirec ... n.href) | tst.js:22:20:22:59 | indirec ... ref)[1] | +| tst.js:22:20:22:56 | indirec ... n.href) | tst.js:22:20:22:59 | indirec ... ref)[1] | +| tst.js:22:34:22:50 | document.location | tst.js:22:34:22:55 | documen ... on.href | +| tst.js:22:34:22:50 | document.location | tst.js:22:34:22:55 | documen ... on.href | +| tst.js:22:34:22:55 | documen ... on.href | tst.js:22:20:22:56 | indirec ... n.href) | #select | sanitizer.js:4:27:4:29 | url | sanitizer.js:2:15:2:25 | window.name | sanitizer.js:4:27:4:29 | url | Untrusted URL redirection due to $@. | sanitizer.js:2:15:2:25 | window.name | user-provided value | | sanitizer.js:16:27:16:29 | url | sanitizer.js:2:15:2:25 | window.name | sanitizer.js:16:27:16:29 | url | Untrusted URL redirection due to $@. | sanitizer.js:2:15:2:25 | window.name | user-provided value | @@ -296,3 +340,7 @@ edges | tst13.js:53:28:53:28 | e | tst13.js:52:34:52:34 | e | tst13.js:53:28:53:28 | e | Untrusted URL redirection due to $@. | tst13.js:52:34:52:34 | e | user-provided value | | tst.js:2:19:2:72 | /.*redi ... ref)[1] | tst.js:2:47:2:63 | document.location | tst.js:2:19:2:72 | /.*redi ... ref)[1] | Untrusted URL redirection due to $@. | tst.js:2:47:2:63 | document.location | user-provided value | | tst.js:6:20:6:59 | indirec ... ref)[1] | tst.js:6:34:6:50 | document.location | tst.js:6:20:6:59 | indirec ... ref)[1] | Untrusted URL redirection due to $@. | tst.js:6:34:6:50 | document.location | user-provided value | +| tst.js:10:19:10:84 | new Reg ... ref)[1] | tst.js:10:59:10:75 | document.location | tst.js:10:19:10:84 | new Reg ... ref)[1] | Untrusted URL redirection due to $@. | tst.js:10:59:10:75 | document.location | user-provided value | +| tst.js:14:20:14:59 | indirec ... ref)[1] | tst.js:14:34:14:50 | document.location | tst.js:14:20:14:59 | indirec ... ref)[1] | Untrusted URL redirection due to $@. | tst.js:14:34:14:50 | document.location | user-provided value | +| tst.js:18:19:18:84 | new Reg ... ref)[1] | tst.js:18:59:18:75 | document.location | tst.js:18:19:18:84 | new Reg ... ref)[1] | Untrusted URL redirection due to $@. | tst.js:18:59:18:75 | document.location | user-provided value | +| tst.js:22:20:22:59 | indirec ... ref)[1] | tst.js:22:34:22:50 | document.location | tst.js:22:20:22:59 | indirec ... ref)[1] | Untrusted URL redirection due to $@. | tst.js:22:34:22:50 | document.location | user-provided value | diff --git a/javascript/ql/test/query-tests/Security/CWE-601/ClientSideUrlRedirect/tst.js b/javascript/ql/test/query-tests/Security/CWE-601/ClientSideUrlRedirect/tst.js index a77903120ab..7994c1e3558 100644 --- a/javascript/ql/test/query-tests/Security/CWE-601/ClientSideUrlRedirect/tst.js +++ b/javascript/ql/test/query-tests/Security/CWE-601/ClientSideUrlRedirect/tst.js @@ -5,3 +5,19 @@ window.location = /.*redirect=([^&]*).*/.exec(document.location.href)[1]; var indirect = /.*redirect=([^&]*).*/; window.location = indirect.exec(document.location.href)[1]; }); + +// NOT OK +window.location = new RegExp('.*redirect=([^&]*).*').exec(document.location.href)[1]; + +(function(){ + var indirect = new RegExp('.*redirect=([^&]*).*') + window.location = indirect.exec(document.location.href)[1]; +}); + +// NOT OK +window.location = new RegExp(/.*redirect=([^&]*).*/).exec(document.location.href)[1]; + +(function(){ + var indirect = new RegExp(/.*redirect=([^&]*).*/) + window.location = indirect.exec(document.location.href)[1]; +}); diff --git a/javascript/ql/test/query-tests/Security/CWE-601/ServerSideUrlRedirect/ServerSideUrlRedirect.expected b/javascript/ql/test/query-tests/Security/CWE-601/ServerSideUrlRedirect/ServerSideUrlRedirect.expected index 299c434f329..5a8ed33b530 100644 --- a/javascript/ql/test/query-tests/Security/CWE-601/ServerSideUrlRedirect/ServerSideUrlRedirect.expected +++ b/javascript/ql/test/query-tests/Security/CWE-601/ServerSideUrlRedirect/ServerSideUrlRedirect.expected @@ -44,6 +44,12 @@ nodes | express.js:136:16:136:36 | 'u' + r ... ms.user | | express.js:136:22:136:36 | req.params.user | | express.js:136:22:136:36 | req.params.user | +| express.js:143:16:143:28 | req.query.foo | +| express.js:143:16:143:28 | req.query.foo | +| express.js:143:16:143:28 | req.query.foo | +| express.js:146:16:146:24 | query.foo | +| express.js:146:16:146:24 | query.foo | +| express.js:146:16:146:24 | query.foo | | koa.js:6:6:6:27 | url | | koa.js:6:12:6:27 | ctx.query.target | | koa.js:6:12:6:27 | ctx.query.target | @@ -128,6 +134,8 @@ edges | express.js:136:22:136:36 | req.params.user | express.js:136:16:136:36 | 'u' + r ... ms.user | | express.js:136:22:136:36 | req.params.user | express.js:136:16:136:36 | 'u' + r ... ms.user | | express.js:136:22:136:36 | req.params.user | express.js:136:16:136:36 | 'u' + r ... ms.user | +| express.js:143:16:143:28 | req.query.foo | express.js:143:16:143:28 | req.query.foo | +| express.js:146:16:146:24 | query.foo | express.js:146:16:146:24 | query.foo | | koa.js:6:6:6:27 | url | koa.js:7:15:7:17 | url | | koa.js:6:6:6:27 | url | koa.js:7:15:7:17 | url | | koa.js:6:6:6:27 | url | koa.js:8:18:8:20 | url | @@ -181,6 +189,8 @@ edges | express.js:134:16:134:36 | '/' + r ... ms.user | express.js:134:22:134:36 | req.params.user | express.js:134:16:134:36 | '/' + r ... ms.user | Untrusted URL redirection due to $@. | express.js:134:22:134:36 | req.params.user | user-provided value | | express.js:135:16:135:37 | '//' + ... ms.user | express.js:135:23:135:37 | req.params.user | express.js:135:16:135:37 | '//' + ... ms.user | Untrusted URL redirection due to $@. | express.js:135:23:135:37 | req.params.user | user-provided value | | express.js:136:16:136:36 | 'u' + r ... ms.user | express.js:136:22:136:36 | req.params.user | express.js:136:16:136:36 | 'u' + r ... ms.user | Untrusted URL redirection due to $@. | express.js:136:22:136:36 | req.params.user | user-provided value | +| express.js:143:16:143:28 | req.query.foo | express.js:143:16:143:28 | req.query.foo | express.js:143:16:143:28 | req.query.foo | Untrusted URL redirection due to $@. | express.js:143:16:143:28 | req.query.foo | user-provided value | +| express.js:146:16:146:24 | query.foo | express.js:146:16:146:24 | query.foo | express.js:146:16:146:24 | query.foo | Untrusted URL redirection due to $@. | express.js:146:16:146:24 | query.foo | user-provided value | | koa.js:7:15:7:17 | url | koa.js:6:12:6:27 | ctx.query.target | koa.js:7:15:7:17 | url | Untrusted URL redirection due to $@. | koa.js:6:12:6:27 | ctx.query.target | user-provided value | | koa.js:8:15:8:26 | `${url}${x}` | koa.js:6:12:6:27 | ctx.query.target | koa.js:8:15:8:26 | `${url}${x}` | Untrusted URL redirection due to $@. | koa.js:6:12:6:27 | ctx.query.target | user-provided value | | koa.js:14:16:14:18 | url | koa.js:6:12:6:27 | ctx.query.target | koa.js:14:16:14:18 | url | Untrusted URL redirection due to $@. | koa.js:6:12:6:27 | ctx.query.target | user-provided value | diff --git a/javascript/ql/test/query-tests/Security/CWE-601/ServerSideUrlRedirect/express.js b/javascript/ql/test/query-tests/Security/CWE-601/ServerSideUrlRedirect/express.js index 4c7c476a060..b319315b985 100644 --- a/javascript/ql/test/query-tests/Security/CWE-601/ServerSideUrlRedirect/express.js +++ b/javascript/ql/test/query-tests/Security/CWE-601/ServerSideUrlRedirect/express.js @@ -138,3 +138,10 @@ app.get('/redirect/:user', function(req, res) { res.redirect('/' + ('/u' + req.params.user)); // BAD - could go to //u.evil.com, but not flagged [INCONSISTENCY] res.redirect('/u' + req.params.user); // GOOD }); + +app.get("foo", (req, res) => { + res.redirect(req.query.foo); // NOT OK +}); +app.get("bar", ({query}, res) => { + res.redirect(query.foo); // NOT OK +}) \ No newline at end of file diff --git a/javascript/ql/test/query-tests/Security/CWE-611/Xxe.expected b/javascript/ql/test/query-tests/Security/CWE-611/Xxe.expected index fd44e6cc756..31bfc72e291 100644 --- a/javascript/ql/test/query-tests/Security/CWE-611/Xxe.expected +++ b/javascript/ql/test/query-tests/Security/CWE-611/Xxe.expected @@ -10,6 +10,18 @@ nodes | libxml.noent.js:6:21:6:41 | req.par ... e-xml") | | libxml.noent.js:6:21:6:41 | req.par ... e-xml") | | libxml.noent.js:6:21:6:41 | req.par ... e-xml") | +| libxml.noent.js:11:21:11:41 | req.par ... e-xml") | +| libxml.noent.js:11:21:11:41 | req.par ... e-xml") | +| libxml.noent.js:11:21:11:41 | req.par ... e-xml") | +| libxml.noent.js:14:27:14:47 | req.par ... e-xml") | +| libxml.noent.js:14:27:14:47 | req.par ... e-xml") | +| libxml.noent.js:14:27:14:47 | req.par ... e-xml") | +| libxml.noent.js:16:27:16:35 | req.files | +| libxml.noent.js:16:27:16:35 | req.files | +| libxml.noent.js:16:27:16:44 | req.files.products | +| libxml.noent.js:16:27:16:49 | req.fil ... ts.data | +| libxml.noent.js:16:27:16:66 | req.fil ... 'utf8') | +| libxml.noent.js:16:27:16:66 | req.fil ... 'utf8') | | libxml.sax.js:6:22:6:42 | req.par ... e-xml") | | libxml.sax.js:6:22:6:42 | req.par ... e-xml") | | libxml.sax.js:6:22:6:42 | req.par ... e-xml") | @@ -25,11 +37,21 @@ edges | domparser.js:2:13:2:29 | document.location | domparser.js:2:13:2:36 | documen ... .search | | domparser.js:2:13:2:36 | documen ... .search | domparser.js:2:7:2:36 | src | | libxml.noent.js:6:21:6:41 | req.par ... e-xml") | libxml.noent.js:6:21:6:41 | req.par ... e-xml") | +| libxml.noent.js:11:21:11:41 | req.par ... e-xml") | libxml.noent.js:11:21:11:41 | req.par ... e-xml") | +| libxml.noent.js:14:27:14:47 | req.par ... e-xml") | libxml.noent.js:14:27:14:47 | req.par ... e-xml") | +| libxml.noent.js:16:27:16:35 | req.files | libxml.noent.js:16:27:16:44 | req.files.products | +| libxml.noent.js:16:27:16:35 | req.files | libxml.noent.js:16:27:16:44 | req.files.products | +| libxml.noent.js:16:27:16:44 | req.files.products | libxml.noent.js:16:27:16:49 | req.fil ... ts.data | +| libxml.noent.js:16:27:16:49 | req.fil ... ts.data | libxml.noent.js:16:27:16:66 | req.fil ... 'utf8') | +| libxml.noent.js:16:27:16:49 | req.fil ... ts.data | libxml.noent.js:16:27:16:66 | req.fil ... 'utf8') | | libxml.sax.js:6:22:6:42 | req.par ... e-xml") | libxml.sax.js:6:22:6:42 | req.par ... e-xml") | | libxml.saxpush.js:6:15:6:35 | req.par ... e-xml") | libxml.saxpush.js:6:15:6:35 | req.par ... e-xml") | #select | domparser.js:11:55:11:57 | src | domparser.js:2:13:2:29 | document.location | domparser.js:11:55:11:57 | src | A $@ is parsed as XML without guarding against external entity expansion. | domparser.js:2:13:2:29 | document.location | user-provided value | | domparser.js:14:57:14:59 | src | domparser.js:2:13:2:29 | document.location | domparser.js:14:57:14:59 | src | A $@ is parsed as XML without guarding against external entity expansion. | domparser.js:2:13:2:29 | document.location | user-provided value | | libxml.noent.js:6:21:6:41 | req.par ... e-xml") | libxml.noent.js:6:21:6:41 | req.par ... e-xml") | libxml.noent.js:6:21:6:41 | req.par ... e-xml") | A $@ is parsed as XML without guarding against external entity expansion. | libxml.noent.js:6:21:6:41 | req.par ... e-xml") | user-provided value | +| libxml.noent.js:11:21:11:41 | req.par ... e-xml") | libxml.noent.js:11:21:11:41 | req.par ... e-xml") | libxml.noent.js:11:21:11:41 | req.par ... e-xml") | A $@ is parsed as XML without guarding against external entity expansion. | libxml.noent.js:11:21:11:41 | req.par ... e-xml") | user-provided value | +| libxml.noent.js:14:27:14:47 | req.par ... e-xml") | libxml.noent.js:14:27:14:47 | req.par ... e-xml") | libxml.noent.js:14:27:14:47 | req.par ... e-xml") | A $@ is parsed as XML without guarding against external entity expansion. | libxml.noent.js:14:27:14:47 | req.par ... e-xml") | user-provided value | +| libxml.noent.js:16:27:16:66 | req.fil ... 'utf8') | libxml.noent.js:16:27:16:35 | req.files | libxml.noent.js:16:27:16:66 | req.fil ... 'utf8') | A $@ is parsed as XML without guarding against external entity expansion. | libxml.noent.js:16:27:16:35 | req.files | user-provided value | | libxml.sax.js:6:22:6:42 | req.par ... e-xml") | libxml.sax.js:6:22:6:42 | req.par ... e-xml") | libxml.sax.js:6:22:6:42 | req.par ... e-xml") | A $@ is parsed as XML without guarding against external entity expansion. | libxml.sax.js:6:22:6:42 | req.par ... e-xml") | user-provided value | | libxml.saxpush.js:6:15:6:35 | req.par ... e-xml") | libxml.saxpush.js:6:15:6:35 | req.par ... e-xml") | libxml.saxpush.js:6:15:6:35 | req.par ... e-xml") | A $@ is parsed as XML without guarding against external entity expansion. | libxml.saxpush.js:6:15:6:35 | req.par ... e-xml") | user-provided value | diff --git a/javascript/ql/test/query-tests/Security/CWE-611/libxml.noent.js b/javascript/ql/test/query-tests/Security/CWE-611/libxml.noent.js index b55e8c9c413..47df4223e0e 100644 --- a/javascript/ql/test/query-tests/Security/CWE-611/libxml.noent.js +++ b/javascript/ql/test/query-tests/Security/CWE-611/libxml.noent.js @@ -5,3 +5,16 @@ express().get('/some/path', function(req) { // NOT OK: unguarded entity expansion libxmljs.parseXml(req.param("some-xml"), { noent: true }); }); + +express().post('/some/path', function(req, res) { + // NOT OK: unguarded entity expansion + libxmljs.parseXml(req.param("some-xml"), { noent: true }); + + // NOT OK: unguarded entity expansion + libxmljs.parseXmlString(req.param("some-xml"), {noent:true}) + // NOT OK: unguarded entity expansion + libxmljs.parseXmlString(req.files.products.data.toString('utf8'), {noent:true}) + + // OK - no entity expansion + libxmljs.parseXmlString(req.files.products.data.toString('utf8'), {noent:false}) +}); diff --git a/python/ql/src/Security/CWE-022/PathInjection.qhelp b/python/ql/src/Security/CWE-022/PathInjection.qhelp index 0abc683b4ca..200a4e78f98 100644 --- a/python/ql/src/Security/CWE-022/PathInjection.qhelp +++ b/python/ql/src/Security/CWE-022/PathInjection.qhelp @@ -55,7 +55,7 @@ known prefix. This ensures that regardless of the user input, the resulting path -
  • OWASP: Path Traversal.
    • +
  • OWASP: Path Traversal.
  • npm: werkzeug.utils.secure_filename.
    •  diff --git a/python/ql/src/Security/CWE-022/TarSlip.qhelp b/python/ql/src/Security/CWE-022/TarSlip.qhelp index a2fbc5dfdc6..07bab31a6ce 100644 --- a/python/ql/src/Security/CWE-022/TarSlip.qhelp +++ b/python/ql/src/Security/CWE-022/TarSlip.qhelp @@ -60,7 +60,7 @@ Snyk:
  • OWASP: -Path Traversal. +Path Traversal.
  • Python Library Reference: diff --git a/python/ql/src/Security/CWE-209/StackTraceExposure.qhelp b/python/ql/src/Security/CWE-209/StackTraceExposure.qhelp index 86ecdbdc0d8..6670aa08284 100644 --- a/python/ql/src/Security/CWE-209/StackTraceExposure.qhelp +++ b/python/ql/src/Security/CWE-209/StackTraceExposure.qhelp @@ -47,6 +47,6 @@ log, but remote users will not see the information. -
  • OWASP: Information Leak.
    • +
  • OWASP: Improper Error Handling.
    • diff --git a/python/ql/src/Security/CWE-327/BrokenCryptoAlgorithm.qhelp b/python/ql/src/Security/CWE-327/BrokenCryptoAlgorithm.qhelp index 1b4031b1cc5..ba5ab4d10c1 100644 --- a/python/ql/src/Security/CWE-327/BrokenCryptoAlgorithm.qhelp +++ b/python/ql/src/Security/CWE-327/BrokenCryptoAlgorithm.qhelp @@ -33,7 +33,7 @@ pycrypto you must specify the encryption algorithm to use. The first example uses DES, which is an older algorithm that is now considered weak. The second - example uses Blowfish, which is a stronger more modern algorithm. + example uses AES, which is a stronger modern algorithm.

    diff --git a/python/ql/src/Security/CWE-327/examples/broken_crypto.py b/python/ql/src/Security/CWE-327/examples/broken_crypto.py index ef9fc75e889..6e229339cdf 100644 --- a/python/ql/src/Security/CWE-327/examples/broken_crypto.py +++ b/python/ql/src/Security/CWE-327/examples/broken_crypto.py @@ -1,4 +1,4 @@ -from Crypto.Cipher import DES, Blowfish +from Crypto.Cipher import DES, AES cipher = DES.new(SECRET_KEY) @@ -6,7 +6,7 @@ def send_encrypted(channel, message): channel.send(cipher.encrypt(message)) # BAD: weak encryption -cipher = Blowfish.new(SECRET_KEY) +cipher = AES.new(SECRET_KEY) def send_encrypted(channel, message): channel.send(cipher.encrypt(message)) # GOOD: strong encryption diff --git a/python/ql/src/experimental/Security-new-dataflow/CWE-078/CommandInjection.ql b/python/ql/src/experimental/Security-new-dataflow/CWE-078/CommandInjection.ql new file mode 100755 index 00000000000..b1672ad995a --- /dev/null +++ b/python/ql/src/experimental/Security-new-dataflow/CWE-078/CommandInjection.ql @@ -0,0 +1,64 @@ +/** + * @name Uncontrolled command line + * @description Using externally controlled strings in a command line may allow a malicious + * user to change the meaning of the command. + * @kind path-problem + * @problem.severity error + * @sub-severity high + * @precision high + * @id py/command-line-injection + * @tags correctness + * security + * external/owasp/owasp-a1 + * external/cwe/cwe-078 + * external/cwe/cwe-088 + */ + +import python +import experimental.dataflow.DataFlow +import experimental.dataflow.TaintTracking +import experimental.semmle.python.Concepts +import experimental.dataflow.RemoteFlowSources +import DataFlow::PathGraph + +class CommandInjectionConfiguration extends TaintTracking::Configuration { + CommandInjectionConfiguration() { this = "CommandInjectionConfiguration" } + + override predicate isSource(DataFlow::Node source) { source instanceof RemoteFlowSource } + + override predicate isSink(DataFlow::Node sink) { + sink = any(SystemCommandExecution e).getCommand() and + // Since the implementation of standard library functions such `os.popen` looks like + // ```py + // def popen(cmd, mode="r", buffering=-1): + // ... + // proc = subprocess.Popen(cmd, ...) + // ``` + // any time we would report flow to the `os.popen` sink, we can ALSO report the flow + // from the `cmd` parameter to the `subprocess.Popen` sink -- obviously we don't + // want that. + // + // However, simply removing taint edges out of a sink is not a good enough solution, + // since we would only flag one of the `os.system` calls in the following example + // due to use-use flow + // ```py + // os.system(cmd) + // os.system(cmd) + // ``` + // + // Best solution I could come up with is to exclude all sinks inside the `os` and + // `subprocess` modules. This does have a downside: If we have overlooked a function + // in any of these, that internally runs a command, we no longer give an alert :| + // + // This does not only affect `os.popen`, but also the helper functions in + // `subprocess`. See: + // https://github.com/python/cpython/blob/fa7ce080175f65d678a7d5756c94f82887fc9803/Lib/os.py#L974 + // https://github.com/python/cpython/blob/fa7ce080175f65d678a7d5756c94f82887fc9803/Lib/subprocess.py#L341 + not sink.getScope().getEnclosingModule().getName() in ["os", "subprocess"] + } +} + +from CommandInjectionConfiguration config, DataFlow::PathNode source, DataFlow::PathNode sink +where config.hasFlowPath(source, sink) +select sink.getNode(), source, sink, "This command depends on $@.", source.getNode(), + "a user-provided value" diff --git a/python/ql/src/experimental/dataflow/TypeTracker.qll b/python/ql/src/experimental/dataflow/TypeTracker.qll index 40b45359f97..4491279a971 100644 --- a/python/ql/src/experimental/dataflow/TypeTracker.qll +++ b/python/ql/src/experimental/dataflow/TypeTracker.qll @@ -154,7 +154,7 @@ private newtype TTypeTracker = MkTypeTracker(Boolean hasCall, OptionalAttributeN * t.start() and * result = < source of myType > * or - * exists (TypeTracker t2 | + * exists (DataFlow::TypeTracker t2 | * result = myType(t2).track(t2, t) * ) * } diff --git a/python/ql/src/experimental/dataflow/internal/DataFlowPrivate.qll b/python/ql/src/experimental/dataflow/internal/DataFlowPrivate.qll index bc904274524..fcc98e9d8ce 100644 --- a/python/ql/src/experimental/dataflow/internal/DataFlowPrivate.qll +++ b/python/ql/src/experimental/dataflow/internal/DataFlowPrivate.qll @@ -16,58 +16,101 @@ class DataFlowCfgNode extends ControlFlowNode { DataFlowCfgNode() { isExpressionNode(this) } } -/** A data flow node which should have an associated post-update node. */ -abstract class PreUpdateNode extends Node { } +/** A data flow node for which we should synthesise an associated pre-update node. */ +abstract class NeedsSyntheticPreUpdateNode extends Node { + /** A label for this kind of node. This will figure in the textual representation of the synthesized pre-update node. */ + abstract string label(); +} + +class SyntheticPreUpdateNode extends Node, TSyntheticPreUpdateNode { + NeedsSyntheticPreUpdateNode post; + + SyntheticPreUpdateNode() { this = TSyntheticPreUpdateNode(post) } + + /** Gets the node for which this is a synthetic pre-update node. */ + Node getPostUpdateNode() { result = post } + + override string toString() { result = "[pre " + post.label() + "] " + post.toString() } + + override Scope getScope() { result = post.getScope() } + + override Location getLocation() { result = post.getLocation() } +} + +/** A data flow node for which we should synthesise an associated post-update node. */ +abstract class NeedsSyntheticPostUpdateNode extends Node { + /** A label for this kind of node. This will figure in the textual representation of the synthesized post-update node. */ + abstract string label(); +} /** An argument might have its value changed as a result of a call. */ -class ArgumentPreUpdateNode extends PreUpdateNode, ArgumentNode { } +class ArgumentPreUpdateNode extends NeedsSyntheticPostUpdateNode, ArgumentNode { + // Certain arguments, such as implicit self arguments are already post-update nodes + // and should not have an extra node synthesised. + ArgumentPreUpdateNode() { + this = any(CallNodeCall c).getArg(_) + or + this = any(SpecialCall c).getArg(_) + or + // Avoid argument 0 of class calls as those have non-synthetic post-update nodes. + exists(ClassCall c, int n | n > 0 | this = c.getArg(n)) + } + + override string label() { result = "arg" } +} /** An object might have its value changed after a store. */ -class StorePreUpdateNode extends PreUpdateNode, CfgNode { +class StorePreUpdateNode extends NeedsSyntheticPostUpdateNode, CfgNode { StorePreUpdateNode() { exists(Attribute a | node = a.getObject().getAFlowNode() and a.getCtx() instanceof Store ) } + + override string label() { result = "store" } } /** A node marking the state change of an object after a read. */ -class ReadPreUpdateNode extends PreUpdateNode, CfgNode { +class ReadPreUpdateNode extends NeedsSyntheticPostUpdateNode, CfgNode { ReadPreUpdateNode() { exists(Attribute a | node = a.getObject().getAFlowNode() and a.getCtx() instanceof Load ) } + + override string label() { result = "read" } } -/** - * A node associated with an object after an operation that might have - * changed its state. - * - * This can be either the argument to a callable after the callable returns - * (which might have mutated the argument), or the qualifier of a field after - * an update to the field. - * - * Nodes corresponding to AST elements, for example `ExprNode`, usually refer - * to the value before the update. - */ -class PostUpdateNode extends Node, TPostUpdateNode { - PreUpdateNode pre; +/** A post-update node is synthesized for all nodes which satisfy `NeedsSyntheticPostUpdateNode`. */ +class SyntheticPostUpdateNode extends PostUpdateNode, TSyntheticPostUpdateNode { + NeedsSyntheticPostUpdateNode pre; - PostUpdateNode() { this = TPostUpdateNode(pre) } + SyntheticPostUpdateNode() { this = TSyntheticPostUpdateNode(pre) } - /** Gets the node before the state update. */ - Node getPreUpdateNode() { result = pre } + override Node getPreUpdateNode() { result = pre } - override string toString() { result = "[post] " + pre.toString() } + override string toString() { result = "[post " + pre.label() + "] " + pre.toString() } override Scope getScope() { result = pre.getScope() } override Location getLocation() { result = pre.getLocation() } } +/** + * Calls to constructors are treated as post-update nodes for the synthesized argument + * that is mapped to the `self` parameter. That way, constructor calls represent the value of the + * object after the constructor (currently only `__init__`) has run. + */ +class ObjectCreationNode extends PostUpdateNode, NeedsSyntheticPreUpdateNode, CfgNode { + ObjectCreationNode() { node.(CallNode) = any(ClassCall c).getNode() } + + override Node getPreUpdateNode() { result.(SyntheticPreUpdateNode).getPostUpdateNode() = this } + + override string label() { result = "objCreate" } +} + class DataFlowExpr = Expr; /** @@ -191,16 +234,18 @@ private Node update(Node node) { //-------- // Global flow //-------- +// /** * IPA type for DataFlowCallable. - * A callable is either a callable value or a class. + * + * A callable is either a callable value or a module (for enclosing `ModuleVariableNode`s). + * A module has no calls. */ newtype TDataFlowCallable = TCallableValue(CallableValue callable) or - TClassValue(ClassValue c) or TModule(Module m) -/** Represents a callable */ +/** Represents a callable. */ abstract class DataFlowCallable extends TDataFlowCallable { /** Gets a textual representation of this element. */ abstract string toString(); @@ -218,6 +263,7 @@ abstract class DataFlowCallable extends TDataFlowCallable { abstract string getName(); } +/** A class representing a callable value. */ class DataFlowCallableValue extends DataFlowCallable, TCallableValue { CallableValue callable; @@ -234,24 +280,6 @@ class DataFlowCallableValue extends DataFlowCallable, TCallableValue { override string getName() { result = callable.getName() } } -class DataFlowClassValue extends DataFlowCallable, TClassValue { - ClassValue c; - - DataFlowClassValue() { this = TClassValue(c) } - - override string toString() { result = c.toString() } - - override CallNode getACall() { result = c.getACall() } - - override Scope getScope() { result = c.getScope() } - - override NameNode getParameter(int n) { - result.getNode() = c.getScope().getInitMethod().getArg(n + 1).asName() - } - - override string getName() { result = c.getName() } -} - /** A class representing the scope in which a `ModuleVariableNode` appears. */ class DataFlowModuleScope extends DataFlowCallable, TModule { Module mod; @@ -269,10 +297,24 @@ class DataFlowModuleScope extends DataFlowCallable, TModule { override string getName() { result = mod.getName() } } +/** + * IPA type for DataFlowCall. + * + * Calls corresponding to `CallNode`s are either to callable values or to classes. + * The latter is directed to the callable corresponding to the `__init__` method of the class. + * + * An `__init__` method can also be called directly, so that the callable can be targeted by + * different types of calls. In that case, the parameter mappings will be different, + * as the class call will synthesize an argument node to be mapped to the `self` parameter. + * + * A call corresponding to a special method call is handled by the corresponding `SpecialMethodCallNode`. + */ newtype TDataFlowCall = - TCallNode(CallNode call) or + TCallNode(CallNode call) { call = any(CallableValue c).getACall() } or + TClassCall(CallNode call) { call = any(ClassValue c).getACall() } or TSpecialCall(SpecialMethodCallNode special) +/** Represents a call. */ abstract class DataFlowCall extends TDataFlowCall { /** Gets a textual representation of this element. */ abstract string toString(); @@ -281,7 +323,7 @@ abstract class DataFlowCall extends TDataFlowCall { abstract DataFlowCallable getCallable(); /** Get the specified argument to this call. */ - abstract ControlFlowNode getArg(int n); + abstract Node getArg(int n); /** Get the control flow node representing this call. */ abstract ControlFlowNode getNode(); @@ -290,7 +332,7 @@ abstract class DataFlowCall extends TDataFlowCall { abstract DataFlowCallable getEnclosingCallable(); } -/** Represents a call to a callable. */ +/** Represents a call to a callable (currently only callable values). */ class CallNodeCall extends DataFlowCall, TCallNode { CallNode call; DataFlowCallable callable; @@ -302,7 +344,7 @@ class CallNodeCall extends DataFlowCall, TCallNode { override string toString() { result = call.toString() } - override ControlFlowNode getArg(int n) { result = call.getArg(n) } + override Node getArg(int n) { result = TCfgNode(call.getArg(n)) } override ControlFlowNode getNode() { result = call } @@ -311,6 +353,36 @@ class CallNodeCall extends DataFlowCall, TCallNode { override DataFlowCallable getEnclosingCallable() { result.getScope() = call.getNode().getScope() } } +/** Represents a call to a class. */ +class ClassCall extends DataFlowCall, TClassCall { + CallNode call; + ClassValue c; + + ClassCall() { + this = TClassCall(call) and + call = c.getACall() + } + + override string toString() { result = call.toString() } + + override Node getArg(int n) { + n > 0 and result = TCfgNode(call.getArg(n - 1)) + or + n = 0 and result = TSyntheticPreUpdateNode(TCfgNode(call)) + } + + override ControlFlowNode getNode() { result = call } + + override DataFlowCallable getCallable() { + exists(CallableValue callable | + result = TCallableValue(callable) and + c.getScope().getInitMethod() = callable.getScope() + ) + } + + override DataFlowCallable getEnclosingCallable() { result.getScope() = call.getScope() } +} + /** Represents a call to a special method. */ class SpecialCall extends DataFlowCall, TSpecialCall { SpecialMethodCallNode special; @@ -319,7 +391,7 @@ class SpecialCall extends DataFlowCall, TSpecialCall { override string toString() { result = special.toString() } - override ControlFlowNode getArg(int n) { result = special.(SpecialMethod::Potential).getArg(n) } + override Node getArg(int n) { result = TCfgNode(special.(SpecialMethod::Potential).getArg(n)) } override ControlFlowNode getNode() { result = special } @@ -333,11 +405,11 @@ class SpecialCall extends DataFlowCall, TSpecialCall { } /** A data flow node that represents a call argument. */ -class ArgumentNode extends CfgNode { - ArgumentNode() { exists(DataFlowCall call, int pos | node = call.getArg(pos)) } +class ArgumentNode extends Node { + ArgumentNode() { this = any(DataFlowCall c).getArg(_) } /** Holds if this argument occurs at the given position in the given call. */ - predicate argumentOf(DataFlowCall call, int pos) { node = call.getArg(pos) } + predicate argumentOf(DataFlowCall call, int pos) { this = call.getArg(pos) } /** Gets the call in which this node is an argument. */ final DataFlowCall getCall() { this.argumentOf(result, _) } @@ -411,7 +483,11 @@ predicate compatibleTypes(DataFlowType t1, DataFlowType t2) { any() } /** * Gets the type of `node`. */ -DataFlowType getNodeType(Node node) { result = TAnyFlow() } +DataFlowType getNodeType(Node node) { + result = TAnyFlow() and + // Suppress unused variable warning + node = node +} /** Gets a string representation of a type returned by `getErasedRepr`. */ string ppReprType(DataFlowType t) { none() } @@ -449,6 +525,8 @@ predicate storeStep(Node nodeFrom, Content c, Node nodeTo) { dictStoreStep(nodeFrom, c, nodeTo) or comprehensionStoreStep(nodeFrom, c, nodeTo) + or + attributeStoreStep(nodeFrom, c, nodeTo) } /** Data flows from an element of a list to the list. */ @@ -458,7 +536,9 @@ predicate listStoreStep(CfgNode nodeFrom, ListElementContent c, CfgNode nodeTo) // nodeFrom is `42`, cfg node // nodeTo is the list, `[..., 42, ...]`, cfg node // c denotes element of list - nodeTo.getNode().(ListNode).getAnElement() = nodeFrom.getNode() + nodeTo.getNode().(ListNode).getAnElement() = nodeFrom.getNode() and + // Suppress unused variable warning + c = c } /** Data flows from an element of a set to the set. */ @@ -468,7 +548,9 @@ predicate setStoreStep(CfgNode nodeFrom, ListElementContent c, CfgNode nodeTo) { // nodeFrom is `42`, cfg node // nodeTo is the set, `{..., 42, ...}`, cfg node // c denotes element of list - nodeTo.getNode().(SetNode).getAnElement() = nodeFrom.getNode() + nodeTo.getNode().(SetNode).getAnElement() = nodeFrom.getNode() and + // Suppress unused variable warning + c = c } /** Data flows from an element of a tuple to the tuple at a specific index. */ @@ -523,6 +605,23 @@ predicate comprehensionStoreStep(CfgNode nodeFrom, Content c, CfgNode nodeTo) { c instanceof ListElementContent } +/** + * Holds if `nodeFrom` flows into an attribute (corresponding to `c`) of `nodeTo` via an attribute assignment. + * + * For example, in + * ```python + * obj.foo = x + * ``` + * data flows from `x` to (the post-update node for) `obj` via assignment to `foo`. + */ +predicate attributeStoreStep(CfgNode nodeFrom, AttributeContent c, PostUpdateNode nodeTo) { + exists(AttrNode attr | + nodeFrom.asCfgNode() = attr.(DefinitionNode).getValue() and + attr.getName() = c.getAttribute() and + attr.getObject() = nodeTo.getPreUpdateNode().(CfgNode).getNode() + ) +} + /** * Holds if data can flow from `nodeFrom` to `nodeTo` via a read of content `c`. */ @@ -532,6 +631,8 @@ predicate readStep(Node nodeFrom, Content c, Node nodeTo) { popReadStep(nodeFrom, c, nodeTo) or comprehensionReadStep(nodeFrom, c, nodeTo) + or + attributeReadStep(nodeFrom, c, nodeTo) } /** Data flows from a sequence to a subscript of the sequence. */ @@ -618,6 +719,24 @@ predicate comprehensionReadStep(CfgNode nodeFrom, Content c, EssaNode nodeTo) { ) } +/** + * Holds if `nodeTo` is a read of an attribute (corresponding to `c`) of the object in `nodeFrom`. + * + * For example, in + * ```python + * obj.foo + * ``` + * data flows from `obj` to `obj.foo` via a read from `foo`. + */ +predicate attributeReadStep(CfgNode nodeFrom, AttributeContent c, CfgNode nodeTo) { + exists(AttrNode attr | + nodeFrom.asCfgNode() = attr.getObject() and + nodeTo.asCfgNode() = attr and + attr.getName() = c.getAttribute() and + attr.isLoad() + ) +} + /** * Holds if values stored inside content `c` are cleared at node `n`. For example, * any value stored inside `f` is cleared at the pre-update node associated with `x` diff --git a/python/ql/src/experimental/dataflow/internal/DataFlowPublic.qll b/python/ql/src/experimental/dataflow/internal/DataFlowPublic.qll index eff7ba5628e..fb68fe3b1f1 100644 --- a/python/ql/src/experimental/dataflow/internal/DataFlowPublic.qll +++ b/python/ql/src/experimental/dataflow/internal/DataFlowPublic.qll @@ -23,8 +23,10 @@ newtype TNode = TEssaNode(EssaVariable var) or /** A node corresponding to a control flow node. */ TCfgNode(DataFlowCfgNode node) or - /** A node representing the value of an object after a state change */ - TPostUpdateNode(PreUpdateNode pre) or + /** A synthetic node representing the value of an object before a state change */ + TSyntheticPreUpdateNode(NeedsSyntheticPreUpdateNode post) or + /** A synthetic node representing the value of an object after a state change */ + TSyntheticPostUpdateNode(NeedsSyntheticPostUpdateNode pre) or /** A node representing a global (module-level) variable in a specific module */ TModuleVariableNode(Module m, GlobalVariable v) { v.getScope() = m and v.escapes() } @@ -151,6 +153,23 @@ class ParameterNode extends EssaNode { override DataFlowCallable getEnclosingCallable() { this.isParameterOf(result, _) } } +/** + * A node associated with an object after an operation that might have + * changed its state. + * + * This can be either the argument to a callable after the callable returns + * (which might have mutated the argument), or the qualifier of a field after + * an update to the field. + * + * Nodes corresponding to AST elements, for example `ExprNode`s, usually refer + * to the value before the update with the exception of `ObjectCreationNode`s, + * which represents the value _after_ the constructor has run. + */ +abstract class PostUpdateNode extends Node { + /** Gets the node before the state update. */ + abstract Node getPreUpdateNode(); +} + /** * A data flow node corresponding to a module-level (global) variable that is accessed outside of the module scope. * @@ -270,7 +289,9 @@ newtype TContent = key = any(Keyword kw).getArg() } or /** An element of a dictionary at any key. */ - TDictionaryElementAnyContent() + TDictionaryElementAnyContent() or + /** An object attribute. */ + TAttributeContent(string attr) { attr = any(Attribute a).getName() } class Content extends TContent { /** Gets a textual representation of this element. */ @@ -278,12 +299,10 @@ class Content extends TContent { } class ListElementContent extends TListElementContent, Content { - /** Gets a textual representation of this element. */ override string toString() { result = "List element" } } class SetElementContent extends TSetElementContent, Content { - /** Gets a textual representation of this element. */ override string toString() { result = "Set element" } } @@ -292,10 +311,9 @@ class TupleElementContent extends TTupleElementContent, Content { TupleElementContent() { this = TTupleElementContent(index) } - /** Gets the index for this tuple element */ + /** Gets the index for this tuple element. */ int getIndex() { result = index } - /** Gets a textual representation of this element. */ override string toString() { result = "Tuple element at index " + index.toString() } } @@ -304,14 +322,23 @@ class DictionaryElementContent extends TDictionaryElementContent, Content { DictionaryElementContent() { this = TDictionaryElementContent(key) } - /** Gets the index for this tuple element */ + /** Gets the key for this dictionary element. */ string getKey() { result = key } - /** Gets a textual representation of this element. */ override string toString() { result = "Dictionary element at key " + key } } class DictionaryElementAnyContent extends TDictionaryElementAnyContent, Content { - /** Gets a textual representation of this element. */ override string toString() { result = "Any dictionary element" } } + +class AttributeContent extends TAttributeContent, Content { + private string attr; + + AttributeContent() { this = TAttributeContent(attr) } + + /** Gets the name of the attribute under which this content is stored. */ + string getAttribute() { result = attr } + + override string toString() { result = "Attribute " + attr } +} diff --git a/python/ql/src/experimental/dataflow/internal/DataFlowUtil.qll b/python/ql/src/experimental/dataflow/internal/DataFlowUtil.qll index c4acc112051..c868e1762ec 100644 --- a/python/ql/src/experimental/dataflow/internal/DataFlowUtil.qll +++ b/python/ql/src/experimental/dataflow/internal/DataFlowUtil.qll @@ -16,3 +16,52 @@ predicate localFlowStep(Node nodeFrom, Node nodeTo) { simpleLocalFlowStep(nodeFr * (intra-procedural) steps. */ predicate localFlow(Node source, Node sink) { localFlowStep*(source, sink) } + +/** + * Gets an EssaNode that holds the module imported by `name`. + * Note that for the statement `import pkg.mod`, the new variable introduced is `pkg` that is a + * reference to the module `pkg`. + * + * This predicate handles (with optional `... as `): + * 1. `import ` + * 2. `from import ` when ` = + "." + ` + * 3. `from import ` when ` = + "." + ` + * + * Note: + * While it is technically possible that `import mypkg.foo` and `from mypkg import foo` can give different values, + * it's highly unlikely that this will be a problem in production level code. + * Example: If `mypkg/__init__.py` contains `foo = 42`, then `from mypkg import foo` will not import the module + * `mypkg/foo.py` but the variable `foo` containing `42` -- however, `import mypkg.foo` will always cause `mypkg.foo` + * to refer to the module. + * + * Also see `DataFlow::importMember` + */ +EssaNode importModule(string name) { + exists(Variable var, Import imp, Alias alias | + alias = imp.getAName() and + alias.getAsname() = var.getAStore() and + ( + name = alias.getValue().(ImportMember).getImportedModuleName() + or + name = alias.getValue().(ImportExpr).getImportedModuleName() + ) and + result.getVar().(AssignmentDefinition).getSourceVariable() = var + ) +} + +/** + * Gets a EssaNode that holds the value imported by using fully qualified name in + *`from import `. + * + * Also see `DataFlow::importModule`. + */ +EssaNode importMember(string moduleName, string memberName) { + exists(Variable var, Import imp, Alias alias, ImportMember member | + alias = imp.getAName() and + member = alias.getValue() and + moduleName = member.getModule().(ImportExpr).getImportedModuleName() and + memberName = member.getName() and + alias.getAsname() = var.getAStore() and + result.getVar().(AssignmentDefinition).getSourceVariable() = var + ) +} diff --git a/python/ql/src/experimental/dataflow/internal/TaintTrackingPrivate.qll b/python/ql/src/experimental/dataflow/internal/TaintTrackingPrivate.qll index 64bdd3bb3ab..d49ff207370 100644 --- a/python/ql/src/experimental/dataflow/internal/TaintTrackingPrivate.qll +++ b/python/ql/src/experimental/dataflow/internal/TaintTrackingPrivate.qll @@ -101,7 +101,7 @@ predicate stringManipulation(DataFlow::CfgNode nodeFrom, DataFlow::CfgNode nodeT nodeFrom.getNode() = object and method_name in ["partition", "rpartition", "rsplit", "split", "splitlines"] or - // List[str] -> str + // Iterable[str] -> str // TODO: check if these should be handled differently in regards to content method_name = "join" and nodeFrom.getNode() = call.getArg(0) @@ -130,7 +130,6 @@ predicate stringManipulation(DataFlow::CfgNode nodeFrom, DataFlow::CfgNode nodeT // f-strings nodeTo.asExpr().(Fstring).getAValue() = nodeFrom.asExpr() // TODO: Handle encode/decode from base64/quopri - // TODO: Handle os.path.join // TODO: Handle functions in https://docs.python.org/3/library/binascii.html } @@ -182,7 +181,7 @@ predicate containerStep(DataFlow::CfgNode nodeFrom, DataFlow::Node nodeTo) { exists(CallNode call, string name | name in ["append", "add"] and call.getFunction().(AttrNode).getObject(name) = - nodeTo.(PostUpdateNode).getPreUpdateNode().asCfgNode() and + nodeTo.(DataFlow::PostUpdateNode).getPreUpdateNode().asCfgNode() and call.getArg(0) = nodeFrom.getNode() ) } diff --git a/python/ql/src/experimental/semmle/python/frameworks/Flask.qll b/python/ql/src/experimental/semmle/python/frameworks/Flask.qll index 1639ceeebfa..5b4079331d7 100644 --- a/python/ql/src/experimental/semmle/python/frameworks/Flask.qll +++ b/python/ql/src/experimental/semmle/python/frameworks/Flask.qll @@ -5,6 +5,160 @@ private import python private import experimental.dataflow.DataFlow private import experimental.dataflow.RemoteFlowSources +private import experimental.dataflow.TaintTracking private import experimental.semmle.python.Concepts +private import experimental.semmle.python.frameworks.Werkzeug -private module Flask { } +// for old improved impl see +// https://github.com/github/codeql/blob/9f95212e103c68d0c1dfa4b6f30fb5d53954ccef/python/ql/src/semmle/python/web/flask/Request.qll +private module Flask { + /** Gets a reference to the `flask` module. */ + DataFlow::Node flask(DataFlow::TypeTracker t) { + t.start() and + result = DataFlow::importModule("flask") + or + exists(DataFlow::TypeTracker t2 | result = flask(t2).track(t2, t)) + } + + /** Gets a reference to the `flask` module. */ + DataFlow::Node flask() { result = flask(DataFlow::TypeTracker::end()) } + + module flask { + /** Gets a reference to the `flask.request` object. */ + DataFlow::Node request(DataFlow::TypeTracker t) { + t.start() and + result = DataFlow::importMember("flask", "request") + or + t.startInAttr("request") and + result = flask() + or + exists(DataFlow::TypeTracker t2 | result = flask::request(t2).track(t2, t)) + } + + /** Gets a reference to the `flask.request` object. */ + DataFlow::Node request() { result = flask::request(DataFlow::TypeTracker::end()) } + } + + // TODO: Do we even need this class? :| + /** + * A source of remote flow from a flask request. + * + * See https://flask.palletsprojects.com/en/1.1.x/api/#flask.Request + */ + private class RequestSource extends RemoteFlowSource::Range { + RequestSource() { this = flask::request() } + + override string getSourceType() { result = "flask.request" } + } + + private module FlaskRequestTracking { + /** Gets a reference to the `get_data` attribute of a Flask request. */ + private DataFlow::Node get_data(DataFlow::TypeTracker t) { + t.startInAttr("get_data") and + result = flask::request() + or + exists(DataFlow::TypeTracker t2 | result = get_data(t2).track(t2, t)) + } + + /** Gets a reference to the `get_data` attribute of a Flask request. */ + DataFlow::Node get_data() { result = get_data(DataFlow::TypeTracker::end()) } + + /** Gets a reference to the `get_json` attribute of a Flask request. */ + private DataFlow::Node get_json(DataFlow::TypeTracker t) { + t.startInAttr("get_json") and + result = flask::request() + or + exists(DataFlow::TypeTracker t2 | result = get_json(t2).track(t2, t)) + } + + /** Gets a reference to the `get_json` attribute of a Flask request. */ + DataFlow::Node get_json() { result = get_json(DataFlow::TypeTracker::end()) } + + /** Gets a reference to either of the `get_json` or `get_data` attributes of a Flask request. */ + DataFlow::Node tainted_methods(string attr_name) { + result = get_data() and + attr_name = "get_data" + or + result = get_json() and + attr_name = "get_json" + } + } + + /** + * A source of remote flow from attributes from a flask request. + * + * See https://flask.palletsprojects.com/en/1.1.x/api/#flask.Request + */ + private class RequestInputAccess extends RemoteFlowSource::Range { + string attr_name; + + RequestInputAccess() { + // attributes + exists(AttrNode attr | + this.asCfgNode() = attr and attr.getObject(attr_name) = flask::request().asCfgNode() + | + attr_name in ["path", + // str + "full_path", "base_url", "url", "access_control_request_method", "content_encoding", + "content_md5", "content_type", "data", "method", "mimetype", "origin", "query_string", + "referrer", "remote_addr", "remote_user", "user_agent", + // dict + "environ", "cookies", "mimetype_params", "view_args", + // json + "json", + // List[str] + "access_route", + // file-like + "stream", "input_stream", + // MultiDict[str, str] + // https://werkzeug.palletsprojects.com/en/1.0.x/datastructures/#werkzeug.datastructures.MultiDict + "args", "values", "form", + // MultiDict[str, FileStorage] + // https://werkzeug.palletsprojects.com/en/1.0.x/datastructures/#werkzeug.datastructures.FileStorage + // TODO: FileStorage needs extra taint steps + "files", + // https://werkzeug.palletsprojects.com/en/1.0.x/datastructures/#werkzeug.datastructures.HeaderSet + "access_control_request_headers", "pragma", + // https://werkzeug.palletsprojects.com/en/1.0.x/datastructures/#werkzeug.datastructures.Accept + // TODO: Kinda badly modeled for now -- has type List[Tuple[value, quality]], and some extra methods + "accept_charsets", "accept_encodings", "accept_languages", "accept_mimetypes", + // https://werkzeug.palletsprojects.com/en/1.0.x/datastructures/#werkzeug.datastructures.Authorization + // TODO: dict subclass with extra attributes like `username` and `password` + "authorization", + // https://werkzeug.palletsprojects.com/en/1.0.x/datastructures/#werkzeug.datastructures.RequestCacheControl + // TODO: has attributes like `no_cache`, and `to_header` method (actually, many of these models do) + "cache_control", + // https://werkzeug.palletsprojects.com/en/1.0.x/datastructures/#werkzeug.datastructures.Headers + // TODO: dict-like with wsgiref.headers.Header compatibility methods + "headers"] + ) + or + // methods (needs special handling to track bound-methods -- see `FlaskRequestMethodCallsAdditionalTaintStep` below) + this = FlaskRequestTracking::tainted_methods(attr_name) + } + + override string getSourceType() { result = "flask.request input" } + } + + private class FlaskRequestMethodCallsAdditionalTaintStep extends TaintTracking::AdditionalTaintStep { + override predicate step(DataFlow::Node nodeFrom, DataFlow::Node nodeTo) { + // NOTE: `request -> request.tainted_method` part is handled as part of RequestInputAccess + // tainted_method -> tainted_method() + nodeFrom = FlaskRequestTracking::tainted_methods(_) and + nodeTo.asCfgNode().(CallNode).getFunction() = nodeFrom.asCfgNode() + } + } + + private class RequestInputMultiDict extends RequestInputAccess, + Werkzeug::Datastructures::MultiDict { + RequestInputMultiDict() { attr_name in ["args", "values", "form", "files"] } + } + + private class RequestInputFiles extends RequestInputMultiDict { + RequestInputFiles() { attr_name = "files" } + } + // TODO: Somehow specify that elements of `RequestInputFiles` are + // Werkzeug::Datastructures::FileStorage and should have those additional taint steps + // AND that the 0-indexed argument to its' save method is a sink for path-injection. + // https://werkzeug.palletsprojects.com/en/1.0.x/datastructures/#werkzeug.datastructures.FileStorage.save +} diff --git a/python/ql/src/experimental/semmle/python/frameworks/Stdlib.qll b/python/ql/src/experimental/semmle/python/frameworks/Stdlib.qll index 60efafbdfde..5fa36c67e09 100644 --- a/python/ql/src/experimental/semmle/python/frameworks/Stdlib.qll +++ b/python/ql/src/experimental/semmle/python/frameworks/Stdlib.qll @@ -5,5 +5,326 @@ private import python private import experimental.dataflow.DataFlow +private import experimental.dataflow.TaintTracking private import experimental.dataflow.RemoteFlowSources private import experimental.semmle.python.Concepts + +/** Provides models for the Python standard library. */ +private module Stdlib { + // --------------------------------------------------------------------------- + // os + // --------------------------------------------------------------------------- + /** Gets a reference to the `os` module. */ + private DataFlow::Node os(DataFlow::TypeTracker t) { + t.start() and + result = DataFlow::importModule("os") + or + exists(DataFlow::TypeTracker t2 | result = os(t2).track(t2, t)) + } + + /** Gets a reference to the `os` module. */ + DataFlow::Node os() { result = os(DataFlow::TypeTracker::end()) } + + /** + * Gets a reference to the attribute `attr_name` of the `os` module. + * WARNING: Only holds for a few predefined attributes. + * + * For example, using `attr_name = "system"` will get all uses of `os.system`. + */ + private DataFlow::Node os_attr(DataFlow::TypeTracker t, string attr_name) { + attr_name in ["system", "popen", + // exec + "execl", "execle", "execlp", "execlpe", "execv", "execve", "execvp", "execvpe", + // spawn + "spawnl", "spawnle", "spawnlp", "spawnlpe", "spawnv", "spawnve", "spawnvp", "spawnvpe", + "posix_spawn", "posix_spawnp", + // modules + "path"] and + ( + t.start() and + result = DataFlow::importMember("os", attr_name) + or + t.startInAttr(attr_name) and + result = DataFlow::importModule("os") + ) + or + // Due to bad performance when using normal setup with `os_attr(t2, attr_name).track(t2, t)` + // we have inlined that code and forced a join + exists(DataFlow::TypeTracker t2 | + exists(DataFlow::StepSummary summary | + os_attr_first_join(t2, attr_name, result, summary) and + t = t2.append(summary) + ) + ) + } + + pragma[nomagic] + private predicate os_attr_first_join( + DataFlow::TypeTracker t2, string attr_name, DataFlow::Node res, DataFlow::StepSummary summary + ) { + DataFlow::StepSummary::step(os_attr(t2, attr_name), res, summary) + } + + /** + * Gets a reference to the attribute `attr_name` of the `os` module. + * WARNING: Only holds for a few predefined attributes. + * + * For example, using `"system"` will get all uses of `os.system`. + */ + private DataFlow::Node os_attr(string attr_name) { + result = os_attr(DataFlow::TypeTracker::end(), attr_name) + } + + /** Provides models for the `os` module. */ + module os { + /** Gets a reference to the `os.path` module. */ + DataFlow::Node path() { result = os_attr("path") } + + /** Provides models for the `os.path` module */ + module path { + /** Gets a reference to the `os.path.join` function. */ + private DataFlow::Node join(DataFlow::TypeTracker t) { + t.start() and + result = DataFlow::importMember("os.path", "join") + or + t.startInAttr("join") and + result = os::path() + or + exists(DataFlow::TypeTracker t2 | result = join(t2).track(t2, t)) + } + + /** Gets a reference to the `os.path.join` function. */ + DataFlow::Node join() { result = join(DataFlow::TypeTracker::end()) } + } + } + + /** + * A call to `os.system`. + * See https://docs.python.org/3/library/os.html#os.system + */ + private class OsSystemCall extends SystemCommandExecution::Range { + OsSystemCall() { this.asCfgNode().(CallNode).getFunction() = os_attr("system").asCfgNode() } + + override DataFlow::Node getCommand() { + result.asCfgNode() = this.asCfgNode().(CallNode).getArg(0) + } + } + + /** + * A call to `os.popen` + * See https://docs.python.org/3/library/os.html#os.popen + */ + private class OsPopenCall extends SystemCommandExecution::Range { + OsPopenCall() { this.asCfgNode().(CallNode).getFunction() = os_attr("popen").asCfgNode() } + + override DataFlow::Node getCommand() { + result.asCfgNode() = this.asCfgNode().(CallNode).getArg(0) + } + } + + /** + * A call to any of the `os.exec*` functions + * See https://docs.python.org/3.8/library/os.html#os.execl + */ + private class OsExecCall extends SystemCommandExecution::Range { + OsExecCall() { + exists(string name | + name in ["execl", "execle", "execlp", "execlpe", "execv", "execve", "execvp", "execvpe"] and + this.asCfgNode().(CallNode).getFunction() = os_attr(name).asCfgNode() + ) + } + + override DataFlow::Node getCommand() { + result.asCfgNode() = this.asCfgNode().(CallNode).getArg(0) + } + } + + /** + * A call to any of the `os.spawn*` functions + * See https://docs.python.org/3.8/library/os.html#os.spawnl + */ + private class OsSpawnCall extends SystemCommandExecution::Range { + OsSpawnCall() { + exists(string name | + name in ["spawnl", "spawnle", "spawnlp", "spawnlpe", "spawnv", "spawnve", "spawnvp", + "spawnvpe"] and + this.asCfgNode().(CallNode).getFunction() = os_attr(name).asCfgNode() + ) + } + + override DataFlow::Node getCommand() { + result.asCfgNode() = this.asCfgNode().(CallNode).getArg(1) + } + } + + /** + * A call to any of the `os.posix_spawn*` functions + * See https://docs.python.org/3.8/library/os.html#os.posix_spawn + */ + private class OsPosixSpawnCall extends SystemCommandExecution::Range { + OsPosixSpawnCall() { + this.asCfgNode().(CallNode).getFunction() = + os_attr(["posix_spawn", "posix_spawnp"]).asCfgNode() + } + + override DataFlow::Node getCommand() { + result.asCfgNode() = this.asCfgNode().(CallNode).getArg(0) + } + } + + /** An additional taint step for calls to `os.path.join` */ + private class OsPathJoinCallAdditionalTaintStep extends TaintTracking::AdditionalTaintStep { + override predicate step(DataFlow::Node nodeFrom, DataFlow::Node nodeTo) { + exists(CallNode call | + nodeTo.asCfgNode() = call and + call.getFunction() = os::path::join().asCfgNode() and + call.getAnArg() = nodeFrom.asCfgNode() + ) + // TODO: Handle pathlib (like we do for os.path.join) + } + } + + // --------------------------------------------------------------------------- + // subprocess + // --------------------------------------------------------------------------- + /** Gets a reference to the `subprocess` module. */ + private DataFlow::Node subprocess(DataFlow::TypeTracker t) { + t.start() and + result = DataFlow::importModule("subprocess") + or + exists(DataFlow::TypeTracker t2 | result = subprocess(t2).track(t2, t)) + } + + /** Gets a reference to the `subprocess` module. */ + DataFlow::Node subprocess() { result = subprocess(DataFlow::TypeTracker::end()) } + + /** + * Gets a reference to the attribute `attr_name` of the `subprocess` module. + * WARNING: Only holds for a few predefined attributes. + * + * For example, using `attr_name = "Popen"` will get all uses of `subprocess.Popen`. + */ + private DataFlow::Node subprocess_attr(DataFlow::TypeTracker t, string attr_name) { + attr_name in ["Popen", "call", "check_call", "check_output", "run"] and + ( + t.start() and + result = DataFlow::importMember("subprocess", attr_name) + or + t.startInAttr(attr_name) and + result = DataFlow::importModule("subprocess") + ) + or + // Due to bad performance when using normal setup with `subprocess_attr(t2, attr_name).track(t2, t)` + // we have inlined that code and forced a join + exists(DataFlow::TypeTracker t2 | + exists(DataFlow::StepSummary summary | + subprocess_attr_first_join(t2, attr_name, result, summary) and + t = t2.append(summary) + ) + ) + } + + pragma[nomagic] + private predicate subprocess_attr_first_join( + DataFlow::TypeTracker t2, string attr_name, DataFlow::Node res, DataFlow::StepSummary summary + ) { + DataFlow::StepSummary::step(subprocess_attr(t2, attr_name), res, summary) + } + + /** + * Gets a reference to the attribute `attr_name` of the `subprocess` module. + * WARNING: Only holds for a few predefined attributes. + * + * For example, using `attr_name = "Popen"` will get all uses of `subprocess.Popen`. + */ + private DataFlow::Node subprocess_attr(string attr_name) { + result = subprocess_attr(DataFlow::TypeTracker::end(), attr_name) + } + + /** + * A call to `subprocess.Popen` or helper functions (call, check_call, check_output, run) + * See https://docs.python.org/3.8/library/subprocess.html#subprocess.Popen + */ + private class SubprocessPopenCall extends SystemCommandExecution::Range { + CallNode call; + + SubprocessPopenCall() { + call = this.asCfgNode() and + exists(string name | + name in ["Popen", "call", "check_call", "check_output", "run"] and + call.getFunction() = subprocess_attr(name).asCfgNode() + ) + } + + /** Gets the ControlFlowNode for the `args` argument, if any. */ + private ControlFlowNode get_args_arg() { + result = call.getArg(0) + or + result = call.getArgByName("args") + } + + /** Gets the ControlFlowNode for the `shell` argument, if any. */ + private ControlFlowNode get_shell_arg() { + result = call.getArg(8) + or + result = call.getArgByName("shell") + } + + private boolean get_shell_arg_value() { + not exists(this.get_shell_arg()) and + result = false + or + exists(ControlFlowNode shell_arg | shell_arg = this.get_shell_arg() | + result = shell_arg.getNode().(ImmutableLiteral).booleanValue() + or + // TODO: Track the "shell" argument to determine possible values + not shell_arg.getNode() instanceof ImmutableLiteral and + ( + result = true + or + result = false + ) + ) + } + + /** Gets the ControlFlowNode for the `executable` argument, if any. */ + private ControlFlowNode get_executable_arg() { + result = call.getArg(2) + or + result = call.getArgByName("executable") + } + + override DataFlow::Node getCommand() { + // TODO: Track arguments ("args" and "shell") + // TODO: Handle using `args=["sh", "-c", ]` + result.asCfgNode() = this.get_executable_arg() + or + exists(ControlFlowNode arg_args, boolean shell | + arg_args = get_args_arg() and + shell = get_shell_arg_value() + | + // When "executable" argument is set, and "shell" argument is `False`, the + // "args" argument will only be used to set the program name and arguments to + // the program, so we should not consider any of them as command execution. + not ( + exists(this.get_executable_arg()) and + shell = false + ) and + ( + // When the "args" argument is an iterable, first element is the command to + // run, so if we're able to, we only mark the first element as the command + // (and not the arguments to the command). + // + result.asCfgNode() = arg_args.(SequenceNode).getElement(0) + or + // Either the "args" argument is not a sequence (which is valid) or we where + // just not able to figure it out. Simply mark the "args" argument as the + // command. + // + not arg_args instanceof SequenceNode and + result.asCfgNode() = arg_args + ) + ) + } + } +} diff --git a/python/ql/src/experimental/semmle/python/frameworks/Werkzeug.qll b/python/ql/src/experimental/semmle/python/frameworks/Werkzeug.qll new file mode 100644 index 00000000000..0c682b71464 --- /dev/null +++ b/python/ql/src/experimental/semmle/python/frameworks/Werkzeug.qll @@ -0,0 +1,75 @@ +/** + * Provides classes modeling security-relevant aspects of the `flask` package. + */ + +private import python +private import experimental.dataflow.DataFlow +private import experimental.dataflow.TaintTracking + +// for old impl see +// https://github.com/github/codeql/blob/9f95212e103c68d0c1dfa4b6f30fb5d53954ccef/python/ql/src/semmle/python/libraries/Werkzeug.qll +module Werkzeug { + module Datastructures { + // ---------------------------------------------------------------------- // + // MultiDict // + // ---------------------------------------------------------------------- // + /** + * A Node representing an instance of a werkzeug.datastructures.MultiDict + * + * See https://werkzeug.palletsprojects.com/en/1.0.x/datastructures/#werkzeug.datastructures.MultiDict + */ + abstract class MultiDict extends DataFlow::Node { } + + private module MultiDictTracking { + private DataFlow::Node getlist(DataFlow::TypeTracker t) { + t.startInAttr("getlist") and + result instanceof MultiDict + or + exists(DataFlow::TypeTracker t2 | result = getlist(t2).track(t2, t)) + } + + DataFlow::Node getlist() { result = getlist(DataFlow::TypeTracker::end()) } + } + + private class MultiDictAdditionalTaintStep extends TaintTracking::AdditionalTaintStep { + override predicate step(DataFlow::Node nodeFrom, DataFlow::Node nodeTo) { + // obj -> obj.getlist + nodeTo.asCfgNode().(AttrNode).getObject("getlist") = nodeFrom.asCfgNode() and + nodeTo = MultiDictTracking::getlist() + or + // getlist -> getlist() + nodeFrom = MultiDictTracking::getlist() and + nodeTo.asCfgNode().(CallNode).getFunction() = nodeFrom.asCfgNode() + } + } + + // ---------------------------------------------------------------------- // + // FileStorage // + // ---------------------------------------------------------------------- // + /** + * A Node representing an instance of a werkzeug.datastructures.FileStorage + * + * See https://werkzeug.palletsprojects.com/en/1.0.x/datastructures/#werkzeug.datastructures.FileStorage + */ + abstract class FileStorage extends DataFlow::Node { } + + private class FileStorageAdditionalTaintStep extends TaintTracking::AdditionalTaintStep { + override predicate step(DataFlow::Node nodeFrom, DataFlow::Node nodeTo) { + // TODO: should be `nodeFrom = tracked(any(FileStorage fs))` + nodeFrom instanceof FileStorage and + exists(string name | + name in ["filename", + // str + "name", "content_type", "mimetype", + // file-like + "stream", + // TODO: werkzeug.datastructures.Headers + "headers", + // dict[str, str] + "mimetype_params"] and + nodeTo.asCfgNode().(AttrNode).getObject(name) = nodeFrom.asCfgNode() + ) + } + } + } +} diff --git a/python/ql/src/semmle/python/essa/SsaCompute.qll b/python/ql/src/semmle/python/essa/SsaCompute.qll index ad348675805..c9b4b28f96d 100644 --- a/python/ql/src/semmle/python/essa/SsaCompute.qll +++ b/python/ql/src/semmle/python/essa/SsaCompute.qll @@ -382,7 +382,9 @@ private module SsaComputeImpl { /** Holds if `v` occurs in `b` or one of `b`'s transitive successors. */ private predicate blockPrecedesVar(SsaSourceVariable v, BasicBlock b) { - varOccursInBlock(v, b.getASuccessor*()) + varOccursInBlock(v, b) + or + SsaDefinitionsImpl::reachesEndOfBlock(v, _, _, b) } /** diff --git a/python/ql/test/experimental/dataflow/basic/local.expected b/python/ql/test/experimental/dataflow/basic/local.expected index 6610a6244d6..bb051d8a30e 100644 --- a/python/ql/test/experimental/dataflow/basic/local.expected +++ b/python/ql/test/experimental/dataflow/basic/local.expected @@ -47,4 +47,4 @@ | test.py:7:5:7:20 | ControlFlowNode for obfuscated_id() | test.py:7:5:7:20 | ControlFlowNode for obfuscated_id() | | test.py:7:5:7:20 | GSSA Variable a | test.py:7:5:7:20 | GSSA Variable a | | test.py:7:19:7:19 | ControlFlowNode for a | test.py:7:19:7:19 | ControlFlowNode for a | -| test.py:7:19:7:19 | [post] ControlFlowNode for a | test.py:7:19:7:19 | [post] ControlFlowNode for a | +| test.py:7:19:7:19 | [post arg] ControlFlowNode for a | test.py:7:19:7:19 | [post arg] ControlFlowNode for a | diff --git a/python/ql/test/experimental/dataflow/basic/sinks.expected b/python/ql/test/experimental/dataflow/basic/sinks.expected index 5b58a0f844f..bfebc2ef31e 100644 --- a/python/ql/test/experimental/dataflow/basic/sinks.expected +++ b/python/ql/test/experimental/dataflow/basic/sinks.expected @@ -23,4 +23,4 @@ | test.py:7:5:7:20 | ControlFlowNode for obfuscated_id() | | test.py:7:5:7:20 | GSSA Variable a | | test.py:7:19:7:19 | ControlFlowNode for a | -| test.py:7:19:7:19 | [post] ControlFlowNode for a | +| test.py:7:19:7:19 | [post arg] ControlFlowNode for a | diff --git a/python/ql/test/experimental/dataflow/basic/sources.expected b/python/ql/test/experimental/dataflow/basic/sources.expected index 5b58a0f844f..bfebc2ef31e 100644 --- a/python/ql/test/experimental/dataflow/basic/sources.expected +++ b/python/ql/test/experimental/dataflow/basic/sources.expected @@ -23,4 +23,4 @@ | test.py:7:5:7:20 | ControlFlowNode for obfuscated_id() | | test.py:7:5:7:20 | GSSA Variable a | | test.py:7:19:7:19 | ControlFlowNode for a | -| test.py:7:19:7:19 | [post] ControlFlowNode for a | +| test.py:7:19:7:19 | [post arg] ControlFlowNode for a | diff --git a/python/ql/test/experimental/dataflow/coverage/argumentRouting1.ql b/python/ql/test/experimental/dataflow/coverage/argumentRouting1.ql index 912c879d34c..90878dc9052 100644 --- a/python/ql/test/experimental/dataflow/coverage/argumentRouting1.ql +++ b/python/ql/test/experimental/dataflow/coverage/argumentRouting1.ql @@ -12,7 +12,7 @@ class ArgumentRoutingConfig extends DataFlow::Configuration { exists(AssignmentDefinition def, DataFlowPrivate::DataFlowCall call | def.getVariable() = node.(DataFlow::EssaNode).getVar() and def.getValue() = call.getNode() and - call.getCallable().getName().matches("With\\_%") + call.getNode().(CallNode).getFunction().(NameNode).getId().matches("With\\_%") ) and node.(DataFlow::EssaNode).getVar().getName().matches("with\\_%") } diff --git a/python/ql/test/experimental/dataflow/coverage/classes.py b/python/ql/test/experimental/dataflow/coverage/classes.py index 350c10d5b04..830ad72d1c5 100644 --- a/python/ql/test/experimental/dataflow/coverage/classes.py +++ b/python/ql/test/experimental/dataflow/coverage/classes.py @@ -50,8 +50,8 @@ def test_new(): class With_init: def __init__(self): - SINK1(self) # Flow not found - OK() # Call not found + SINK1(self) + OK() def test_init(): diff --git a/python/ql/test/experimental/dataflow/coverage/classesCallGraph.expected b/python/ql/test/experimental/dataflow/coverage/classesCallGraph.expected index 1276083cecc..cf688c17a49 100644 --- a/python/ql/test/experimental/dataflow/coverage/classesCallGraph.expected +++ b/python/ql/test/experimental/dataflow/coverage/classesCallGraph.expected @@ -1,4 +1,5 @@ | classes.py:41:16:41:35 | ControlFlowNode for Attribute() | classes.py:41:16:41:35 | ControlFlowNode for Attribute() | +| classes.py:58:17:58:27 | [pre objCreate] ControlFlowNode for With_init() | classes.py:52:18:52:21 | SSA variable self | | classes.py:264:9:264:24 | ControlFlowNode for set() | classes.py:264:9:264:24 | ControlFlowNode for set() | | classes.py:269:9:269:30 | ControlFlowNode for frozenset() | classes.py:269:9:269:30 | ControlFlowNode for frozenset() | | classes.py:274:9:274:28 | ControlFlowNode for dict() | classes.py:274:9:274:28 | ControlFlowNode for dict() | diff --git a/python/ql/test/experimental/dataflow/coverage/dataflow.expected b/python/ql/test/experimental/dataflow/coverage/dataflow.expected index 81d6bb1800a..379b44a3090 100644 --- a/python/ql/test/experimental/dataflow/coverage/dataflow.expected +++ b/python/ql/test/experimental/dataflow/coverage/dataflow.expected @@ -1,4 +1,7 @@ edges +| datamodel.py:0:0:0:0 | ModuleVariableNode for Global Variable SOURCE in Module datamodel | datamodel.py:152:14:152:19 | ControlFlowNode for SOURCE | +| datamodel.py:13:1:13:6 | GSSA Variable SOURCE | datamodel.py:0:0:0:0 | ModuleVariableNode for Global Variable SOURCE in Module datamodel | +| datamodel.py:13:10:13:17 | ControlFlowNode for Str | datamodel.py:13:1:13:6 | GSSA Variable SOURCE | | datamodel.py:13:10:13:17 | ControlFlowNode for Str | datamodel.py:38:8:38:13 | ControlFlowNode for SOURCE | | datamodel.py:13:10:13:17 | ControlFlowNode for Str | datamodel.py:71:15:71:20 | ControlFlowNode for SOURCE | | datamodel.py:13:10:13:17 | ControlFlowNode for Str | datamodel.py:72:18:72:23 | ControlFlowNode for SOURCE | @@ -21,6 +24,10 @@ edges | datamodel.py:80:20:80:25 | ControlFlowNode for SOURCE | datamodel.py:80:6:80:26 | ControlFlowNode for Attribute() | | datamodel.py:80:20:80:25 | ControlFlowNode for SOURCE | datamodel.py:81:20:81:25 | ControlFlowNode for SOURCE | | datamodel.py:81:20:81:25 | ControlFlowNode for SOURCE | datamodel.py:81:6:81:26 | ControlFlowNode for Attribute() | +| datamodel.py:152:5:152:8 | [post store] ControlFlowNode for self [Attribute b] | datamodel.py:155:14:155:25 | ControlFlowNode for Customized() [Attribute b] | +| datamodel.py:152:14:152:19 | ControlFlowNode for SOURCE | datamodel.py:152:5:152:8 | [post store] ControlFlowNode for self [Attribute b] | +| datamodel.py:155:14:155:25 | ControlFlowNode for Customized() [Attribute b] | datamodel.py:159:6:159:15 | ControlFlowNode for customized [Attribute b] | +| datamodel.py:159:6:159:15 | ControlFlowNode for customized [Attribute b] | datamodel.py:159:6:159:17 | ControlFlowNode for Attribute | | test.py:0:0:0:0 | ModuleVariableNode for Global Variable SOURCE in Module test | test.py:36:21:36:26 | ControlFlowNode for SOURCE | | test.py:0:0:0:0 | ModuleVariableNode for Global Variable SOURCE in Module test | test.py:50:9:50:14 | ControlFlowNode for SOURCE | | test.py:0:0:0:0 | ModuleVariableNode for Global Variable SOURCE in Module test | test.py:84:10:84:15 | ControlFlowNode for SOURCE | @@ -145,6 +152,8 @@ edges | test.py:504:9:504:14 | ControlFlowNode for SOURCE | test.py:506:10:506:10 | ControlFlowNode for a | | test.py:504:9:504:14 | ControlFlowNode for SOURCE | test.py:511:10:511:10 | ControlFlowNode for b | nodes +| datamodel.py:0:0:0:0 | ModuleVariableNode for Global Variable SOURCE in Module datamodel | semmle.label | ModuleVariableNode for Global Variable SOURCE in Module datamodel | +| datamodel.py:13:1:13:6 | GSSA Variable SOURCE | semmle.label | GSSA Variable SOURCE | | datamodel.py:13:10:13:17 | ControlFlowNode for Str | semmle.label | ControlFlowNode for Str | | datamodel.py:38:6:38:17 | ControlFlowNode for f() | semmle.label | ControlFlowNode for f() | | datamodel.py:38:8:38:13 | ControlFlowNode for SOURCE | semmle.label | ControlFlowNode for SOURCE | @@ -157,6 +166,11 @@ nodes | datamodel.py:80:20:80:25 | ControlFlowNode for SOURCE | semmle.label | ControlFlowNode for SOURCE | | datamodel.py:81:6:81:26 | ControlFlowNode for Attribute() | semmle.label | ControlFlowNode for Attribute() | | datamodel.py:81:20:81:25 | ControlFlowNode for SOURCE | semmle.label | ControlFlowNode for SOURCE | +| datamodel.py:152:5:152:8 | [post store] ControlFlowNode for self [Attribute b] | semmle.label | [post store] ControlFlowNode for self [Attribute b] | +| datamodel.py:152:14:152:19 | ControlFlowNode for SOURCE | semmle.label | ControlFlowNode for SOURCE | +| datamodel.py:155:14:155:25 | ControlFlowNode for Customized() [Attribute b] | semmle.label | ControlFlowNode for Customized() [Attribute b] | +| datamodel.py:159:6:159:15 | ControlFlowNode for customized [Attribute b] | semmle.label | ControlFlowNode for customized [Attribute b] | +| datamodel.py:159:6:159:17 | ControlFlowNode for Attribute | semmle.label | ControlFlowNode for Attribute | | test.py:0:0:0:0 | ModuleVariableNode for Global Variable SOURCE in Module test | semmle.label | ModuleVariableNode for Global Variable SOURCE in Module test | | test.py:14:1:14:6 | GSSA Variable SOURCE | semmle.label | GSSA Variable SOURCE | | test.py:14:10:14:17 | ControlFlowNode for Str | semmle.label | ControlFlowNode for Str | @@ -286,77 +300,79 @@ nodes | test.py:506:10:506:10 | ControlFlowNode for a | semmle.label | ControlFlowNode for a | | test.py:511:10:511:10 | ControlFlowNode for b | semmle.label | ControlFlowNode for b | #select -| datamodel.py:38:6:38:17 | ControlFlowNode for f() | datamodel.py:13:10:13:17 | ControlFlowNode for Str | datamodel.py:38:6:38:17 | ControlFlowNode for f() | | -| datamodel.py:38:6:38:17 | ControlFlowNode for f() | datamodel.py:38:8:38:13 | ControlFlowNode for SOURCE | datamodel.py:38:6:38:17 | ControlFlowNode for f() | | -| datamodel.py:71:6:71:24 | ControlFlowNode for Attribute() | datamodel.py:13:10:13:17 | ControlFlowNode for Str | datamodel.py:71:6:71:24 | ControlFlowNode for Attribute() | | -| datamodel.py:71:6:71:24 | ControlFlowNode for Attribute() | datamodel.py:38:8:38:13 | ControlFlowNode for SOURCE | datamodel.py:71:6:71:24 | ControlFlowNode for Attribute() | | -| datamodel.py:71:6:71:24 | ControlFlowNode for Attribute() | datamodel.py:71:15:71:20 | ControlFlowNode for SOURCE | datamodel.py:71:6:71:24 | ControlFlowNode for Attribute() | | -| datamodel.py:72:6:72:27 | ControlFlowNode for Attribute() | datamodel.py:13:10:13:17 | ControlFlowNode for Str | datamodel.py:72:6:72:27 | ControlFlowNode for Attribute() | | -| datamodel.py:72:6:72:27 | ControlFlowNode for Attribute() | datamodel.py:38:8:38:13 | ControlFlowNode for SOURCE | datamodel.py:72:6:72:27 | ControlFlowNode for Attribute() | | -| datamodel.py:72:6:72:27 | ControlFlowNode for Attribute() | datamodel.py:71:15:71:20 | ControlFlowNode for SOURCE | datamodel.py:72:6:72:27 | ControlFlowNode for Attribute() | | -| datamodel.py:72:6:72:27 | ControlFlowNode for Attribute() | datamodel.py:72:18:72:23 | ControlFlowNode for SOURCE | datamodel.py:72:6:72:27 | ControlFlowNode for Attribute() | | -| datamodel.py:80:6:80:26 | ControlFlowNode for Attribute() | datamodel.py:13:10:13:17 | ControlFlowNode for Str | datamodel.py:80:6:80:26 | ControlFlowNode for Attribute() | | -| datamodel.py:80:6:80:26 | ControlFlowNode for Attribute() | datamodel.py:38:8:38:13 | ControlFlowNode for SOURCE | datamodel.py:80:6:80:26 | ControlFlowNode for Attribute() | | -| datamodel.py:80:6:80:26 | ControlFlowNode for Attribute() | datamodel.py:71:15:71:20 | ControlFlowNode for SOURCE | datamodel.py:80:6:80:26 | ControlFlowNode for Attribute() | | -| datamodel.py:80:6:80:26 | ControlFlowNode for Attribute() | datamodel.py:72:18:72:23 | ControlFlowNode for SOURCE | datamodel.py:80:6:80:26 | ControlFlowNode for Attribute() | | -| datamodel.py:80:6:80:26 | ControlFlowNode for Attribute() | datamodel.py:73:18:73:23 | ControlFlowNode for SOURCE | datamodel.py:80:6:80:26 | ControlFlowNode for Attribute() | | -| datamodel.py:80:6:80:26 | ControlFlowNode for Attribute() | datamodel.py:80:20:80:25 | ControlFlowNode for SOURCE | datamodel.py:80:6:80:26 | ControlFlowNode for Attribute() | | -| datamodel.py:81:6:81:26 | ControlFlowNode for Attribute() | datamodel.py:13:10:13:17 | ControlFlowNode for Str | datamodel.py:81:6:81:26 | ControlFlowNode for Attribute() | | -| datamodel.py:81:6:81:26 | ControlFlowNode for Attribute() | datamodel.py:38:8:38:13 | ControlFlowNode for SOURCE | datamodel.py:81:6:81:26 | ControlFlowNode for Attribute() | | -| datamodel.py:81:6:81:26 | ControlFlowNode for Attribute() | datamodel.py:71:15:71:20 | ControlFlowNode for SOURCE | datamodel.py:81:6:81:26 | ControlFlowNode for Attribute() | | -| datamodel.py:81:6:81:26 | ControlFlowNode for Attribute() | datamodel.py:72:18:72:23 | ControlFlowNode for SOURCE | datamodel.py:81:6:81:26 | ControlFlowNode for Attribute() | | -| datamodel.py:81:6:81:26 | ControlFlowNode for Attribute() | datamodel.py:73:18:73:23 | ControlFlowNode for SOURCE | datamodel.py:81:6:81:26 | ControlFlowNode for Attribute() | | -| datamodel.py:81:6:81:26 | ControlFlowNode for Attribute() | datamodel.py:80:20:80:25 | ControlFlowNode for SOURCE | datamodel.py:81:6:81:26 | ControlFlowNode for Attribute() | | -| datamodel.py:81:6:81:26 | ControlFlowNode for Attribute() | datamodel.py:81:20:81:25 | ControlFlowNode for SOURCE | datamodel.py:81:6:81:26 | ControlFlowNode for Attribute() | | -| test.py:38:10:38:10 | ControlFlowNode for y | test.py:14:10:14:17 | ControlFlowNode for Str | test.py:38:10:38:10 | ControlFlowNode for y | | -| test.py:38:10:38:10 | ControlFlowNode for y | test.py:36:21:36:26 | ControlFlowNode for SOURCE | test.py:38:10:38:10 | ControlFlowNode for y | | -| test.py:51:10:51:10 | ControlFlowNode for x | test.py:14:10:14:17 | ControlFlowNode for Str | test.py:51:10:51:10 | ControlFlowNode for x | | -| test.py:51:10:51:10 | ControlFlowNode for x | test.py:50:9:50:14 | ControlFlowNode for SOURCE | test.py:51:10:51:10 | ControlFlowNode for x | | -| test.py:58:10:58:10 | ControlFlowNode for x | test.py:57:9:57:16 | ControlFlowNode for Str | test.py:58:10:58:10 | ControlFlowNode for x | | -| test.py:63:10:63:10 | ControlFlowNode for x | test.py:62:9:62:17 | ControlFlowNode for Str | test.py:63:10:63:10 | ControlFlowNode for x | | -| test.py:68:10:68:10 | ControlFlowNode for x | test.py:67:9:67:10 | ControlFlowNode for IntegerLiteral | test.py:68:10:68:10 | ControlFlowNode for x | | -| test.py:73:10:73:10 | ControlFlowNode for x | test.py:72:9:72:12 | ControlFlowNode for FloatLiteral | test.py:73:10:73:10 | ControlFlowNode for x | | -| test.py:85:10:85:10 | ControlFlowNode for x | test.py:14:10:14:17 | ControlFlowNode for Str | test.py:85:10:85:10 | ControlFlowNode for x | | -| test.py:85:10:85:10 | ControlFlowNode for x | test.py:84:10:84:15 | ControlFlowNode for SOURCE | test.py:85:10:85:10 | ControlFlowNode for x | | -| test.py:92:10:92:13 | ControlFlowNode for Subscript | test.py:14:10:14:17 | ControlFlowNode for Str | test.py:92:10:92:13 | ControlFlowNode for Subscript | | -| test.py:92:10:92:13 | ControlFlowNode for Subscript | test.py:91:10:91:15 | ControlFlowNode for SOURCE | test.py:92:10:92:13 | ControlFlowNode for Subscript | | -| test.py:102:10:102:13 | ControlFlowNode for Subscript | test.py:14:10:14:17 | ControlFlowNode for Str | test.py:102:10:102:13 | ControlFlowNode for Subscript | | -| test.py:102:10:102:13 | ControlFlowNode for Subscript | test.py:101:10:101:15 | ControlFlowNode for SOURCE | test.py:102:10:102:13 | ControlFlowNode for Subscript | | -| test.py:107:10:107:13 | ControlFlowNode for Subscript | test.py:14:10:14:17 | ControlFlowNode for Str | test.py:107:10:107:13 | ControlFlowNode for Subscript | | -| test.py:107:10:107:13 | ControlFlowNode for Subscript | test.py:106:22:106:27 | ControlFlowNode for SOURCE | test.py:107:10:107:13 | ControlFlowNode for Subscript | | -| test.py:113:10:113:13 | ControlFlowNode for Subscript | test.py:14:10:14:17 | ControlFlowNode for Str | test.py:113:10:113:13 | ControlFlowNode for Subscript | | -| test.py:113:10:113:13 | ControlFlowNode for Subscript | test.py:111:10:111:15 | ControlFlowNode for SOURCE | test.py:113:10:113:13 | ControlFlowNode for Subscript | | -| test.py:125:10:125:16 | ControlFlowNode for Attribute() | test.py:14:10:14:17 | ControlFlowNode for Str | test.py:125:10:125:16 | ControlFlowNode for Attribute() | | -| test.py:125:10:125:16 | ControlFlowNode for Attribute() | test.py:124:10:124:15 | ControlFlowNode for SOURCE | test.py:125:10:125:16 | ControlFlowNode for Attribute() | | -| test.py:130:10:130:16 | ControlFlowNode for Attribute() | test.py:14:10:14:17 | ControlFlowNode for Str | test.py:130:10:130:16 | ControlFlowNode for Attribute() | | -| test.py:130:10:130:16 | ControlFlowNode for Attribute() | test.py:129:10:129:15 | ControlFlowNode for SOURCE | test.py:130:10:130:16 | ControlFlowNode for Attribute() | | -| test.py:135:10:135:16 | ControlFlowNode for Attribute() | test.py:14:10:14:17 | ControlFlowNode for Str | test.py:135:10:135:16 | ControlFlowNode for Attribute() | | -| test.py:135:10:135:16 | ControlFlowNode for Attribute() | test.py:134:22:134:27 | ControlFlowNode for SOURCE | test.py:135:10:135:16 | ControlFlowNode for Attribute() | | -| test.py:141:10:141:16 | ControlFlowNode for Attribute() | test.py:14:10:14:17 | ControlFlowNode for Str | test.py:141:10:141:16 | ControlFlowNode for Attribute() | | -| test.py:141:10:141:16 | ControlFlowNode for Attribute() | test.py:139:10:139:15 | ControlFlowNode for SOURCE | test.py:141:10:141:16 | ControlFlowNode for Attribute() | | -| test.py:153:10:153:15 | ControlFlowNode for Subscript | test.py:14:10:14:17 | ControlFlowNode for Str | test.py:153:10:153:15 | ControlFlowNode for Subscript | | -| test.py:153:10:153:15 | ControlFlowNode for Subscript | test.py:152:15:152:20 | ControlFlowNode for SOURCE | test.py:153:10:153:15 | ControlFlowNode for Subscript | | -| test.py:158:10:158:19 | ControlFlowNode for Attribute() | test.py:14:10:14:17 | ControlFlowNode for Str | test.py:158:10:158:19 | ControlFlowNode for Attribute() | | -| test.py:158:10:158:19 | ControlFlowNode for Attribute() | test.py:157:15:157:20 | ControlFlowNode for SOURCE | test.py:158:10:158:19 | ControlFlowNode for Attribute() | | -| test.py:185:10:185:13 | ControlFlowNode for Subscript | test.py:14:10:14:17 | ControlFlowNode for Str | test.py:185:10:185:13 | ControlFlowNode for Subscript | | -| test.py:185:10:185:13 | ControlFlowNode for Subscript | test.py:184:23:184:28 | ControlFlowNode for SOURCE | test.py:185:10:185:13 | ControlFlowNode for Subscript | | -| test.py:190:10:190:13 | ControlFlowNode for Subscript | test.py:14:10:14:17 | ControlFlowNode for Str | test.py:190:10:190:13 | ControlFlowNode for Subscript | | -| test.py:190:10:190:13 | ControlFlowNode for Subscript | test.py:189:25:189:30 | ControlFlowNode for SOURCE | test.py:190:10:190:13 | ControlFlowNode for Subscript | | -| test.py:201:10:201:13 | ControlFlowNode for Subscript | test.py:14:10:14:17 | ControlFlowNode for Str | test.py:201:10:201:13 | ControlFlowNode for Subscript | | -| test.py:201:10:201:13 | ControlFlowNode for Subscript | test.py:200:34:200:39 | ControlFlowNode for SOURCE | test.py:201:10:201:13 | ControlFlowNode for Subscript | | -| test.py:344:10:344:21 | ControlFlowNode for Subscript | test.py:14:10:14:17 | ControlFlowNode for Str | test.py:344:10:344:21 | ControlFlowNode for Subscript | | -| test.py:344:10:344:21 | ControlFlowNode for Subscript | test.py:344:11:344:16 | ControlFlowNode for SOURCE | test.py:344:10:344:21 | ControlFlowNode for Subscript | | -| test.py:348:10:348:20 | ControlFlowNode for Subscript | test.py:14:10:14:17 | ControlFlowNode for Str | test.py:348:10:348:20 | ControlFlowNode for Subscript | | -| test.py:348:10:348:20 | ControlFlowNode for Subscript | test.py:348:11:348:16 | ControlFlowNode for SOURCE | test.py:348:10:348:20 | ControlFlowNode for Subscript | | -| test.py:352:10:352:27 | ControlFlowNode for Subscript | test.py:14:10:14:17 | ControlFlowNode for Str | test.py:352:10:352:27 | ControlFlowNode for Subscript | | -| test.py:352:10:352:27 | ControlFlowNode for Subscript | test.py:352:16:352:21 | ControlFlowNode for SOURCE | test.py:352:10:352:27 | ControlFlowNode for Subscript | | -| test.py:375:10:375:34 | ControlFlowNode for second() | test.py:14:10:14:17 | ControlFlowNode for Str | test.py:375:10:375:34 | ControlFlowNode for second() | | -| test.py:375:10:375:34 | ControlFlowNode for second() | test.py:375:28:375:33 | ControlFlowNode for SOURCE | test.py:375:10:375:34 | ControlFlowNode for second() | | -| test.py:457:10:457:18 | ControlFlowNode for f() | test.py:14:10:14:17 | ControlFlowNode for Str | test.py:457:10:457:18 | ControlFlowNode for f() | | -| test.py:457:10:457:18 | ControlFlowNode for f() | test.py:457:12:457:17 | ControlFlowNode for SOURCE | test.py:457:10:457:18 | ControlFlowNode for f() | | -| test.py:462:10:462:34 | ControlFlowNode for second() | test.py:14:10:14:17 | ControlFlowNode for Str | test.py:462:10:462:34 | ControlFlowNode for second() | | -| test.py:462:10:462:34 | ControlFlowNode for second() | test.py:462:28:462:33 | ControlFlowNode for SOURCE | test.py:462:10:462:34 | ControlFlowNode for second() | | -| test.py:506:10:506:10 | ControlFlowNode for a | test.py:14:10:14:17 | ControlFlowNode for Str | test.py:506:10:506:10 | ControlFlowNode for a | | -| test.py:506:10:506:10 | ControlFlowNode for a | test.py:504:9:504:14 | ControlFlowNode for SOURCE | test.py:506:10:506:10 | ControlFlowNode for a | | -| test.py:511:10:511:10 | ControlFlowNode for b | test.py:14:10:14:17 | ControlFlowNode for Str | test.py:511:10:511:10 | ControlFlowNode for b | | -| test.py:511:10:511:10 | ControlFlowNode for b | test.py:504:9:504:14 | ControlFlowNode for SOURCE | test.py:511:10:511:10 | ControlFlowNode for b | | +| datamodel.py:38:6:38:17 | ControlFlowNode for f() | datamodel.py:13:10:13:17 | ControlFlowNode for Str | datamodel.py:38:6:38:17 | ControlFlowNode for f() | Flow found | +| datamodel.py:38:6:38:17 | ControlFlowNode for f() | datamodel.py:38:8:38:13 | ControlFlowNode for SOURCE | datamodel.py:38:6:38:17 | ControlFlowNode for f() | Flow found | +| datamodel.py:71:6:71:24 | ControlFlowNode for Attribute() | datamodel.py:13:10:13:17 | ControlFlowNode for Str | datamodel.py:71:6:71:24 | ControlFlowNode for Attribute() | Flow found | +| datamodel.py:71:6:71:24 | ControlFlowNode for Attribute() | datamodel.py:38:8:38:13 | ControlFlowNode for SOURCE | datamodel.py:71:6:71:24 | ControlFlowNode for Attribute() | Flow found | +| datamodel.py:71:6:71:24 | ControlFlowNode for Attribute() | datamodel.py:71:15:71:20 | ControlFlowNode for SOURCE | datamodel.py:71:6:71:24 | ControlFlowNode for Attribute() | Flow found | +| datamodel.py:72:6:72:27 | ControlFlowNode for Attribute() | datamodel.py:13:10:13:17 | ControlFlowNode for Str | datamodel.py:72:6:72:27 | ControlFlowNode for Attribute() | Flow found | +| datamodel.py:72:6:72:27 | ControlFlowNode for Attribute() | datamodel.py:38:8:38:13 | ControlFlowNode for SOURCE | datamodel.py:72:6:72:27 | ControlFlowNode for Attribute() | Flow found | +| datamodel.py:72:6:72:27 | ControlFlowNode for Attribute() | datamodel.py:71:15:71:20 | ControlFlowNode for SOURCE | datamodel.py:72:6:72:27 | ControlFlowNode for Attribute() | Flow found | +| datamodel.py:72:6:72:27 | ControlFlowNode for Attribute() | datamodel.py:72:18:72:23 | ControlFlowNode for SOURCE | datamodel.py:72:6:72:27 | ControlFlowNode for Attribute() | Flow found | +| datamodel.py:80:6:80:26 | ControlFlowNode for Attribute() | datamodel.py:13:10:13:17 | ControlFlowNode for Str | datamodel.py:80:6:80:26 | ControlFlowNode for Attribute() | Flow found | +| datamodel.py:80:6:80:26 | ControlFlowNode for Attribute() | datamodel.py:38:8:38:13 | ControlFlowNode for SOURCE | datamodel.py:80:6:80:26 | ControlFlowNode for Attribute() | Flow found | +| datamodel.py:80:6:80:26 | ControlFlowNode for Attribute() | datamodel.py:71:15:71:20 | ControlFlowNode for SOURCE | datamodel.py:80:6:80:26 | ControlFlowNode for Attribute() | Flow found | +| datamodel.py:80:6:80:26 | ControlFlowNode for Attribute() | datamodel.py:72:18:72:23 | ControlFlowNode for SOURCE | datamodel.py:80:6:80:26 | ControlFlowNode for Attribute() | Flow found | +| datamodel.py:80:6:80:26 | ControlFlowNode for Attribute() | datamodel.py:73:18:73:23 | ControlFlowNode for SOURCE | datamodel.py:80:6:80:26 | ControlFlowNode for Attribute() | Flow found | +| datamodel.py:80:6:80:26 | ControlFlowNode for Attribute() | datamodel.py:80:20:80:25 | ControlFlowNode for SOURCE | datamodel.py:80:6:80:26 | ControlFlowNode for Attribute() | Flow found | +| datamodel.py:81:6:81:26 | ControlFlowNode for Attribute() | datamodel.py:13:10:13:17 | ControlFlowNode for Str | datamodel.py:81:6:81:26 | ControlFlowNode for Attribute() | Flow found | +| datamodel.py:81:6:81:26 | ControlFlowNode for Attribute() | datamodel.py:38:8:38:13 | ControlFlowNode for SOURCE | datamodel.py:81:6:81:26 | ControlFlowNode for Attribute() | Flow found | +| datamodel.py:81:6:81:26 | ControlFlowNode for Attribute() | datamodel.py:71:15:71:20 | ControlFlowNode for SOURCE | datamodel.py:81:6:81:26 | ControlFlowNode for Attribute() | Flow found | +| datamodel.py:81:6:81:26 | ControlFlowNode for Attribute() | datamodel.py:72:18:72:23 | ControlFlowNode for SOURCE | datamodel.py:81:6:81:26 | ControlFlowNode for Attribute() | Flow found | +| datamodel.py:81:6:81:26 | ControlFlowNode for Attribute() | datamodel.py:73:18:73:23 | ControlFlowNode for SOURCE | datamodel.py:81:6:81:26 | ControlFlowNode for Attribute() | Flow found | +| datamodel.py:81:6:81:26 | ControlFlowNode for Attribute() | datamodel.py:80:20:80:25 | ControlFlowNode for SOURCE | datamodel.py:81:6:81:26 | ControlFlowNode for Attribute() | Flow found | +| datamodel.py:81:6:81:26 | ControlFlowNode for Attribute() | datamodel.py:81:20:81:25 | ControlFlowNode for SOURCE | datamodel.py:81:6:81:26 | ControlFlowNode for Attribute() | Flow found | +| datamodel.py:159:6:159:17 | ControlFlowNode for Attribute | datamodel.py:13:10:13:17 | ControlFlowNode for Str | datamodel.py:159:6:159:17 | ControlFlowNode for Attribute | Flow found | +| datamodel.py:159:6:159:17 | ControlFlowNode for Attribute | datamodel.py:152:14:152:19 | ControlFlowNode for SOURCE | datamodel.py:159:6:159:17 | ControlFlowNode for Attribute | Flow found | +| test.py:38:10:38:10 | ControlFlowNode for y | test.py:14:10:14:17 | ControlFlowNode for Str | test.py:38:10:38:10 | ControlFlowNode for y | Flow found | +| test.py:38:10:38:10 | ControlFlowNode for y | test.py:36:21:36:26 | ControlFlowNode for SOURCE | test.py:38:10:38:10 | ControlFlowNode for y | Flow found | +| test.py:51:10:51:10 | ControlFlowNode for x | test.py:14:10:14:17 | ControlFlowNode for Str | test.py:51:10:51:10 | ControlFlowNode for x | Flow found | +| test.py:51:10:51:10 | ControlFlowNode for x | test.py:50:9:50:14 | ControlFlowNode for SOURCE | test.py:51:10:51:10 | ControlFlowNode for x | Flow found | +| test.py:58:10:58:10 | ControlFlowNode for x | test.py:57:9:57:16 | ControlFlowNode for Str | test.py:58:10:58:10 | ControlFlowNode for x | Flow found | +| test.py:63:10:63:10 | ControlFlowNode for x | test.py:62:9:62:17 | ControlFlowNode for Str | test.py:63:10:63:10 | ControlFlowNode for x | Flow found | +| test.py:68:10:68:10 | ControlFlowNode for x | test.py:67:9:67:10 | ControlFlowNode for IntegerLiteral | test.py:68:10:68:10 | ControlFlowNode for x | Flow found | +| test.py:73:10:73:10 | ControlFlowNode for x | test.py:72:9:72:12 | ControlFlowNode for FloatLiteral | test.py:73:10:73:10 | ControlFlowNode for x | Flow found | +| test.py:85:10:85:10 | ControlFlowNode for x | test.py:14:10:14:17 | ControlFlowNode for Str | test.py:85:10:85:10 | ControlFlowNode for x | Flow found | +| test.py:85:10:85:10 | ControlFlowNode for x | test.py:84:10:84:15 | ControlFlowNode for SOURCE | test.py:85:10:85:10 | ControlFlowNode for x | Flow found | +| test.py:92:10:92:13 | ControlFlowNode for Subscript | test.py:14:10:14:17 | ControlFlowNode for Str | test.py:92:10:92:13 | ControlFlowNode for Subscript | Flow found | +| test.py:92:10:92:13 | ControlFlowNode for Subscript | test.py:91:10:91:15 | ControlFlowNode for SOURCE | test.py:92:10:92:13 | ControlFlowNode for Subscript | Flow found | +| test.py:102:10:102:13 | ControlFlowNode for Subscript | test.py:14:10:14:17 | ControlFlowNode for Str | test.py:102:10:102:13 | ControlFlowNode for Subscript | Flow found | +| test.py:102:10:102:13 | ControlFlowNode for Subscript | test.py:101:10:101:15 | ControlFlowNode for SOURCE | test.py:102:10:102:13 | ControlFlowNode for Subscript | Flow found | +| test.py:107:10:107:13 | ControlFlowNode for Subscript | test.py:14:10:14:17 | ControlFlowNode for Str | test.py:107:10:107:13 | ControlFlowNode for Subscript | Flow found | +| test.py:107:10:107:13 | ControlFlowNode for Subscript | test.py:106:22:106:27 | ControlFlowNode for SOURCE | test.py:107:10:107:13 | ControlFlowNode for Subscript | Flow found | +| test.py:113:10:113:13 | ControlFlowNode for Subscript | test.py:14:10:14:17 | ControlFlowNode for Str | test.py:113:10:113:13 | ControlFlowNode for Subscript | Flow found | +| test.py:113:10:113:13 | ControlFlowNode for Subscript | test.py:111:10:111:15 | ControlFlowNode for SOURCE | test.py:113:10:113:13 | ControlFlowNode for Subscript | Flow found | +| test.py:125:10:125:16 | ControlFlowNode for Attribute() | test.py:14:10:14:17 | ControlFlowNode for Str | test.py:125:10:125:16 | ControlFlowNode for Attribute() | Flow found | +| test.py:125:10:125:16 | ControlFlowNode for Attribute() | test.py:124:10:124:15 | ControlFlowNode for SOURCE | test.py:125:10:125:16 | ControlFlowNode for Attribute() | Flow found | +| test.py:130:10:130:16 | ControlFlowNode for Attribute() | test.py:14:10:14:17 | ControlFlowNode for Str | test.py:130:10:130:16 | ControlFlowNode for Attribute() | Flow found | +| test.py:130:10:130:16 | ControlFlowNode for Attribute() | test.py:129:10:129:15 | ControlFlowNode for SOURCE | test.py:130:10:130:16 | ControlFlowNode for Attribute() | Flow found | +| test.py:135:10:135:16 | ControlFlowNode for Attribute() | test.py:14:10:14:17 | ControlFlowNode for Str | test.py:135:10:135:16 | ControlFlowNode for Attribute() | Flow found | +| test.py:135:10:135:16 | ControlFlowNode for Attribute() | test.py:134:22:134:27 | ControlFlowNode for SOURCE | test.py:135:10:135:16 | ControlFlowNode for Attribute() | Flow found | +| test.py:141:10:141:16 | ControlFlowNode for Attribute() | test.py:14:10:14:17 | ControlFlowNode for Str | test.py:141:10:141:16 | ControlFlowNode for Attribute() | Flow found | +| test.py:141:10:141:16 | ControlFlowNode for Attribute() | test.py:139:10:139:15 | ControlFlowNode for SOURCE | test.py:141:10:141:16 | ControlFlowNode for Attribute() | Flow found | +| test.py:153:10:153:15 | ControlFlowNode for Subscript | test.py:14:10:14:17 | ControlFlowNode for Str | test.py:153:10:153:15 | ControlFlowNode for Subscript | Flow found | +| test.py:153:10:153:15 | ControlFlowNode for Subscript | test.py:152:15:152:20 | ControlFlowNode for SOURCE | test.py:153:10:153:15 | ControlFlowNode for Subscript | Flow found | +| test.py:158:10:158:19 | ControlFlowNode for Attribute() | test.py:14:10:14:17 | ControlFlowNode for Str | test.py:158:10:158:19 | ControlFlowNode for Attribute() | Flow found | +| test.py:158:10:158:19 | ControlFlowNode for Attribute() | test.py:157:15:157:20 | ControlFlowNode for SOURCE | test.py:158:10:158:19 | ControlFlowNode for Attribute() | Flow found | +| test.py:185:10:185:13 | ControlFlowNode for Subscript | test.py:14:10:14:17 | ControlFlowNode for Str | test.py:185:10:185:13 | ControlFlowNode for Subscript | Flow found | +| test.py:185:10:185:13 | ControlFlowNode for Subscript | test.py:184:23:184:28 | ControlFlowNode for SOURCE | test.py:185:10:185:13 | ControlFlowNode for Subscript | Flow found | +| test.py:190:10:190:13 | ControlFlowNode for Subscript | test.py:14:10:14:17 | ControlFlowNode for Str | test.py:190:10:190:13 | ControlFlowNode for Subscript | Flow found | +| test.py:190:10:190:13 | ControlFlowNode for Subscript | test.py:189:25:189:30 | ControlFlowNode for SOURCE | test.py:190:10:190:13 | ControlFlowNode for Subscript | Flow found | +| test.py:201:10:201:13 | ControlFlowNode for Subscript | test.py:14:10:14:17 | ControlFlowNode for Str | test.py:201:10:201:13 | ControlFlowNode for Subscript | Flow found | +| test.py:201:10:201:13 | ControlFlowNode for Subscript | test.py:200:34:200:39 | ControlFlowNode for SOURCE | test.py:201:10:201:13 | ControlFlowNode for Subscript | Flow found | +| test.py:344:10:344:21 | ControlFlowNode for Subscript | test.py:14:10:14:17 | ControlFlowNode for Str | test.py:344:10:344:21 | ControlFlowNode for Subscript | Flow found | +| test.py:344:10:344:21 | ControlFlowNode for Subscript | test.py:344:11:344:16 | ControlFlowNode for SOURCE | test.py:344:10:344:21 | ControlFlowNode for Subscript | Flow found | +| test.py:348:10:348:20 | ControlFlowNode for Subscript | test.py:14:10:14:17 | ControlFlowNode for Str | test.py:348:10:348:20 | ControlFlowNode for Subscript | Flow found | +| test.py:348:10:348:20 | ControlFlowNode for Subscript | test.py:348:11:348:16 | ControlFlowNode for SOURCE | test.py:348:10:348:20 | ControlFlowNode for Subscript | Flow found | +| test.py:352:10:352:27 | ControlFlowNode for Subscript | test.py:14:10:14:17 | ControlFlowNode for Str | test.py:352:10:352:27 | ControlFlowNode for Subscript | Flow found | +| test.py:352:10:352:27 | ControlFlowNode for Subscript | test.py:352:16:352:21 | ControlFlowNode for SOURCE | test.py:352:10:352:27 | ControlFlowNode for Subscript | Flow found | +| test.py:375:10:375:34 | ControlFlowNode for second() | test.py:14:10:14:17 | ControlFlowNode for Str | test.py:375:10:375:34 | ControlFlowNode for second() | Flow found | +| test.py:375:10:375:34 | ControlFlowNode for second() | test.py:375:28:375:33 | ControlFlowNode for SOURCE | test.py:375:10:375:34 | ControlFlowNode for second() | Flow found | +| test.py:457:10:457:18 | ControlFlowNode for f() | test.py:14:10:14:17 | ControlFlowNode for Str | test.py:457:10:457:18 | ControlFlowNode for f() | Flow found | +| test.py:457:10:457:18 | ControlFlowNode for f() | test.py:457:12:457:17 | ControlFlowNode for SOURCE | test.py:457:10:457:18 | ControlFlowNode for f() | Flow found | +| test.py:462:10:462:34 | ControlFlowNode for second() | test.py:14:10:14:17 | ControlFlowNode for Str | test.py:462:10:462:34 | ControlFlowNode for second() | Flow found | +| test.py:462:10:462:34 | ControlFlowNode for second() | test.py:462:28:462:33 | ControlFlowNode for SOURCE | test.py:462:10:462:34 | ControlFlowNode for second() | Flow found | +| test.py:506:10:506:10 | ControlFlowNode for a | test.py:14:10:14:17 | ControlFlowNode for Str | test.py:506:10:506:10 | ControlFlowNode for a | Flow found | +| test.py:506:10:506:10 | ControlFlowNode for a | test.py:504:9:504:14 | ControlFlowNode for SOURCE | test.py:506:10:506:10 | ControlFlowNode for a | Flow found | +| test.py:511:10:511:10 | ControlFlowNode for b | test.py:14:10:14:17 | ControlFlowNode for Str | test.py:511:10:511:10 | ControlFlowNode for b | Flow found | +| test.py:511:10:511:10 | ControlFlowNode for b | test.py:504:9:504:14 | ControlFlowNode for SOURCE | test.py:511:10:511:10 | ControlFlowNode for b | Flow found | diff --git a/python/ql/test/experimental/dataflow/coverage/dataflow.ql b/python/ql/test/experimental/dataflow/coverage/dataflow.ql index 18b21324e47..868f24a598f 100644 --- a/python/ql/test/experimental/dataflow/coverage/dataflow.ql +++ b/python/ql/test/experimental/dataflow/coverage/dataflow.ql @@ -8,4 +8,4 @@ import DataFlow::PathGraph from TestConfiguration config, DataFlow::PathNode source, DataFlow::PathNode sink where config.hasFlowPath(source, sink) -select sink.getNode(), source, sink, "" +select sink.getNode(), source, sink, "Flow found" diff --git a/python/ql/test/experimental/dataflow/coverage/datamodel.py b/python/ql/test/experimental/dataflow/coverage/datamodel.py index a3fc54d11a0..f5f3680dd55 100644 --- a/python/ql/test/experimental/dataflow/coverage/datamodel.py +++ b/python/ql/test/experimental/dataflow/coverage/datamodel.py @@ -156,4 +156,4 @@ customized = Customized() SINK(Customized.a) SINK_F(Customized.b) SINK(customized.a) -SINK(customized.b) +SINK(customized.b) # Flow found diff --git a/python/ql/test/experimental/dataflow/fieldflow/allLocalFlow.expected b/python/ql/test/experimental/dataflow/fieldflow/allLocalFlow.expected new file mode 100644 index 00000000000..0f33cbc6bda --- /dev/null +++ b/python/ql/test/experimental/dataflow/fieldflow/allLocalFlow.expected @@ -0,0 +1,147 @@ +| examples.py:0:0:0:0 | GSSA Variable SINK | examples.py:1:1:1:66 | GSSA Variable SINK | +| examples.py:0:0:0:0 | GSSA Variable SINK | examples.py:28:1:28:4 | ControlFlowNode for SINK | +| examples.py:0:0:0:0 | GSSA Variable SINK_F | examples.py:1:1:1:66 | GSSA Variable SINK_F | +| examples.py:0:0:0:0 | GSSA Variable SOURCE | examples.py:1:1:1:66 | GSSA Variable SOURCE | +| examples.py:0:0:0:0 | GSSA Variable SOURCE | examples.py:27:15:27:20 | ControlFlowNode for SOURCE | +| examples.py:0:0:0:0 | GSSA Variable __name__ | examples.py:1:1:1:66 | GSSA Variable __name__ | +| examples.py:0:0:0:0 | GSSA Variable __package__ | examples.py:1:1:1:66 | GSSA Variable __package__ | +| examples.py:0:0:0:0 | GSSA Variable a | examples.py:1:1:1:66 | GSSA Variable a | +| examples.py:0:0:0:0 | GSSA Variable fields_with_local_flow | examples.py:1:1:1:66 | GSSA Variable fields_with_local_flow | +| examples.py:0:0:0:0 | GSSA Variable myobj | examples.py:1:1:1:66 | GSSA Variable myobj | +| examples.py:0:0:0:0 | GSSA Variable obj | examples.py:1:1:1:66 | GSSA Variable obj | +| examples.py:0:0:0:0 | GSSA Variable object | examples.py:1:1:1:66 | GSSA Variable object | +| examples.py:0:0:0:0 | GSSA Variable object | examples.py:5:13:5:18 | ControlFlowNode for object | +| examples.py:0:0:0:0 | GSSA Variable x | examples.py:1:1:1:66 | GSSA Variable x | +| examples.py:0:0:0:0 | SSA variable $ | examples.py:1:1:1:66 | SSA variable $ | +| examples.py:0:0:0:0 | SSA variable * | examples.py:1:1:1:66 | SSA variable * | +| examples.py:1:1:1:66 | GSSA Variable SOURCE | examples.py:41:7:41:19 | GSSA Variable SOURCE | +| examples.py:5:1:5:20 | ControlFlowNode for ClassExpr | examples.py:5:7:5:11 | GSSA Variable MyObj | +| examples.py:5:7:5:11 | GSSA Variable MyObj | examples.py:25:9:25:13 | ControlFlowNode for MyObj | +| examples.py:5:13:5:18 | ControlFlowNode for object | examples.py:11:17:11:22 | ControlFlowNode for object | +| examples.py:7:5:7:28 | ControlFlowNode for FunctionExpr | examples.py:7:9:7:16 | SSA variable __init__ | +| examples.py:7:18:7:21 | SSA variable self | examples.py:8:9:8:12 | ControlFlowNode for self | +| examples.py:7:18:7:21 | SSA variable self | examples.py:8:9:8:16 | SSA variable self | +| examples.py:7:24:7:26 | SSA variable foo | examples.py:8:20:8:22 | ControlFlowNode for foo | +| examples.py:11:1:11:24 | ControlFlowNode for ClassExpr | examples.py:11:7:11:15 | GSSA Variable NestedObj | +| examples.py:11:7:11:15 | GSSA Variable NestedObj | examples.py:33:5:33:13 | ControlFlowNode for NestedObj | +| examples.py:13:5:13:23 | ControlFlowNode for FunctionExpr | examples.py:13:9:13:16 | SSA variable __init__ | +| examples.py:13:5:13:23 | GSSA Variable MyObj | examples.py:14:20:14:24 | ControlFlowNode for MyObj | +| examples.py:13:18:13:21 | SSA variable self | examples.py:14:9:14:12 | ControlFlowNode for self | +| examples.py:13:18:13:21 | SSA variable self | examples.py:14:9:14:16 | SSA variable self | +| examples.py:16:5:16:21 | ControlFlowNode for FunctionExpr | examples.py:16:9:16:14 | SSA variable getObj | +| examples.py:16:16:16:19 | SSA variable self | examples.py:17:16:17:19 | ControlFlowNode for self | +| examples.py:21:1:21:19 | ControlFlowNode for FunctionExpr | examples.py:21:5:21:10 | GSSA Variable setFoo | +| examples.py:21:1:21:19 | GSSA Variable SINK_F | examples.py:22:5:22:10 | ControlFlowNode for SINK_F | +| examples.py:21:5:21:10 | GSSA Variable setFoo | examples.py:27:1:27:6 | ControlFlowNode for setFoo | +| examples.py:21:12:21:14 | SSA variable obj | examples.py:22:12:22:14 | ControlFlowNode for obj | +| examples.py:21:12:21:14 | SSA variable obj | examples.py:23:5:23:11 | SSA variable obj | +| examples.py:21:17:21:17 | SSA variable x | examples.py:23:15:23:15 | ControlFlowNode for x | +| examples.py:22:12:22:14 | ControlFlowNode for obj | examples.py:23:5:23:7 | ControlFlowNode for obj | +| examples.py:22:12:22:14 | [post read] ControlFlowNode for obj | examples.py:23:5:23:7 | ControlFlowNode for obj | +| examples.py:25:1:25:5 | GSSA Variable myobj | examples.py:27:1:27:21 | GSSA Variable myobj | +| examples.py:25:1:25:5 | GSSA Variable myobj | examples.py:27:8:27:12 | ControlFlowNode for myobj | +| examples.py:25:9:25:13 | ControlFlowNode for MyObj | examples.py:41:7:41:11 | ControlFlowNode for MyObj | +| examples.py:25:9:25:19 | ControlFlowNode for MyObj() | examples.py:25:1:25:5 | GSSA Variable myobj | +| examples.py:27:8:27:12 | ControlFlowNode for myobj | examples.py:28:6:28:10 | ControlFlowNode for myobj | +| examples.py:27:8:27:12 | [post arg] ControlFlowNode for myobj | examples.py:28:6:28:10 | ControlFlowNode for myobj | +| examples.py:27:15:27:20 | ControlFlowNode for SOURCE | examples.py:31:5:31:10 | ControlFlowNode for SOURCE | +| examples.py:27:15:27:20 | [post arg] ControlFlowNode for SOURCE | examples.py:31:5:31:10 | ControlFlowNode for SOURCE | +| examples.py:28:1:28:4 | ControlFlowNode for SINK | examples.py:38:1:38:4 | ControlFlowNode for SINK | +| examples.py:31:1:31:1 | GSSA Variable x | examples.py:35:13:35:13 | ControlFlowNode for x | +| examples.py:31:5:31:10 | ControlFlowNode for SOURCE | examples.py:31:1:31:1 | GSSA Variable x | +| examples.py:31:5:31:10 | ControlFlowNode for SOURCE | examples.py:41:13:41:18 | ControlFlowNode for SOURCE | +| examples.py:33:1:33:1 | GSSA Variable a | examples.py:35:1:35:1 | ControlFlowNode for a | +| examples.py:33:1:33:1 | GSSA Variable a | examples.py:36:1:36:10 | GSSA Variable a | +| examples.py:33:5:33:15 | ControlFlowNode for NestedObj() | examples.py:33:1:33:1 | GSSA Variable a | +| examples.py:35:1:35:1 | ControlFlowNode for a | examples.py:36:1:36:1 | ControlFlowNode for a | +| examples.py:35:1:35:1 | [post read] ControlFlowNode for a | examples.py:36:1:36:1 | ControlFlowNode for a | +| examples.py:35:13:35:13 | ControlFlowNode for x | examples.py:36:18:36:18 | ControlFlowNode for x | +| examples.py:36:1:36:1 | ControlFlowNode for a | examples.py:38:6:38:6 | ControlFlowNode for a | +| examples.py:36:1:36:1 | [post read] ControlFlowNode for a | examples.py:38:6:38:6 | ControlFlowNode for a | +| examples.py:38:1:38:4 | ControlFlowNode for SINK | examples.py:42:1:42:4 | ControlFlowNode for SINK | +| examples.py:41:1:41:3 | GSSA Variable obj | examples.py:42:6:42:8 | ControlFlowNode for obj | +| examples.py:41:7:41:19 | ControlFlowNode for MyObj() | examples.py:41:1:41:3 | GSSA Variable obj | +| examples.py:41:7:41:19 | GSSA Variable SOURCE | examples.py:50:6:50:35 | GSSA Variable SOURCE | +| examples.py:41:13:41:18 | ControlFlowNode for SOURCE | examples.py:50:29:50:34 | ControlFlowNode for SOURCE | +| examples.py:41:13:41:18 | [post arg] ControlFlowNode for SOURCE | examples.py:50:29:50:34 | ControlFlowNode for SOURCE | +| examples.py:42:1:42:4 | ControlFlowNode for SINK | examples.py:50:1:50:4 | ControlFlowNode for SINK | +| examples.py:45:1:45:30 | ControlFlowNode for FunctionExpr | examples.py:45:5:45:26 | GSSA Variable fields_with_local_flow | +| examples.py:45:1:45:30 | GSSA Variable MyObj | examples.py:46:9:46:13 | ControlFlowNode for MyObj | +| examples.py:45:5:45:26 | GSSA Variable fields_with_local_flow | examples.py:50:6:50:27 | ControlFlowNode for fields_with_local_flow | +| examples.py:45:28:45:28 | SSA variable x | examples.py:46:9:46:16 | SSA variable x | +| examples.py:45:28:45:28 | SSA variable x | examples.py:46:15:46:15 | ControlFlowNode for x | +| examples.py:46:3:46:5 | SSA variable obj | examples.py:47:7:47:9 | ControlFlowNode for obj | +| examples.py:46:9:46:16 | ControlFlowNode for MyObj() | examples.py:46:3:46:5 | SSA variable obj | +| examples.py:47:3:47:3 | SSA variable a | examples.py:48:10:48:10 | ControlFlowNode for a | +| examples.py:47:7:47:13 | ControlFlowNode for Attribute | examples.py:47:3:47:3 | SSA variable a | +| test.py:0:0:0:0 | GSSA Variable SINK | test.py:1:1:1:66 | GSSA Variable SINK | +| test.py:0:0:0:0 | GSSA Variable SINK_F | test.py:1:1:1:66 | GSSA Variable SINK_F | +| test.py:0:0:0:0 | GSSA Variable SOURCE | test.py:1:1:1:66 | GSSA Variable SOURCE | +| test.py:0:0:0:0 | GSSA Variable __name__ | test.py:1:1:1:66 | GSSA Variable __name__ | +| test.py:0:0:0:0 | GSSA Variable __package__ | test.py:1:1:1:66 | GSSA Variable __package__ | +| test.py:0:0:0:0 | GSSA Variable object | test.py:1:1:1:66 | GSSA Variable object | +| test.py:0:0:0:0 | GSSA Variable object | test.py:6:13:6:18 | ControlFlowNode for object | +| test.py:0:0:0:0 | SSA variable $ | test.py:1:1:1:66 | SSA variable $ | +| test.py:0:0:0:0 | SSA variable * | test.py:1:1:1:66 | SSA variable * | +| test.py:6:1:6:20 | ControlFlowNode for ClassExpr | test.py:6:7:6:11 | GSSA Variable MyObj | +| test.py:6:13:6:18 | ControlFlowNode for object | test.py:12:17:12:22 | ControlFlowNode for object | +| test.py:8:5:8:28 | ControlFlowNode for FunctionExpr | test.py:8:9:8:16 | SSA variable __init__ | +| test.py:8:18:8:21 | SSA variable self | test.py:9:9:9:12 | ControlFlowNode for self | +| test.py:8:18:8:21 | SSA variable self | test.py:9:9:9:16 | SSA variable self | +| test.py:8:24:8:26 | SSA variable foo | test.py:9:20:9:22 | ControlFlowNode for foo | +| test.py:12:1:12:24 | ControlFlowNode for ClassExpr | test.py:12:7:12:15 | GSSA Variable NestedObj | +| test.py:14:5:14:23 | ControlFlowNode for FunctionExpr | test.py:14:9:14:16 | SSA variable __init__ | +| test.py:14:5:14:23 | GSSA Variable MyObj | test.py:15:20:15:24 | ControlFlowNode for MyObj | +| test.py:14:18:14:21 | SSA variable self | test.py:15:9:15:12 | ControlFlowNode for self | +| test.py:14:18:14:21 | SSA variable self | test.py:15:9:15:16 | SSA variable self | +| test.py:17:5:17:21 | ControlFlowNode for FunctionExpr | test.py:17:9:17:14 | SSA variable getObj | +| test.py:17:16:17:19 | SSA variable self | test.py:18:16:18:19 | ControlFlowNode for self | +| test.py:21:1:21:19 | ControlFlowNode for FunctionExpr | test.py:21:5:21:10 | GSSA Variable setFoo | +| test.py:21:1:21:19 | GSSA Variable SINK_F | test.py:22:5:22:10 | ControlFlowNode for SINK_F | +| test.py:21:12:21:14 | SSA variable obj | test.py:22:12:22:14 | ControlFlowNode for obj | +| test.py:21:12:21:14 | SSA variable obj | test.py:23:5:23:11 | SSA variable obj | +| test.py:21:17:21:17 | SSA variable x | test.py:23:15:23:15 | ControlFlowNode for x | +| test.py:22:12:22:14 | ControlFlowNode for obj | test.py:23:5:23:7 | ControlFlowNode for obj | +| test.py:22:12:22:14 | [post read] ControlFlowNode for obj | test.py:23:5:23:7 | ControlFlowNode for obj | +| test.py:26:1:26:20 | ControlFlowNode for FunctionExpr | test.py:26:5:26:17 | GSSA Variable test_example1 | +| test.py:26:1:26:20 | GSSA Variable MyObj | test.py:27:13:27:17 | ControlFlowNode for MyObj | +| test.py:26:1:26:20 | GSSA Variable SINK | test.py:30:5:30:8 | ControlFlowNode for SINK | +| test.py:26:1:26:20 | GSSA Variable SOURCE | test.py:29:19:29:24 | ControlFlowNode for SOURCE | +| test.py:26:1:26:20 | GSSA Variable setFoo | test.py:29:5:29:10 | ControlFlowNode for setFoo | +| test.py:27:5:27:9 | SSA variable myobj | test.py:29:5:29:25 | SSA variable myobj | +| test.py:27:5:27:9 | SSA variable myobj | test.py:29:12:29:16 | ControlFlowNode for myobj | +| test.py:27:13:27:23 | ControlFlowNode for MyObj() | test.py:27:5:27:9 | SSA variable myobj | +| test.py:29:12:29:16 | ControlFlowNode for myobj | test.py:30:10:30:14 | ControlFlowNode for myobj | +| test.py:29:12:29:16 | [post arg] ControlFlowNode for myobj | test.py:30:10:30:14 | ControlFlowNode for myobj | +| test.py:33:1:33:20 | ControlFlowNode for FunctionExpr | test.py:33:5:33:17 | GSSA Variable test_example2 | +| test.py:33:1:33:20 | GSSA Variable NestedObj | test.py:36:9:36:17 | ControlFlowNode for NestedObj | +| test.py:33:1:33:20 | GSSA Variable SINK | test.py:41:5:41:8 | ControlFlowNode for SINK | +| test.py:33:1:33:20 | GSSA Variable SOURCE | test.py:34:9:34:14 | ControlFlowNode for SOURCE | +| test.py:34:5:34:5 | SSA variable x | test.py:38:17:38:17 | ControlFlowNode for x | +| test.py:34:9:34:14 | ControlFlowNode for SOURCE | test.py:34:5:34:5 | SSA variable x | +| test.py:36:5:36:5 | SSA variable a | test.py:38:5:38:5 | ControlFlowNode for a | +| test.py:36:5:36:5 | SSA variable a | test.py:39:5:39:14 | SSA variable a | +| test.py:36:9:36:19 | ControlFlowNode for NestedObj() | test.py:36:5:36:5 | SSA variable a | +| test.py:38:5:38:5 | ControlFlowNode for a | test.py:39:5:39:5 | ControlFlowNode for a | +| test.py:38:5:38:5 | [post read] ControlFlowNode for a | test.py:39:5:39:5 | ControlFlowNode for a | +| test.py:38:17:38:17 | ControlFlowNode for x | test.py:39:22:39:22 | ControlFlowNode for x | +| test.py:39:5:39:5 | ControlFlowNode for a | test.py:41:10:41:10 | ControlFlowNode for a | +| test.py:39:5:39:5 | [post read] ControlFlowNode for a | test.py:41:10:41:10 | ControlFlowNode for a | +| test.py:44:1:44:20 | ControlFlowNode for FunctionExpr | test.py:44:5:44:17 | GSSA Variable test_example3 | +| test.py:44:1:44:20 | GSSA Variable MyObj | test.py:45:11:45:15 | ControlFlowNode for MyObj | +| test.py:44:1:44:20 | GSSA Variable SINK | test.py:46:5:46:8 | ControlFlowNode for SINK | +| test.py:44:1:44:20 | GSSA Variable SOURCE | test.py:45:17:45:22 | ControlFlowNode for SOURCE | +| test.py:45:5:45:7 | SSA variable obj | test.py:46:10:46:12 | ControlFlowNode for obj | +| test.py:45:11:45:23 | ControlFlowNode for MyObj() | test.py:45:5:45:7 | SSA variable obj | +| test.py:49:1:49:30 | ControlFlowNode for FunctionExpr | test.py:49:5:49:26 | GSSA Variable fields_with_local_flow | +| test.py:49:1:49:30 | GSSA Variable MyObj | test.py:50:11:50:15 | ControlFlowNode for MyObj | +| test.py:49:28:49:28 | SSA variable x | test.py:50:11:50:18 | SSA variable x | +| test.py:49:28:49:28 | SSA variable x | test.py:50:17:50:17 | ControlFlowNode for x | +| test.py:50:5:50:7 | SSA variable obj | test.py:51:9:51:11 | ControlFlowNode for obj | +| test.py:50:11:50:18 | ControlFlowNode for MyObj() | test.py:50:5:50:7 | SSA variable obj | +| test.py:51:5:51:5 | SSA variable a | test.py:52:12:52:12 | ControlFlowNode for a | +| test.py:51:9:51:15 | ControlFlowNode for Attribute | test.py:51:5:51:5 | SSA variable a | +| test.py:55:1:55:18 | ControlFlowNode for FunctionExpr | test.py:55:5:55:15 | GSSA Variable test_fields | +| test.py:55:1:55:18 | GSSA Variable SINK | test.py:56:5:56:8 | ControlFlowNode for SINK | +| test.py:55:1:55:18 | GSSA Variable SOURCE | test.py:56:33:56:38 | ControlFlowNode for SOURCE | +| test.py:55:1:55:18 | GSSA Variable fields_with_local_flow | test.py:56:10:56:31 | ControlFlowNode for fields_with_local_flow | diff --git a/python/ql/test/experimental/dataflow/fieldflow/allLocalFlow.ql b/python/ql/test/experimental/dataflow/fieldflow/allLocalFlow.ql new file mode 100644 index 00000000000..4df2ae7fdfd --- /dev/null +++ b/python/ql/test/experimental/dataflow/fieldflow/allLocalFlow.ql @@ -0,0 +1,8 @@ +import python +import experimental.dataflow.DataFlow + +from DataFlow::Node nodeFrom, DataFlow::Node nodeTo +where + DataFlow::localFlowStep(nodeFrom, nodeTo) and + nodeFrom.getLocation().getFile().getParent().getBaseName() = "fieldflow" +select nodeFrom, nodeTo diff --git a/python/ql/test/experimental/dataflow/fieldflow/dataflow.expected b/python/ql/test/experimental/dataflow/fieldflow/dataflow.expected new file mode 100644 index 00000000000..76e77ef58b4 --- /dev/null +++ b/python/ql/test/experimental/dataflow/fieldflow/dataflow.expected @@ -0,0 +1,83 @@ +edges +| examples.py:27:8:27:12 | [post arg] ControlFlowNode for myobj [Attribute foo] | examples.py:28:6:28:10 | ControlFlowNode for myobj [Attribute foo] | +| examples.py:27:15:27:20 | ControlFlowNode for SOURCE | examples.py:27:8:27:12 | [post arg] ControlFlowNode for myobj [Attribute foo] | +| examples.py:27:15:27:20 | ControlFlowNode for SOURCE | examples.py:35:13:35:13 | ControlFlowNode for x | +| examples.py:27:15:27:20 | ControlFlowNode for SOURCE | examples.py:41:13:41:18 | ControlFlowNode for SOURCE | +| examples.py:27:15:27:20 | ControlFlowNode for SOURCE | examples.py:50:29:50:34 | ControlFlowNode for SOURCE | +| examples.py:28:6:28:10 | ControlFlowNode for myobj [Attribute foo] | examples.py:28:6:28:14 | ControlFlowNode for Attribute | +| examples.py:31:5:31:10 | ControlFlowNode for SOURCE | examples.py:35:13:35:13 | ControlFlowNode for x | +| examples.py:31:5:31:10 | ControlFlowNode for SOURCE | examples.py:41:13:41:18 | ControlFlowNode for SOURCE | +| examples.py:31:5:31:10 | ControlFlowNode for SOURCE | examples.py:50:29:50:34 | ControlFlowNode for SOURCE | +| examples.py:35:1:35:1 | [post read] ControlFlowNode for a [Attribute obj, Attribute foo] | examples.py:38:6:38:6 | ControlFlowNode for a [Attribute obj, Attribute foo] | +| examples.py:35:1:35:5 | [post store] ControlFlowNode for Attribute [Attribute foo] | examples.py:35:1:35:1 | [post read] ControlFlowNode for a [Attribute obj, Attribute foo] | +| examples.py:35:13:35:13 | ControlFlowNode for x | examples.py:35:1:35:5 | [post store] ControlFlowNode for Attribute [Attribute foo] | +| examples.py:38:6:38:6 | ControlFlowNode for a [Attribute obj, Attribute foo] | examples.py:38:6:38:10 | ControlFlowNode for Attribute [Attribute foo] | +| examples.py:38:6:38:10 | ControlFlowNode for Attribute [Attribute foo] | examples.py:38:6:38:14 | ControlFlowNode for Attribute | +| examples.py:41:7:41:19 | ControlFlowNode for MyObj() [Attribute foo] | examples.py:42:6:42:8 | ControlFlowNode for obj [Attribute foo] | +| examples.py:41:13:41:18 | ControlFlowNode for SOURCE | examples.py:41:7:41:19 | ControlFlowNode for MyObj() [Attribute foo] | +| examples.py:41:13:41:18 | ControlFlowNode for SOURCE | examples.py:50:29:50:34 | ControlFlowNode for SOURCE | +| examples.py:42:6:42:8 | ControlFlowNode for obj [Attribute foo] | examples.py:42:6:42:12 | ControlFlowNode for Attribute | +| examples.py:50:29:50:34 | ControlFlowNode for SOURCE | examples.py:50:6:50:35 | ControlFlowNode for fields_with_local_flow() | +| test.py:29:12:29:16 | [post arg] ControlFlowNode for myobj [Attribute foo] | test.py:30:10:30:14 | ControlFlowNode for myobj [Attribute foo] | +| test.py:29:19:29:24 | ControlFlowNode for SOURCE | test.py:29:12:29:16 | [post arg] ControlFlowNode for myobj [Attribute foo] | +| test.py:30:10:30:14 | ControlFlowNode for myobj [Attribute foo] | test.py:30:10:30:18 | ControlFlowNode for Attribute | +| test.py:34:9:34:14 | ControlFlowNode for SOURCE | test.py:38:17:38:17 | ControlFlowNode for x | +| test.py:38:5:38:5 | [post read] ControlFlowNode for a [Attribute obj, Attribute foo] | test.py:41:10:41:10 | ControlFlowNode for a [Attribute obj, Attribute foo] | +| test.py:38:5:38:9 | [post store] ControlFlowNode for Attribute [Attribute foo] | test.py:38:5:38:5 | [post read] ControlFlowNode for a [Attribute obj, Attribute foo] | +| test.py:38:17:38:17 | ControlFlowNode for x | test.py:38:5:38:9 | [post store] ControlFlowNode for Attribute [Attribute foo] | +| test.py:41:10:41:10 | ControlFlowNode for a [Attribute obj, Attribute foo] | test.py:41:10:41:14 | ControlFlowNode for Attribute [Attribute foo] | +| test.py:41:10:41:14 | ControlFlowNode for Attribute [Attribute foo] | test.py:41:10:41:18 | ControlFlowNode for Attribute | +| test.py:45:11:45:23 | ControlFlowNode for MyObj() [Attribute foo] | test.py:46:10:46:12 | ControlFlowNode for obj [Attribute foo] | +| test.py:45:17:45:22 | ControlFlowNode for SOURCE | test.py:45:11:45:23 | ControlFlowNode for MyObj() [Attribute foo] | +| test.py:46:10:46:12 | ControlFlowNode for obj [Attribute foo] | test.py:46:10:46:16 | ControlFlowNode for Attribute | +| test.py:56:33:56:38 | ControlFlowNode for SOURCE | test.py:56:10:56:39 | ControlFlowNode for fields_with_local_flow() | +nodes +| examples.py:27:8:27:12 | [post arg] ControlFlowNode for myobj [Attribute foo] | semmle.label | [post arg] ControlFlowNode for myobj [Attribute foo] | +| examples.py:27:15:27:20 | ControlFlowNode for SOURCE | semmle.label | ControlFlowNode for SOURCE | +| examples.py:28:6:28:10 | ControlFlowNode for myobj [Attribute foo] | semmle.label | ControlFlowNode for myobj [Attribute foo] | +| examples.py:28:6:28:14 | ControlFlowNode for Attribute | semmle.label | ControlFlowNode for Attribute | +| examples.py:31:5:31:10 | ControlFlowNode for SOURCE | semmle.label | ControlFlowNode for SOURCE | +| examples.py:35:1:35:1 | [post read] ControlFlowNode for a [Attribute obj, Attribute foo] | semmle.label | [post read] ControlFlowNode for a [Attribute obj, Attribute foo] | +| examples.py:35:1:35:5 | [post store] ControlFlowNode for Attribute [Attribute foo] | semmle.label | [post store] ControlFlowNode for Attribute [Attribute foo] | +| examples.py:35:13:35:13 | ControlFlowNode for x | semmle.label | ControlFlowNode for x | +| examples.py:38:6:38:6 | ControlFlowNode for a [Attribute obj, Attribute foo] | semmle.label | ControlFlowNode for a [Attribute obj, Attribute foo] | +| examples.py:38:6:38:10 | ControlFlowNode for Attribute [Attribute foo] | semmle.label | ControlFlowNode for Attribute [Attribute foo] | +| examples.py:38:6:38:14 | ControlFlowNode for Attribute | semmle.label | ControlFlowNode for Attribute | +| examples.py:41:7:41:19 | ControlFlowNode for MyObj() [Attribute foo] | semmle.label | ControlFlowNode for MyObj() [Attribute foo] | +| examples.py:41:13:41:18 | ControlFlowNode for SOURCE | semmle.label | ControlFlowNode for SOURCE | +| examples.py:42:6:42:8 | ControlFlowNode for obj [Attribute foo] | semmle.label | ControlFlowNode for obj [Attribute foo] | +| examples.py:42:6:42:12 | ControlFlowNode for Attribute | semmle.label | ControlFlowNode for Attribute | +| examples.py:50:6:50:35 | ControlFlowNode for fields_with_local_flow() | semmle.label | ControlFlowNode for fields_with_local_flow() | +| examples.py:50:29:50:34 | ControlFlowNode for SOURCE | semmle.label | ControlFlowNode for SOURCE | +| test.py:29:12:29:16 | [post arg] ControlFlowNode for myobj [Attribute foo] | semmle.label | [post arg] ControlFlowNode for myobj [Attribute foo] | +| test.py:29:19:29:24 | ControlFlowNode for SOURCE | semmle.label | ControlFlowNode for SOURCE | +| test.py:30:10:30:14 | ControlFlowNode for myobj [Attribute foo] | semmle.label | ControlFlowNode for myobj [Attribute foo] | +| test.py:30:10:30:18 | ControlFlowNode for Attribute | semmle.label | ControlFlowNode for Attribute | +| test.py:34:9:34:14 | ControlFlowNode for SOURCE | semmle.label | ControlFlowNode for SOURCE | +| test.py:38:5:38:5 | [post read] ControlFlowNode for a [Attribute obj, Attribute foo] | semmle.label | [post read] ControlFlowNode for a [Attribute obj, Attribute foo] | +| test.py:38:5:38:9 | [post store] ControlFlowNode for Attribute [Attribute foo] | semmle.label | [post store] ControlFlowNode for Attribute [Attribute foo] | +| test.py:38:17:38:17 | ControlFlowNode for x | semmle.label | ControlFlowNode for x | +| test.py:41:10:41:10 | ControlFlowNode for a [Attribute obj, Attribute foo] | semmle.label | ControlFlowNode for a [Attribute obj, Attribute foo] | +| test.py:41:10:41:14 | ControlFlowNode for Attribute [Attribute foo] | semmle.label | ControlFlowNode for Attribute [Attribute foo] | +| test.py:41:10:41:18 | ControlFlowNode for Attribute | semmle.label | ControlFlowNode for Attribute | +| test.py:45:11:45:23 | ControlFlowNode for MyObj() [Attribute foo] | semmle.label | ControlFlowNode for MyObj() [Attribute foo] | +| test.py:45:17:45:22 | ControlFlowNode for SOURCE | semmle.label | ControlFlowNode for SOURCE | +| test.py:46:10:46:12 | ControlFlowNode for obj [Attribute foo] | semmle.label | ControlFlowNode for obj [Attribute foo] | +| test.py:46:10:46:16 | ControlFlowNode for Attribute | semmle.label | ControlFlowNode for Attribute | +| test.py:56:10:56:39 | ControlFlowNode for fields_with_local_flow() | semmle.label | ControlFlowNode for fields_with_local_flow() | +| test.py:56:33:56:38 | ControlFlowNode for SOURCE | semmle.label | ControlFlowNode for SOURCE | +#select +| examples.py:28:6:28:14 | ControlFlowNode for Attribute | examples.py:27:15:27:20 | ControlFlowNode for SOURCE | examples.py:28:6:28:14 | ControlFlowNode for Attribute | Flow found | +| examples.py:38:6:38:14 | ControlFlowNode for Attribute | examples.py:27:15:27:20 | ControlFlowNode for SOURCE | examples.py:38:6:38:14 | ControlFlowNode for Attribute | Flow found | +| examples.py:38:6:38:14 | ControlFlowNode for Attribute | examples.py:31:5:31:10 | ControlFlowNode for SOURCE | examples.py:38:6:38:14 | ControlFlowNode for Attribute | Flow found | +| examples.py:42:6:42:12 | ControlFlowNode for Attribute | examples.py:27:15:27:20 | ControlFlowNode for SOURCE | examples.py:42:6:42:12 | ControlFlowNode for Attribute | Flow found | +| examples.py:42:6:42:12 | ControlFlowNode for Attribute | examples.py:31:5:31:10 | ControlFlowNode for SOURCE | examples.py:42:6:42:12 | ControlFlowNode for Attribute | Flow found | +| examples.py:42:6:42:12 | ControlFlowNode for Attribute | examples.py:41:13:41:18 | ControlFlowNode for SOURCE | examples.py:42:6:42:12 | ControlFlowNode for Attribute | Flow found | +| examples.py:50:6:50:35 | ControlFlowNode for fields_with_local_flow() | examples.py:27:15:27:20 | ControlFlowNode for SOURCE | examples.py:50:6:50:35 | ControlFlowNode for fields_with_local_flow() | Flow found | +| examples.py:50:6:50:35 | ControlFlowNode for fields_with_local_flow() | examples.py:31:5:31:10 | ControlFlowNode for SOURCE | examples.py:50:6:50:35 | ControlFlowNode for fields_with_local_flow() | Flow found | +| examples.py:50:6:50:35 | ControlFlowNode for fields_with_local_flow() | examples.py:41:13:41:18 | ControlFlowNode for SOURCE | examples.py:50:6:50:35 | ControlFlowNode for fields_with_local_flow() | Flow found | +| examples.py:50:6:50:35 | ControlFlowNode for fields_with_local_flow() | examples.py:50:29:50:34 | ControlFlowNode for SOURCE | examples.py:50:6:50:35 | ControlFlowNode for fields_with_local_flow() | Flow found | +| test.py:30:10:30:18 | ControlFlowNode for Attribute | test.py:29:19:29:24 | ControlFlowNode for SOURCE | test.py:30:10:30:18 | ControlFlowNode for Attribute | Flow found | +| test.py:41:10:41:18 | ControlFlowNode for Attribute | test.py:34:9:34:14 | ControlFlowNode for SOURCE | test.py:41:10:41:18 | ControlFlowNode for Attribute | Flow found | +| test.py:46:10:46:16 | ControlFlowNode for Attribute | test.py:45:17:45:22 | ControlFlowNode for SOURCE | test.py:46:10:46:16 | ControlFlowNode for Attribute | Flow found | +| test.py:56:10:56:39 | ControlFlowNode for fields_with_local_flow() | test.py:56:33:56:38 | ControlFlowNode for SOURCE | test.py:56:10:56:39 | ControlFlowNode for fields_with_local_flow() | Flow found | diff --git a/python/ql/test/experimental/dataflow/fieldflow/dataflow.ql b/python/ql/test/experimental/dataflow/fieldflow/dataflow.ql new file mode 100644 index 00000000000..807f2e91931 --- /dev/null +++ b/python/ql/test/experimental/dataflow/fieldflow/dataflow.ql @@ -0,0 +1,10 @@ +/** + * @kind path-problem + */ + +import experimental.dataflow.testConfig +import DataFlow::PathGraph + +from TestConfiguration config, DataFlow::PathNode source, DataFlow::PathNode sink +where config.hasFlowPath(source, sink) +select sink.getNode(), source, sink, "Flow found" diff --git a/python/ql/test/experimental/dataflow/fieldflow/examples.py b/python/ql/test/experimental/dataflow/fieldflow/examples.py new file mode 100644 index 00000000000..ed961438822 --- /dev/null +++ b/python/ql/test/experimental/dataflow/fieldflow/examples.py @@ -0,0 +1,50 @@ +from python.ql.test.experimental.dataflow.testDefinitions import * + +# Preamble + +class MyObj(object): + + def __init__(self, foo): + self.foo = foo + + +class NestedObj(object): + + def __init__(self): + self.obj = MyObj("OK") + + def getObj(self): + return self.obj + + +# Example 1 +def setFoo(obj, x): + SINK_F(obj.foo) + obj.foo = x + +myobj = MyObj("OK") + +setFoo(myobj, SOURCE) +SINK(myobj.foo) + +# Example 2 +x = SOURCE + +a = NestedObj() + +a.obj.foo = x +a.getObj().foo = x + +SINK(a.obj.foo) + +# Example 3 +obj = MyObj(SOURCE) +SINK(obj.foo) + +# Local flow +def fields_with_local_flow(x): + obj = MyObj(x) + a = obj.foo + return a + +SINK(fields_with_local_flow(SOURCE)) \ No newline at end of file diff --git a/python/ql/test/experimental/dataflow/fieldflow/globalStep.expected b/python/ql/test/experimental/dataflow/fieldflow/globalStep.expected new file mode 100644 index 00000000000..ec077fa4b64 --- /dev/null +++ b/python/ql/test/experimental/dataflow/fieldflow/globalStep.expected @@ -0,0 +1,806 @@ +| examples.py:0:0:0:0 | GSSA Variable SINK | examples.py:1:1:1:66 | GSSA Variable SINK | +| examples.py:0:0:0:0 | GSSA Variable SINK | examples.py:1:1:1:66 | GSSA Variable SINK | +| examples.py:0:0:0:0 | GSSA Variable SINK | examples.py:28:1:28:4 | ControlFlowNode for SINK | +| examples.py:0:0:0:0 | GSSA Variable SINK | examples.py:28:1:28:4 | ControlFlowNode for SINK | +| examples.py:0:0:0:0 | GSSA Variable SINK | examples.py:38:1:38:4 | ControlFlowNode for SINK | +| examples.py:0:0:0:0 | GSSA Variable SINK | examples.py:38:1:38:4 | ControlFlowNode for SINK | +| examples.py:0:0:0:0 | GSSA Variable SINK | examples.py:42:1:42:4 | ControlFlowNode for SINK | +| examples.py:0:0:0:0 | GSSA Variable SINK | examples.py:42:1:42:4 | ControlFlowNode for SINK | +| examples.py:0:0:0:0 | GSSA Variable SINK | examples.py:50:1:50:4 | ControlFlowNode for SINK | +| examples.py:0:0:0:0 | GSSA Variable SINK | examples.py:50:1:50:4 | ControlFlowNode for SINK | +| examples.py:0:0:0:0 | GSSA Variable SINK_F | examples.py:1:1:1:66 | GSSA Variable SINK_F | +| examples.py:0:0:0:0 | GSSA Variable SINK_F | examples.py:1:1:1:66 | GSSA Variable SINK_F | +| examples.py:0:0:0:0 | GSSA Variable SOURCE | examples.py:1:1:1:66 | GSSA Variable SOURCE | +| examples.py:0:0:0:0 | GSSA Variable SOURCE | examples.py:1:1:1:66 | GSSA Variable SOURCE | +| examples.py:0:0:0:0 | GSSA Variable SOURCE | examples.py:27:15:27:20 | ControlFlowNode for SOURCE | +| examples.py:0:0:0:0 | GSSA Variable SOURCE | examples.py:27:15:27:20 | ControlFlowNode for SOURCE | +| examples.py:0:0:0:0 | GSSA Variable SOURCE | examples.py:31:1:31:1 | GSSA Variable x | +| examples.py:0:0:0:0 | GSSA Variable SOURCE | examples.py:31:1:31:1 | GSSA Variable x | +| examples.py:0:0:0:0 | GSSA Variable SOURCE | examples.py:31:5:31:10 | ControlFlowNode for SOURCE | +| examples.py:0:0:0:0 | GSSA Variable SOURCE | examples.py:31:5:31:10 | ControlFlowNode for SOURCE | +| examples.py:0:0:0:0 | GSSA Variable SOURCE | examples.py:35:13:35:13 | ControlFlowNode for x | +| examples.py:0:0:0:0 | GSSA Variable SOURCE | examples.py:35:13:35:13 | ControlFlowNode for x | +| examples.py:0:0:0:0 | GSSA Variable SOURCE | examples.py:36:18:36:18 | ControlFlowNode for x | +| examples.py:0:0:0:0 | GSSA Variable SOURCE | examples.py:36:18:36:18 | ControlFlowNode for x | +| examples.py:0:0:0:0 | GSSA Variable SOURCE | examples.py:41:7:41:19 | GSSA Variable SOURCE | +| examples.py:0:0:0:0 | GSSA Variable SOURCE | examples.py:41:7:41:19 | GSSA Variable SOURCE | +| examples.py:0:0:0:0 | GSSA Variable SOURCE | examples.py:41:13:41:18 | ControlFlowNode for SOURCE | +| examples.py:0:0:0:0 | GSSA Variable SOURCE | examples.py:41:13:41:18 | ControlFlowNode for SOURCE | +| examples.py:0:0:0:0 | GSSA Variable SOURCE | examples.py:50:6:50:35 | GSSA Variable SOURCE | +| examples.py:0:0:0:0 | GSSA Variable SOURCE | examples.py:50:6:50:35 | GSSA Variable SOURCE | +| examples.py:0:0:0:0 | GSSA Variable SOURCE | examples.py:50:29:50:34 | ControlFlowNode for SOURCE | +| examples.py:0:0:0:0 | GSSA Variable SOURCE | examples.py:50:29:50:34 | ControlFlowNode for SOURCE | +| examples.py:0:0:0:0 | GSSA Variable __name__ | examples.py:1:1:1:66 | GSSA Variable __name__ | +| examples.py:0:0:0:0 | GSSA Variable __name__ | examples.py:1:1:1:66 | GSSA Variable __name__ | +| examples.py:0:0:0:0 | GSSA Variable __package__ | examples.py:1:1:1:66 | GSSA Variable __package__ | +| examples.py:0:0:0:0 | GSSA Variable __package__ | examples.py:1:1:1:66 | GSSA Variable __package__ | +| examples.py:0:0:0:0 | GSSA Variable a | examples.py:1:1:1:66 | GSSA Variable a | +| examples.py:0:0:0:0 | GSSA Variable a | examples.py:1:1:1:66 | GSSA Variable a | +| examples.py:0:0:0:0 | GSSA Variable fields_with_local_flow | examples.py:1:1:1:66 | GSSA Variable fields_with_local_flow | +| examples.py:0:0:0:0 | GSSA Variable fields_with_local_flow | examples.py:1:1:1:66 | GSSA Variable fields_with_local_flow | +| examples.py:0:0:0:0 | GSSA Variable myobj | examples.py:1:1:1:66 | GSSA Variable myobj | +| examples.py:0:0:0:0 | GSSA Variable myobj | examples.py:1:1:1:66 | GSSA Variable myobj | +| examples.py:0:0:0:0 | GSSA Variable obj | examples.py:1:1:1:66 | GSSA Variable obj | +| examples.py:0:0:0:0 | GSSA Variable obj | examples.py:1:1:1:66 | GSSA Variable obj | +| examples.py:0:0:0:0 | GSSA Variable object | examples.py:1:1:1:66 | GSSA Variable object | +| examples.py:0:0:0:0 | GSSA Variable object | examples.py:1:1:1:66 | GSSA Variable object | +| examples.py:0:0:0:0 | GSSA Variable object | examples.py:5:13:5:18 | ControlFlowNode for object | +| examples.py:0:0:0:0 | GSSA Variable object | examples.py:5:13:5:18 | ControlFlowNode for object | +| examples.py:0:0:0:0 | GSSA Variable object | examples.py:11:17:11:22 | ControlFlowNode for object | +| examples.py:0:0:0:0 | GSSA Variable object | examples.py:11:17:11:22 | ControlFlowNode for object | +| examples.py:0:0:0:0 | GSSA Variable x | examples.py:1:1:1:66 | GSSA Variable x | +| examples.py:0:0:0:0 | GSSA Variable x | examples.py:1:1:1:66 | GSSA Variable x | +| examples.py:0:0:0:0 | ModuleVariableNode for Global Variable MyObj in Module examples | examples.py:14:20:14:24 | ControlFlowNode for MyObj | +| examples.py:0:0:0:0 | ModuleVariableNode for Global Variable MyObj in Module examples | examples.py:14:20:14:24 | ControlFlowNode for MyObj | +| examples.py:0:0:0:0 | ModuleVariableNode for Global Variable MyObj in Module examples | examples.py:46:9:46:13 | ControlFlowNode for MyObj | +| examples.py:0:0:0:0 | ModuleVariableNode for Global Variable MyObj in Module examples | examples.py:46:9:46:13 | ControlFlowNode for MyObj | +| examples.py:0:0:0:0 | ModuleVariableNode for Global Variable SINK_F in Module examples | examples.py:22:5:22:10 | ControlFlowNode for SINK_F | +| examples.py:0:0:0:0 | ModuleVariableNode for Global Variable SINK_F in Module examples | examples.py:22:5:22:10 | ControlFlowNode for SINK_F | +| examples.py:0:0:0:0 | SSA variable $ | examples.py:1:1:1:66 | SSA variable $ | +| examples.py:0:0:0:0 | SSA variable $ | examples.py:1:1:1:66 | SSA variable $ | +| examples.py:0:0:0:0 | SSA variable * | examples.py:1:1:1:66 | SSA variable * | +| examples.py:0:0:0:0 | SSA variable * | examples.py:1:1:1:66 | SSA variable * | +| examples.py:1:1:1:66 | GSSA Variable SOURCE | examples.py:41:7:41:19 | GSSA Variable SOURCE | +| examples.py:1:1:1:66 | GSSA Variable SOURCE | examples.py:41:7:41:19 | GSSA Variable SOURCE | +| examples.py:1:1:1:66 | GSSA Variable SOURCE | examples.py:50:6:50:35 | GSSA Variable SOURCE | +| examples.py:1:1:1:66 | GSSA Variable SOURCE | examples.py:50:6:50:35 | GSSA Variable SOURCE | +| examples.py:5:1:5:20 | ControlFlowNode for ClassExpr | examples.py:5:7:5:11 | GSSA Variable MyObj | +| examples.py:5:1:5:20 | ControlFlowNode for ClassExpr | examples.py:5:7:5:11 | GSSA Variable MyObj | +| examples.py:5:1:5:20 | ControlFlowNode for ClassExpr | examples.py:25:9:25:13 | ControlFlowNode for MyObj | +| examples.py:5:1:5:20 | ControlFlowNode for ClassExpr | examples.py:25:9:25:13 | ControlFlowNode for MyObj | +| examples.py:5:1:5:20 | ControlFlowNode for ClassExpr | examples.py:41:7:41:11 | ControlFlowNode for MyObj | +| examples.py:5:1:5:20 | ControlFlowNode for ClassExpr | examples.py:41:7:41:11 | ControlFlowNode for MyObj | +| examples.py:5:7:5:11 | GSSA Variable MyObj | examples.py:0:0:0:0 | ModuleVariableNode for Global Variable MyObj in Module examples | +| examples.py:5:7:5:11 | GSSA Variable MyObj | examples.py:0:0:0:0 | ModuleVariableNode for Global Variable MyObj in Module examples | +| examples.py:5:7:5:11 | GSSA Variable MyObj | examples.py:25:9:25:13 | ControlFlowNode for MyObj | +| examples.py:5:7:5:11 | GSSA Variable MyObj | examples.py:25:9:25:13 | ControlFlowNode for MyObj | +| examples.py:5:7:5:11 | GSSA Variable MyObj | examples.py:41:7:41:11 | ControlFlowNode for MyObj | +| examples.py:5:7:5:11 | GSSA Variable MyObj | examples.py:41:7:41:11 | ControlFlowNode for MyObj | +| examples.py:5:13:5:18 | ControlFlowNode for object | examples.py:11:17:11:22 | ControlFlowNode for object | +| examples.py:5:13:5:18 | ControlFlowNode for object | examples.py:11:17:11:22 | ControlFlowNode for object | +| examples.py:7:5:7:28 | ControlFlowNode for FunctionExpr | examples.py:7:9:7:16 | SSA variable __init__ | +| examples.py:7:5:7:28 | ControlFlowNode for FunctionExpr | examples.py:7:9:7:16 | SSA variable __init__ | +| examples.py:7:18:7:21 | SSA variable self | examples.py:8:9:8:12 | ControlFlowNode for self | +| examples.py:7:18:7:21 | SSA variable self | examples.py:8:9:8:12 | ControlFlowNode for self | +| examples.py:7:18:7:21 | SSA variable self | examples.py:8:9:8:12 | ControlFlowNode for self | +| examples.py:7:18:7:21 | SSA variable self | examples.py:8:9:8:12 | ControlFlowNode for self | +| examples.py:7:18:7:21 | SSA variable self | examples.py:8:9:8:16 | SSA variable self | +| examples.py:7:18:7:21 | SSA variable self | examples.py:8:9:8:16 | SSA variable self | +| examples.py:7:18:7:21 | SSA variable self | examples.py:8:9:8:16 | SSA variable self | +| examples.py:7:18:7:21 | SSA variable self | examples.py:8:9:8:16 | SSA variable self | +| examples.py:7:24:7:26 | SSA variable foo | examples.py:8:20:8:22 | ControlFlowNode for foo | +| examples.py:7:24:7:26 | SSA variable foo | examples.py:8:20:8:22 | ControlFlowNode for foo | +| examples.py:7:24:7:26 | SSA variable foo | examples.py:8:20:8:22 | ControlFlowNode for foo | +| examples.py:7:24:7:26 | SSA variable foo | examples.py:8:20:8:22 | ControlFlowNode for foo | +| examples.py:8:9:8:12 | [post store] ControlFlowNode for self | examples.py:14:20:14:30 | ControlFlowNode for MyObj() | +| examples.py:8:9:8:12 | [post store] ControlFlowNode for self | examples.py:14:20:14:30 | ControlFlowNode for MyObj() | +| examples.py:8:9:8:12 | [post store] ControlFlowNode for self | examples.py:25:9:25:19 | ControlFlowNode for MyObj() | +| examples.py:8:9:8:12 | [post store] ControlFlowNode for self | examples.py:25:9:25:19 | ControlFlowNode for MyObj() | +| examples.py:8:9:8:12 | [post store] ControlFlowNode for self | examples.py:41:7:41:19 | ControlFlowNode for MyObj() | +| examples.py:8:9:8:12 | [post store] ControlFlowNode for self | examples.py:41:7:41:19 | ControlFlowNode for MyObj() | +| examples.py:8:9:8:12 | [post store] ControlFlowNode for self | examples.py:46:9:46:16 | ControlFlowNode for MyObj() | +| examples.py:8:9:8:12 | [post store] ControlFlowNode for self | examples.py:46:9:46:16 | ControlFlowNode for MyObj() | +| examples.py:8:9:8:12 | [post store] ControlFlowNode for self [Attribute foo] | examples.py:14:20:14:30 | ControlFlowNode for MyObj() [Attribute foo] | +| examples.py:8:9:8:12 | [post store] ControlFlowNode for self [Attribute foo] | examples.py:25:9:25:19 | ControlFlowNode for MyObj() [Attribute foo] | +| examples.py:8:9:8:12 | [post store] ControlFlowNode for self [Attribute foo] | examples.py:41:7:41:19 | ControlFlowNode for MyObj() [Attribute foo] | +| examples.py:8:9:8:12 | [post store] ControlFlowNode for self [Attribute foo] | examples.py:46:9:46:16 | ControlFlowNode for MyObj() [Attribute foo] | +| examples.py:8:20:8:22 | ControlFlowNode for foo | examples.py:8:9:8:12 | [post store] ControlFlowNode for self [Attribute foo] | +| examples.py:8:20:8:22 | ControlFlowNode for foo | examples.py:8:9:8:12 | [post store] ControlFlowNode for self [Attribute foo] | +| examples.py:11:1:11:24 | ControlFlowNode for ClassExpr | examples.py:11:7:11:15 | GSSA Variable NestedObj | +| examples.py:11:1:11:24 | ControlFlowNode for ClassExpr | examples.py:11:7:11:15 | GSSA Variable NestedObj | +| examples.py:11:1:11:24 | ControlFlowNode for ClassExpr | examples.py:33:5:33:13 | ControlFlowNode for NestedObj | +| examples.py:11:1:11:24 | ControlFlowNode for ClassExpr | examples.py:33:5:33:13 | ControlFlowNode for NestedObj | +| examples.py:11:7:11:15 | GSSA Variable NestedObj | examples.py:33:5:33:13 | ControlFlowNode for NestedObj | +| examples.py:11:7:11:15 | GSSA Variable NestedObj | examples.py:33:5:33:13 | ControlFlowNode for NestedObj | +| examples.py:13:5:13:23 | ControlFlowNode for FunctionExpr | examples.py:13:9:13:16 | SSA variable __init__ | +| examples.py:13:5:13:23 | ControlFlowNode for FunctionExpr | examples.py:13:9:13:16 | SSA variable __init__ | +| examples.py:13:5:13:23 | GSSA Variable MyObj | examples.py:14:20:14:24 | ControlFlowNode for MyObj | +| examples.py:13:5:13:23 | GSSA Variable MyObj | examples.py:14:20:14:24 | ControlFlowNode for MyObj | +| examples.py:13:18:13:21 | SSA variable self | examples.py:14:9:14:12 | ControlFlowNode for self | +| examples.py:13:18:13:21 | SSA variable self | examples.py:14:9:14:12 | ControlFlowNode for self | +| examples.py:13:18:13:21 | SSA variable self | examples.py:14:9:14:12 | ControlFlowNode for self | +| examples.py:13:18:13:21 | SSA variable self | examples.py:14:9:14:12 | ControlFlowNode for self | +| examples.py:13:18:13:21 | SSA variable self | examples.py:14:9:14:16 | SSA variable self | +| examples.py:13:18:13:21 | SSA variable self | examples.py:14:9:14:16 | SSA variable self | +| examples.py:13:18:13:21 | SSA variable self | examples.py:14:9:14:16 | SSA variable self | +| examples.py:13:18:13:21 | SSA variable self | examples.py:14:9:14:16 | SSA variable self | +| examples.py:14:9:14:12 | [post store] ControlFlowNode for self | examples.py:33:5:33:15 | ControlFlowNode for NestedObj() | +| examples.py:14:9:14:12 | [post store] ControlFlowNode for self | examples.py:33:5:33:15 | ControlFlowNode for NestedObj() | +| examples.py:14:9:14:12 | [post store] ControlFlowNode for self [Attribute obj, Attribute foo] | examples.py:33:5:33:15 | ControlFlowNode for NestedObj() [Attribute obj, Attribute foo] | +| examples.py:14:9:14:12 | [post store] ControlFlowNode for self [Attribute obj] | examples.py:33:5:33:15 | ControlFlowNode for NestedObj() [Attribute obj] | +| examples.py:14:20:14:30 | ControlFlowNode for MyObj() | examples.py:14:9:14:12 | [post store] ControlFlowNode for self [Attribute obj] | +| examples.py:14:20:14:30 | ControlFlowNode for MyObj() [Attribute foo] | examples.py:14:9:14:12 | [post store] ControlFlowNode for self [Attribute obj, Attribute foo] | +| examples.py:14:20:14:30 | [pre objCreate] ControlFlowNode for MyObj() | examples.py:7:18:7:21 | SSA variable self | +| examples.py:14:20:14:30 | [pre objCreate] ControlFlowNode for MyObj() | examples.py:7:18:7:21 | SSA variable self | +| examples.py:14:26:14:29 | ControlFlowNode for Str | examples.py:7:24:7:26 | SSA variable foo | +| examples.py:14:26:14:29 | ControlFlowNode for Str | examples.py:7:24:7:26 | SSA variable foo | +| examples.py:14:26:14:29 | ControlFlowNode for Str | examples.py:14:20:14:30 | ControlFlowNode for MyObj() [Attribute foo] | +| examples.py:16:5:16:21 | ControlFlowNode for FunctionExpr | examples.py:16:9:16:14 | SSA variable getObj | +| examples.py:16:5:16:21 | ControlFlowNode for FunctionExpr | examples.py:16:9:16:14 | SSA variable getObj | +| examples.py:16:16:16:19 | SSA variable self | examples.py:17:16:17:19 | ControlFlowNode for self | +| examples.py:16:16:16:19 | SSA variable self | examples.py:17:16:17:19 | ControlFlowNode for self | +| examples.py:21:1:21:19 | ControlFlowNode for FunctionExpr | examples.py:21:5:21:10 | GSSA Variable setFoo | +| examples.py:21:1:21:19 | ControlFlowNode for FunctionExpr | examples.py:21:5:21:10 | GSSA Variable setFoo | +| examples.py:21:1:21:19 | ControlFlowNode for FunctionExpr | examples.py:27:1:27:6 | ControlFlowNode for setFoo | +| examples.py:21:1:21:19 | ControlFlowNode for FunctionExpr | examples.py:27:1:27:6 | ControlFlowNode for setFoo | +| examples.py:21:1:21:19 | GSSA Variable SINK_F | examples.py:22:5:22:10 | ControlFlowNode for SINK_F | +| examples.py:21:1:21:19 | GSSA Variable SINK_F | examples.py:22:5:22:10 | ControlFlowNode for SINK_F | +| examples.py:21:5:21:10 | GSSA Variable setFoo | examples.py:27:1:27:6 | ControlFlowNode for setFoo | +| examples.py:21:5:21:10 | GSSA Variable setFoo | examples.py:27:1:27:6 | ControlFlowNode for setFoo | +| examples.py:21:12:21:14 | SSA variable obj | examples.py:22:12:22:14 | ControlFlowNode for obj | +| examples.py:21:12:21:14 | SSA variable obj | examples.py:22:12:22:14 | ControlFlowNode for obj | +| examples.py:21:12:21:14 | SSA variable obj | examples.py:22:12:22:14 | ControlFlowNode for obj | +| examples.py:21:12:21:14 | SSA variable obj | examples.py:22:12:22:14 | ControlFlowNode for obj | +| examples.py:21:12:21:14 | SSA variable obj | examples.py:23:5:23:7 | ControlFlowNode for obj | +| examples.py:21:12:21:14 | SSA variable obj | examples.py:23:5:23:7 | ControlFlowNode for obj | +| examples.py:21:12:21:14 | SSA variable obj | examples.py:23:5:23:7 | ControlFlowNode for obj | +| examples.py:21:12:21:14 | SSA variable obj | examples.py:23:5:23:7 | ControlFlowNode for obj | +| examples.py:21:12:21:14 | SSA variable obj | examples.py:23:5:23:11 | SSA variable obj | +| examples.py:21:12:21:14 | SSA variable obj | examples.py:23:5:23:11 | SSA variable obj | +| examples.py:21:12:21:14 | SSA variable obj | examples.py:23:5:23:11 | SSA variable obj | +| examples.py:21:12:21:14 | SSA variable obj | examples.py:23:5:23:11 | SSA variable obj | +| examples.py:21:12:21:14 | SSA variable obj [Attribute foo] | examples.py:22:12:22:14 | ControlFlowNode for obj [Attribute foo] | +| examples.py:21:17:21:17 | SSA variable x | examples.py:23:15:23:15 | ControlFlowNode for x | +| examples.py:21:17:21:17 | SSA variable x | examples.py:23:15:23:15 | ControlFlowNode for x | +| examples.py:21:17:21:17 | SSA variable x | examples.py:23:15:23:15 | ControlFlowNode for x | +| examples.py:21:17:21:17 | SSA variable x | examples.py:23:15:23:15 | ControlFlowNode for x | +| examples.py:22:12:22:14 | ControlFlowNode for obj | examples.py:23:5:23:7 | ControlFlowNode for obj | +| examples.py:22:12:22:14 | ControlFlowNode for obj | examples.py:23:5:23:7 | ControlFlowNode for obj | +| examples.py:22:12:22:14 | ControlFlowNode for obj | examples.py:23:5:23:7 | ControlFlowNode for obj | +| examples.py:22:12:22:14 | ControlFlowNode for obj | examples.py:23:5:23:7 | ControlFlowNode for obj | +| examples.py:22:12:22:14 | ControlFlowNode for obj [Attribute foo] | examples.py:22:12:22:18 | ControlFlowNode for Attribute | +| examples.py:22:12:22:14 | ControlFlowNode for obj [Attribute foo] | examples.py:22:12:22:18 | ControlFlowNode for Attribute | +| examples.py:22:12:22:14 | [post read] ControlFlowNode for obj | examples.py:23:5:23:7 | ControlFlowNode for obj | +| examples.py:22:12:22:14 | [post read] ControlFlowNode for obj | examples.py:23:5:23:7 | ControlFlowNode for obj | +| examples.py:22:12:22:14 | [post read] ControlFlowNode for obj | examples.py:27:8:27:12 | [post arg] ControlFlowNode for myobj | +| examples.py:22:12:22:14 | [post read] ControlFlowNode for obj | examples.py:27:8:27:12 | [post arg] ControlFlowNode for myobj | +| examples.py:23:5:23:7 | [post store] ControlFlowNode for obj | examples.py:27:8:27:12 | [post arg] ControlFlowNode for myobj | +| examples.py:23:5:23:7 | [post store] ControlFlowNode for obj | examples.py:27:8:27:12 | [post arg] ControlFlowNode for myobj | +| examples.py:23:5:23:7 | [post store] ControlFlowNode for obj [Attribute foo] | examples.py:27:8:27:12 | [post arg] ControlFlowNode for myobj [Attribute foo] | +| examples.py:23:15:23:15 | ControlFlowNode for x | examples.py:23:5:23:7 | [post store] ControlFlowNode for obj [Attribute foo] | +| examples.py:23:15:23:15 | ControlFlowNode for x | examples.py:23:5:23:7 | [post store] ControlFlowNode for obj [Attribute foo] | +| examples.py:25:1:25:5 | GSSA Variable myobj | examples.py:27:1:27:21 | GSSA Variable myobj | +| examples.py:25:1:25:5 | GSSA Variable myobj | examples.py:27:1:27:21 | GSSA Variable myobj | +| examples.py:25:1:25:5 | GSSA Variable myobj | examples.py:27:8:27:12 | ControlFlowNode for myobj | +| examples.py:25:1:25:5 | GSSA Variable myobj | examples.py:27:8:27:12 | ControlFlowNode for myobj | +| examples.py:25:1:25:5 | GSSA Variable myobj | examples.py:28:6:28:10 | ControlFlowNode for myobj | +| examples.py:25:1:25:5 | GSSA Variable myobj | examples.py:28:6:28:10 | ControlFlowNode for myobj | +| examples.py:25:1:25:5 | GSSA Variable myobj [Attribute foo] | examples.py:27:8:27:12 | ControlFlowNode for myobj [Attribute foo] | +| examples.py:25:1:25:5 | GSSA Variable myobj [Attribute foo] | examples.py:28:6:28:10 | ControlFlowNode for myobj [Attribute foo] | +| examples.py:25:9:25:13 | ControlFlowNode for MyObj | examples.py:41:7:41:11 | ControlFlowNode for MyObj | +| examples.py:25:9:25:13 | ControlFlowNode for MyObj | examples.py:41:7:41:11 | ControlFlowNode for MyObj | +| examples.py:25:9:25:19 | ControlFlowNode for MyObj() | examples.py:25:1:25:5 | GSSA Variable myobj | +| examples.py:25:9:25:19 | ControlFlowNode for MyObj() | examples.py:25:1:25:5 | GSSA Variable myobj | +| examples.py:25:9:25:19 | ControlFlowNode for MyObj() | examples.py:27:1:27:21 | GSSA Variable myobj | +| examples.py:25:9:25:19 | ControlFlowNode for MyObj() | examples.py:27:1:27:21 | GSSA Variable myobj | +| examples.py:25:9:25:19 | ControlFlowNode for MyObj() | examples.py:27:8:27:12 | ControlFlowNode for myobj | +| examples.py:25:9:25:19 | ControlFlowNode for MyObj() | examples.py:27:8:27:12 | ControlFlowNode for myobj | +| examples.py:25:9:25:19 | ControlFlowNode for MyObj() | examples.py:28:6:28:10 | ControlFlowNode for myobj | +| examples.py:25:9:25:19 | ControlFlowNode for MyObj() | examples.py:28:6:28:10 | ControlFlowNode for myobj | +| examples.py:25:9:25:19 | ControlFlowNode for MyObj() [Attribute foo] | examples.py:25:1:25:5 | GSSA Variable myobj [Attribute foo] | +| examples.py:25:9:25:19 | ControlFlowNode for MyObj() [Attribute foo] | examples.py:27:8:27:12 | ControlFlowNode for myobj [Attribute foo] | +| examples.py:25:9:25:19 | ControlFlowNode for MyObj() [Attribute foo] | examples.py:28:6:28:10 | ControlFlowNode for myobj [Attribute foo] | +| examples.py:25:9:25:19 | [pre objCreate] ControlFlowNode for MyObj() | examples.py:7:18:7:21 | SSA variable self | +| examples.py:25:9:25:19 | [pre objCreate] ControlFlowNode for MyObj() | examples.py:7:18:7:21 | SSA variable self | +| examples.py:25:15:25:18 | ControlFlowNode for Str | examples.py:7:24:7:26 | SSA variable foo | +| examples.py:25:15:25:18 | ControlFlowNode for Str | examples.py:7:24:7:26 | SSA variable foo | +| examples.py:25:15:25:18 | ControlFlowNode for Str | examples.py:25:9:25:19 | ControlFlowNode for MyObj() [Attribute foo] | +| examples.py:27:8:27:12 | ControlFlowNode for myobj | examples.py:21:12:21:14 | SSA variable obj | +| examples.py:27:8:27:12 | ControlFlowNode for myobj | examples.py:21:12:21:14 | SSA variable obj | +| examples.py:27:8:27:12 | ControlFlowNode for myobj | examples.py:28:6:28:10 | ControlFlowNode for myobj | +| examples.py:27:8:27:12 | ControlFlowNode for myobj | examples.py:28:6:28:10 | ControlFlowNode for myobj | +| examples.py:27:8:27:12 | ControlFlowNode for myobj [Attribute foo] | examples.py:21:12:21:14 | SSA variable obj [Attribute foo] | +| examples.py:27:8:27:12 | ControlFlowNode for myobj [Attribute foo] | examples.py:28:6:28:10 | ControlFlowNode for myobj [Attribute foo] | +| examples.py:27:8:27:12 | [post arg] ControlFlowNode for myobj | examples.py:28:6:28:10 | ControlFlowNode for myobj | +| examples.py:27:8:27:12 | [post arg] ControlFlowNode for myobj | examples.py:28:6:28:10 | ControlFlowNode for myobj | +| examples.py:27:8:27:12 | [post arg] ControlFlowNode for myobj [Attribute foo] | examples.py:28:6:28:10 | ControlFlowNode for myobj [Attribute foo] | +| examples.py:27:15:27:20 | ControlFlowNode for SOURCE | examples.py:21:17:21:17 | SSA variable x | +| examples.py:27:15:27:20 | ControlFlowNode for SOURCE | examples.py:21:17:21:17 | SSA variable x | +| examples.py:27:15:27:20 | ControlFlowNode for SOURCE | examples.py:27:8:27:12 | [post arg] ControlFlowNode for myobj [Attribute foo] | +| examples.py:27:15:27:20 | ControlFlowNode for SOURCE | examples.py:31:1:31:1 | GSSA Variable x | +| examples.py:27:15:27:20 | ControlFlowNode for SOURCE | examples.py:31:1:31:1 | GSSA Variable x | +| examples.py:27:15:27:20 | ControlFlowNode for SOURCE | examples.py:31:5:31:10 | ControlFlowNode for SOURCE | +| examples.py:27:15:27:20 | ControlFlowNode for SOURCE | examples.py:31:5:31:10 | ControlFlowNode for SOURCE | +| examples.py:27:15:27:20 | ControlFlowNode for SOURCE | examples.py:35:13:35:13 | ControlFlowNode for x | +| examples.py:27:15:27:20 | ControlFlowNode for SOURCE | examples.py:35:13:35:13 | ControlFlowNode for x | +| examples.py:27:15:27:20 | ControlFlowNode for SOURCE | examples.py:36:18:36:18 | ControlFlowNode for x | +| examples.py:27:15:27:20 | ControlFlowNode for SOURCE | examples.py:36:18:36:18 | ControlFlowNode for x | +| examples.py:27:15:27:20 | ControlFlowNode for SOURCE | examples.py:41:13:41:18 | ControlFlowNode for SOURCE | +| examples.py:27:15:27:20 | ControlFlowNode for SOURCE | examples.py:41:13:41:18 | ControlFlowNode for SOURCE | +| examples.py:27:15:27:20 | ControlFlowNode for SOURCE | examples.py:50:29:50:34 | ControlFlowNode for SOURCE | +| examples.py:27:15:27:20 | ControlFlowNode for SOURCE | examples.py:50:29:50:34 | ControlFlowNode for SOURCE | +| examples.py:27:15:27:20 | [post arg] ControlFlowNode for SOURCE | examples.py:31:1:31:1 | GSSA Variable x | +| examples.py:27:15:27:20 | [post arg] ControlFlowNode for SOURCE | examples.py:31:1:31:1 | GSSA Variable x | +| examples.py:27:15:27:20 | [post arg] ControlFlowNode for SOURCE | examples.py:31:5:31:10 | ControlFlowNode for SOURCE | +| examples.py:27:15:27:20 | [post arg] ControlFlowNode for SOURCE | examples.py:31:5:31:10 | ControlFlowNode for SOURCE | +| examples.py:27:15:27:20 | [post arg] ControlFlowNode for SOURCE | examples.py:35:13:35:13 | ControlFlowNode for x | +| examples.py:27:15:27:20 | [post arg] ControlFlowNode for SOURCE | examples.py:35:13:35:13 | ControlFlowNode for x | +| examples.py:27:15:27:20 | [post arg] ControlFlowNode for SOURCE | examples.py:36:18:36:18 | ControlFlowNode for x | +| examples.py:27:15:27:20 | [post arg] ControlFlowNode for SOURCE | examples.py:36:18:36:18 | ControlFlowNode for x | +| examples.py:27:15:27:20 | [post arg] ControlFlowNode for SOURCE | examples.py:41:13:41:18 | ControlFlowNode for SOURCE | +| examples.py:27:15:27:20 | [post arg] ControlFlowNode for SOURCE | examples.py:41:13:41:18 | ControlFlowNode for SOURCE | +| examples.py:27:15:27:20 | [post arg] ControlFlowNode for SOURCE | examples.py:50:29:50:34 | ControlFlowNode for SOURCE | +| examples.py:27:15:27:20 | [post arg] ControlFlowNode for SOURCE | examples.py:50:29:50:34 | ControlFlowNode for SOURCE | +| examples.py:28:1:28:4 | ControlFlowNode for SINK | examples.py:38:1:38:4 | ControlFlowNode for SINK | +| examples.py:28:1:28:4 | ControlFlowNode for SINK | examples.py:38:1:38:4 | ControlFlowNode for SINK | +| examples.py:28:1:28:4 | ControlFlowNode for SINK | examples.py:42:1:42:4 | ControlFlowNode for SINK | +| examples.py:28:1:28:4 | ControlFlowNode for SINK | examples.py:42:1:42:4 | ControlFlowNode for SINK | +| examples.py:28:1:28:4 | ControlFlowNode for SINK | examples.py:50:1:50:4 | ControlFlowNode for SINK | +| examples.py:28:1:28:4 | ControlFlowNode for SINK | examples.py:50:1:50:4 | ControlFlowNode for SINK | +| examples.py:28:6:28:10 | ControlFlowNode for myobj [Attribute foo] | examples.py:28:6:28:14 | ControlFlowNode for Attribute | +| examples.py:28:6:28:10 | ControlFlowNode for myobj [Attribute foo] | examples.py:28:6:28:14 | ControlFlowNode for Attribute | +| examples.py:31:1:31:1 | GSSA Variable x | examples.py:35:13:35:13 | ControlFlowNode for x | +| examples.py:31:1:31:1 | GSSA Variable x | examples.py:35:13:35:13 | ControlFlowNode for x | +| examples.py:31:1:31:1 | GSSA Variable x | examples.py:36:18:36:18 | ControlFlowNode for x | +| examples.py:31:1:31:1 | GSSA Variable x | examples.py:36:18:36:18 | ControlFlowNode for x | +| examples.py:31:5:31:10 | ControlFlowNode for SOURCE | examples.py:31:1:31:1 | GSSA Variable x | +| examples.py:31:5:31:10 | ControlFlowNode for SOURCE | examples.py:31:1:31:1 | GSSA Variable x | +| examples.py:31:5:31:10 | ControlFlowNode for SOURCE | examples.py:35:13:35:13 | ControlFlowNode for x | +| examples.py:31:5:31:10 | ControlFlowNode for SOURCE | examples.py:35:13:35:13 | ControlFlowNode for x | +| examples.py:31:5:31:10 | ControlFlowNode for SOURCE | examples.py:36:18:36:18 | ControlFlowNode for x | +| examples.py:31:5:31:10 | ControlFlowNode for SOURCE | examples.py:36:18:36:18 | ControlFlowNode for x | +| examples.py:31:5:31:10 | ControlFlowNode for SOURCE | examples.py:41:13:41:18 | ControlFlowNode for SOURCE | +| examples.py:31:5:31:10 | ControlFlowNode for SOURCE | examples.py:41:13:41:18 | ControlFlowNode for SOURCE | +| examples.py:31:5:31:10 | ControlFlowNode for SOURCE | examples.py:50:29:50:34 | ControlFlowNode for SOURCE | +| examples.py:31:5:31:10 | ControlFlowNode for SOURCE | examples.py:50:29:50:34 | ControlFlowNode for SOURCE | +| examples.py:33:1:33:1 | GSSA Variable a | examples.py:35:1:35:1 | ControlFlowNode for a | +| examples.py:33:1:33:1 | GSSA Variable a | examples.py:35:1:35:1 | ControlFlowNode for a | +| examples.py:33:1:33:1 | GSSA Variable a | examples.py:36:1:36:1 | ControlFlowNode for a | +| examples.py:33:1:33:1 | GSSA Variable a | examples.py:36:1:36:1 | ControlFlowNode for a | +| examples.py:33:1:33:1 | GSSA Variable a | examples.py:36:1:36:10 | GSSA Variable a | +| examples.py:33:1:33:1 | GSSA Variable a | examples.py:36:1:36:10 | GSSA Variable a | +| examples.py:33:1:33:1 | GSSA Variable a | examples.py:38:6:38:6 | ControlFlowNode for a | +| examples.py:33:1:33:1 | GSSA Variable a | examples.py:38:6:38:6 | ControlFlowNode for a | +| examples.py:33:1:33:1 | GSSA Variable a [Attribute obj, Attribute foo] | examples.py:35:1:35:1 | ControlFlowNode for a [Attribute obj, Attribute foo] | +| examples.py:33:1:33:1 | GSSA Variable a [Attribute obj, Attribute foo] | examples.py:36:1:36:1 | ControlFlowNode for a [Attribute obj, Attribute foo] | +| examples.py:33:1:33:1 | GSSA Variable a [Attribute obj, Attribute foo] | examples.py:38:6:38:6 | ControlFlowNode for a [Attribute obj, Attribute foo] | +| examples.py:33:1:33:1 | GSSA Variable a [Attribute obj] | examples.py:35:1:35:1 | ControlFlowNode for a [Attribute obj] | +| examples.py:33:1:33:1 | GSSA Variable a [Attribute obj] | examples.py:36:1:36:1 | ControlFlowNode for a [Attribute obj] | +| examples.py:33:1:33:1 | GSSA Variable a [Attribute obj] | examples.py:38:6:38:6 | ControlFlowNode for a [Attribute obj] | +| examples.py:33:5:33:15 | ControlFlowNode for NestedObj() | examples.py:33:1:33:1 | GSSA Variable a | +| examples.py:33:5:33:15 | ControlFlowNode for NestedObj() | examples.py:33:1:33:1 | GSSA Variable a | +| examples.py:33:5:33:15 | ControlFlowNode for NestedObj() | examples.py:35:1:35:1 | ControlFlowNode for a | +| examples.py:33:5:33:15 | ControlFlowNode for NestedObj() | examples.py:35:1:35:1 | ControlFlowNode for a | +| examples.py:33:5:33:15 | ControlFlowNode for NestedObj() | examples.py:36:1:36:1 | ControlFlowNode for a | +| examples.py:33:5:33:15 | ControlFlowNode for NestedObj() | examples.py:36:1:36:1 | ControlFlowNode for a | +| examples.py:33:5:33:15 | ControlFlowNode for NestedObj() | examples.py:36:1:36:10 | GSSA Variable a | +| examples.py:33:5:33:15 | ControlFlowNode for NestedObj() | examples.py:36:1:36:10 | GSSA Variable a | +| examples.py:33:5:33:15 | ControlFlowNode for NestedObj() | examples.py:38:6:38:6 | ControlFlowNode for a | +| examples.py:33:5:33:15 | ControlFlowNode for NestedObj() | examples.py:38:6:38:6 | ControlFlowNode for a | +| examples.py:33:5:33:15 | ControlFlowNode for NestedObj() [Attribute obj, Attribute foo] | examples.py:33:1:33:1 | GSSA Variable a [Attribute obj, Attribute foo] | +| examples.py:33:5:33:15 | ControlFlowNode for NestedObj() [Attribute obj, Attribute foo] | examples.py:35:1:35:1 | ControlFlowNode for a [Attribute obj, Attribute foo] | +| examples.py:33:5:33:15 | ControlFlowNode for NestedObj() [Attribute obj, Attribute foo] | examples.py:36:1:36:1 | ControlFlowNode for a [Attribute obj, Attribute foo] | +| examples.py:33:5:33:15 | ControlFlowNode for NestedObj() [Attribute obj, Attribute foo] | examples.py:38:6:38:6 | ControlFlowNode for a [Attribute obj, Attribute foo] | +| examples.py:33:5:33:15 | ControlFlowNode for NestedObj() [Attribute obj] | examples.py:33:1:33:1 | GSSA Variable a [Attribute obj] | +| examples.py:33:5:33:15 | ControlFlowNode for NestedObj() [Attribute obj] | examples.py:35:1:35:1 | ControlFlowNode for a [Attribute obj] | +| examples.py:33:5:33:15 | ControlFlowNode for NestedObj() [Attribute obj] | examples.py:36:1:36:1 | ControlFlowNode for a [Attribute obj] | +| examples.py:33:5:33:15 | ControlFlowNode for NestedObj() [Attribute obj] | examples.py:38:6:38:6 | ControlFlowNode for a [Attribute obj] | +| examples.py:33:5:33:15 | [pre objCreate] ControlFlowNode for NestedObj() | examples.py:13:18:13:21 | SSA variable self | +| examples.py:33:5:33:15 | [pre objCreate] ControlFlowNode for NestedObj() | examples.py:13:18:13:21 | SSA variable self | +| examples.py:35:1:35:1 | ControlFlowNode for a | examples.py:36:1:36:1 | ControlFlowNode for a | +| examples.py:35:1:35:1 | ControlFlowNode for a | examples.py:36:1:36:1 | ControlFlowNode for a | +| examples.py:35:1:35:1 | ControlFlowNode for a | examples.py:38:6:38:6 | ControlFlowNode for a | +| examples.py:35:1:35:1 | ControlFlowNode for a | examples.py:38:6:38:6 | ControlFlowNode for a | +| examples.py:35:1:35:1 | ControlFlowNode for a [Attribute obj, Attribute foo] | examples.py:36:1:36:1 | ControlFlowNode for a [Attribute obj, Attribute foo] | +| examples.py:35:1:35:1 | ControlFlowNode for a [Attribute obj, Attribute foo] | examples.py:38:6:38:6 | ControlFlowNode for a [Attribute obj, Attribute foo] | +| examples.py:35:1:35:1 | ControlFlowNode for a [Attribute obj] | examples.py:35:1:35:5 | ControlFlowNode for Attribute | +| examples.py:35:1:35:1 | ControlFlowNode for a [Attribute obj] | examples.py:35:1:35:5 | ControlFlowNode for Attribute | +| examples.py:35:1:35:1 | ControlFlowNode for a [Attribute obj] | examples.py:36:1:36:1 | ControlFlowNode for a [Attribute obj] | +| examples.py:35:1:35:1 | ControlFlowNode for a [Attribute obj] | examples.py:38:6:38:6 | ControlFlowNode for a [Attribute obj] | +| examples.py:35:1:35:1 | [post read] ControlFlowNode for a | examples.py:36:1:36:1 | ControlFlowNode for a | +| examples.py:35:1:35:1 | [post read] ControlFlowNode for a | examples.py:36:1:36:1 | ControlFlowNode for a | +| examples.py:35:1:35:1 | [post read] ControlFlowNode for a | examples.py:38:6:38:6 | ControlFlowNode for a | +| examples.py:35:1:35:1 | [post read] ControlFlowNode for a | examples.py:38:6:38:6 | ControlFlowNode for a | +| examples.py:35:1:35:1 | [post read] ControlFlowNode for a [Attribute obj, Attribute foo] | examples.py:36:1:36:1 | ControlFlowNode for a [Attribute obj, Attribute foo] | +| examples.py:35:1:35:1 | [post read] ControlFlowNode for a [Attribute obj, Attribute foo] | examples.py:38:6:38:6 | ControlFlowNode for a [Attribute obj, Attribute foo] | +| examples.py:35:1:35:1 | [post read] ControlFlowNode for a [Attribute obj] | examples.py:36:1:36:1 | ControlFlowNode for a [Attribute obj] | +| examples.py:35:1:35:1 | [post read] ControlFlowNode for a [Attribute obj] | examples.py:38:6:38:6 | ControlFlowNode for a [Attribute obj] | +| examples.py:35:1:35:5 | [post store] ControlFlowNode for Attribute | examples.py:35:1:35:1 | [post read] ControlFlowNode for a [Attribute obj] | +| examples.py:35:1:35:5 | [post store] ControlFlowNode for Attribute [Attribute foo] | examples.py:35:1:35:1 | [post read] ControlFlowNode for a [Attribute obj, Attribute foo] | +| examples.py:35:13:35:13 | ControlFlowNode for x | examples.py:35:1:35:5 | [post store] ControlFlowNode for Attribute [Attribute foo] | +| examples.py:35:13:35:13 | ControlFlowNode for x | examples.py:36:18:36:18 | ControlFlowNode for x | +| examples.py:35:13:35:13 | ControlFlowNode for x | examples.py:36:18:36:18 | ControlFlowNode for x | +| examples.py:36:1:36:1 | ControlFlowNode for a | examples.py:38:6:38:6 | ControlFlowNode for a | +| examples.py:36:1:36:1 | ControlFlowNode for a | examples.py:38:6:38:6 | ControlFlowNode for a | +| examples.py:36:1:36:1 | ControlFlowNode for a [Attribute obj, Attribute foo] | examples.py:38:6:38:6 | ControlFlowNode for a [Attribute obj, Attribute foo] | +| examples.py:36:1:36:1 | ControlFlowNode for a [Attribute obj] | examples.py:38:6:38:6 | ControlFlowNode for a [Attribute obj] | +| examples.py:36:1:36:1 | [post read] ControlFlowNode for a | examples.py:38:6:38:6 | ControlFlowNode for a | +| examples.py:36:1:36:1 | [post read] ControlFlowNode for a | examples.py:38:6:38:6 | ControlFlowNode for a | +| examples.py:38:1:38:4 | ControlFlowNode for SINK | examples.py:42:1:42:4 | ControlFlowNode for SINK | +| examples.py:38:1:38:4 | ControlFlowNode for SINK | examples.py:42:1:42:4 | ControlFlowNode for SINK | +| examples.py:38:1:38:4 | ControlFlowNode for SINK | examples.py:50:1:50:4 | ControlFlowNode for SINK | +| examples.py:38:1:38:4 | ControlFlowNode for SINK | examples.py:50:1:50:4 | ControlFlowNode for SINK | +| examples.py:38:6:38:6 | ControlFlowNode for a [Attribute obj, Attribute foo] | examples.py:38:6:38:10 | ControlFlowNode for Attribute [Attribute foo] | +| examples.py:38:6:38:6 | ControlFlowNode for a [Attribute obj] | examples.py:38:6:38:10 | ControlFlowNode for Attribute | +| examples.py:38:6:38:6 | ControlFlowNode for a [Attribute obj] | examples.py:38:6:38:10 | ControlFlowNode for Attribute | +| examples.py:38:6:38:10 | ControlFlowNode for Attribute [Attribute foo] | examples.py:38:6:38:14 | ControlFlowNode for Attribute | +| examples.py:38:6:38:10 | ControlFlowNode for Attribute [Attribute foo] | examples.py:38:6:38:14 | ControlFlowNode for Attribute | +| examples.py:41:1:41:3 | GSSA Variable obj | examples.py:42:6:42:8 | ControlFlowNode for obj | +| examples.py:41:1:41:3 | GSSA Variable obj | examples.py:42:6:42:8 | ControlFlowNode for obj | +| examples.py:41:1:41:3 | GSSA Variable obj [Attribute foo] | examples.py:42:6:42:8 | ControlFlowNode for obj [Attribute foo] | +| examples.py:41:7:41:19 | ControlFlowNode for MyObj() | examples.py:41:1:41:3 | GSSA Variable obj | +| examples.py:41:7:41:19 | ControlFlowNode for MyObj() | examples.py:41:1:41:3 | GSSA Variable obj | +| examples.py:41:7:41:19 | ControlFlowNode for MyObj() | examples.py:42:6:42:8 | ControlFlowNode for obj | +| examples.py:41:7:41:19 | ControlFlowNode for MyObj() | examples.py:42:6:42:8 | ControlFlowNode for obj | +| examples.py:41:7:41:19 | ControlFlowNode for MyObj() [Attribute foo] | examples.py:41:1:41:3 | GSSA Variable obj [Attribute foo] | +| examples.py:41:7:41:19 | ControlFlowNode for MyObj() [Attribute foo] | examples.py:42:6:42:8 | ControlFlowNode for obj [Attribute foo] | +| examples.py:41:7:41:19 | GSSA Variable SOURCE | examples.py:50:6:50:35 | GSSA Variable SOURCE | +| examples.py:41:7:41:19 | GSSA Variable SOURCE | examples.py:50:6:50:35 | GSSA Variable SOURCE | +| examples.py:41:7:41:19 | [pre objCreate] ControlFlowNode for MyObj() | examples.py:7:18:7:21 | SSA variable self | +| examples.py:41:7:41:19 | [pre objCreate] ControlFlowNode for MyObj() | examples.py:7:18:7:21 | SSA variable self | +| examples.py:41:13:41:18 | ControlFlowNode for SOURCE | examples.py:7:24:7:26 | SSA variable foo | +| examples.py:41:13:41:18 | ControlFlowNode for SOURCE | examples.py:7:24:7:26 | SSA variable foo | +| examples.py:41:13:41:18 | ControlFlowNode for SOURCE | examples.py:41:7:41:19 | ControlFlowNode for MyObj() [Attribute foo] | +| examples.py:41:13:41:18 | ControlFlowNode for SOURCE | examples.py:50:29:50:34 | ControlFlowNode for SOURCE | +| examples.py:41:13:41:18 | ControlFlowNode for SOURCE | examples.py:50:29:50:34 | ControlFlowNode for SOURCE | +| examples.py:41:13:41:18 | [post arg] ControlFlowNode for SOURCE | examples.py:50:29:50:34 | ControlFlowNode for SOURCE | +| examples.py:41:13:41:18 | [post arg] ControlFlowNode for SOURCE | examples.py:50:29:50:34 | ControlFlowNode for SOURCE | +| examples.py:42:1:42:4 | ControlFlowNode for SINK | examples.py:50:1:50:4 | ControlFlowNode for SINK | +| examples.py:42:1:42:4 | ControlFlowNode for SINK | examples.py:50:1:50:4 | ControlFlowNode for SINK | +| examples.py:42:6:42:8 | ControlFlowNode for obj [Attribute foo] | examples.py:42:6:42:12 | ControlFlowNode for Attribute | +| examples.py:42:6:42:8 | ControlFlowNode for obj [Attribute foo] | examples.py:42:6:42:12 | ControlFlowNode for Attribute | +| examples.py:45:1:45:30 | ControlFlowNode for FunctionExpr | examples.py:45:5:45:26 | GSSA Variable fields_with_local_flow | +| examples.py:45:1:45:30 | ControlFlowNode for FunctionExpr | examples.py:45:5:45:26 | GSSA Variable fields_with_local_flow | +| examples.py:45:1:45:30 | ControlFlowNode for FunctionExpr | examples.py:50:6:50:27 | ControlFlowNode for fields_with_local_flow | +| examples.py:45:1:45:30 | ControlFlowNode for FunctionExpr | examples.py:50:6:50:27 | ControlFlowNode for fields_with_local_flow | +| examples.py:45:1:45:30 | GSSA Variable MyObj | examples.py:46:9:46:13 | ControlFlowNode for MyObj | +| examples.py:45:1:45:30 | GSSA Variable MyObj | examples.py:46:9:46:13 | ControlFlowNode for MyObj | +| examples.py:45:5:45:26 | GSSA Variable fields_with_local_flow | examples.py:50:6:50:27 | ControlFlowNode for fields_with_local_flow | +| examples.py:45:5:45:26 | GSSA Variable fields_with_local_flow | examples.py:50:6:50:27 | ControlFlowNode for fields_with_local_flow | +| examples.py:45:28:45:28 | SSA variable x | examples.py:46:9:46:16 | SSA variable x | +| examples.py:45:28:45:28 | SSA variable x | examples.py:46:9:46:16 | SSA variable x | +| examples.py:45:28:45:28 | SSA variable x | examples.py:46:9:46:16 | SSA variable x | +| examples.py:45:28:45:28 | SSA variable x | examples.py:46:9:46:16 | SSA variable x | +| examples.py:45:28:45:28 | SSA variable x | examples.py:46:15:46:15 | ControlFlowNode for x | +| examples.py:45:28:45:28 | SSA variable x | examples.py:46:15:46:15 | ControlFlowNode for x | +| examples.py:45:28:45:28 | SSA variable x | examples.py:46:15:46:15 | ControlFlowNode for x | +| examples.py:45:28:45:28 | SSA variable x | examples.py:46:15:46:15 | ControlFlowNode for x | +| examples.py:46:3:46:5 | SSA variable obj | examples.py:47:7:47:9 | ControlFlowNode for obj | +| examples.py:46:3:46:5 | SSA variable obj | examples.py:47:7:47:9 | ControlFlowNode for obj | +| examples.py:46:3:46:5 | SSA variable obj [Attribute foo] | examples.py:47:7:47:9 | ControlFlowNode for obj [Attribute foo] | +| examples.py:46:3:46:5 | SSA variable obj [Attribute foo] | examples.py:47:7:47:9 | ControlFlowNode for obj [Attribute foo] | +| examples.py:46:9:46:16 | ControlFlowNode for MyObj() | examples.py:46:3:46:5 | SSA variable obj | +| examples.py:46:9:46:16 | ControlFlowNode for MyObj() | examples.py:46:3:46:5 | SSA variable obj | +| examples.py:46:9:46:16 | ControlFlowNode for MyObj() | examples.py:47:7:47:9 | ControlFlowNode for obj | +| examples.py:46:9:46:16 | ControlFlowNode for MyObj() | examples.py:47:7:47:9 | ControlFlowNode for obj | +| examples.py:46:9:46:16 | ControlFlowNode for MyObj() [Attribute foo] | examples.py:46:3:46:5 | SSA variable obj [Attribute foo] | +| examples.py:46:9:46:16 | ControlFlowNode for MyObj() [Attribute foo] | examples.py:46:3:46:5 | SSA variable obj [Attribute foo] | +| examples.py:46:9:46:16 | ControlFlowNode for MyObj() [Attribute foo] | examples.py:47:7:47:9 | ControlFlowNode for obj [Attribute foo] | +| examples.py:46:9:46:16 | ControlFlowNode for MyObj() [Attribute foo] | examples.py:47:7:47:9 | ControlFlowNode for obj [Attribute foo] | +| examples.py:46:9:46:16 | [pre objCreate] ControlFlowNode for MyObj() | examples.py:7:18:7:21 | SSA variable self | +| examples.py:46:9:46:16 | [pre objCreate] ControlFlowNode for MyObj() | examples.py:7:18:7:21 | SSA variable self | +| examples.py:46:15:46:15 | ControlFlowNode for x | examples.py:7:24:7:26 | SSA variable foo | +| examples.py:46:15:46:15 | ControlFlowNode for x | examples.py:7:24:7:26 | SSA variable foo | +| examples.py:46:15:46:15 | ControlFlowNode for x | examples.py:7:24:7:26 | SSA variable foo | +| examples.py:46:15:46:15 | ControlFlowNode for x | examples.py:7:24:7:26 | SSA variable foo | +| examples.py:46:15:46:15 | ControlFlowNode for x | examples.py:46:9:46:16 | ControlFlowNode for MyObj() [Attribute foo] | +| examples.py:46:15:46:15 | ControlFlowNode for x | examples.py:46:9:46:16 | ControlFlowNode for MyObj() [Attribute foo] | +| examples.py:46:15:46:15 | [post arg] ControlFlowNode for x | examples.py:50:29:50:34 | [post arg] ControlFlowNode for SOURCE | +| examples.py:46:15:46:15 | [post arg] ControlFlowNode for x | examples.py:50:29:50:34 | [post arg] ControlFlowNode for SOURCE | +| examples.py:47:3:47:3 | SSA variable a | examples.py:48:10:48:10 | ControlFlowNode for a | +| examples.py:47:3:47:3 | SSA variable a | examples.py:48:10:48:10 | ControlFlowNode for a | +| examples.py:47:3:47:3 | SSA variable a | examples.py:48:10:48:10 | ControlFlowNode for a | +| examples.py:47:3:47:3 | SSA variable a | examples.py:48:10:48:10 | ControlFlowNode for a | +| examples.py:47:7:47:9 | ControlFlowNode for obj [Attribute foo] | examples.py:47:7:47:13 | ControlFlowNode for Attribute | +| examples.py:47:7:47:9 | ControlFlowNode for obj [Attribute foo] | examples.py:47:7:47:13 | ControlFlowNode for Attribute | +| examples.py:47:7:47:9 | ControlFlowNode for obj [Attribute foo] | examples.py:47:7:47:13 | ControlFlowNode for Attribute | +| examples.py:47:7:47:9 | ControlFlowNode for obj [Attribute foo] | examples.py:47:7:47:13 | ControlFlowNode for Attribute | +| examples.py:47:7:47:13 | ControlFlowNode for Attribute | examples.py:47:3:47:3 | SSA variable a | +| examples.py:47:7:47:13 | ControlFlowNode for Attribute | examples.py:47:3:47:3 | SSA variable a | +| examples.py:47:7:47:13 | ControlFlowNode for Attribute | examples.py:47:3:47:3 | SSA variable a | +| examples.py:47:7:47:13 | ControlFlowNode for Attribute | examples.py:47:3:47:3 | SSA variable a | +| examples.py:47:7:47:13 | ControlFlowNode for Attribute | examples.py:48:10:48:10 | ControlFlowNode for a | +| examples.py:47:7:47:13 | ControlFlowNode for Attribute | examples.py:48:10:48:10 | ControlFlowNode for a | +| examples.py:47:7:47:13 | ControlFlowNode for Attribute | examples.py:48:10:48:10 | ControlFlowNode for a | +| examples.py:47:7:47:13 | ControlFlowNode for Attribute | examples.py:48:10:48:10 | ControlFlowNode for a | +| examples.py:48:10:48:10 | ControlFlowNode for a | examples.py:50:6:50:35 | ControlFlowNode for fields_with_local_flow() | +| examples.py:48:10:48:10 | ControlFlowNode for a | examples.py:50:6:50:35 | ControlFlowNode for fields_with_local_flow() | +| examples.py:50:29:50:34 | ControlFlowNode for SOURCE | examples.py:45:28:45:28 | SSA variable x | +| examples.py:50:29:50:34 | ControlFlowNode for SOURCE | examples.py:45:28:45:28 | SSA variable x | +| examples.py:50:29:50:34 | ControlFlowNode for SOURCE | examples.py:50:6:50:35 | ControlFlowNode for fields_with_local_flow() | +| examples.py:50:29:50:34 | ControlFlowNode for SOURCE | examples.py:50:6:50:35 | ControlFlowNode for fields_with_local_flow() | +| test.py:0:0:0:0 | GSSA Variable SINK | test.py:1:1:1:66 | GSSA Variable SINK | +| test.py:0:0:0:0 | GSSA Variable SINK | test.py:1:1:1:66 | GSSA Variable SINK | +| test.py:0:0:0:0 | GSSA Variable SINK_F | test.py:1:1:1:66 | GSSA Variable SINK_F | +| test.py:0:0:0:0 | GSSA Variable SINK_F | test.py:1:1:1:66 | GSSA Variable SINK_F | +| test.py:0:0:0:0 | GSSA Variable SOURCE | test.py:1:1:1:66 | GSSA Variable SOURCE | +| test.py:0:0:0:0 | GSSA Variable SOURCE | test.py:1:1:1:66 | GSSA Variable SOURCE | +| test.py:0:0:0:0 | GSSA Variable __name__ | test.py:1:1:1:66 | GSSA Variable __name__ | +| test.py:0:0:0:0 | GSSA Variable __name__ | test.py:1:1:1:66 | GSSA Variable __name__ | +| test.py:0:0:0:0 | GSSA Variable __package__ | test.py:1:1:1:66 | GSSA Variable __package__ | +| test.py:0:0:0:0 | GSSA Variable __package__ | test.py:1:1:1:66 | GSSA Variable __package__ | +| test.py:0:0:0:0 | GSSA Variable object | test.py:1:1:1:66 | GSSA Variable object | +| test.py:0:0:0:0 | GSSA Variable object | test.py:1:1:1:66 | GSSA Variable object | +| test.py:0:0:0:0 | GSSA Variable object | test.py:6:13:6:18 | ControlFlowNode for object | +| test.py:0:0:0:0 | GSSA Variable object | test.py:6:13:6:18 | ControlFlowNode for object | +| test.py:0:0:0:0 | GSSA Variable object | test.py:12:17:12:22 | ControlFlowNode for object | +| test.py:0:0:0:0 | GSSA Variable object | test.py:12:17:12:22 | ControlFlowNode for object | +| test.py:0:0:0:0 | ModuleVariableNode for Global Variable MyObj in Module test | test.py:15:20:15:24 | ControlFlowNode for MyObj | +| test.py:0:0:0:0 | ModuleVariableNode for Global Variable MyObj in Module test | test.py:15:20:15:24 | ControlFlowNode for MyObj | +| test.py:0:0:0:0 | ModuleVariableNode for Global Variable MyObj in Module test | test.py:27:13:27:17 | ControlFlowNode for MyObj | +| test.py:0:0:0:0 | ModuleVariableNode for Global Variable MyObj in Module test | test.py:27:13:27:17 | ControlFlowNode for MyObj | +| test.py:0:0:0:0 | ModuleVariableNode for Global Variable MyObj in Module test | test.py:45:11:45:15 | ControlFlowNode for MyObj | +| test.py:0:0:0:0 | ModuleVariableNode for Global Variable MyObj in Module test | test.py:45:11:45:15 | ControlFlowNode for MyObj | +| test.py:0:0:0:0 | ModuleVariableNode for Global Variable MyObj in Module test | test.py:50:11:50:15 | ControlFlowNode for MyObj | +| test.py:0:0:0:0 | ModuleVariableNode for Global Variable MyObj in Module test | test.py:50:11:50:15 | ControlFlowNode for MyObj | +| test.py:0:0:0:0 | ModuleVariableNode for Global Variable NestedObj in Module test | test.py:36:9:36:17 | ControlFlowNode for NestedObj | +| test.py:0:0:0:0 | ModuleVariableNode for Global Variable NestedObj in Module test | test.py:36:9:36:17 | ControlFlowNode for NestedObj | +| test.py:0:0:0:0 | ModuleVariableNode for Global Variable SINK in Module test | test.py:30:5:30:8 | ControlFlowNode for SINK | +| test.py:0:0:0:0 | ModuleVariableNode for Global Variable SINK in Module test | test.py:30:5:30:8 | ControlFlowNode for SINK | +| test.py:0:0:0:0 | ModuleVariableNode for Global Variable SINK in Module test | test.py:41:5:41:8 | ControlFlowNode for SINK | +| test.py:0:0:0:0 | ModuleVariableNode for Global Variable SINK in Module test | test.py:41:5:41:8 | ControlFlowNode for SINK | +| test.py:0:0:0:0 | ModuleVariableNode for Global Variable SINK in Module test | test.py:46:5:46:8 | ControlFlowNode for SINK | +| test.py:0:0:0:0 | ModuleVariableNode for Global Variable SINK in Module test | test.py:46:5:46:8 | ControlFlowNode for SINK | +| test.py:0:0:0:0 | ModuleVariableNode for Global Variable SINK in Module test | test.py:56:5:56:8 | ControlFlowNode for SINK | +| test.py:0:0:0:0 | ModuleVariableNode for Global Variable SINK in Module test | test.py:56:5:56:8 | ControlFlowNode for SINK | +| test.py:0:0:0:0 | ModuleVariableNode for Global Variable SINK_F in Module test | test.py:22:5:22:10 | ControlFlowNode for SINK_F | +| test.py:0:0:0:0 | ModuleVariableNode for Global Variable SINK_F in Module test | test.py:22:5:22:10 | ControlFlowNode for SINK_F | +| test.py:0:0:0:0 | ModuleVariableNode for Global Variable SOURCE in Module test | test.py:29:19:29:24 | ControlFlowNode for SOURCE | +| test.py:0:0:0:0 | ModuleVariableNode for Global Variable SOURCE in Module test | test.py:29:19:29:24 | ControlFlowNode for SOURCE | +| test.py:0:0:0:0 | ModuleVariableNode for Global Variable SOURCE in Module test | test.py:34:9:34:14 | ControlFlowNode for SOURCE | +| test.py:0:0:0:0 | ModuleVariableNode for Global Variable SOURCE in Module test | test.py:34:9:34:14 | ControlFlowNode for SOURCE | +| test.py:0:0:0:0 | ModuleVariableNode for Global Variable SOURCE in Module test | test.py:45:17:45:22 | ControlFlowNode for SOURCE | +| test.py:0:0:0:0 | ModuleVariableNode for Global Variable SOURCE in Module test | test.py:45:17:45:22 | ControlFlowNode for SOURCE | +| test.py:0:0:0:0 | ModuleVariableNode for Global Variable SOURCE in Module test | test.py:56:33:56:38 | ControlFlowNode for SOURCE | +| test.py:0:0:0:0 | ModuleVariableNode for Global Variable SOURCE in Module test | test.py:56:33:56:38 | ControlFlowNode for SOURCE | +| test.py:0:0:0:0 | ModuleVariableNode for Global Variable fields_with_local_flow in Module test | test.py:56:10:56:31 | ControlFlowNode for fields_with_local_flow | +| test.py:0:0:0:0 | ModuleVariableNode for Global Variable fields_with_local_flow in Module test | test.py:56:10:56:31 | ControlFlowNode for fields_with_local_flow | +| test.py:0:0:0:0 | ModuleVariableNode for Global Variable setFoo in Module test | test.py:29:5:29:10 | ControlFlowNode for setFoo | +| test.py:0:0:0:0 | ModuleVariableNode for Global Variable setFoo in Module test | test.py:29:5:29:10 | ControlFlowNode for setFoo | +| test.py:0:0:0:0 | SSA variable $ | test.py:1:1:1:66 | SSA variable $ | +| test.py:0:0:0:0 | SSA variable $ | test.py:1:1:1:66 | SSA variable $ | +| test.py:0:0:0:0 | SSA variable * | test.py:1:1:1:66 | SSA variable * | +| test.py:0:0:0:0 | SSA variable * | test.py:1:1:1:66 | SSA variable * | +| test.py:6:1:6:20 | ControlFlowNode for ClassExpr | test.py:6:7:6:11 | GSSA Variable MyObj | +| test.py:6:1:6:20 | ControlFlowNode for ClassExpr | test.py:6:7:6:11 | GSSA Variable MyObj | +| test.py:6:7:6:11 | GSSA Variable MyObj | test.py:0:0:0:0 | ModuleVariableNode for Global Variable MyObj in Module test | +| test.py:6:7:6:11 | GSSA Variable MyObj | test.py:0:0:0:0 | ModuleVariableNode for Global Variable MyObj in Module test | +| test.py:6:13:6:18 | ControlFlowNode for object | test.py:12:17:12:22 | ControlFlowNode for object | +| test.py:6:13:6:18 | ControlFlowNode for object | test.py:12:17:12:22 | ControlFlowNode for object | +| test.py:8:5:8:28 | ControlFlowNode for FunctionExpr | test.py:8:9:8:16 | SSA variable __init__ | +| test.py:8:5:8:28 | ControlFlowNode for FunctionExpr | test.py:8:9:8:16 | SSA variable __init__ | +| test.py:8:18:8:21 | SSA variable self | test.py:9:9:9:12 | ControlFlowNode for self | +| test.py:8:18:8:21 | SSA variable self | test.py:9:9:9:12 | ControlFlowNode for self | +| test.py:8:18:8:21 | SSA variable self | test.py:9:9:9:12 | ControlFlowNode for self | +| test.py:8:18:8:21 | SSA variable self | test.py:9:9:9:12 | ControlFlowNode for self | +| test.py:8:18:8:21 | SSA variable self | test.py:9:9:9:16 | SSA variable self | +| test.py:8:18:8:21 | SSA variable self | test.py:9:9:9:16 | SSA variable self | +| test.py:8:18:8:21 | SSA variable self | test.py:9:9:9:16 | SSA variable self | +| test.py:8:18:8:21 | SSA variable self | test.py:9:9:9:16 | SSA variable self | +| test.py:8:24:8:26 | SSA variable foo | test.py:9:20:9:22 | ControlFlowNode for foo | +| test.py:8:24:8:26 | SSA variable foo | test.py:9:20:9:22 | ControlFlowNode for foo | +| test.py:8:24:8:26 | SSA variable foo | test.py:9:20:9:22 | ControlFlowNode for foo | +| test.py:8:24:8:26 | SSA variable foo | test.py:9:20:9:22 | ControlFlowNode for foo | +| test.py:9:9:9:12 | [post store] ControlFlowNode for self | test.py:15:20:15:30 | ControlFlowNode for MyObj() | +| test.py:9:9:9:12 | [post store] ControlFlowNode for self | test.py:15:20:15:30 | ControlFlowNode for MyObj() | +| test.py:9:9:9:12 | [post store] ControlFlowNode for self | test.py:27:13:27:23 | ControlFlowNode for MyObj() | +| test.py:9:9:9:12 | [post store] ControlFlowNode for self | test.py:27:13:27:23 | ControlFlowNode for MyObj() | +| test.py:9:9:9:12 | [post store] ControlFlowNode for self | test.py:45:11:45:23 | ControlFlowNode for MyObj() | +| test.py:9:9:9:12 | [post store] ControlFlowNode for self | test.py:45:11:45:23 | ControlFlowNode for MyObj() | +| test.py:9:9:9:12 | [post store] ControlFlowNode for self | test.py:50:11:50:18 | ControlFlowNode for MyObj() | +| test.py:9:9:9:12 | [post store] ControlFlowNode for self | test.py:50:11:50:18 | ControlFlowNode for MyObj() | +| test.py:9:9:9:12 | [post store] ControlFlowNode for self [Attribute foo] | test.py:15:20:15:30 | ControlFlowNode for MyObj() [Attribute foo] | +| test.py:9:9:9:12 | [post store] ControlFlowNode for self [Attribute foo] | test.py:27:13:27:23 | ControlFlowNode for MyObj() [Attribute foo] | +| test.py:9:9:9:12 | [post store] ControlFlowNode for self [Attribute foo] | test.py:45:11:45:23 | ControlFlowNode for MyObj() [Attribute foo] | +| test.py:9:9:9:12 | [post store] ControlFlowNode for self [Attribute foo] | test.py:50:11:50:18 | ControlFlowNode for MyObj() [Attribute foo] | +| test.py:9:20:9:22 | ControlFlowNode for foo | test.py:9:9:9:12 | [post store] ControlFlowNode for self [Attribute foo] | +| test.py:9:20:9:22 | ControlFlowNode for foo | test.py:9:9:9:12 | [post store] ControlFlowNode for self [Attribute foo] | +| test.py:12:1:12:24 | ControlFlowNode for ClassExpr | test.py:12:7:12:15 | GSSA Variable NestedObj | +| test.py:12:1:12:24 | ControlFlowNode for ClassExpr | test.py:12:7:12:15 | GSSA Variable NestedObj | +| test.py:12:7:12:15 | GSSA Variable NestedObj | test.py:0:0:0:0 | ModuleVariableNode for Global Variable NestedObj in Module test | +| test.py:12:7:12:15 | GSSA Variable NestedObj | test.py:0:0:0:0 | ModuleVariableNode for Global Variable NestedObj in Module test | +| test.py:14:5:14:23 | ControlFlowNode for FunctionExpr | test.py:14:9:14:16 | SSA variable __init__ | +| test.py:14:5:14:23 | ControlFlowNode for FunctionExpr | test.py:14:9:14:16 | SSA variable __init__ | +| test.py:14:5:14:23 | GSSA Variable MyObj | test.py:15:20:15:24 | ControlFlowNode for MyObj | +| test.py:14:5:14:23 | GSSA Variable MyObj | test.py:15:20:15:24 | ControlFlowNode for MyObj | +| test.py:14:18:14:21 | SSA variable self | test.py:15:9:15:12 | ControlFlowNode for self | +| test.py:14:18:14:21 | SSA variable self | test.py:15:9:15:12 | ControlFlowNode for self | +| test.py:14:18:14:21 | SSA variable self | test.py:15:9:15:12 | ControlFlowNode for self | +| test.py:14:18:14:21 | SSA variable self | test.py:15:9:15:12 | ControlFlowNode for self | +| test.py:14:18:14:21 | SSA variable self | test.py:15:9:15:16 | SSA variable self | +| test.py:14:18:14:21 | SSA variable self | test.py:15:9:15:16 | SSA variable self | +| test.py:14:18:14:21 | SSA variable self | test.py:15:9:15:16 | SSA variable self | +| test.py:14:18:14:21 | SSA variable self | test.py:15:9:15:16 | SSA variable self | +| test.py:15:9:15:12 | [post store] ControlFlowNode for self | test.py:36:9:36:19 | ControlFlowNode for NestedObj() | +| test.py:15:9:15:12 | [post store] ControlFlowNode for self | test.py:36:9:36:19 | ControlFlowNode for NestedObj() | +| test.py:15:9:15:12 | [post store] ControlFlowNode for self [Attribute obj, Attribute foo] | test.py:36:9:36:19 | ControlFlowNode for NestedObj() [Attribute obj, Attribute foo] | +| test.py:15:9:15:12 | [post store] ControlFlowNode for self [Attribute obj] | test.py:36:9:36:19 | ControlFlowNode for NestedObj() [Attribute obj] | +| test.py:15:20:15:30 | ControlFlowNode for MyObj() | test.py:15:9:15:12 | [post store] ControlFlowNode for self [Attribute obj] | +| test.py:15:20:15:30 | ControlFlowNode for MyObj() [Attribute foo] | test.py:15:9:15:12 | [post store] ControlFlowNode for self [Attribute obj, Attribute foo] | +| test.py:15:20:15:30 | [pre objCreate] ControlFlowNode for MyObj() | test.py:8:18:8:21 | SSA variable self | +| test.py:15:20:15:30 | [pre objCreate] ControlFlowNode for MyObj() | test.py:8:18:8:21 | SSA variable self | +| test.py:15:26:15:29 | ControlFlowNode for Str | test.py:8:24:8:26 | SSA variable foo | +| test.py:15:26:15:29 | ControlFlowNode for Str | test.py:8:24:8:26 | SSA variable foo | +| test.py:15:26:15:29 | ControlFlowNode for Str | test.py:15:20:15:30 | ControlFlowNode for MyObj() [Attribute foo] | +| test.py:17:5:17:21 | ControlFlowNode for FunctionExpr | test.py:17:9:17:14 | SSA variable getObj | +| test.py:17:5:17:21 | ControlFlowNode for FunctionExpr | test.py:17:9:17:14 | SSA variable getObj | +| test.py:17:16:17:19 | SSA variable self | test.py:18:16:18:19 | ControlFlowNode for self | +| test.py:17:16:17:19 | SSA variable self | test.py:18:16:18:19 | ControlFlowNode for self | +| test.py:21:1:21:19 | ControlFlowNode for FunctionExpr | test.py:21:5:21:10 | GSSA Variable setFoo | +| test.py:21:1:21:19 | ControlFlowNode for FunctionExpr | test.py:21:5:21:10 | GSSA Variable setFoo | +| test.py:21:1:21:19 | GSSA Variable SINK_F | test.py:22:5:22:10 | ControlFlowNode for SINK_F | +| test.py:21:1:21:19 | GSSA Variable SINK_F | test.py:22:5:22:10 | ControlFlowNode for SINK_F | +| test.py:21:5:21:10 | GSSA Variable setFoo | test.py:0:0:0:0 | ModuleVariableNode for Global Variable setFoo in Module test | +| test.py:21:5:21:10 | GSSA Variable setFoo | test.py:0:0:0:0 | ModuleVariableNode for Global Variable setFoo in Module test | +| test.py:21:12:21:14 | SSA variable obj | test.py:22:12:22:14 | ControlFlowNode for obj | +| test.py:21:12:21:14 | SSA variable obj | test.py:22:12:22:14 | ControlFlowNode for obj | +| test.py:21:12:21:14 | SSA variable obj | test.py:22:12:22:14 | ControlFlowNode for obj | +| test.py:21:12:21:14 | SSA variable obj | test.py:22:12:22:14 | ControlFlowNode for obj | +| test.py:21:12:21:14 | SSA variable obj | test.py:23:5:23:7 | ControlFlowNode for obj | +| test.py:21:12:21:14 | SSA variable obj | test.py:23:5:23:7 | ControlFlowNode for obj | +| test.py:21:12:21:14 | SSA variable obj | test.py:23:5:23:7 | ControlFlowNode for obj | +| test.py:21:12:21:14 | SSA variable obj | test.py:23:5:23:7 | ControlFlowNode for obj | +| test.py:21:12:21:14 | SSA variable obj | test.py:23:5:23:11 | SSA variable obj | +| test.py:21:12:21:14 | SSA variable obj | test.py:23:5:23:11 | SSA variable obj | +| test.py:21:12:21:14 | SSA variable obj | test.py:23:5:23:11 | SSA variable obj | +| test.py:21:12:21:14 | SSA variable obj | test.py:23:5:23:11 | SSA variable obj | +| test.py:21:12:21:14 | SSA variable obj [Attribute foo] | test.py:22:12:22:14 | ControlFlowNode for obj [Attribute foo] | +| test.py:21:17:21:17 | SSA variable x | test.py:23:15:23:15 | ControlFlowNode for x | +| test.py:21:17:21:17 | SSA variable x | test.py:23:15:23:15 | ControlFlowNode for x | +| test.py:21:17:21:17 | SSA variable x | test.py:23:15:23:15 | ControlFlowNode for x | +| test.py:21:17:21:17 | SSA variable x | test.py:23:15:23:15 | ControlFlowNode for x | +| test.py:22:12:22:14 | ControlFlowNode for obj | test.py:23:5:23:7 | ControlFlowNode for obj | +| test.py:22:12:22:14 | ControlFlowNode for obj | test.py:23:5:23:7 | ControlFlowNode for obj | +| test.py:22:12:22:14 | ControlFlowNode for obj | test.py:23:5:23:7 | ControlFlowNode for obj | +| test.py:22:12:22:14 | ControlFlowNode for obj | test.py:23:5:23:7 | ControlFlowNode for obj | +| test.py:22:12:22:14 | ControlFlowNode for obj [Attribute foo] | test.py:22:12:22:18 | ControlFlowNode for Attribute | +| test.py:22:12:22:14 | ControlFlowNode for obj [Attribute foo] | test.py:22:12:22:18 | ControlFlowNode for Attribute | +| test.py:22:12:22:14 | [post read] ControlFlowNode for obj | test.py:23:5:23:7 | ControlFlowNode for obj | +| test.py:22:12:22:14 | [post read] ControlFlowNode for obj | test.py:23:5:23:7 | ControlFlowNode for obj | +| test.py:22:12:22:14 | [post read] ControlFlowNode for obj | test.py:29:12:29:16 | [post arg] ControlFlowNode for myobj | +| test.py:22:12:22:14 | [post read] ControlFlowNode for obj | test.py:29:12:29:16 | [post arg] ControlFlowNode for myobj | +| test.py:23:5:23:7 | [post store] ControlFlowNode for obj | test.py:29:12:29:16 | [post arg] ControlFlowNode for myobj | +| test.py:23:5:23:7 | [post store] ControlFlowNode for obj | test.py:29:12:29:16 | [post arg] ControlFlowNode for myobj | +| test.py:23:5:23:7 | [post store] ControlFlowNode for obj [Attribute foo] | test.py:29:12:29:16 | [post arg] ControlFlowNode for myobj [Attribute foo] | +| test.py:23:15:23:15 | ControlFlowNode for x | test.py:23:5:23:7 | [post store] ControlFlowNode for obj [Attribute foo] | +| test.py:23:15:23:15 | ControlFlowNode for x | test.py:23:5:23:7 | [post store] ControlFlowNode for obj [Attribute foo] | +| test.py:26:1:26:20 | ControlFlowNode for FunctionExpr | test.py:26:5:26:17 | GSSA Variable test_example1 | +| test.py:26:1:26:20 | ControlFlowNode for FunctionExpr | test.py:26:5:26:17 | GSSA Variable test_example1 | +| test.py:26:1:26:20 | GSSA Variable MyObj | test.py:27:13:27:17 | ControlFlowNode for MyObj | +| test.py:26:1:26:20 | GSSA Variable MyObj | test.py:27:13:27:17 | ControlFlowNode for MyObj | +| test.py:26:1:26:20 | GSSA Variable SINK | test.py:30:5:30:8 | ControlFlowNode for SINK | +| test.py:26:1:26:20 | GSSA Variable SINK | test.py:30:5:30:8 | ControlFlowNode for SINK | +| test.py:26:1:26:20 | GSSA Variable SOURCE | test.py:29:19:29:24 | ControlFlowNode for SOURCE | +| test.py:26:1:26:20 | GSSA Variable SOURCE | test.py:29:19:29:24 | ControlFlowNode for SOURCE | +| test.py:26:1:26:20 | GSSA Variable setFoo | test.py:29:5:29:10 | ControlFlowNode for setFoo | +| test.py:26:1:26:20 | GSSA Variable setFoo | test.py:29:5:29:10 | ControlFlowNode for setFoo | +| test.py:27:5:27:9 | SSA variable myobj | test.py:29:5:29:25 | SSA variable myobj | +| test.py:27:5:27:9 | SSA variable myobj | test.py:29:5:29:25 | SSA variable myobj | +| test.py:27:5:27:9 | SSA variable myobj | test.py:29:12:29:16 | ControlFlowNode for myobj | +| test.py:27:5:27:9 | SSA variable myobj | test.py:29:12:29:16 | ControlFlowNode for myobj | +| test.py:27:5:27:9 | SSA variable myobj | test.py:30:10:30:14 | ControlFlowNode for myobj | +| test.py:27:5:27:9 | SSA variable myobj | test.py:30:10:30:14 | ControlFlowNode for myobj | +| test.py:27:5:27:9 | SSA variable myobj [Attribute foo] | test.py:29:12:29:16 | ControlFlowNode for myobj [Attribute foo] | +| test.py:27:5:27:9 | SSA variable myobj [Attribute foo] | test.py:30:10:30:14 | ControlFlowNode for myobj [Attribute foo] | +| test.py:27:13:27:23 | ControlFlowNode for MyObj() | test.py:27:5:27:9 | SSA variable myobj | +| test.py:27:13:27:23 | ControlFlowNode for MyObj() | test.py:27:5:27:9 | SSA variable myobj | +| test.py:27:13:27:23 | ControlFlowNode for MyObj() | test.py:29:5:29:25 | SSA variable myobj | +| test.py:27:13:27:23 | ControlFlowNode for MyObj() | test.py:29:5:29:25 | SSA variable myobj | +| test.py:27:13:27:23 | ControlFlowNode for MyObj() | test.py:29:12:29:16 | ControlFlowNode for myobj | +| test.py:27:13:27:23 | ControlFlowNode for MyObj() | test.py:29:12:29:16 | ControlFlowNode for myobj | +| test.py:27:13:27:23 | ControlFlowNode for MyObj() | test.py:30:10:30:14 | ControlFlowNode for myobj | +| test.py:27:13:27:23 | ControlFlowNode for MyObj() | test.py:30:10:30:14 | ControlFlowNode for myobj | +| test.py:27:13:27:23 | ControlFlowNode for MyObj() [Attribute foo] | test.py:27:5:27:9 | SSA variable myobj [Attribute foo] | +| test.py:27:13:27:23 | ControlFlowNode for MyObj() [Attribute foo] | test.py:29:12:29:16 | ControlFlowNode for myobj [Attribute foo] | +| test.py:27:13:27:23 | ControlFlowNode for MyObj() [Attribute foo] | test.py:30:10:30:14 | ControlFlowNode for myobj [Attribute foo] | +| test.py:27:13:27:23 | [pre objCreate] ControlFlowNode for MyObj() | test.py:8:18:8:21 | SSA variable self | +| test.py:27:13:27:23 | [pre objCreate] ControlFlowNode for MyObj() | test.py:8:18:8:21 | SSA variable self | +| test.py:27:19:27:22 | ControlFlowNode for Str | test.py:8:24:8:26 | SSA variable foo | +| test.py:27:19:27:22 | ControlFlowNode for Str | test.py:8:24:8:26 | SSA variable foo | +| test.py:27:19:27:22 | ControlFlowNode for Str | test.py:27:13:27:23 | ControlFlowNode for MyObj() [Attribute foo] | +| test.py:29:12:29:16 | ControlFlowNode for myobj | test.py:21:12:21:14 | SSA variable obj | +| test.py:29:12:29:16 | ControlFlowNode for myobj | test.py:21:12:21:14 | SSA variable obj | +| test.py:29:12:29:16 | ControlFlowNode for myobj | test.py:30:10:30:14 | ControlFlowNode for myobj | +| test.py:29:12:29:16 | ControlFlowNode for myobj | test.py:30:10:30:14 | ControlFlowNode for myobj | +| test.py:29:12:29:16 | ControlFlowNode for myobj [Attribute foo] | test.py:21:12:21:14 | SSA variable obj [Attribute foo] | +| test.py:29:12:29:16 | ControlFlowNode for myobj [Attribute foo] | test.py:30:10:30:14 | ControlFlowNode for myobj [Attribute foo] | +| test.py:29:12:29:16 | [post arg] ControlFlowNode for myobj | test.py:30:10:30:14 | ControlFlowNode for myobj | +| test.py:29:12:29:16 | [post arg] ControlFlowNode for myobj | test.py:30:10:30:14 | ControlFlowNode for myobj | +| test.py:29:12:29:16 | [post arg] ControlFlowNode for myobj [Attribute foo] | test.py:30:10:30:14 | ControlFlowNode for myobj [Attribute foo] | +| test.py:29:19:29:24 | ControlFlowNode for SOURCE | test.py:21:17:21:17 | SSA variable x | +| test.py:29:19:29:24 | ControlFlowNode for SOURCE | test.py:21:17:21:17 | SSA variable x | +| test.py:29:19:29:24 | ControlFlowNode for SOURCE | test.py:29:12:29:16 | [post arg] ControlFlowNode for myobj [Attribute foo] | +| test.py:30:10:30:14 | ControlFlowNode for myobj [Attribute foo] | test.py:30:10:30:18 | ControlFlowNode for Attribute | +| test.py:30:10:30:14 | ControlFlowNode for myobj [Attribute foo] | test.py:30:10:30:18 | ControlFlowNode for Attribute | +| test.py:33:1:33:20 | ControlFlowNode for FunctionExpr | test.py:33:5:33:17 | GSSA Variable test_example2 | +| test.py:33:1:33:20 | ControlFlowNode for FunctionExpr | test.py:33:5:33:17 | GSSA Variable test_example2 | +| test.py:33:1:33:20 | GSSA Variable NestedObj | test.py:36:9:36:17 | ControlFlowNode for NestedObj | +| test.py:33:1:33:20 | GSSA Variable NestedObj | test.py:36:9:36:17 | ControlFlowNode for NestedObj | +| test.py:33:1:33:20 | GSSA Variable SINK | test.py:41:5:41:8 | ControlFlowNode for SINK | +| test.py:33:1:33:20 | GSSA Variable SINK | test.py:41:5:41:8 | ControlFlowNode for SINK | +| test.py:33:1:33:20 | GSSA Variable SOURCE | test.py:34:5:34:5 | SSA variable x | +| test.py:33:1:33:20 | GSSA Variable SOURCE | test.py:34:5:34:5 | SSA variable x | +| test.py:33:1:33:20 | GSSA Variable SOURCE | test.py:34:9:34:14 | ControlFlowNode for SOURCE | +| test.py:33:1:33:20 | GSSA Variable SOURCE | test.py:34:9:34:14 | ControlFlowNode for SOURCE | +| test.py:33:1:33:20 | GSSA Variable SOURCE | test.py:38:17:38:17 | ControlFlowNode for x | +| test.py:33:1:33:20 | GSSA Variable SOURCE | test.py:38:17:38:17 | ControlFlowNode for x | +| test.py:33:1:33:20 | GSSA Variable SOURCE | test.py:39:22:39:22 | ControlFlowNode for x | +| test.py:33:1:33:20 | GSSA Variable SOURCE | test.py:39:22:39:22 | ControlFlowNode for x | +| test.py:34:5:34:5 | SSA variable x | test.py:38:17:38:17 | ControlFlowNode for x | +| test.py:34:5:34:5 | SSA variable x | test.py:38:17:38:17 | ControlFlowNode for x | +| test.py:34:5:34:5 | SSA variable x | test.py:39:22:39:22 | ControlFlowNode for x | +| test.py:34:5:34:5 | SSA variable x | test.py:39:22:39:22 | ControlFlowNode for x | +| test.py:34:9:34:14 | ControlFlowNode for SOURCE | test.py:34:5:34:5 | SSA variable x | +| test.py:34:9:34:14 | ControlFlowNode for SOURCE | test.py:34:5:34:5 | SSA variable x | +| test.py:34:9:34:14 | ControlFlowNode for SOURCE | test.py:38:17:38:17 | ControlFlowNode for x | +| test.py:34:9:34:14 | ControlFlowNode for SOURCE | test.py:38:17:38:17 | ControlFlowNode for x | +| test.py:34:9:34:14 | ControlFlowNode for SOURCE | test.py:39:22:39:22 | ControlFlowNode for x | +| test.py:34:9:34:14 | ControlFlowNode for SOURCE | test.py:39:22:39:22 | ControlFlowNode for x | +| test.py:36:5:36:5 | SSA variable a | test.py:38:5:38:5 | ControlFlowNode for a | +| test.py:36:5:36:5 | SSA variable a | test.py:38:5:38:5 | ControlFlowNode for a | +| test.py:36:5:36:5 | SSA variable a | test.py:39:5:39:5 | ControlFlowNode for a | +| test.py:36:5:36:5 | SSA variable a | test.py:39:5:39:5 | ControlFlowNode for a | +| test.py:36:5:36:5 | SSA variable a | test.py:39:5:39:14 | SSA variable a | +| test.py:36:5:36:5 | SSA variable a | test.py:39:5:39:14 | SSA variable a | +| test.py:36:5:36:5 | SSA variable a | test.py:41:10:41:10 | ControlFlowNode for a | +| test.py:36:5:36:5 | SSA variable a | test.py:41:10:41:10 | ControlFlowNode for a | +| test.py:36:5:36:5 | SSA variable a [Attribute obj, Attribute foo] | test.py:38:5:38:5 | ControlFlowNode for a [Attribute obj, Attribute foo] | +| test.py:36:5:36:5 | SSA variable a [Attribute obj, Attribute foo] | test.py:39:5:39:5 | ControlFlowNode for a [Attribute obj, Attribute foo] | +| test.py:36:5:36:5 | SSA variable a [Attribute obj, Attribute foo] | test.py:41:10:41:10 | ControlFlowNode for a [Attribute obj, Attribute foo] | +| test.py:36:5:36:5 | SSA variable a [Attribute obj] | test.py:38:5:38:5 | ControlFlowNode for a [Attribute obj] | +| test.py:36:5:36:5 | SSA variable a [Attribute obj] | test.py:39:5:39:5 | ControlFlowNode for a [Attribute obj] | +| test.py:36:5:36:5 | SSA variable a [Attribute obj] | test.py:41:10:41:10 | ControlFlowNode for a [Attribute obj] | +| test.py:36:9:36:19 | ControlFlowNode for NestedObj() | test.py:36:5:36:5 | SSA variable a | +| test.py:36:9:36:19 | ControlFlowNode for NestedObj() | test.py:36:5:36:5 | SSA variable a | +| test.py:36:9:36:19 | ControlFlowNode for NestedObj() | test.py:38:5:38:5 | ControlFlowNode for a | +| test.py:36:9:36:19 | ControlFlowNode for NestedObj() | test.py:38:5:38:5 | ControlFlowNode for a | +| test.py:36:9:36:19 | ControlFlowNode for NestedObj() | test.py:39:5:39:5 | ControlFlowNode for a | +| test.py:36:9:36:19 | ControlFlowNode for NestedObj() | test.py:39:5:39:5 | ControlFlowNode for a | +| test.py:36:9:36:19 | ControlFlowNode for NestedObj() | test.py:39:5:39:14 | SSA variable a | +| test.py:36:9:36:19 | ControlFlowNode for NestedObj() | test.py:39:5:39:14 | SSA variable a | +| test.py:36:9:36:19 | ControlFlowNode for NestedObj() | test.py:41:10:41:10 | ControlFlowNode for a | +| test.py:36:9:36:19 | ControlFlowNode for NestedObj() | test.py:41:10:41:10 | ControlFlowNode for a | +| test.py:36:9:36:19 | ControlFlowNode for NestedObj() [Attribute obj, Attribute foo] | test.py:36:5:36:5 | SSA variable a [Attribute obj, Attribute foo] | +| test.py:36:9:36:19 | ControlFlowNode for NestedObj() [Attribute obj, Attribute foo] | test.py:38:5:38:5 | ControlFlowNode for a [Attribute obj, Attribute foo] | +| test.py:36:9:36:19 | ControlFlowNode for NestedObj() [Attribute obj, Attribute foo] | test.py:39:5:39:5 | ControlFlowNode for a [Attribute obj, Attribute foo] | +| test.py:36:9:36:19 | ControlFlowNode for NestedObj() [Attribute obj, Attribute foo] | test.py:41:10:41:10 | ControlFlowNode for a [Attribute obj, Attribute foo] | +| test.py:36:9:36:19 | ControlFlowNode for NestedObj() [Attribute obj] | test.py:36:5:36:5 | SSA variable a [Attribute obj] | +| test.py:36:9:36:19 | ControlFlowNode for NestedObj() [Attribute obj] | test.py:38:5:38:5 | ControlFlowNode for a [Attribute obj] | +| test.py:36:9:36:19 | ControlFlowNode for NestedObj() [Attribute obj] | test.py:39:5:39:5 | ControlFlowNode for a [Attribute obj] | +| test.py:36:9:36:19 | ControlFlowNode for NestedObj() [Attribute obj] | test.py:41:10:41:10 | ControlFlowNode for a [Attribute obj] | +| test.py:36:9:36:19 | [pre objCreate] ControlFlowNode for NestedObj() | test.py:14:18:14:21 | SSA variable self | +| test.py:36:9:36:19 | [pre objCreate] ControlFlowNode for NestedObj() | test.py:14:18:14:21 | SSA variable self | +| test.py:38:5:38:5 | ControlFlowNode for a | test.py:39:5:39:5 | ControlFlowNode for a | +| test.py:38:5:38:5 | ControlFlowNode for a | test.py:39:5:39:5 | ControlFlowNode for a | +| test.py:38:5:38:5 | ControlFlowNode for a | test.py:41:10:41:10 | ControlFlowNode for a | +| test.py:38:5:38:5 | ControlFlowNode for a | test.py:41:10:41:10 | ControlFlowNode for a | +| test.py:38:5:38:5 | ControlFlowNode for a [Attribute obj, Attribute foo] | test.py:39:5:39:5 | ControlFlowNode for a [Attribute obj, Attribute foo] | +| test.py:38:5:38:5 | ControlFlowNode for a [Attribute obj, Attribute foo] | test.py:41:10:41:10 | ControlFlowNode for a [Attribute obj, Attribute foo] | +| test.py:38:5:38:5 | ControlFlowNode for a [Attribute obj] | test.py:38:5:38:9 | ControlFlowNode for Attribute | +| test.py:38:5:38:5 | ControlFlowNode for a [Attribute obj] | test.py:38:5:38:9 | ControlFlowNode for Attribute | +| test.py:38:5:38:5 | ControlFlowNode for a [Attribute obj] | test.py:39:5:39:5 | ControlFlowNode for a [Attribute obj] | +| test.py:38:5:38:5 | ControlFlowNode for a [Attribute obj] | test.py:41:10:41:10 | ControlFlowNode for a [Attribute obj] | +| test.py:38:5:38:5 | [post read] ControlFlowNode for a | test.py:39:5:39:5 | ControlFlowNode for a | +| test.py:38:5:38:5 | [post read] ControlFlowNode for a | test.py:39:5:39:5 | ControlFlowNode for a | +| test.py:38:5:38:5 | [post read] ControlFlowNode for a | test.py:41:10:41:10 | ControlFlowNode for a | +| test.py:38:5:38:5 | [post read] ControlFlowNode for a | test.py:41:10:41:10 | ControlFlowNode for a | +| test.py:38:5:38:5 | [post read] ControlFlowNode for a [Attribute obj, Attribute foo] | test.py:39:5:39:5 | ControlFlowNode for a [Attribute obj, Attribute foo] | +| test.py:38:5:38:5 | [post read] ControlFlowNode for a [Attribute obj, Attribute foo] | test.py:41:10:41:10 | ControlFlowNode for a [Attribute obj, Attribute foo] | +| test.py:38:5:38:5 | [post read] ControlFlowNode for a [Attribute obj] | test.py:39:5:39:5 | ControlFlowNode for a [Attribute obj] | +| test.py:38:5:38:5 | [post read] ControlFlowNode for a [Attribute obj] | test.py:41:10:41:10 | ControlFlowNode for a [Attribute obj] | +| test.py:38:5:38:9 | [post store] ControlFlowNode for Attribute | test.py:38:5:38:5 | [post read] ControlFlowNode for a [Attribute obj] | +| test.py:38:5:38:9 | [post store] ControlFlowNode for Attribute [Attribute foo] | test.py:38:5:38:5 | [post read] ControlFlowNode for a [Attribute obj, Attribute foo] | +| test.py:38:17:38:17 | ControlFlowNode for x | test.py:38:5:38:9 | [post store] ControlFlowNode for Attribute [Attribute foo] | +| test.py:38:17:38:17 | ControlFlowNode for x | test.py:39:22:39:22 | ControlFlowNode for x | +| test.py:38:17:38:17 | ControlFlowNode for x | test.py:39:22:39:22 | ControlFlowNode for x | +| test.py:39:5:39:5 | ControlFlowNode for a | test.py:41:10:41:10 | ControlFlowNode for a | +| test.py:39:5:39:5 | ControlFlowNode for a | test.py:41:10:41:10 | ControlFlowNode for a | +| test.py:39:5:39:5 | ControlFlowNode for a [Attribute obj, Attribute foo] | test.py:41:10:41:10 | ControlFlowNode for a [Attribute obj, Attribute foo] | +| test.py:39:5:39:5 | ControlFlowNode for a [Attribute obj] | test.py:41:10:41:10 | ControlFlowNode for a [Attribute obj] | +| test.py:39:5:39:5 | [post read] ControlFlowNode for a | test.py:41:10:41:10 | ControlFlowNode for a | +| test.py:39:5:39:5 | [post read] ControlFlowNode for a | test.py:41:10:41:10 | ControlFlowNode for a | +| test.py:41:10:41:10 | ControlFlowNode for a [Attribute obj, Attribute foo] | test.py:41:10:41:14 | ControlFlowNode for Attribute [Attribute foo] | +| test.py:41:10:41:10 | ControlFlowNode for a [Attribute obj] | test.py:41:10:41:14 | ControlFlowNode for Attribute | +| test.py:41:10:41:10 | ControlFlowNode for a [Attribute obj] | test.py:41:10:41:14 | ControlFlowNode for Attribute | +| test.py:41:10:41:14 | ControlFlowNode for Attribute [Attribute foo] | test.py:41:10:41:18 | ControlFlowNode for Attribute | +| test.py:41:10:41:14 | ControlFlowNode for Attribute [Attribute foo] | test.py:41:10:41:18 | ControlFlowNode for Attribute | +| test.py:44:1:44:20 | ControlFlowNode for FunctionExpr | test.py:44:5:44:17 | GSSA Variable test_example3 | +| test.py:44:1:44:20 | ControlFlowNode for FunctionExpr | test.py:44:5:44:17 | GSSA Variable test_example3 | +| test.py:44:1:44:20 | GSSA Variable MyObj | test.py:45:11:45:15 | ControlFlowNode for MyObj | +| test.py:44:1:44:20 | GSSA Variable MyObj | test.py:45:11:45:15 | ControlFlowNode for MyObj | +| test.py:44:1:44:20 | GSSA Variable SINK | test.py:46:5:46:8 | ControlFlowNode for SINK | +| test.py:44:1:44:20 | GSSA Variable SINK | test.py:46:5:46:8 | ControlFlowNode for SINK | +| test.py:44:1:44:20 | GSSA Variable SOURCE | test.py:45:17:45:22 | ControlFlowNode for SOURCE | +| test.py:44:1:44:20 | GSSA Variable SOURCE | test.py:45:17:45:22 | ControlFlowNode for SOURCE | +| test.py:45:5:45:7 | SSA variable obj | test.py:46:10:46:12 | ControlFlowNode for obj | +| test.py:45:5:45:7 | SSA variable obj | test.py:46:10:46:12 | ControlFlowNode for obj | +| test.py:45:5:45:7 | SSA variable obj [Attribute foo] | test.py:46:10:46:12 | ControlFlowNode for obj [Attribute foo] | +| test.py:45:11:45:23 | ControlFlowNode for MyObj() | test.py:45:5:45:7 | SSA variable obj | +| test.py:45:11:45:23 | ControlFlowNode for MyObj() | test.py:45:5:45:7 | SSA variable obj | +| test.py:45:11:45:23 | ControlFlowNode for MyObj() | test.py:46:10:46:12 | ControlFlowNode for obj | +| test.py:45:11:45:23 | ControlFlowNode for MyObj() | test.py:46:10:46:12 | ControlFlowNode for obj | +| test.py:45:11:45:23 | ControlFlowNode for MyObj() [Attribute foo] | test.py:45:5:45:7 | SSA variable obj [Attribute foo] | +| test.py:45:11:45:23 | ControlFlowNode for MyObj() [Attribute foo] | test.py:46:10:46:12 | ControlFlowNode for obj [Attribute foo] | +| test.py:45:11:45:23 | [pre objCreate] ControlFlowNode for MyObj() | test.py:8:18:8:21 | SSA variable self | +| test.py:45:11:45:23 | [pre objCreate] ControlFlowNode for MyObj() | test.py:8:18:8:21 | SSA variable self | +| test.py:45:17:45:22 | ControlFlowNode for SOURCE | test.py:8:24:8:26 | SSA variable foo | +| test.py:45:17:45:22 | ControlFlowNode for SOURCE | test.py:8:24:8:26 | SSA variable foo | +| test.py:45:17:45:22 | ControlFlowNode for SOURCE | test.py:45:11:45:23 | ControlFlowNode for MyObj() [Attribute foo] | +| test.py:46:10:46:12 | ControlFlowNode for obj [Attribute foo] | test.py:46:10:46:16 | ControlFlowNode for Attribute | +| test.py:46:10:46:12 | ControlFlowNode for obj [Attribute foo] | test.py:46:10:46:16 | ControlFlowNode for Attribute | +| test.py:49:1:49:30 | ControlFlowNode for FunctionExpr | test.py:49:5:49:26 | GSSA Variable fields_with_local_flow | +| test.py:49:1:49:30 | ControlFlowNode for FunctionExpr | test.py:49:5:49:26 | GSSA Variable fields_with_local_flow | +| test.py:49:1:49:30 | GSSA Variable MyObj | test.py:50:11:50:15 | ControlFlowNode for MyObj | +| test.py:49:1:49:30 | GSSA Variable MyObj | test.py:50:11:50:15 | ControlFlowNode for MyObj | +| test.py:49:5:49:26 | GSSA Variable fields_with_local_flow | test.py:0:0:0:0 | ModuleVariableNode for Global Variable fields_with_local_flow in Module test | +| test.py:49:5:49:26 | GSSA Variable fields_with_local_flow | test.py:0:0:0:0 | ModuleVariableNode for Global Variable fields_with_local_flow in Module test | +| test.py:49:28:49:28 | SSA variable x | test.py:50:11:50:18 | SSA variable x | +| test.py:49:28:49:28 | SSA variable x | test.py:50:11:50:18 | SSA variable x | +| test.py:49:28:49:28 | SSA variable x | test.py:50:11:50:18 | SSA variable x | +| test.py:49:28:49:28 | SSA variable x | test.py:50:11:50:18 | SSA variable x | +| test.py:49:28:49:28 | SSA variable x | test.py:50:17:50:17 | ControlFlowNode for x | +| test.py:49:28:49:28 | SSA variable x | test.py:50:17:50:17 | ControlFlowNode for x | +| test.py:49:28:49:28 | SSA variable x | test.py:50:17:50:17 | ControlFlowNode for x | +| test.py:49:28:49:28 | SSA variable x | test.py:50:17:50:17 | ControlFlowNode for x | +| test.py:50:5:50:7 | SSA variable obj | test.py:51:9:51:11 | ControlFlowNode for obj | +| test.py:50:5:50:7 | SSA variable obj | test.py:51:9:51:11 | ControlFlowNode for obj | +| test.py:50:5:50:7 | SSA variable obj [Attribute foo] | test.py:51:9:51:11 | ControlFlowNode for obj [Attribute foo] | +| test.py:50:5:50:7 | SSA variable obj [Attribute foo] | test.py:51:9:51:11 | ControlFlowNode for obj [Attribute foo] | +| test.py:50:11:50:18 | ControlFlowNode for MyObj() | test.py:50:5:50:7 | SSA variable obj | +| test.py:50:11:50:18 | ControlFlowNode for MyObj() | test.py:50:5:50:7 | SSA variable obj | +| test.py:50:11:50:18 | ControlFlowNode for MyObj() | test.py:51:9:51:11 | ControlFlowNode for obj | +| test.py:50:11:50:18 | ControlFlowNode for MyObj() | test.py:51:9:51:11 | ControlFlowNode for obj | +| test.py:50:11:50:18 | ControlFlowNode for MyObj() [Attribute foo] | test.py:50:5:50:7 | SSA variable obj [Attribute foo] | +| test.py:50:11:50:18 | ControlFlowNode for MyObj() [Attribute foo] | test.py:50:5:50:7 | SSA variable obj [Attribute foo] | +| test.py:50:11:50:18 | ControlFlowNode for MyObj() [Attribute foo] | test.py:51:9:51:11 | ControlFlowNode for obj [Attribute foo] | +| test.py:50:11:50:18 | ControlFlowNode for MyObj() [Attribute foo] | test.py:51:9:51:11 | ControlFlowNode for obj [Attribute foo] | +| test.py:50:11:50:18 | [pre objCreate] ControlFlowNode for MyObj() | test.py:8:18:8:21 | SSA variable self | +| test.py:50:11:50:18 | [pre objCreate] ControlFlowNode for MyObj() | test.py:8:18:8:21 | SSA variable self | +| test.py:50:17:50:17 | ControlFlowNode for x | test.py:8:24:8:26 | SSA variable foo | +| test.py:50:17:50:17 | ControlFlowNode for x | test.py:8:24:8:26 | SSA variable foo | +| test.py:50:17:50:17 | ControlFlowNode for x | test.py:8:24:8:26 | SSA variable foo | +| test.py:50:17:50:17 | ControlFlowNode for x | test.py:8:24:8:26 | SSA variable foo | +| test.py:50:17:50:17 | ControlFlowNode for x | test.py:50:11:50:18 | ControlFlowNode for MyObj() [Attribute foo] | +| test.py:50:17:50:17 | ControlFlowNode for x | test.py:50:11:50:18 | ControlFlowNode for MyObj() [Attribute foo] | +| test.py:50:17:50:17 | [post arg] ControlFlowNode for x | test.py:56:33:56:38 | [post arg] ControlFlowNode for SOURCE | +| test.py:50:17:50:17 | [post arg] ControlFlowNode for x | test.py:56:33:56:38 | [post arg] ControlFlowNode for SOURCE | +| test.py:51:5:51:5 | SSA variable a | test.py:52:12:52:12 | ControlFlowNode for a | +| test.py:51:5:51:5 | SSA variable a | test.py:52:12:52:12 | ControlFlowNode for a | +| test.py:51:5:51:5 | SSA variable a | test.py:52:12:52:12 | ControlFlowNode for a | +| test.py:51:5:51:5 | SSA variable a | test.py:52:12:52:12 | ControlFlowNode for a | +| test.py:51:9:51:11 | ControlFlowNode for obj [Attribute foo] | test.py:51:9:51:15 | ControlFlowNode for Attribute | +| test.py:51:9:51:11 | ControlFlowNode for obj [Attribute foo] | test.py:51:9:51:15 | ControlFlowNode for Attribute | +| test.py:51:9:51:11 | ControlFlowNode for obj [Attribute foo] | test.py:51:9:51:15 | ControlFlowNode for Attribute | +| test.py:51:9:51:11 | ControlFlowNode for obj [Attribute foo] | test.py:51:9:51:15 | ControlFlowNode for Attribute | +| test.py:51:9:51:15 | ControlFlowNode for Attribute | test.py:51:5:51:5 | SSA variable a | +| test.py:51:9:51:15 | ControlFlowNode for Attribute | test.py:51:5:51:5 | SSA variable a | +| test.py:51:9:51:15 | ControlFlowNode for Attribute | test.py:51:5:51:5 | SSA variable a | +| test.py:51:9:51:15 | ControlFlowNode for Attribute | test.py:51:5:51:5 | SSA variable a | +| test.py:51:9:51:15 | ControlFlowNode for Attribute | test.py:52:12:52:12 | ControlFlowNode for a | +| test.py:51:9:51:15 | ControlFlowNode for Attribute | test.py:52:12:52:12 | ControlFlowNode for a | +| test.py:51:9:51:15 | ControlFlowNode for Attribute | test.py:52:12:52:12 | ControlFlowNode for a | +| test.py:51:9:51:15 | ControlFlowNode for Attribute | test.py:52:12:52:12 | ControlFlowNode for a | +| test.py:52:12:52:12 | ControlFlowNode for a | test.py:56:10:56:39 | ControlFlowNode for fields_with_local_flow() | +| test.py:52:12:52:12 | ControlFlowNode for a | test.py:56:10:56:39 | ControlFlowNode for fields_with_local_flow() | +| test.py:55:1:55:18 | ControlFlowNode for FunctionExpr | test.py:55:5:55:15 | GSSA Variable test_fields | +| test.py:55:1:55:18 | ControlFlowNode for FunctionExpr | test.py:55:5:55:15 | GSSA Variable test_fields | +| test.py:55:1:55:18 | GSSA Variable SINK | test.py:56:5:56:8 | ControlFlowNode for SINK | +| test.py:55:1:55:18 | GSSA Variable SINK | test.py:56:5:56:8 | ControlFlowNode for SINK | +| test.py:55:1:55:18 | GSSA Variable SOURCE | test.py:56:33:56:38 | ControlFlowNode for SOURCE | +| test.py:55:1:55:18 | GSSA Variable SOURCE | test.py:56:33:56:38 | ControlFlowNode for SOURCE | +| test.py:55:1:55:18 | GSSA Variable fields_with_local_flow | test.py:56:10:56:31 | ControlFlowNode for fields_with_local_flow | +| test.py:55:1:55:18 | GSSA Variable fields_with_local_flow | test.py:56:10:56:31 | ControlFlowNode for fields_with_local_flow | +| test.py:56:33:56:38 | ControlFlowNode for SOURCE | test.py:49:28:49:28 | SSA variable x | +| test.py:56:33:56:38 | ControlFlowNode for SOURCE | test.py:49:28:49:28 | SSA variable x | +| test.py:56:33:56:38 | ControlFlowNode for SOURCE | test.py:56:10:56:39 | ControlFlowNode for fields_with_local_flow() | +| test.py:56:33:56:38 | ControlFlowNode for SOURCE | test.py:56:10:56:39 | ControlFlowNode for fields_with_local_flow() | diff --git a/python/ql/test/experimental/dataflow/fieldflow/globalStep.ql b/python/ql/test/experimental/dataflow/fieldflow/globalStep.ql new file mode 100644 index 00000000000..64d565339b7 --- /dev/null +++ b/python/ql/test/experimental/dataflow/fieldflow/globalStep.ql @@ -0,0 +1,7 @@ +import experimental.dataflow.basic.allFlowsConfig + +from DataFlow::PathNode fromNode, DataFlow::PathNode toNode +where + toNode = fromNode.getASuccessor() and + fromNode.getNode().getLocation().getFile().getParent().getBaseName() = "fieldflow" +select fromNode, toNode diff --git a/python/ql/test/experimental/dataflow/fieldflow/localFlow.expected b/python/ql/test/experimental/dataflow/fieldflow/localFlow.expected new file mode 100644 index 00000000000..4be38a469f6 --- /dev/null +++ b/python/ql/test/experimental/dataflow/fieldflow/localFlow.expected @@ -0,0 +1,14 @@ +| examples.py:45:1:45:30 | GSSA Variable MyObj | examples.py:46:9:46:13 | ControlFlowNode for MyObj | +| examples.py:45:28:45:28 | SSA variable x | examples.py:46:9:46:16 | SSA variable x | +| examples.py:45:28:45:28 | SSA variable x | examples.py:46:15:46:15 | ControlFlowNode for x | +| examples.py:46:3:46:5 | SSA variable obj | examples.py:47:7:47:9 | ControlFlowNode for obj | +| examples.py:46:9:46:16 | ControlFlowNode for MyObj() | examples.py:46:3:46:5 | SSA variable obj | +| examples.py:47:3:47:3 | SSA variable a | examples.py:48:10:48:10 | ControlFlowNode for a | +| examples.py:47:7:47:13 | ControlFlowNode for Attribute | examples.py:47:3:47:3 | SSA variable a | +| test.py:49:1:49:30 | GSSA Variable MyObj | test.py:50:11:50:15 | ControlFlowNode for MyObj | +| test.py:49:28:49:28 | SSA variable x | test.py:50:11:50:18 | SSA variable x | +| test.py:49:28:49:28 | SSA variable x | test.py:50:17:50:17 | ControlFlowNode for x | +| test.py:50:5:50:7 | SSA variable obj | test.py:51:9:51:11 | ControlFlowNode for obj | +| test.py:50:11:50:18 | ControlFlowNode for MyObj() | test.py:50:5:50:7 | SSA variable obj | +| test.py:51:5:51:5 | SSA variable a | test.py:52:12:52:12 | ControlFlowNode for a | +| test.py:51:9:51:15 | ControlFlowNode for Attribute | test.py:51:5:51:5 | SSA variable a | diff --git a/python/ql/test/experimental/dataflow/fieldflow/localFlow.ql b/python/ql/test/experimental/dataflow/fieldflow/localFlow.ql new file mode 100644 index 00000000000..1b413bedeca --- /dev/null +++ b/python/ql/test/experimental/dataflow/fieldflow/localFlow.ql @@ -0,0 +1,8 @@ +import python +import experimental.dataflow.DataFlow + +from DataFlow::Node nodeFrom, DataFlow::Node nodeTo +where + DataFlow::localFlowStep(nodeFrom, nodeTo) and + nodeFrom.getEnclosingCallable().getName().matches("%\\_with\\_local\\_flow") +select nodeFrom, nodeTo diff --git a/python/ql/test/experimental/dataflow/fieldflow/postupdates.expected b/python/ql/test/experimental/dataflow/fieldflow/postupdates.expected new file mode 100644 index 00000000000..4dcabdd4b86 --- /dev/null +++ b/python/ql/test/experimental/dataflow/fieldflow/postupdates.expected @@ -0,0 +1,52 @@ +| examples.py:8:9:8:12 | [post store] ControlFlowNode for self | examples.py:8:9:8:12 | ControlFlowNode for self | +| examples.py:14:9:14:12 | [post store] ControlFlowNode for self | examples.py:14:9:14:12 | ControlFlowNode for self | +| examples.py:14:20:14:30 | ControlFlowNode for MyObj() | examples.py:14:20:14:30 | [pre objCreate] ControlFlowNode for MyObj() | +| examples.py:14:26:14:29 | [post arg] ControlFlowNode for Str | examples.py:14:26:14:29 | ControlFlowNode for Str | +| examples.py:17:16:17:19 | [post read] ControlFlowNode for self | examples.py:17:16:17:19 | ControlFlowNode for self | +| examples.py:22:12:22:14 | [post read] ControlFlowNode for obj | examples.py:22:12:22:14 | ControlFlowNode for obj | +| examples.py:23:5:23:7 | [post store] ControlFlowNode for obj | examples.py:23:5:23:7 | ControlFlowNode for obj | +| examples.py:25:9:25:19 | ControlFlowNode for MyObj() | examples.py:25:9:25:19 | [pre objCreate] ControlFlowNode for MyObj() | +| examples.py:25:15:25:18 | [post arg] ControlFlowNode for Str | examples.py:25:15:25:18 | ControlFlowNode for Str | +| examples.py:27:8:27:12 | [post arg] ControlFlowNode for myobj | examples.py:27:8:27:12 | ControlFlowNode for myobj | +| examples.py:27:15:27:20 | [post arg] ControlFlowNode for SOURCE | examples.py:27:15:27:20 | ControlFlowNode for SOURCE | +| examples.py:28:6:28:10 | [post read] ControlFlowNode for myobj | examples.py:28:6:28:10 | ControlFlowNode for myobj | +| examples.py:33:5:33:15 | ControlFlowNode for NestedObj() | examples.py:33:5:33:15 | [pre objCreate] ControlFlowNode for NestedObj() | +| examples.py:35:1:35:1 | [post read] ControlFlowNode for a | examples.py:35:1:35:1 | ControlFlowNode for a | +| examples.py:35:1:35:5 | [post store] ControlFlowNode for Attribute | examples.py:35:1:35:5 | ControlFlowNode for Attribute | +| examples.py:36:1:36:1 | [post read] ControlFlowNode for a | examples.py:36:1:36:1 | ControlFlowNode for a | +| examples.py:36:1:36:10 | [post store] ControlFlowNode for Attribute() | examples.py:36:1:36:10 | ControlFlowNode for Attribute() | +| examples.py:38:6:38:6 | [post read] ControlFlowNode for a | examples.py:38:6:38:6 | ControlFlowNode for a | +| examples.py:38:6:38:10 | [post read] ControlFlowNode for Attribute | examples.py:38:6:38:10 | ControlFlowNode for Attribute | +| examples.py:41:7:41:19 | ControlFlowNode for MyObj() | examples.py:41:7:41:19 | [pre objCreate] ControlFlowNode for MyObj() | +| examples.py:41:13:41:18 | [post arg] ControlFlowNode for SOURCE | examples.py:41:13:41:18 | ControlFlowNode for SOURCE | +| examples.py:42:6:42:8 | [post read] ControlFlowNode for obj | examples.py:42:6:42:8 | ControlFlowNode for obj | +| examples.py:46:9:46:16 | ControlFlowNode for MyObj() | examples.py:46:9:46:16 | [pre objCreate] ControlFlowNode for MyObj() | +| examples.py:46:15:46:15 | [post arg] ControlFlowNode for x | examples.py:46:15:46:15 | ControlFlowNode for x | +| examples.py:47:7:47:9 | [post read] ControlFlowNode for obj | examples.py:47:7:47:9 | ControlFlowNode for obj | +| examples.py:50:29:50:34 | [post arg] ControlFlowNode for SOURCE | examples.py:50:29:50:34 | ControlFlowNode for SOURCE | +| test.py:9:9:9:12 | [post store] ControlFlowNode for self | test.py:9:9:9:12 | ControlFlowNode for self | +| test.py:15:9:15:12 | [post store] ControlFlowNode for self | test.py:15:9:15:12 | ControlFlowNode for self | +| test.py:15:20:15:30 | ControlFlowNode for MyObj() | test.py:15:20:15:30 | [pre objCreate] ControlFlowNode for MyObj() | +| test.py:15:26:15:29 | [post arg] ControlFlowNode for Str | test.py:15:26:15:29 | ControlFlowNode for Str | +| test.py:18:16:18:19 | [post read] ControlFlowNode for self | test.py:18:16:18:19 | ControlFlowNode for self | +| test.py:22:12:22:14 | [post read] ControlFlowNode for obj | test.py:22:12:22:14 | ControlFlowNode for obj | +| test.py:23:5:23:7 | [post store] ControlFlowNode for obj | test.py:23:5:23:7 | ControlFlowNode for obj | +| test.py:27:13:27:23 | ControlFlowNode for MyObj() | test.py:27:13:27:23 | [pre objCreate] ControlFlowNode for MyObj() | +| test.py:27:19:27:22 | [post arg] ControlFlowNode for Str | test.py:27:19:27:22 | ControlFlowNode for Str | +| test.py:29:12:29:16 | [post arg] ControlFlowNode for myobj | test.py:29:12:29:16 | ControlFlowNode for myobj | +| test.py:29:19:29:24 | [post arg] ControlFlowNode for SOURCE | test.py:29:19:29:24 | ControlFlowNode for SOURCE | +| test.py:30:10:30:14 | [post read] ControlFlowNode for myobj | test.py:30:10:30:14 | ControlFlowNode for myobj | +| test.py:36:9:36:19 | ControlFlowNode for NestedObj() | test.py:36:9:36:19 | [pre objCreate] ControlFlowNode for NestedObj() | +| test.py:38:5:38:5 | [post read] ControlFlowNode for a | test.py:38:5:38:5 | ControlFlowNode for a | +| test.py:38:5:38:9 | [post store] ControlFlowNode for Attribute | test.py:38:5:38:9 | ControlFlowNode for Attribute | +| test.py:39:5:39:5 | [post read] ControlFlowNode for a | test.py:39:5:39:5 | ControlFlowNode for a | +| test.py:39:5:39:14 | [post store] ControlFlowNode for Attribute() | test.py:39:5:39:14 | ControlFlowNode for Attribute() | +| test.py:41:10:41:10 | [post read] ControlFlowNode for a | test.py:41:10:41:10 | ControlFlowNode for a | +| test.py:41:10:41:14 | [post read] ControlFlowNode for Attribute | test.py:41:10:41:14 | ControlFlowNode for Attribute | +| test.py:45:11:45:23 | ControlFlowNode for MyObj() | test.py:45:11:45:23 | [pre objCreate] ControlFlowNode for MyObj() | +| test.py:45:17:45:22 | [post arg] ControlFlowNode for SOURCE | test.py:45:17:45:22 | ControlFlowNode for SOURCE | +| test.py:46:10:46:12 | [post read] ControlFlowNode for obj | test.py:46:10:46:12 | ControlFlowNode for obj | +| test.py:50:11:50:18 | ControlFlowNode for MyObj() | test.py:50:11:50:18 | [pre objCreate] ControlFlowNode for MyObj() | +| test.py:50:17:50:17 | [post arg] ControlFlowNode for x | test.py:50:17:50:17 | ControlFlowNode for x | +| test.py:51:9:51:11 | [post read] ControlFlowNode for obj | test.py:51:9:51:11 | ControlFlowNode for obj | +| test.py:56:33:56:38 | [post arg] ControlFlowNode for SOURCE | test.py:56:33:56:38 | ControlFlowNode for SOURCE | diff --git a/python/ql/test/experimental/dataflow/fieldflow/postupdates.ql b/python/ql/test/experimental/dataflow/fieldflow/postupdates.ql new file mode 100644 index 00000000000..b034ad9c9be --- /dev/null +++ b/python/ql/test/experimental/dataflow/fieldflow/postupdates.ql @@ -0,0 +1,4 @@ +import experimental.dataflow.testConfig + +from DataFlow::PostUpdateNode pun +select pun, pun.getPreUpdateNode() diff --git a/python/ql/test/experimental/dataflow/fieldflow/test.py b/python/ql/test/experimental/dataflow/fieldflow/test.py new file mode 100644 index 00000000000..a747db14e24 --- /dev/null +++ b/python/ql/test/experimental/dataflow/fieldflow/test.py @@ -0,0 +1,56 @@ +from python.ql.test.experimental.dataflow.testDefinitions import * + +# Preamble + + +class MyObj(object): + + def __init__(self, foo): + self.foo = foo + + +class NestedObj(object): + + def __init__(self): + self.obj = MyObj("OK") + + def getObj(self): + return self.obj + + +def setFoo(obj, x): + SINK_F(obj.foo) + obj.foo = x + + +def test_example1(): + myobj = MyObj("OK") + + setFoo(myobj, SOURCE) + SINK(myobj.foo) + + +def test_example2(): + x = SOURCE + + a = NestedObj() + + a.obj.foo = x + a.getObj().foo = x + + SINK(a.obj.foo) + + +def test_example3(): + obj = MyObj(SOURCE) + SINK(obj.foo) + + +def fields_with_local_flow(x): + obj = MyObj(x) + a = obj.foo + return a + + +def test_fields(): + SINK(fields_with_local_flow(SOURCE)) diff --git a/python/ql/test/experimental/dataflow/import-helper/ImportHelper.expected b/python/ql/test/experimental/dataflow/import-helper/ImportHelper.expected new file mode 100644 index 00000000000..521bb780a56 --- /dev/null +++ b/python/ql/test/experimental/dataflow/import-helper/ImportHelper.expected @@ -0,0 +1,20 @@ +importModule +| test1.py:1:8:1:12 | GSSA Variable mypkg | mypkg | +| test2.py:1:19:1:21 | GSSA Variable foo | mypkg.foo | +| test2.py:1:24:1:26 | GSSA Variable bar | mypkg.bar | +| test3.py:2:8:2:16 | GSSA Variable mypkg | mypkg | +| test4.py:1:21:1:24 | GSSA Variable _foo | mypkg.foo | +| test4.py:2:21:2:24 | GSSA Variable _bar | mypkg.bar | +| test5.py:1:8:1:12 | GSSA Variable mypkg | mypkg | +| test5.py:9:26:9:29 | GSSA Variable _bar | mypkg.bar | +| test6.py:1:8:1:12 | GSSA Variable mypkg | mypkg | +| test6.py:5:8:5:16 | GSSA Variable mypkg | mypkg | +| test7.py:1:19:1:21 | GSSA Variable foo | mypkg.foo | +| test7.py:5:8:5:16 | GSSA Variable mypkg | mypkg | +| test7.py:9:19:9:21 | GSSA Variable foo | mypkg.foo | +importMember +| test2.py:1:19:1:21 | GSSA Variable foo | mypkg | foo | +| test2.py:1:24:1:26 | GSSA Variable bar | mypkg | bar | +| test5.py:9:26:9:29 | GSSA Variable _bar | mypkg | bar | +| test7.py:1:19:1:21 | GSSA Variable foo | mypkg | foo | +| test7.py:9:19:9:21 | GSSA Variable foo | mypkg | foo | diff --git a/python/ql/test/experimental/dataflow/import-helper/ImportHelper.ql b/python/ql/test/experimental/dataflow/import-helper/ImportHelper.ql new file mode 100644 index 00000000000..68d794f1636 --- /dev/null +++ b/python/ql/test/experimental/dataflow/import-helper/ImportHelper.ql @@ -0,0 +1,8 @@ +import python +import experimental.dataflow.DataFlow + +query predicate importModule(DataFlow::Node res, string name) { res = DataFlow::importModule(name) } + +query predicate importMember(DataFlow::Node res, string moduleName, string memberName) { + res = DataFlow::importMember(moduleName, memberName) +} diff --git a/python/ql/test/experimental/dataflow/import-helper/README.md b/python/ql/test/experimental/dataflow/import-helper/README.md new file mode 100644 index 00000000000..bcee9b620db --- /dev/null +++ b/python/ql/test/experimental/dataflow/import-helper/README.md @@ -0,0 +1 @@ +Small tests that explore difference between `import mypkg.foo` and `from mypkg import foo`. diff --git a/python/ql/test/experimental/dataflow/import-helper/mypkg/__init__.py b/python/ql/test/experimental/dataflow/import-helper/mypkg/__init__.py new file mode 100644 index 00000000000..c84a9b135a3 --- /dev/null +++ b/python/ql/test/experimental/dataflow/import-helper/mypkg/__init__.py @@ -0,0 +1 @@ +foo = 42 diff --git a/python/ql/test/experimental/dataflow/import-helper/mypkg/bar.py b/python/ql/test/experimental/dataflow/import-helper/mypkg/bar.py new file mode 100644 index 00000000000..2ae28399f5f --- /dev/null +++ b/python/ql/test/experimental/dataflow/import-helper/mypkg/bar.py @@ -0,0 +1 @@ +pass diff --git a/python/ql/test/experimental/dataflow/import-helper/mypkg/foo.py b/python/ql/test/experimental/dataflow/import-helper/mypkg/foo.py new file mode 100644 index 00000000000..2ae28399f5f --- /dev/null +++ b/python/ql/test/experimental/dataflow/import-helper/mypkg/foo.py @@ -0,0 +1 @@ +pass diff --git a/python/ql/test/experimental/dataflow/import-helper/test1.py b/python/ql/test/experimental/dataflow/import-helper/test1.py new file mode 100644 index 00000000000..fa4d1464942 --- /dev/null +++ b/python/ql/test/experimental/dataflow/import-helper/test1.py @@ -0,0 +1,6 @@ +import mypkg +print(mypkg.foo) # 42 +try: + print(mypkg.bar) +except AttributeError as e: + print(e) # module 'mypkg' has no attribute 'bar' diff --git a/python/ql/test/experimental/dataflow/import-helper/test2.py b/python/ql/test/experimental/dataflow/import-helper/test2.py new file mode 100644 index 00000000000..a706d61fd0c --- /dev/null +++ b/python/ql/test/experimental/dataflow/import-helper/test2.py @@ -0,0 +1,3 @@ +from mypkg import foo, bar +print(foo) +print(bar) diff --git a/python/ql/test/experimental/dataflow/import-helper/test3.py b/python/ql/test/experimental/dataflow/import-helper/test3.py new file mode 100644 index 00000000000..4fc602432c2 --- /dev/null +++ b/python/ql/test/experimental/dataflow/import-helper/test3.py @@ -0,0 +1,4 @@ +import mypkg.foo +import mypkg.bar +print(mypkg.foo) # /') +def test_taint(name = "World!", number="0", foo="foo"): + ensure_tainted(name, number) + ensure_not_tainted(foo) + + # Manually inspected all fields of the Request object + # https://flask.palletsprojects.com/en/1.1.x/api/#flask.Request + + ensure_tainted( + + request.environ, + request.environ.get('HTTP_AUTHORIZATION'), + + request.path, + request.full_path, + request.base_url, + request.url, + + # These request.accept_* properties are instances of subclasses of werkzeug.datastructures.Accept + request.accept_charsets.best, + request.accept_charsets.best_match(["utf-8", "utf-16"]), + request.accept_charsets[0], + request.accept_encodings, + request.accept_languages, + request.accept_mimetypes, + + # werkzeug.datastructures.HeaderSet (subclass of collections_abc.MutableSet) + request.access_control_request_headers, + + request.access_control_request_method, + + request.access_route, + request.access_route[0], + + # By default werkzeug.datastructures.ImmutableMultiDict -- although can be changed :\ + request.args, + request.args['key'], + request.args.getlist('key'), + + # werkzeug.datastructures.Authorization (a dict, with some properties) + request.authorization, + request.authorization['username'], + request.authorization.username, + + # werkzeug.datastructures.RequestCacheControl + request.cache_control, + # These should be `int`s, but can be strings... see debug method below + request.cache_control.max_age, + request.cache_control.max_stale, + request.cache_control.min_fresh, + + request.content_encoding, + + request.content_md5, + + request.content_type, + + # werkzeug.datastructures.ImmutableTypeConversionDict (which is basically just a dict) + request.cookies, + request.cookies['key'], + + request.data, + + # a werkzeug.datastructures.MultiDict, mapping [str, werkzeug.datastructures.FileStorage] + request.files, + request.files['key'], + request.files['key'].filename, + request.files['key'].stream, + request.files.getlist('key'), + request.files.getlist('key')[0].filename, + request.files.getlist('key')[0].stream, + + # By default werkzeug.datastructures.ImmutableMultiDict -- although can be changed :\ + request.form, + request.form['key'], + request.form.getlist('key'), + + request.get_data(), + + request.get_json(), + request.get_json()['foo'], + request.get_json()['foo']['bar'], + + # werkzeug.datastructures.EnvironHeaders, + # which has same interface as werkzeug.datastructures.Headers + request.headers, + request.headers['key'], + request.headers.get_all('key'), + request.headers.getlist('key'), + list(request.headers), # (k, v) list + request.headers.to_wsgi_list(), # (k, v) list + + request.json, + request.json['foo'], + request.json['foo']['bar'], + + request.method, + + request.mimetype, + + request.mimetype_params, + + request.origin, + + # werkzeug.datastructures.HeaderSet (subclass of collections_abc.MutableSet) + request.pragma, + + request.query_string, + + request.referrer, + + request.remote_addr, + + request.remote_user, + + # file-like object + request.stream, + request.input_stream, + + request.url, + + request.user_agent, + + # werkzeug.datastructures.CombinedMultiDict, which is basically just a werkzeug.datastructures.MultiDict + request.values, + request.values['key'], + request.values.getlist('key'), + + # dict + request.view_args, + request.view_args['key'], + ) + + ensure_not_tainted( + request.script_root, + request.url_root, + + # The expected charset for parsing request data / urls. Can not be changed by client. + # https://github.com/pallets/werkzeug/blob/4dc8d6ab840d4b78cbd5789cef91b01e3bde01d5/src/werkzeug/wrappers/base_request.py#L71-L72 + request.charset, + request.url_charset, + + # request.date is a parsed `datetime` + # https://github.com/pallets/werkzeug/blob/4dc8d6ab840d4b78cbd5789cef91b01e3bde01d5/src/werkzeug/wrappers/common_descriptors.py#L76-L83 + request.date, + + # Assuming that endpoints are not created by user-input seems fair + request.endpoint, + + # In some rare circumstances a client could spoof the host, but by default they + # should not be able to. See + # https://werkzeug.palletsprojects.com/en/1.0.x/wrappers/#werkzeug.wrappers.BaseRequest.trusted_hosts + request.host, + request.host_url, + + request.scheme, + + request.script_root, + ) + + # Testing some more tricky data-flow still works + a = request.args + b = a + gl = b.getlist + ensure_tainted( + request.args, + a, + b, + + request.args['key'], + a['key'], + b['key'], + + request.args.getlist('key'), + a.getlist('key'), + b.getlist('key'), + gl('key'), + ) + + # aliasing tests + req = request + gd = request.get_data + ensure_tainted( + req.path, + gd(), + ) + + + + +@app.route('/debug//', methods=['GET']) +def debug(foo, bar): + print("request.view_args", request.view_args) + + print("request.headers {!r}".format(request.headers)) + print("request.headers['accept'] {!r}".format(request.headers['accept'])) + + print("request.pragma {!r}".format(request.pragma)) + + return 'ok' + +@app.route('/stream', methods=['POST']) +def stream(): + print(request.path) + s = request.stream + print(s) + # just works :) + print(s.read()) + + return 'ok' + +@app.route('/input_stream', methods=['POST']) +def input_stream(): + print(request.path) + s = request.input_stream + print(s) + # hangs until client stops connection, since max number of bytes to read must + # be handled manually + print(s.read()) + + return 'ok' + +@app.route('/form', methods=['POST']) +def form(): + print(request.path) + print("request.form", request.form) + + return 'ok' + +@app.route('/cache_control', methods=['POST']) +def cache_control(): + print(request.path) + print("request.cache_control.max_age", request.cache_control.max_age, type(request.cache_control.max_age)) + print("request.cache_control.max_stale", request.cache_control.max_stale, type(request.cache_control.max_stale)) + print("request.cache_control.min_fresh", request.cache_control.min_fresh, type(request.cache_control.min_fresh)) + + return 'ok' + +@app.route('/file_upload', methods=['POST']) +def file_upload(): + print(request.path) + for k,v in request.files.items(): + print(k, v, v.name, v.filename, v.stream) + + return 'ok' + +@app.route('/args', methods=['GET']) +def args(): + print(request.path) + print("request.args", request.args) + + return 'ok' + +# curl --header "My-Header: some-value" http://localhost:5000/debug/fooval/barval +# curl --header "Pragma: foo, bar" --header "Pragma: stuff, foo" http://localhost:5000/debug/fooval/barval + +# curl -X POST --data 'wat' http://localhost:5000/stream +# curl -X POST --data 'wat' http://localhost:5000/input_stream + +# curl --form foo=foo --form foo=123 http://localhost:5000/form + +# curl --header "Cache-Control: max-age=foo, max-stale=bar, min-fresh=baz" http://localhost:5000/cache_control +# curl --header "Cache-Control: max-age=1, max-stale=2, min-fresh=3" http://localhost:5000/cache_control + +# curl -F myfile=@ localhost:5000/file_upload + +# curl http://localhost:5000/args?foo=42&bar=bar + +if __name__ == "__main__": + app.run(debug=True) diff --git a/python/ql/test/experimental/library-tests/frameworks/modeling-example/NaiveModel.expected b/python/ql/test/experimental/library-tests/frameworks/modeling-example/NaiveModel.expected new file mode 100644 index 00000000000..159c2f3f256 --- /dev/null +++ b/python/ql/test/experimental/library-tests/frameworks/modeling-example/NaiveModel.expected @@ -0,0 +1,35 @@ +edges +| test.py:21:11:21:18 | ControlFlowNode for source() | test.py:22:10:22:24 | ControlFlowNode for Attribute() | +| test.py:29:11:29:18 | ControlFlowNode for source() | test.py:33:10:33:12 | ControlFlowNode for val | +| test.py:40:11:40:25 | ControlFlowNode for Attribute() | test.py:41:10:41:12 | ControlFlowNode for val | +| test.py:45:11:45:18 | ControlFlowNode for source() | test.py:40:11:40:25 | ControlFlowNode for Attribute() | +| test.py:53:11:53:25 | ControlFlowNode for Attribute() | test.py:54:10:54:12 | ControlFlowNode for val | +| test.py:70:11:70:18 | ControlFlowNode for source() | test.py:53:11:53:25 | ControlFlowNode for Attribute() | +| test.py:78:11:78:14 | ControlFlowNode for bm() | test.py:79:10:79:12 | ControlFlowNode for val | +| test.py:83:11:83:18 | ControlFlowNode for source() | test.py:78:11:78:14 | ControlFlowNode for bm() | +| test.py:90:11:90:14 | ControlFlowNode for bm() | test.py:91:10:91:12 | ControlFlowNode for val | +| test.py:107:11:107:18 | ControlFlowNode for source() | test.py:90:11:90:14 | ControlFlowNode for bm() | +nodes +| test.py:21:11:21:18 | ControlFlowNode for source() | semmle.label | ControlFlowNode for source() | +| test.py:22:10:22:24 | ControlFlowNode for Attribute() | semmle.label | ControlFlowNode for Attribute() | +| test.py:29:11:29:18 | ControlFlowNode for source() | semmle.label | ControlFlowNode for source() | +| test.py:33:10:33:12 | ControlFlowNode for val | semmle.label | ControlFlowNode for val | +| test.py:40:11:40:25 | ControlFlowNode for Attribute() | semmle.label | ControlFlowNode for Attribute() | +| test.py:41:10:41:12 | ControlFlowNode for val | semmle.label | ControlFlowNode for val | +| test.py:45:11:45:18 | ControlFlowNode for source() | semmle.label | ControlFlowNode for source() | +| test.py:53:11:53:25 | ControlFlowNode for Attribute() | semmle.label | ControlFlowNode for Attribute() | +| test.py:54:10:54:12 | ControlFlowNode for val | semmle.label | ControlFlowNode for val | +| test.py:70:11:70:18 | ControlFlowNode for source() | semmle.label | ControlFlowNode for source() | +| test.py:78:11:78:14 | ControlFlowNode for bm() | semmle.label | ControlFlowNode for bm() | +| test.py:79:10:79:12 | ControlFlowNode for val | semmle.label | ControlFlowNode for val | +| test.py:83:11:83:18 | ControlFlowNode for source() | semmle.label | ControlFlowNode for source() | +| test.py:90:11:90:14 | ControlFlowNode for bm() | semmle.label | ControlFlowNode for bm() | +| test.py:91:10:91:12 | ControlFlowNode for val | semmle.label | ControlFlowNode for val | +| test.py:107:11:107:18 | ControlFlowNode for source() | semmle.label | ControlFlowNode for source() | +#select +| test.py:22:10:22:24 | ControlFlowNode for Attribute() | test.py:21:11:21:18 | ControlFlowNode for source() | test.py:22:10:22:24 | ControlFlowNode for Attribute() | test flow (naive): test_simple | +| test.py:33:10:33:12 | ControlFlowNode for val | test.py:29:11:29:18 | ControlFlowNode for source() | test.py:33:10:33:12 | ControlFlowNode for val | test flow (naive): test_alias | +| test.py:41:10:41:12 | ControlFlowNode for val | test.py:45:11:45:18 | ControlFlowNode for source() | test.py:41:10:41:12 | ControlFlowNode for val | test flow (naive): test_accross_functions | +| test.py:54:10:54:12 | ControlFlowNode for val | test.py:70:11:70:18 | ControlFlowNode for source() | test.py:54:10:54:12 | ControlFlowNode for val | test flow (naive): test_deeply_nested | +| test.py:79:10:79:12 | ControlFlowNode for val | test.py:83:11:83:18 | ControlFlowNode for source() | test.py:79:10:79:12 | ControlFlowNode for val | test flow (naive): test_pass_bound_method | +| test.py:91:10:91:12 | ControlFlowNode for val | test.py:107:11:107:18 | ControlFlowNode for source() | test.py:91:10:91:12 | ControlFlowNode for val | test flow (naive): test_deeply_nested_bound_method | diff --git a/python/ql/test/experimental/library-tests/frameworks/modeling-example/NaiveModel.ql b/python/ql/test/experimental/library-tests/frameworks/modeling-example/NaiveModel.ql new file mode 100644 index 00000000000..6203923378a --- /dev/null +++ b/python/ql/test/experimental/library-tests/frameworks/modeling-example/NaiveModel.ql @@ -0,0 +1,24 @@ +/** + * @kind path-problem + */ + +private import python +private import experimental.dataflow.DataFlow +private import experimental.dataflow.TaintTracking +import DataFlow::PathGraph +import SharedCode + +class MyClassGetValueAdditionalTaintStep extends TaintTracking::AdditionalTaintStep { + override predicate step(DataFlow::Node nodeFrom, DataFlow::Node nodeTo) { + // obj -> obj.get_value() + exists(DataFlow::Node bound_method | + bound_method = myClassGetValue(nodeFrom) and + nodeTo.asCfgNode().(CallNode).getFunction() = bound_method.asCfgNode() + ) + } +} + +from SharedConfig config, DataFlow::PathNode source, DataFlow::PathNode sink +where config.hasFlowPath(source, sink) +select sink.getNode(), source, sink, + "test flow (naive): " + source.getNode().asCfgNode().getScope().getName() diff --git a/python/ql/test/experimental/library-tests/frameworks/modeling-example/ProperModel.expected b/python/ql/test/experimental/library-tests/frameworks/modeling-example/ProperModel.expected new file mode 100644 index 00000000000..9c6f6ad30c1 --- /dev/null +++ b/python/ql/test/experimental/library-tests/frameworks/modeling-example/ProperModel.expected @@ -0,0 +1,67 @@ +edges +| test.py:21:11:21:18 | ControlFlowNode for source() | test.py:22:10:22:24 | ControlFlowNode for Attribute() | +| test.py:29:11:29:18 | ControlFlowNode for source() | test.py:33:10:33:12 | ControlFlowNode for val | +| test.py:39:15:39:17 | SSA variable arg | test.py:41:10:41:12 | ControlFlowNode for val | +| test.py:45:11:45:18 | ControlFlowNode for source() | test.py:46:15:46:17 | ControlFlowNode for src | +| test.py:46:15:46:17 | ControlFlowNode for src | test.py:39:15:39:17 | SSA variable arg | +| test.py:52:24:52:26 | SSA variable arg | test.py:54:10:54:12 | ControlFlowNode for val | +| test.py:57:33:57:35 | SSA variable arg | test.py:58:24:58:26 | ControlFlowNode for arg | +| test.py:58:24:58:26 | ControlFlowNode for arg | test.py:52:24:52:26 | SSA variable arg | +| test.py:61:33:61:35 | SSA variable arg | test.py:62:33:62:35 | ControlFlowNode for arg | +| test.py:62:33:62:35 | ControlFlowNode for arg | test.py:57:33:57:35 | SSA variable arg | +| test.py:65:33:65:35 | SSA variable arg | test.py:66:33:66:35 | ControlFlowNode for arg | +| test.py:66:33:66:35 | ControlFlowNode for arg | test.py:61:33:61:35 | SSA variable arg | +| test.py:70:11:70:18 | ControlFlowNode for source() | test.py:71:33:71:35 | ControlFlowNode for src | +| test.py:71:33:71:35 | ControlFlowNode for src | test.py:65:33:65:35 | SSA variable arg | +| test.py:77:23:77:24 | SSA variable bm | test.py:79:10:79:12 | ControlFlowNode for val | +| test.py:83:11:83:18 | ControlFlowNode for source() | test.py:84:23:84:35 | ControlFlowNode for Attribute | +| test.py:84:23:84:35 | ControlFlowNode for Attribute | test.py:77:23:77:24 | SSA variable bm | +| test.py:89:37:89:38 | SSA variable bm | test.py:91:10:91:12 | ControlFlowNode for val | +| test.py:94:46:94:47 | SSA variable bm | test.py:95:37:95:38 | ControlFlowNode for bm | +| test.py:95:37:95:38 | ControlFlowNode for bm | test.py:89:37:89:38 | SSA variable bm | +| test.py:98:46:98:47 | SSA variable bm | test.py:99:46:99:47 | ControlFlowNode for bm | +| test.py:99:46:99:47 | ControlFlowNode for bm | test.py:94:46:94:47 | SSA variable bm | +| test.py:102:46:102:47 | SSA variable bm | test.py:103:46:103:47 | ControlFlowNode for bm | +| test.py:103:46:103:47 | ControlFlowNode for bm | test.py:98:46:98:47 | SSA variable bm | +| test.py:107:11:107:18 | ControlFlowNode for source() | test.py:108:46:108:58 | ControlFlowNode for Attribute | +| test.py:108:46:108:58 | ControlFlowNode for Attribute | test.py:102:46:102:47 | SSA variable bm | +nodes +| test.py:21:11:21:18 | ControlFlowNode for source() | semmle.label | ControlFlowNode for source() | +| test.py:22:10:22:24 | ControlFlowNode for Attribute() | semmle.label | ControlFlowNode for Attribute() | +| test.py:29:11:29:18 | ControlFlowNode for source() | semmle.label | ControlFlowNode for source() | +| test.py:33:10:33:12 | ControlFlowNode for val | semmle.label | ControlFlowNode for val | +| test.py:39:15:39:17 | SSA variable arg | semmle.label | SSA variable arg | +| test.py:41:10:41:12 | ControlFlowNode for val | semmle.label | ControlFlowNode for val | +| test.py:45:11:45:18 | ControlFlowNode for source() | semmle.label | ControlFlowNode for source() | +| test.py:46:15:46:17 | ControlFlowNode for src | semmle.label | ControlFlowNode for src | +| test.py:52:24:52:26 | SSA variable arg | semmle.label | SSA variable arg | +| test.py:54:10:54:12 | ControlFlowNode for val | semmle.label | ControlFlowNode for val | +| test.py:57:33:57:35 | SSA variable arg | semmle.label | SSA variable arg | +| test.py:58:24:58:26 | ControlFlowNode for arg | semmle.label | ControlFlowNode for arg | +| test.py:61:33:61:35 | SSA variable arg | semmle.label | SSA variable arg | +| test.py:62:33:62:35 | ControlFlowNode for arg | semmle.label | ControlFlowNode for arg | +| test.py:65:33:65:35 | SSA variable arg | semmle.label | SSA variable arg | +| test.py:66:33:66:35 | ControlFlowNode for arg | semmle.label | ControlFlowNode for arg | +| test.py:70:11:70:18 | ControlFlowNode for source() | semmle.label | ControlFlowNode for source() | +| test.py:71:33:71:35 | ControlFlowNode for src | semmle.label | ControlFlowNode for src | +| test.py:77:23:77:24 | SSA variable bm | semmle.label | SSA variable bm | +| test.py:79:10:79:12 | ControlFlowNode for val | semmle.label | ControlFlowNode for val | +| test.py:83:11:83:18 | ControlFlowNode for source() | semmle.label | ControlFlowNode for source() | +| test.py:84:23:84:35 | ControlFlowNode for Attribute | semmle.label | ControlFlowNode for Attribute | +| test.py:89:37:89:38 | SSA variable bm | semmle.label | SSA variable bm | +| test.py:91:10:91:12 | ControlFlowNode for val | semmle.label | ControlFlowNode for val | +| test.py:94:46:94:47 | SSA variable bm | semmle.label | SSA variable bm | +| test.py:95:37:95:38 | ControlFlowNode for bm | semmle.label | ControlFlowNode for bm | +| test.py:98:46:98:47 | SSA variable bm | semmle.label | SSA variable bm | +| test.py:99:46:99:47 | ControlFlowNode for bm | semmle.label | ControlFlowNode for bm | +| test.py:102:46:102:47 | SSA variable bm | semmle.label | SSA variable bm | +| test.py:103:46:103:47 | ControlFlowNode for bm | semmle.label | ControlFlowNode for bm | +| test.py:107:11:107:18 | ControlFlowNode for source() | semmle.label | ControlFlowNode for source() | +| test.py:108:46:108:58 | ControlFlowNode for Attribute | semmle.label | ControlFlowNode for Attribute | +#select +| test.py:22:10:22:24 | ControlFlowNode for Attribute() | test.py:21:11:21:18 | ControlFlowNode for source() | test.py:22:10:22:24 | ControlFlowNode for Attribute() | test flow (proper): test_simple | +| test.py:33:10:33:12 | ControlFlowNode for val | test.py:29:11:29:18 | ControlFlowNode for source() | test.py:33:10:33:12 | ControlFlowNode for val | test flow (proper): test_alias | +| test.py:41:10:41:12 | ControlFlowNode for val | test.py:45:11:45:18 | ControlFlowNode for source() | test.py:41:10:41:12 | ControlFlowNode for val | test flow (proper): test_accross_functions | +| test.py:54:10:54:12 | ControlFlowNode for val | test.py:70:11:70:18 | ControlFlowNode for source() | test.py:54:10:54:12 | ControlFlowNode for val | test flow (proper): test_deeply_nested | +| test.py:79:10:79:12 | ControlFlowNode for val | test.py:83:11:83:18 | ControlFlowNode for source() | test.py:79:10:79:12 | ControlFlowNode for val | test flow (proper): test_pass_bound_method | +| test.py:91:10:91:12 | ControlFlowNode for val | test.py:107:11:107:18 | ControlFlowNode for source() | test.py:91:10:91:12 | ControlFlowNode for val | test flow (proper): test_deeply_nested_bound_method | diff --git a/python/ql/test/experimental/library-tests/frameworks/modeling-example/ProperModel.ql b/python/ql/test/experimental/library-tests/frameworks/modeling-example/ProperModel.ql new file mode 100644 index 00000000000..235276e630f --- /dev/null +++ b/python/ql/test/experimental/library-tests/frameworks/modeling-example/ProperModel.ql @@ -0,0 +1,26 @@ +/** + * @kind path-problem + */ + +private import python +private import experimental.dataflow.DataFlow +private import experimental.dataflow.TaintTracking +import DataFlow::PathGraph +import SharedCode + +class MyClassGetValueAdditionalTaintStep extends TaintTracking::AdditionalTaintStep { + override predicate step(DataFlow::Node nodeFrom, DataFlow::Node nodeTo) { + // obj -> obj.get_value + nodeTo.asCfgNode().(AttrNode).getObject("get_value") = nodeFrom.asCfgNode() and + nodeTo = myClassGetValue(_) + or + // get_value -> get_value() + nodeFrom = myClassGetValue(_) and + nodeTo.asCfgNode().(CallNode).getFunction() = nodeFrom.asCfgNode() + } +} + +from SharedConfig config, DataFlow::PathNode source, DataFlow::PathNode sink +where config.hasFlowPath(source, sink) +select sink.getNode(), source, sink, + "test flow (proper): " + source.getNode().asCfgNode().getScope().getName() diff --git a/python/ql/test/experimental/library-tests/frameworks/modeling-example/README.md b/python/ql/test/experimental/library-tests/frameworks/modeling-example/README.md new file mode 100644 index 00000000000..09a01d043f4 --- /dev/null +++ b/python/ql/test/experimental/library-tests/frameworks/modeling-example/README.md @@ -0,0 +1,29 @@ +This test illustrates that you need to be very careful when adding additional taint-steps or dataflow steps using `TypeTracker`. + +The basic setup is that we're modeling the behavior of a (fictitious) external library class `MyClass`, and (fictitious) source of such an instance (the `source` function). + +```py3 +class MyClass: + def __init__(self, value): + self.value = value + + def get_value(self): + return self.value +``` + +We want to extend our analysis to `obj.get_value()` is also tainted if `obj` is a tainted instance of `MyClass`. + +The actual type-tracking is done in `SharedCode.qll`, but it's the _way_ we use it that matters. + +In `NaiveModel.ql` we add an additional taint step from an instance of `MyClass` to calls of the bound method `get_value` (that we have tracked). It provides us with the correct results, but the path explanations are not very useful, since we are now able to cross functions in _one step_. + +In `ProperModel.ql` we split the additional taint step in two: + +1. from tracked `obj` that is instance of `MyClass`, to `obj.get_value` **but only** exactly where the attribute is accessed (by an `AttrNode`). This is important, since if we allowed `.get_value` we would again be able to cross functions in one step. +2. from tracked `get_value` bound method to calls of it, **but only** exactly where the call is (by a `CallNode`). for same reason as above. + +**Try running the queries in VS Code to see the difference** + +### Possible improvements + +Using `AttrNode` directly in the code here means there is no easy way to add `getattr` support too all such predicates. Not really sure how to handle this in a generalized way though :| diff --git a/python/ql/test/experimental/library-tests/frameworks/modeling-example/SharedCode.qll b/python/ql/test/experimental/library-tests/frameworks/modeling-example/SharedCode.qll new file mode 100644 index 00000000000..1bf6e3930ab --- /dev/null +++ b/python/ql/test/experimental/library-tests/frameworks/modeling-example/SharedCode.qll @@ -0,0 +1,36 @@ +private import python +private import experimental.dataflow.DataFlow +private import experimental.dataflow.TaintTracking + +// Helpers modeling MyClass +/** A data-flow Node representing an instance of MyClass. */ +abstract class MyClass extends DataFlow::Node { } + +private DataFlow::Node myClassGetValue(MyClass qualifier, DataFlow::TypeTracker t) { + t.startInAttr("get_value") and + result = qualifier + or + exists(DataFlow::TypeTracker t2 | result = myClassGetValue(qualifier, t2).track(t2, t)) +} + +DataFlow::Node myClassGetValue(MyClass qualifier) { + result = myClassGetValue(qualifier, DataFlow::TypeTracker::end()) +} + +// Config +class SourceCall extends DataFlow::Node, MyClass { + SourceCall() { this.asCfgNode().(CallNode).getFunction().(NameNode).getId() = "source" } +} + +class SharedConfig extends TaintTracking::Configuration { + SharedConfig() { this = "SharedConfig" } + + override predicate isSource(DataFlow::Node source) { source instanceof SourceCall } + + override predicate isSink(DataFlow::Node sink) { + exists(CallNode call | + call.getFunction().(NameNode).getId() = "sink" and + call.getArg(0) = sink.asCfgNode() + ) + } +} diff --git a/python/ql/test/experimental/library-tests/frameworks/modeling-example/test.py b/python/ql/test/experimental/library-tests/frameworks/modeling-example/test.py new file mode 100644 index 00000000000..6fe7bb3fcad --- /dev/null +++ b/python/ql/test/experimental/library-tests/frameworks/modeling-example/test.py @@ -0,0 +1,108 @@ +class MyClass: + def __init__(self, value): + self.value = value + + def get_value(self): + return self.value + + +def source(): + return MyClass("tainted") + + +def sink(obj): + print("sink", obj) + + +################################################################################ + + +def test_simple(): + src = source() + sink(src.get_value()) + + +################################################################################ + + +def test_alias(): + src = source() + foo = src + bound_method = foo.get_value + val = bound_method() + sink(val) + + +################################################################################ + + +def sink_func(arg): + val = arg.get_value() + sink(val) + + +def test_accross_functions(): + src = source() + sink_func(src) + + +################################################################################ + + +def deeply_nested_sink(arg): + val = arg.get_value() + sink(val) + + +def deeply_nested_passthrough_1(arg): + deeply_nested_sink(arg) + + +def deeply_nested_passthrough_2(arg): + deeply_nested_passthrough_1(arg) + + +def deeply_nested_passthrough_3(arg): + deeply_nested_passthrough_2(arg) + + +def test_deeply_nested(): + src = source() + deeply_nested_passthrough_3(src) + + +################################################################################ + + +def recv_bound_method(bm): + val = bm() + sink(val) + + +def test_pass_bound_method(): + src = source() + recv_bound_method(src.get_value) + + +################################################################################ + +def deeply_nested_bound_method_sink(bm): + val = bm() + sink(val) + + +def deeply_nested_bound_method_passthrough_1(bm): + deeply_nested_bound_method_sink(bm) + + +def deeply_nested_bound_method_passthrough_2(bm): + deeply_nested_bound_method_passthrough_1(bm) + + +def deeply_nested_bound_method_passthrough_3(bm): + deeply_nested_bound_method_passthrough_2(bm) + + +def test_deeply_nested_bound_method(): + src = source() + deeply_nested_bound_method_passthrough_3(src.get_value) diff --git a/python/ql/test/experimental/library-tests/frameworks/stdlib/ConceptsTest.expected b/python/ql/test/experimental/library-tests/frameworks/stdlib/ConceptsTest.expected new file mode 100644 index 00000000000..e69de29bb2d diff --git a/python/ql/test/experimental/library-tests/frameworks/stdlib/ConceptsTest.ql b/python/ql/test/experimental/library-tests/frameworks/stdlib/ConceptsTest.ql new file mode 100644 index 00000000000..b557a0bccb6 --- /dev/null +++ b/python/ql/test/experimental/library-tests/frameworks/stdlib/ConceptsTest.ql @@ -0,0 +1,2 @@ +import python +import experimental.meta.ConceptsTest diff --git a/python/ql/test/experimental/library-tests/frameworks/stdlib/SystemCommandExecution.py b/python/ql/test/experimental/library-tests/frameworks/stdlib/SystemCommandExecution.py new file mode 100644 index 00000000000..ad4223a8c4b --- /dev/null +++ b/python/ql/test/experimental/library-tests/frameworks/stdlib/SystemCommandExecution.py @@ -0,0 +1,141 @@ +# Note: Some of these commands will technically not allow an attacker to execute +# arbitrary system commands, but only specify the program to be executed. The general +# consensus was that even this is still a high security risk, so we also treat them as +# system command executions. +# +# As an example, executing `subprocess.Popen(["rm -rf /"])` will result in +# `FileNotFoundError: [Errno 2] No such file or directory: 'rm -rf /'` + +######################################## + + +import os + +# can't use a string literal with spaces in the tags of an InlineExpectationsTest, so using variables :| + +os.popen("cmd1; cmd2") # $getCommand="cmd1; cmd2" +os.system("cmd1; cmd2") # $getCommand="cmd1; cmd2" + + +def os_members(): + # hmm, it's kinda annoying to check that we handle this import correctly for + # everything. It's quite useful since I messed it up initially and didn't have a + # test for it, but in the long run it's just cumbersome to duplicate all the tests + # :| + from os import popen, system + + popen("cmd1; cmd2") # $getCommand="cmd1; cmd2" + system("cmd1; cmd2") # $getCommand="cmd1; cmd2" + + +######################################## +# https://docs.python.org/3.8/library/os.html#os.execl +# +# VS Code extension will ignore rest of program if encountering one of these, which we +# don't want. We could use `if False`, but just to be 100% sure we don't do anything too +# clever in our analysis that discards that code, I used `if UNKNOWN` instead +if UNKNOWN: + env = {"FOO": "foo"} + os.execl("executable", "", "arg0") # $getCommand="executable" + os.execle("executable", "", "arg0", env) # $getCommand="executable" + os.execlp("executable", "", "arg0") # $getCommand="executable" + os.execlpe("executable", "", "arg0", env) # $getCommand="executable" + os.execv("executable", ["", "arg0"]) # $getCommand="executable" + os.execve("executable", ["", "arg0"], env) # $getCommand="executable" + os.execvp("executable", ["", "arg0"]) # $getCommand="executable" + os.execvpe("executable", ["", "arg0"], env) # $getCommand="executable" + + +######################################## +# https://docs.python.org/3.8/library/os.html#os.spawnl +env = {"FOO": "foo"} +os.spawnl(os.P_WAIT, "executable", "", "arg0") # $getCommand="executable" +os.spawnle(os.P_WAIT, "executable", "", "arg0", env) # $getCommand="executable" +os.spawnlp(os.P_WAIT, "executable", "", "arg0") # $getCommand="executable" +os.spawnlpe(os.P_WAIT, "executable", "", "arg0", env) # $getCommand="executable" +os.spawnv(os.P_WAIT, "executable", ["", "arg0"]) # $getCommand="executable" +os.spawnve(os.P_WAIT, "executable", ["", "arg0"], env) # $getCommand="executable" +os.spawnvp(os.P_WAIT, "executable", ["", "arg0"]) # $getCommand="executable" +os.spawnvpe(os.P_WAIT, "executable", ["", "arg0"], env) # $getCommand="executable" + +# Added in Python 3.8 +os.posix_spawn("executable", ["", "arg0"], env) # $getCommand="executable" +os.posix_spawnp("executable", ["", "arg0"], env) # $getCommand="executable" + +######################################## + +import subprocess + +subprocess.Popen("cmd1; cmd2", shell=True) # $getCommand="cmd1; cmd2" +subprocess.Popen("cmd1; cmd2", shell="truthy string") # $getCommand="cmd1; cmd2" +subprocess.Popen(["cmd1; cmd2", "shell-arg"], shell=True) # $getCommand="cmd1; cmd2" +subprocess.Popen("cmd1; cmd2", shell=True, executable="/bin/bash") # $getCommand="cmd1; cmd2" $getCommand="/bin/bash" + +subprocess.Popen("executable") # $getCommand="executable" +subprocess.Popen(["executable", "arg0"]) # $getCommand="executable" +subprocess.Popen("", executable="executable") # $getCommand="executable" +subprocess.Popen(["", "arg0"], executable="executable") # $getCommand="executable" + +# call/check_call/check_output/run all work like Popen from a command execution point of view +subprocess.call(["executable", "arg0"]) # $getCommand="executable" +subprocess.check_call(["executable", "arg0"]) # $getCommand="executable" +subprocess.check_output(["executable", "arg0"]) # $getCommand="executable" +subprocess.run(["executable", "arg0"]) # $getCommand="executable" + + +######################################## +# actively using known shell as the executable + +subprocess.Popen(["/bin/sh", "-c", "vuln"]) # $getCommand="/bin/sh" $f-:getCommand="vuln" +subprocess.Popen(["/bin/bash", "-c", "vuln"]) # $getCommand="/bin/bash" $f-:getCommand="vuln" +subprocess.Popen(["/bin/dash", "-c", "vuln"]) # $getCommand="/bin/dash" $f-:getCommand="vuln" +subprocess.Popen(["/bin/zsh", "-c", "vuln"]) # $getCommand="/bin/zsh" $f-:getCommand="vuln" + +subprocess.Popen(["sh", "-c", "vuln"]) # $getCommand="sh" $f-:getCommand="vuln" +subprocess.Popen(["bash", "-c", "vuln"]) # $getCommand="bash" $f-:getCommand="vuln" +subprocess.Popen(["dash", "-c", "vuln"]) # $getCommand="dash" $f-:getCommand="vuln" +subprocess.Popen(["zsh", "-c", "vuln"]) # $getCommand="zsh" $f-:getCommand="vuln" + +# Check that we don't consider ANY argument a command injection sink +subprocess.Popen(["sh", "/bin/python"]) # $getCommand="sh" + +subprocess.Popen(["cmd.exe", "/c", "vuln"]) # $getCommand="cmd.exe" $f-:getCommand="vuln" +subprocess.Popen(["cmd.exe", "/C", "vuln"]) # $getCommand="cmd.exe" $f-:getCommand="vuln" +subprocess.Popen(["cmd", "/c", "vuln"]) # $getCommand="cmd" $f-:getCommand="vuln" +subprocess.Popen(["cmd", "/C", "vuln"]) # $getCommand="cmd" $f-:getCommand="vuln" + +subprocess.Popen(["", "-c", "vuln"], executable="/bin/bash") # $getCommand="/bin/bash" $f-:getCommand="vuln" + +if UNKNOWN: + os.execl("/bin/sh", "", "-c", "vuln") # $getCommand="/bin/sh" $f-:getCommand="vuln" + +os.spawnl(os.P_WAIT, "/bin/sh", "", "-c", "vuln") # $getCommand="/bin/sh" $f-:getCommand="vuln" + + +######################################## +# Passing arguments by reference + +args = ["/bin/sh", "-c", "vuln"] +subprocess.Popen(args) # $getCommand=args + +args = "" +use_shell = False +exe = "executable" +subprocess.Popen(args, shell=use_shell, executable=exe) # $f+:getCommand=args $getCommand=exe + + +################################################################################ +# Taint related + +import shlex + +cmd = shlex.join(["echo", tainted]) +args = shlex.split(tainted) + +# will handle tainted = 'foo; rm -rf /' +safe_cmd = "ls {}".format(shlex.quote(tainted)) + +# not how you are supposed to use shlex.quote +wrong_use = shlex.quote("ls {}".format(tainted)) +# still dangerous, for example +cmd = "sh -c " + wrong_use diff --git a/python/ql/test/experimental/library-tests/options b/python/ql/test/experimental/library-tests/options new file mode 100644 index 00000000000..eb214fc2931 --- /dev/null +++ b/python/ql/test/experimental/library-tests/options @@ -0,0 +1 @@ +semmle-extractor-options: --max-import-depth=1 diff --git a/python/ql/test/experimental/meta/ConceptsTest.qll b/python/ql/test/experimental/meta/ConceptsTest.qll new file mode 100644 index 00000000000..eb798823d50 --- /dev/null +++ b/python/ql/test/experimental/meta/ConceptsTest.qll @@ -0,0 +1,34 @@ +import python +import experimental.dataflow.DataFlow +import experimental.semmle.python.Concepts +import TestUtilities.InlineExpectationsTest + +string value_from_expr(Expr e) { + // TODO: This one is starting to look like `repr` predicate from TestTaintLib + result = + e.(StrConst).getPrefix() + e.(StrConst).getText() + + e.(StrConst).getPrefix().regexpReplaceAll("[a-zA-Z]+", "") + or + result = e.(Name).getId() + or + not e instanceof StrConst and + not e instanceof Name and + result = e.toString() +} + +class SystemCommandExecutionTest extends InlineExpectationsTest { + SystemCommandExecutionTest() { this = "SystemCommandExecutionTest" } + + override string getARelevantTag() { result = "getCommand" } + + override predicate hasActualResult(Location location, string element, string tag, string value) { + exists(SystemCommandExecution sce, DataFlow::Node command | + exists(location.getFile().getRelativePath()) and + command = sce.getCommand() and + location = command.getLocation() and + element = command.toString() and + value = value_from_expr(command.asExpr()) and + tag = "getCommand" + ) + } +} diff --git a/python/ql/test/experimental/query-tests/Security-new-dataflow/CWE-078/CommandInjection.expected b/python/ql/test/experimental/query-tests/Security-new-dataflow/CWE-078/CommandInjection.expected new file mode 100644 index 00000000000..03748127520 --- /dev/null +++ b/python/ql/test/experimental/query-tests/Security-new-dataflow/CWE-078/CommandInjection.expected @@ -0,0 +1,42 @@ +edges +| command_injection.py:10:13:10:24 | ControlFlowNode for Attribute | command_injection.py:12:15:12:27 | ControlFlowNode for BinaryExpr | +| command_injection.py:17:13:17:24 | ControlFlowNode for Attribute | command_injection.py:19:22:19:34 | ControlFlowNode for BinaryExpr | +| command_injection.py:24:11:24:22 | ControlFlowNode for Attribute | command_injection.py:25:23:25:25 | ControlFlowNode for cmd | +| command_injection.py:30:13:30:24 | ControlFlowNode for Attribute | command_injection.py:32:14:32:26 | ControlFlowNode for BinaryExpr | +| command_injection.py:36:15:36:26 | ControlFlowNode for Attribute | command_injection.py:39:15:39:21 | ControlFlowNode for command | +| command_injection.py:36:15:36:26 | ControlFlowNode for Attribute | command_injection.py:40:15:40:21 | ControlFlowNode for command | +| command_injection.py:52:15:52:26 | ControlFlowNode for Attribute | command_injection.py:53:15:53:21 | ControlFlowNode for command | +| command_injection.py:52:15:52:26 | ControlFlowNode for Attribute | command_injection.py:54:14:54:20 | ControlFlowNode for command | +| command_injection.py:52:15:52:26 | ControlFlowNode for Attribute | command_injection.py:55:21:55:27 | ControlFlowNode for command | +| command_injection.py:52:15:52:26 | ControlFlowNode for Attribute | command_injection.py:56:27:56:33 | ControlFlowNode for command | +| command_injection.py:52:15:52:26 | ControlFlowNode for Attribute | command_injection.py:57:20:57:26 | ControlFlowNode for command | +nodes +| command_injection.py:10:13:10:24 | ControlFlowNode for Attribute | semmle.label | ControlFlowNode for Attribute | +| command_injection.py:12:15:12:27 | ControlFlowNode for BinaryExpr | semmle.label | ControlFlowNode for BinaryExpr | +| command_injection.py:17:13:17:24 | ControlFlowNode for Attribute | semmle.label | ControlFlowNode for Attribute | +| command_injection.py:19:22:19:34 | ControlFlowNode for BinaryExpr | semmle.label | ControlFlowNode for BinaryExpr | +| command_injection.py:24:11:24:22 | ControlFlowNode for Attribute | semmle.label | ControlFlowNode for Attribute | +| command_injection.py:25:23:25:25 | ControlFlowNode for cmd | semmle.label | ControlFlowNode for cmd | +| command_injection.py:30:13:30:24 | ControlFlowNode for Attribute | semmle.label | ControlFlowNode for Attribute | +| command_injection.py:32:14:32:26 | ControlFlowNode for BinaryExpr | semmle.label | ControlFlowNode for BinaryExpr | +| command_injection.py:36:15:36:26 | ControlFlowNode for Attribute | semmle.label | ControlFlowNode for Attribute | +| command_injection.py:39:15:39:21 | ControlFlowNode for command | semmle.label | ControlFlowNode for command | +| command_injection.py:40:15:40:21 | ControlFlowNode for command | semmle.label | ControlFlowNode for command | +| command_injection.py:52:15:52:26 | ControlFlowNode for Attribute | semmle.label | ControlFlowNode for Attribute | +| command_injection.py:53:15:53:21 | ControlFlowNode for command | semmle.label | ControlFlowNode for command | +| command_injection.py:54:14:54:20 | ControlFlowNode for command | semmle.label | ControlFlowNode for command | +| command_injection.py:55:21:55:27 | ControlFlowNode for command | semmle.label | ControlFlowNode for command | +| command_injection.py:56:27:56:33 | ControlFlowNode for command | semmle.label | ControlFlowNode for command | +| command_injection.py:57:20:57:26 | ControlFlowNode for command | semmle.label | ControlFlowNode for command | +#select +| command_injection.py:12:15:12:27 | ControlFlowNode for BinaryExpr | command_injection.py:10:13:10:24 | ControlFlowNode for Attribute | command_injection.py:12:15:12:27 | ControlFlowNode for BinaryExpr | This command depends on $@. | command_injection.py:10:13:10:24 | ControlFlowNode for Attribute | a user-provided value | +| command_injection.py:19:22:19:34 | ControlFlowNode for BinaryExpr | command_injection.py:17:13:17:24 | ControlFlowNode for Attribute | command_injection.py:19:22:19:34 | ControlFlowNode for BinaryExpr | This command depends on $@. | command_injection.py:17:13:17:24 | ControlFlowNode for Attribute | a user-provided value | +| command_injection.py:25:23:25:25 | ControlFlowNode for cmd | command_injection.py:24:11:24:22 | ControlFlowNode for Attribute | command_injection.py:25:23:25:25 | ControlFlowNode for cmd | This command depends on $@. | command_injection.py:24:11:24:22 | ControlFlowNode for Attribute | a user-provided value | +| command_injection.py:32:14:32:26 | ControlFlowNode for BinaryExpr | command_injection.py:30:13:30:24 | ControlFlowNode for Attribute | command_injection.py:32:14:32:26 | ControlFlowNode for BinaryExpr | This command depends on $@. | command_injection.py:30:13:30:24 | ControlFlowNode for Attribute | a user-provided value | +| command_injection.py:39:15:39:21 | ControlFlowNode for command | command_injection.py:36:15:36:26 | ControlFlowNode for Attribute | command_injection.py:39:15:39:21 | ControlFlowNode for command | This command depends on $@. | command_injection.py:36:15:36:26 | ControlFlowNode for Attribute | a user-provided value | +| command_injection.py:40:15:40:21 | ControlFlowNode for command | command_injection.py:36:15:36:26 | ControlFlowNode for Attribute | command_injection.py:40:15:40:21 | ControlFlowNode for command | This command depends on $@. | command_injection.py:36:15:36:26 | ControlFlowNode for Attribute | a user-provided value | +| command_injection.py:53:15:53:21 | ControlFlowNode for command | command_injection.py:52:15:52:26 | ControlFlowNode for Attribute | command_injection.py:53:15:53:21 | ControlFlowNode for command | This command depends on $@. | command_injection.py:52:15:52:26 | ControlFlowNode for Attribute | a user-provided value | +| command_injection.py:54:14:54:20 | ControlFlowNode for command | command_injection.py:52:15:52:26 | ControlFlowNode for Attribute | command_injection.py:54:14:54:20 | ControlFlowNode for command | This command depends on $@. | command_injection.py:52:15:52:26 | ControlFlowNode for Attribute | a user-provided value | +| command_injection.py:55:21:55:27 | ControlFlowNode for command | command_injection.py:52:15:52:26 | ControlFlowNode for Attribute | command_injection.py:55:21:55:27 | ControlFlowNode for command | This command depends on $@. | command_injection.py:52:15:52:26 | ControlFlowNode for Attribute | a user-provided value | +| command_injection.py:56:27:56:33 | ControlFlowNode for command | command_injection.py:52:15:52:26 | ControlFlowNode for Attribute | command_injection.py:56:27:56:33 | ControlFlowNode for command | This command depends on $@. | command_injection.py:52:15:52:26 | ControlFlowNode for Attribute | a user-provided value | +| command_injection.py:57:20:57:26 | ControlFlowNode for command | command_injection.py:52:15:52:26 | ControlFlowNode for Attribute | command_injection.py:57:20:57:26 | ControlFlowNode for command | This command depends on $@. | command_injection.py:52:15:52:26 | ControlFlowNode for Attribute | a user-provided value | diff --git a/python/ql/test/experimental/query-tests/Security-new-dataflow/CWE-078/CommandInjection.qlref b/python/ql/test/experimental/query-tests/Security-new-dataflow/CWE-078/CommandInjection.qlref new file mode 100644 index 00000000000..2d70a75c372 --- /dev/null +++ b/python/ql/test/experimental/query-tests/Security-new-dataflow/CWE-078/CommandInjection.qlref @@ -0,0 +1 @@ +experimental/Security-new-dataflow/CWE-078/CommandInjection.ql diff --git a/python/ql/test/experimental/query-tests/Security-new-dataflow/CWE-078/command_injection.py b/python/ql/test/experimental/query-tests/Security-new-dataflow/CWE-078/command_injection.py new file mode 100644 index 00000000000..75125f59fca --- /dev/null +++ b/python/ql/test/experimental/query-tests/Security-new-dataflow/CWE-078/command_injection.py @@ -0,0 +1,61 @@ + +import os +import subprocess + +from flask import Flask, request +app = Flask(__name__) + +@app.route("/command1") +def command_injection1(): + files = request.args.get('files', '') + # Don't let files be `; rm -rf /` + os.system("ls " + files) + + +@app.route("/command2") +def command_injection2(): + files = request.args.get('files', '') + # Don't let files be `; rm -rf /` + subprocess.Popen("ls " + files, shell=True) + + +@app.route("/command3") +def first_arg_injection(): + cmd = request.args.get('cmd', '') + subprocess.Popen([cmd, "param1"]) + + +@app.route("/other_cases") +def others(): + files = request.args.get('files', '') + # Don't let files be `; rm -rf /` + os.popen("ls " + files) + +@app.route("/multiple") +def multiple(): + command = request.args.get('command', '') + # We should mark flow to both calls here, which conflicts with removing flow out of + # a sink due to use-use flow. + os.system(command) + os.system(command) + + +@app.route("/not-into-sink-impl") +def not_into_sink_impl(): + """When there is flow to a sink such as `os.popen(cmd)`, we don't want to highlight that there is also + flow through the actual `popen` function to the internal call to `subprocess.Popen` -- we would usually + see that flow since we extract the `os.py` file from the standard library. + + os.popen implementation: https://github.com/python/cpython/blob/fa7ce080175f65d678a7d5756c94f82887fc9803/Lib/os.py#L974 + subprocess.call implementation: https://github.com/python/cpython/blob/fa7ce080175f65d678a7d5756c94f82887fc9803/Lib/subprocess.py#L341 + """ + command = request.args.get('command', '') + os.system(command) + os.popen(command) + subprocess.call(command) + subprocess.check_call(command) + subprocess.run(command) + + +# TODO: popen2 module for Python 2 only https://devdocs.io/python~2.7/library/popen2 +# (deprecated since Python 2.6, but still functional in Python 2.7.17) diff --git a/python/ql/test/query-tests/Security/CWE-078/CommandInjection.expected b/python/ql/test/query-tests/Security/CWE-078/CommandInjection.expected index 3d1cf22a2d1..d417f55c68c 100644 --- a/python/ql/test/query-tests/Security/CWE-078/CommandInjection.expected +++ b/python/ql/test/query-tests/Security/CWE-078/CommandInjection.expected @@ -7,10 +7,10 @@ edges | command_injection.py:12:23:12:27 | externally controlled string | command_injection.py:12:15:12:27 | externally controlled string | | command_injection.py:17:13:17:24 | dict of externally controlled string | command_injection.py:17:13:17:41 | externally controlled string | | command_injection.py:17:13:17:24 | dict of externally controlled string | command_injection.py:17:13:17:41 | externally controlled string | -| command_injection.py:17:13:17:41 | externally controlled string | command_injection.py:19:29:19:33 | externally controlled string | -| command_injection.py:17:13:17:41 | externally controlled string | command_injection.py:19:29:19:33 | externally controlled string | -| command_injection.py:19:29:19:33 | externally controlled string | command_injection.py:19:22:19:34 | sequence of externally controlled string | -| command_injection.py:19:29:19:33 | externally controlled string | command_injection.py:19:22:19:34 | sequence of externally controlled string | +| command_injection.py:17:13:17:41 | externally controlled string | command_injection.py:19:30:19:34 | externally controlled string | +| command_injection.py:17:13:17:41 | externally controlled string | command_injection.py:19:30:19:34 | externally controlled string | +| command_injection.py:19:30:19:34 | externally controlled string | command_injection.py:19:22:19:34 | externally controlled string | +| command_injection.py:19:30:19:34 | externally controlled string | command_injection.py:19:22:19:34 | externally controlled string | | command_injection.py:24:11:24:22 | dict of externally controlled string | command_injection.py:24:11:24:37 | externally controlled string | | command_injection.py:24:11:24:22 | dict of externally controlled string | command_injection.py:24:11:24:37 | externally controlled string | | command_injection.py:24:11:24:37 | externally controlled string | command_injection.py:25:23:25:25 | externally controlled string | @@ -25,6 +25,6 @@ edges | command_injection.py:32:22:32:26 | externally controlled string | command_injection.py:32:14:32:26 | externally controlled string | #select | command_injection.py:12:15:12:27 | BinaryExpr | command_injection.py:10:13:10:24 | dict of externally controlled string | command_injection.py:12:15:12:27 | externally controlled string | This command depends on $@. | command_injection.py:10:13:10:24 | Attribute | a user-provided value | -| command_injection.py:19:22:19:34 | List | command_injection.py:17:13:17:24 | dict of externally controlled string | command_injection.py:19:22:19:34 | sequence of externally controlled string | This command depends on $@. | command_injection.py:17:13:17:24 | Attribute | a user-provided value | +| command_injection.py:19:22:19:34 | BinaryExpr | command_injection.py:17:13:17:24 | dict of externally controlled string | command_injection.py:19:22:19:34 | externally controlled string | This command depends on $@. | command_injection.py:17:13:17:24 | Attribute | a user-provided value | | command_injection.py:25:22:25:36 | List | command_injection.py:24:11:24:22 | dict of externally controlled string | command_injection.py:25:22:25:36 | first item in sequence of externally controlled string | This command depends on $@. | command_injection.py:24:11:24:22 | Attribute | a user-provided value | | command_injection.py:32:14:32:26 | BinaryExpr | command_injection.py:30:13:30:24 | dict of externally controlled string | command_injection.py:32:14:32:26 | externally controlled string | This command depends on $@. | command_injection.py:30:13:30:24 | Attribute | a user-provided value | diff --git a/python/ql/test/query-tests/Security/CWE-078/command_injection.py b/python/ql/test/query-tests/Security/CWE-078/command_injection.py index 2122a076ec9..ee5629c1b4a 100644 --- a/python/ql/test/query-tests/Security/CWE-078/command_injection.py +++ b/python/ql/test/query-tests/Security/CWE-078/command_injection.py @@ -16,7 +16,7 @@ def command_injection1(): def command_injection2(): files = request.args.get('files', '') # Don't let files be `; rm -rf /` - subprocess.Popen(["ls", files], shell = True) + subprocess.Popen("ls " + files, shell=True) @app.route("/command3")