Merge remote-tracking branch 'upstream/master' into FalsySanitizer

This commit is contained in:
Erik Krogh Kristensen
2020-02-12 10:28:48 +01:00
27 changed files with 3986 additions and 58 deletions

View File

@@ -19,6 +19,7 @@ The following changes in version 1.24 affect C/C++ analysis in all applications.
| Memory is never freed (`cpp/memory-never-freed`) | More true positive results | This query now identifies a wider variety of buffer allocations using the `semmle.code.cpp.models.interfaces.Allocation` library. |
| Memory may not be freed (`cpp/memory-may-not-be-freed`) | More true positive results | This query now identifies a wider variety of buffer allocations using the `semmle.code.cpp.models.interfaces.Allocation` library. |
| Missing return statement (`cpp/missing-return`) | Fewer false positive results | Functions containing `asm` statements are no longer highlighted by this query. |
| No space for zero terminator (`cpp/no-space-for-terminator`) | More correct results | String arguments to formatting functions are now (usually) expected to be null terminated strings. |
| Hard-coded Japanese era start date (`cpp/japanese-era/exact-era-date`) | | This query is no longer run on LGTM. |
| No space for zero terminator (`cpp/no-space-for-terminator`) | Fewer false positive results | This query has been modified to be more conservative when identifying which pointers point to null-terminated strings. This approach produces fewer, more accurate results. |
| Overloaded assignment does not return 'this' (`cpp/assignment-does-not-return-this`) | Fewer false positive results | This query no longer reports incorrect results in template classes. |

View File

@@ -27,7 +27,9 @@ The following changes in version 1.24 affect C# analysis in all applications.
## Changes to code extraction
* Tuple expressions, for example `(int,bool)` in `default((int,bool))` are now extracted correctly.
* Expression nullability flow state is extracted.
* Expression nullability flow state is extracted.
* Implicitly typed `stackalloc` expressions are now extracted correctly.
* The difference between `stackalloc` array creations and normal array creations is extracted.
## Changes to libraries
@@ -38,5 +40,6 @@ The following changes in version 1.24 affect C# analysis in all applications.
* The taint tracking library now tracks flow through (implicit or explicit) conversion operator calls.
* [Code contracts](https://docs.microsoft.com/en-us/dotnet/framework/debug-trace-profile/code-contracts) are now recognized, and are treated like any other assertion methods.
* Expression nullability flow state is given by the predicates `Expr.hasNotNullFlowState()` and `Expr.hasMaybeNullFlowState()`.
* `stackalloc` array creations are now represented by the QL class `Stackalloc`. Previously they were represented by the class `ArrayCreation`.
## Changes to autobuilder

View File

@@ -22,16 +22,25 @@ import semmle.code.cpp.models.interfaces.Allocation
predicate terminationProblem(AllocationExpr malloc, string msg) {
// malloc(strlen(...))
exists(StrlenCall strlen | DataFlow::localExprFlow(strlen, malloc.getSizeExpr())) and
// flows into a null-terminated string function
// flows to a call that implies this is a null-terminated string
exists(ArrayFunction af, FunctionCall fc, int arg |
DataFlow::localExprFlow(malloc, fc.getArgument(arg)) and
fc.getTarget() = af and
(
// null terminated string
// flows into null terminated string argument
af.hasArrayWithNullTerminator(arg)
or
// likely a null terminated string (such as `strcpy`, `strcat`)
// flows into likely null terminated string argument (such as `strcpy`, `strcat`)
af.hasArrayWithUnknownSize(arg)
or
// flows into string argument to a formatting function (such as `printf`)
exists(int n, FormatLiteral fl |
fc.getArgument(arg) = fc.(FormattingFunctionCall).getConversionArgument(n) and
fl = fc.(FormattingFunctionCall).getFormat() and
fl.getConversionType(n) instanceof PointerType and // `%s`, `%ws` etc
not fl.getConversionType(n) instanceof VoidPointerType and // exclude: `%p`
not fl.hasPrecision(n) // exclude: `%.*s`
)
)
) and
msg = "This allocation does not include space to null-terminate the string."

View File

@@ -365,10 +365,10 @@ private predicate modelFlow(Instruction iFrom, Instruction iTo) {
modelOut.isReturnValueDeref() and
iTo = call
or
exists(WriteSideEffectInstruction outNode |
modelOut.isParameterDeref(outNode.getIndex()) and
exists(int index, WriteSideEffectInstruction outNode |
modelOut.isParameterDeref(index) and
iTo = outNode and
outNode.getPrimaryInstruction() = call
outNode = getSideEffectFor(call, index)
)
// TODO: add write side effects for qualifiers
) and
@@ -380,8 +380,7 @@ private predicate modelFlow(Instruction iFrom, Instruction iTo) {
or
exists(int index, ReadSideEffectInstruction read |
modelIn.isParameterDeref(index) and
read.getIndex() = index and
read.getPrimaryInstruction() = call and
read = getSideEffectFor(call, index) and
iFrom = read.getSideEffectOperand().getAnyDef()
)
or
@@ -392,6 +391,18 @@ private predicate modelFlow(Instruction iFrom, Instruction iTo) {
)
}
/**
* Holds if the result is a side effect for instruction `call` on argument
* index `argument`. This helper predicate makes it easy to join on both of
* these columns at once, avoiding pathological join orders in case the
* argument index should get joined first.
*/
pragma[noinline]
SideEffectInstruction getSideEffectFor(CallInstruction call, int argument) {
call = result.getPrimaryInstruction() and
argument = result.(IndexedInstruction).getIndex()
}
/**
* Holds if data flows from `source` to `sink` in zero or more local
* (intra-procedural) steps.

View File

@@ -220,9 +220,12 @@ class VariableMemoryLocation extends TVariableMemoryLocation, AllocationMemoryLo
/**
* Holds if this memory location covers the entire variable.
*/
final predicate coversEntireVariable() {
startBitOffset = 0 and
endBitOffset = var.getIRType().getByteSize() * 8
final predicate coversEntireVariable() { varIRTypeHasBitRange(startBitOffset, endBitOffset) }
pragma[noinline]
private predicate varIRTypeHasBitRange(int start, int end) {
start = 0 and
end = var.getIRType().getByteSize() * 8
}
}

View File

@@ -94,6 +94,22 @@ abstract class FunctionWithWrappers extends Function {
)
}
/**
* Whether 'func' is a (possibly nested) wrapper function that feeds a parameter at the given index
* through to an interesting parameter of 'this' function.
*
* The 'cause' gives the name of 'this' interesting function and its relevant parameter
* at the end of the call chain.
*
* If there is more than one possible 'cause', a unique one is picked (by lexicographic order).
*/
pragma[nomagic]
private string wrapperFunctionAnyDepthUnique(Function func, int paramIndex) {
result =
toCause(func, paramIndex) + ", which ends up calling " +
min(string targetCause | this.wrapperFunctionAnyDepth(func, paramIndex, targetCause))
}
/**
* Whether 'func' is a (possibly nested) wrapper function that feeds a parameter at the given index
* through to an interesting parameter of 'this' function.
@@ -114,13 +130,7 @@ abstract class FunctionWithWrappers extends Function {
)
or
not this.wrapperFunctionLimitedDepth(func, paramIndex, _, _) and
cause =
min(string targetCause, string possibleCause |
this.wrapperFunctionAnyDepth(func, paramIndex, targetCause) and
possibleCause = toCause(func, paramIndex) + ", which ends up calling " + targetCause
|
possibleCause
)
cause = wrapperFunctionAnyDepthUnique(func, paramIndex)
}
/**

View File

@@ -5,6 +5,7 @@
| test.c:49:20:49:25 | call to malloc | This allocation does not include space to null-terminate the string. |
| test.cpp:24:35:24:40 | call to malloc | This allocation does not include space to null-terminate the string. |
| test.cpp:45:28:45:33 | call to malloc | This allocation does not include space to null-terminate the string. |
| test.cpp:55:28:55:33 | call to malloc | This allocation does not include space to null-terminate the string. |
| test.cpp:63:28:63:33 | call to malloc | This allocation does not include space to null-terminate the string. |
| test.cpp:71:28:71:33 | call to malloc | This allocation does not include space to null-terminate the string. |
| test.cpp:79:28:79:33 | call to malloc | This allocation does not include space to null-terminate the string. |

View File

@@ -51,7 +51,7 @@ void decode(char *dest, char *src);
void wdecode(wchar_t *dest, wchar_t *src);
void bad4(char *str) {
// BAD -- zero-termination proved by wprintf (as parameter) [NOT DETECTED]
// BAD -- zero-termination proved by wprintf (as parameter)
char *buffer = (char *)malloc(strlen(str));
decode(buffer, str);
wprintf(L"%s", buffer);
@@ -107,3 +107,19 @@ void bad9(wchar_t *wstr) {
wcscpy(wbuffer, wstr);
delete wbuffer;
}
void good3(char *str) {
// GOOD -- zero-termination not required for this printf
char *buffer = (char *)malloc(strlen(str));
decode(buffer, str);
wprintf(L"%p", buffer);
free(buffer);
}
void good4(char *str) {
// GOOD -- zero-termination not required for this printf
char *buffer = (char *)malloc(strlen(str));
decode(buffer, str);
wprintf(L"%.*s", strlen(str), buffer);
free(buffer);
}

View File

@@ -90,9 +90,29 @@ namespace Semmle.Extraction.CSharp.Entities.Expressions
public override InitializerExpressionSyntax Initializer => Syntax.Initializer;
protected override void PopulateExpression(TextWriter trapFile)
{
base.PopulateExpression(trapFile);
trapFile.stackalloc_array_creation(this);
}
public static Expression Create(ExpressionNodeInfo info) => new StackAllocArrayCreation(info).TryPopulate();
}
class ImplicitStackAllocArrayCreation : ArrayCreation<ImplicitStackAllocArrayCreationExpressionSyntax>
{
ImplicitStackAllocArrayCreation(ExpressionNodeInfo info) : base(info.SetKind(ExprKind.ARRAY_CREATION)) { }
public static Expression Create(ExpressionNodeInfo info) => new ImplicitStackAllocArrayCreation(info).TryPopulate();
protected override void PopulateExpression(TextWriter trapFile)
{
ArrayInitializer.Create(new ExpressionNodeInfo(cx, Syntax.Initializer, this, -1));
trapFile.implicitly_typed_array_creation(this);
trapFile.stackalloc_array_creation(this);
}
}
class ImplicitArrayCreation : ArrayCreation<ImplicitArrayCreationExpressionSyntax>
{
ImplicitArrayCreation(ExpressionNodeInfo info) : base(info.SetKind(ExprKind.ARRAY_CREATION)) { }

View File

@@ -207,6 +207,9 @@ namespace Semmle.Extraction.CSharp.Entities.Expressions
case SyntaxKind.StackAllocArrayCreationExpression:
return StackAllocArrayCreation.Create(info);
case SyntaxKind.ImplicitStackAllocArrayCreationExpression:
return ImplicitStackAllocArrayCreation.Create(info);
case SyntaxKind.ArgListExpression:
return ArgList.Create(info);

View File

@@ -466,6 +466,11 @@ namespace Semmle.Extraction.CSharp
trapFile.WriteTuple("specific_type_parameter_nullability", constraints, baseType, nullability);
}
internal static void stackalloc_array_creation(this TextWriter trapFile, Expression array)
{
trapFile.WriteTuple("stackalloc_array_creation", array);
}
internal static void stmt_location(this TextWriter trapFile, Statement stmt, Location location)
{
trapFile.WriteTuple("stmt_location", stmt, location);

View File

@@ -11,6 +11,7 @@
import semmle.code.csharp.serialization.Serialization
import semmle.code.csharp.controlflow.Guards
import semmle.code.csharp.dataflow.DataFlow
/**
* The result is a write to the field `f`, assigning it the value
@@ -29,7 +30,11 @@ GuardedExpr checkedWrite(Field f, Variable v, IfStmt check) {
Expr uncheckedWrite(Callable callable, Field f) {
result = f.getAnAssignedValue() and
result.getEnclosingCallable() = callable and
not callable.calls*(checkedWrite(f, _, _).getEnclosingCallable())
not callable.calls*(checkedWrite(f, _, _).getEnclosingCallable()) and
// Exclude object creations because they were not deserialized
not exists(Expr src | DataFlow::localExprFlow(src, result) |
src instanceof ObjectCreation or src.hasValue()
)
}
from BinarySerializableType t, Field f, IfStmt check, Expr write, Expr unsafeWrite

View File

@@ -372,6 +372,13 @@ class ArrayCreation extends Expr, @array_creation_expr {
override string toString() { result = "array creation of type " + this.getType().getName() }
}
/**
* A `stackalloc` array creation, for example `stackalloc char[] { 'x', 'y' }`.
*/
class Stackalloc extends ArrayCreation {
Stackalloc() { stackalloc_array_creation(this) }
}
/**
* An anonymous function. Either a lambda expression (`LambdaExpr`) or an
* anonymous method expression (`AnonymousMethodExpr`).

View File

@@ -1084,6 +1084,9 @@ implicitly_typed_array_creation(
explicitly_sized_array_creation(
unique int id: @array_creation_expr ref);
stackalloc_array_creation(
unique int id: @array_creation_expr ref);
mutator_invocation_mode(
unique int id: @operator_invocation_expr ref,
int mode: int ref /* prefix = 1, postfix = 2*/);

View File

@@ -28440,6 +28440,17 @@
<dependencies/>
</relation>
<relation>
<name>stackalloc_array_creation</name>
<cardinality>50</cardinality>
<columnsizes>
<e>
<k>id</k>
<v>50</v>
</e>
</columnsizes>
<dependencies/>
</relation>
<relation>
<name>mutator_invocation_mode</name>
<cardinality>0</cardinality>
<columnsizes>

View File

@@ -1,6 +1,20 @@
arrayCreation
| csharp73.cs:9:20:9:49 | array creation of type Char* | 0 | csharp73.cs:9:20:9:49 | 2 |
| csharp73.cs:10:20:10:45 | array creation of type Char* | 0 | csharp73.cs:10:36:10:36 | 1 |
| csharp73.cs:11:20:11:37 | array creation of type Char[] | 0 | csharp73.cs:11:20:11:37 | 1 |
| csharp73.cs:12:20:12:38 | array creation of type Char* | 0 | csharp73.cs:12:36:12:37 | 10 |
| csharp73.cs:13:20:13:31 | array creation of type Char[] | 0 | csharp73.cs:13:29:13:30 | 10 |
| csharp73.cs:21:23:21:33 | array creation of type Int32[] | 0 | csharp73.cs:21:31:21:32 | 10 |
| csharp73.cs:22:23:22:33 | array creation of type Int32[] | 0 | csharp73.cs:22:31:22:32 | 10 |
arrayElement
| csharp73.cs:9:20:9:49 | array creation of type Char* | 0 | csharp73.cs:9:40:9:42 | x |
| csharp73.cs:9:20:9:49 | array creation of type Char* | 1 | csharp73.cs:9:45:9:47 | y |
| csharp73.cs:10:20:10:45 | array creation of type Char* | 0 | csharp73.cs:10:41:10:43 | x |
| csharp73.cs:11:20:11:37 | array creation of type Char[] | 0 | csharp73.cs:11:33:11:35 | x |
| csharp73.cs:14:20:14:43 | array creation of type Int32* | 0 | csharp73.cs:14:35:14:35 | 1 |
| csharp73.cs:14:20:14:43 | array creation of type Int32* | 1 | csharp73.cs:14:38:14:38 | 2 |
| csharp73.cs:14:20:14:43 | array creation of type Int32* | 2 | csharp73.cs:14:41:14:41 | 3 |
stackalloc
| csharp73.cs:9:20:9:49 | array creation of type Char* |
| csharp73.cs:10:20:10:45 | array creation of type Char* |
| csharp73.cs:12:20:12:38 | array creation of type Char* |
| csharp73.cs:14:20:14:43 | array creation of type Int32* |

View File

@@ -1,4 +1,11 @@
import csharp
from ArrayCreation creation, int i
select creation, i, creation.getLengthArgument(i)
query predicate arrayCreation(ArrayCreation creation, int i, Expr length) {
length = creation.getLengthArgument(i)
}
query predicate arrayElement(ArrayCreation array, int i, Expr element) {
element = array.getInitializer().getElement(i)
}
query predicate stackalloc(Stackalloc a) { any() }

View File

@@ -1,4 +0,0 @@
| csharp73.cs:9:20:9:49 | array creation of type Char* | 0 | csharp73.cs:9:40:9:42 | x |
| csharp73.cs:9:20:9:49 | array creation of type Char* | 1 | csharp73.cs:9:45:9:47 | y |
| csharp73.cs:10:20:10:45 | array creation of type Char* | 0 | csharp73.cs:10:41:10:43 | x |
| csharp73.cs:11:20:11:37 | array creation of type Char[] | 0 | csharp73.cs:11:33:11:35 | x |

View File

@@ -1,4 +0,0 @@
import csharp
from ArrayCreation array, int i
select array, i, array.getInitializer().getElement(i)

View File

@@ -11,6 +11,7 @@ class StackAllocs
var arr3 = new char[] { 'x' };
var arr4 = stackalloc char[10];
var arr5 = new char[10];
var arr6 = stackalloc[] { 1, 2, 3 };
}
}

View File

@@ -10,14 +10,14 @@ public class Test1
{
if (v == "valid")
{
f = v /* safe write */;
f = v; // GOOD
}
}
[OnDeserializing]
public void Deserialize()
{
f = "invalid" /* unsafe write */;
f = $"invalid"; // BAD
}
}
@@ -30,19 +30,19 @@ public class Test2
{
if (v == "valid")
{
f = v /* safe write */;
f = v; // GOOD
}
}
[OnDeserializing]
public void Deserialize()
{
var v = "invalid";
f = v /* unsafe write -- false negative */;
var v = $"invalid";
f = v; // BAD: False negative
if (v == "valid")
{
f = v; /* safe write */
f = v; // GOOD
}
}
}
@@ -56,25 +56,25 @@ public class Test3
{
if (v == "valid")
{
f = v /* safe write */;
f = v; // GOOD
}
}
[OnDeserializing]
public void Deserialize()
{
var v = "invalid";
f = v /* unsafe write -- false negative */;
var v = $"invalid";
f = v; // GOOD: False negative
Assign(v);
}
private void Assign(string v)
{
f = v /* unsafe write -- false negative */;
f = v; // GOOD: False negative
if (v == "valid")
{
f = v /* safe write */;
f = v; // GOOD
}
}
}
@@ -88,21 +88,21 @@ public class Test4
{
if (v == "valid")
{
f = v /* safe write */;
f = v; // GOOD
}
}
[OnDeserializing]
public void Deserialize()
{
var v = "invalid";
var v = $"invalid";
if (v == "valid")
Assign(v);
}
private void Assign(string v)
{
f = v /* safe write */;
f = v; // GOOD
}
}
@@ -115,13 +115,13 @@ public class Test5 : ISerializable
{
if (age < 0)
throw new ArgumentException(nameof(age));
Age = age /* safe write */;
Age = age; // GOOD
}
[OnDeserializing]
void ISerializable.GetObjectData(SerializationInfo info, StreamingContext context)
{
Age = info.GetInt32("age"); /* unsafe write */;
Age = info.GetInt32("age"); // BAD
}
}
@@ -134,7 +134,7 @@ public class Test6 : ISerializable
{
if (age < 0)
throw new ArgumentException(nameof(age));
Age = age /* safe write */;
Age = age; // GOOD
}
[OnDeserializing]
@@ -143,7 +143,7 @@ public class Test6 : ISerializable
int age = info.GetInt32("age");
if (age < 0)
throw new SerializationException("age");
Age = age; /* safe write */;
Age = age; // GOOD
}
}
@@ -156,7 +156,7 @@ public class Test7 : ISerializable
{
if (age < 0)
throw new ArgumentException(nameof(age));
Age = age /* safe write */;
Age = age; // GOOD
}
[OnDeserializing]
@@ -165,6 +165,27 @@ public class Test7 : ISerializable
int age = info.GetInt32("age");
if (false)
throw new SerializationException("age");
Age = age; /* unsafe write */;
Age = age; // BAD
}
}
[Serializable]
public class Test8 : ISerializable
{
string Options;
public int Age;
public Test8(string options)
{
if (options == null)
throw new ArgumentNullException(nameof(options));
Options = options; // GOOD
}
[OnDeserializing]
void ISerializable.GetObjectData(SerializationInfo info, StreamingContext context)
{
Options = new string(""); // GOOD: A created object
}
}

View File

@@ -1,4 +1,4 @@
| RuntimeChecksBypass.cs:20:13:20:21 | "invalid" | This write to $@ may be circumventing a $@. | RuntimeChecksBypass.cs:7:19:7:19 | f | f | RuntimeChecksBypass.cs:11:9:14:9 | if (...) ... | check |
| RuntimeChecksBypass.cs:20:13:20:22 | $"..." | This write to $@ may be circumventing a $@. | RuntimeChecksBypass.cs:7:19:7:19 | f | f | RuntimeChecksBypass.cs:11:9:14:9 | if (...) ... | check |
| RuntimeChecksBypass.cs:124:15:124:34 | call to method GetInt32 | This write to $@ may be circumventing a $@. | RuntimeChecksBypass.cs:112:16:112:18 | Age | Age | RuntimeChecksBypass.cs:116:9:117:53 | if (...) ... | check |
| RuntimeChecksBypass.cs:168:15:168:17 | access to local variable age | This write to $@ may be circumventing a $@. | RuntimeChecksBypass.cs:153:16:153:18 | Age | Age | RuntimeChecksBypass.cs:157:9:158:53 | if (...) ... | check |
| RuntimeChecksBypassBad.cs:19:15:19:34 | call to method GetInt32 | This write to $@ may be circumventing a $@. | RuntimeChecksBypassBad.cs:7:16:7:18 | Age | Age | RuntimeChecksBypassBad.cs:11:9:12:53 | if (...) ... | check |

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,2 @@
description: Adds information about `stackalloc` array creations
compatibility: backwards

View File

@@ -113,7 +113,7 @@ Then we can make the source more specific, for example an access to a public par
where
fileReader.getDeclaringType().hasQualifiedName("java.io", "FileReader") and
call.getCallee() = fileReader and
DataFlow::localFlow(DataFlow::parameterNode(p), DataFlow::exprNode(fc.getArgument(0)))
DataFlow::localFlow(DataFlow::parameterNode(p), DataFlow::exprNode(call.getArgument(0)))
select p
The following example finds calls to formatting functions where the format string is not hard-coded.

View File

@@ -502,7 +502,7 @@ Identifiers are used in following syntactic constructs:
simpleId ::= lowerId | upperId
modulename ::= simpleId
classname ::= upperId
dbasetype ::= atlowerId
dbasetype ::= atLowerId
predicateRef ::= (moduleId "::")? literalId
predicateName ::= lowerId
varname ::= simpleId
@@ -1804,7 +1804,7 @@ The complete grammar for QL is as follows:
::
ql ::= moduleBody ;
ql ::= moduleBody
module ::= annotation* "module" modulename "{" moduleBody "}"
@@ -1976,11 +1976,11 @@ The complete grammar for QL is as follows:
simpleId ::= lowerId | upperId
modulename :: = simpleId
modulename ::= simpleId
classname ::= upperId
dbasetype ::= atlowerId
dbasetype ::= atLowerId
predicateRef ::= (moduleId "::")? literalId