Commit Graph

17948 Commits

Author SHA1 Message Date
Rasmus Lerchedahl Petersen
9c8e968cba Python: Fix bad merge 2020-10-15 11:47:34 +02:00
Erik Krogh Kristensen
ab7542c0d2 improve printing of JSON values 2020-10-15 11:05:22 +02:00
Taus
c8b93148a2 Merge pull request #4424 from RasmusWL/python-model-python2-specific-command-execution
Python: model Python 2 specific command execution
2020-10-15 10:52:43 +02:00
Erik Krogh Kristensen
1ebd49b0eb remove location from "mapping i" print node 2020-10-15 10:51:34 +02:00
Erik Krogh Kristensen
3e2d266343 improve YAMLMapping printing 2020-10-15 10:49:37 +02:00
Anders Schack-Mulligen
60ce02ac18 Merge pull request #4469 from JLLeitschuh/additional-file-taint
Java: Track taint through java.io.File::toPath & java.nio.file.Path::toFile
2020-10-15 10:46:35 +02:00
Erik Krogh Kristensen
1b908ce030 improve printing of DeclStmt, and remove escaped whitespace chars from printed output 2020-10-15 10:43:32 +02:00
Tom Hvitved
8728017328 C#: Increase fieldFlowBranchLimit in test
68014fd3bf means that more accessors are properly
extracted, and consequently the calls to `get_Item` in the test have more dispatch
targets. Increasing `fieldFlowBranchLimit` makes the test pass again.
2020-10-15 10:40:19 +02:00
Rasmus Wriedt Larsen
c5810d623b Merge pull request #4474 from tausbn/python-fix-tostring-divergence
Python: Fix divergence in tuple/subscripted type `toString`
2020-10-15 10:29:33 +02:00
Erik Krogh Kristensen
c033ae9b7f add one more case to getAPrimaryQlClass 2020-10-15 10:05:07 +02:00
Arthur Baars
a10c0138e9 Merge commit '78c58c24158e3ee4fd78318194d56591af90da69' into lgtm.com 2020-10-15 10:00:43 +02:00
Rasmus Wriedt Larsen
ce967e1249 Merge branch 'main' into python-model-python2-specific-command-execution 2020-10-15 10:00:02 +02:00
Erik Krogh Kristensen
ab10c28cc4 change the default sorting order for print children to be location based 2020-10-15 09:53:52 +02:00
Rasmus Lerchedahl Petersen
0766eef49b Merge branch 'main' of github.com:github/codeql into SharedDataflow_ArgumentPassing 2020-10-15 09:49:21 +02:00
Erik Krogh Kristensen
74243d39aa remove location for arguments/parameters print node 2020-10-15 09:48:55 +02:00
Robert Marsh
fade75b0fb C++: add operator-- tests for insert iterators 2020-10-14 15:54:34 -07:00
Dave Bartolomeo
dfb687fd47 C++: Add ability to dump local dataflow info in IR dumps
This change adds a new module, `PrintIRLocalFlow.qll`, which can be imported into any query that uses both `PrintIR.qll` and the IR dataflow library. The IR dump printed by `PrintIR.qll` will be annotated with information about how each operand and instruction participates in dataflow.

For each operand and instruction, the following propeties are displayed:
- `flow`: Which local operands/instructions have flow to this node, and which local operands/instruction this node has flow to.
- `source`: `true` if this node is a source
- `sink`: `true` if this node is a sink
- `barrier`: Lists which kinds of barrier this node is. Can be zero or more of `full`, `in`, `out`, and `guard`. If the node is a guard barrier, the IR of the guarding instruction is also printed.

We already had a way to print additional properties for instructions and blocks, but not for operands. I added support for operand properties to `IRPropertyProvider`. These are now printed in a curly-brace-enclosed list immediately after the corresponding operand.

When printing flow, instructions are identified by their result ID (e.g., `m128`). Operands are identified by both the result ID of their instruction and their kind (e.g., `r145.left`). For flow from an operand to its use instruction, it just prints `result` at the operand, and prints only the operand kind on the instruction.

Example output:
```
#  344|     m344_34(vector<int, allocator<int>>)                                               = Chi                             : total:m344_20{flow:def->@, @->result}, partial:m344_33{flow:def->@, @->result}
#  344|         flow = total->@, partial->@, +m344_33->@, @->+r347_3, @->v347_7.side_effect, @->m347_9.total, @->m344_20.1
```
The `+` annotations indicate when the flow came from `isAdditionalFlowStep()`, rather than built-in local flow.
2020-10-14 18:09:54 -04:00
Dave Bartolomeo
794a6729bc C++: Add ability to dump local dataflow info in IR dumps
This change adds a new module, `PrintIRLocalFlow.qll`, which can be imported into any query that uses both `PrintIR.qll` and the IR dataflow library. The IR dump printed by `PrintIR.qll` will be annotated with information about how each operand and instruction participates in dataflow.

For each operand and instruction, the following propeties are displayed:
- `flow`: Which local operands/instructions have flow to this node, and which local operands/instruction this node has flow to.
- `source`: `true` if this node is a source
- `sink`: `true` if this node is a sink
- `barrier`: Lists which kinds of barrier this node is. Can be zero or more of `full`, `in`, `out`, and `guard`. If the node is a guard barrier, the IR of the guarding instruction is also printed.

We already had a way to print additional properties for instructions and blocks, but not for operands. I added support for operand properties to `IRPropertyProvider`. These are now printed in a curly-brace-enclosed list immediately after the corresponding operand.

When printing flow, instructions are identified by their result ID (e.g., `m128`). Operands are identified by both the result ID of their instruction and their kind (e.g., `r145.left`). For flow from an operand to its use instruction, it just prints `result` at the operand, and prints only the operand kind on the instruction.

Example output:
```
#  344|     m344_34(vector<int, allocator<int>>)                                               = Chi                             : total:m344_20{flow:def->@, @->result}, partial:m344_33{flow:def->@, @->result}
#  344|         flow = total->@, partial->@, +m344_33->@, @->+r347_3, @->v347_7.side_effect, @->m347_9.total, @->m344_20.1
```
The `+` annotations indicate when the flow came from `isAdditionalFlowStep()`, rather than built-in local flow.
2020-10-14 18:02:45 -04:00
Robert Marsh
126615a7ee C++: add prefix operator++ in inserter tests 2020-10-14 13:33:50 -07:00
Robert Marsh
fbe9bc8587 C++: add comments in new iterator models 2020-10-14 13:13:16 -07:00
Robert Marsh
a1a441d759 C++: output iterator flow through operator= models 2020-10-14 13:06:11 -07:00
Robert Marsh
f3843b8a40 Revert "C++: output iterator flow with user-defined operators"
This reverts commit 28fa26629c.
2020-10-14 13:01:37 -07:00
james
98d8ec488e add banner to sphinx template 2020-10-14 20:09:19 +01:00
Geoffrey White
09d4b3eb6d Merge branch 'main' into callderef 2020-10-14 17:41:14 +01:00
Rasmus Lerchedahl Petersen
d2b90662a3 Python: implement ToString on mappings 2020-10-14 17:31:13 +02:00
Taus
466c22f4a8 Merge pull request #4435 from RasmusWL/python-port-code-injection
Python: port code injection query
2020-10-14 16:41:42 +02:00
Rasmus Lerchedahl Petersen
6a3aed337f Python self -> range 2020-10-14 16:35:43 +02:00
Rasmus Lerchedahl Petersen
352418cb5d Python: track safe loaders 2020-10-14 16:33:55 +02:00
Taus Brock-Nannestad
f8190feef2 Python: Fix divergence in tuple/subscripted type toString
A slightly more complicated version of the situation in
https://github.com/github/codeql/pull/2507 could cause the `toString`
calculation to diverge. Although the previous PR took tuples nested
inside tuples into account (and subscripted types cannot be nested
inside each other in our modelling), it did not account for having
this nesting be interleaved, and this is what caused the divergence.

I have not done the usual "test case first to show the problem
exists", since this would also diverge and take forever to fail. The
instance observed in `scipy` was likely caused by something akin to

```python
x = ()
while True:
    x = x[(x,)]
```

Finally, to prevent this from happening with other types, I went
through and checked each instance where the string representation of
an `ObjectInternal` might potentially contain a reference to
itself (and thus explode). I encapsulated this in a
`bounded_toString` helper predicate, and used this in all the cases
where I was able to determine that the above _could_ happen.
2020-10-14 16:13:03 +02:00
yoff
5f6f85c998 Merge pull request #4465 from tausbn/python-remove-essa-flow
Python: Remove flow between ESSA variables
2020-10-14 15:37:39 +02:00
Rasmus Lerchedahl Petersen
b8cba381cf Merge branch 'main' of github.com:github/codeql into python-port-unsafe-deserialization 2020-10-14 15:01:30 +02:00
Tamas Vajk
5a91736b7a C#: Change public fields to properties 2020-10-14 14:44:01 +02:00
Rasmus Lerchedahl Petersen
3a281a1bd6 Python: Adjust comments and tests 2020-10-14 14:40:11 +02:00
Rasmus Wriedt Larsen
5db4f906d0 Merge branch 'main' into python-port-code-injection 2020-10-14 14:22:02 +02:00
Tom Hvitved
91806da2fa C#: Address review comments 2020-10-14 14:15:34 +02:00
Tom Hvitved
5d1a5920c7 C#: Reimplement flow-summary compilation 2020-10-14 14:15:34 +02:00
Tom Hvitved
444e607338 C#: Add missing flow through library code using params arguments 2020-10-14 14:15:34 +02:00
Tom Hvitved
f2dc2d912a C#: Add inter-procedural data-flow test for StringBuilder 2020-10-14 14:15:34 +02:00
yoff
ffe79f688d Apply suggestions from code review
Co-authored-by: Rasmus Wriedt Larsen <rasmuswriedtlarsen@gmail.com>
2020-10-14 14:08:16 +02:00
Ian Lynagh
9e518d2555 C++: Accept test change for p#n -> (unnamed parameter n) 2020-10-14 12:59:47 +01:00
Tom Hvitved
2af7e1c213 C#: Use CFG nodes instead of AST nodes in sign/modulus analysis 2020-10-14 13:39:44 +02:00
Taus
92ccb795fd Merge pull request #4415 from RasmusWL/python-flask-routed-parameter
Python: Add support for routed parameters in flask
2020-10-14 13:29:51 +02:00
Tamas Vajk
1094d192ab C#: Change extraction of NamespaceDeclarations to have ID based on the location 2020-10-14 13:26:01 +02:00
Rasmus Wriedt Larsen
1fde477a8f Python: Refactor argument matching 2020-10-14 13:22:35 +02:00
Rasmus Wriedt Larsen
680a6eb2a6 Python: Refactor argument matching (more) 2020-10-14 13:21:04 +02:00
Rasmus Wriedt Larsen
61ecec7d17 Merge pull request #4467 from tausbn/python-fix-import-type-tracking
Python: Fix unwanted module type tracking
2020-10-14 13:08:57 +02:00
Tamas Vajk
5fae440a58 C#: Reverse assignment child nodes in AST 2020-10-14 12:49:08 +02:00
Tamas Vajk
ee9a40e16d C#: Remove location from base types in the AST 2020-10-14 12:49:08 +02:00
yoff
27f474f0e9 Merge pull request #4429 from RasmusWL/python-model-invoke
Python: model invoke library
2020-10-14 12:13:35 +02:00
Rasmus Lerchedahl Petersen
dc7e7890f0 Python: Clearer naming and comments (I hope) 2020-10-14 12:03:05 +02:00