Commit Graph

16916 Commits

Author SHA1 Message Date
Geoffrey White
1b8d14077a Merge pull request #4481 from rvermeulen/patch-1
C++: Fix qldoc for getIncludeText
2020-10-15 13:00:33 +01:00
Tamás Vajk
e62c9b1382 Merge pull request #4472 from tamasvajk/feature/cleanup-3
C#: Change public fields to properties
2020-10-15 12:16:53 +02:00
Tom Hvitved
36f6e97cad Merge pull request #4371 from hvitved/csharp/library-flow-refactor
C#: Reimplement flow-summary compilation
2020-10-15 11:56:32 +02:00
Remco Vermeulen
7848c5f54d Fix qldoc for getIncludeText
The '<' was HTML encoded for some reason.
2020-10-15 11:49:18 +02:00
Taus
c8b93148a2 Merge pull request #4424 from RasmusWL/python-model-python2-specific-command-execution
Python: model Python 2 specific command execution
2020-10-15 10:52:43 +02:00
Anders Schack-Mulligen
60ce02ac18 Merge pull request #4469 from JLLeitschuh/additional-file-taint
Java: Track taint through java.io.File::toPath & java.nio.file.Path::toFile
2020-10-15 10:46:35 +02:00
Tom Hvitved
8728017328 C#: Increase fieldFlowBranchLimit in test
68014fd3bf means that more accessors are properly
extracted, and consequently the calls to `get_Item` in the test have more dispatch
targets. Increasing `fieldFlowBranchLimit` makes the test pass again.
2020-10-15 10:40:19 +02:00
Rasmus Wriedt Larsen
c5810d623b Merge pull request #4474 from tausbn/python-fix-tostring-divergence
Python: Fix divergence in tuple/subscripted type `toString`
2020-10-15 10:29:33 +02:00
Rasmus Wriedt Larsen
ce967e1249 Merge branch 'main' into python-model-python2-specific-command-execution 2020-10-15 10:00:02 +02:00
Taus
466c22f4a8 Merge pull request #4435 from RasmusWL/python-port-code-injection
Python: port code injection query
2020-10-14 16:41:42 +02:00
Taus Brock-Nannestad
f8190feef2 Python: Fix divergence in tuple/subscripted type toString
A slightly more complicated version of the situation in
https://github.com/github/codeql/pull/2507 could cause the `toString`
calculation to diverge. Although the previous PR took tuples nested
inside tuples into account (and subscripted types cannot be nested
inside each other in our modelling), it did not account for having
this nesting be interleaved, and this is what caused the divergence.

I have not done the usual "test case first to show the problem
exists", since this would also diverge and take forever to fail. The
instance observed in `scipy` was likely caused by something akin to

```python
x = ()
while True:
    x = x[(x,)]
```

Finally, to prevent this from happening with other types, I went
through and checked each instance where the string representation of
an `ObjectInternal` might potentially contain a reference to
itself (and thus explode). I encapsulated this in a
`bounded_toString` helper predicate, and used this in all the cases
where I was able to determine that the above _could_ happen.
2020-10-14 16:13:03 +02:00
yoff
5f6f85c998 Merge pull request #4465 from tausbn/python-remove-essa-flow
Python: Remove flow between ESSA variables
2020-10-14 15:37:39 +02:00
Tamas Vajk
5a91736b7a C#: Change public fields to properties 2020-10-14 14:44:01 +02:00
Rasmus Wriedt Larsen
5db4f906d0 Merge branch 'main' into python-port-code-injection 2020-10-14 14:22:02 +02:00
Tom Hvitved
91806da2fa C#: Address review comments 2020-10-14 14:15:34 +02:00
Tom Hvitved
5d1a5920c7 C#: Reimplement flow-summary compilation 2020-10-14 14:15:34 +02:00
Tom Hvitved
444e607338 C#: Add missing flow through library code using params arguments 2020-10-14 14:15:34 +02:00
Tom Hvitved
f2dc2d912a C#: Add inter-procedural data-flow test for StringBuilder 2020-10-14 14:15:34 +02:00
Taus
92ccb795fd Merge pull request #4415 from RasmusWL/python-flask-routed-parameter
Python: Add support for routed parameters in flask
2020-10-14 13:29:51 +02:00
Rasmus Wriedt Larsen
1fde477a8f Python: Refactor argument matching 2020-10-14 13:22:35 +02:00
Rasmus Wriedt Larsen
680a6eb2a6 Python: Refactor argument matching (more) 2020-10-14 13:21:04 +02:00
Rasmus Wriedt Larsen
61ecec7d17 Merge pull request #4467 from tausbn/python-fix-import-type-tracking
Python: Fix unwanted module type tracking
2020-10-14 13:08:57 +02:00
yoff
27f474f0e9 Merge pull request #4429 from RasmusWL/python-model-invoke
Python: model invoke library
2020-10-14 12:13:35 +02:00
Taus Brock-Nannestad
f3c07e3849 Python: Fix up import helper tests 2020-10-14 11:58:14 +02:00
Tamás Vajk
8127d9b93e Merge pull request #4404 from tamasvajk/feature/cleanup-2
C# extractor code cleanup
2020-10-14 11:02:40 +02:00
Rasmus Wriedt Larsen
b0cfa1d92d Python: Make "..Call" modeling classes extend DataFlow::CfgNode 2020-10-14 10:53:18 +02:00
Rasmus Wriedt Larsen
bfa5d18476 Python: Use new importNode 2020-10-14 10:49:38 +02:00
Rasmus Wriedt Larsen
7d600e4e8e Merge branch 'main' into python-port-code-injection 2020-10-14 10:48:38 +02:00
Rasmus Wriedt Larsen
4d9d2155fc Python: Make "..Call" modeling classes extend DataFlow::CfgNode 2020-10-14 10:44:58 +02:00
Rasmus Wriedt Larsen
b0e79890e6 Python: Use new importNode 2020-10-14 10:43:22 +02:00
Rasmus Wriedt Larsen
4597ba64d0 Merge branch 'main' into python-model-invoke 2020-10-14 10:41:37 +02:00
Rasmus Wriedt Larsen
eff47457bf Python: Refactor argument matching 2020-10-14 10:37:38 +02:00
Rasmus Wriedt Larsen
2ea71f574c Python: Make "..Call" modeling classes extend DataFlow::CfgNode 2020-10-14 10:37:37 +02:00
Rasmus Wriedt Larsen
2e30f58aa2 Python: Use new importNode 2020-10-14 10:37:36 +02:00
Rasmus Wriedt Larsen
ecf70c5f30 Merge branch 'main' into python-model-python2-specific-command-execution 2020-10-14 10:36:43 +02:00
Rasmus Wriedt Larsen
74bd045488 Python: Make "..Call" modeling classes extend DataFlow::CfgNode 2020-10-14 10:24:46 +02:00
Rasmus Wriedt Larsen
ba158f3317 Python: Use new importNode 2020-10-14 10:17:35 +02:00
Rasmus Wriedt Larsen
49d2e68d12 Merge branch 'main' into python-flask-routed-parameter 2020-10-14 10:16:00 +02:00
Jonathan Leitschuh
fc71ca747d Java: Track taint through java.io.File::toPath & java.nio.file.Path::toFile 2020-10-13 21:15:09 -04:00
Taus Brock-Nannestad
7d86b53b71 Python: Fix unwanted module type tracking 2020-10-13 22:47:57 +02:00
Taus Brock-Nannestad
76e5b59dab Python: Add test case for unwanted module type tracking 2020-10-13 22:47:03 +02:00
Robert Marsh
b49aa677d0 Merge pull request #4459 from geoffw0/setex
C++: Additional taint flows through std::set
2020-10-13 15:17:54 -04:00
Taus
83937bacae Merge pull request #4448 from RasmusWL/python-simplify-import-modeling
Python: simplify import modeling
2020-10-13 18:08:07 +02:00
Rasmus Wriedt Larsen
2c5996f694 Python: Refactor subprocess_attr type-tracker
Co-authored-by: Taus <tausbn@github.com>
2020-10-13 17:21:21 +02:00
Taus Brock-Nannestad
fdb489fc93 Python: Remove flow between ESSA variables
This required a minor change in the type tracker implementation, but
apart from that no other changes appear to be needed. Seems to clean
up the test output quite a bit.
2020-10-13 16:35:41 +02:00
Rasmus Wriedt Larsen
b895641a83 Merge pull request #4464 from tausbn/python-remove-dataflowcfgnode
Python: Get rid of `DataFlowCfgNode`
2020-10-13 15:08:28 +02:00
Rasmus Wriedt Larsen
76c9b8c49f Python: Expose importNode instead of importModule/importMember
Since predicate name `import` is not allowed, I adopted `importNode` as it sort
of matches what `exprNode` does.

---

Due to only using `importMember` in `os_attr` we previously didn't handle
`import os.path as alias` :|

I did creat a hotfix for this (https://github.com/github/codeql/pull/4446), but
in doing so I realized the core of the problem: We're exposing ourselves to
making these kinds of mistakes by having BOTH importModule and importMember, and
we don't really gain anything from doing this!

We do loose the ability to easily only modeling `from mod import val` and not
`import mod.val`, but I don't think that will ever be relevant.

This change will also make us to recognize some invalid code, for example in

    import os.system as runtime_error

we would now model that `runtime_error` is a reference to the `os.system`
function (although the actual import would result in a runtime error).

Overall these are tradeoffs I'm willing to make, as it does makes things simpler
from a QL modeling point of view, and THAT sounds nice 👍
2020-10-13 15:03:22 +02:00
Tamas Vajk
ce9624e61d C#: Remove unneeded vscode settings from settings.json 2020-10-13 14:50:46 +02:00
Rasmus Wriedt Larsen
4bfd55f1af Python: Show problem with os.path modeling
This is not a very good test for showing that we don't handle direct imports,
but it was the best I had available without inventing something new. It's very
fragile, since any of these would propagate taint (due to handling all `join`
calls as if the qualifier was a string):

    ospath_alias.join(ts)
    ospath_alias.join(ts, "foo", "bar")

But this test DOES serve the purpose of illustrating that my fix works :D
2020-10-13 14:50:00 +02:00
Tamas Vajk
ce793c357f C#: Adjust parameters of DefinitionField ctor 2020-10-13 14:45:38 +02:00