Commit Graph

34755 Commits

Author SHA1 Message Date
Erik Krogh Kristensen
e06f6529f1 refactor API-graph labels to an IPA type 2022-02-01 17:32:08 +01:00
Rasmus Wriedt Larsen
2bc4a60496 Python: Unify normal dataflow test setup
I went with NormalDataflowTest to signify that if you don't know what
you're looking for, this is probably the one. I did not want to just
call it DataflowTest, since that becomes a big vague when there are also
`FlowTest.qll` and `MaximalFlowTest.qll` -- I'm open to renaming this
though 👍
2022-02-01 17:31:31 +01:00
Rasmus Wriedt Larsen
41319607a9 Python: Use InlineExpectationsTest for field-flow tests
I deleted the old tests, so it's very clear what tests to look for
2022-02-01 17:31:31 +01:00
Rasmus Wriedt Larsen
d6f415bae2 Python: Run match tests if Python 3.10 or newer
Also fixes a bug in the tests
2022-02-01 17:31:31 +01:00
liangjinhuang
1885b683f7 style:formatDocument 2022-02-02 00:21:26 +08:00
liangjinhuang
af2e8ff8c6 feat:modify source regular matching rules 2022-02-02 00:10:15 +08:00
Henry Mercer
368839edfc JS: Fix QLDoc style in ExtractMisclassifiedEndpointFeatures.ql 2022-02-01 15:39:15 +00:00
Arthur Baars
ea901adb3c Merge pull request #7799 from github/aibaars/fix-ruby-workflows
Ruby: use ruby specific cache key
2022-02-01 16:28:14 +01:00
Henry Mercer
db0b4fc463 JS: Add model building pack for ML-powered queries
Tests are currently still internal. They will be migrated to
`github/codeql` in a subsequent PR.
2022-02-01 15:03:26 +00:00
Arthur Baars
73d60550ce QL-QL: fix cache keys 2022-02-01 15:57:59 +01:00
Erik Krogh Kristensen
0f85a52f09 Merge pull request #7773 from erik-krogh/CWE-367
JS: add a js/file-system-race query
2022-02-01 15:36:13 +01:00
Rasmus Wriedt Larsen
a4bb0cc5d8 Python: Run tests for fieldflow/test.py 2022-02-01 15:32:07 +01:00
Rasmus Wriedt Larsen
1390f034f3 Python: Delete duplicated tests
All the same tests are present in `fieldflow/test.py`
2022-02-01 15:31:30 +01:00
Arthur Baars
6451a71a78 Ruby: use ruby specific cache key 2022-02-01 15:18:09 +01:00
Rasmus Wriedt Larsen
1394b38032 Python: Improve customSanitizer tests
Before we didn't show how we treated the value _after_ the check. But we
do actually handle this nicely 💪
2022-02-01 15:09:29 +01:00
Mathias Vorreiter Pedersen
3597d80340 Merge pull request #7787 from Yonah125/main
C/C++ : Useless test
2022-02-01 14:01:27 +00:00
Jeroen Ketema
ff1c971100 Add query for missing mode argument in open/openat calls 2022-02-01 14:52:22 +01:00
Rasmus Lerchedahl Petersen
1e2428cb6b python: create LDAP module in Concepts 2022-02-01 14:39:58 +01:00
Nick Rolfe
5828a61fec Merge pull request #7795 from github/nickrolfe/graph_test_edge_ordering
Ruby/C#: add semmle.order attribute to edges in CFG tests
2022-02-01 13:36:15 +00:00
Rasmus Lerchedahl Petersen
c2cd58edc4 python: rewrite to separate configurations
source nodes get duplicated, so perhaps flow states
are actually better for performance?
2022-02-01 14:36:11 +01:00
Erik Krogh Kristensen
a51f892a99 move dot in qhelp
Co-authored-by: Esben Sparre Andreasen <esbena@github.com>
2022-02-01 14:34:30 +01:00
Michael Nebel
2c6e35f55e C#: Add more attribute testcases. 2022-02-01 14:06:45 +01:00
Rasmus Lerchedahl Petersen
7df5c70c6f python: update frameworks.rst 2022-02-01 13:54:34 +01:00
Rasmus Lerchedahl Petersen
bec8c0daea python: update change note 2022-02-01 13:39:03 +01:00
Rasmus Lerchedahl Petersen
c587084758 python: use standard InstanceSource construction 2022-02-01 13:31:16 +01:00
Michael Nebel
31a70a17a4 C#: Add attribute tests for delegate types. 2022-02-01 13:29:26 +01:00
Michael Nebel
8ee27be908 C#: Small change in the attribute elements test. We now require that the attribute should be in source code and not the attributable. 2022-02-01 13:29:26 +01:00
Michael Nebel
7d1d2e792c C#: Add specialized ql classes for each attribute kind and update AST printing. 2022-02-01 13:29:26 +01:00
Michael Nebel
5a6667efc5 C#: Update attribute test cases. 2022-02-01 13:29:26 +01:00
Michael Nebel
c94cdfa79a C#: Update the extractor to include the kind of the extracted attributes. Furthermore, include method return attributes. 2022-02-01 13:29:26 +01:00
Michael Nebel
e86ac73628 C#: Add attribute kind to the dbscheme for the attribute relation. 2022-02-01 13:29:26 +01:00
Erik Krogh Kristensen
e6c90670e6 Merge pull request #7740 from erik-krogh/CWE-347
JS: promote the js/jwt-missing-verification query out of experimental
2022-02-01 13:10:35 +01:00
museljh
012434b152 Update python/ql/src/experimental/Security/CWE-338/InsecureRandomness.ql
Co-authored-by: Rasmus Wriedt Larsen <rasmuswriedtlarsen@gmail.com>
2022-02-01 19:00:06 +08:00
museljh
a6002186bd Update python/ql/src/experimental/Security/CWE-338/InsecureRandomness.ql
Co-authored-by: Rasmus Wriedt Larsen <rasmuswriedtlarsen@gmail.com>
2022-02-01 18:59:12 +08:00
Jeroen Ketema
dbac927721 Take into account that the 4th argument of openat may be omitted
This matches what is done for `open`.
2022-02-01 11:41:07 +01:00
Rasmus Wriedt Larsen
f7a0b17ed6 Merge pull request #7687 from yoff/python/PathInjection-FlowState
python: Rewrite path injection query to use flow state
2022-02-01 11:33:37 +01:00
yoff
b120721942 Update python/ql/src/Security/CWE-090/LdapInjection.ql
Co-authored-by: Rasmus Wriedt Larsen <rasmuswriedtlarsen@gmail.com>
2022-02-01 11:02:47 +01:00
Rasmus Lerchedahl Petersen
119a7e4f34 python: provide links for Flask 2022-02-01 10:55:45 +01:00
Rasmus Lerchedahl Petersen
ecea392a08 python: rewrite qhelp overview
(combining the Java version and the JS version)
2022-02-01 10:47:18 +01:00
Rasmus Lerchedahl Petersen
26befebfc2 python: drop precision and add severity score
Given both the original FP score and our concerns
regarding sanitizers, `@precision medium`, which
is aligned with other languages, feels appropriate.
2022-02-01 10:34:36 +01:00
Rasmus Lerchedahl Petersen
7511b33512 python: "command" -> "log" 2022-02-01 10:23:16 +01:00
yoff
45f0bfd8f0 Apply suggestions from code review
Co-authored-by: Rasmus Wriedt Larsen <rasmuswriedtlarsen@gmail.com>
2022-02-01 10:06:37 +01:00
yoff
c03f89d712 Apply suggestions from code review
Co-authored-by: Rasmus Wriedt Larsen <rasmuswriedtlarsen@gmail.com>
2022-02-01 10:04:26 +01:00
Mathias Vorreiter Pedersen
a59a9ba82b C++: Autoformat. 2022-02-01 08:28:53 +00:00
yoff
8df04c58e9 Merge pull request #7793 from tausbn/python-fix-bad-TPythonTuple-join-order
Python: Fix bad join order in `TPythonTuple`
2022-01-31 22:39:58 +01:00
Harry Maclean
e5b7478028 Merge pull request #7780 from github/hmac/split-tests
Ruby: Split up CI jobs
2022-02-01 09:10:01 +13:00
Nick Rolfe
990e07b986 Ruby/C#: add semmle.order attribute to edges in CFG tests 2022-01-31 20:08:24 +00:00
BACK Yonah
46c1744204 C/C++: getFullyConverted replaced by getConversion* 2022-01-31 18:51:18 +01:00
Taus
4a29095e3b Python: Fix bad join order in TPythonTuple
TL;DR: Something introduced the following bad join order:
```
(227s) Tuple counts for dom#TObject::TPythonTuple#ff/2@i2#8f58670w after 3m46s:
25000      ~0%     {2} r1 = SCAN PointsToContext::PointsToContext::appliesToScope_dispred#ff#prev_delta OUTPUT In.1, In.0 'context'
24000      ~1%     {2} r2 = JOIN r1 WITH @py_scope#f ON FIRST 1 OUTPUT Lhs.1 'context', Lhs.0
1076876712 ~6%     {3} r3 = JOIN r2 WITH Flow::TupleNode#class#f CARTESIAN PRODUCT OUTPUT Rhs.0, Lhs.0 'context', Lhs.1
870129666  ~0%     {3} r4 = JOIN r3 WITH Flow::ControlFlowNode::isLoad_dispred#f ON FIRST 1 OUTPUT Lhs.1 'context', Lhs.2, Lhs.0 'origin'
870129000  ~0%     {3} r5 = r4 AND NOT dom#TObject::TPythonTuple#ff#prev(Lhs.2 'origin', Lhs.0 'context')
870129000  ~1%     {3} r6 = SCAN r5 OUTPUT In.2 'origin', In.1, In.0 'context'
9000       ~0%     {2} r7 = JOIN r6 WITH Flow::ControlFlowNode::getScope_dispred#ff ON FIRST 2 OUTPUT Lhs.0 'origin', Lhs.2 'context'
                    return r7
```
(...the above being the tuple counts _at the point when I cancelled the
query_!)

Rewriting the code to force a join between `TupleNode#class` and
`getScope` results in the following join orders:

```
(0s) Tuple counts for TObject::scope_loads_tuplenode#ff/2@b3cf0bo5 after 13ms:
37369 ~3%     {1} r1 = JOIN Flow::TupleNode#class#f WITH Flow::ControlFlowNode::isLoad_dispred#f ON FIRST 1 OUTPUT Lhs.0 'origin'
37369 ~3%     {2} r2 = JOIN r1 WITH Flow::ControlFlowNode::getScope_dispred#ff ON FIRST 1 OUTPUT Rhs.1 's', Lhs.0 'origin'
            return r2
```
and
```
(78s) Tuple counts for dom#TObject::TPythonTuple#ff/2@i53#121c440w after 6ms:
34736 ~3%     {2} r1 = SCAN PointsToContext::PointsToContext::appliesToScope_dispred#ff#prev_delta OUTPUT In.1, In.0 'context'
7370  ~5%     {2} r2 = JOIN r1 WITH TObject::scope_loads_tuplenode#ff ON FIRST 1 OUTPUT Lhs.1 'context', Rhs.1 'origin'
7370  ~5%     {2} r3 = r2 AND NOT dom#TObject::TPythonTuple#ff#prev(Lhs.1 'origin', Lhs.0 'context')
7370  ~1%     {2} r4 = SCAN r3 OUTPUT In.1 'origin', In.0 'context'
            return r4
```
the latter being the largest iteration of `dom#TPythonTuple` throughout
the log.

No other major performance issues were observed.
2022-01-31 16:59:50 +00:00
jorgectf
080775c873 Merge branch 'jorgectf/python/deserialization' of https://github.com/jorgectf/codeql into jorgectf/python/deserialization 2022-01-31 17:48:47 +01:00