Commit Graph

5928 Commits

Author SHA1 Message Date
Ahmed Farid
e7742bd87c Create CryptographicOperation.qll
Provides models for Python's Cryptography-related libraries
2022-07-20 12:58:13 +01:00
Ahmed Farid
4f082e28e5 Update and rename TimingAttackAgainstSignature.py to TimingAttackAgainstHash.py 2022-07-20 12:26:57 +01:00
Ahmed Farid
b3925ae988 Update PossibleTimingAttackAgainstSignature.qlref 2022-07-20 00:57:26 +01:00
Ahmed Farid
3d092f9569 Update TimingAttackAgainstSignature.ql 2022-07-20 00:56:52 +01:00
Ahmed Farid
27d81548a7 Update PossibleTimingAttackAgainstHash.ql 2022-07-20 00:55:22 +01:00
Ahmed Farid
bfce1898b9 Update and rename PossibleTimingAttackAgainstSignature.ql to PossibleTimingAttackAgainstHash.ql 2022-07-20 00:49:09 +01:00
Asger F
b9bdee6651 Merge branch 'main' into post-release-prep/codeql-cli-2.10.1 2022-07-19 16:24:35 +02:00
Taus
bfe90413e2 Merge pull request #9847 from alexet/alexet/fix-predicate-binding
Python: Fix binding incorrect predicate.
2022-07-19 13:59:13 +02:00
alexet
f9b6ca76e5 Python: Fix binding incorrect predicate. 2022-07-18 16:28:19 +01:00
yo-h
d4443592eb Merge pull request #9776 from raulgarciamsft/azure-sdk-client-encryption-version
New queries to detect unsafe client side encryption in Azure Storage
2022-07-16 14:59:51 -04:00
Raul Garcia
6b17890e4f Fixing warning on usage of a deprecated feature. 2022-07-16 08:30:06 -07:00
Andrew Eisenberg
b897a40228 Move python contextual queries to lib folders
This will ensure that python projects can use jump to ref/def in
vscode when the core libraries are not installed.
2022-07-15 13:12:17 -07:00
Ahmed Farid
7406273346 Update TimingAttack.qhelp 2022-07-14 17:56:58 +01:00
Ahmed Farid
f4654136d6 Update TimingAttack.qhelp 2022-07-14 17:56:13 +01:00
github-actions[bot]
0ee476129a Post-release preparation for codeql-cli-2.10.1 2022-07-14 14:38:49 +00:00
github-actions[bot]
d1aa0d7dd3 Release preparation for version 2.10.1 2022-07-14 08:56:03 +00:00
Raul Garcia
f7c47b6c75 Update python/ql/src/experimental/Security/CWE-327/Azure/UnsafeUsageOfClientSideEncryptionVersion.py
Co-authored-by: Taus <tausbn@github.com>
2022-07-13 08:34:48 -07:00
Erik Krogh Kristensen
c4f44bb67f sync files 2022-07-13 10:01:26 +02:00
Raul Garcia
0dbb03f732 Adding CVE information. 2022-07-12 21:49:19 -07:00
Raul Garcia
d929b1338b Addressing API::Node feedback for all predicates 2022-07-12 11:55:06 -07:00
Raul Garcia
d5791e2d56 Addressing feedback from the PR 2022-07-11 15:45:15 -07:00
Raul Garcia
ac05577966 Making various changes based on the feedback. Pending: 2 non-trivial fixes for Java & Python. 2022-07-11 13:25:35 -07:00
Raul Garcia
e5702d0e15 Update python/ql/src/experimental/Security/CWE-327/Azure/UnsafeUsageOfClientSideEncryptionVersion.ql
Co-authored-by: Taus <tausbn@github.com>
2022-07-11 13:07:37 -07:00
Raul Garcia
7fc9ae6c49 Update python/ql/src/experimental/Security/CWE-327/Azure/UnsafeUsageOfClientSideEncryptionVersion.ql
Co-authored-by: Taus <tausbn@github.com>
2022-07-11 13:07:20 -07:00
Taus
ec363166ba Python: Make UserInputMsgConfig public 2022-07-11 15:24:31 +02:00
Raul Garcia
dd1a9a22e3 Update UnsafeUsageOfClientSideEncryptionVersion.qhelp 2022-07-05 13:58:38 -07:00
Raul Garcia
e43e5810cf New queries to detect unsafe client side encryption in Azure Storage 2022-07-01 17:08:35 -07:00
CodeQL CI
5b5a52fa25 Merge pull request #9551 from yoff/python/port-tarslip
Approved by RasmusWL
2022-07-01 12:58:25 +01:00
yoff
cf9b69b5f2 python: More helpful comment 2022-06-30 13:07:13 +00:00
yoff
b0a29b146a Update python/ql/lib/semmle/python/security/dataflow/TarSlipQuery.qll
Co-authored-by: Rasmus Wriedt Larsen <rasmuswriedtlarsen@gmail.com>
2022-06-30 14:54:01 +02:00
yoff
df7ffb2880 Update python/ql/lib/semmle/python/security/dataflow/TarSlipCustomizations.qll
Co-authored-by: Rasmus Wriedt Larsen <rasmuswriedtlarsen@gmail.com>
2022-06-30 14:53:49 +02:00
Andrew Eisenberg
fbeecd6c08 Merge pull request #9744 from github/aeisenberg/move-contextual-queries 2022-06-29 11:44:33 -07:00
Andrew Eisenberg
7864a7580e Fix import statements 2022-06-29 10:22:45 -07:00
Andrew Eisenberg
ddf06f8617 Add change notes and qldoc for moved files 2022-06-29 10:03:12 -07:00
Andrew Eisenberg
a3f4d1bf66 Move contextual queries from src to lib
With this change, users are now able to run View AST command in
vscode within vscode workspaces that do not include the core libraries.
The relevant core library only needs to be installed in the package
cache.
2022-06-29 07:51:26 -07:00
yoff
8988a02806 Merge pull request #9733 from tausbn/python-fix-bad-mro-flatten-list-join
Python: Fix bad join in MRO `flatten_list`
2022-06-29 13:29:48 +02:00
yoff
f122af81ea Merge pull request #9741 from tausbn/python-fix-bad-join-in-regexpbackref-getgroup
Python: Fix bad join in `RegExpBackRef::getGroup`
2022-06-29 13:23:07 +02:00
yoff
731f866242 Merge pull request #9717 from tausbn/python-fix-bad-mro-linearization-of-bases-join
Python: Fix bad join in MRO
2022-06-29 13:08:18 +02:00
Jeroen Ketema
55e052af26 Merge pull request #9686 from aschackmull/dataflow/no-node-scan
Dataflow performance: Avoid node scans
2022-06-29 10:38:56 +02:00
Ahmed Farid
f5d0791b4f Update TimingAttack.qll 2022-06-29 00:56:15 +01:00
Ahmed Farid
98909c2069 Update TimingAttackAgainstSensitiveInfo.ql 2022-06-29 00:55:21 +01:00
Ahmed Farid
41b4c06f2d Update TimingAttackAgainstSignature.ql 2022-06-29 00:54:44 +01:00
Ahmed Farid
e20fefc3ad Update TimingAttackAgainstHeader.ql 2022-06-29 00:54:03 +01:00
Ahmed Farid
5742046edf Update PossibleTimingAttackAgainstSignature.ql 2022-06-29 00:51:51 +01:00
Ahmed Farid
acbb4042df Update TimingAttack.qhelp 2022-06-29 00:51:12 +01:00
yoff
1105cd569b Merge branch 'main' into python/port-tarslip 2022-06-28 22:17:28 +02:00
yoff
ac0c8d238f python: only clear taint on false-edge 2022-06-28 20:14:52 +00:00
Taus
38b8640582 Python: Fix bad join in RegExpBackRef::getGroup
Although this wasn't (as far as I know) causing any performance issues,
it was making the join-order badness report quite noisy, and so I
figured it was worth fixing.

Before:
```
Tuple counts for RegexTreeView::RegExpBackRef::getGroup#dispred#f0820431#ff/2@d3441d0b after 84ms:
1501195 ~3%     {2} r1 = JOIN RegexTreeView::RegExpTerm::getLiteral#dispred#f0820431#ff_10#join_rhs WITH RegexTreeView::RegExpTerm::getLiteral#dispred#f0820431#ff_10#join_rhs ON FIRST 1 OUTPUT Rhs.1 'result', Lhs.1 'result'
149     ~0%     {5} r2 = JOIN r1 WITH RegexTreeView::RegExpBackRef#class#31aac2a7#ffff ON FIRST 1 OUTPUT Rhs.1, Rhs.2, Rhs.3, Lhs.1 'result', Lhs.0 'this'
149     ~1%     {3} r3 = JOIN r2 WITH regex::RegexString::numbered_backreference#dispred#f0820431#ffff ON FIRST 3 OUTPUT Lhs.3 'result', Rhs.3, Lhs.4 'this'
4       ~0%     {2} r4 = JOIN r3 WITH RegexTreeView::RegExpGroup::getNumber#dispred#f0820431#ff ON FIRST 2 OUTPUT Lhs.2 'this', Lhs.0 'result'

1501195 ~3%     {2} r5 = JOIN RegexTreeView::RegExpTerm::getLiteral#dispred#f0820431#ff_10#join_rhs WITH RegexTreeView::RegExpTerm::getLiteral#dispred#f0820431#ff_10#join_rhs ON FIRST 1 OUTPUT Lhs.1 'result', Rhs.1 'result'
42526   ~0%     {5} r6 = JOIN r5 WITH RegexTreeView::RegExpGroup#31aac2a7#ffff ON FIRST 1 OUTPUT Lhs.1 'this', Lhs.0 'result', Rhs.1, Rhs.2, Rhs.3
22      ~0%     {8} r7 = JOIN r6 WITH RegexTreeView::RegExpBackRef#class#31aac2a7#ffff ON FIRST 1 OUTPUT Lhs.2, Lhs.3, Lhs.4, Lhs.1 'result', Lhs.0 'this', Rhs.1, Rhs.2, Rhs.3
0       ~0%     {6} r8 = JOIN r7 WITH regex::RegexString::getGroupName#dispred#f0820431#ffff ON FIRST 3 OUTPUT Lhs.5, Lhs.6, Lhs.7, Rhs.3, Lhs.3 'result', Lhs.4 'this'
0       ~0%     {2} r9 = JOIN r8 WITH regex::RegexString::named_backreference#dispred#f0820431#ffff ON FIRST 4 OUTPUT Lhs.5 'this', Lhs.4 'result'

4       ~0%     {2} r10 = r4 UNION r9
                return r10
```

In this case I opted for a classical solution: tying together the
literal and number (or name) part of the backreference in order to
encourage a two-column join.

After:
```
Tuple counts for RegexTreeView::RegExpBackRef::getGroup#dispred#f0820431#ff/2@b0cc4d5n after 0ms:
898  ~1%     {3} r1 = JOIN RegexTreeView::RegExpTerm::getLiteral#dispred#f0820431#ff WITH RegexTreeView::RegExpGroup::getNumber#dispred#f0820431#ff ON FIRST 1 OUTPUT Lhs.1, Rhs.1, Lhs.0 'result'
4    ~0%     {2} r2 = JOIN r1 WITH RegexTreeView::RegExpBackRef::hasLiteralAndNumber#f0820431#fff_120#join_rhs ON FIRST 2 OUTPUT Rhs.2 'this', Lhs.2 'result'

1110 ~0%     {5} r3 = JOIN RegexTreeView::RegExpGroup#31aac2a7#ffff WITH RegexTreeView::RegExpTerm::getLiteral#dispred#f0820431#ff ON FIRST 1 OUTPUT Lhs.1, Lhs.2, Lhs.3, Lhs.0 'result', Rhs.1
146  ~0%     {3} r4 = JOIN r3 WITH regex::RegexString::getGroupName#dispred#f0820431#ffff ON FIRST 3 OUTPUT Lhs.4, Rhs.3, Lhs.3 'result'
0    ~0%     {2} r5 = JOIN r4 WITH RegexTreeView::RegExpBackRef::hasLiteralAndName#f0820431#fff_120#join_rhs ON FIRST 2 OUTPUT Rhs.2 'this', Lhs.2 'result'

4    ~0%     {2} r6 = r2 UNION r5
            return r6
```
2022-06-28 16:51:09 +00:00
Taus
b98c482c47 Python: Fix bad join in MRO flatten_list
This bad join was identified by the join-order-badness report, which
showed that:

py/use-of-input:MRO::flatten_list#f4eaf05f#fff#9c5fe54whnlqffdgu65vhb8uhpg# (order_500000)

calculated a whopping 212,820,108 tuples in order to produce an output of
size 55516, roughly 3833 times more effort than needed.

Here's a snippet of the slowest iteration of that predicate:
```
Tuple counts for MRO::flatten_list#f4eaf05f#fff/3@i1839#0265eb3w after 14ms:
0     ~0%     {3} r1 = JOIN MRO::need_flattening#f4eaf05f#f#prev_delta WITH MRO::ConsList#f4eaf05f#fff#reorder_2_0_1#prev ON FIRST 1 OUTPUT Rhs.1, Lhs.0 'list', Rhs.2
0     ~0%     {3} r2 = JOIN r1 WITH MRO::ClassList::length#f0820431#ff#prev ON FIRST 1 OUTPUT Lhs.2, Lhs.1 'list', Rhs.1 'n'
0     ~0%     {3} r3 = JOIN r2 WITH MRO::ClassListList::flatten#dispred#f0820431#ff#prev ON FIRST 1 OUTPUT Lhs.1 'list', Lhs.2 'n', Rhs.1 'result'

0     ~0%     {3} r4 = SCAN MRO::ConsList#f4eaf05f#fff#prev_delta OUTPUT In.2 'list', In.0, In.1
0     ~0%     {3} r5 = JOIN r4 WITH MRO::need_flattening#f4eaf05f#f#prev ON FIRST 1 OUTPUT Lhs.1, Lhs.2, Lhs.0 'list'
0     ~0%     {3} r6 = JOIN r5 WITH MRO::ClassList::length#f0820431#ff#prev ON FIRST 1 OUTPUT Lhs.1, Lhs.2 'list', Rhs.1 'n'
0     ~0%     {3} r7 = JOIN r6 WITH MRO::ClassListList::flatten#dispred#f0820431#ff#prev ON FIRST 1 OUTPUT Lhs.1 'list', Lhs.2 'n', Rhs.1 'result'

0     ~0%     {3} r8 = r3 UNION r7

26355 ~2%     {3} r9 = SCAN MRO::ConsList#f4eaf05f#fff#prev OUTPUT In.2 'list', In.0, In.1

0     ~0%     {3} r10 = JOIN r9 WITH MRO::need_flattening#f4eaf05f#f#prev ON FIRST 1 OUTPUT Lhs.1, Lhs.2, Lhs.0 'list'
0     ~0%     {3} r11 = JOIN r10 WITH MRO::ClassList::length#f0820431#ff#prev_delta ON FIRST 1 OUTPUT Lhs.1, Lhs.2 'list', Rhs.1 'n'
0     ~0%     {3} r12 = JOIN r11 WITH MRO::ClassListList::flatten#dispred#f0820431#ff#prev ON FIRST 1 OUTPUT Lhs.1 'list', Lhs.2 'n', Rhs.1 'result'
...
```
(... and a bunch more lines. The same construction appears several times,
but the join order is the same each time.)

Clearly it would be better to start with whatever is in `need_flattening`,
and then do the other joins. This is what the present fix does (by
unbinding `list` in all but the `needs_flattening` call).

After the fix, the slowest iteration is as follows:

```
Tuple counts for MRO::flatten_list#f4eaf05f#fff/3@i2617#8155ab3w after 9ms:
0 ~0%     {2} r1 = SCAN MRO::need_flattening#f4eaf05f#f#prev_delta OUTPUT In.0 'list', In.0 'list'

0 ~0%     {3} r2 = JOIN r1 WITH MRO::ConsList#f4eaf05f#fff#reorder_2_0_1#prev ON FIRST 1 OUTPUT Rhs.1, Lhs.1 'list', Rhs.2
0 ~0%     {3} r3 = JOIN r2 WITH MRO::ClassList::length#f0820431#ff#prev ON FIRST 1 OUTPUT Lhs.2, Lhs.1 'list', Rhs.1 'n'
0 ~0%     {3} r4 = JOIN r3 WITH MRO::ClassListList::flatten#dispred#f0820431#ff#prev ON FIRST 1 OUTPUT Lhs.1 'list', Lhs.2 'n', Rhs.1 'result'

1 ~0%     {2} r5 = SCAN MRO::need_flattening#f4eaf05f#f#prev OUTPUT In.0 'list', In.0 'list'

0 ~0%     {3} r6 = JOIN r5 WITH MRO::ConsList#f4eaf05f#fff#reorder_2_0_1#prev_delta ON FIRST 1 OUTPUT Rhs.1, Lhs.1 'list', Rhs.2
0 ~0%     {3} r7 = JOIN r6 WITH MRO::ClassList::length#f0820431#ff#prev ON FIRST 1 OUTPUT Lhs.2, Lhs.1 'list', Rhs.1 'n'
0 ~0%     {3} r8 = JOIN r7 WITH MRO::ClassListList::flatten#dispred#f0820431#ff#prev ON FIRST 1 OUTPUT Lhs.1 'list', Lhs.2 'n', Rhs.1 'result'
...
```
(... and so on. The remainder is 0 tuples all the way.)

In total, we went from
```
40.6s |  7614 |  15ms @ 1839 | MRO::flatten_list#f4eaf05f#fff@0265eb3w
```
to
```
7.8s |  7614 |  11ms @ 2617 | MRO::flatten_list#f4eaf05f#fff@8155ab3w
```
2022-06-28 14:17:47 +00:00
Asger F
a522562f93 Merge pull request #9369 from asgerf/python/api-graph-api
Python: API graph renaming and documentation
2022-06-28 14:48:12 +02:00