Commit Graph

4589 Commits

Author SHA1 Message Date
Ahmed Farid
466f75bad8 Update Concepts.qll 2022-03-06 23:53:00 +01:00
Ahmed Farid
be7c619ca8 Update zipslip_bad.py 2022-03-04 00:48:45 +01:00
Ahmed Farid
5e14d89714 Update ZipSlip.qll 2022-03-03 17:12:06 +01:00
Ahmed Farid
c45b67c316 Create zipslip_good.py 2022-03-02 18:10:24 +01:00
Ahmed Farid
aef1df122b Create zipslip_bad.py 2022-03-02 18:09:45 +01:00
Ahmed Farid
c8f73ec845 Create ZipSlip.qlref 2022-03-02 18:08:32 +01:00
Ahmed Farid
70c0c7e461 Update zipslip_bad.py 2022-03-01 00:24:33 +01:00
Ahmed Farid
85bcaa96ce Update Concepts.qll 2022-03-01 00:23:06 +01:00
Ahmed Farid
c22b032bbe Update Zip.qll 2022-03-01 00:11:33 +01:00
Ahmed Farid
67d3498891 Update ZipSlip.ql 2022-03-01 00:07:37 +01:00
Ahmed Farid
b29936716d Update Frameworks.qll 2022-03-01 00:06:22 +01:00
Ahmed Farid
76bd3317eb Create Zip.qll 2022-03-01 00:05:30 +01:00
Ahmed Farid
abe25da3df Create ZipSlip.qll 2022-03-01 00:04:02 +01:00
Ahmed Farid
3eae13161f Delete ZipSlipCheckLib.ql 2022-03-01 00:01:34 +01:00
Ahmed Farid
21f6ad5190 Update and rename ZipSlipCheck.ql to ZipSlip.ql 2022-03-01 00:01:06 +01:00
Ahmed Farid
c207294dfc Update zipslip_good.py 2022-02-26 18:31:22 +01:00
Ahmed Farid
d0d14be693 Update ZipSlip.qhelp 2022-02-26 18:25:13 +01:00
root
5ed5e0b105 Add query to detect ZipSlip 2022-02-13 16:44:27 -05:00
Chuan-kai Lin
9b4dbb9dd8 Merge pull request #7895 from github/cklin/upgrades-initial-dbscheme
Upgrade scripts testing: set initial dbschemes
2022-02-11 11:06:12 -08:00
Taus
d7f30de5b0 Merge pull request #7874 from RasmusWL/set-store-step
Python: Fix setStoreStep to use `SetElementContent`
2022-02-11 12:50:02 +01:00
Taus Brock-Nannestad
be323bafaf Merge remote-tracking branch 'upstream/main' into python-normalise-prefixes 2022-02-10 12:55:49 +01:00
Tamás Vajk
6483a92587 Merge pull request #7865 from github/post-release-prep/codeql-cli-2.8.0
Post-release preparation for codeql-cli-2.8.0
2022-02-09 16:42:38 +01:00
Rasmus Wriedt Larsen
9d5e8d5bd8 Merge pull request #7842 from RasmusWL/consistency-queires
Misc: Streamline `consistency-queries/qlpack.yml`
2022-02-09 13:42:18 +01:00
Tom Hvitved
9440a45015 Merge branch 'main' into post-release-prep/codeql-cli-2.8.0 2022-02-09 09:40:33 +01:00
Taus Brock-Nannestad
54ae744b2c Python: Also update Python 2 file 2022-02-08 22:08:53 +01:00
Chuan-kai Lin
a7f1ee574c Upgrade scripts testing: set initial dbschemes
This commit sets initial dbschemes for cpp, csharp, java, javascript, and
python so that automated testing for upgrade scripts would also cover legacy
upgrades.
2022-02-08 11:11:41 -08:00
Taus Brock-Nannestad
6ea8986daa Python: Normalise string prefixes 2022-02-08 16:48:17 +01:00
Rasmus Wriedt Larsen
a8edd44a3c Python: Update .expected 2022-02-08 11:12:34 +01:00
Rasmus Wriedt Larsen
eb109828c0 Merge pull request #7252 from museljh/feature/cwe-338
Python: CWE-338 insecureRandomness
2022-02-07 19:30:06 +01:00
Rasmus Wriedt Larsen
62702d0ca9 Python: Fix setStoreStep to use SetElementContent 2022-02-07 13:18:36 +01:00
Rasmus Wriedt Larsen
32cd7d6fa7 Add groups to all consistency-queries/qlpack.yml
as discussed in PR review
2022-02-07 11:15:48 +01:00
github-actions[bot]
b4ab86c020 Post-release preparation for codeql-cli-2.8.0 2022-02-06 23:34:07 +00:00
yoff
182c62f5c3 Merge pull request #7838 from tausbn/python-fix-charset-performance-problem
Python: Fix performance issue in `charSet`
2022-02-04 14:18:13 +01:00
Taus
67be20f368 Python: Remove implied inequalities
Also gets rid of `inner_end`, since we're already doing `end - 1 = ...`
in the other fix (and so this is more consistent).
2022-02-04 12:46:06 +00:00
Rasmus Wriedt Larsen
c817ba5718 Python: Add consistency-queries/qlpack.yml
But no queries yet
2022-02-04 12:08:54 +01:00
Rasmus Wriedt Larsen
e9b496ba73 Merge pull request #7831 from RasmusWL/printast-remove-regexp
Python: Remove `RegExpTerm` from PrintAST
2022-02-04 11:38:58 +01:00
Harry Maclean
ab7fd89653 Merge pull request #7663 from github/hmac/api-graph-subclass
Ruby: Add basic subclassing support to API Graphs
2022-02-04 10:19:07 +13:00
Taus
22aa4c9379 Python: Fix performance issue in charSet
Observed on `mozilla/bugbug` on the 2.8.0 CLI branch, we had the
following line in the timing report:
```
FullServerSideRequestForgery.ql-17:regex::RegexString::charSet_dispred#fff#antijoin_rhs ............... 1m13s
```

Inspecting the logs, we see the following join:

```
(644s) Tuple counts for regex::RegexString::charSet_dispred#fff#antijoin_rhs/5@f295d1bk after 1m13s:
1         ~0%         {1} r1 = CONSTANT(unique string)["]"]
2389      ~4%         {3} r2 = JOIN r1 WITH regex::RegexString::nonEscapedCharAt_dispred#fff_201#join_rhs ON FIRST 1 OUTPUT Rhs.1 'arg0', Rhs.2 'arg1', (Rhs.2 'arg1' + 1)
668873    ~0%         {6} r3 = JOIN r2 WITH regex::RegexString::char_set_start_dispred#fff ON FIRST 1 OUTPUT Lhs.0 'arg0', "]", Lhs.1 'arg1', Lhs.2 'arg2', Rhs.1 'arg3', Rhs.2 'arg4'
537501371 ~4%         {7} r4 = JOIN r3 WITH regex::RegexString::nonEscapedCharAt_dispred#fff_021#join_rhs ON FIRST 2 OUTPUT Lhs.0 'arg0', Lhs.2 'arg1', Lhs.3 'arg2', Lhs.4 'arg3', Lhs.5 'arg4', "]", Rhs.2
269085087 ~0%         {7} r5 = SELECT r4 ON In.6 > In.4 'arg4'
89583155  ~3%         {7} r6 = SELECT r5 ON In.6 < In.1 'arg1'
89583155  ~26634%     {5} r7 = SCAN r6 OUTPUT In.0 'arg0', In.1 'arg1', In.2 'arg2', In.3 'arg3', In.4 'arg4'
                    return r7
```
Now, this is problematic not just because of the large intermediary join
but also because of the large number of tuples being materialised at the
end. The culprit in this case turns out to be this bit of `charSet`:
```
not exists(int mid | this.nonEscapedCharAt(mid) = "]" | mid > inner_start and mid < inner_end)
```

Rewriting this to instead look for the minimum index at which a `]`
appears resulted in a much nicer join.

I also fixed up a similar issue surrounding the `\N` unicode escape.
Not that I think this will necessarily be relevant, but the `min`-based
solution is more robust either way.
2022-02-03 20:42:04 +00:00
Chuan-kai Lin
c8bc5cfa75 Merge pull request #7825 from github/cklin/python-downgrade-scripts
Python: adjust downgrade script location and format
2022-02-03 11:40:07 -08:00
Rasmus Wriedt Larsen
8386b36217 Python: Apply suggestions from code review
Co-authored-by: Nick Rolfe <nickrolfe@github.com>
2022-02-03 15:00:04 +01:00
Rasmus Wriedt Larsen
cf68148316 Python: Add change-note 2022-02-03 14:29:02 +01:00
Rasmus Wriedt Larsen
e2de0e61ca Python: Remove RegExpTerm from PrintAST
Since this caused bad performance (as we had to evaluate points-to).

Fixes https://github.com/github/codeql/issues/6964

This approach was motivated by the comment on the issue from @tausbn:

> We discussed this internally in the CodeQL Python team, and have
> agreed that the best approach for now is to disable the printing of
> regex ASTs.

I tried to keep our RegExpTerm logic, but doing the fix below did not
work, and still evaluated RegExpTerm :| I guess we will just have to
revert this PR if we want it back

```diff
   TRegExpTermNode(RegExpTerm term) {
+    none() and
     exists(StrConst str | term.getRootTerm() = getParsedRegExp(str) and shouldPrint(str, _))
   }
```
2022-02-03 14:22:14 +01:00
Erik Krogh Kristensen
e93c46ad31 Merge pull request #7811 from erik-krogh/pyApiIpa
Python: refactor API-graph labels to an IPA type
2022-02-03 12:31:39 +01:00
Tom Hvitved
6bb71f051b Merge pull request #7791 from hvitved/dataflow/inline-local-flow-star
Data flow: Inline `local(Expr|Instruction)?(Flow|Taint)`
2022-02-03 09:02:43 +01:00
Chuan-kai Lin
df91ee6616 Python: adjust downgrade script location and format 2022-02-02 14:23:21 -08:00
Arthur Baars
33b97f3e0c Update synchronized files 2022-02-02 13:30:45 +01:00
CodeQL CI
7bb11b837c Merge pull request #7788 from yoff/python/remove-library-annotation
Approved by tausbn
2022-02-02 03:51:00 -08:00
liangjinhuang
1dd15fa235 style:auto format 2022-02-02 01:30:54 +08:00
liangjinhuang
976e484c57 style:move all source files under src/experimental & feat:modify source regular matching rules 2022-02-02 01:14:51 +08:00
Erik Krogh Kristensen
e06f6529f1 refactor API-graph labels to an IPA type 2022-02-01 17:32:08 +01:00