Commit Graph

71517 Commits

Author SHA1 Message Date
Michael Nebel
16fe7a2fbe Merge pull request #17815 from michaelnebel/csharp/logforgingreplacelineending
C#: Add sanitizer to `cs/log-forging`.
2024-10-22 15:41:17 +02:00
Chris Smowton
2312f9d8df Merge pull request #17811 from github/smowton/admin/package-maven-plugin-change-note
Add change-note for Java buildless packaging its required Maven plugin
2024-10-22 14:23:01 +01:00
Cornelius Riemenschneider
cdffa09a7b Merge pull request #17821 from github/criemen/win-make-bazel
Go: Fix makefile to use bash to look up bazel path.
2024-10-22 12:50:10 +02:00
Cornelius Riemenschneider
1ccadbc3f6 Add comment. 2024-10-22 11:26:02 +02:00
Cornelius Riemenschneider
ad1ef65539 Go: Fix makefile to use bash to look up bazel path.
On Windows, make's path resolution algorithm is incorrect.
It picks up a bazel.exe in PATH that's _after_ a bazel binary.
In particular, on actions, the non-exe binary is a bazelisk
instance, whereas bazel.exe is a bazel (at the current time 7.3.2)
installation.
This means we pick up the wrong bazel version, and
if the differences between the bazel we want and that we actually
get are too big, the build fails.
2024-10-22 10:51:10 +02:00
Tom Hvitved
60c205ff37 Merge pull request #17818 from hvitved/rust/summary-stats-perf
Rust: Speedup `SummaryStats.ql`
2024-10-22 10:15:56 +02:00
Owen Mansel-Chan
23a1ea7191 Merge pull request #17717 from owen-mc/go/update-incorrect-integer-conversion-qhelp
Go: Update `go/incorrect-integer-conversion` qhelp to explain possible source of FPs
2024-10-21 21:53:32 +01:00
Jeroen Ketema
9ef1a9c3f9 Merge pull request #17225 from geoffw0/aliasperf2
C++: Improve AliasedSSA performance
2024-10-21 20:20:24 +02:00
Taus
9803bbdc4b Python: Update class parser test 2024-10-21 15:35:48 +00:00
Taus
1cd04c96c7 Python: Fix bug in handling of **kwargs in class bases
This caused a dataset check error on the `python/cpython` database, as
we had a `DictUnpacking` node whose parent was not a `dict_item_list`,
but rather an `expr_list`.

Investigating a bit further revealed that this was because in a
construction like

```python
class C[T](base, foo=bar, **kwargs): ...
```
we were mistakenly adding `**kwargs` to the same list as `base` (which
is just a list of expressions), rather than the same list as `foo=bar`
(which is a list of dictionary items)

The ultimate cause of this was the use of `! name` in `python.tsg` to
distinguish between bases and keyword arguments (only the latter of
which have the `name` field). Because `dictionary_splat` doesn't have a
`name` field either, these were mistakenly put in the wrong list,
leading to the error.

Also, because our previous test of `class` statements did not include a
`**kwargs` construction, we were not checking that the new parser
behaved correctly in this case. For the most part this was not a
problem, but on files that use syntax not supported by the old parser
(like type parameters on classes), this became an issue. This is also
why we did not see this error previously.

To fix this, we added `! value` (which is a field present on
`dictionary_splat` nodes) as a secondary filter, and added a third
stanza to handle `dictionary_splat` nodes.
2024-10-21 15:35:47 +00:00
Taus
ae4a4bb881 Python: Flip test expectation
This test should now validate that we no longer have dataset check
errors even when there are unencodable characters.
2024-10-21 15:32:23 +00:00
Taus
cc39ae57dc Python: Fix dataset check error for string encoding
Here's an example of one of these errors:
```
INVALID_KEY predicate py_cobjectnames(@py_cobject obj, string name)

The key set {obj} does not functionally determine all fields. Here is a
pair of tuples that agree on the key set but differ at index 1: Tuple 1
in row 63874: (72088,"u'<X>'") Tuple 2 in row 63875: (72088,"u'<?>'")
```
(Here, the substring `X` should really be the Unicode character U+FFFD,
but for some reason I'm not allowed to put that in this commit message.)

Inside the extractor, we assign IDs based on the string type (bytestring
or Unicode) and a hash of the UTF-8 encoded content of the string. In
this case, however, certain _different_ strings were receiving the same
hash, due to replacement characters in the encoding process.

In particular, we were converting unencodable characters to question
marks in one place, and to U+FFFD in another place. This caused a
discrepancy that lead to the dataset check error.

To fix this, we put in a custom error handler that always puts the
U+FFFD character in place of unencodable characters. With this, the
strings now agree, and hence there is no clash.
2024-10-21 15:31:16 +00:00
Porcupiney Hairs
c7610b3539 Include change-note 2024-10-21 20:14:58 +05:30
Porcupiney Hairs
c93f0ed851 Include change-note 2024-10-21 20:12:46 +05:30
Jeroen Ketema
e36a1c717d Merge branch 'main' into aliasperf2 2024-10-21 16:40:01 +02:00
Porcupiney Hairs
c74f6f587f Merge branch 'main' into pyloadSsl 2024-10-21 20:09:05 +05:30
Porcupiney Hairs
f6369a6ed7 Include changes from review 2024-10-21 20:01:44 +05:30
Arthur Baars
490f81437b Merge pull request #17804 from github/aibaars/local-defs-1
Rust: add ide-contextual-queries/localDefinitions.ql
2024-10-21 15:30:44 +02:00
Tom Hvitved
f72af4f1f3 Rust: Use Callable to define CfgScope 2024-10-21 15:22:04 +02:00
Tom Hvitved
c4c936d6fa Rust: Speedup SummaryStats.ql 2024-10-21 15:21:13 +02:00
Michael Nebel
dec2c61e5d Java: Update LdapInjection expected test output. 2024-10-21 15:19:46 +02:00
Michael Nebel
d59df1f938 Java: Re-generate JDK 17 models. 2024-10-21 15:19:45 +02:00
Michael Nebel
23d285c698 Java: Update model generator expected output. 2024-10-21 15:19:44 +02:00
Michael Nebel
786d04e939 Java: Add the clone method to the model generation exclusions. 2024-10-21 15:19:43 +02:00
Michael Nebel
7919dcfb12 Java: Add modelgenerator clone example. 2024-10-21 15:19:42 +02:00
Michael Nebel
e2ada2536b Java: Update java.net expected output. 2024-10-21 15:19:41 +02:00
Michael Nebel
97f0037a7b Java: Manually model InetSocketAddress as the model generator doesn't correctly taint the hostname. 2024-10-21 15:19:40 +02:00
Michael Nebel
0a931aa69f Java: Add change note. 2024-10-21 15:19:38 +02:00
Michael Nebel
9a44eec04c Java: Add manual models for FileReader (they would also have disappeared if models were re-generated without using mixed mode). 2024-10-21 15:19:37 +02:00
Michael Nebel
b356c3cd48 Java: Manually model ZipFile (due to CWE-522 compression bombs test failure). 2024-10-21 15:19:36 +02:00
Michael Nebel
f537e04532 Java: Update LdapInjection expected test output. 2024-10-21 15:19:35 +02:00
Michael Nebel
f7b38a8955 Java: Add some less precise models for BasicAttributes to get the models to work with search sink and re-generate SDK models. 2024-10-21 15:19:34 +02:00
Michael Nebel
e94cacd449 Java: Update test expected output where the query results are not affected. 2024-10-21 15:19:33 +02:00
Michael Nebel
24d1e9927b Java: Update expected test output for the model editor tests. 2024-10-21 15:19:32 +02:00
Michael Nebel
ea14547643 Java: Update TopJdkApisTest expected output. 2024-10-21 15:19:31 +02:00
Michael Nebel
cbd9cc6dae Java: Update request forgery expected output. 2024-10-21 15:19:30 +02:00
Michael Nebel
3b6f39931b Java: Re-add generated (mixed) summaries and neutrals for the Java SDK 17. 2024-10-21 15:19:28 +02:00
Michael Nebel
f50734f0ee Java: Delete all generated Java JDK models. 2024-10-21 15:19:27 +02:00
Tom Hvitved
e9adbf231f Merge pull request #17816 from hvitved/rust/expr-trees-module
Rust: Move all expression CFG trees inside an `ExprTrees` module
2024-10-21 15:12:51 +02:00
Tom Hvitved
d2623cf4c3 Merge pull request #17814 from hvitved/rust/fix-bad-join
Rust: Fix bad join
2024-10-21 15:01:31 +02:00
Tom Hvitved
a6a68ef8be Apply suggestions from code review
Co-authored-by: Simon Friis Vindum <paldepind@github.com>
2024-10-21 14:43:22 +02:00
Taus
d01593e571 Python: Add test for string encoding dataset check
Note that this test checks that the current setup creates dataset check
violations. A later commit will fix this (and flip the negation in the
test).
2024-10-21 12:08:46 +00:00
Tom Hvitved
1f5e02f539 Rust: Move all expression CFG trees inside an ExprTrees module 2024-10-21 13:41:11 +02:00
Simon Friis Vindum
85e59c9920 Merge branch 'main' into rust-saa-additions 2024-10-21 13:30:02 +02:00
Michael Nebel
1217c55c36 C#: Add change note. 2024-10-21 12:08:03 +02:00
Michael Nebel
191658f637 C#: Update expected test output. 2024-10-21 12:04:31 +02:00
Michael Nebel
b2b1a3ea65 C#: Consider string.ReplaceLineEndings(string) as a sanitizer for log forging. 2024-10-21 12:03:59 +02:00
Michael Nebel
0b8e83dc87 C#: Add log forging false positive example using ReplaceLineEndings. 2024-10-21 11:55:09 +02:00
Cornelius Riemenschneider
ce53964edf Merge pull request #17812 from github/redsun82/rust-move-integration-tests
Rust: move integration tests to where other languages have them
2024-10-21 11:41:16 +02:00
Tom Hvitved
7e82595cae Rust: Fix bad join
Before
```
Evaluated relational algebra for predicate MatchExprImpl::Impl::MatchExpr.getLastArm/0#dispred#24e5f4cf@9cf607tl with tuple counts:
            660677  ~0%    {4} r1 = SCAN `MatchExprImpl::Impl::MatchExpr.getArm/1#dispred#817de8a3` OUTPUT _, In.0, In.2, In.1
            660677  ~0%    {3}    | REWRITE WITH Tmp.0 := 1, Out.0 := (Tmp.0 + In.3) KEEPING 3
        5342095756  ~0%    {3}    | JOIN WITH `MatchArmList::Generated::MatchArmList.getNumberOfArms/0#dispred#9ad72f08_10#join_rhs` ON FIRST 1 OUTPUT Lhs.1, Rhs.1, Lhs.2
             96597  ~3%    {2}    | JOIN WITH `MatchExpr::Generated::MatchExpr.getMatchArmList/0#dispred#11f1a73e` ON FIRST 2 OUTPUT Lhs.0, Lhs.2
                           return r1
```

After
```
Evaluated relational algebra for predicate MatchExprImpl::Impl::MatchExpr.getLastArm/0#dispred#24e5f4cf@9d7a92pu with tuple counts:
        660677   ~0%    {5} r1 = JOIN `MatchExprImpl::Impl::MatchExpr.getArm/1#344daffc` WITH `MatchExprImpl::Impl::MatchExpr.getNumberOfArms/0#ab0d8732` ON FIRST 1 OUTPUT Lhs.0, Lhs.1, Lhs.2, _, Rhs.1
                        {4}    | REWRITE WITH Tmp.3 := 1, Out.3 := (In.4 - Tmp.3), TEST Out.3 = InOut.1 KEEPING 4
         96597   ~3%    {2}    | SCAN OUTPUT In.0, In.2
                        return r1
```
2024-10-21 11:40:13 +02:00