Commit Graph

1234 Commits

Author SHA1 Message Date
Jeroen Ketema
46b21af3ef Dataflow: Make default field flow branch limit configurable per language 2025-04-24 11:02:49 +02:00
Michael Nebel
617f4729d8 Shared: Match line information on Alert and Sink locations. 2025-04-23 12:35:17 +02:00
Tom Hvitved
ae5ac11387 Shared: Fix join in FileSystem.qll
Before
```
Evaluated relational algebra for predicate FileSystem::Folder::Append<PathResolution::shouldAppend>::appendStep/3#bed54f6d@d2a7eeoa on iteration 2 running pipeline standard with tuple counts:
         120   ~2%    {4} r1 = SCAN `FileSystem::Folder::Append<PathResolution::shouldAppend>::appendStep/3#bed54f6d#prev_delta` OUTPUT In.1, In.0, In.2, In.3

           0   ~0%    {6} r2 = JOIN r1 WITH `_FileSystem::Folder::Append<PathResolution::shouldAppend>::getComponent/2#a6e36a04#join_rhs#1` ON FIRST 1 OUTPUT Lhs.0, Rhs.1, Lhs.1, Lhs.2, Lhs.3, _
                      {6}    | REWRITE WITH Tmp.5 := 1, Out.5 := (InOut.1 - Tmp.5), TEST Out.5 = InOut.3
           0   ~0%    {4}    | SCAN OUTPUT In.2, In.0, In.1, In.4

          91   ~3%    {6} r3 = JOIN r1 WITH `_FileSystem::Folder::Append<PathResolution::shouldAppend>::getComponent/2#a6e36a04#join_rhs` ON FIRST 1 OUTPUT Lhs.0, Rhs.1, Lhs.1, Lhs.2, Lhs.3, _
                      {6}    | REWRITE WITH Tmp.5 := 1, Out.5 := (InOut.1 - Tmp.5), TEST Out.5 = InOut.3
          81   ~0%    {4}    | SCAN OUTPUT In.4, In.0, In.1, In.2
          81   ~3%    {4}    | JOIN WITH containerparent_10#join_rhs ON FIRST 1 OUTPUT Lhs.3, Lhs.1, Lhs.2, Rhs.1

         269   ~0%    {7} r4 = JOIN r1 WITH `_FileSystem::Folder::Append<PathResolution::shouldAppend>::getComponent/2#a6e36a04#join_rhs#2` ON FIRST 1 OUTPUT Lhs.0, Rhs.1, Rhs.2, Lhs.1, Lhs.2, Lhs.3, _
                      {7}    | REWRITE WITH Tmp.6 := 1, Out.6 := (InOut.1 - Tmp.6), TEST Out.6 = InOut.4
          39   ~1%    {5}    | SCAN OUTPUT In.5, In.0, In.1, In.2, In.3
        1295   ~0%    {6}    | JOIN WITH containerparent ON FIRST 1 OUTPUT Rhs.1, Lhs.3, _, Lhs.1, Lhs.2, Lhs.4
        1295   ~0%    {6}    | REWRITE WITH Out.2 := 1
          34   ~1%    {4}    | JOIN WITH `cached_FileSystem::Container.splitAbsolutePath/2#dispred#dc97b0cc` ON FIRST 3 OUTPUT Lhs.5, Lhs.3, Lhs.4, Lhs.0

         115   ~0%    {4} r5 = r2 UNION r3 UNION r4
         115   ~0%    {4}    | AND NOT `FileSystem::Folder::Append<PathResolution::shouldAppend>::appendStep/3#bed54f6d#prev`(FIRST 4)
                      return r5
```

After
```
Evaluated relational algebra for predicate FileSystem::Folder::Append<PathResolution::shouldAppend>::appendStep/3#bed54f6d@4fb6e6v7 on iteration 2 running pipeline standard with tuple counts:
        120   ~0%    {4} r1 = SCAN `FileSystem::Folder::Append<PathResolution::shouldAppend>::appendStep/3#bed54f6d#prev_delta` OUTPUT In.1, In.0, In.2, In.3

          0   ~0%    {6} r2 = JOIN r1 WITH `_FileSystem::Folder::Append<PathResolution::shouldAppend>::getComponent/2#a6e36a04#join_rhs#1` ON FIRST 1 OUTPUT Lhs.0, Rhs.1, Lhs.1, Lhs.2, Lhs.3, _
                     {6}    | REWRITE WITH Tmp.5 := 1, Out.5 := (InOut.1 - Tmp.5), TEST Out.5 = InOut.3
          0   ~0%    {4}    | SCAN OUTPUT In.2, In.0, In.1, In.4

         91   ~0%    {6} r3 = JOIN r1 WITH `_FileSystem::Folder::Append<PathResolution::shouldAppend>::getComponent/2#a6e36a04#join_rhs` ON FIRST 1 OUTPUT Lhs.0, Rhs.1, Lhs.1, Lhs.2, Lhs.3, _
                     {6}    | REWRITE WITH Tmp.5 := 1, Out.5 := (InOut.1 - Tmp.5), TEST Out.5 = InOut.3
         81   ~0%    {4}    | SCAN OUTPUT In.4, In.0, In.1, In.2
         81   ~5%    {4}    | JOIN WITH containerparent_10#join_rhs ON FIRST 1 OUTPUT Lhs.3, Lhs.1, Lhs.2, Rhs.1

        269   ~0%    {7} r4 = JOIN r1 WITH `_FileSystem::Folder::Append<PathResolution::shouldAppend>::getComponent/2#a6e36a04#join_rhs#2` ON FIRST 1 OUTPUT Lhs.0, Rhs.1, Rhs.2, Lhs.1, Lhs.2, Lhs.3, _
                     {7}    | REWRITE WITH Tmp.6 := 1, Out.6 := (InOut.1 - Tmp.6), TEST Out.6 = InOut.4
         39   ~3%    {5}    | SCAN OUTPUT In.5, In.2, In.0, In.1, In.3
         34   ~0%    {4}    | JOIN WITH `FileSystem::Folder::Append<PathResolution::shouldAppend>::getAChildContainer/2#2e91feca` ON FIRST 2 OUTPUT Lhs.4, Lhs.2, Lhs.3, Rhs.2

        115   ~2%    {4} r5 = r2 UNION r3 UNION r4
        115   ~2%    {4}    | AND NOT `FileSystem::Folder::Append<PathResolution::shouldAppend>::appendStep/3#bed54f6d#prev`(FIRST 4)
                     return r5
```
2025-04-22 12:16:59 +02:00
github-actions[bot]
d78736b1bf Post-release preparation for codeql-cli-2.21.1 2025-04-15 16:33:15 +00:00
github-actions[bot]
b961c5961d Release preparation for version 2.21.1 2025-04-14 09:53:06 +00:00
Mathias Vorreiter Pedersen
11aef7019e Merge pull request #19273 from MathiasVP/prepare-shared-mad-generation-for-cpp
Shared: Prepare model generation for C++ adoption
2025-04-11 07:22:56 -07:00
Mathias Vorreiter Pedersen
ea3bb8cf0c Shared: Provide a hook to MaD generation to modify the 'ReturnValue' string. 2025-04-10 14:02:31 +01:00
Mathias Vorreiter Pedersen
c484945f39 Shared: Move 'getEnclosingCallable' and 'getAsExprEnclosingCallable' out of the class signature. 2025-04-10 14:00:11 +01:00
Mathias Vorreiter Pedersen
732fcbf1c9 Shared: Move 'asParameter' out of the class signature. 2025-04-10 13:58:59 +01:00
Mathias Vorreiter Pedersen
a1dc87496a Shared: Replace a 'count' with a 'strictcount' to prevent a CP when testing on C++. 2025-04-10 13:56:38 +01:00
Tom Hvitved
1ba06ab3bf Merge pull request #19216 from hvitved/rust/path-resolution-path-attr
Rust: Handle path attributes in path resolution
2025-04-09 18:57:56 +02:00
Tom Hvitved
52401aaa73 Address review comments 2025-04-09 17:19:25 +02:00
Geoffrey White
9c1567375d Shared: Implement getInconsistencyCounts for SSA. 2025-04-07 16:20:42 +01:00
Tom Hvitved
13f4a6afa6 Rust: Handle path attributes in path resolution 2025-04-07 15:24:17 +02:00
Geoffrey White
2c2506c4f8 Rust: Add Rust SSA inconsistency infrastructure. 2025-04-07 12:16:45 +01:00
Anders Schack-Mulligen
ea8cd6f758 Merge pull request #19160 from aschackmull/ssa/deprecate-definitionext
Ssa: Deprecate the public DefinitionExt and PhiReadNode
2025-04-04 09:23:08 +02:00
Anders Schack-Mulligen
ce19972aef SSA: Reinstate consistency check. 2025-04-03 16:16:20 +02:00
Asger F
c49ffa01ee JS: Enable post-processed inline expectations for query predicates 2025-04-03 13:24:17 +02:00
Anders Schack-Mulligen
6ac4cb71cb SSA: Add change note. 2025-04-03 09:57:48 +02:00
Anders Schack-Mulligen
34554fd000 SSA: Deprecate the public DefinitionExt. 2025-04-03 09:47:50 +02:00
Tom Hvitved
a01b211f79 Merge pull request #19185 from hvitved/rust/ssa-capture-psudeo-read-dominate
Rust: Fix `capturedCallRead`
2025-04-02 19:16:55 +02:00
Taus
f461763938 Merge pull request #19186 from github/tausbn/actions-fix-gettargetpath-performance
Actions: Fix bad performance in `getTargetPath`
2025-04-02 12:53:56 +02:00
Anders Schack-Mulligen
47b1c3d3ce Merge pull request #19154 from aschackmull/ssa/variablecapture
Ssa: Replace phi-read references in VariableCapture with default use-use flow
2025-04-02 10:16:17 +02:00
Taus
8afdf25e9a Actions: Mark helper predicate as noinline. 2025-04-01 14:50:00 +00:00
Taus
c9fc7882e6 Actions: Fix bad performance in getTargetPath
Seen on `github/codeql`, some queries had very poor performance:
```
 [2/24 eval 36m4s] Evaluation done; writing results to
codeql/actions-queries/Security/CWE-312/ExcessiveSecretsExposure.bqrs
```

Investigating further lead to the following worrying sequence of joins
(after I ran out of patience and cancelled the query):
```
[2025-04-01 12:31:03] Tuple counts for
Yaml::YamlInclude.getTargetPath/0#dispred#32565107#fb#reorder_1_0/2@i6#9f4b2jw1
after 8m40s:
...
    559418    ~33%        {1} r5 = SCAN
`Yaml::YamlNode.getLocation/0#dispred#24555c57#prev_delta` OUTPUT In.1
...
    909345525 ~821%       {3} r7 = JOIN r5 WITH
`Yaml::YamlNode.getLocation/0#dispred#24555c57#prev` CARTESIAN PRODUCT
OUTPUT Rhs.1, Lhs.0 'result', Rhs.0
    909342139 ~779%       {3}    | JOIN WITH
`Locations::Location.getFile/0#dispred#dcf38c8d#prev` ON FIRST 1 OUTPUT
Rhs.1, Lhs.1 'result', Lhs.2
    909338753 ~794%       {3}    | JOIN WITH containerparent_10#join_rhs
ON FIRST 1 OUTPUT Rhs.1, Lhs.1 'result', Lhs.2
    909335367 ~824%       {3}    | JOIN WITH
`FileSystem::Container.getAbsolutePath/0#dispred#d234e6fa` ON FIRST 1
OUTPUT Lhs.2, Lhs.1 'result', Rhs.1
    883246724 ~812%       {3}    | JOIN WITH
`Yaml::YamlNode.getDocument/0#dispred#ee1eb3bf#bf_10#join_rhs` ON FIRST
1 OUTPUT Rhs.1 'this', Lhs.1 'result', Lhs.2
    760047185 ~838%       {5}    | JOIN WITH yaml_scalars ON FIRST 1
OUTPUT Lhs.1 'result', Lhs.0 'this', Rhs.2, _, Lhs.2
    0         ~0%         {4}    | REWRITE WITH Tmp.3 := "/", Out.3 :=
(In.4 ++ Tmp.3 ++ InOut.2), TEST Out.3 = InOut.0 KEEPING 4
                        {4}    | REWRITE WITH NOT [TEST InOut.2
startsWith "/"]
...
```

The culprit turned out to be the following method on class `YamlInclude`
```ql
private string getTargetPath() {
    exists(string path | path = this.getValue() |
    if path.matches("/%")
    then result = path
    else
        result =

this.getDocument().getLocation().getFile().getParentContainer().getAbsolutePath()
+ "/" +
            path
    )
}
```

Basically, in the `else` branch, the evaluator was producing all
possible values of `result` before filtering out the ones where the
`path` component started with a forward slash.

To fix this, I opted to factor out the logic into two helper predicates,
each accounting for whether `this.getValue()` does or does not start
with a `/`. With this, evaluating the original query from a clean cache
takes roughly 3.3s.
2025-04-01 13:59:19 +00:00
Tom Hvitved
9dea9f5046 SSA: Make phiWithoutTwoPriorRefs consistency check more informative 2025-04-01 15:07:46 +02:00
github-actions[bot]
10205cb990 Post-release preparation for codeql-cli-2.21.0 2025-04-01 11:30:43 +00:00
github-actions[bot]
84f6564cc0 Release preparation for version 2.21.0 2025-03-31 17:35:15 +00:00
Anders Schack-Mulligen
b4daba30a5 SSA: Remove dead code. 2025-03-31 10:49:10 +02:00
Anders Schack-Mulligen
70e53c2f8b SSA: Push includeWriteDefsInFlowStep constraint into newtype. 2025-03-31 10:49:10 +02:00
Anders Schack-Mulligen
ca6444ce98 VariableCapture: Replace phi-read reference with SSA data flow integration module. 2025-03-31 10:49:09 +02:00
Anders Schack-Mulligen
a8b19d2b21 Merge pull request #19147 from aschackmull/ssa/writedef-source-refactor
Ssa: Refactor data flow integration to make the input signature simpler
2025-03-31 10:07:09 +02:00
Paolo Tranquilli
96f7dc7eb4 Merge pull request #19083 from github/redsun82/rust-analyzer-update
Rust: update rust-analyzer from `0.0.266` to `0.0.270`
2025-03-28 15:04:09 +01:00
Simon Friis Vindum
884c828452 Merge pull request #19102 from paldepind/rust-ti-refactor
Shared, type inference: Add inference for type parameters with constraints (base type mentions)
2025-03-28 13:58:16 +01:00
Simon Friis Vindum
989c14485d Rust: Minor doc tweaks based on PR comments 2025-03-28 13:39:42 +01:00
Paolo Tranquilli
76b1349068 Merge branch 'main' into redsun82/rust-analyzer-update 2025-03-28 13:36:48 +01:00
Anders Schack-Mulligen
0d1ac7789b SSA/Ruby: Address review comments. 2025-03-28 13:27:56 +01:00
Simon Friis Vindum
623bc232bf Rust: Address PR comments 2025-03-28 13:04:55 +01:00
Anders Schack-Mulligen
5a986f5327 SSA: Remove empty predicates and dead code. 2025-03-28 12:00:38 +01:00
Anders Schack-Mulligen
1ded4df3fd SSA: Add an alternative to ssaDefAssigns/ssaDefInitializesParam. 2025-03-28 11:57:26 +01:00
Anders Schack-Mulligen
0c74f21107 Merge pull request #19044 from aschackmull/ssa/useuse-trim
Ssa: Trim the use-use relation to skip irrelevant nodes
2025-03-28 11:55:34 +01:00
Anders Schack-Mulligen
c6cee489e4 SSA: Address review comments. 2025-03-28 10:53:03 +01:00
Michael Nebel
8763d18c91 C#: Correct printing of out and ref notes in the model generator. 2025-03-26 15:06:26 +01:00
Paolo Tranquilli
7cfed3c4a7 Merge branch 'main' into redsun82/rust-analyzer-update 2025-03-25 18:29:39 +01:00
Simon Friis Vindum
520e27ccfa Shared: Fix typos in qldoc 2025-03-25 15:49:03 +01:00
Simon Friis Vindum
54e7bb7f1a Rust: Fix a bad join by adding bindingset to resolveTypeMentionRoot 2025-03-25 14:26:24 +01:00
Anders Schack-Mulligen
d5d0274ce7 Java/SSA: Keep proper distinction between cached stages. 2025-03-25 13:43:55 +01:00
Anders Schack-Mulligen
4d04391b70 C++: Keep all phi input back edges. 2025-03-25 13:21:13 +01:00
Anders Schack-Mulligen
0162b84d20 SSA: Fix a poor join-order and avoid SSA recomputation. 2025-03-25 12:31:03 +01:00
Anders Schack-Mulligen
36532bc58c SSA: Skip identity steps. 2025-03-25 12:31:02 +01:00