Commit Graph

1961 Commits

Author SHA1 Message Date
Arthur Baars
8e7eedc172 Update codegen/grammar 2024-12-17 14:05:50 +01:00
Arthur Baars
3928efe05f Rust: update rust.ungram 2024-12-17 14:05:12 +01:00
Tom Hvitved
8efd870192 Merge pull request #18292 from hvitved/rust/never-skip-lhs
Rust: Never skip assignment LHS in data flow
2024-12-17 13:18:17 +01:00
Tom Hvitved
d8c05b5388 Merge pull request #18290 from hvitved/rust/perf-fixes
Rust: Fix two bad joins
2024-12-17 13:18:05 +01:00
Simon Friis Vindum
ee87d4c948 Merge branch 'main' into rust-data-flow-models 2024-12-17 13:12:32 +01:00
Simon Friis Vindum
402d4e11c4 Rust: Re-add inline expectations query tags 2024-12-16 16:36:30 +01:00
Tom Hvitved
ddd05b5d1b Rust: Never skip match scrutinee/patterns in data flow 2024-12-16 15:12:16 +01:00
Tom Hvitved
9f2b436d35 Rust: Never skip assignment LHS in data flow 2024-12-16 15:12:15 +01:00
Paolo Tranquilli
4975e7b739 Merge branch 'main' into redsun82/extract-self-param-ref 2024-12-16 15:06:16 +01:00
Tom Hvitved
5ed03e266a Rust: Fix semantic merge conflicts 2024-12-16 14:47:13 +01:00
Paolo Tranquilli
4c4a8d7619 Rust: extract isRef for SelfParam 2024-12-16 14:24:56 +01:00
Michael Nebel
aaf0cd5dee Merge pull request #17968 from michaelnebel/java/movetestutils
Move test utilities to the query pack.
2024-12-16 13:41:30 +01:00
Simon Friis Vindum
cad4f39aee Rust: Database name capitalization 2024-12-16 13:15:42 +01:00
Simon Friis Vindum
defbbb2a24 Rust: Add additional models for stdlib and sqlx 2024-12-16 11:46:57 +01:00
Simon Friis Vindum
aab3428bc7 Rust: Model address-of and dereference as stores and loads 2024-12-16 11:31:15 +01:00
Simon Friis Vindum
df0375103c Rust: Add data flow tests 2024-12-16 11:09:22 +01:00
Tom Hvitved
aabcc108dd Rust: Fix bad join
```
[2024-12-16 10:10:36] (247s) Tuple counts for DataFlowImpl::RustDataFlow::storeStep/3#98e80e57/3@0618fdm6 after 3m8s:
                      33711       ~0%        {3} r1 = SCAN `DataFlowImpl::VariableCapture::storeStep/3#cb0fdcf6` OUTPUT In.1, In.0 'node1', In.2 'node2'
                      33711       ~6%        {3}    | JOIN WITH DataFlowImpl::TSingletonContentSet#9b15eaba ON FIRST 1 OUTPUT Lhs.1 'node1', Rhs.1 'cs', Lhs.2 'node2'

                      0           ~0%        {3} r2 = JOIN `FlowSummaryImpl::Private::Steps::summaryStoreStep/3#2c853d0d` WITH DataFlowImpl::TFlowSummaryNode#2b28ecb7 ON FIRST 1 OUTPUT Lhs.2, Lhs.1 'cs', Rhs.1 'node1'
                      0           ~0%        {3}    | JOIN WITH DataFlowImpl::TFlowSummaryNode#2b28ecb7 ON FIRST 1 OUTPUT Lhs.2 'node1', Lhs.1 'cs', Rhs.1 'node1'

                      1554        ~0%        {3} r3 = JOIN _DataFlowImpl::TExprNode#83a34c2e__DataFlowImpl::TArrayElement#b9fb9b7b_DataFlowImpl::TSingletonCont__#shared WITH `CfgNodes::ArrayRepeatExprCfgNode.getRepeatOperand/0#dispred#b264e402_10#join_rhs` ON FIRST 1 OUTPUT Rhs.1, Lhs.1 'cs', Lhs.2 'node1'
                      1554        ~0%        {3}    | JOIN WITH DataFlowImpl::TExprNode#83a34c2e ON FIRST 1 OUTPUT Lhs.2 'node1', Lhs.1 'cs', Rhs.1 'node2'

                      870         ~2%        {3} r4 = SCAN `DataFlowImpl::RustDataFlow::tupleAssignment/3#bf3c8690` OUTPUT In.2, In.0 'node1', In.1
                      870         ~0%        {3}    | JOIN WITH DataFlowImpl::TSingletonContentSet#9b15eaba ON FIRST 1 OUTPUT Lhs.2, Rhs.1 'cs', Lhs.1 'node1'
                      870         ~0%        {3}    | JOIN WITH `DataFlowImpl::Node::PostUpdateNode.getPreUpdateNode/0#dispred#53daedc2_10#join_rhs` ON FIRST 1 OUTPUT Lhs.2 'node1', Lhs.1 'cs', Rhs.1 'node2'

                      40037       ~4%        {3} r5 = JOIN _DataFlowImpl::TExprNode#83a34c2e__DataFlowImpl::TArrayElement#b9fb9b7b_DataFlowImpl::TSingletonCont__#shared WITH `CfgNodes::ArrayExprCfgNode.getAnExpr/0#dispred#9d00a6f1_10#join_rhs` ON FIRST 1 OUTPUT Rhs.1, Lhs.1 'cs', Lhs.2 'node1'
                      36929       ~4%        {3}    | JOIN WITH CfgNodes::ArrayListExprCfgNode#07eee614 ON FIRST 1 OUTPUT Lhs.0, Lhs.1 'cs', Lhs.2 'node1'
                      36929       ~0%        {3}    | JOIN WITH DataFlowImpl::TExprNode#83a34c2e ON FIRST 1 OUTPUT Lhs.2 'node1', Lhs.1 'cs', Rhs.1 'node2'

                      14          ~0%        {2} r6 = JOIN DataFlowImpl::TTuplePositionContent#f1d90606_10#join_rhs WITH DataFlowImpl::TSingletonContentSet#9b15eaba ON FIRST 1 OUTPUT Lhs.1, Rhs.1 'cs'
                      47949       ~0%        {3}    | JOIN WITH `CfgNodes::TupleExprCfgNode.getField/1#dispred#9f7c9c63_102#join_rhs` ON FIRST 1 OUTPUT Rhs.1, Lhs.1 'cs', Rhs.2
                      47949       ~0%        {3}    | JOIN WITH DataFlowImpl::TExprNode#83a34c2e ON FIRST 1 OUTPUT Lhs.2, Lhs.1 'cs', Rhs.1 'node2'
                      47949       ~2%        {3}    | JOIN WITH DataFlowImpl::TExprNode#83a34c2e ON FIRST 1 OUTPUT Rhs.1 'node2', Lhs.1 'cs', Lhs.2 'node2'

                      59801       ~0%        {3} r7 = JOIN _DataFlowImpl::TSingletonContentSet#9b15eaba_DataFlowImpl::TVariantPositionContent#ca6baca0_201#join__#shared WITH `DataFlowImpl::RustDataFlow::tupleVariantConstruction/2#10613c55_10#join_rhs` ON FIRST 1 OUTPUT Rhs.1, Lhs.1 'cs', Lhs.2
                      45509       ~0%        {3}    | JOIN WITH CfgNodes::CallExprCfgNode#9c2a4686_10#join_rhs ON FIRST 1 OUTPUT Rhs.1, Lhs.1 'cs', Lhs.2
                      45509       ~2%        {4}    | JOIN WITH DataFlowImpl::TExprNode#83a34c2e ON FIRST 1 OUTPUT Lhs.0, Lhs.2, Lhs.1 'cs', Rhs.1 'node2'
                      45509       ~0%        {3}    | JOIN WITH `CfgNodes::CallExprBaseCfgNode.getArgument/1#dispred#9ebb27c0` ON FIRST 2 OUTPUT Rhs.2, Lhs.2 'cs', Lhs.3 'node2'
                      45509       ~0%        {3}    | JOIN WITH DataFlowImpl::TExprNode#83a34c2e ON FIRST 1 OUTPUT Rhs.1 'node2', Lhs.1 'cs', Lhs.2 'node2'

                      75147       ~1%        {3} r8 = JOIN _DataFlowImpl::TSingletonContentSet#9b15eaba_DataFlowImpl::TStructFieldContent#1d6d7b05_201#join_rhs#shared WITH `DataFlowImpl::RustDataFlow::structConstruction/2#a9656db0_10#join_rhs` ON FIRST 1 OUTPUT Rhs.1, Lhs.1 'cs', Lhs.2
                      59186       ~3%        {3}    | JOIN WITH `CfgNodes::RecordExprCfgNode.getRecordExpr/0#dispred#659ad1af_10#join_rhs` ON FIRST 1 OUTPUT Rhs.1, Lhs.1 'cs', Lhs.2

                      5641        ~2%        {3} r9 = JOIN _DataFlowImpl::TSingletonContentSet#9b15eaba_DataFlowImpl::TVariantFieldContent#4e05bcf1_201#join_rh__#shared WITH `DataFlowImpl::RustDataFlow::recordVariantConstruction/2#34b016f6_10#join_rhs` ON FIRST 1 OUTPUT Rhs.1, Lhs.1 'cs', Lhs.2
                      5268        ~0%        {3}    | JOIN WITH `CfgNodes::RecordExprCfgNode.getRecordExpr/0#dispred#659ad1af_10#join_rhs` ON FIRST 1 OUTPUT Rhs.1, Lhs.1 'cs', Lhs.2

                      64454       ~1%        {3} r10 = r8 UNION r9
                      64454       ~0%        {4}    | JOIN WITH DataFlowImpl::TExprNode#83a34c2e ON FIRST 1 OUTPUT Lhs.0, Lhs.2, Lhs.1 'cs', Rhs.1 'node2'
                      25923       ~0%        {3}    | JOIN WITH `CfgNodes::RecordExprCfgNode.getFieldExpr/1#d72dca6e` ON FIRST 2 OUTPUT Rhs.2, Lhs.2 'cs', Lhs.3 'node2'
                      25923       ~0%        {3}    | JOIN WITH DataFlowImpl::TExprNode#83a34c2e ON FIRST 1 OUTPUT Rhs.1 'node2', Lhs.1 'cs', Lhs.2 'node2'

                      67759289500 ~251%      {4} r11 = JOIN DataFlowImpl::TSingletonContentSet#9b15eaba WITH DataFlowImpl::TExprNode#83a34c2e CARTESIAN PRODUCT OUTPUT Lhs.0, Lhs.1 'cs', Rhs.0, Rhs.1 'node2'
                      3568000     ~1488%     {3}    | JOIN WITH DataFlowImpl::TArrayElement#b9fb9b7b ON FIRST 1 OUTPUT Lhs.3, Lhs.1 'cs', Lhs.2
                      1223000     ~1291%     {3}    | JOIN WITH `DataFlowImpl::Node::PostUpdateNode.getPreUpdateNode/0#dispred#53daedc2_10#join_rhs` ON FIRST 1 OUTPUT Lhs.2, Lhs.1 'cs', Rhs.1 'node2'
                      11500       ~0%        {3}    | JOIN WITH `CfgNodes::IndexExprCfgNode.getBase/0#dispred#19aba7d8_10#join_rhs` ON FIRST 1 OUTPUT Rhs.1, Lhs.1 'cs', Lhs.2 'node2'
                      1000        ~3%        {3}    | JOIN WITH `CfgNodes::BinaryExprCfgNode.getLhs/0#dispred#bd1c02e7_10#join_rhs` ON FIRST 1 OUTPUT Rhs.1, Lhs.1 'cs', Lhs.2 'node2'
                      500         ~3%        {3}    | JOIN WITH CfgNodes::AssignmentExprCfgNode#a9a5c022 ON FIRST 1 OUTPUT Lhs.0, Lhs.1 'cs', Lhs.2 'node2'
                      0           ~0%        {3}    | JOIN WITH `CfgNodes::BinaryExprCfgNode.getRhs/0#dispred#4a1146e4` ON FIRST 1 OUTPUT Rhs.1, Lhs.1 'cs', Lhs.2 'node2'
                      0           ~0%        {3}    | JOIN WITH DataFlowImpl::TExprNode#83a34c2e ON FIRST 1 OUTPUT Rhs.1 'node2', Lhs.1 'cs', Lhs.2 'node2'

                      192445      ~1%        {3} r12 = r1 UNION r2 UNION r3 UNION r4 UNION r5 UNION r6 UNION r7 UNION r10 UNION r11
                                             return r12
```
2024-12-16 10:20:30 +01:00
Tom Hvitved
2d16b5276d Rust: Fix bad join
```
Evaluated relational algebra for predicate DataFlowImpl::RustDataFlow::pathResolveToVariantCanonicalPath/2#dc73aca0@34414869 with tuple counts:
          422639   ~3%    {3} r1 = JOIN `DataFlowImpl::resolveExtendedCanonicalPath/3#0454a346` WITH Synth::Synth::TPathAstNode#a7913307 ON FIRST 1 OUTPUT Lhs.1, Lhs.0, Lhs.2
        73033499   ~7%    {6}    | JOIN WITH DataFlowImpl::MkVariantCanonicalPath#ab1ecb00 ON FIRST 1 OUTPUT Lhs.1, Lhs.2, Rhs.3, _, Rhs.1, Rhs.2
                          {4}    | REWRITE WITH Tmp.3 := "::", Out.3 := (In.4 ++ Tmp.3 ++ In.5), TEST Out.3 = InOut.1 KEEPING 4
          170993   ~1%    {2}    | SCAN OUTPUT In.0, In.2
                          return r1
```
2024-12-16 10:20:01 +01:00
Simon Friis Vindum
31717524f0 Merge pull request #18270 from paldepind/rust-captured-variables
Rust: Flow through captured variables
2024-12-16 10:08:53 +01:00
Simon Friis Vindum
9da5d7128b Rust: Add test with data flow inconsistency 2024-12-16 09:40:13 +01:00
Simon Friis Vindum
1d8e7fd9ea Rust: Accept differences 2024-12-12 16:47:51 +01:00
Simon Friis Vindum
0fa40fcdcc Rust: Fix captured variable data flow inconsistency 2024-12-12 16:28:19 +01:00
Geoffrey White
03f962ed86 Merge pull request #18226 from geoffw0/badcrypto
Rust: Weak encryption algorithm query.
2024-12-12 14:21:16 +00:00
Michael Nebel
0bfc1b6ea8 Also move the postprocessing queries to the library pack. 2024-12-12 15:03:03 +01:00
Michael Nebel
941b0abbf6 Move modules to the library packs. 2024-12-12 15:03:01 +01:00
Michael Nebel
864c34fc03 Rust: Update all test util paths to point to the new location. 2024-12-12 15:02:59 +01:00
Geoffrey White
44a0ad2942 Update data-flow -> data flow in all versions of ConceptsShared.qll. 2024-12-12 13:36:26 +00:00
Michael Nebel
cbae2cf7fa Rust: Move test utilities into the query pack. 2024-12-12 13:54:42 +01:00
Geoffrey White
611d04e221 Rust: Revert stylistic change in shared file. 2024-12-12 11:30:23 +00:00
Geoffrey White
1d72b750b7 Rust: data-flow -> data flow. 2024-12-12 11:29:23 +00:00
Geoffrey White
d2cfcb4c9e Update rust/ql/lib/codeql/rust/internal/ConceptsShared.qll
Co-authored-by: Simon Friis Vindum <paldepind@github.com>
2024-12-12 11:28:04 +00:00
Simon Friis Vindum
16dcc5c278 Rust: Add variable capture consistency query 2024-12-12 11:23:22 +01:00
Simon Friis Vindum
9fe7bb3e2b Rust: Address PR comments 2024-12-12 11:19:14 +01:00
Simon Friis Vindum
2cf043cfbc Rust: Address PR comments 2024-12-12 10:19:53 +01:00
Simon Friis Vindum
59f3f1f1e9 Apply suggestions from code review
Co-authored-by: intrigus-lgtm <60750685+intrigus-lgtm@users.noreply.github.com>
2024-12-12 08:58:35 +01:00
Geoffrey White
591db05610 Rust: Formatting. 2024-12-11 16:49:29 +00:00
Geoffrey White
ad75906672 Apply suggestions from code review
Co-authored-by: Tom Hvitved <hvitved@github.com>
2024-12-11 16:44:00 +00:00
Simon Friis Vindum
94b037fad1 Rust: Instantiate variable capture library for data flow 2024-12-11 15:44:17 +01:00
Simon Friis Vindum
e8357a648d Rust: Add additional data flow tests 2024-12-11 14:56:16 +01:00
Tom Hvitved
22aaf74184 Merge pull request #18231 from hvitved/rust/models-as-data-summaries
Rust: Models-as-data for flow summaries
2024-12-11 13:37:09 +01:00
Tom Hvitved
baf186fed7 Address review comments 2024-12-11 10:28:32 +01:00
Cornelius Riemenschneider
53ca5083a9 Upgrade bazel to 8.0.0.
Previously, we were using 8.0.0rc1.
In particular, this upgrade means we need to explicitly
import more rules, as they've been moved out of the core bazel repo.
2024-12-10 12:05:37 +01:00
Tom Hvitved
2f8b04b225 Rust: Models-as-data for flow summaries 2024-12-09 13:41:55 +01:00
Geoffrey White
97ab31ab00 Merge pull request #18229 from geoffw0/self
Rust: Fix unwanted 'self' results from rust/unused-variable
2024-12-09 08:50:03 +00:00
Simon Friis Vindum
70501008b0 Merge pull request #18232 from paldepind/rust-self-capture
Rust: Fix a bug where captured `self` parameters where not considered captured
2024-12-06 20:23:38 +01:00
Simon Friis Vindum
9b34615a64 Rust: Update test assertions to match results 2024-12-06 19:05:53 +01:00
Paolo Tranquilli
a555a44a20 Merge pull request #18236 from geoffw0/percent
Rust: Add % of files extracted without errors to summary stats.
2024-12-06 18:01:04 +01:00
Geoffrey White
1ceee76919 Rust: Get the .expected values right this time. 2024-12-06 15:53:42 +00:00
Geoffrey White
4024968e46 Rust: Accept integration test changes. 2024-12-06 15:30:02 +00:00
Geoffrey White
d7d4658aae Rust: Add % of files extractor without errors to summary stats. 2024-12-06 14:54:21 +00:00