Commit Graph

58293 Commits

Author SHA1 Message Date
Mathias Vorreiter Pedersen
81d20be1ee Update cpp/ql/src/experimental/Security/CWE/CWE-416/UseAfterExpiredLifetime.qhelp
Co-authored-by: Geoffrey White <40627776+geoffw0@users.noreply.github.com>
2023-09-13 11:37:32 +01:00
Mathias Vorreiter Pedersen
1232120d42 C++: Naively copy the 'surprising lifetimes' query from Coding Standards and add required metadata. 2023-09-04 16:21:28 +01:00
AlexDenisov
e2602fbbc4 Merge pull request #14119 from github/alexdenisov/sequence-expr
Swift: fix SequenceExpr extraction
2023-09-04 12:29:07 +02:00
Paolo Tranquilli
7d89028688 Merge pull request #14106 from github/alexdenisov/rename-xcode-autobuilder
Swift: rename autobuilder. NFC
2023-09-04 12:28:47 +02:00
Alex Denisov
888dd786b4 Swift: fix SequenceExpr extraction
Before we extracted all the subexpressions from the `SequenceExpr` while we should've only extracted the expressions at odd indices:
```
...
/// SequenceExpr - A list of binary operations which has not yet been
/// folded into a tree.  The operands all have even indices, while the
/// subexpressions with odd indices are all (potentially overloaded)
/// references to binary operators.
class SequenceExpr final : public Expr,
...
```

The AST for a `SequenceExpr` looks like this:

```
sequence_expr:
  unresolved_dot_expr:
    ...
  assign_expr:
    member_ref_expr:
      ...
    dot_syntax_call_expr:
      ...
  unresolved_member_chain_expr:
    ...
```

however, what's is not visible with the "final" AST is that `unresolved_dot_expr` is the unresolved version of `assign_expr.member_ref_expr` and the `unresolved_member_chain_expr` is the unresolved version of `assign_expr.dot_syntax_call_expr`.
This becomes visible when I enable typechecker debugging:

```c++
auto &typeCheckerOptions = invocation.getTypeCheckerOptions();
typeCheckerOptions.DebugConstraintSolver = true;
```

Which prints the following snippets:

```
---Initial constraints for the given expression---
(assign_expr type='()' location=foo.swift:25:54 range=[foo.swift:25:13 - line:25:57]
  (unresolved_dot_expr type='$T2' location=foo.swift:25:29 range=[foo.swift:25:13 - line:25:29] field 'preferredDatePickerStyle' function_ref=unapplied
    (unresolved_dot_expr type='$T1' location=foo.swift:25:18 range=[foo.swift:25:13 - line:25:18] field 'datePicker' function_ref=unapplied
      (declref_expr type='DatePickerCell' location=foo.swift:25:13 range=[foo.swift:25:13 - line:25:13] decl=foo.(file).DatePickerRowProtocol extension.configurePickerStyle(_:_:).cell@foo.swift:15:33 function_ref=unapplied)))
  (unresolved_member_chain_expr implicit type='$T5' location=foo.swift:25:57 range=[foo.swift:25:56 - line:25:57]
    (unresolved_member_expr type='$T4' location=foo.swift:25:57 range=[foo.swift:25:56 - line:25:57] name='wheels' function_ref=unapplied)))

// ...

---Type-checked expression---
(assign_expr type='()' location=foo.swift:25:54 range=[foo.swift:25:13 - line:25:57]
  (member_ref_expr type='@lvalue UIDatePickerStyle' location=foo.swift:25:29 range=[foo.swift:25:13 - line:25:29] decl=UIKit.(file).UIDatePicker.preferredDatePickerStyle
    (force_value_expr implicit type='UIDatePicker' location=foo.swift:25:18 range=[foo.swift:25:13 - line:25:18] implicit_iuo_unwrap
      (load_expr implicit type='UIDatePicker?' location=foo.swift:25:18 range=[foo.swift:25:13 - line:25:18]
        (member_ref_expr type='@lvalue UIDatePicker?' location=foo.swift:25:18 range=[foo.swift:25:13 - line:25:18] decl=foo.(file).DatePickerCell.datePicker@foo.swift:10:29
          (declref_expr type='DatePickerCell' location=foo.swift:25:13 range=[foo.swift:25:13 - line:25:13] decl=foo.(file).DatePickerRowProtocol extension.configurePickerStyle(_:_:).cell@foo.swift:15:33 function_ref=unapplied)))))
  (dot_syntax_call_expr type='UIDatePickerStyle' location=foo.swift:25:57 range=[foo.swift:25:56 - line:25:57]
    (declref_expr type='(UIDatePickerStyle.Type) -> UIDatePickerStyle' location=foo.swift:25:57 range=[foo.swift:25:57 - line:25:57] decl=UIKit.(file).UIDatePickerStyle.wheels function_ref=unapplied)
    (argument_list implicit
      (argument
        (type_expr implicit type='UIDatePickerStyle.Type' location=foo.swift:25:56 range=[foo.swift:25:56 - line:25:56] typerepr='UIDatePickerStyle')))))
```

The proposed solution is to only extract subexpressions at indices from `SequenceExpr` thus ignoring all the unresolved leftovers.

Note: I'm not entirely sure about the case when there is only child (`elements.size() == 1`) so I'm always extracting it.

This patch fixes the last source of unresolved expressions.
2023-09-04 11:42:12 +02:00
Rasmus Wriedt Larsen
a58c9e91ea Merge pull request #14130 from RasmusWL/fixup-accept-ci-changes
Misc: Fixup for `accept-expected-changes-from-ci.py`
2023-09-04 09:47:31 +02:00
Rasmus Wriedt Larsen
301133ad94 Misc: Fixup for accept-expected-changes-from-ci.py
Allow empty lines while looking for diff (between `---expected` and `FAILED`). This can happen when there is `Locations outside the test directory` since an empty line is printed after that warning message (this output can be interleaved with the diff due to parallel execution).
2023-09-04 09:32:06 +02:00
Erik Krogh Kristensen
284ca5e4ee Merge pull request #14129 from github/dependabot/cargo/ql/regex-1.9.5
Bump regex from 1.9.3 to 1.9.5 in /ql
2023-09-04 08:11:51 +02:00
dependabot[bot]
4919cc4c4e Bump regex from 1.9.3 to 1.9.5 in /ql
Bumps [regex](https://github.com/rust-lang/regex) from 1.9.3 to 1.9.5.
- [Release notes](https://github.com/rust-lang/regex/releases)
- [Changelog](https://github.com/rust-lang/regex/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/regex/compare/1.9.3...1.9.5)

---
updated-dependencies:
- dependency-name: regex
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-09-04 03:47:05 +00:00
Mathias Vorreiter Pedersen
67a0112fcb Merge pull request #14115 from MathiasVP/fix-incorrect-load-of-constant
C++: Don't generate `Load`s for constant expressions
2023-09-01 08:50:44 +01:00
Mathias Vorreiter Pedersen
72d9812fea C++: Accept more test changes. 2023-08-31 21:56:35 +01:00
Mathias Vorreiter Pedersen
dfefd62089 C++: Accept test changes. 2023-08-31 20:38:46 +01:00
Mathias Vorreiter Pedersen
aed14f2924 C++: Don't insert loads for constants. 2023-08-31 20:38:40 +01:00
Mathias Vorreiter Pedersen
b575747357 C++: Add testcase with invalid IR. 2023-08-31 20:35:33 +01:00
yoff
da64ea40b9 Merge pull request #13782 from jorgectf/jorgectf/shlex-quote
Python: Add `shlex.quote` as `py/shell-command-constructed-from-input` sanitizer
2023-08-31 21:08:58 +02:00
Mathias Vorreiter Pedersen
10548b57d7 Merge pull request #14103 from MathiasVP/non-certain-def-is-a-use
C++: Non-certain definitions should always be uses
2023-08-31 16:15:30 +01:00
Erik Krogh Kristensen
cd590d356d Merge pull request #14053 from erik-krogh/ts52
JS: Add support for TypeScript 5.2
2023-08-31 14:39:50 +02:00
Tom Hvitved
23857267db Merge pull request #14110 from hvitved/ruby/remove-emptiness-successor
Ruby: Get rid of unused `EmptinessSuccessor`
2023-08-31 13:41:25 +02:00
Tom Hvitved
50db6916c8 Ruby: Get rid of unused EmptinessSuccessor 2023-08-31 13:17:05 +02:00
Tom Hvitved
73370e7282 Merge pull request #14100 from hvitved/dataflow/consistency-pack
Data flow: Add consistency checks to shared ql pack
2023-08-31 11:47:40 +02:00
Tom Hvitved
756886808d Merge pull request #14098 from hvitved/csharp/cil-best-impl
C#: Speedup `bestImplementation`
2023-08-31 10:57:28 +02:00
Mathias Vorreiter Pedersen
f5509da4bb Merge pull request #14038 from alexet/delete-ir
CPP: Add delete/delete[] calls to the IR.
2023-08-31 09:22:34 +01:00
Asger F
2d5c40db31 Merge pull request #14048 from asgerf/shared/variable-capture-write-source-node
Variable capture: allow arbitrary data-flow nodes to be the source of a write
2023-08-31 10:20:48 +02:00
Tom Hvitved
c68d0bc936 Merge pull request #14099 from hvitved/csharp/transitive-capture-call-unique
C#: Do not embed target callable in `TransitiveCapturedCall`
2023-08-31 10:12:26 +02:00
Alex Denisov
c1da2dfa73 Swift: fix autobuild.sh 2023-08-31 09:40:13 +02:00
Michael Nebel
ded49015e0 Merge pull request #13970 from michaelnebel/csharp/usereferenceassemlblies
C#: Compile against the reference assemblies in the standalone extractor (if possible)
2023-08-31 09:00:30 +02:00
Erik Krogh Kristensen
cd289f8207 Merge pull request #14105 from github/dependabot/cargo/ql/chrono-0.4.28
Bump chrono from 0.4.27 to 0.4.28 in /ql
2023-08-31 08:16:42 +02:00
Alex Denisov
803adae9c6 Swift: rename autobuilder. NFC 2023-08-31 08:12:52 +02:00
dependabot[bot]
4770eb0328 Bump chrono from 0.4.27 to 0.4.28 in /ql
Bumps [chrono](https://github.com/chronotope/chrono) from 0.4.27 to 0.4.28.
- [Release notes](https://github.com/chronotope/chrono/releases)
- [Changelog](https://github.com/chronotope/chrono/blob/main/CHANGELOG.md)
- [Commits](https://github.com/chronotope/chrono/compare/v0.4.27...v0.4.28)

---
updated-dependencies:
- dependency-name: chrono
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-08-31 03:44:01 +00:00
Mathias Vorreiter Pedersen
1159508e4f Merge pull request #14008 from MathiasVP/reuse-even-more-nodes
C++: Reuse even more `DataFlow::Node`s
2023-08-30 19:38:52 +01:00
Alex Eyers-Taylor
59a77666a6 CPP: Remove uneeded indirection around delete calls in the IR. 2023-08-30 16:31:51 +01:00
Michael Nebel
3afa4aa91e C#: Add change note. 2023-08-30 16:51:16 +02:00
Mathias Vorreiter Pedersen
261ba8e02d C++: Add QLDoc to 'isDereference'. 2023-08-30 14:34:30 +01:00
Tom Hvitved
253f932d2a Python: Use data flow consistency checks from shared pack 2023-08-30 15:29:41 +02:00
Tom Hvitved
9af706c2a5 Swift: Use data flow consistency checks from shared pack 2023-08-30 15:29:41 +02:00
Tom Hvitved
db304d118b C++: Use data flow consistency checks from shared pack 2023-08-30 15:29:41 +02:00
Tom Hvitved
fefe64bf0c Java: Use data flow consistency checks from shared pack 2023-08-30 15:29:41 +02:00
Tom Hvitved
5c8367a695 C#: Use data flow consistency checks from shared pack 2023-08-30 15:29:41 +02:00
Tom Hvitved
c4b626a416 Ruby: Use data flow consistency checks from shared pack 2023-08-30 15:29:41 +02:00
Tom Hvitved
de7c9bdd9b Data flow: Add consistency checks to shared ql pack 2023-08-30 15:29:41 +02:00
Michael Nebel
291d7b3e05 C#: Use reference assemblies instead of implementation assemblies. 2023-08-30 14:19:11 +02:00
Michael Nebel
a966c0e1eb C#: Do not include the mscorlib assembly and remove the option. 2023-08-30 14:19:11 +02:00
Tom Hvitved
ce229fe21f Merge pull request #14096 from hvitved/ruby/expected-test-fix
Ruby: Update expected test output
2023-08-30 13:50:49 +02:00
Tom Hvitved
29982fe30e C#: Do not embed target callable in TransitiveCapturedCall 2023-08-30 13:48:44 +02:00
Tom Hvitved
66f5e4a05b C#: Speedup bestImplementation
Avoids an expensive anti-join:

```
[2023-08-29 15:25:48] Evaluated non-recursive predicate _FileSystem#df18ed9a::Make#File#1a556f64::Input#::Container::toString#0#dispred#bf_Method#621e9e2e::__#antijoin_rhs@96d08bc8 in 272332ms (size: 1841891).
Evaluated relational algebra for predicate _FileSystem#df18ed9a::Make#File#1a556f64::Input#::Container::toString#0#dispred#bf_Method#621e9e2e::__#antijoin_rhs@96d08bc8 with tuple counts:
         4632443     ~2%    {3} r1 = JOIN _cil_instruction_3#antijoin_rhs_cil_method_implementation#shared WITH cil_method_implementation ON FIRST 1 OUTPUT Rhs.1, Lhs.0, Lhs.1

        71945701     ~3%    {3} r2 = JOIN r1 WITH cil_method_implementation_10#join_rhs ON FIRST 1 OUTPUT Rhs.1, Lhs.1, Lhs.2
        71945701  ~1329%    {3} r3 = JOIN r2 WITH Method#621e9e2e::MethodImplementation::getNumberOfInstructions#0#dispred#ff ON FIRST 1 OUTPUT Lhs.1, Lhs.2, Rhs.1
         5016836     ~4%    {4} r4 = JOIN r3 WITH Method#621e9e2e::MethodImplementation::getNumberOfInstructions#0#dispred#ff ON FIRST 1 OUTPUT Lhs.0, Lhs.1, Lhs.2, Rhs.1
                            {4} r5 = SELECT r4 ON In.3 < In.2
           65637     ~3%    {2} r6 = SCAN r5 OUTPUT In.0, In.1

        71945701     ~0%    {3} r7 = JOIN r1 WITH cil_method_implementation_10#join_rhs ON FIRST 1 OUTPUT Lhs.2, Lhs.1, Rhs.1
        71945701     ~1%    {4} r8 = JOIN r7 WITH assemblies ON FIRST 1 OUTPUT Lhs.2, Lhs.1, Lhs.0, Rhs.1
        71945701     ~0%    {5} r9 = JOIN r8 WITH cil_method_implementation ON FIRST 1 OUTPUT Rhs.2, Lhs.1, Lhs.2, Lhs.0, Lhs.3
        71945701     ~0%    {5} r10 = JOIN r9 WITH assemblies ON FIRST 1 OUTPUT Rhs.1, Lhs.1, Lhs.2, Lhs.3, Lhs.4
        71945701     ~0%    {5} r11 = JOIN r10 WITH FileSystem#df18ed9a::Make#File#1a556f64::Input#::Container::toString#0#dispred#bf ON FIRST 1 OUTPUT Lhs.4, Lhs.1, Lhs.2, Lhs.3, Rhs.1
        71945701     ~2%    {5} r12 = JOIN r11 WITH FileSystem#df18ed9a::Make#File#1a556f64::Input#::Container::toString#0#dispred#bf ON FIRST 1 OUTPUT Lhs.1, Lhs.2, Lhs.3, Lhs.4, Rhs.1
                            {5} r13 = SELECT r12 ON In.4 > In.3
        33509342     ~0%    {3} r14 = SCAN r13 OUTPUT In.0, In.1, In.2
        33509342     ~0%    {4} r15 = JOIN r14 WITH Method#621e9e2e::MethodImplementation::getNumberOfInstructions#0#dispred#ff ON FIRST 1 OUTPUT Lhs.2, Rhs.1, Lhs.0, Lhs.1
        33051362  ~1670%    {2} r16 = JOIN r15 WITH Method#621e9e2e::MethodImplementation::getNumberOfInstructions#0#dispred#ff ON FIRST 2 OUTPUT Lhs.2, Lhs.3

        33116999  ~1646%    {2} r17 = r6 UNION r16
                            return r17
```
2023-08-30 13:46:11 +02:00
Tom Hvitved
7e77c77d92 Ruby: Update expected test output 2023-08-30 13:33:48 +02:00
erik-krogh
984795ee46 fix off-by-one 2023-08-30 13:29:23 +02:00
Anders Starcke Henriksen
44a83a71a8 Merge pull request #13879 from github/starcke/automodel-pack
Create separate automodel pack
2023-08-30 13:24:25 +02:00
Rasmus Wriedt Larsen
62c2316124 Merge pull request #14084 from RasmusWL/flask-jsonify
Python: Remove XSS FP from use of `flask.jsonify`
2023-08-30 13:07:54 +02:00
Mathias Vorreiter Pedersen
4ca259b200 C++: Non-exact definitions should always be a use. 2023-08-30 11:50:57 +01:00