Wrapping an optional capture in a node meant building the wrapper inside
an `Option::map`, which buried the shape of the output in a closure:
(break_expr label: {lbl.map(|l| tree!((identifier #{l})))})
A field's value can now be marked with `?` instead. If a `#{expr}`
anywhere beneath it interpolates an absent value, the subtree is
abandoned and the field is left unset:
(break_expr label: (identifier #{lbl})?)
The marker follows the value, as quantifiers do in the query language
(`label: _? @@lbl`). Absence propagates through as many levels as
necessary, and a nested `?` catches first, so an inner absent value need
not discard the outer node.
Only `#{expr}` propagates absence, since it supplies a node's content:
with no value there is no leaf to build. A `{expr}` splice supplies
children, where yielding nothing already leaves the field unset, so `?`
is rejected on one. Outside a `?`, interpolating an `Option` with
`#{expr}` remains a compile error, keeping the choice between leaving a
field unset and unwrapping explicit; `YeastDisplay` now carries an
`on_unimplemented` note pointing at the new syntax.
Inside a fallible field, interpolations route through a new
`MaybeYeastValue` trait, whose impls are enumerated rather than blanket
for the same coherence reason `YeastDisplay`'s are.
Codegen outside a `?` is unchanged, so `tree!` and `trees!` keep their
return types. Converting the ten `Option::map` sites in the Swift rules
leaves the corpus byte-identical; four captures become `@@` now that
their values are only ever interpolated.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Adds rules for mapping things like `=` and `as!` to `infix_operator`s,
so that they are present in the output (which for unresolved operators
expects an alternating sequence of values and `infix_operator`s).
For ternary operators, we expand this into _two_ unresolved infix
operators, `?` and `:` respectively.
Add corpus test cases that witness Swift constructs the swift-syntax
mapping does not yet handle, so they map to `unsupported_node` (or, for
unfoldable operator chains, `unresolved_operator_sequence`). These
document the current behaviour and give us a place to observe the diff
when each construct is eventually supported.
The cases are placed alongside the feature they exercise:
- functions: `inout` parameter types and `&`-prefixed inout arguments.
- expressions: key paths, generic specialization in expression position
(`Array<Int>()`), `copy`/`consume` expressions, and `unsafe`
expressions.
- operators: unresolved operator sequences (pointwise operators the
parser cannot fold), custom postfix operators, and partial ranges.
- control-flow: `fallthrough`, `defer`, and `discard` statements.
- types: `actor` declarations, inline array types (`[3 of Int]`),
function-type attributes (`@convention(c)`, `@Sendable`), noncopyable
(`~Copyable`) types, and conditional compilation in a class body.
- literals: the `#line` magic literal.
The conditional-compilation case is worth calling out because it
swallows members: swift-syntax reports a structured `ifConfigDecl` whose
branches hold ordinary member items, but with no rule for it the whole
`#if` block collapses into a single `unsupported_node`, so the
declarations inside are not extracted at all.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Pure deletion of the files the previous commit left unreferenced:
- the vendored `unified/extractor/tree-sitter-swift` crate — grammar,
generated parser tables, editor queries and Node bindings;
- `scripts/regenerate-grammar.sh`, which regenerated those tables from
the grammar;
- the `rules_macro_smoke` test, which type-checked the `rules!` macro
against the crate's `node-types.yml` and so cannot outlive it.
The extractor now builds with no Swift grammar and no Swift toolchain;
the Swift dependency lives solely in the separate `swift-syntax-parse`
binary.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
With the Swift front-end fully switched to swift-syntax, nothing links
the tree-sitter Swift grammar any more. Remove every reference to it;
the vendored crate itself is deleted in the following commit, so that
this one shows only what actually changed.
- Drop `unified/extractor/tree-sitter-swift` as a workspace member, path
dependency and BUILD.bazel dependency.
- Drop the now-unused `tree-sitter` and `tree-sitter-embedded-template`
direct dependencies from the extractor (neither is referenced any
longer; the tree-sitter runtime is still pulled in transitively where
the shared extractor needs it).
- Rewrite the "Swift Parser" section of `AGENTS.md`, which still pointed
at `grammar.js` and `node-types.yml`, to describe `swift-syntax-parse`
and the hand-maintained `swift_node_types.yml`, and note that the
tests need the parser binary.
The mapping's comments also explained many rules by how the tree-sitter
path had behaved. That is now of historical interest only, so each is
restated in terms of swift-syntax and the target AST alone — no rule
changes, and the corpus is unaffected. Two were more than stylistic:
- The `subscriptCallExpr` rule and its corpus case said the parser
reports `xs[0]` and `xs(0)` identically. swift-syntax distinguishes
them, so the collapse to `call_expr` is now purely ours, and a
dedicated `subscript_expr` would need only a schema addition and a
remap.
- `discardAssignmentExpr` mapped to `name_expr` "because tree-sitter
treated `_` as a name". The standing reason is that the target AST has
no expression-level discard — only `ignore_pattern`, which is a
pattern.
References to tree-sitter's *node model* are kept: yeast is built on it,
so `adapter.rs` still explains named/anonymous nodes, `extra` tokens and
byte-offset conventions in those terms.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Map a nominal type's inheritance clause (`class C: Base, Proto`, and
likewise for enum/struct/protocol/extension) to `base_type` children on
the `class_like_declaration`, one per inherited type. The tree-sitter
path dropped these (no corpus target had a `base_type`) and the mapping
matched that for parity; swift-syntax exposes the clause cleanly.
Adds a focused `class-with-multiple-base-types` corpus case and updates
`class-inheritance` to witness the restored base types.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Map a generic type applied with explicit arguments (`Set<Int>`,
`Dictionary<String, Array<Int>>`) to a `generic_type_expr` whose `base`
is the type name and whose `type_argument`s are the structured,
recursively-mapped arguments — the same shape the sugared `?`/`[]`/`[:]`
types already desugar to.
Previously the whole application was kept opaquely as a
`named_type_expr` whose name was the raw source text, matching the
tree-sitter path for corpus parity.
Adds a focused `generic-type-arguments` corpus case (multiple and nested
arguments) and updates `set-literal` to witness the structured
arguments.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Emit a parameter's declared type on the mapped `parameter` node. The
tree-sitter path dropped it — its untyped-parameter rule was ordered
before the typed one and shadowed it — and the mapping matched that for
corpus parity; swift-syntax models the type as a required
`functionParameter.type`, so emitting it is a correctness improvement.
The existing function-parameter corpus cases (and the initializer cases
from the previous commit) witness the restored types.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Emit an initializer's parameters on the `constructor_declaration`,
captured from the `initializerDecl` signature (as for `functionDecl`).
The tree-sitter path dropped them -- its positional `(parameter)*`
capture missed the field-attached parameters -- and the mapping matched
that for corpus parity; swift-syntax exposes them cleanly, so emitting
them is a correctness improvement.
Adds a focused `constructor-with-parameters` corpus case and updates
`class-with-initializer` to witness the restored parameters.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Adds a few tests that validate that higher-order functions are parsed
correctly into the commonAST representation.
Also removes a redundant assignment to `ctx.in_function_type` that
happend after all translations had taken place (and so nothing would
actually read this field).
The extractor shells out to a separate `swift-syntax-parse` binary, but
nothing placed it in the extractor pack, so a shipped Swift extraction
failed at the first spawn. Package it next to the extractor, the same
way `//swift/extractor` ships its Swift-linked binary: a small wrapper
points the dynamic loader at its own directory and execs the real
binary, whose Swift runtime libraries travel alongside it.
- `swift-syntax-parse.sh`: wrapper that sets `LD_LIBRARY_PATH` /
`DYLD_LIBRARY_PATH` to its directory and execs
`swift-syntax-parse.real`
(mirrors `swift/extractor/extractor.sh`).
- `runtime.bzl`: a `swift_runtime_libs` rule that selects just the Linux
Swift runtime shared objects (`usr/lib/swift/linux/*.so`) out of the
full toolchain, so only they — not the whole toolchain — travel with
the binary.
- `swift-syntax-rs/BUILD.bazel`: the `rust_binary` becomes
`swift-syntax-parse.real`
and carries the runtime libraries as runfiles on Linux; a `sh_binary`
(`swift-syntax-parse`) is the wrapper; `codeql_pkg_runfiles` flattens
the three (wrapper, real binary, runtime) into one directory.
- `BUILD.bazel`: ship that group under `tools/{CODEQL_PLATFORM}` next to
the extractor, on the platforms where swift-syntax builds
(Linux/macOS).
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
`languages::swift::adapter` embeds the swift-syntax node-types schema
with `include_str!("../../../swift_node_types.yml")`, but the file was
never listed in the extractor's Bazel `compile_data`. The `cargo` build
finds it on disk, so this went unnoticed, but the sandboxed Bazel build
cannot see it and fails to compile the extractor. List it alongside
`ast_types.yml`.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Two fixes prompted by review of the swift-syntax switch-over.
Parser resolution (`parse.rs`): `parse_bin` now resolves the
`swift-syntax-parse` executable in priority order — the
`CODEQL_EXTRACTOR_UNIFIED_SWIFT_SYNTAX_PARSE` override, then a copy next
to the extractor executable (as a shipped extractor pack lays it out:
`tools/<platform>/{extractor,swift-syntax-parse}`), then a
bare `PATH` lookup. This lets a packaged extractor find its parser with
no environment setup. (Bundling the binary into the pack, together with
its Swift runtime, is a separate follow-up.)
Corpus test guard (`corpus_tests.rs`): `parser_available` previously
treated *any* parser error as "unavailable" and skipped the entire
corpus suite, so a parser that was present but crashed or emitted
invalid JSON would silently skip the exact regressions the suite exists
to catch. It now uses the new `binary_available`, which reports whether
the *executable* can be launched (false only when it cannot be found,
e.g. no Swift toolchain); a launchable-but-failing parser makes the
suite run and fail.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
`swift-syntax-rs` is a workspace member, so its build script runs on a
plain `cargo check`/`fmt`/`clippy` at the repo root. Previously it
panicked when `swift build` could not be run, breaking those Swift-free
workflows for anyone without a Swift toolchain.
Instead, when `swift build` cannot be spawned, emit a `cargo:warning`
and skip the link directives rather than panicking. `cargo
check`/`fmt`/`clippy` don't link, so they keep working; only `cargo
build`/`cargo test` then fail, at link time — which is fair, since those
genuinely need Swift (and CI builds go through Bazel). A Swift toolchain
that is present but whose build fails is still surfaced as a hard error.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Regenerate `types/property-with-getter-and-setter`, whose mapped AST now
differs from the tree-sitter output: the backing `private var _v`
retains its `private` modifier (`modifier "var"` + `modifier
"private"`), whereas the tree-sitter path dropped it (its
`visibility_modifier` node carried no text). The swift-syntax
front-end preserves the modifier, so the mapped AST is strictly richer
here.
The raw (second) section is regenerated to the swift-syntax AST like the
other cases; the mapped (third) section gains the retained `private`
modifier.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
