Introduce NodeRef as a typed wrapper around node arena IDs. Captures in
desugaring rules are now bound as NodeRef instead of raw usize, which
prevents accidental misuse and enables source-text-aware rendering.
Add the YeastDisplay trait as an alternative to Display: its
yeast_to_string method receives the Ast, allowing NodeRef to resolve to
the captured node's source text instead of printing a numeric ID.
Store the original source bytes in the Ast so that NodeContent::Range
values (from synthesized literal nodes) can be resolved back to text.
Update yeast-macros to emit NodeRef-typed capture bindings and use
Into::<usize>::into where raw IDs are needed. The #{expr} template
syntax now uses YeastDisplay instead of Display.
The effect is visible in the corpus tests: operator nodes now correctly
render as e.g. operator "+" instead of operator "3".
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Canonicalize `current_dir()` to match canonicalized file paths (avoids
`\\?\` prefix mismatch on Windows), and normalize backslashes to
forward slashes in relative diagnostic paths.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Drop the `url` crate dependency. When a path can't be relativized
against the source root, emit it as a bare absolute path and let the
CLI's SARIF generator handle URI conversion downstream.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Diagnostic `location.file` entries were using absolute paths (e.g.
`/home/runner/work/...`), causing broken links in the GitHub UI.
Now relativize against CWD (the source root during extraction), falling
back to a properly percent-encoded `file:` URI for paths outside it.
Fixes https://github.com/github/codeql/issues/21802
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add BUILD.bazel files for the yeast and yeast-macros crates, register
them as dependencies of the shared tree-sitter extractor, and refresh
the vendored crate dependencies via update_tree_sitter_extractors_deps.sh.
Language and LanguageSpec gain optional output_node_types field.
When set, the generator produces dbscheme/QL from the output types
and the extractor validates TRAP against them.
All existing extractors pass None (no behavior change).
Ruby extract() calls gain vec![] for the new rules parameter.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
extract() gains a rules parameter. When empty, uses tree-sitter native
traversal (no behavior change). When non-empty, runs yeast desugaring
and extracts via traverse_yeast.
Adds AstNode trait abstracting over tree_sitter::Node and yeast::Node,
with minimal changes to existing Visitor methods (Node -> &N in 6
signatures).
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Also:
* remove new warnings raised by the rust toolchain
* run new formatting and linting
* update the rust toolchain used by `cargo`
While we keep `bazel` builds using the same toolchain as internally
(now a nightly one), I opted for using a stable toolchain for `cargo`.
The nightly toolchain is only required internally for build reasons, we
should keep not using any unstable rust features in our sources.
Supports only a minimal subset of the project layout specification;
enough to work with the transformers produced by the CLI when building
an overlay database.
The rust-analyzer update will need more work as it seems to break rust
analysis on windows.
This was carried out using `cargo upgrade` from `cargo-edit`:
* getting exclusions options for rust-analyzer with
```bash
cargo upgrade -i --dry-run | grep -o 'ra_ap_\S\+' | sort -u | sed 's/^/--exclude=/' > /tmp/exclude
```
* running
```bash
cargo upgrade -i $(cat /tmp/exclude)
misc/bazel/3rdparty/update_cargo_deps.sh
```
We've been observing some performance issues using crate_universe on CI.
Therefore, we're moving to vendor the auto-generated BUILD files
in our repository. This should provide a nice speed boost, while
getting rid of the complexity of the "rust cache" job we've been using
when we had a lot of git dependencies.
This PR includes a vendor script, and I'll put up a CI job internally
that runs that vendor script on Cargo.toml and Cargo.lock changes, to check
that the vendored files are in sync.
There was a recent round of tree-sitter-* package releases,
so the latest code is now a) released and b) available on crates.io.
Therefore, move away from the (super slow on CI) git dependencies to released crates instead.
This also includes a run of `cargo update`, so there's a bunch of more changes to the lockfile.
