This adds the possibility to add a special `proc_macro.rs` source file
to QL tests, which will be generated into a `proc_macro` crate the
usual `lib` crate depends on.
This allow to define procedural macros in QL tests, and is here used to
move the `macro-expansion` integration test to be a language test
instead.
As the generated manifests involved were starting to get a bit complex,
they are now generated from a `mustache` template.
This required some code changes because of some breaking changes in
`clap` and `tree-sitter`.
Also needed to assign a new bazel repo name to the `crates_vendor` to
avoid name conflicts in `MODULE.bazel`.
The rust-analyzer update will need more work as it seems to break rust
analysis on windows.
This was carried out using `cargo upgrade` from `cargo-edit`:
* getting exclusions options for rust-analyzer with
```bash
cargo upgrade -i --dry-run | grep -o 'ra_ap_\S\+' | sort -u | sed 's/^/--exclude=/' > /tmp/exclude
```
* running
```bash
cargo upgrade -i $(cat /tmp/exclude)
misc/bazel/3rdparty/update_cargo_deps.sh
```
* The ungram file is now taken from the rust-analyzer dependencies
pulled in by bazel
* the grammar parsing code is not published, so it must be taken
directly from rust-analyzer code. That part should be less prone to be
updated than the ungram file, so it does not necessarily need to be
in sync with the rust-analyzer version is used elsewhere.
* both need some patches. The former is patched during build, the latter
during loading in `MODULE.bazel`.
This outputs some duration counts for various parts of the extraction
process in the database in the form of telemetry diagnostics.
The diagnostics format was preferred to putting things in the relational
database as that will scale better to code scanning and is more flexible
as for the data we can put into it without passing through the dbscheme.
Also, although it's not the case yet, it will be possible to output
diagnostics even if creation of the database fails.
We've been observing some performance issues using crate_universe on CI.
Therefore, we're moving to vendor the auto-generated BUILD files
in our repository. This should provide a nice speed boost, while
getting rid of the complexity of the "rust cache" job we've been using
when we had a lot of git dependencies.
This PR includes a vendor script, and I'll put up a CI job internally
that runs that vendor script on Cargo.toml and Cargo.lock changes, to check
that the vendored files are in sync.
