* The ungram file is now taken from the rust-analyzer dependencies
pulled in by bazel
* the grammar parsing code is not published, so it must be taken
directly from rust-analyzer code. That part should be less prone to be
updated than the ungram file, so it does not necessarily need to be
in sync with the rust-analyzer version is used elsewhere.
* both need some patches. The former is patched during build, the latter
during loading in `MODULE.bazel`.
This outputs some duration counts for various parts of the extraction
process in the database in the form of telemetry diagnostics.
The diagnostics format was preferred to putting things in the relational
database as that will scale better to code scanning and is more flexible
as for the data we can put into it without passing through the dbscheme.
Also, although it's not the case yet, it will be possible to output
diagnostics even if creation of the database fails.
We've been observing some performance issues using crate_universe on CI.
Therefore, we're moving to vendor the auto-generated BUILD files
in our repository. This should provide a nice speed boost, while
getting rid of the complexity of the "rust cache" job we've been using
when we had a lot of git dependencies.
This PR includes a vendor script, and I'll put up a CI job internally
that runs that vendor script on Cargo.toml and Cargo.lock changes, to check
that the vendored files are in sync.
