If the cache is prefilled, LFS rules were still trying to query LFS
urls.
Now the strategy is to first try to fetch the files from the repository
cache (which is possible by providing an empty url list and `allow_fail`
to `repository_ctx.download`), and only run the LFS protocol if that
fails. Technically this is possible by enhancing `git_lfs_probe.py` with
a `--hash-only` flag.
This is also an optimization where no uneeded access is done (including
the slightly slow SSH call) if the repository cache is warm.
When using a sufficiently new version of Python, it will give a warning
about the escape sequence `\_` in `¯\_(ツ)_/¯` not being a valid escape
:D fix is to make the docstring a raw string.
Thanks @owen-mc
This reintroduces lazy lfs file rules that were removed in
https://github.com/github/codeql/pull/16117, now improved.
The new rules will make the actual file download go through bazel's
download manager, which includes:
* caching into the repository cache
* sane limiting of concurrent downloads
* retries
The bulk of the work is done by `git_lfs_probe.py`, which will use the
LFS protocol (with authentication via SSH) to output short lived
download URLs that can be consumed by `repository_ctx.download`.
This introduces tooling and enforcement for formatting bazel files.
The tooling is provided as a bazel run target from
[keith/buildifier-prebuilt](https://github.com/keith/buildifier-prebuilt).
This is used in a [`pre-commit`](https://pre-commit.com/) hook for those
having that installed. In turn this is used in a CI check. Relying on a
`pre-commit` action gives us easy checking that buildifying did not
change anything in the files and printing the diff, without having to
hand-roll the check ourselves.
This enforcement will make usage of gazelle easier, as gazelle itself
might reformat files, even outside of `go`. Having them properly
formatted will allow gazelle to leave them unchanged, without needing
to configure awkward exclude directives.
