Commit Graph

10 Commits

Author SHA1 Message Date
Paolo Tranquilli
d01d657f89 Bazel: accept new SSH keys in git_lfs_probe.py 2024-05-17 16:39:18 +01:00
Paolo Tranquilli
170e2231d4 Bazel: allow LFS rules to use cached downloads without internet
If the cache is prefilled, LFS rules were still trying to query LFS
urls.

Now the strategy is to first try to fetch the files from the repository
cache (which is possible by providing an empty url list and `allow_fail`
to `repository_ctx.download`), and only run the LFS protocol if that
fails. Technically this is possible by enhancing `git_lfs_probe.py` with
a `--hash-only` flag.

This is also an optimization where no uneeded access is done (including
the slightly slow SSH call) if the repository cache is warm.
2024-05-17 16:24:38 +01:00
Óscar San José
9105faa3fd make sure GITHUB_TOKEN exists and is not empty before using it in auth headers 2024-05-14 11:22:07 +02:00
Óscar San José
faa2dcee24 test GITHUB_TOKEN non-emptyness before using it in auth headers 2024-05-14 11:18:18 +02:00
Óscar San José
5b572a2c2a make GITHUB_TOKEN in lfs_probe a fallback rather than an override 2024-05-14 10:24:02 +02:00
Paolo Tranquilli
9f27eb3eda Bazel: make git_lfs_probe.py compatible with python 3.8 2024-05-07 12:44:28 +02:00
Paolo Tranquilli
54c9aea251 Bazel: fix lfs lazy rule when all objects are local 2024-05-06 15:47:43 +02:00
Paolo Tranquilli
ecdf62376d Bazel: clean up git_lfs_probe.py 2024-05-03 09:21:57 +02:00
Paolo Tranquilli
daea674095 Bazel: cover standard https git credentials in git_lfs_probe.py 2024-05-02 18:02:22 +02:00
Paolo Tranquilli
677520aa8e Bazel: improved lazy lfs files
This reintroduces lazy lfs file rules that were removed in
https://github.com/github/codeql/pull/16117, now improved.

The new rules will make the actual file download go through bazel's
download manager, which includes:
* caching into the repository cache
* sane limiting of concurrent downloads
* retries

The bulk of the work is done by `git_lfs_probe.py`, which will use the
LFS protocol (with authentication via SSH) to output short lived
download URLs that can be consumed by `repository_ctx.download`.
2024-05-02 06:38:26 +02:00