Commit Graph

36 Commits

Author SHA1 Message Date
Paolo Tranquilli
61847bc58b Ripunzip: provide installer 2024-06-05 14:46:59 +02:00
Cornelius Riemenschneider
5875ec272d Merge pull request #16627 from github/criemen/pkg-fix-install
`pkg.bzl`: Fix install targets.
2024-06-03 15:31:43 +02:00
Paolo Tranquilli
8479673419 Bazel: ignore non-default SSH endpoints in git_lfs_probe.py 2024-06-03 12:26:37 +02:00
Paolo Tranquilli
07f4288e1f Fix zipmerge build and test 2024-05-31 12:12:58 +02:00
Cornelius Riemenschneider
e2cb5631d9 pkg.bzl: Fix install targets.
First, the labels used in this file need to be wrapped in `Label`,
so that we can use the installer target from the internal repo.
Second, when unpacking zips, the path passed to `ripunzip` included a trailing
`\n`, which caused `ripunzip` to fail.
2024-05-30 21:06:46 +02:00
Paolo Tranquilli
2606d3f6d1 Reapply "Bazel: add codeql specific packaging library"
This reverts commit 31d6b9be4d.
2024-05-29 15:46:29 +02:00
Paolo Tranquilli
31d6b9be4d Revert "Bazel: add codeql specific packaging library" 2024-05-29 15:27:10 +02:00
Paolo Tranquilli
5672ddf8f3 Fix bazel formatting 2024-05-29 09:53:31 +02:00
Paolo Tranquilli
fbe1b56f2d Zipmerge: link test statically 2024-05-29 08:55:06 +02:00
Paolo Tranquilli
332d178298 Zipmerge: allow test to be run from internal repo 2024-05-28 17:37:34 +02:00
Paolo Tranquilli
5eb12b8503 Zipmerge: substitute stripped down slf4j jars with dummy ones 2024-05-28 16:15:20 +02:00
Paolo Tranquilli
c3ccf4d5a3 Zipmerge: substitute CPython archives with dummy ones 2024-05-28 15:47:24 +02:00
Paolo Tranquilli
67d622fa9d Bazel: actually run the zipmerge tests 2024-05-28 15:44:53 +02:00
Paolo Tranquilli
00ed00e1e5 Bazel: avoid unneeded operations if no imported zips are present 2024-05-28 15:01:35 +02:00
Paolo Tranquilli
6d798410ce Bazel: add --no-cleanup to installer script 2024-05-28 12:51:52 +02:00
Paolo Tranquilli
a8543d4a88 Zipmerge: port tests from internal repo 2024-05-28 12:01:57 +02:00
Paolo Tranquilli
392ef09d19 Zipmerge: make lib public for internal testing 2024-05-27 17:28:33 +02:00
Paolo Tranquilli
cde71a915b Bazel: address review comments 2024-05-27 17:00:39 +02:00
Paolo Tranquilli
2f95944244 Bazel: add documentation to install.py 2024-05-27 16:58:58 +02:00
Paolo Tranquilli
2f53c0e640 Bazel: fix codeql_pack installation on Windows 2024-05-27 16:09:42 +02:00
Paolo Tranquilli
5d4b61c365 Bazel: replace prebuilt ripunzip from workflow 2024-05-24 17:44:39 +02:00
Paolo Tranquilli
b9064c5446 Bazel: fail install on ripunzip failing 2024-05-24 15:50:16 +02:00
Paolo Tranquilli
fa2c626e49 Bazel: add fat macOS ripunzip binary 2024-05-24 14:29:02 +02:00
Paolo Tranquilli
dcbf42d29c Bazel: reorganize LFS files and add licensing information 2024-05-24 14:26:22 +02:00
Paolo Tranquilli
e8b857b79e Bazel/Swift: add zip imports to packs 2024-05-24 12:18:42 +02:00
Paolo Tranquilli
60cf77be7e Bazel: add codeql specific packaging library
This encapsulate arch specific logic, local installation and separation
of zip files into generic and arch-specific parts as required by the
internal build.
2024-05-24 10:27:44 +02:00
Paolo Tranquilli
d01d657f89 Bazel: accept new SSH keys in git_lfs_probe.py 2024-05-17 16:39:18 +01:00
Paolo Tranquilli
170e2231d4 Bazel: allow LFS rules to use cached downloads without internet
If the cache is prefilled, LFS rules were still trying to query LFS
urls.

Now the strategy is to first try to fetch the files from the repository
cache (which is possible by providing an empty url list and `allow_fail`
to `repository_ctx.download`), and only run the LFS protocol if that
fails. Technically this is possible by enhancing `git_lfs_probe.py` with
a `--hash-only` flag.

This is also an optimization where no uneeded access is done (including
the slightly slow SSH call) if the repository cache is warm.
2024-05-17 16:24:38 +01:00
Óscar San José
9105faa3fd make sure GITHUB_TOKEN exists and is not empty before using it in auth headers 2024-05-14 11:22:07 +02:00
Óscar San José
faa2dcee24 test GITHUB_TOKEN non-emptyness before using it in auth headers 2024-05-14 11:18:18 +02:00
Óscar San José
5b572a2c2a make GITHUB_TOKEN in lfs_probe a fallback rather than an override 2024-05-14 10:24:02 +02:00
Paolo Tranquilli
9f27eb3eda Bazel: make git_lfs_probe.py compatible with python 3.8 2024-05-07 12:44:28 +02:00
Paolo Tranquilli
54c9aea251 Bazel: fix lfs lazy rule when all objects are local 2024-05-06 15:47:43 +02:00
Paolo Tranquilli
ecdf62376d Bazel: clean up git_lfs_probe.py 2024-05-03 09:21:57 +02:00
Paolo Tranquilli
daea674095 Bazel: cover standard https git credentials in git_lfs_probe.py 2024-05-02 18:02:22 +02:00
Paolo Tranquilli
677520aa8e Bazel: improved lazy lfs files
This reintroduces lazy lfs file rules that were removed in
https://github.com/github/codeql/pull/16117, now improved.

The new rules will make the actual file download go through bazel's
download manager, which includes:
* caching into the repository cache
* sane limiting of concurrent downloads
* retries

The bulk of the work is done by `git_lfs_probe.py`, which will use the
LFS protocol (with authentication via SSH) to output short lived
download URLs that can be consumed by `repository_ctx.download`.
2024-05-02 06:38:26 +02:00