Previously, we were using 8.0.0rc1.
In particular, this upgrade means we need to explicitly
import more rules, as they've been moved out of the core bazel repo.
This PR introduces a `codeql_pack_rule` that does the heavy lifting
of extracting arch- and common zip files for production dist building.
It also factors out the installer targets for individual packs,
as well as pack groups.
This changes the contract between the internal build system and the pack
definition significantly, which is why an accompanying internal PR is required.
No backwards compatibility layer is provided, as the PR as complex enough as-is.
The individual `codeql_pack` rules are now much simpler,
as they mostly stuff their inputs into a new `_CodeQLPackInfo` provider,
and let the installer and `codeql_pack_group` rules do the heavy lifting.
For working in the external repo with self-contained packs,
the per-pack installer targets are still available.
Internally, we'll only use the new `codeql_pack_group` targets
going forward, both for defining intree-dists and for building
the production zip files.
This allows the bazel cache to cache this genrule invocation.
It shouldn't depend on system-specific binaries, so
I believe this is correct.
This is the only part of our build where we otherwise
need to recompute parts after pulling in a full cache.
The test sdk that we were prebuilding to run ql tests is actually not
needed, as the `resource-dir` we package for cross-version compatibility
is enough for running qltests as well.
This fetches the resource directory directly from the released
toolchains, allowing us to stop prebuilding and assembling them.
Moreover insertion of our resource directory is moved to the lua
tracing configuration (solving a `TODO`) and enhanced. Now all options
that start with the original resource directory (either explicit or
implied) are redirected to our resource directory.
This solves a problem where `-I <original resource dir>/some/path` was
passed to the extractor and did not work.
This works around the 5.9 linux compatibility problem by including the
`PackageDescription` swift modules in the in-dist toolchain. Copying the
toolchain and fixing the `-I` flag was not enough as for some reason
compilation of `PackageDescription.swiftinterface` was causing a crash
in the SIL pass. We work around that by pre-compiling those modules
during the build and including `.swiftmodule` files in the resource
directory.
TODO (apart from testing):
* the libraries included in the macOS toolchain are now fat (they were
intel only before), occupying more space. We should see if we need to
trim them down.
* there might be other swiftinterface files causing problems on linux
lurking around...
* if we go with this, we can simplify and trim down the prebuilding we
do leaving out the resource directory.
`std::result_of` was removed in C++20, though the actual removal from
the STL library implementations seems to depend on the version. For
example using xcode 14.2 one gets away with a deprecation warning, but
xcode 14.3 will fail.
As Swift 5.8.1 is still compiled with C++14, we cannot replace
`std::result_of` with `std::invoke_result` in the prebuilding patches
just yet, but we can do that for the extractor itself, patching the
prebuilt package.
This required a bit of an overhaul of the original integration of
JSON diagnostics into binlog.
The problem is that it is quite hard to add a kind of metadata to
binlog entries without changing its code. Another problem is that when
wanting to avoid double evaluation of logging macro arguments one
cannot really add a separate "diagnose" step easily.
The proposed solution consists in two things:
* hook into a binlog plumbing function by providing a better overload
resolution match, which happens after logging macro expansion,
bypassing the problem of double evaluation
* in that hook, produce the diagnostic directly, without waiting to
reconstruct the diagnostics entry from the binlog serialized entry.
This allows to forgo the weird category to diagnostic mapping, and now a
diagnostics emission simply happens when a diagnostic source is given
as the first argument after the log format string. A flavour of
diganostics sources with locations is then added with the same
mechanism, allowing to write something like
```cpp
LOG_ERROR("[{}] ouch!", internalError.withLocation("foo.swift", 32));
```
