This fetches the resource directory directly from the released
toolchains, allowing us to stop prebuilding and assembling them.
Moreover insertion of our resource directory is moved to the lua
tracing configuration (solving a `TODO`) and enhanced. Now all options
that start with the original resource directory (either explicit or
implied) are redirected to our resource directory.
This solves a problem where `-I <original resource dir>/some/path` was
passed to the extractor and did not work.
This works around the 5.9 linux compatibility problem by including the
`PackageDescription` swift modules in the in-dist toolchain. Copying the
toolchain and fixing the `-I` flag was not enough as for some reason
compilation of `PackageDescription.swiftinterface` was causing a crash
in the SIL pass. We work around that by pre-compiling those modules
during the build and including `.swiftmodule` files in the resource
directory.
TODO (apart from testing):
* the libraries included in the macOS toolchain are now fat (they were
intel only before), occupying more space. We should see if we need to
trim them down.
* there might be other swiftinterface files causing problems on linux
lurking around...
* if we go with this, we can simplify and trim down the prebuilding we
do leaving out the resource directory.
`std::result_of` was removed in C++20, though the actual removal from
the STL library implementations seems to depend on the version. For
example using xcode 14.2 one gets away with a deprecation warning, but
xcode 14.3 will fail.
As Swift 5.8.1 is still compiled with C++14, we cannot replace
`std::result_of` with `std::invoke_result` in the prebuilding patches
just yet, but we can do that for the extractor itself, patching the
prebuilt package.
This required a bit of an overhaul of the original integration of
JSON diagnostics into binlog.
The problem is that it is quite hard to add a kind of metadata to
binlog entries without changing its code. Another problem is that when
wanting to avoid double evaluation of logging macro arguments one
cannot really add a separate "diagnose" step easily.
The proposed solution consists in two things:
* hook into a binlog plumbing function by providing a better overload
resolution match, which happens after logging macro expansion,
bypassing the problem of double evaluation
* in that hook, produce the diagnostic directly, without waiting to
reconstruct the diagnostics entry from the binlog serialized entry.
This allows to forgo the weird category to diagnostic mapping, and now a
diagnostics emission simply happens when a diagnostic source is given
as the first argument after the log format string. A flavour of
diganostics sources with locations is then added with the same
mechanism, allowing to write something like
```cpp
LOG_ERROR("[{}] ouch!", internalError.withLocation("foo.swift", 32));
```
An interesting byproduct was finding a problematic `assert` in the
Swift headers. An incomplete `FallthroughStmt` was asserting on having
a destination. I did not find any other sensible way of getting rid of
the crash when running in debug mode than to patch the header.
This replaces usages of `llvm::fs` and string manipulation with
`std::filesystem`, also replacing `std::string` with
`std::filesystem::path` where it made sense.
Moreover MD5 hashing used in macOS file remapping was replaced by
SHA256 hashing using a small header-only SHA256 C++ library with an
MIT license, https://github.com/okdshin/PicoSHA2.
File contents hashing was relocated to the newly created `file` library
for later planned reuse.
