File hashing is now done internally in `SwiftFileInterception` (and
exported as a `getHashOfRealFile` function for future use in linkage
awareness), and using a per-process in-memory cache. The persistent
caching of paths is removed, so the solution is now robust against input
file changes during the build.
For the same reason, the hash to artifact mapping have the symlinks
reversed now. The artifacts themselves are stored using the hash as
filenames, and the original paths of the artifacts are reacreated in the
scratch dir with symlinks mostly for debugging purposes (to understand
what artifact each hash corresponds to, and to follow what was built by
the extractor).
The hash map mechanism that was already in use for reading swiftmodule
files on macOS is now in use also on Linux. The output replacing
mechanism has been also reworked so that:
* frontend module emission modes have the remapping done directly in
the internal frontend options instead of painstakingly modifying input
flags (this requires a patch on the swift headers though)
* object emission mode is silenced to be just a type checking pass,
thus producing no output files
* all other passes but some debugging and version related ones become
noops
The open file read redirection uses a global weak pointer instance to
maximize robustness in the face of possibly multi-threaded calls to open
happening while `main` is exiting. Possibly overkill, but better safe
than sorry.
if `CODEQL_EXTRACTOR_SWIFT_RUN_UNDER` env variable is set, and either
* `CODEQL_EXTRACTOR_SWIFT_RUN_UNDER_FILTER` is not set, or
* it is set to a regexp matching any substring of the extractor call
then the extractor process is substituted with the command (and possibly
options) stated in `CODEQL_EXTRACTOR_SWIFT_RUN_UNDER`, followed by the
system arguments of the extractor itself (which should include the
extractor program itself at the start).
Before calling `exec`, `CODEQL_EXTRACTOR_SWIFT_RUN_UNDER` is unset to
avoid unpleasant loops.
An example usage is to run the extractor under `gdbserver :1234` when
the arguments match a given source file.
This replaces usages of `llvm::fs` and string manipulation with
`std::filesystem`, also replacing `std::string` with
`std::filesystem::path` where it made sense.
Moreover MD5 hashing used in macOS file remapping was replaced by
SHA256 hashing using a small header-only SHA256 C++ library with an
MIT license, https://github.com/okdshin/PicoSHA2.
File contents hashing was relocated to the newly created `file` library
for later planned reuse.
This should cover `-merge-modules` mode.
Dumping of the configuration to the target files was moved to a
separate pair of header/source files, as now it is also done in
`SwiftOutputRewrite.cpp`.
Previously we were not extracting any `swiftmodule` file that was not
a system or a built-in one. This was done to avoid re-extracting
`swiftmodule` files that were built previously in the same build, but it
turned out to be too eager, as there are legitimate cases where a
non-system, non-built-in precompiled swift module can be used. An
example of that is the `PackageDescription` module used in Swift
Package Manager manifest files (`Package.swift`).
We now relax the test and trigger module extraction on all loaded
modules that do not have source files (we trigger source file extraction
for those). The catch, is that we also create empty trap files for
current output `swiftmodule` files (including possible alias locations
set up by XCode).
This means that if a following extractor run loads a previously built
`swiftmodule` file, although it will trigger module extraction, this
will however be skipped as it will find its target file already present
(this is done via the `TargetFile` semantics).
Currently, we have a number of assertions in the codebase and certain
assumptions about the AST. These don't always hold, sometimes leading to
a crash in the extractor.
The crashes leave incomplete TRAP files that cannot be imported into the
database.
With this change, we still get those incomplete TRAP files, but we also
get a database in the end (even thoough it is also incomplete as we
cannot import everything).
This adds a first dummy extractor for swift.
Running `bazel run //swift:install` will create an `extractor_pack`
directory in `swift`. From that moment providing `--search-path=swift`
will pick up the extractor.
