* do not mention any more that one might make analysis happen on Linux with
advanced setup
* say that outright Swift analysis is only supported on macOS, not just
autobuild.
* emit the error diagnostics even for traced builds, not only for autobuilds
(by using a dummy `extractor` executable).
Previously, we were using 8.0.0rc1.
In particular, this upgrade means we need to explicitly
import more rules, as they've been moved out of the core bazel repo.
This encapsulate arch specific logic, local installation and separation
of zip files into generic and arch-specific parts as required by the
internal build.
This introduces tooling and enforcement for formatting bazel files.
The tooling is provided as a bazel run target from
[keith/buildifier-prebuilt](https://github.com/keith/buildifier-prebuilt).
This is used in a [`pre-commit`](https://pre-commit.com/) hook for those
having that installed. In turn this is used in a CI check. Relying on a
`pre-commit` action gives us easy checking that buildifying did not
change anything in the files and printing the diff, without having to
hand-roll the check ourselves.
This enforcement will make usage of gazelle easier, as gazelle itself
might reformat files, even outside of `go`. Having them properly
formatted will allow gazelle to leave them unchanged, without needing
to configure awkward exclude directives.
The bazel -> cmake generator is currently not capable of handling
separate included generated cmake files making use of common C/C++
dependencies.
To work around this limitation, a single generated cmake is now in
place. Long-term, we should either:
* make the cmake generator handle common dependencies gracefully, or
* make the cmake generation aspect travel up `pkg_` rules `srcs`
attributes
so to avoid having to list the targets to be generated in the top-level
`BUILD` file.
Other things fixed:
* removed some warning spam about redefined `BAZEL_CURRENT_REPOSITORY`
* fixed the final link step, that was failing because `libswiftCore.so`
was not being linked.
* visiting now happens in a later stage than fetching labels. While
fetching a list of entities to be visited is created, and then acted
upon in actual extraction. This partially flattens the recursive
nature of `fetchLabel` into a loop inside `SwiftVisitor::extract`.
Recursion in `fetchLabel` will only happen on labels fetched while
naming an entity (calling into `SwiftMangler`).
* The choice whether to name a declaration or type has been moved from
the translators to `SwiftMangler`. Acting on this choice is contained
in `SwiftDispatcher::createLabel`.
* The choice whether to emit a body of a declaration has been moved from
`DeclTranslator` to the dispatcher. This choice is also contained in
`SwiftDispatcher::createLabel`.
* The simple functionality of the `LabelStore` has been moved to the
`SwiftDispatcher` as well.
This also removes the TODO about using `absl::StrJoin` to dump the environment because we can't easily get a range from a null-terminated `envp`. It also doesn't suffer from the usual awkwardness around inserting a separator *between* elements but not after the last one, so a for loop is clear enough.
This is in preparation for the extractor to use shared libraries
packaged alongside it.
We could probably also move the `CODEQL_EXTRACTOR_SWIFT_RUN_UNDER` logic
in it, where it would be simpler and more robust.
The hash map mechanism that was already in use for reading swiftmodule
files on macOS is now in use also on Linux. The output replacing
mechanism has been also reworked so that:
* frontend module emission modes have the remapping done directly in
the internal frontend options instead of painstakingly modifying input
flags (this requires a patch on the swift headers though)
* object emission mode is silenced to be just a type checking pass,
thus producing no output files
* all other passes but some debugging and version related ones become
noops
The open file read redirection uses a global weak pointer instance to
maximize robustness in the face of possibly multi-threaded calls to open
happening while `main` is exiting. Possibly overkill, but better safe
than sorry.
A cmake generator in bazel is introduced allowing to import the Swift
extractor as a CMake project while keeping Bazel files as the source of
truth for the build.
Using the CMake project:
* requires bazel and clang to be installed and available on the command
line
* does not require a previous bazel build, however
* will require a CMake reconfiguration for changes to generated code
(like changes to the schema)
This should cover `-merge-modules` mode.
Dumping of the configuration to the target files was moved to a
separate pair of header/source files, as now it is also done in
`SwiftOutputRewrite.cpp`.
Visitor code has been split between header and sources to speed up
incremental build. Moreover the code was reorganized using a new `infra`
bazel package (and `visitors` got promoted to a bazel package as well).
This allows to avoid bypassing label type correcness in the extractor,
and allows to independently resolve TBD extractions, as with this
approach TBD nodes do have the correctly typed trap label. The TBD
status is now a predicate on the QL side.
This requires:
* a default visit using the correct type, which is achieved via macro
metaprogramming in `VisitorBase.h`, following the way
`swift::ASTVisitor` is programmed
* a mapping from labels to corresponding binding trap entries. The
functor is defined in `TrapTagTraits.h` and instantiated in generated
`TrapEntries.h`
* Binding trap entries for TBD unknown entities must not have any other
field than the `id` (after all, we are supposed to not extract them
yet). This is why all unextracted fields in `schema.yml` have been
commented out, and will be uncommentend when visitors are added
Providing `--dynamic_mode=fully` (for example setting it in
`local.bazelrc`) will now work.
All runfiles are now copied in the extractor pack: in dynamic mode,
those will be the executable and the dynamic libraries, while in static
mode only the executable will be part of the runfiles.
Setting the correct `LD_LIBRARY_PATH` in `qltest.sh` then allows to
run tests with this pakcage. If we need something more, we can switch to
a wrapper script in place of `extractor` in the future.
Notice that `LD_LIBRARY_PATH` is also set in static mode, but that has
no consequence.
This checks in the trapgen script generating trap entries in C++.
The codegen suite has been slightly reorganized, moving the templates
directory up one level and chopping everything into smaller bazel
packages. Running tests is now done via
```
bazel run //swift/codegen/test
```
With respect to the PoC, the nested `codeql::trap` namespace has been
dropped in favour of a `Trap` prefix (or suffix in case of entries)
within the `codeql` namespace. Also, generated C++ code is not checked
in in git any more, and generated during build. Finally, labels get
printed in hex in the trap file.
`TrapLabel` is for the moment only default-constructible, so only one
single label is possible. `TrapArena`, that is responsible for creating
disjoint labels will come in a later commit.
This adds a first dummy extractor for swift.
Running `bazel run //swift:install` will create an `extractor_pack`
directory in `swift`. From that moment providing `--search-path=swift`
will pick up the extractor.
