This transfers the current state of `StmtVisitor` in the PoC, plus some
changes required for the update to swift 5.6.
Also `getLabel` in `SwiftDispatcher` got renamed to `createLabel`, and
is now correctly outputting the label assignment to the trap file.
This transfers the current state of `DeclVisitor` from the
proof-of-concept.
TODO: make the `declarations` tests in `extractor-tests` more
comprehensive.
This allows to avoid bypassing label type correcness in the extractor,
and allows to independently resolve TBD extractions, as with this
approach TBD nodes do have the correctly typed trap label. The TBD
status is now a predicate on the QL side.
This requires:
* a default visit using the correct type, which is achieved via macro
metaprogramming in `VisitorBase.h`, following the way
`swift::ASTVisitor` is programmed
* a mapping from labels to corresponding binding trap entries. The
functor is defined in `TrapTagTraits.h` and instantiated in generated
`TrapEntries.h`
* Binding trap entries for TBD unknown entities must not have any other
field than the `id` (after all, we are supposed to not extract them
yet). This is why all unextracted fields in `schema.yml` have been
commented out, and will be uncommentend when visitors are added
This is solving a papercut, where the C++ build was relying on the
local dbscheme file to be up-to-date, even if all the information for
building is actually in `schema.yml`. This made a pure C++ development
cycle with changes to `schema.yml` clumsy, as it required a further
dbscheme generation step.
Now for C++ the dbscheme is generated internally in the build files, and
thus a change in `schema.yml` is reflected immediately in the C++ build.
A `swift/codegen` step for checked in generated code (including the
dbscheme) is still required, but a developer can do it just before
running QL tests or committing, instead of during each C++
recompilation.
Some directory reorganization was also carried out, moving specific
generator modules to a new `generators` python package, and only leaving
the two drivers at the top level.
This allows to avoid bypassing label type correcness in the extractor,
and allows to independently resolve TBD extractions, as with this
approach TBD nodes do have the correctly typed trap label. The TBD
status is now a predicate on the QL side.
This requires:
* a default visit using the correct type, which is achieved via macro
metaprogramming in `VisitorBase.h`, following the way
`swift::ASTVisitor` is programmed
* a mapping from labels to corresponding binding trap entries. The
functor is defined in `TrapTagTraits.h` and instantiated in generated
`TrapEntries.h`
* Binding trap entries for TBD unknown entities must not have any other
field than the `id` (after all, we are supposed to not extract them
yet). This is why all unextracted fields in `schema.yml` have been
commented out, and will be uncommentend when visitors are added
This turned out easier than expected previously. `llvm::PointerUnion`
was also considered, which would have less memory footprint, but it
would require more effort as it is lacking the same implicit conversions
and operators that `std::variant` provides.
Also renamed `ToTag<E>` to `TrapTagOf<E>` and introduced a derived
convenience functor `TrapLabelOf<E>`.
These changes are required to allow a new type-safe approach to TBD
nodes, that will come in a separate commit.
This introduces:
* the possibility to add properties to the root `Element`
* a functor taking tags to the corresponding binding trap entry
* `hasProp()` methods for optional properties in QL
* `getPrimaryQlClass()` method
Providing `--dynamic_mode=fully` (for example setting it in
`local.bazelrc`) will now work.
All runfiles are now copied in the extractor pack: in dynamic mode,
those will be the executable and the dynamic libraries, while in static
mode only the executable will be part of the runfiles.
Setting the correct `LD_LIBRARY_PATH` in `qltest.sh` then allows to
run tests with this pakcage. If we need something more, we can switch to
a wrapper script in place of `extractor` in the future.
Notice that `LD_LIBRARY_PATH` is also set in static mode, but that has
no consequence.
This checks in the trapgen script generating trap entries in C++.
The codegen suite has been slightly reorganized, moving the templates
directory up one level and chopping everything into smaller bazel
packages. Running tests is now done via
```
bazel run //swift/codegen/test
```
With respect to the PoC, the nested `codeql::trap` namespace has been
dropped in favour of a `Trap` prefix (or suffix in case of entries)
within the `codeql` namespace. Also, generated C++ code is not checked
in in git any more, and generated during build. Finally, labels get
printed in hex in the trap file.
`TrapLabel` is for the moment only default-constructible, so only one
single label is possible. `TrapArena`, that is responsible for creating
disjoint labels will come in a later commit.
