2cd58817d7
This is a developer QoL improvement, where running codegen will skip writing (and especially formatting) any files that were not changed. **Why?** While code generation in itself was pretty much instant, QL formatting of generated code was starting to take a long time. This made unconditionally running codegen quite annoying, for example before each test run as part of an IDE workflow or as part of the pre-commit hook. **How?** This was not completely straightforward as we could not work with the contents of the file prior to code generation as that was already post-processed by the QL formatting, so we had no chance of comparing the output of template rendering with that. We therefore store the hashes of the files _prior_ to QL formatting in a checked-in file (`swift/ql/.generated.list`). We can therefore load those hashes at the beginning of code generation, use them to compare the template rendering output and update them in this special registry file. **What else?** We also extend this mechanism to detect accidental modification of generated files in a more robust way. Before this patch, we were doing it with a rough regexp based heuristic. Now, we just store the hashes of the files _after_ QL formatting in the same checked file, so we can check that and stop generation if a generated file was modified, or a stub was modified without removing the `// generated` header.
Code generation suite
This directory contains the code generation suite used by the Swift extractor and the QL library. This suite will use
the abstract class specification of schema.yml to generate:
- the
dbschemefile (seedbschemegen.py) - the QL generated code and when
appropriate the corresponding stubs (see
qlgen.py) - C++ tags and trap entries (see
trapgen.py) - C++ structured classes (see
cppgen.py)
Usage
By default bazel run //swift/codegen will update all checked-in generated files (dbscheme and QL sources). You can
append -- followed by other options to tweak the behaviour, which is mainly intended for debugging.
See bazel run //swift/codegen -- --help for a list of all options. In particular --generate can be used with a comma
separated list to select what to generate (choosing among dbscheme, ql, trap and cpp).
C++ code is generated during build (see swift/extractor/trap/BUILD.bazel). After a
build you can browse the generated code in bazel-bin/swift/extractor/trap/generated.
Implementation notes
The suite uses mustache templating for generation. Templates are
in the templates directory, prefixed with the generation target they are used for.
Rather than passing dictionaries to the templating engine, python dataclasses are used as defined
in the lib directory. For each of the four generation targets the entry point for the implementation is
specified as the generate function in the modules within the generators directory.
Finally, codegen.py is the driver script gluing everything together and specifying the command line
options.
Unit tests are in the test directory and can be run via bazel test //swift/codegen/test.
For more details about each specific generation target, please refer to the module docstrings
in the generators directory.