It turns out the threshold of 5 lines for stub modification detection
was too strict: in case of a long class name the QL formatter will put
the closing brace of the empty class definition on a new line, leading
to codegen fail with an error thinking the stub was modified.
On the other side of things, also adding a base to a stub class was not
being detected as a modification.
Now the modification test is slightly smarter. If the stub still marked
as generated and
* has more than 6 lines, or
* the contents does not match a regexp aproximation of a plain stub
then codegen will abort. The test will still avoid reading the whole
contents of all the stubs.
If one modifies a QL stub but forgets to remove the `// generated`
header comment, codegen will now abort with an error rather than
silently reverting the change.
This is based on the rough heuristic of just counting the lines. If any
change is done to the stub class, the number of lines is bound to be
5 or more.
This adds:
* a base `README.md` file to `codegen`
* module docstrings for the modules in `generators`
* help strings on all command line flags
Moreover some unneeded command line flags (`--namespace`,
`--include-dir` and `--trap-affix`) have been dropped.
By explicitly marking children in the `schema.yml` file, an internal
`getAChild` predicate is implemented, that is in turn used in `AstNode`
to implement `getParent`.
This is yet to be used in the control flow library to replace the
hand-rolled implementation.
A further, more complex step is to use the same information to fully
generate the core implementation of `PrintAst` (including the
accessor string). This will be done later.
The `parent` tests use the same swift code as the extractor tests, and
this is currently enforced by `sync-files.py`. Notice that `qltest.sh`
had to be modified to deal with multiple files, which was not working
yet.
This is solving a papercut, where the C++ build was relying on the
local dbscheme file to be up-to-date, even if all the information for
building is actually in `schema.yml`. This made a pure C++ development
cycle with changes to `schema.yml` clumsy, as it required a further
dbscheme generation step.
Now for C++ the dbscheme is generated internally in the build files, and
thus a change in `schema.yml` is reflected immediately in the C++ build.
A `swift/codegen` step for checked in generated code (including the
dbscheme) is still required, but a developer can do it just before
running QL tests or committing, instead of during each C++
recompilation.
Some directory reorganization was also carried out, moving specific
generator modules to a new `generators` python package, and only leaving
the two drivers at the top level.
These changes are required to allow a new type-safe approach to TBD
nodes, that will come in a separate commit.
This introduces:
* the possibility to add properties to the root `Element`
* a functor taking tags to the corresponding binding trap entry
* `hasProp()` methods for optional properties in QL
* `getPrimaryQlClass()` method
Properties marked with `predicate` in the schema are now accepted.
* in the dbscheme, they will translate to a table with a single `id`
column (and the table name will not be pluralized)
* in C++ classes, they will translate to `bool` fields
* in QL classes, they will translate to predicates
Closes https://github.com/github/codeql-c-team/issues/1016
This checks in the trapgen script generating trap entries in C++.
The codegen suite has been slightly reorganized, moving the templates
directory up one level and chopping everything into smaller bazel
packages. Running tests is now done via
```
bazel run //swift/codegen/test
```
With respect to the PoC, the nested `codeql::trap` namespace has been
dropped in favour of a `Trap` prefix (or suffix in case of entries)
within the `codeql` namespace. Also, generated C++ code is not checked
in in git any more, and generated during build. Finally, labels get
printed in hex in the trap file.
`TrapLabel` is for the moment only default-constructible, so only one
single label is possible. `TrapArena`, that is responsible for creating
disjoint labels will come in a later commit.
Tests can be run with
```
bazel test //swift/codegen:tests
```
Coverage can be checked installing `pytest-cov` and running
```
pytest --cov=swift/codegen swift/codegen/test
```
