This collapses all generated test QL sources into a single one per
directory, using query predicates to run the different tests.
This should improve the time required to run generated tests.
This improves the implementation of the generated parent/child
relationship by adding a new `all_children` field to `ql.Class` which
lists all children (both direct and inherited) of a class, carefully
avoiding duplicating children in case of diamond inheritance. This:
* simplifies the generated code,
* avoid children ambiguities in case of diamond inheritance.
This only comes with some changes in the order of children in the
generated tests (we were previously sorting bases alphabetically there).
For the rest this should be a non-functional change.
This adds the possibility to add a special `proc_macro.rs` source file
to QL tests, which will be generated into a `proc_macro` crate the
usual `lib` crate depends on.
This allow to define procedural macros in QL tests, and is here used to
move the `macro-expansion` integration test to be a language test
instead.
As the generated manifests involved were starting to get a bit complex,
they are now generated from a `mustache` template.
Turns out this is important for the black formatting tool to work
correctly. The formatting won't generally change between python versions
(it only depends on `black`'s version), but the formatted code needs to
be parseable by the system python version. One script uses
```python
def foo[T](x: T) -> T:
```
syntax, which is only supported in Python 3.12 and later.
Running `pre-commit` will now require a python 3.12 installation (which
is already what we mandate for internal developer environment setup).
The error in case of absence of such a version is pretty clear though.
The standalone MaD generator now uses `0` for threads and throttles the
RAM to use 2GB per thread by default.
Also, replaced the hand-written argument parsing with `argparse`.
* fix a bug where the order of model generation was determined by the
order in the `download.json` file of the experiment rather than the
order in the config file
* allow configuring `--ram` and `--threads` in the MaD generator scripts
* use no `--ram` and `--threads=0` by default in the bulk generator
(single generator defaults are left unchanged)
* allow to pass `--dca` multiple times, taking DBs from experiments
listed last. This allows to run a subset of the sources in a "fixup"
experiment and use it to "patch" a previous run without rerunning
everything.
* fix a bug where the order of model generation was determined by the
order in the `download.json` file of the experiment rather than the
order in the config file
* allow configuring `--ram` and `--threads` in the MaD generator scripts
* use no `--ram` and `--threads=0` by default in the bulk generator
(single generator defaults are left unchanged)
* allow to pass `--dca` multiple times, taking DBs from experiments
listed last. This allows to run a subset of the sources in a "fixup"
experiment and use it to "patch" a previous run without rerunning
everything.
This switched `codegen` from the `autopep8` formatting to the `black`
one, and applies it to `bulk_mad_generator.py` as well. We can enroll
more python scripts to it in the future.
