Paolo Tranquilli
492b0cd2e8
Bazel: regenerate cargo vendored files
2025-06-20 17:37:58 +02:00
Tamas Vajk
40274dcd69
Add code-quality-extended query suites
2025-06-18 13:10:34 +02:00
Paolo Tranquilli
299fe2eb3f
Rust: add proc-macro capabilities to QL tests
...
This adds the possibility to add a special `proc_macro.rs` source file
to QL tests, which will be generated into a `proc_macro` crate the
usual `lib` crate depends on.
This allow to define procedural macros in QL tests, and is here used to
move the `macro-expansion` integration test to be a language test
instead.
As the generated manifests involved were starting to get a bit complex,
they are now generated from a `mustache` template.
2025-06-17 16:58:33 +02:00
Paolo Tranquilli
fe1e562f8d
CI: fix python version
...
Turns out this is important for the black formatting tool to work
correctly. The formatting won't generally change between python versions
(it only depends on `black`'s version), but the formatted code needs to
be parseable by the system python version. One script uses
```python
def foo[T](x: T) -> T:
```
syntax, which is only supported in Python 3.12 and later.
Running `pre-commit` will now require a python 3.12 installation (which
is already what we mandate for internal developer environment setup).
The error in case of absence of such a version is pretty clear though.
2025-06-13 18:19:30 +02:00
Paolo Tranquilli
55eb9fb838
Merge pull request #19712 from github/redsun82/python-black
...
Add `black` pre-commit hook
2025-06-12 20:49:56 +02:00
Paolo Tranquilli
22c490ce4e
Merge pull request #19667 from github/redsun82/rust-mad
...
Rust: Use QL computed canonical paths in MaD `Field` tokens
2025-06-12 20:48:41 +02:00
Paolo Tranquilli
ad7fb9c061
Merge branch 'main' into redsun82/rust-has-implementation
2025-06-12 15:25:59 +02:00
Paolo Tranquilli
c6c52edbee
MaD generator: tweak the scripts
...
* fix a bug where the order of model generation was determined by the
order in the `download.json` file of the experiment rather than the
order in the config file
* allow configuring `--ram` and `--threads` in the MaD generator scripts
* use no `--ram` and `--threads=0` by default in the bulk generator
(single generator defaults are left unchanged)
* allow to pass `--dca` multiple times, taking DBs from experiments
listed last. This allows to run a subset of the sources in a "fixup"
experiment and use it to "patch" a previous run without rerunning
everything.
2025-06-11 16:42:00 +02:00
Paolo Tranquilli
051704e59d
Merge branch 'main' into redsun82/rust-mad
2025-06-11 16:39:10 +02:00
Paolo Tranquilli
14d48e9d58
Add black pre-commit hook
...
This switched `codegen` from the `autopep8` formatting to the `black`
one, and applies it to `bulk_mad_generator.py` as well. We can enroll
more python scripts to it in the future.
2025-06-10 12:25:39 +02:00
Paolo Tranquilli
ca99add7d6
MaD generator: address review
2025-06-10 12:14:39 +02:00
Paolo Tranquilli
ecc35e5a07
MaD generator: run formatter
2025-06-10 12:11:00 +02:00
Paolo Tranquilli
bcfc009228
Merge branch 'main' into redsun82/mad
2025-06-10 10:53:52 +02:00
github-actions[bot]
21463a9653
Post-release preparation for codeql-cli-2.22.0
2025-06-09 18:50:20 +00:00
github-actions[bot]
88ba02edf8
Release preparation for version 2.22.0
2025-06-09 18:14:51 +00:00
Chuan-kai Lin
631502e129
Merge branch 'main' into cklin/rc-3.18-mergeback
2025-06-09 07:19:40 -07:00
Paolo Tranquilli
d6d13b9164
MaD generator: use decompress terminology instead of extract
2025-06-06 17:36:05 +02:00
Paolo Tranquilli
e1eb1f6345
Rust: address review
...
Also fix some minor things in `bulk_generate_mad.py`.
2025-06-05 16:03:40 +02:00
Paolo Tranquilli
530b990dd5
MaD generator: some final minor tweaks
2025-06-05 10:08:01 +02:00
Paolo Tranquilli
ee7eb86a1d
MaD: make bulk generator cleanup downloaded DBs
2025-06-05 09:31:24 +02:00
Paolo Tranquilli
4f47ee2e72
MaD: make bulk generator DCA strategy download DBs in parallel
2025-06-05 09:30:02 +02:00
Paolo Tranquilli
31954fa794
MaD generator: make bulk generator executable
2025-06-05 08:49:15 +02:00
Paolo Tranquilli
d5c16d6092
MaD generator: reformat
2025-06-05 08:48:01 +02:00
Paolo Tranquilli
900a3b0992
MaD generator: only pick up last database on comparison DCAs
2025-06-05 08:47:45 +02:00
Paolo Tranquilli
31d1604337
Bulk model generator: switch from json to yml configuration files
2025-06-05 08:37:43 +02:00
Paolo Tranquilli
5f474a7185
Rust: Use QL computed canonical paths in MaD Field tokens
...
Also regenerate all auto-generated models with latest extractor.
2025-06-04 09:43:45 +02:00
Paolo Tranquilli
77c40da51a
Rust/Codegen: fix detached predicates
2025-06-03 10:41:31 +02:00
Paolo Tranquilli
bd2573a2b0
Ripunzip: fix macos archive
2025-06-02 18:20:29 +02:00
Paolo Tranquilli
ddbe29a8e2
Ripunzip: update to 2.0.2
2025-06-02 18:04:57 +02:00
Mathias Vorreiter Pedersen
f6231a37e1
Merge pull request #19627 from MathiasVP/generalize-bulk-generation
...
Bulk MAD generator: Support databases from DCA runs
2025-05-30 14:46:36 +01:00
Mathias Vorreiter Pedersen
7cb9024cc6
Bulk generator: Flip default values for summaries, sources, and sinks.
2025-05-30 13:33:24 +01:00
Mathias Vorreiter Pedersen
0f30644afd
Bulk generator: Snake case things.
2025-05-30 13:26:53 +01:00
Mathias Vorreiter Pedersen
3444c986ec
Bulk generator: Fix field name.
2025-05-30 13:25:12 +01:00
Mathias Vorreiter Pedersen
bdf411afbc
Bulk generator: Make 'database_results' a map to simplify away the explicit sorting.
2025-05-30 13:09:55 +01:00
Mathias Vorreiter Pedersen
cdd869a970
Bulk generator: Autoformat.
2025-05-30 12:49:12 +01:00
Mathias Vorreiter Pedersen
3ddca32705
Update misc/scripts/models-as-data/bulk_generate_mad.py
...
Co-authored-by: Simon Friis Vindum <paldepind@github.com >
2025-05-30 12:48:50 +01:00
Mathias Vorreiter Pedersen
7c2612a6a1
Bulk generator: Specify a path to the PAT instead of the PAT itself.
2025-05-30 12:47:07 +01:00
Mathias Vorreiter Pedersen
1228080914
Bulk generator: Specify 'language' in the config file.
2025-05-30 12:40:21 +01:00
Mathias Vorreiter Pedersen
fc165db8ac
Bulk generator: Specify 'with-summaries', 'with-sources', and 'with-sinks' in the config file.
2025-05-30 12:40:20 +01:00
Mathias Vorreiter Pedersen
7121f5c57e
Bulk generator: Use the 'Project' type throughout the file.
2025-05-30 12:08:42 +01:00
Mathias Vorreiter Pedersen
7c89d6d6dd
Bulk generator: Rename 'get_destination_for_project' to 'get_mad_destination_for_project'.
2025-05-30 11:49:48 +01:00
Mathias Vorreiter Pedersen
5d79a8de89
Update misc/scripts/models-as-data/bulk_generate_mad.py
...
Co-authored-by: Simon Friis Vindum <paldepind@github.com >
2025-05-30 11:48:30 +01:00
Mathias Vorreiter Pedersen
b640474a61
Bulk generator: Remove 'Phase' part of log message.
2025-05-30 11:43:30 +01:00
Mathias Vorreiter Pedersen
566bf431d7
Bulk generator: Rename 'github' to 'get_json_from_github'.
2025-05-30 11:42:15 +01:00
Mathias Vorreiter Pedersen
7ecf8c8ea2
Bulk generator: Format file and add a note at the top of the file specifying the formatting requirements.
2025-05-30 11:40:55 +01:00
Mathias Vorreiter Pedersen
cb938701a1
Bulk generator: Rename file since it is no longer Rust specific.
2025-05-29 18:14:24 +01:00
Mathias Vorreiter Pedersen
5051790e24
Bulk generator: Add DCA support.
2025-05-29 18:14:22 +01:00
Mathias Vorreiter Pedersen
e721fc07aa
Bulk generator: Prepare for adding DCA support. This commits just generalizes the existing functionality to be independent of Rust and instead depend on the configuration file and the command-line arguments.
2025-05-29 18:14:21 +01:00
Mathias Vorreiter Pedersen
6ff2bebbc2
Bulk generator: Add command-line arguments.
2025-05-29 18:14:19 +01:00
Mathias Vorreiter Pedersen
b87ba31c43
Bulk generator: Get rid of the hardcoded project list and move it into a configuration file.
2025-05-29 18:14:18 +01:00
