69 Commits

Author SHA1 Message Date
Paolo Tranquilli
de72e68d2c Merge branch 'main' into redsun82/codegen-new-parent-child 2025-06-26 12:14:53 +02:00
Paolo Tranquilli
355fd85c23 Codegen: remove unneeded has|getNumberOf in instance tests 2025-06-25 12:02:13 +02:00
Paolo Tranquilli
6bbf1e3bc1 Codegen: use one generated test file per directory
This collapses all generated test QL sources into a single one per
directory, using query predicates to run the different tests.

This should improve the time required to run generated tests.
2025-06-25 11:44:54 +02:00
Paolo Tranquilli
1dcd60527c Codegen: improve implementation of generated parent/child relationship
This improves the implementation of the generated parent/child
relationship by adding a new `all_children` field to `ql.Class` which
lists all children (both direct and inherited) of a class, carefully
avoiding duplicating children in case of diamond inheritance. This:
* simplifies the generated code,
* avoid children ambiguities in case of diamond inheritance.

This only comes with some changes in the order of children in the
generated tests (we were previously sorting bases alphabetically there).
For the rest this should be a non-functional change.
2025-06-24 17:26:24 +02:00
Paolo Tranquilli
14d48e9d58 Add black pre-commit hook
This switched `codegen` from the `autopep8` formatting to the `black`
one, and applies it to `bulk_mad_generator.py` as well. We can enroll
more python scripts to it in the future.
2025-06-10 12:25:39 +02:00
Paolo Tranquilli
9958cc7784 Codegen: consider windows paths in local codeql binary heuristic 2025-04-30 11:43:03 +02:00
Paolo Tranquilli
ba89a5de6f Codegen: make missing codeql error clearer 2025-04-30 11:38:52 +02:00
Paolo Tranquilli
a762e518c6 Merge branch 'main' into redsun82/codegen-rename-dbscheme 2025-03-20 15:52:49 +01:00
Paolo Tranquilli
4110636032 Rust: preserve ordering in rust generated code
This is a small devex improvement to the rust code generator.

Usage of `sorted` in `rustgen.py` was causing the generated code to be
completely reshuffled on renames, which made diffs hard to follow. As an
example see [this generated file diff](https://github.com/github/codeql/pull/19059/files#diff-c938ba77a3398dd4c633ada5702a03477705c24740a2f7d1e40d4b270d8c3f86).

This will make the order deterministically based on the order of
definitions in the schema file. This means that renames will find the
same place in the generated file, and the place in the generated file
will generally be more predictable with respect to the schema.

However, that does mean this change is heavily reshuffling the generated
code.
2025-03-20 12:12:52 +01:00
Paolo Tranquilli
7c8eb9ea0d Codegen: implement db_table_name in cppgen 2025-03-19 15:14:15 +01:00
Paolo Tranquilli
9639d6c8bb Codegen: implement db_table_name in rustgen 2025-03-19 15:14:15 +01:00
Paolo Tranquilli
f48aa79927 Codegen: implement db_table_name in qlgen 2025-03-19 15:14:15 +01:00
Paolo Tranquilli
fc9e066ecd Codegen: implement db_table_name in dbschemegen 2025-03-19 15:14:14 +01:00
Paolo Tranquilli
06b349e307 Codegen: introduce name conflict error in dbschemegen 2025-03-19 12:15:45 +01:00
Cornelius Riemenschneider
53ca5083a9 Upgrade bazel to 8.0.0.
Previously, we were using 8.0.0rc1.
In particular, this upgrade means we need to explicitly
import more rules, as they've been moved out of the core bazel repo.
2024-12-10 12:05:37 +01:00
Paolo Tranquilli
31b9b3adaa Merge branch 'main' into redsun82/rust-perf-measures 2024-12-04 10:50:37 +01:00
Paolo Tranquilli
db18d1046b Rust: rename getTy -> getTypeRepr 2024-12-03 11:17:08 +01:00
Paolo Tranquilli
b57a37479b Rust: make File usable in codegen 2024-12-02 15:15:46 +01:00
Paolo Tranquilli
99bd75c429 Rust: expand Ty in documentation 2024-12-02 09:51:15 +01:00
Paolo Tranquilli
5d3c9d050d Rust: expand Pat in documentation 2024-12-02 09:48:10 +01:00
Paolo Tranquilli
261e0a1a53 Codegen/Rust: allow renaming in QL
This adds a `ql.name` codegen pragma to change the name of a property on
the QL side. This is useful to give more meaningful names than what we
get from the generated rust AST.
2024-11-25 17:04:03 +01:00
Tom Hvitved
c8736e8a3d Rust: Auto-generate CfgNodes.qll 2024-11-21 10:21:11 +01:00
Arthur Baars
04e3b39ffb Merge pull request #17537 from github/redsun82/rust-doctest-gen
Rust: take test code also from property descriptions
2024-09-23 11:41:32 +02:00
Arthur Baars
05173fa7ac Merge pull request #17539 from github/redsun82/rust-codegen-detach
Rust/Codegen: allow to "detach" property emission
2024-09-23 11:40:11 +02:00
Paolo Tranquilli
c74b6be136 Rust/Codegen: allow to "detach" property emission
By using the `rust.detach` pragma on a property, we make that property
not appear in the generated struct as a field, and provide instead
a `generated::Class::emit_property` function that can be used to emit
the corresponding TRAP entry independently.
2024-09-20 16:06:22 +02:00
Paolo Tranquilli
2a95068a0a Rust: take test code also from property descriptions 2024-09-20 15:12:13 +02:00
Paolo Tranquilli
2533f18a6e Codegen: move ql.hideable to pragmas 2024-09-20 12:47:44 +02:00
Paolo Tranquilli
8d291ab938 Codegen: move qltest.test_with to parametrized pragmas 2024-09-20 12:47:44 +02:00
Paolo Tranquilli
9d6ee09f65 Codegen: move rust.doc_test_signature to parametrized pragmas 2024-09-20 11:09:21 +02:00
Paolo Tranquilli
70997e8189 Codegen: move default_doc_name to parametrized pragma 2024-09-20 11:09:21 +02:00
Tom Hvitved
5554c0f28f Rust: Use ```rust for code block examples
Also added some missing code block terminators.
2024-09-18 13:47:45 +02:00
Tom Hvitved
98b5ef5e01 Rust: Halt codegen on code block without ``` terminator 2024-09-18 13:47:44 +02:00
Tom Hvitved
1433363523 Codegen: Create internal folders 2024-09-17 20:06:35 +02:00
Paolo Tranquilli
1ce4707ff9 Merge branch 'main' into redsun82/rust-typed-labels 2024-09-16 16:44:55 +02:00
Paolo Tranquilli
95b32fb541 Pre-commit: bump up autopep8 check version and fix formatting 2024-09-16 14:03:48 +02:00
Tom Hvitved
9c0cafeeb8 Codegen: Make public AST classes final 2024-09-16 13:39:53 +02:00
Paolo Tranquilli
f1233b14e8 Rust: fix generated hierarchy 2024-09-16 10:06:01 +02:00
Paolo Tranquilli
8f93f5e34b Rust: move to Label<T> and mark unsafety of from_untyped 2024-09-13 17:17:19 +02:00
Paolo Tranquilli
faf1eeeb0d Rust: introduce typed labels 2024-09-13 13:57:14 +02:00
Paolo Tranquilli
56e127849f Rust/Codegen: fix generated list files 2024-09-09 12:31:09 +02:00
Paolo Tranquilli
3cd8aaf4b0 Rust: simplify rust doc test annotation 2024-09-09 08:59:17 +02:00
Paolo Tranquilli
928f3f11f1 Rust: remove <> from function wrapper 2024-09-06 14:34:12 +02:00
Paolo Tranquilli
8c5cc2efdc Rust: generate test code from schema docstrings
This generates test source files from code blocks in class docstrings.

By default the test code is generated as is, but it can optionally:
* be wrapped in a function providing an adequate context using
  `@rust.doc_test_function(name, *, lifetimes=(), return_type="()", **kwargs)`,
  with `kwargs` providing both generic and normal params depending on
  capitalization
* be skipped altogether using `@rust.skip_doc_test`

So for example an annotation like
```python
@rust.doc_test_function("foo",
                        lifetimes=("a",),
                        T="Eq",
                        x="&'a T",
                        y="&'a T",
                        return_type="&'a T")
```
will result in the following wrapper:
```rust
fn foo<'a, T: Eq>(x: &'a T, y: &'a T) -> &'a T {
    // example code here
}
```
2024-09-06 14:24:42 +02:00
Paolo Tranquilli
7e1290aa74 Rust: reuse shared rust trap library 2024-08-30 16:08:37 +02:00
Paolo Tranquilli
4f0fe1ce3a Rust: bazel packaging 2024-08-30 13:05:46 +02:00
Paolo Tranquilli
f40901f391 Rust: archiving + skeleton def translator 2024-08-28 17:15:49 +02:00
Paolo Tranquilli
9f5782b67b Bazel: introduce buildifier formatting
This introduces tooling and enforcement for formatting bazel files.

The tooling is provided as a bazel run target from
[keith/buildifier-prebuilt](https://github.com/keith/buildifier-prebuilt).

This is used in a [`pre-commit`](https://pre-commit.com/) hook for those
having that installed. In turn this is used in a CI check. Relying on a
`pre-commit` action gives us easy checking that buildifying did not
change anything in the files and printing the diff, without having to
hand-roll the check ourselves.

This enforcement will make usage of gazelle easier, as gazelle itself
might reformat files, even outside of `go`. Having them properly
formatted will allow gazelle to leave them unchanged, without needing
to configure awkward exclude directives.
2024-04-24 15:49:48 +02:00
Paolo Tranquilli
9264b2a6d6 Codegen: add internal to properties, rename ql_internal->internal 2023-11-24 06:56:55 +01:00
Paolo Tranquilli
a18f1ef7cd Codegen: fix bug where stub rewriting globbles too much code 2023-11-23 17:14:32 +01:00
Paolo Tranquilli
ff529c34b4 Codegen: use re.escape 2023-11-23 09:37:44 +01:00