hohn/codeql
1
0
Fork 0
mirror of https://github.com/github/codeql.git synced 2026-05-14 11:19:27 +02:00
Code Issues Packages Projects Releases Wiki Activity

Yeast: Update documentation to match current API

Browse Source 
yeast.md:
- Template section: document implicit context from rule!, show
  explicit BuildCtx::new with 3 params (ast, captures, fresh)
- Complete example: rewrite using rule! macro instead of manual
  Rule::new with closures
- Add rule! macro section documenting full and shorthand forms
- Document capture multiplicity (Id, Vec<Id>, Option<Id>)

yeast-macros/src/lib.rs:
- query!: remove references to field*: and standalone @capture
- tree!: document #{expr}, $fresh, {expr} syntax
- trees!: document {..expr} splice and (@capture)* splice
- rule!: document shorthand form, Option<Id> for ? captures

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This commit is contained in:
Taus
2026-05-01 16:08:51 +00:00
parent 112b38f78e
commit a3d7a9b8f2
4 changed files with 99 additions and 55 deletions
Download Patch File Download Diff File Expand all files Collapse all files

51
shared/yeast-macros/src/lib.rs
View File

@@ -8,16 +8,14 @@ mod parse;
/// # Syntax
///
/// ```text
/// (_) - match any node
/// (_) - match any named node (skips unnamed tokens)
/// (kind) - match a named node of the given kind
/// ("literal") - match an unnamed node (e.g. operators, keywords)
/// ("literal") - match an unnamed token by its text
/// (kind field: (pattern)) - match with named field
/// (kind child*: (patterns...)) - match unnamed children (yeast-specific)
/// (kind (pat) (pat)...) - match unnamed children (after all fields)
/// (pattern) @capture - capture the matched node
/// @capture - capture any node (shorthand for (_) @capture)
/// (pat1 pat2)* - zero or more repetitions
/// (pat1 pat2)+ - one or more repetitions
/// (pat1 pat2)? - zero or one repetitions
/// (pattern)* @capture - capture each repeated match
/// (pattern)? - zero or one
/// ```
#[proc_macro]
pub fn query(input: TokenStream) -> TokenStream {
@@ -30,6 +28,16 @@ pub fn query(input: TokenStream) -> TokenStream {
/// Build a single AST node from a template, returning its `Id`.
///
/// # Template syntax
///
/// ```text
/// (kind field: @capture) - node with captured child
/// (kind "literal") - leaf with static content
/// (kind #{expr}) - leaf with computed content (expr.to_string())
/// (kind $fresh) - leaf with auto-generated unique name
/// {expr} - embed a Rust expression returning Id
/// ```
///
/// Can be called with an explicit context or using the implicit context
/// from an enclosing `rule!`:
///
@@ -48,6 +56,13 @@ pub fn tree(input: TokenStream) -> TokenStream {
/// Build a list of AST nodes from a template, returning `Vec<Id>`.
///
/// Supports all syntax from `tree!`, plus:
///
/// ```text
/// {..expr} - splice an iterable of Id
/// (@capture)* - splice a repeated capture
/// ```
///
/// Can be called with an explicit context or using the implicit context
/// from an enclosing `rule!`:
///
@@ -68,21 +83,21 @@ pub fn trees(input: TokenStream) -> TokenStream {
///
/// ```text
/// rule!(
/// (query_pattern
/// field: (_) @capture_name
/// (kind)* @repeated
/// )
/// (query_pattern field: (_) @name (kind)* @repeated (_)? @optional)
/// =>
/// (output_template
/// field: {capture_name}
/// {..repeated}
/// )
/// (output_template field: {name} {..repeated})
/// )
///
/// // Shorthand: captures become fields on the output node
/// rule!((query ...) => output_kind)
/// ```
///
/// Captures become Rust variables: `@name` binds `name: Id` (single)
/// or `name: Vec<Id>` (after `*`/`+`). The transform can use `tree!`
/// and `trees!` without an explicit context.
/// Captures become Rust variables automatically:
/// - `@name` (no quantifier) → `name: Id`
/// - `@name` (after `*`/`+`) → `name: Vec<Id>`
/// - `@name` (after `?`) → `name: Option<Id>`
///
/// `tree!` and `trees!` can be used without explicit context inside `{...}`.
#[proc_macro]
pub fn rule(input: TokenStream) -> TokenStream {
let input2: TokenStream2 = input.into();

3
shared/yeast-macros/src/parse.rs
View File

@@ -602,9 +602,6 @@ pub fn parse_rule_top(input: TokenStream) -> Result<TokenStream> {
let name = Ident::new(&cap.name, Span::call_site());
let name_str = &cap.name;
match cap.multiplicity {
CaptureMultiplicity::Repeated => quote! {
let __field_id = #ctx_ident.ast.field_id_for_name(#name_str)
.unwrap_or_else(|| panic!("field '{}' not found", #name_str));
CaptureMultiplicity::Repeated => quote! {
let __field_id = #ctx_ident.ast.field_id_for_name(#name_str)
.unwrap_or_else(|| panic!("field '{}' not found", #name_str));

96
shared/yeast/doc/yeast.md
View File

@@ -130,35 +130,47 @@ children. Named node patterns like `(_)` automatically skip unnamed tokens
## Template language
Templates construct new AST nodes using the `tree!` and `trees!` macros.
Both take a `BuildCtx` as their first argument, which holds the AST,
captures from the query match, and a fresh identifier scope.
When used inside a `rule!` macro, the context is implicit — no explicit
`BuildCtx` argument is needed. When used standalone, they take a `BuildCtx`
as the first argument:
```rust
let mut ctx = BuildCtx::new(ast, &captures);
// Inside rule! — implicit context
yeast::rule!(
(assignment left: (_) @left right: (_) @right)
=>
(assignment left: {right} right: {left})
);
// Standalone — explicit context
let fresh = yeast::tree_builder::FreshScope::new();
let mut ctx = BuildCtx::new(ast, &captures, &fresh);
let id = yeast::tree!(ctx, (assignment left: @lhs right: @rhs));
```
### `tree!` — build a single node
`tree!(ctx, ...)` returns a single node `Id`:
`tree!(...)` returns a single node `Id`:
```rust
let id = yeast::tree!(ctx,
yeast::tree!(ctx,
(assignment
left: @lhs
right: @rhs
)
);
)
```
### `trees!` — build multiple nodes
`trees!(ctx, ...)` returns `Vec<Id>`:
`trees!(...)` returns `Vec<Id>`:
```rust
let ids = yeast::trees!(ctx,
yeast::trees!(ctx,
(assignment left: @tmp right: @right)
(@body)*
);
)
```
### Capture references
@@ -245,39 +257,59 @@ This rule rewrites Ruby's `for pat in val do body end` into
`val.each { |tmp| pat = tmp; body }`:
```rust
let query = yeast::query!(
let for_rule = yeast::rule!(
(for
pattern: (_) @pat
value: (in (_) @val)
body: (do (_)* @body)
)
);
let transform = |ast: &mut Ast, match_: Captures| {
let mut ctx = BuildCtx::new(ast, &match_);
vec![yeast::tree!(ctx,
(call
receiver: @val
method: (identifier "each")
block: (block
parameters: (block_parameters
(identifier $tmp)
)
body: (block_body
(assignment
left: @pat
right: (identifier $tmp)
)
(@body)*
=>
(call
receiver: {val}
method: (identifier "each")
block: (block
parameters: (block_parameters
(identifier $tmp)
)
body: (block_body
(assignment
left: {pat}
right: (identifier $tmp)
)
{..body}
)
)
)]
};
let rule = Rule::new(query, Box::new(transform));
)
);
```
Captures from the query (`@pat`, `@val`, `@body`) become Rust variables
automatically: single captures bind as `Id`, repeated captures (after
`*` or `+`) as `Vec<Id>`, and optional captures (after `?`) as
`Option<Id>`.
## The `rule!` macro
`rule!` combines a query and a transform into a single declaration:
```rust
// Full template form
yeast::rule!(
(query_pattern field: (_) @capture)
=>
(output_template field: {capture})
)
// Shorthand form — captures become fields on the output node
yeast::rule!(
(query_pattern field: (_) @capture)
=> output_kind
)
```
The shorthand `=> kind` form auto-generates the template, mapping each
capture name to a field of the same name on the output node.
## Integration with the extractor
YEAST integrates with the shared tree-sitter extractor via two mechanisms:

4
shared/yeast/src/lib.rs
View File

@@ -206,14 +206,14 @@ impl Ast {
fields: BTreeMap<FieldId, Vec<Id>>,
is_named: bool,
) -> Id {
self.create_node_with_range(kind, content, children, is_named, None)
self.create_node_with_range(kind, content, fields, is_named, None)
}
pub fn create_node_with_range(
&mut self,
kind: KindId,
content: NodeContent,
children: Vec<(FieldId, Id)>,
fields: BTreeMap<FieldId, Vec<Id>>,
is_named: bool,
source_range: Option<tree_sitter::Range>,
) -> Id {