Compare commits

..

4 Commits

Author SHA1 Message Date
yoff
e50e81390f Python: visit function parameter and return annotations in new CFG
The new (shared-CFG-based) Python control flow graph in
`semmle.python.controlflow.internal.Cfg` previously did not emit CFG
nodes for parameter type annotations (`def f(x: T): ...`) or for the
return type annotation (`-> T`). The legacy CFG emitted both, and a
small number of framework models rely on this: `LocalSources.qll`'s
`annotatedInstance` walks the parameter annotation expression by way
of its CFG node to track that a parameter receives an instance of the
annotated class.

After the dataflow flip to the new CFG/SSA this regression manifested
as lost flows in any test exercising annotation-based parameter
tracking: FastAPI `Depends()` receivers, Pydantic request bodies,
Starlette `WebSocket`, the call-graph type-annotation test, and so on.
Extend `FunctionDefExpr` to visit each annotation as a child of the
function-def expression, in CPython evaluation order: positional
parameter annotations, `*args` annotation, keyword-only parameter
annotations, `**kwargs` annotation, then the return annotation. (Lambda
expressions have no annotations in Python syntax, so `LambdaExpr` is
unchanged.) PEP 695 type parameters remain out of scope; they belong
to the inner annotation scope, not the enclosing CFG.

Restored test results across `framework/aiohttp`, `framework/fastapi`,
`framework/lxml`, the `CallGraph-type-annotations` test, and
`CWE-022-PathInjection`. Two FastAPI list-comprehension MISSING markers
become positive (`taint_test.py:41,55`). CPython CFG consistency
remains clean.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-06-29 19:37:01 +00:00
yoff
66900c7d62 Python: model exception edges for raise-prone expressions inside try/with
The new CFG previously only emitted exception edges for explicit `raise`
and `assert` statements. As a result, code that became reachable only
via the exception path of an arbitrary expression (e.g., the body of an
`except` handler following a try-body whose `call()` could raise) was
classified as dead, breaking analyses like StackTraceExposure,
FileNotAlwaysClosed, ExceptionInfo, UseOfExit, and CatchingBaseException.

This commit adds a `mayThrow` predicate over expressions that are known
sources of implicit exceptions in Python (calls, attribute access,
subscripts, arithmetic/comparison operators, imports, await/yield/yield
from) plus `from m import *` at the statement level, and routes them
through the shared CFG's `beginAbruptCompletion(_, _, ExceptionSuccessor,
always=false)` hook.

The set of exception sources is restricted to nodes that are
syntactically inside a `try`/`with` statement in the same scope.
This mirrors Java's `ControlFlowGraph::mayThrow`, which only emits
exception edges where local handling can observe them — outside such
contexts, the edges add CFG complexity (weakening BarrierGuard
precision and breaking SSA continuity around augmented assignments and
subscript stores) without analysis benefit, since exceptions just
propagate to the function exit anyway.

Net effect on the test suite: ~100 alerts restored across the exception-
related query tests (StackTraceExposure +29, ExceptionInfo +17,
FileNotAlwaysClosed +52, UseOfExit +1, CatchingBaseException restored)
with no precision regressions. Affected `.expected` files and the
regression-guard `dead_under_no_raise.py` are updated accordingly.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-06-29 19:37:01 +00:00
yoff
956d2dbec4 Python: add new shared-CFG-backed control flow graph facade (Cfg)
Adds the public facade on top of the AstNodeImpl adapter from the
previous commit. Re-exposes the same API surface as
semmle/python/Flow.qll (ControlFlowNode, CallNode, BasicBlock,
NameNode, DefinitionNode, CompareNode, ...), backed by the shared
codeql.controlflow.ControlFlowGraph library.

- semmle.python.controlflow.internal.Cfg — public facade.
- ControlFlow/store-load/* — basic store/load coverage via the facade.

The new CFG library is added additively: it has zero callers in lib/
and src/, and the legacy CFG in semmle/python/Flow.qll remains the
default. Dataflow, SSA, and production query migration land in
follow-up PRs.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-06-29 19:37:01 +00:00
yoff
872c08148e Python: add shared-CFG AstSig adapter (AstNodeImpl)
Preparatory refactor for the shared-CFG dataflow migration. Adds the
adapter that mediates between the Python AST and the shared
codeql.controlflow.ControlFlowGraph signature, plus the test suites
that validate the new CFG directly against this adapter. The public
facade is added in the following commit.

Library additions:

- semmle.python.controlflow.internal.AstNodeImpl — wraps Python's
  Stmt/Expr/Scope/Pattern and adds two synthetic kinds of node
  (BlockStmt for body slots, intermediate nodes for multi-operand
  boolean expressions) to satisfy the shared CFG signature.

- lib/ide-contextual-queries/printCfg.ql — the IDE "Print CFG" query, retargeted to the new CFG.

- consistency-queries/CfgConsistency.ql — consistency query running
  the shared CFG's standard checks against Python.

Test additions (all driven directly off AstNodeImpl):

- ControlFlow/bindings/* — annotation-driven SSA-binding tests
  (annassign, compound, comprehension, decorated, except_handler,
  imports, match_pattern, parameters, simple, type_params,
  walrus_starred, with_stmt, dead_under_no_raise).

- ControlFlow/evaluation-order/NewCfg*.ql — mirrors of the existing
  OldCfg evaluation-order self-validation suite, run against the
  new CFG via NewCfgImpl.qll.

- Minor extensions to existing test_if.py / test_boolean.py +
  cosmetic .expected churn on a handful of OldCfg tests.

No dataflow, SSA, or production query is migrated yet.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-06-29 18:49:41 +00:00
77 changed files with 4337 additions and 879 deletions

View File

@@ -28,6 +28,7 @@
/swift/extractor/ @github/codeql-swift @github/code-scanning-language-coverage
/misc/codegen/ @github/codeql-swift
/java/kotlin-extractor/ @github/codeql-kotlin @github/code-scanning-language-coverage
/java/ql/test-kotlin1/ @github/codeql-kotlin
/java/ql/test-kotlin2/ @github/codeql-kotlin
# Experimental CodeQL cryptography

View File

@@ -75,9 +75,6 @@ def get_version():
def install(version: str, quiet: bool):
if install_dir.exists():
return
if quiet:
info_out = subprocess.DEVNULL
info = lambda *args: None
@@ -86,6 +83,8 @@ def install(version: str, quiet: bool):
info = lambda *args: print(*args, file=sys.stderr)
file = file_template.format(version=version)
url = url_template.format(version=version)
if install_dir.exists():
shutil.rmtree(install_dir)
install_dir.mkdir()
zips_dir.mkdir(exist_ok=True)
zip = zips_dir / file
@@ -157,11 +156,8 @@ def main(opts, forwarded_opts):
selected_version = current_version or DEFAULT_VERSION
if selected_version != current_version:
# don't print information about install procedure unless explicitly using --select
if install_dir.exists():
shutil.rmtree(install_dir)
install(selected_version, quiet=opts.select is None)
version_file.write_text(selected_version)
# don't print information about install procedure unless explicitly using --select
install(selected_version, quiet=opts.select is None)
if opts.select and not forwarded_opts and not opts.version:
print(f"selected {selected_version}")
return

View File

@@ -6,8 +6,6 @@ import com.github.codeql.utils.*
import com.github.codeql.utils.versions.*
import com.semmle.extractor.java.OdasaOutput
import java.io.Closeable
import java.nio.file.Files
import java.nio.file.Path
import java.util.*
import kotlin.collections.ArrayList
import org.jetbrains.kotlin.backend.common.extensions.IrPluginContext
@@ -52,7 +50,6 @@ import org.jetbrains.kotlin.load.java.structure.JavaMethod
import org.jetbrains.kotlin.load.java.structure.JavaTypeParameter
import org.jetbrains.kotlin.load.java.structure.JavaTypeParameterListOwner
import org.jetbrains.kotlin.load.java.structure.impl.classFiles.BinaryJavaClass
import org.jetbrains.kotlin.fir.java.VirtualFileBasedSourceElement
import org.jetbrains.kotlin.name.FqName
import org.jetbrains.kotlin.types.Variance
import org.jetbrains.kotlin.util.OperatorNameConventions
@@ -164,100 +161,23 @@ open class KotlinFileExtractor(
}
}
private fun javaBinaryDeclaresMethod(c: IrClass, name: String): Boolean? {
// K1 path: source is JavaSourceElement wrapping a BinaryJavaClass - inspect class metadata
val binaryJavaClass = (c.source as? JavaSourceElement)?.javaElement as? BinaryJavaClass
if (binaryJavaClass != null) {
return binaryJavaClass.methods.any { it.name.asString() == name }
private fun javaBinaryDeclaresMethod(c: IrClass, name: String) =
((c.source as? JavaSourceElement)?.javaElement as? BinaryJavaClass)?.methods?.any {
it.name.asString() == name
}
// K2 path: binary Java classes use VirtualFileBasedSourceElement instead of
// JavaSourceElement. The BinaryJavaClass is not stored in the source element, so we parse
// the class bytes directly using ASM to check if the method is explicitly declared.
if (c.source is VirtualFileBasedSourceElement) {
val virtualFile = (c.source as VirtualFileBasedSourceElement).virtualFile
if (!virtualFile.name.endsWith(".class")) return null
return try {
val bytes = virtualFile.contentsToByteArray()
var found = false
var hasKotlinMetadata = false
val reader = org.jetbrains.org.objectweb.asm.ClassReader(bytes)
reader.accept(
object : org.jetbrains.org.objectweb.asm.ClassVisitor(
org.jetbrains.org.objectweb.asm.Opcodes.ASM9
) {
override fun visitAnnotation(
descriptor: String,
visible: Boolean
): org.jetbrains.org.objectweb.asm.AnnotationVisitor? {
if (descriptor == "Lkotlin/Metadata;") hasKotlinMetadata = true
return null
}
override fun visitMethod(
access: Int,
methodName: String,
descriptor: String,
signature: String?,
exceptions: Array<String>?
): org.jetbrains.org.objectweb.asm.MethodVisitor? {
if (methodName == name) found = true
return null
}
},
org.jetbrains.org.objectweb.asm.ClassReader.SKIP_CODE or
org.jetbrains.org.objectweb.asm.ClassReader.SKIP_DEBUG or
org.jetbrains.org.objectweb.asm.ClassReader.SKIP_FRAMES
)
if (hasKotlinMetadata) false else found
} catch (e: Exception) {
logger.warn("Failed to check binary class methods for ${c.fqNameWhenAvailable}: $e")
null
}
}
return null
}
private fun isJavaBinaryDeclaration(f: IrFunction) =
f.parentClassOrNull?.let { javaBinaryDeclaresMethod(it, f.name.asString()) } ?: false
private fun hasConcreteSiblingObjectMethod(f: IrFunction): Boolean {
val parentClass = f.parentClassOrNull ?: return false
return parentClass.declarations
.asSequence()
.filterIsInstance<IrFunction>()
.filter { sibling ->
sibling !== f &&
sibling.name == f.name &&
sibling.codeQlValueParameters.size == f.codeQlValueParameters.size
}
.any { sibling ->
val hasInvisibleFakeVisibility =
sibling.visibility.let {
it is DelegatedDescriptorVisibility && it.delegate == Visibilities.InvisibleFake
}
!sibling.isFakeOverride && !hasInvisibleFakeVisibility
}
}
private fun isJavaBinaryObjectMethodRedeclaration(d: IrDeclaration) =
when (d) {
is IrFunction ->
d.parentClassOrNull?.typeParameters?.isEmpty() == true &&
when (d.name.asString()) {
"toString" -> d.codeQlValueParameters.isEmpty()
"hashCode" -> d.codeQlValueParameters.isEmpty()
// Under K2 (language version 2.0+), the Object.equals(Object) parameter is
// typed as Any (non-nullable) rather than Any? (nullable). Accept both.
"equals" ->
d.codeQlValueParameters
.singleOrNull()
?.type
?.let { it.isNullableAny() || it.isAny() } ?: false
"equals" -> d.codeQlValueParameters.singleOrNull()?.type?.isNullableAny() ?: false
else -> false
} &&
!hasConcreteSiblingObjectMethod(d) &&
isJavaBinaryDeclaration(d)
} && isJavaBinaryDeclaration(d)
else -> false
}
@@ -1392,28 +1312,27 @@ open class KotlinFileExtractor(
): TypeResults {
with("value parameter", vp) {
val location = locOverride ?: getLocation(vp, classTypeArgsIncludingOuterClasses)
val parentFunction = vp.parent as? IrFunction
val javaCallable = parentFunction?.let { getJavaCallable(it) }
val maybeAlteredType =
parentFunction?.let {
(vp.parent as? IrFunction)?.let {
if (overridesCollectionsMethodWithAlteredParameterTypes(it))
eraseCollectionsMethodParameterType(vp.type, it.name.asString(), idx)
else if (
(parentFunction as? IrConstructor)?.parentClassOrNull?.kind ==
(vp.parent as? IrConstructor)?.parentClassOrNull?.kind ==
ClassKind.ANNOTATION_CLASS
)
kClassToJavaClass(vp.type)
else null
} ?: vp.type
val javaType = javaCallable?.let { jCallable -> getJavaValueParameterType(jCallable, idx) }
val addParameterWildcardsByDefault =
!getInnermostWildcardSupppressionAnnotation(vp) &&
!(javaCallable == null &&
parentFunction?.origin == IrDeclarationOrigin.IR_EXTERNAL_JAVA_DECLARATION_STUB)
val javaType =
(vp.parent as? IrFunction)?.let {
getJavaCallable(it)?.let { jCallable ->
getJavaValueParameterType(jCallable, idx)
}
}
val typeWithWildcards =
addJavaLoweringWildcards(
maybeAlteredType,
addParameterWildcardsByDefault,
!getInnermostWildcardSupppressionAnnotation(vp),
javaType
)
val substitutedType =
@@ -1427,9 +1346,9 @@ open class KotlinFileExtractor(
vp.origin == IrDeclarationOrigin.UNDERSCORE_PARAMETER ||
((vp.parent as? IrFunction)?.let { hasSynthesizedParameterNames(it) } ?: true)
val javaParameter =
when (javaCallable) {
is JavaConstructor -> javaCallable.valueParameters.getOrNull(idx)
is JavaMethod -> javaCallable.valueParameters.getOrNull(idx)
when (val callable = (vp.parent as? IrFunction)?.let { getJavaCallable(it) }) {
is JavaConstructor -> callable.valueParameters.getOrNull(idx)
is JavaMethod -> callable.valueParameters.getOrNull(idx)
else -> null
}
val extraAnnotations =
@@ -2955,45 +2874,6 @@ open class KotlinFileExtractor(
return v
}
private val sourceTextCache = mutableMapOf<String, String?>()
private fun getCurrentFileSourceText() =
sourceTextCache.getOrPut(filePath) {
runCatching { Files.readString(Path.of(filePath)) }.getOrNull()
}
private fun getVariableNameLocation(v: IrVariable): Label<DbLocation>? {
if (v.startOffset < 0 || v.endOffset < v.startOffset) return null
val source = getCurrentFileSourceText() ?: return null
if (v.startOffset >= source.length) return null
val name = v.name.asString()
if (name.isEmpty()) return null
val endExclusive = minOf(v.endOffset + 1, source.length)
val declarationText = source.substring(v.startOffset, endExclusive)
val nameOffsetInDeclaration = declarationText.indexOf(name)
if (nameOffsetInDeclaration < 0) return null
val nameStartOffset = v.startOffset + nameOffsetInDeclaration
val nameEndOffset = nameStartOffset + name.length - 1
return tw.getLocation(nameStartOffset, nameEndOffset)
}
private fun shouldUseVariableNameLocation(v: IrVariable): Boolean {
val initializer = v.initializer
return initializer is IrTypeOperatorCall && initializer.operator == IrTypeOperator.IMPLICIT_NOTNULL
}
private fun getVariableLocation(v: IrVariable): Label<DbLocation> {
if (shouldUseVariableNameLocation(v)) {
val nameLocation = getVariableNameLocation(v)
if (nameLocation != null) return nameLocation
}
return tw.getLocation(getVariableLocationProvider(v))
}
private fun extractVariable(
v: IrVariable,
callable: Label<out DbCallable>,
@@ -3002,7 +2882,7 @@ open class KotlinFileExtractor(
) {
with("variable", v) {
val stmtId = tw.getFreshIdLabel<DbLocalvariabledeclstmt>()
val locId = getVariableLocation(v)
val locId = tw.getLocation(getVariableLocationProvider(v))
tw.writeStmts_localvariabledeclstmt(stmtId, parent, idx, callable)
tw.writeHasLocation(stmtId, locId)
extractVariableExpr(v, callable, stmtId, 1, stmtId)
@@ -3020,7 +2900,7 @@ open class KotlinFileExtractor(
with("variable expr", v) {
val varId = useVariable(v)
val exprId = tw.getFreshIdLabel<DbLocalvariabledeclexpr>()
val locId = getVariableLocation(v)
val locId = tw.getLocation(getVariableLocationProvider(v))
val type = useType(v.type)
tw.writeLocalvars(varId, v.name.asString(), type.javaResult.id, exprId)
tw.writeLocalvarsKotlinType(varId, type.kotlinResult.id)
@@ -4186,28 +4066,6 @@ open class KotlinFileExtractor(
else -> false
}
private fun getCallResultType(c: IrCall, syntacticCallTarget: IrFunction): IrType {
if (syntacticCallTarget.origin != IrDeclarationOrigin.IR_EXTERNAL_JAVA_DECLARATION_STUB) {
return c.type
}
val primitiveInfo =
(c.type as? IrSimpleType)?.let { primitiveTypeMapping.getPrimitiveInfo(it) } ?: return c.type
val parentClass = syntacticCallTarget.parentClassOrNull ?: return c.type
val returnIsClassifier =
javaBinaryMethodReturnIsClassifierType(
parentClass,
getFunctionShortName(syntacticCallTarget).nameInDB,
syntacticCallTarget.codeQlValueParameters.size,
syntacticCallTarget is IrConstructor
)
return if (returnIsClassifier == true) {
primitiveInfo.javaClass.symbol.typeWith()
} else {
c.type
}
}
private fun isGenericArrayType(typeName: String) =
when (typeName) {
"Array" -> true
@@ -4253,7 +4111,7 @@ open class KotlinFileExtractor(
extractRawMethodAccess(
syntacticCallTarget,
c,
getCallResultType(c, syntacticCallTarget),
c.type,
callable,
parent,
idx,

View File

@@ -36,7 +36,6 @@ import org.jetbrains.kotlin.load.java.BuiltinMethodsWithSpecialGenericSignature
import org.jetbrains.kotlin.load.java.JvmAbi
import org.jetbrains.kotlin.load.java.sources.JavaSourceElement
import org.jetbrains.kotlin.load.java.structure.*
import org.jetbrains.kotlin.load.java.structure.impl.classFiles.BinaryJavaClass
import org.jetbrains.kotlin.load.java.typeEnhancement.hasEnhancedNullability
import org.jetbrains.kotlin.name.FqName
import org.jetbrains.kotlin.name.NameUtils
@@ -997,20 +996,7 @@ open class KotlinUsesExtractor(
)
return null
}
val fileClassId = extractFileClass(fqName)
// Under K2, external file class members sit directly under IrExternalPackageFragment
// rather than under their IrClass parent. In that case the file class entity won't
// get a location set through the normal extractClassSource path.
if (d is IrMemberWithContainerSource && tw.lm.externalFileClassLocationsExtracted.add(fqName)) {
val binaryPath =
getContainerSourceBinaryPath(d.containerSource)
?.let { normalizeExternalFileClassBinaryPath(it, fqName) }
if (binaryPath != null && shouldUseConcreteExternalFileClassLocation(binaryPath)) {
val fileId = tw.mkFileId(binaryPath, true)
tw.writeHasLocation(fileClassId, tw.getWholeFileLocation(fileId))
}
}
return fileClassId
return extractFileClass(fqName)
}
return useDeclarationParent(parent, canBeTopLevel, classTypeArguments, inReceiverContext)
}
@@ -1385,13 +1371,8 @@ open class KotlinUsesExtractor(
parentId: Label<out DbElement>,
classTypeArgsIncludingOuterClasses: List<IrTypeArgument>?,
maybeParameterList: List<IrValueParameter>? = null
): String {
val javaCallable = getJavaCallable(f)
val addParameterWildcardsByDefault =
!getInnermostWildcardSupppressionAnnotation(f) &&
!(javaCallable == null && f.origin == IrDeclarationOrigin.IR_EXTERNAL_JAVA_DECLARATION_STUB)
return getFunctionLabel(
): String =
getFunctionLabel(
f.parent,
parentId,
getFunctionShortName(f).nameInDB,
@@ -1401,10 +1382,9 @@ open class KotlinUsesExtractor(
getFunctionTypeParameters(f),
classTypeArgsIncludingOuterClasses,
overridesCollectionsMethodWithAlteredParameterTypes(f),
javaCallable,
addParameterWildcardsByDefault
getJavaCallable(f),
!getInnermostWildcardSupppressionAnnotation(f)
)
}
/*
* This function actually generates the label for a function.
@@ -1491,41 +1471,15 @@ open class KotlinUsesExtractor(
// Finally, mimic the Java extractor's behaviour by naming functions with type
// parameters for their erased types;
// those without type parameters are named for the generic type.
var maybeErased =
val maybeErased =
if (functionTypeParameters.isEmpty()) maybeSubbed else erase(maybeSubbed)
// K2 compatibility: under K2, Java @NotNull reference types such as @NotNull Integer
// are enhanced to Kotlin primitives (e.g. kotlin.Int). But the Java extractor uses
// the original reference type (java.lang.Integer) in callable labels. When we detect
// that the original Java parameter type is a reference (classifier) type but the
// Kotlin IR type is a primitive, revert to the boxed Java class so both extractors
// produce matching callable IDs.
if (functionTypeParameters.isEmpty()) {
val primitiveInfo = (maybeErased as? IrSimpleType)?.let {
primitiveTypeMapping.getPrimitiveInfo(it)
}
if (primitiveInfo != null) {
val parentClass = parent as? IrClass
if (parentClass != null) {
val isClassifierType = javaBinaryMethodParamIsClassifierType(
parentClass,
name,
allParamTypes.size,
name == "<init>",
it.index
)
if (isClassifierType == true) {
maybeErased = primitiveInfo.javaClass.symbol.typeWith()
}
}
}
}
"{${useType(maybeErased).javaResult.id}}"
}
val paramTypeIds =
allParamTypes
.withIndex()
.joinToString(separator = ",", transform = getIdForFunctionLabel)
var labelReturnType =
val labelReturnType =
if (name == "<init>") pluginContext.irBuiltIns.unitType
else
erase(
@@ -1535,28 +1489,6 @@ open class KotlinUsesExtractor(
pluginContext
)
)
// K2 compatibility: same as for parameters, if the Java binary method return type is a
// reference type but K2 enhanced it to a Kotlin primitive, use the boxed Java class.
if (functionTypeParameters.isEmpty() && name != "<init>") {
val primitiveInfo = (labelReturnType as? IrSimpleType)?.let {
primitiveTypeMapping.getPrimitiveInfo(it)
}
if (primitiveInfo != null) {
val parentClass = parent as? IrClass
if (parentClass != null) {
val returnIsClassifier =
javaBinaryMethodReturnIsClassifierType(
parentClass,
name,
allParamTypes.size,
false
)
if (returnIsClassifier == true) {
labelReturnType = primitiveInfo.javaClass.symbol.typeWith()
}
}
}
}
// Note that `addJavaLoweringWildcards` is not required here because the return type used to
// form the function
// label is always erased.
@@ -1662,23 +1594,9 @@ open class KotlinUsesExtractor(
}
@OptIn(ObsoleteDescriptorBasedAPI::class)
fun getJavaCallable(f: IrFunction): JavaMember? {
val fromDescriptor = (f.descriptor.source as? JavaSourceElement)?.javaElement as? JavaMember
if (fromDescriptor != null) return fromDescriptor
fun getJavaCallable(f: IrFunction) =
(f.descriptor.source as? JavaSourceElement)?.javaElement as? JavaMember
// K2 fallback: under K2, descriptor.source may not carry JavaSourceElement for binary Java
// methods. Try to get the JavaMember from the parent class's binary class directly.
val parentClass = f.parentClassOrNull ?: return null
val binaryJavaClass = (parentClass.source as? JavaSourceElement)?.javaElement as? BinaryJavaClass
?: return null
val name = getFunctionShortName(f).nameInDB
val nParams = f.codeQlValueParameters.size
return if (f is IrConstructor) {
binaryJavaClass.constructors.find { it.valueParameters.size == nParams }
} else {
binaryJavaClass.methods.find { it.name.asString() == name && it.valueParameters.size == nParams }
}
}
fun getJavaValueParameterType(m: JavaMember, idx: Int) =
when (m) {
is JavaMethod -> m.valueParameters[idx].type

View File

@@ -51,13 +51,6 @@ class TrapLabelManager {
* to avoid duplication.
*/
val fileClassLocationsExtracted = HashSet<IrFile>()
/**
* Tracks external file classes (by FqName) whose location has been set from a binary path.
* Used to avoid writing duplicate hasLocation facts for external file class entities extracted
* through the K2 code path where declarations sit directly under IrExternalPackageFragment.
*/
val externalFileClassLocationsExtracted = HashSet<org.jetbrains.kotlin.name.FqName>()
}
/**

View File

@@ -17,7 +17,6 @@ import org.jetbrains.kotlin.load.kotlin.JvmPackagePartSource
import org.jetbrains.kotlin.load.kotlin.KotlinJvmBinarySourceElement
import org.jetbrains.kotlin.load.kotlin.VirtualFileKotlinClass
import org.jetbrains.kotlin.name.FqName
import org.jetbrains.kotlin.serialization.deserialization.descriptors.DeserializedContainerSource
// Adapted from Kotlin's interpreter/Utils.kt function 'internalName'
// Translates class names into their JLS section 13.1 binary name,
@@ -177,238 +176,15 @@ fun getIrDeclarationBinaryPath(d: IrDeclaration): String? {
// This is in a file class.
val fqName = getFileClassFqName(d)
if (fqName != null) {
if (d is IrMemberWithContainerSource) {
val containerBinaryPath = getContainerSourceBinaryPath(d.containerSource)
if (containerBinaryPath != null) {
return normalizeExternalFileClassBinaryPath(containerBinaryPath, fqName)
}
}
return getUnknownBinaryLocation(fqName.asString())
}
}
return null
}
/**
* Attempts to get the binary file path from a container source (typically a
* [JvmPackagePartSource]). Returns null if the path is unavailable.
*/
fun getContainerSourceBinaryPath(containerSource: org.jetbrains.kotlin.serialization.deserialization.descriptors.DeserializedContainerSource?): String? {
if (containerSource !is JvmPackagePartSource) return null
val binaryClass = containerSource.knownJvmBinaryClass ?: return null
return when (binaryClass) {
is VirtualFileKotlinClass -> {
val vf = binaryClass.file
val path = vf.path
if (vf.fileSystem.protocol == StandardFileSystems.JRT_PROTOCOL)
"/${path.split("!/", limit = 2)[1]}"
else path
}
else -> binaryClass.location.takeIf { it.isNotEmpty() }
}
}
private fun getUnknownBinaryLocation(s: String): String {
return "/!unknown-binary-location/${s.replace(".", "/")}.class"
}
fun normalizeExternalFileClassBinaryPath(path: String, fqName: FqName): String {
if (path.contains(".kotlinc_installed")) {
return getUnknownBinaryLocation(fqName.asString())
}
val normalizedPath = path.replace('\\', '/')
val classInternalPath = "${fqName.asString().replace(".", "/")}.class"
val classSuffix = "/$classInternalPath"
if (normalizedPath.endsWith(classSuffix)) {
val classpathRoot = normalizedPath.removeSuffix(classSuffix).substringAfterLast('/')
if (classpathRoot.isNotEmpty()) {
return "$classpathRoot/$classInternalPath"
}
}
return path
}
fun shouldUseConcreteExternalFileClassLocation(path: String): Boolean {
val normalizedPath = path.replace('\\', '/')
return normalizedPath.contains("/") &&
!normalizedPath.startsWith("/!unknown-binary-location/")
}
fun getJavaEquivalentClassId(c: IrClass) =
c.fqNameWhenAvailable?.toUnsafe()?.let { JavaToKotlinClassMap.mapKotlinToJava(it) }
/**
* Checks whether a specific parameter of a Java binary method (identified by [methodName] and
* [paramIndex]) is a reference type (as opposed to a Java primitive). This is used to detect
* cases where K2 FIR has enhanced a reference type parameter (e.g. `@NotNull Integer`) to a
* Kotlin primitive (e.g. `kotlin.Int`), so that callable labels can use the original reference
* type and remain compatible with the Java extractor's callable IDs.
*
* Under K1, binary Java classes use [JavaSourceElement] and we can check [BinaryJavaClass.methods]
* directly. Under K2, they use [VirtualFileBasedSourceElement] and we fall back to reading the
* class bytes with ASM.
*
* Returns `null` if the information cannot be determined.
*/
fun javaBinaryMethodParamIsClassifierType(
parentClass: IrClass,
methodName: String,
nParams: Int,
isConstructor: Boolean,
paramIndex: Int
): Boolean? {
// K1 path: binary Java class has JavaSourceElement with a BinaryJavaClass.
val k1ParamKinds =
((parentClass.source as? JavaSourceElement)?.javaElement as? BinaryJavaClass)?.let {
binaryJavaClass ->
if (isConstructor)
binaryJavaClass.constructors
.asSequence()
.filter { it.valueParameters.size == nParams }
.mapNotNull { it.valueParameters.getOrNull(paramIndex)?.type }
.map { it is org.jetbrains.kotlin.load.java.structure.JavaClassifierType }
.toSet()
else
binaryJavaClass.methods
.asSequence()
.filter { it.name.asString() == methodName && it.valueParameters.size == nParams }
.mapNotNull { it.valueParameters.getOrNull(paramIndex)?.type }
.map { it is org.jetbrains.kotlin.load.java.structure.JavaClassifierType }
.toSet()
}
if (k1ParamKinds != null && k1ParamKinds.isNotEmpty()) {
return k1ParamKinds.singleOrNull()
}
// K2 path: binary Java class has VirtualFileBasedSourceElement
if (parentClass.source !is VirtualFileBasedSourceElement) return null
val vf = (parentClass.source as VirtualFileBasedSourceElement).virtualFile
if (!vf.name.endsWith(".class")) return null
return try {
val bytes = vf.contentsToByteArray()
val expectedMethodName = if (isConstructor) "<init>" else methodName
val descriptorKinds = mutableSetOf<Boolean>()
val reader = org.jetbrains.org.objectweb.asm.ClassReader(bytes)
reader.accept(
object : org.jetbrains.org.objectweb.asm.ClassVisitor(
org.jetbrains.org.objectweb.asm.Opcodes.ASM9
) {
override fun visitMethod(
access: Int,
name: String,
descriptor: String,
signature: String?,
exceptions: Array<String>?
): org.jetbrains.org.objectweb.asm.MethodVisitor? {
if (name != expectedMethodName) return null
val paramDescriptors = parseAsmMethodDescriptorParams(descriptor)
if (paramDescriptors.size != nParams) return null
val paramDesc = paramDescriptors.getOrNull(paramIndex) ?: return null
// Reference types start with 'L' or '['; Java primitives are single chars
descriptorKinds.add(paramDesc.startsWith("L") || paramDesc.startsWith("["))
return null
}
},
org.jetbrains.org.objectweb.asm.ClassReader.SKIP_CODE or
org.jetbrains.org.objectweb.asm.ClassReader.SKIP_DEBUG or
org.jetbrains.org.objectweb.asm.ClassReader.SKIP_FRAMES
)
descriptorKinds.singleOrNull()
} catch (e: Exception) {
null
}
}
/**
* Checks whether the return type of a Java binary method (identified by [methodName] and
* [nParams]) is a reference type (as opposed to a Java primitive).
*
* Returns `null` if the information cannot be determined.
*/
fun javaBinaryMethodReturnIsClassifierType(
parentClass: IrClass,
methodName: String,
nParams: Int,
isConstructor: Boolean
): Boolean? {
if (isConstructor) return false
// K1 path: binary Java class has JavaSourceElement with a BinaryJavaClass.
val k1ReturnKinds =
((parentClass.source as? JavaSourceElement)?.javaElement as? BinaryJavaClass)?.methods
?.asSequence()
?.filter { it.name.asString() == methodName && it.valueParameters.size == nParams }
?.map { it.returnType is org.jetbrains.kotlin.load.java.structure.JavaClassifierType }
?.toSet()
if (k1ReturnKinds != null && k1ReturnKinds.isNotEmpty()) {
return k1ReturnKinds.singleOrNull()
}
// K2 path: binary Java class has VirtualFileBasedSourceElement
if (parentClass.source !is VirtualFileBasedSourceElement) return null
val vf = (parentClass.source as VirtualFileBasedSourceElement).virtualFile
if (!vf.name.endsWith(".class")) return null
return try {
val bytes = vf.contentsToByteArray()
val returnKinds = mutableSetOf<Boolean>()
val reader = org.jetbrains.org.objectweb.asm.ClassReader(bytes)
reader.accept(
object : org.jetbrains.org.objectweb.asm.ClassVisitor(
org.jetbrains.org.objectweb.asm.Opcodes.ASM9
) {
override fun visitMethod(
access: Int,
name: String,
descriptor: String,
signature: String?,
exceptions: Array<String>?
): org.jetbrains.org.objectweb.asm.MethodVisitor? {
if (name != methodName) return null
if (parseAsmMethodDescriptorParams(descriptor).size != nParams) return null
val returnDescriptor = descriptor.substring(descriptor.lastIndexOf(')') + 1)
returnKinds.add(
returnDescriptor.startsWith("L") || returnDescriptor.startsWith("[")
)
return null
}
},
org.jetbrains.org.objectweb.asm.ClassReader.SKIP_CODE or
org.jetbrains.org.objectweb.asm.ClassReader.SKIP_DEBUG or
org.jetbrains.org.objectweb.asm.ClassReader.SKIP_FRAMES
)
returnKinds.singleOrNull()
} catch (e: Exception) {
null
}
}
fun parseAsmMethodDescriptorParams(descriptor: String): List<String> {
val params = mutableListOf<String>()
var i = descriptor.indexOf('(') + 1
val end = descriptor.lastIndexOf(')')
while (i < end) {
when (val c = descriptor[i]) {
'L' -> {
val semi = descriptor.indexOf(';', i)
params.add(descriptor.substring(i, semi + 1))
i = semi + 1
}
'[' -> {
var j = i + 1
while (j < end && descriptor[j] == '[') j++
if (descriptor[j] == 'L') {
val semi = descriptor.indexOf(';', j)
params.add(descriptor.substring(i, semi + 1))
i = semi + 1
} else {
params.add(descriptor.substring(i, j + 1))
i = j + 1
}
}
else -> { params.add(c.toString()); i++ }
}
}
return params
}

View File

@@ -1,11 +1,11 @@
import pathlib
def test(codeql, java_full):
def test(codeql, java_full, kotlinc_2_3_20):
java_srcs = " ".join([str(s) for s in pathlib.Path().glob("*.java")])
codeql.database.create(
command=[
f"javac {java_srcs} -d build",
"kotlinc -language-version 2.0 user.kt -cp build",
"kotlinc -language-version 1.9 user.kt -cp build",
]
)

View File

@@ -1,6 +1,6 @@
import commands
def test(codeql, java_full):
commands.run("kotlinc -language-version 2.0 test.kt -d lib")
codeql.database.create(command="kotlinc -language-version 2.0 user.kt -cp lib")
def test(codeql, java_full, kotlinc_2_3_20):
commands.run("kotlinc -language-version 1.9 test.kt -d lib")
codeql.database.create(command="kotlinc -language-version 1.9 user.kt -cp lib")

View File

@@ -9,4 +9,4 @@
| Percentage of calls with call target | 100 |
| Total number of lines | 3 |
| Total number of lines with extension kt | 3 |
| Uses Kotlin 2: true | 1 |
| Uses Kotlin 2: false | 1 |

View File

@@ -1,2 +1,2 @@
def test(codeql, java_full):
codeql.database.create(command="kotlinc -J-Xmx2G -language-version 2.0 SomeClass.kt")
def test(codeql, java_full, kotlinc_2_3_20):
codeql.database.create(command=f"kotlinc -J-Xmx2G -language-version 1.9 SomeClass.kt")

View File

@@ -1,6 +1,6 @@
import commands
def test(codeql, java_full):
commands.run("kotlinc -language-version 2.0 A.kt")
codeql.database.create(command="kotlinc -cp . -language-version 2.0 B.kt C.kt")
def test(codeql, java_full, kotlinc_2_3_20):
commands.run("kotlinc -language-version 1.9 A.kt")
codeql.database.create(command="kotlinc -cp . -language-version 1.9 B.kt C.kt")

View File

@@ -1,6 +1,6 @@
import commands
def test(codeql, java_full):
def test(codeql, java_full, kotlinc_2_3_20):
commands.run(["javac", "Test.java", "-d", "bin"])
codeql.database.create(command="kotlinc -language-version 2.0 user.kt -cp bin")
codeql.database.create(command="kotlinc -language-version 1.9 user.kt -cp bin")

View File

@@ -1,13 +1,13 @@
import commands
def test(codeql, java_full):
def test(codeql, java_full, kotlinc_2_3_20):
# Compile the JavaDefns2 copy outside tracing, to make sure the Kotlin view of it matches the Java view seen by the traced javac compilation of JavaDefns.java below.
commands.run(["javac", "JavaDefns2.java"])
codeql.database.create(
command=[
"kotlinc kotlindefns.kt",
"javac JavaUser.java JavaDefns.java -cp .",
"kotlinc -language-version 2.0 -cp . kotlinuser.kt",
"kotlinc -language-version 1.9 -cp . kotlinuser.kt",
]
)

View File

@@ -0,0 +1,2 @@
import semmle.python.controlflow.internal.AstNodeImpl
import ControlFlow::Consistency

View File

@@ -0,0 +1,4 @@
---
category: minorAnalysis
---
* A new Python control flow graph implementation has been added under `semmle.python.controlflow.internal.Cfg` (backed by `AstNodeImpl.qll`), built on the shared `codeql.controlflow.ControlFlowGraph` library. It is not yet used by the dataflow library or any production query; the legacy CFG in `semmle/python/Flow.qll` remains the default. The new library is exposed for tests and for upcoming migrations.

View File

@@ -0,0 +1,4 @@
---
category: minorAnalysis
---
* The new (shared-CFG-based) Python control flow graph now visits parameter and return type annotations as CFG nodes for function definitions, matching the legacy CFG. This restores annotation-based type tracking through framework models such as FastAPI's `Depends()`, Pydantic request models, Starlette `WebSocket` handlers, and any other models that flow a class reference through `Parameter.getAnnotation()` to identify instances of the annotated class.

View File

@@ -0,0 +1,42 @@
/**
* @name Print CFG
* @description Produces a representation of a file's Control Flow Graph.
* This query is used by the VS Code extension.
* @id py/print-cfg
* @kind graph
* @tags ide-contextual-queries/print-cfg
*/
import semmle.python.Files as Files
// import semmle.python.Scope
import semmle.python.controlflow.internal.AstNodeImpl
external string selectedSourceFile();
private predicate selectedSourceFileAlias = selectedSourceFile/0;
external int selectedSourceLine();
private predicate selectedSourceLineAlias = selectedSourceLine/0;
external int selectedSourceColumn();
private predicate selectedSourceColumnAlias = selectedSourceColumn/0;
module ViewCfgQueryInput implements ControlFlow::ViewCfgQueryInputSig<Files::File> {
predicate selectedSourceFile = selectedSourceFileAlias/0;
predicate selectedSourceLine = selectedSourceLineAlias/0;
predicate selectedSourceColumn = selectedSourceColumnAlias/0;
predicate cfgScopeSpan(
Ast::Callable scope, Files::File file, int startLine, int startColumn, int endLine,
int endColumn
) {
file = scope.getLocation().getFile() and
scope.getLocation().hasLocationInfo(_, startLine, startColumn, endLine, endColumn)
}
}
import ControlFlow::ViewCfgQuery<Files::File, ViewCfgQueryInput>

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,4 @@
consistencyOverview
| deadEnd | 1 |
deadEnd
| without_loop.py:7:5:7:9 | Break |

View File

@@ -0,0 +1,32 @@
/**
* Phase -1 of the dataflow CFG migration: verifies that every variable
* binding visible to the AST (`Name.defines(v)`) corresponds to a CFG node
* in the new CFG (`semmle.python.controlflow.internal.AstNodeImpl`).
*
* The expected tag is `cfgdefines=<name>`. Each binding annotation in the
* test sources looks like `# $ cfgdefines=x` for a binding currently
* covered by the new CFG, or `# $ MISSING: cfgdefines=x` for a binding
* that is known to be uncovered (a "red" test case that should be
* green-flipped once the corresponding `cfg-ext-*` extension lands).
*/
import python
import semmle.python.controlflow.internal.AstNodeImpl as CfgImpl
import utils.test.InlineExpectationsTest
module CfgBindingsTest implements TestSig {
string getARelevantTag() { result = "cfgdefines" }
predicate hasActualResult(Location location, string element, string tag, string value) {
exists(Name n, Variable v, CfgImpl::ControlFlowNode cfg |
n.defines(v) and
cfg.getAstNode().asExpr() = n and
location = n.getLocation() and
element = n.toString() and
tag = "cfgdefines" and
value = v.getId()
)
}
}
import MakeTest<CfgBindingsTest>

View File

@@ -0,0 +1,13 @@
# Annotated assignment (PEP 526). Both with and without an initializer.
a: int = 1 # $ cfgdefines=a
b: str = "hi" # $ cfgdefines=b
# Annotation without value: the AST records `c` as defined,
# and the new CFG now visits it via the AnnAssignStmt wrapper.
c: int # $ cfgdefines=c
class K: # $ cfgdefines=K
field: int = 0 # $ cfgdefines=field

View File

@@ -0,0 +1,14 @@
# Compound (tuple/list) assignment targets — actually wired in the new CFG.
a, b = (1, 2) # $ cfgdefines=a cfgdefines=b
[c, d] = [3, 4] # $ cfgdefines=c cfgdefines=d
# Nested unpacking.
(e, (f, g)) = (1, (2, 3)) # $ cfgdefines=e cfgdefines=f cfgdefines=g
# Star unpacking.
h, *i = [1, 2, 3] # $ cfgdefines=h cfgdefines=i
# Chained assignment with compound target.
j = k, l = (5, 6) # $ cfgdefines=j cfgdefines=k cfgdefines=l

View File

@@ -0,0 +1,21 @@
# Comprehension and `for` loop targets — wired in the new CFG.
# Comprehensions are nested function scopes with a synthetic `.0` parameter
# bound to the iterable.
# Bare-name `for` target.
for i in range(3): # $ cfgdefines=i
pass
# Compound `for` target.
for k, v in [(1, 2)]: # $ cfgdefines=k cfgdefines=v
pass
# Comprehension targets.
_ = [x for x in range(3)] # $ cfgdefines=_ cfgdefines=x cfgdefines=.0
_ = {y: z for y, z in []} # $ cfgdefines=_ cfgdefines=y cfgdefines=z cfgdefines=.0
_ = (a for a in []) # $ cfgdefines=_ cfgdefines=a cfgdefines=.0
# Nested comprehensions.
_ = [b for c in [] for b in c] # $ cfgdefines=_ cfgdefines=c cfgdefines=b cfgdefines=.0

View File

@@ -0,0 +1,53 @@
# Reachability of code following a try whose body always returns.
#
# The new CFG models exception edges for raise-prone expressions when
# they appear inside a `try` (or `with`) statement, mirroring Java's
# `mayThrow`. This means the body of a `try` has both a normal
# completion edge and an exception edge to its handlers, so code
# following the try-statement is reachable via the except-handler path
# even when the try-body would otherwise always return.
#
# Code that is not reachable under either normal or exception flow
# (for example, the `else` clause of a try whose body unconditionally
# raises) remains correctly classified as dead.
def f(obj): # $ cfgdefines=f cfgdefines=obj
try:
return len(obj)
except TypeError:
pass
# The try-body always returns, but `len(obj)` can raise (it is
# inside the try, so we model its exception edge). The
# `except TypeError: pass` handler falls through to here, making
# the code below reachable.
try:
hint = type(obj).__length_hint__ # $ cfgdefines=hint
except AttributeError:
return None
return hint
def g(): # $ cfgdefines=g
try:
raise Exception("inner")
except:
raise Exception("outer")
else:
# Unreachable: the inner try body always raises (via an explicit
# `raise`, which is modelled unconditionally), so the `else:`
# clause never runs.
hit_inner_else = True
def h(cache, key): # $ cfgdefines=h cfgdefines=cache cfgdefines=key
try:
return cache[key]
except KeyError:
pass
# Same pattern as `f`: reachable via the except-handler fall-through.
value = compute(key) # $ cfgdefines=value
cache[key] = value
return value

View File

@@ -0,0 +1,30 @@
# Decorated `def`/`class` — wired in the new CFG.
def deco(f): # $ cfgdefines=deco cfgdefines=f
return f
@deco
def decorated_func(): # $ cfgdefines=decorated_func
pass
@deco
class DecoratedClass: # $ cfgdefines=DecoratedClass
pass
# Stacked decorators.
@deco
@deco
def doubly(): # $ cfgdefines=doubly
pass
# Inside a class body.
class Outer: # $ cfgdefines=Outer
@staticmethod
def inner(): # $ cfgdefines=inner
pass

View File

@@ -0,0 +1,19 @@
# Exception-handler name bindings. These are already wired in the new
# CFG provided the try body can raise; `raise` statements are reliably
# treated as exception sources.
try:
raise ValueError("oops")
except ValueError as e: # $ cfgdefines=e
pass
try:
raise TypeError("oops")
except (TypeError, KeyError) as err: # $ cfgdefines=err
pass
# Exception groups (Python 3.11+).
try:
raise ValueError("oops")
except* ValueError as eg: # $ cfgdefines=eg
pass

View File

@@ -0,0 +1,14 @@
# Import aliases — all bound names below are now reachable via the new
# CFG's `ImportStmt` wrapper.
import os # $ cfgdefines=os
import os.path # $ cfgdefines=os
import os as o # $ cfgdefines=o
from os import path # $ cfgdefines=path
from os import path as p # $ cfgdefines=p
from os import sep, linesep # $ cfgdefines=sep cfgdefines=linesep
from os import (
getcwd, # $ cfgdefines=getcwd
getcwdb, # $ cfgdefines=getcwdb
)

View File

@@ -0,0 +1,24 @@
# Match-statement pattern bindings — wired in the new CFG.
def f(subject): # $ cfgdefines=f cfgdefines=subject
match subject:
case x: # $ cfgdefines=x
pass
case [a, b]: # $ cfgdefines=a cfgdefines=b
pass
case {"k": v}: # $ cfgdefines=v
pass
case Point(p, q): # $ cfgdefines=p cfgdefines=q
pass
case [_, *rest]: # $ cfgdefines=rest
pass
case (1 | 2) as n: # $ cfgdefines=n
pass
class Point: # $ cfgdefines=Point
__match_args__ = ("x", "y") # $ cfgdefines=__match_args__
x: int # $ cfgdefines=x
y: int # $ cfgdefines=y

View File

@@ -0,0 +1,42 @@
# Function parameters.
def positional(a, b): # $ cfgdefines=positional cfgdefines=a cfgdefines=b
pass
def with_default(x=1, y=2): # $ cfgdefines=with_default cfgdefines=x cfgdefines=y
pass
def with_vararg(*args): # $ cfgdefines=with_vararg cfgdefines=args
pass
def with_kwarg(**kwargs): # $ cfgdefines=with_kwarg cfgdefines=kwargs
pass
def with_kwonly(*, k1, k2=5): # $ cfgdefines=with_kwonly cfgdefines=k1 cfgdefines=k2
pass
def kitchen_sink(a, b=2, *args, k1, k2=5, **kw): # $ cfgdefines=kitchen_sink cfgdefines=a cfgdefines=b cfgdefines=args cfgdefines=k1 cfgdefines=k2 cfgdefines=kw
pass
# Methods get `self` / `cls`.
class C: # $ cfgdefines=C
def method(self, x): # $ cfgdefines=method cfgdefines=self cfgdefines=x
pass
@classmethod
def cmethod(cls, x): # $ cfgdefines=cmethod cfgdefines=cls cfgdefines=x
pass
# Lambda parameter.
_ = lambda p: p + 1 # $ cfgdefines=_ cfgdefines=p
# PEP 570 positional-only.
def pos_only(a, b, /, c): # $ cfgdefines=pos_only cfgdefines=a cfgdefines=b cfgdefines=c
pass

View File

@@ -0,0 +1,14 @@
# Simple bindings that should already work in the new CFG.
# No MISSING annotations expected.
x = 1 # $ cfgdefines=x
y = x + 1 # $ cfgdefines=y
def f(): # $ cfgdefines=f
pass
class C: # $ cfgdefines=C
pass
# Re-assignment.
x = 2 # $ cfgdefines=x

View File

@@ -0,0 +1,21 @@
# PEP 695 type parameters (Python 3.12+).
# PEP 695 type-param names on `def`/`class` bind in an annotation scope
# that nests the function/class body — they have no CFG node in the
# enclosing scope (matching the legacy CFG).
def func[T](x: T) -> T: # $ cfgdefines=func cfgdefines=x
return x
class Box[T]: # $ cfgdefines=Box
item: T # $ cfgdefines=item
# Multi-parameter, with bound and variadics.
def multi[T: int, *Ts, **P](x: T, *args: *Ts, **kwargs: P.kwargs) -> T: # $ cfgdefines=multi cfgdefines=x cfgdefines=args cfgdefines=kwargs
return x
# `type` statement (PEP 695).
type Alias[T] = list[T] # $ cfgdefines=Alias cfgdefines=T

View File

@@ -0,0 +1,14 @@
# Walrus and starred-target edge cases — wired in the new CFG.
# Walrus in expression context.
if (y := 5) > 0: # $ cfgdefines=y
pass
# Walrus in a comprehension. The comprehension introduces a synthetic
# `.0` parameter bound to the iterable.
_ = [w for _ in range(3) if (w := 1)] # $ cfgdefines=_ cfgdefines=w cfgdefines=.0
# Starred target in a Tuple LHS.
*head, tail = [1, 2, 3] # $ cfgdefines=head cfgdefines=tail

View File

@@ -0,0 +1,21 @@
# `with cm() as x:` bindings — wired in the new CFG.
class CM: # $ cfgdefines=CM
def __enter__(self): return self # $ cfgdefines=__enter__ cfgdefines=self
def __exit__(self, *a): pass # $ cfgdefines=__exit__ cfgdefines=self cfgdefines=a
with CM() as x: # $ cfgdefines=x
pass
# Multiple items.
with CM() as a, CM() as b: # $ cfgdefines=a cfgdefines=b
pass
# Parenthesised form (Python 3.10+).
with (CM() as p, CM() as q): # $ cfgdefines=p cfgdefines=q
pass
# Compound target in `with`.
with CM() as (m, n): # $ cfgdefines=m cfgdefines=n
pass

View File

@@ -0,0 +1,14 @@
/** New-CFG version of AllLiveReachable. */
import python
import TimerUtils
import NewCfgImpl
private module Utils = EvalOrderCfgUtils<NewCfg>;
private import Utils
private import Utils::CfgTests
from TimerCfgNode a, TestFunction f
where allLiveReachable(a, f)
select a, "Unreachable live annotation; entry of $@ does not reach this node", f, f.getName()

View File

@@ -0,0 +1,18 @@
/**
* New-CFG version of AnnotationHasCfgNode.
*
* Checks that every timer annotation has a corresponding CFG node.
*/
import python
import TimerUtils
import NewCfgImpl
private module Utils = EvalOrderCfgUtils<NewCfg>;
private import Utils::CfgTests
from TimerAnnotation ann
where annotationWithoutCfgNode(ann)
select ann, "Annotation in $@ has no CFG node", ann.getTestFunction(),
ann.getTestFunction().getName()

View File

@@ -0,0 +1,26 @@
/**
* New-CFG version of BasicBlockAnnotationGap.
*
* Original:
* Checks that within a basic block, if a node is annotated then its
* successor is also annotated (or excluded). A gap in annotations
* within a basic block indicates a missing annotation, since there
* are no branches to justify the gap.
*
* Nodes with exceptional successors are excluded, as the exception
* edge leaves the basic block and the normal successor may be dead.
*/
import python
import TimerUtils
import NewCfgImpl
private module Utils = EvalOrderCfgUtils<NewCfg>;
private import Utils
private import Utils::CfgTests
from TimerCfgNode a, CfgNode succ
where basicBlockAnnotationGap(a, succ)
select a, "Annotated node followed by unannotated $@ in the same basic block", succ,
succ.getNode().toString()

View File

@@ -0,0 +1,21 @@
/**
* New-CFG version of BasicBlockOrdering.
*
* Original:
* Checks that within a single basic block, annotations appear in
* increasing minimum-timestamp order.
*/
import python
import TimerUtils
import NewCfgImpl
private module Utils = EvalOrderCfgUtils<NewCfg>;
private import Utils
private import Utils::CfgTests
from TimerCfgNode a, TimerCfgNode b, int minA, int minB
where basicBlockOrdering(a, b, minA, minB)
select a, "Basic block ordering: $@ appears before $@", a.getTimestampExpr(minA),
"timestamp " + minA, b.getTimestampExpr(minB), "timestamp " + minB

View File

@@ -0,0 +1,80 @@
/**
* New-CFG version of BranchTimestamps.
*
* Checks that when a node has both a true and false successor, the
* live timestamps on one branch are marked as dead on the other.
* This ensures that boolean branches are fully annotated with dead()
* markers for the paths not taken.
*
* Limitation: the `@ t[ts, ...]` / `dead(ts)` annotation scheme can only
* model branch-dead-ness for plain boolean control flow that reconverges
* linearly after the split — i.e. `if`-with-else and `if`-expression.
* It cannot model:
*
* * loops (`while` / `for`): body timestamps repeat across iterations,
* so the loop-exit annotation can't list them as dead;
* * `match` statements: each `case` body is a syntactically distinct
* sub-tree, and the branches don't reconverge through a common
* annotation point in the timeline;
* * `try` / `with` and `raise` / `assert`: exception edges are modelled
* as true/false but flow to syntactically distinct handlers, with no
* reconvergence in the linear annotation order;
* * short-circuit `and` / `or` (`BoolExpr`): the branches reconverge at
* the BoolExpr's after-node, so timestamps on one branch are live
* downstream of the other rather than dead;
* * `if` without an `else` clause, and `if`/`elif` chains: the false
* branch reconverges with the true branch at the post-if statement
* (no-else) or fans out across multiple elif-test annotations,
* neither of which fit the binary annotation scheme.
*
* Branch nodes inside those constructs are therefore whitelisted out
* below. The check still fires (and is useful) for plain `if`/`else`
* and conditional-expression branching.
*/
import python
import TimerUtils
import NewCfgImpl
private module Utils = EvalOrderCfgUtils<NewCfg>;
private import Utils
private import Utils::CfgTests
/**
* Holds if `f` contains a construct whose branches the linear-timestamp
* annotation scheme cannot describe (see file-level comment).
*/
private predicate hasUnmodellableBranching(Function f) {
exists(AstNode bad |
bad.getScope() = f and
(
bad instanceof While
or
bad instanceof For
or
bad instanceof MatchStmt
or
bad instanceof Try
or
bad instanceof With
or
bad instanceof Raise
or
bad instanceof Assert
or
bad instanceof BoolExpr
or
bad instanceof If and
(not exists(bad.(If).getAnOrelse()) or bad.(If).isElif())
)
)
}
from TimerCfgNode node, int ts, string branch
where
missingBranchTimestamp(node, ts, branch) and
not hasUnmodellableBranching(node.getTestFunction())
select node,
"Timestamp " + ts + " on true/false branch is missing a dead() annotation on the " + branch +
" successor in $@", node.getTestFunction(), node.getTestFunction().getName()

View File

@@ -0,0 +1,22 @@
/**
* New-CFG version of ConsecutivePredecessorTimestamps.
*
* Checks that each annotated node (except the minimum timestamp) has
* a predecessor annotation with timestamp `a - 1`. This is the reverse
* of ConsecutiveTimestamps: it catches nodes that are reachable but
* arrived at from the wrong place (skipping an intermediate node).
*/
import python
import TimerUtils
import NewCfgImpl
private module Utils = EvalOrderCfgUtils<NewCfg>;
private import Utils
private import Utils::CfgTests
from TimerAnnotation ann, int a
where consecutivePredecessorTimestamps(ann, a)
select ann, "$@ in $@ has no consecutive predecessor (expected " + (a - 1) + ")",
ann.getTimestampExpr(a), "Timestamp " + a, ann.getTestFunction(), ann.getTestFunction().getName()

View File

@@ -0,0 +1,29 @@
/**
* New-CFG version of ConsecutiveTimestamps.
*
* Original:
* Checks that consecutive annotated nodes have consecutive timestamps:
* for each annotation with timestamp `a`, some CFG node for that annotation
* must have a next annotation containing `a + 1`.
*
* Handles CFG splitting (e.g., finally blocks duplicated for normal/exceptional
* flow) by checking that at least one split has the required successor.
*
* Only applies to functions where all annotations are in the function's
* own scope (excludes tests with generators, async, comprehensions, or
* lambdas that have annotations in nested scopes).
*/
import python
import TimerUtils
import NewCfgImpl
private module Utils = EvalOrderCfgUtils<NewCfg>;
private import Utils
private import Utils::CfgTests
from TimerAnnotation ann, int a
where consecutiveTimestamps(ann, a)
select ann, "$@ in $@ has no consecutive successor (expected " + (a + 1) + ")",
ann.getTimestampExpr(a), "Timestamp " + a, ann.getTestFunction(), ann.getTestFunction().getName()

View File

@@ -0,0 +1,120 @@
/**
* Implementation of the evaluation-order CFG signature using the new
* shared control flow graph from AstNodeImpl.
*/
private import python as Py
import TimerUtils
private import semmle.python.controlflow.internal.AstNodeImpl as CfgImpl
private import codeql.controlflow.SuccessorType
private class NewControlFlowNode = CfgImpl::ControlFlowNode;
private class NewBasicBlock = CfgImpl::BasicBlock;
/** New (shared) CFG implementation of the evaluation-order signature. */
module NewCfg implements EvalOrderCfgSig {
class CfgNode instanceof NewControlFlowNode {
// We must pick a *unique* representative CFG node for each AST node. The
// shared CFG has several nodes per AST node (before / in-post-order / after
// / after-value splits), but the timer test framework keys annotations on
// `getNode()` and assumes one CFG node per annotated AST node. Without a
// filter, an annotated `f()` would map to both `f()` and `After f()`, which
// breaks two framework invariants: (1) the "no shared reachable" check
// requires that two distinct nodes sharing a timestamp be mutually
// unreachable (true/false branches of a condition), but `Before f()`,
// `f()` and `After f()` share the annotation's timestamp *and* lie on one
// linear path; and (2) the annotation walk (`nextTimerAnnotation`) halts at
// the first reachable representative, so a second node for the same AST
// node would stall the walk on the same timestamp instead of advancing to
// the next evaluation event.
//
// We use the "after" node (`isAfter`) rather than the canonical `injects`
// node, because `injects` represents short-circuit / conditional
// expressions (`and`/`or`/`not`/ternary) by their *before* node, placing
// them ahead of their operands — wrong for evaluation order. `isAfter`
// instead picks the post-evaluation node: the merged before/after node for
// simple leaves, the `TAfterNode` for post-order expressions, and the
// `AfterValueNode`(s) for pre-order conditionals, all positioned after the
// operands. The two value-split nodes of a conditional are genuinely
// distinct evaluation outcomes (handled by `getATrueSuccessor` /
// `getAFalseSuccessor`), so they do not violate the uniqueness assumption.
CfgNode() { NewControlFlowNode.super.isAfter(_) }
string toString() { result = NewControlFlowNode.super.toString() }
Py::Location getLocation() { result = NewControlFlowNode.super.getLocation() }
Py::AstNode getNode() {
result = CfgImpl::astNodeToPyNode(NewControlFlowNode.super.getAstNode())
}
CfgNode getASuccessor() { nextCfgNode(this, result) }
CfgNode getATrueSuccessor() {
NewControlFlowNode.super.isAfterTrue(_) and
// Only where there's also a false branch (true boolean split)
exists(NewControlFlowNode other | other.isAfterFalse(NewControlFlowNode.super.getAstNode())) and
nextCfgNodeFrom(this, result)
}
CfgNode getAFalseSuccessor() {
NewControlFlowNode.super.isAfterFalse(_) and
// Only where there's also a true branch (true boolean split)
exists(NewControlFlowNode other | other.isAfterTrue(NewControlFlowNode.super.getAstNode())) and
nextCfgNodeFrom(this, result)
}
CfgNode getAnExceptionalSuccessor() {
exists(NewControlFlowNode mid |
mid = NewControlFlowNode.super.getAnExceptionSuccessor() and
nextCfgNodeFrom(mid, result)
)
}
Py::Scope getScope() { result = NewControlFlowNode.super.getEnclosingCallable().asScope() }
BasicBlock getBasicBlock() {
exists(NewBasicBlock bb, int i | bb.getNode(i) = this and result = bb)
}
}
/**
* Holds if `next` is the nearest CfgNode reachable from `n` via
* one or more raw CFG successor edges, skipping non-CfgNode intermediaries.
*/
private predicate nextCfgNodeFrom(NewControlFlowNode n, CfgNode next) {
next = n.getASuccessor()
or
exists(NewControlFlowNode mid |
mid = n.getASuccessor() and
not mid instanceof CfgNode and
nextCfgNodeFrom(mid, next)
)
}
/**
* Holds if `next` is the nearest CfgNode successor of `n`,
* skipping synthetic intermediate nodes.
*/
private predicate nextCfgNode(CfgNode n, CfgNode next) { nextCfgNodeFrom(n, next) }
class BasicBlock instanceof NewBasicBlock {
string toString() { result = NewBasicBlock.super.toString() }
CfgNode getNode(int n) { result = NewBasicBlock.super.getNode(n) }
predicate reaches(BasicBlock bb) { this = bb or this.strictlyReaches(bb) }
predicate strictlyReaches(BasicBlock bb) { NewBasicBlock.super.getASuccessor+() = bb }
predicate strictlyDominates(BasicBlock bb) { NewBasicBlock.super.strictlyDominates(bb) }
}
CfgNode scopeGetEntryNode(Py::Scope s) {
exists(CfgImpl::ControlFlow::EntryNode entry |
entry.getEnclosingCallable().asScope() = s and
nextCfgNodeFrom(entry, result)
)
}
}

View File

@@ -0,0 +1,21 @@
/**
* New-CFG version of NeverReachable.
*
* Original:
* Checks that expressions annotated with `t.never` either have no CFG
* node, or if they do, that the node is not reachable from its scope's
* entry (including within the same basic block).
*/
import python
import TimerUtils
import NewCfgImpl
private module Utils = EvalOrderCfgUtils<NewCfg>;
private import Utils::CfgTests
from TimerAnnotation ann
where neverReachable(ann)
select ann, "Node annotated with t.never is reachable in $@", ann.getTestFunction(),
ann.getTestFunction().getName()

View File

@@ -0,0 +1,22 @@
/**
* New-CFG version of NoBackwardFlow.
*
* Original:
* Checks that time never flows backward between consecutive timer annotations
* in the CFG. For each pair of consecutive annotated nodes (A -> B), there must
* exist timestamps a in A and b in B with a < b.
*/
import python
import TimerUtils
import NewCfgImpl
private module Utils = EvalOrderCfgUtils<NewCfg>;
private import Utils
private import Utils::CfgTests
from TimerCfgNode a, TimerCfgNode b, int minA, int maxB
where noBackwardFlow(a, b, minA, maxB)
select a, "Backward flow: $@ flows to $@ (max timestamp $@)", a.getTimestampExpr(minA),
minA.toString(), b, b.getNode().toString(), b.getTimestampExpr(maxB), maxB.toString()

View File

@@ -0,0 +1,18 @@
/**
* New-CFG version of NoBasicBlock.
*
* Checks that every annotated CFG node belongs to a basic block.
*/
import python
import TimerUtils
import NewCfgImpl
private module Utils = EvalOrderCfgUtils<NewCfg>;
private import Utils
private import Utils::CfgTests
from CfgNode n, TestFunction f
where noBasicBlock(n, f)
select n, "CFG node in $@ does not belong to any basic block", f, f.getName()

View File

@@ -0,0 +1,21 @@
/**
* New-CFG version of NoSharedReachable.
*
* Original:
* Checks that two annotations sharing a timestamp value are on
* mutually exclusive CFG paths (neither can reach the other).
*/
import python
import TimerUtils
import NewCfgImpl
private module Utils = EvalOrderCfgUtils<NewCfg>;
private import Utils
private import Utils::CfgTests
from TimerCfgNode a, TimerCfgNode b, int ts
where noSharedReachable(a, b, ts)
select a, "Shared timestamp $@ but this node reaches $@", a.getTimestampExpr(ts), ts.toString(), b,
b.getNode().toString()

View File

@@ -0,0 +1,22 @@
/**
* New-CFG version of StrictForward.
*
* Original:
* Stronger version of NoBackwardFlow: for consecutive annotated nodes
* A -> B that both have a single timestamp (non-loop code) and B does
* NOT dominate A (forward edge), requires max(A) < min(B).
*/
import python
import TimerUtils
import NewCfgImpl
private module Utils = EvalOrderCfgUtils<NewCfg>;
private import Utils
private import Utils::CfgTests
from TimerCfgNode a, TimerCfgNode b, int maxA, int minB
where strictForward(a, b, maxA, minB)
select a, "Strict forward violation: $@ flows to $@", a.getTimestampExpr(maxA), "timestamp " + maxA,
b.getTimestampExpr(minB), "timestamp " + minB

View File

@@ -3,14 +3,14 @@
* Python control flow graph.
*/
private import python as PY
private import python as Py
import TimerUtils
/** Existing Python CFG implementation of the evaluation-order signature. */
module OldCfg implements EvalOrderCfgSig {
class CfgNode = PY::ControlFlowNode;
class CfgNode = Py::ControlFlowNode;
class BasicBlock = PY::BasicBlock;
class BasicBlock = Py::BasicBlock;
CfgNode scopeGetEntryNode(PY::Scope s) { result = s.getEntryNode() }
CfgNode scopeGetEntryNode(Py::Scope s) { result = s.getEntryNode() }
}

View File

@@ -85,7 +85,7 @@ def test_nested_if_else(t):
else:
z = 2 @ t[dead(4)]
else:
z = 3 @ t[dead(4)]
z = 3 @ t[dead(3), dead(4)]
w = 0 @ t[5]

View File

@@ -0,0 +1,41 @@
/**
* Inline-expectations test for the store/load/delete/parameter
* classification predicates on the new-CFG facade.
*
* Each tag fires when the corresponding predicate (`isLoad`,
* `isStore`, `isDelete`, `isParameter`, `isAugLoad`, `isAugStore`)
* holds on the canonical CFG node wrapping a `Py::Name` with the
* given identifier. Subscript and attribute stores are not covered
* by these tags — only the `Name`-typed targets/loads they involve.
*/
import python
import semmle.python.controlflow.internal.Cfg as Cfg
import utils.test.InlineExpectationsTest
module StoreLoadTest implements TestSig {
string getARelevantTag() { result = ["load", "store", "delete", "param", "augload", "augstore"] }
predicate hasActualResult(Location location, string element, string tag, string value) {
exists(Cfg::NameNode n |
location = n.getLocation() and
element = n.toString() and
value = n.getId() and
(
n.isLoad() and not n.isAugLoad() and tag = "load"
or
n.isStore() and not n.isAugStore() and tag = "store"
or
n.isDelete() and tag = "delete"
or
n.isParameter() and tag = "param"
or
n.isAugLoad() and tag = "augload"
or
n.isAugStore() and tag = "augstore"
)
)
}
}
import MakeTest<StoreLoadTest>

View File

@@ -0,0 +1,56 @@
# Store/load/delete/parameter classification on the new-CFG facade.
#
# Each annotated location carries the (sorted, deduplicated) set of
# kinds the CFG facade reports there. Comparing against the legacy
# 'semmle.python.Flow' classification is done by the comparison query
# 'StoreLoadParity.ql' — annotations here are only the positive
# assertions for the new facade.
#
# Tags:
# load=<id> -- isLoad() fires on the Name
# store=<id> -- isStore() fires
# delete=<id> -- isDelete() fires
# param=<id> -- isParameter() fires
# augload=<id> -- isAugLoad() fires (the LHS of x += ... when read)
# augstore=<id> -- isAugStore() fires (the LHS of x += ... when written)
# --- plain load / store / delete ---
x = 1 # $ store=x
y = x + 1 # $ store=y load=x
print(y) # $ load=print load=y
del x # $ delete=x
# --- function definitions (parameters) ---
def f(a, b=2, *args, c, **kwargs): # $ store=f param=a param=b param=args param=c param=kwargs
return a + b + c # $ load=a load=b load=c
# --- augmented assignment splits one Name into load + store halves ---
def aug(): # $ store=aug
n = 0 # $ store=n
n += 1 # $ augload=n augstore=n
return n # $ load=n
# --- subscript / attribute stores ---
class C: # $ store=C
pass
def stores(obj, container, idx): # $ store=stores param=obj param=container param=idx
obj.attr = 1 # $ load=obj
container[idx] = 2 # $ load=container load=idx
return obj # $ load=obj
# --- tuple unpacking ---
def unpack(pair): # $ store=unpack param=pair
a, b = pair # $ store=a store=b load=pair
return a + b # $ load=a load=b

View File

@@ -66,7 +66,7 @@ impl<'a> AstNode for Node<'a> {
impl AstNode for yeast::Node {
fn kind(&self) -> &str {
yeast::Node::kind_name(self)
yeast::Node::kind(self)
}
fn is_named(&self) -> bool {
yeast::Node::is_named(self)
@@ -882,6 +882,7 @@ fn emit_extras_in(visitor: &mut Visitor, node: Node<'_>) {
}
fn traverse_yeast(tree: &yeast::Ast, visitor: &mut Visitor) {
use yeast::Cursor;
let mut cursor = tree.walk();
visitor.enter_node(cursor.node());
let mut recurse = true;

View File

@@ -41,14 +41,22 @@ pub fn query(input: TokenStream) -> TokenStream {
/// (kind "literal") - leaf with static content
/// (kind #{expr}) - leaf with computed content (expr.to_string())
/// (kind $fresh) - leaf with auto-generated unique name
/// {expr} - embed a Rust expression, dispatched via
/// the `IntoFieldIds` trait: `Id` pushes a
/// single id; iterables (`Vec<Id>`,
/// `Option<Id>`, iterator chains) splice
/// their elements
/// field: {expr} - extend a named field with `{expr}`'s ids
/// {expr} - embed a Rust expression returning Id
/// {..expr} - splice an iterable of Id (in child/field position)
/// field: {..expr} - splice into a named field
/// {expr}.map(p -> tpl) - apply tpl to each element; splice result
/// {expr}.reduce_left(f -> init, acc, e -> fold)
/// - fold with per-element init; splice 0 or 1 result
/// ```
///
/// Chain syntax after `{expr}` or `{..expr}`:
/// - `.map(param -> template)` — one output node per input element.
/// - `.reduce_left(first -> init, acc, elem -> fold)` — fold left; the first
/// element is converted by `init`, subsequent elements are folded by `fold`
/// with the accumulator bound to `acc`. An empty iterable yields nothing.
/// - Chains always splice (the result is iterable).
/// - Multiple chains can be chained, e.g. `.map(...).reduce_left(...)`.
///
/// Can be called with an explicit context or using the implicit context
/// from an enclosing `rule!`:
///
@@ -92,7 +100,7 @@ pub fn trees(input: TokenStream) -> TokenStream {
/// rule!(
/// (query_pattern field: (_) @name (kind)* @repeated (_)? @optional)
/// =>
/// (output_template field: {name} {repeated})
/// (output_template field: {name} {..repeated})
/// )
///
/// // Shorthand: captures become fields on the output node

View File

@@ -304,8 +304,7 @@ fn parse_ctx_or_implicit(tokens: &mut Tokens) -> Ident {
&& matches!(lookahead.next(), Some(TokenTree::Punct(p)) if p.as_char() == ',');
if is_explicit {
let ctx = expect_ident(tokens, "unreachable: ident was just peeked")
.expect("unreachable: ident was just peeked");
let ctx = expect_ident(tokens, "").unwrap();
let _ = tokens.next(); // consume comma
ctx
} else {
@@ -343,7 +342,7 @@ pub fn parse_trees_top(input: TokenStream) -> Result<TokenStream> {
}
Ok(quote! {
{
let mut __nodes: Vec<yeast::Id> = Vec::new();
let mut __nodes: Vec<usize> = Vec::new();
#(#items)*
__nodes
}
@@ -357,7 +356,7 @@ fn parse_direct_node(tokens: &mut Tokens, ctx: &Ident) -> Result<TokenStream> {
Some(TokenTree::Group(g)) if g.delimiter() == Delimiter::Brace => {
let group = expect_group(tokens, Delimiter::Brace)?;
let expr = group.stream();
Ok(quote! { ::std::convert::Into::<yeast::Id>::into({ #expr }) })
Ok(quote! { ::std::convert::Into::<usize>::into({ #expr }) })
}
Some(TokenTree::Group(g)) if g.delimiter() == Delimiter::Parenthesis => {
let group = expect_group(tokens, Delimiter::Parenthesis)?;
@@ -430,24 +429,49 @@ fn parse_direct_node_inner(tokens: &mut Tokens, ctx: &Ident) -> Result<TokenStre
);
field_counter += 1;
// Plain `field: {expr}` — trait-dispatched extend.
// Check for field: {..expr}.chain or field: {expr}.chain — splice a Vec<Id> into the field
if peek_is_group(tokens, Delimiter::Brace) {
let group = expect_group(tokens, Delimiter::Brace)?;
let expr = group.stream();
stmts.push(quote! {
let mut #temp: Vec<yeast::Id> = Vec::new();
yeast::IntoFieldIds::extend_into({ #expr }, &mut #temp);
});
// An empty `{expr}` means the field is absent — skip it
// entirely rather than emitting an empty named field.
field_args.push(quote! {
if !#temp.is_empty() { __fields.push((#field_str, #temp)); }
});
continue;
let group_clone = tokens.clone().next().unwrap();
if let TokenTree::Group(g) = &group_clone {
let mut inner_check = g.stream().into_iter();
let is_splice = matches!(inner_check.next(), Some(TokenTree::Punct(p)) if p.as_char() == '.')
&& matches!(inner_check.next(), Some(TokenTree::Punct(p)) if p.as_char() == '.');
// Determine if a chain (.map(..)) follows the `{}` group.
let mut after = tokens.clone();
after.next(); // skip the brace group
let has_chain =
matches!(after.peek(), Some(TokenTree::Punct(p)) if p.as_char() == '.');
if is_splice || has_chain {
let group = expect_group(tokens, Delimiter::Brace)?;
let base: TokenStream = if is_splice {
let mut inner = group.stream().into_iter().peekable();
inner.next(); // consume first .
inner.next(); // consume second .
let expr: TokenStream = inner.collect();
quote! {
{ #expr }.into_iter().map(::std::convert::Into::<usize>::into)
}
} else {
let expr = group.stream();
quote! { { #expr }.into_iter() }
};
let chained = parse_chain_suffix(tokens, ctx, base)?;
stmts.push(quote! {
let #temp: Vec<usize> = #chained.collect();
});
// An empty splice means the field is absent — skip it
// entirely rather than emitting an empty named field.
field_args.push(quote! {
if !#temp.is_empty() { __fields.push((#field_str, #temp)); }
});
continue;
}
}
}
let value = parse_direct_node(tokens, ctx)?;
stmts.push(quote! { let #temp: yeast::Id = #value; });
stmts.push(quote! { let #temp: usize = #value; });
field_args.push(quote! { __fields.push((#field_str, vec![#temp])); });
}
@@ -464,13 +488,101 @@ fn parse_direct_node_inner(tokens: &mut Tokens, ctx: &Ident) -> Result<TokenStre
Ok(quote! {
{
#(#stmts)*
let mut __fields: Vec<(&str, Vec<yeast::Id>)> = Vec::new();
let mut __fields: Vec<(&str, Vec<usize>)> = Vec::new();
#(#field_args)*
#ctx.node(#kind_str, __fields)
}
})
}
/// Parse a chain of `.method(args)` suffixes after a `{expr}` or `{..expr}`
/// placeholder in tree templates. Currently supports:
///
/// ```text
/// .map(param -> template) -- iterator map: produces Vec<usize>
/// ```
///
/// The chain may be empty (returns `base` unchanged). Multiple chained calls
/// are supported, e.g. `.map(p -> ...).map(q -> ...)`.
///
/// Each call expects the receiver to be an iterator. The `base` argument
/// should therefore already be an iterator (use `.into_iter()` on it before
/// calling this function).
fn parse_chain_suffix(tokens: &mut Tokens, ctx: &Ident, base: TokenStream) -> Result<TokenStream> {
let mut current = base;
while matches!(tokens.peek(), Some(TokenTree::Punct(p)) if p.as_char() == '.') {
tokens.next(); // consume .
let method = expect_ident(tokens, "expected method name after `.`")?;
let method_str = method.to_string();
let args_group = expect_group(tokens, Delimiter::Parenthesis)?;
match method_str.as_str() {
"map" => {
let mut inner = args_group.stream().into_iter().peekable();
let param = expect_ident(&mut inner, "expected lambda parameter name")?;
expect_punct(&mut inner, '-', "expected `->` after lambda parameter")?;
expect_punct(&mut inner, '>', "expected `->` after lambda parameter")?;
let body = parse_direct_node(&mut inner, ctx)?;
if let Some(tok) = inner.next() {
return Err(syn::Error::new_spanned(
tok,
"unexpected token after lambda body",
));
}
current = quote! {
#current.map(|#param| #body)
};
}
"reduce_left" => {
// Syntax: reduce_left(first -> init_tpl, acc, elem -> fold_tpl)
// - first -> init_tpl : converts the first element to the initial accumulator
// - acc, elem -> fold_tpl : fold step (acc = current accumulator, elem = next element)
// Empty iterator produces an empty iterator; non-empty produces a single-element iterator.
let mut inner = args_group.stream().into_iter().peekable();
let init_param = expect_ident(&mut inner, "expected initial lambda parameter")?;
expect_punct(&mut inner, '-', "expected `->` after init parameter")?;
expect_punct(&mut inner, '>', "expected `->` after init parameter")?;
let init_body = parse_direct_node(&mut inner, ctx)?;
expect_punct(&mut inner, ',', "expected `,` after init template")?;
let acc_param = expect_ident(&mut inner, "expected accumulator parameter")?;
expect_punct(&mut inner, ',', "expected `,` after accumulator parameter")?;
let elem_param = expect_ident(&mut inner, "expected element parameter")?;
expect_punct(&mut inner, '-', "expected `->` after element parameter")?;
expect_punct(&mut inner, '>', "expected `->` after element parameter")?;
let fold_body = parse_direct_node(&mut inner, ctx)?;
if let Some(tok) = inner.next() {
return Err(syn::Error::new_spanned(
tok,
"unexpected token after fold template",
));
}
current = quote! {
{
let mut __iter = #current;
let __result: Option<usize> = if let Some(#init_param) = __iter.next() {
let mut __acc: usize = #init_body;
for #elem_param in __iter {
let #acc_param: usize = __acc;
__acc = #fold_body;
}
Some(__acc)
} else {
None
};
__result.into_iter()
}
};
}
_ => {
return Err(syn::Error::new_spanned(
method,
format!("unknown builtin method `.{method_str}()`"),
));
}
}
}
Ok(current)
}
/// Parse the top-level list of a `trees!` template.
/// Each item is a node template or `{expr}` splice.
fn parse_direct_list(tokens: &mut Tokens, ctx: &Ident) -> Result<Vec<TokenStream>> {
@@ -491,14 +603,35 @@ fn parse_direct_list(tokens: &mut Tokens, ctx: &Ident) -> Result<Vec<TokenStream
continue;
}
// `{expr}` — extend `__nodes` via `IntoFieldIds`, which handles
// single ids and iterables uniformly.
// {expr} or {..expr} (with optional .chain) — single node or splice
if peek_is_group(tokens, Delimiter::Brace) {
let group = expect_group(tokens, Delimiter::Brace)?;
let expr = group.stream();
items.push(quote! {
yeast::IntoFieldIds::extend_into({ #expr }, &mut __nodes);
});
let has_chain =
matches!(tokens.peek(), Some(TokenTree::Punct(p)) if p.as_char() == '.');
let mut inner = group.stream().into_iter().peekable();
let is_splice = peek_is_dotdot(&inner);
if is_splice || has_chain {
let base: TokenStream = if is_splice {
inner.next(); // consume first .
inner.next(); // consume second .
let expr: TokenStream = inner.collect();
quote! {
{ #expr }.into_iter().map(::std::convert::Into::<usize>::into)
}
} else {
let expr = group.stream();
quote! { { #expr }.into_iter() }
};
let chained = parse_chain_suffix(tokens, ctx, base)?;
items.push(quote! {
__nodes.extend(#chained);
});
} else {
let expr = group.stream();
items.push(quote! {
__nodes.push(::std::convert::Into::<usize>::into({ #expr }));
});
}
continue;
}
@@ -516,7 +649,7 @@ struct CaptureInfo {
name: String,
multiplicity: CaptureMultiplicity,
/// `true` for `@@name` captures: the auto-translate prefix skips them,
/// so the bound `Id` refers to the raw (input-schema) node.
/// so the bound `NodeRef` refers to the raw (input-schema) node.
raw: bool,
}
@@ -671,17 +804,22 @@ pub fn parse_rule_top(input: TokenStream) -> Result<TokenStream> {
match cap.multiplicity {
CaptureMultiplicity::Repeated => {
quote! {
let #name: Vec<yeast::Id> = __captures.get_all(#name_str);
let #name: Vec<yeast::NodeRef> = __captures.get_all(#name_str)
.into_iter()
.map(yeast::NodeRef)
.collect();
}
}
CaptureMultiplicity::Optional => {
quote! {
let #name: Option<yeast::Id> = __captures.get_opt(#name_str);
let #name: Option<yeast::NodeRef> =
__captures.get_opt(#name_str).map(yeast::NodeRef);
}
}
CaptureMultiplicity::Single => {
quote! {
let #name: yeast::Id = __captures.get_var(#name_str).unwrap();
let #name: yeast::NodeRef =
yeast::NodeRef(__captures.get_var(#name_str).unwrap());
}
}
}
@@ -712,7 +850,7 @@ pub fn parse_rule_top(input: TokenStream) -> Result<TokenStream> {
__fields.insert(
__field_id,
#name.into_iter()
.map(::std::convert::Into::<yeast::Id>::into)
.map(::std::convert::Into::<usize>::into)
.collect(),
);
},
@@ -721,14 +859,14 @@ pub fn parse_rule_top(input: TokenStream) -> Result<TokenStream> {
.unwrap_or_else(|| panic!("field '{}' not found", #name_str));
if let Some(__id) = #name {
__fields.entry(__field_id).or_insert_with(Vec::new)
.push(::std::convert::Into::<yeast::Id>::into(__id));
.push(::std::convert::Into::<usize>::into(__id));
}
},
CaptureMultiplicity::Single => quote! {
let __field_id = #ctx_ident.ast.field_id_for_name(#name_str)
.unwrap_or_else(|| panic!("field '{}' not found", #name_str));
__fields.entry(__field_id).or_insert_with(Vec::new)
.push(::std::convert::Into::<yeast::Id>::into(#name));
.push(::std::convert::Into::<usize>::into(#name));
},
}
})
@@ -760,7 +898,7 @@ pub fn parse_rule_top(input: TokenStream) -> Result<TokenStream> {
}
quote! {
let mut __nodes: Vec<yeast::Id> = Vec::new();
let mut __nodes: Vec<usize> = Vec::new();
#(#transform_items)*
__nodes
}
@@ -781,7 +919,7 @@ pub fn parse_rule_top(input: TokenStream) -> Result<TokenStream> {
__translator.auto_translate_captures(&mut __captures, __ast, __user_ctx, __skip)?;
#(#bindings)*
let mut #ctx_ident = yeast::build::BuildCtx::with_translator(__ast, &__captures, __fresh, __source_range, __user_ctx, __translator);
let __result: Vec<yeast::Id> = { #transform_body };
let __result: Vec<usize> = { #transform_body };
Ok(__result)
}))
}
@@ -818,6 +956,13 @@ fn peek_is_hash(tokens: &mut Tokens) -> bool {
matches!(tokens.peek(), Some(TokenTree::Punct(p)) if p.as_char() == '#')
}
/// Check for `..` (two consecutive dot punctuation tokens).
fn peek_is_dotdot(tokens: &Tokens) -> bool {
let mut lookahead = tokens.clone();
matches!(lookahead.next(), Some(TokenTree::Punct(p)) if p.as_char() == '.')
&& matches!(lookahead.next(), Some(TokenTree::Punct(p)) if p.as_char() == '.')
}
fn peek_is_underscore(tokens: &mut Tokens) -> bool {
matches!(tokens.peek(), Some(TokenTree::Ident(id)) if *id == "_")
}

View File

@@ -214,7 +214,7 @@ yeast::tree!(ctx,
```rust
yeast::trees!(ctx,
(assignment left: {tmp} right: {right})
{body}
{..body}
)
```
@@ -256,26 +256,12 @@ occurrences of the same `$name` within one `BuildCtx` share the same value:
### Embedded Rust expressions
`{expr}` embeds a Rust expression whose value is appended to the
enclosing field (or to the rule body's id list). Dispatch happens via
the [`IntoFieldIds`] trait, which is implemented for:
- `Id` — pushes the single id.
- Any `IntoIterator<Item: Into<Id>>` — extends with all yielded ids
(covers `Vec<Id>`, `Option<Id>`, iterator chains, etc.).
So the same `{expr}` syntax handles single ids, splices, and zero-or-many
options uniformly:
`{expr}` embeds a Rust expression that returns a single node `Id`:
```rust
(assignment
left: {some_node_id} // a single Id
right: {rhs} // a captured value (inside rule!)
)
yeast::trees!(ctx,
(assignment left: {tmp} right: {right})
{extra_nodes} // splices a Vec<Id>
left: {some_node_id} // insert a pre-built node
right: {rhs} // insert a captured value (inside rule!)
)
```
@@ -291,17 +277,21 @@ expressions (with `let` bindings) work too:
})
```
Inside `rule!`, captures are Rust variables — `{name}` works for
single, optional, and repeated captures alike:
`{..expr}` splices a `Vec<Id>` (or any iterable of `Id`); the contents
are likewise a Rust block, so the splice can be the result of arbitrary
computation:
```rust
rule!(
(assignment left: @lhs right: _* @parts)
=>
(assignment left: {lhs} right: (block stmt: {parts}))
yeast::trees!(ctx,
(assignment left: {tmp} right: {right})
{..extra_nodes} // splice a Vec<Id>
)
```
Inside `rule!`, captures are Rust variables, so `{name}` inserts a
single capture (`Id`) and `{..name}` splices a repeated capture
(`Vec<Id>`).
### Raw captures (`@@name`)
The default `@name` capture marker is *auto-translated*: in OneShot
@@ -312,7 +302,7 @@ already conforms to the output schema.
For rules that need the raw (input-schema) capture — typically to read
its source text or to translate it explicitly with mutable context
state between calls — use `@@name` instead. The body sees the original
input-schema `Id`:
input-schema `NodeRef`:
```rust
yeast::rule!(
@@ -320,7 +310,7 @@ yeast::rule!(
=>
{
// raw_lhs is untranslated: read its original source text.
let text = ctx.ast.source_text(raw_lhs);
let text = ctx.ast.source_text(raw_lhs.into());
// rhs is already translated by the auto-translate prefix.
tree!((call
method: (identifier #{text.as_str()})

View File

@@ -158,6 +158,15 @@ impl<'a, C> BuildCtx<'a, C> {
self.ast
.create_named_token_with_range(kind, generated, self.source_range)
}
/// Prepend a value to a field of an existing node.
pub fn prepend_field(&mut self, node_id: Id, field_name: &str, value_id: Id) {
let field_id = self
.ast
.field_id_for_name(field_name)
.unwrap_or_else(|| panic!("build: field '{field_name}' not found"));
self.ast.prepend_field_child(node_id, field_id, value_id);
}
}
impl<C: Clone> BuildCtx<'_, C> {
@@ -167,6 +176,9 @@ impl<C: Clone> BuildCtx<'_, C> {
/// (translation is not meaningful when input and output share a
/// schema).
///
/// Accepts any value convertible to [`Id`] (including [`crate::NodeRef`]),
/// so manual rules can pass capture bindings directly without unwrapping.
///
/// Errors if this `BuildCtx` was constructed by hand (without a
/// translator handle) — for example, in unit tests that don't go
/// through the rule driver.
@@ -177,6 +189,20 @@ impl<C: Clone> BuildCtx<'_, C> {
None => Err("translate() called on a BuildCtx without a translator handle".into()),
}
}
/// Translate an optional capture, returning the first translated id or
/// `None`. Convenience for `?`-quantifier captures (`Option<NodeRef>`).
///
/// If the underlying translation produces multiple ids for a single
/// input, only the first is returned. For most use cases (e.g.
/// translating a single type annotation) this is what you want; if
/// you need all ids, use [`translate`] directly.
pub fn translate_opt<I: Into<Id>>(&mut self, id: Option<I>) -> Result<Option<Id>, String> {
match id {
Some(id) => Ok(self.translate(id)?.into_iter().next()),
None => Ok(None),
}
}
}
impl<C> std::ops::Deref for BuildCtx<'_, C> {

View File

@@ -54,15 +54,37 @@ impl Captures {
self.captures.entry(key).or_default().push(id);
}
/// Apply a fallible function to every captured id, replacing each id
/// with the results. A function returning an empty vector removes
/// the capture; returning multiple ids splices them into the
/// capture's value list (suitable for `*`/`+` captures). Captures
/// whose name appears in `skip` are left untouched. Stops and
/// returns the error on the first failure.
///
/// Used by the `rule!` macro's auto-translate prefix to translate
/// every capture except those marked `@@name` (raw).
pub fn map_captures(&mut self, kind: &str, f: &mut impl FnMut(Id) -> Id) {
if let Some(ids) = self.captures.get_mut(kind) {
for id in ids {
*id = f(*id);
}
}
}
/// Apply a fallible function to every captured id (across all keys),
/// replacing each id with the results. A function returning an empty
/// vector removes the capture; returning multiple ids splices them
/// into the capture's value list (suitable for `*`/`+` captures).
/// Stops and returns the error on the first failure.
pub fn try_map_all_captures<E>(
&mut self,
mut f: impl FnMut(Id) -> Result<Vec<Id>, E>,
) -> Result<(), E> {
for ids in self.captures.values_mut() {
let mut new_ids = Vec::with_capacity(ids.len());
for &id in ids.iter() {
new_ids.extend(f(id)?);
}
*ids = new_ids;
}
Ok(())
}
/// Like [`try_map_all_captures`] but leaves captures whose name appears
/// in `skip` untouched. Used by the `rule!` macro to support `@@name`
/// (raw) captures alongside the default auto-translated `@name`
/// captures.
pub fn try_map_captures_except<E>(
&mut self,
skip: &[&str],
@@ -80,6 +102,12 @@ impl Captures {
}
Ok(())
}
pub fn map_captures_to(&mut self, from: &str, to: &'static str, f: &mut impl FnMut(Id) -> Id) {
if let Some(from_ids) = self.captures.get(from) {
let new_values = from_ids.iter().copied().map(f).collect();
self.captures.insert(to, new_values);
}
}
pub fn merge(&mut self, other: &Captures) {
for (key, ids) in &other.captures {

View File

@@ -0,0 +1,8 @@
pub trait Cursor<'a, T, N, F> {
fn node(&self) -> &'a N;
fn field_id(&self) -> Option<F>;
fn field_name(&self) -> Option<&'static str>;
fn goto_first_child(&mut self) -> bool;
fn goto_next_sibling(&mut self) -> bool;
fn goto_parent(&mut self) -> bool;
}

View File

@@ -1,6 +1,6 @@
use std::fmt::Write;
use crate::{schema::Schema, Ast, Id, Node, NodeContent, CHILD_FIELD};
use crate::{schema::Schema, Ast, Node, NodeContent, CHILD_FIELD};
/// Options for controlling AST dump output.
pub struct DumpOptions {
@@ -34,11 +34,16 @@ impl Default for DumpOptions {
/// method:
/// identifier "foo"
/// ```
pub fn dump_ast(ast: &Ast, root: Id, source: &str) -> String {
pub fn dump_ast(ast: &Ast, root: usize, source: &str) -> String {
dump_ast_with_options(ast, root, source, &DumpOptions::default())
}
pub fn dump_ast_with_options(ast: &Ast, root: Id, source: &str, options: &DumpOptions) -> String {
pub fn dump_ast_with_options(
ast: &Ast,
root: usize,
source: &str,
options: &DumpOptions,
) -> String {
let mut out = String::new();
dump_node(ast, root, source, options, 0, None, &mut out);
out
@@ -48,7 +53,7 @@ pub fn dump_ast_with_options(ast: &Ast, root: Id, source: &str, options: &DumpOp
///
/// Any node that does not match the expected type set for its parent field is
/// rendered with a trailing `" <-- ERROR: ..."` annotation on the same line.
pub fn dump_ast_with_type_errors(ast: &Ast, root: Id, source: &str, schema: &Schema) -> String {
pub fn dump_ast_with_type_errors(ast: &Ast, root: usize, source: &str, schema: &Schema) -> String {
dump_ast_with_type_errors_and_options(ast, root, source, schema, &DumpOptions::default())
}
@@ -58,7 +63,7 @@ pub fn dump_ast_with_type_errors(ast: &Ast, root: Id, source: &str, schema: &Sch
/// rendered with a trailing `" <-- ERROR: ..."` annotation on the same line.
pub fn dump_ast_with_type_errors_and_options(
ast: &Ast,
root: Id,
root: usize,
source: &str,
schema: &Schema,
options: &DumpOptions,
@@ -171,7 +176,7 @@ fn expected_for_field<'a>(
fn dump_node(
ast: &Ast,
id: Id,
id: usize,
source: &str,
options: &DumpOptions,
indent: usize,
@@ -310,7 +315,7 @@ fn dump_node(
/// Dump a leaf node inline (no newline prefix, caller provides context).
fn dump_node_inline(
ast: &Ast,
id: Id,
id: usize,
source: &str,
options: &DumpOptions,
type_check: Option<(

View File

@@ -7,6 +7,7 @@ use serde_json::{json, Value};
pub mod build;
pub mod captures;
pub mod cursor;
pub mod dump;
pub mod node_types_yaml;
pub mod query;
@@ -18,61 +19,38 @@ mod visitor;
pub use yeast_macros::{query, rule, tree, trees};
use captures::Captures;
pub use cursor::Cursor;
use query::QueryNode;
/// Node id: an index into the [`Ast`] arena. A newtype around `usize`
/// rather than a bare alias so that it can carry its own
/// [`YeastDisplay`] / [`YeastSourceRange`] / [`IntoFieldIds`] impls
/// without colliding with the impls for plain integers.
///
/// Use `id.0` (or `id.into()`) to obtain the raw arena index.
#[repr(transparent)]
#[derive(Copy, Clone, Eq, PartialEq, Ord, PartialOrd, Debug, Hash, Serialize)]
pub struct Id(pub usize);
impl From<usize> for Id {
fn from(value: usize) -> Self {
Id(value)
}
}
impl From<Id> for usize {
fn from(value: Id) -> Self {
value.0
}
}
/// Node ids are indexes into the arena
pub type Id = usize;
/// Field and Kind ids are provided by tree-sitter
type FieldId = u16;
type KindId = u16;
/// Trait for values that can be appended to a field's id list inside a
/// `tree!`/`trees!`/`rule!` template (in `{expr}` placeholders).
///
/// `Id` pushes a single id; the blanket impl for
/// `IntoIterator<Item: Into<Id>>` handles `Vec<Id>`, `Option<Id>`,
/// arbitrary iterators yielding `Id`, etc.
///
/// This lets `{expr}` interpolate any of these shapes without a
/// dedicated splice syntax — the macro emits the same trait-dispatched
/// call regardless of the value's type.
pub trait IntoFieldIds {
fn extend_into(self, out: &mut Vec<Id>);
}
/// A typed reference to a node in an [`Ast`] arena. Wraps an [`Id`] but
/// deliberately does not implement [`std::fmt::Display`]: rendering a node
/// requires the [`Ast`] it lives in (to resolve [`NodeContent::Range`] back
/// to source text). Use [`YeastDisplay::yeast_to_string`] to format it.
#[derive(Copy, Clone, Eq, PartialEq, Debug, Hash)]
pub struct NodeRef(pub Id);
impl IntoFieldIds for Id {
fn extend_into(self, out: &mut Vec<Id>) {
out.push(self);
impl NodeRef {
pub fn id(self) -> Id {
self.0
}
}
impl<I, T> IntoFieldIds for I
where
I: IntoIterator<Item = T>,
T: Into<Id>,
{
fn extend_into(self, out: &mut Vec<Id>) {
out.extend(self.into_iter().map(Into::into));
impl From<NodeRef> for Id {
fn from(value: NodeRef) -> Self {
value.0
}
}
impl From<Id> for NodeRef {
fn from(value: Id) -> Self {
NodeRef(value)
}
}
@@ -89,21 +67,21 @@ pub trait YeastDisplay {
/// Optional source range for values used in `#{expr}` interpolations.
///
/// By default this returns `None`, so synthesized leaves inherit the matched
/// rule's source range. `Id` returns the referenced node's range, letting
/// rule's source range. `NodeRef` returns the referenced node's range, letting
/// `(kind #{capture})` carry the captured node's location.
pub trait YeastSourceRange {
fn yeast_source_range(&self, ast: &Ast) -> Option<tree_sitter::Range>;
}
impl YeastDisplay for Id {
impl YeastDisplay for NodeRef {
fn yeast_to_string(&self, ast: &Ast) -> String {
ast.source_text(*self)
ast.source_text(self.0)
}
}
impl YeastSourceRange for Id {
impl YeastSourceRange for NodeRef {
fn yeast_source_range(&self, ast: &Ast) -> Option<tree_sitter::Range> {
ast.get_node(*self).and_then(|n| match &n.content {
ast.get_node(self.0).and_then(|n| match &n.content {
NodeContent::Range(r) => Some(r.clone()),
_ => n.source_range,
})
@@ -172,36 +150,6 @@ impl<'a> AstCursor<'a> {
self.node_id
}
pub fn node(&self) -> &'a Node {
&self.ast.nodes[self.node_id.0]
}
pub fn field_id(&self) -> Option<FieldId> {
let (_, children) = self.parents.last()?;
children.current_field()
}
pub fn field_name(&self) -> Option<&'static str> {
if self.field_id() == Some(CHILD_FIELD) {
None
} else {
self.field_id()
.and_then(|id| self.ast.field_name_for_id(id))
}
}
pub fn goto_first_child(&mut self) -> bool {
self.goto_first_child_opt().is_some()
}
pub fn goto_next_sibling(&mut self) -> bool {
self.goto_next_sibling_opt().is_some()
}
pub fn goto_parent(&mut self) -> bool {
self.goto_parent_opt().is_some()
}
fn goto_next_sibling_opt(&mut self) -> Option<()> {
self.node_id = self.parents.last_mut()?.1.next()?;
Some(())
@@ -222,6 +170,37 @@ impl<'a> AstCursor<'a> {
Some(())
}
}
impl<'a> Cursor<'a, Ast, Node, FieldId> for AstCursor<'a> {
fn node(&self) -> &'a Node {
&self.ast.nodes[self.node_id]
}
fn field_id(&self) -> Option<FieldId> {
let (_, children) = self.parents.last()?;
children.current_field()
}
fn field_name(&self) -> Option<&'static str> {
if self.field_id() == Some(CHILD_FIELD) {
None
} else {
self.field_id()
.and_then(|id| self.ast.field_name_for_id(id))
}
}
fn goto_first_child(&mut self) -> bool {
self.goto_first_child_opt().is_some()
}
fn goto_next_sibling(&mut self) -> bool {
self.goto_next_sibling_opt().is_some()
}
fn goto_parent(&mut self) -> bool {
self.goto_parent_opt().is_some()
}
}
/// An iterator over the child Ids of a node.
#[derive(Debug)]
@@ -368,16 +347,16 @@ impl Ast {
///
/// This reflects the effective AST after desugaring and excludes orphaned
/// arena nodes left behind by rewrite operations.
pub fn reachable_node_ids(&self) -> Vec<Id> {
pub fn reachable_node_ids(&self) -> Vec<usize> {
let mut reachable = Vec::new();
let mut stack = vec![self.root];
let mut seen = vec![false; self.nodes.len()];
while let Some(id) = stack.pop() {
if id.0 >= self.nodes.len() || seen[id.0] {
if id >= self.nodes.len() || seen[id] {
continue;
}
seen[id.0] = true;
seen[id] = true;
reachable.push(id);
if let Some(node) = self.get_node(id) {
@@ -401,11 +380,11 @@ impl Ast {
}
pub fn get_node(&self, id: Id) -> Option<&Node> {
self.nodes.get(id.0)
self.nodes.get(id)
}
pub fn print(&self, source: &str, root_id: Id) -> Value {
let root = &self.nodes()[root_id.0];
let root = &self.nodes()[root_id];
self.print_node(root, source)
}
@@ -448,7 +427,7 @@ impl Ast {
is_named,
source_range,
});
Id(id)
id
}
fn union_source_range_of_children(
@@ -515,6 +494,15 @@ impl Ast {
self.create_named_token_with_range(kind, content, None)
}
/// Prepend a child id to the given field of the given node.
pub fn prepend_field_child(&mut self, node_id: Id, field_id: FieldId, value_id: Id) {
let node = self
.nodes
.get_mut(node_id)
.expect("prepend_field_child: invalid node id");
node.fields.entry(field_id).or_default().insert(0, value_id);
}
pub fn create_named_token_with_range(
&mut self,
kind: &'static str,
@@ -536,7 +524,7 @@ impl Ast {
fields: BTreeMap::new(),
content: NodeContent::DynamicString(content),
});
Id(id)
id
}
pub fn field_name_for_id(&self, id: FieldId) -> Option<&'static str> {
@@ -620,6 +608,10 @@ pub struct Node {
}
impl Node {
pub fn kind(&self) -> &'static str {
self.kind_name
}
pub fn kind_name(&self) -> &'static str {
self.kind_name
}
@@ -964,7 +956,7 @@ fn apply_repeating_rules_inner<C: Clone>(
));
}
let node_kind = ast.get_node(id).map(|n| n.kind_name()).unwrap_or("");
let node_kind = ast.get_node(id).map(|n| n.kind()).unwrap_or("");
for rule in index.rules_for_kind(node_kind) {
let rule_ptr = *rule as *const Rule<C>;
if Some(rule_ptr) == skip_rule {
@@ -1016,7 +1008,7 @@ fn apply_repeating_rules_inner<C: Clone>(
//
// Child traversal does not increment rewrite depth and starts fresh
// (no rule is skipped on child subtrees).
let mut fields = std::mem::take(&mut ast.nodes[id.0].fields);
let mut fields = std::mem::take(&mut ast.nodes[id].fields);
for children in fields.values_mut() {
let mut new_children: Option<Vec<Id>> = None;
for (i, &child_id) in children.iter().enumerate() {
@@ -1049,7 +1041,7 @@ fn apply_repeating_rules_inner<C: Clone>(
*children = new;
}
}
ast.nodes[id.0].fields = fields;
ast.nodes[id].fields = fields;
Ok(vec![id])
}
@@ -1083,7 +1075,7 @@ fn apply_one_shot_rules_inner<C: Clone>(
));
}
let node_kind = ast.get_node(id).map(|n| n.kind_name()).unwrap_or("");
let node_kind = ast.get_node(id).map(|n| n.kind()).unwrap_or("");
for rule in index.rules_for_kind(node_kind) {
if let Some(captures) = rule.try_match(ast, id)? {

View File

@@ -49,7 +49,7 @@ impl Visitor {
pub fn build_with_schema(self, schema: crate::schema::Schema) -> Ast {
Ast {
root: Id(0),
root: 0,
schema,
nodes: self.nodes.into_iter().map(|n| n.inner).collect(),
source: Vec::new(),
@@ -72,7 +72,7 @@ impl Visitor {
},
parent: self.current,
});
Id(id)
id
}
fn enter_node(&mut self, node: tree_sitter::Node<'_>) -> bool {
@@ -83,10 +83,10 @@ impl Visitor {
fn leave_node(&mut self, field_name: Option<&'static str>, _node: tree_sitter::Node<'_>) {
let node_id = self.current.unwrap();
let node_parent = self.nodes[node_id.0].parent;
let node_parent = self.nodes[node_id].parent;
if let Some(parent_id) = node_parent {
let parent = self.nodes.get_mut(parent_id.0).unwrap();
let parent = self.nodes.get_mut(parent_id).unwrap();
if let Some(field) = field_name {
let field_id = self.language.field_id_for_name(field).unwrap().get();
parent

View File

@@ -300,7 +300,7 @@ fn test_query_skips_extras_in_positional_match() {
let mut cursor = AstCursor::new(&ast);
cursor.goto_first_child();
let array_id = cursor.node_id();
assert_eq!(ast.get_node(array_id).unwrap().kind_name(), "array");
assert_eq!(ast.get_node(array_id).unwrap().kind(), "array");
// Two positional wildcards should bind to the two integers, skipping
// the comment that sits between them.
@@ -309,15 +309,11 @@ fn test_query_skips_extras_in_positional_match() {
let matched = query.do_match(&ast, array_id, &mut captures).unwrap();
assert!(matched);
assert_eq!(
ast.get_node(captures.get_var("a").unwrap())
.unwrap()
.kind_name(),
ast.get_node(captures.get_var("a").unwrap()).unwrap().kind(),
"integer"
);
assert_eq!(
ast.get_node(captures.get_var("b").unwrap())
.unwrap()
.kind_name(),
ast.get_node(captures.get_var("b").unwrap()).unwrap().kind(),
"integer"
);
}
@@ -395,7 +391,7 @@ fn test_capture_unnamed_node_parenthesized() {
assert!(matched);
let op_id = captures.get_var("op").unwrap();
let op_node = ast.get_node(op_id).unwrap();
assert_eq!(op_node.kind_name(), "=");
assert_eq!(op_node.kind(), "=");
assert!(!op_node.is_named());
}
@@ -418,7 +414,7 @@ fn test_capture_bare_underscore_repeated() {
let all = captures.get_all("all");
assert_eq!(all.len(), 1);
assert_eq!(ast.get_node(all[0]).unwrap().kind_name(), "=");
assert_eq!(ast.get_node(all[0]).unwrap().kind(), "=");
assert!(!ast.get_node(all[0]).unwrap().is_named());
}
@@ -445,7 +441,7 @@ fn test_capture_unnamed_node_bare_literal() {
assert!(matched);
let op_id = captures.get_var("op").unwrap();
let op_node = ast.get_node(op_id).unwrap();
assert_eq!(op_node.kind_name(), "=");
assert_eq!(op_node.kind(), "=");
assert!(!op_node.is_named());
}
@@ -483,7 +479,7 @@ fn test_bare_underscore_matches_unnamed() {
.unwrap();
assert!(matched, "_ should match the unnamed `=`");
let any_node = ast.get_node(captures.get_var("any").unwrap()).unwrap();
assert_eq!(any_node.kind_name(), "=");
assert_eq!(any_node.kind(), "=");
assert!(!any_node.is_named());
}
@@ -510,7 +506,7 @@ fn test_bare_forms_in_field_position() {
assert_eq!(
ast.get_node(captures.get_var("lhs").unwrap())
.unwrap()
.kind_name(),
.kind(),
"identifier"
);
@@ -520,7 +516,7 @@ fn test_bare_forms_in_field_position() {
let matched = query.do_match(&ast, assignment_id, &mut captures).unwrap();
assert!(matched);
let op = ast.get_node(captures.get_var("op").unwrap()).unwrap();
assert_eq!(op.kind_name(), "=");
assert_eq!(op.kind(), "=");
assert!(!op.is_named());
}
@@ -539,7 +535,7 @@ fn test_forward_scan_finds_unnamed_token_late() {
let mut cursor = AstCursor::new(&ast);
cursor.goto_first_child(); // for
cursor.goto_first_child(); // do (the body)
while cursor.node().kind_name() != "do" || !cursor.node().is_named() {
while cursor.node().kind() != "do" || !cursor.node().is_named() {
assert!(cursor.goto_next_sibling(), "expected to find named `do`");
}
let do_id = cursor.node_id();
@@ -549,7 +545,7 @@ fn test_forward_scan_finds_unnamed_token_late() {
let matched = query.do_match(&ast, do_id, &mut captures).unwrap();
assert!(matched, "forward-scan should find the `end` keyword");
let kw = ast.get_node(captures.get_var("kw").unwrap()).unwrap();
assert_eq!(kw.kind_name(), "end");
assert_eq!(kw.kind(), "end");
assert!(!kw.is_named());
}
@@ -565,7 +561,7 @@ fn test_forward_scan_preserves_order() {
let mut cursor = AstCursor::new(&ast);
cursor.goto_first_child();
cursor.goto_first_child();
while cursor.node().kind_name() != "do" || !cursor.node().is_named() {
while cursor.node().kind() != "do" || !cursor.node().is_named() {
assert!(cursor.goto_next_sibling(), "expected to find named `do`");
}
let do_id = cursor.node_id();
@@ -639,7 +635,7 @@ fn ruby_rules() -> Vec<Rule> {
left: (identifier $tmp)
right: {right}
)
{left.iter().enumerate().map(|(i, &lhs)|
{..left.iter().enumerate().map(|(i, &lhs)|
yeast::tree!(
(assignment
left: {lhs}
@@ -671,7 +667,7 @@ fn ruby_rules() -> Vec<Rule> {
left: {pat}
right: (identifier $tmp)
)
stmt: {body}
stmt: {..body}
)
)
)
@@ -907,7 +903,7 @@ fn one_shot_xeq1_rules() -> Vec<Rule> {
yeast::rule!(
(program (_)* @stmts)
=>
(program stmt: {stmts})
(program stmt: {..stmts})
),
yeast::rule!(
(assignment left: (_) @left right: (_) @right)
@@ -983,7 +979,7 @@ fn test_one_shot_recurses_into_returned_capture() {
yeast::rule!(
(program (_)* @stmts)
=>
(program stmt: {stmts})
(program stmt: {..stmts})
),
// Returns the captured `left` verbatim, discarding `right`.
yeast::rule!(
@@ -1025,7 +1021,7 @@ fn test_one_shot_does_not_recurse_into_wrapper_output() {
yeast::rule!(
(program (_)* @stmts)
=>
(program stmt: {stmts})
(program stmt: {..stmts})
),
// Wraps `left` in nested `first_node`/`second_node` output kinds.
// Neither wrapper kind has a matching rule, so a buggy implementation
@@ -1063,7 +1059,7 @@ fn test_one_shot_does_not_recurse_into_wrapper_output() {
}
/// Verify that `@@name` capture markers skip the auto-translate prefix:
/// the body sees the *raw* (input-schema) `Id` and can read its
/// the body sees the *raw* (input-schema) NodeRef and can read its
/// source text or call `ctx.translate(...)` explicitly. Compare with
/// the bare `@name` form, where the auto-translate prefix runs the
/// same translation up front and the body sees the post-translate id.
@@ -1076,7 +1072,7 @@ fn test_raw_capture_marker() {
yeast::rule!(
(program (_)* @stmts)
=>
(program stmt: {stmts})
(program stmt: {..stmts})
),
// `@@raw_lhs` is untranslated: the body reads its source text
// ("x") and embeds it directly as the identifier content. `@rhs`
@@ -1085,7 +1081,7 @@ fn test_raw_capture_marker() {
(assignment left: (_) @@raw_lhs right: (_) @rhs)
=>
{
let text = ctx.ast.source_text(raw_lhs);
let text = ctx.ast.source_text(raw_lhs.into());
tree!((call
method: (identifier #{text.as_str()})
receiver: {rhs}))
@@ -1120,7 +1116,7 @@ fn test_raw_capture_marker() {
}
/// Companion to `test_raw_capture_marker`: confirms that calling
/// `ctx.translate(raw)` on a `@@`-captured `Id` from the rule body
/// `ctx.translate(raw)` on a `@@`-captured NodeRef from the rule body
/// produces the correctly-translated output-schema node. With `@`, the
/// translation has already happened, so `ctx.translate(...)` inside the
/// body would attempt to re-translate an output node (which has no
@@ -1134,7 +1130,7 @@ fn test_raw_capture_marker_explicit_translate() {
yeast::rule!(
(program (_)* @stmts)
=>
(program stmt: {stmts})
(program stmt: {..stmts})
),
yeast::rule!(
(assignment left: (_) @@raw_lhs right: (_) @rhs)
@@ -1142,7 +1138,7 @@ fn test_raw_capture_marker_explicit_translate() {
{
let translated_lhs = ctx.translate(raw_lhs)?;
tree!((call
method: {translated_lhs}
method: {..translated_lhs}
receiver: {rhs}))
}
),
@@ -1176,11 +1172,11 @@ fn test_cursor_navigation() {
let mut cursor = AstCursor::new(&ast);
// Start at root
assert_eq!(cursor.node().kind_name(), "program");
assert_eq!(cursor.node().kind(), "program");
// Go to first child (assignment)
assert!(cursor.goto_first_child());
assert_eq!(cursor.node().kind_name(), "assignment");
assert_eq!(cursor.node().kind(), "assignment");
// No sibling
assert!(!cursor.goto_next_sibling());
@@ -1191,10 +1187,10 @@ fn test_cursor_navigation() {
// Go back up
assert!(cursor.goto_parent());
assert_eq!(cursor.node().kind_name(), "assignment");
assert_eq!(cursor.node().kind(), "assignment");
assert!(cursor.goto_parent());
assert_eq!(cursor.node().kind_name(), "program");
assert_eq!(cursor.node().kind(), "program");
// Can't go further up
assert!(!cursor.goto_parent());
@@ -1239,8 +1235,10 @@ fn test_desugar_for_with_multiple_assignment() {
}
/// Regression test: `#{capture}` in a template must render the *source text*
/// of the captured node, not its arena `Id`. Captures are bound as `Id`,
/// whose `YeastDisplay` impl resolves to the captured node's source text.
/// of the captured node, not its arena `Id`. Previously, captures were bound
/// as `usize`, so `#{cap}` printed the integer id (e.g. `"3"`) via `Display`.
/// Captures are now bound as `NodeRef`, which has no `Display` impl and
/// resolves to the captured node's source text via `YeastDisplay`.
#[test]
fn test_hash_brace_renders_capture_source_text() {
let rule: Rule = rule!(
@@ -1268,7 +1266,7 @@ fn test_hash_brace_renders_capture_source_text() {
);
}
/// Regression test: non-`Id` values in `#{expr}` still render via their
/// Regression test: non-`NodeRef` values in `#{expr}` still render via their
/// `Display` impl (covered by `YeastDisplay`'s blanket impls for primitives).
#[test]
fn test_hash_brace_renders_integer_expression() {
@@ -1306,12 +1304,12 @@ fn test_hash_brace_uses_capture_location_for_leaf() {
let ast = run_and_ast("foo.bar()", vec![rule]);
let mut bar_ids: Vec<yeast::Id> = Vec::new();
let mut bar_ids: Vec<usize> = Vec::new();
for id in ast.reachable_node_ids() {
let Some(node) = ast.get_node(id) else {
continue;
};
if node.kind_name() == "identifier" && ast.source_text(id) == "bar" {
if node.kind() == "identifier" && ast.source_text(id) == "bar" {
bar_ids.push(id);
}
}

View File

@@ -15,26 +15,26 @@ struct SwiftContext {
/// (`computed_getter`/`computed_setter`/`computed_modify`/
/// `willset_clause`/`didset_clause`/`getter_specifier`/
/// `setter_specifier`).
property_name: Option<yeast::Id>,
property_name: Option<yeast::NodeRef>,
/// Translated type node for the property type. Set by the outer
/// `property_binding` rule (computed accessors variant) and
/// `protocol_property_declaration` when present; read by the
/// accessor inner rules.
property_type: Option<yeast::Id>,
property_type: Option<yeast::NodeRef>,
/// Default-value expression for the next translated `parameter`. Set
/// by the outer `function_parameter` rule; read by the `parameter`
/// rules.
default_value: Option<yeast::Id>,
default_value: Option<yeast::NodeRef>,
/// Translated outer modifiers (e.g. visibility, attributes) to
/// attach to each child of a flattening outer rule. Set by
/// `property_declaration`, `enum_entry`, and
/// `protocol_property_declaration`.
outer_modifiers: Vec<yeast::Id>,
outer_modifiers: Vec<yeast::NodeRef>,
/// The `let`/`var` binding modifier for a `property_declaration`.
/// Set by `property_declaration`; read by the inner declaration
/// rules (`property_binding` variants, accessor rules) so they
/// emit it as part of the output node's `modifier:` field.
binding_modifier: Option<yeast::Id>,
binding_modifier: Option<yeast::NodeRef>,
/// True when the current child of a flattening outer rule is not
/// the first one — its inner rule should emit a
/// `chained_declaration` modifier so the original grouping can be
@@ -45,10 +45,10 @@ struct SwiftContext {
/// Build a freshly-created `chained_declaration` modifier node if
/// `ctx.is_chained`, else `None`. Used by inner declaration rules to
/// emit the chained tag for non-first children of a flattening outer
/// rule. Returns `Option<Id>` so it splices via `{…}` to 0 or 1 ids.
fn chained_modifier(ctx: &mut yeast::build::BuildCtx<'_, SwiftContext>) -> Option<yeast::Id> {
/// rule. Returns `Option<NodeRef>` so it splices via `{..…}` to 0 or 1 ids.
fn chained_modifier(ctx: &mut yeast::build::BuildCtx<'_, SwiftContext>) -> Option<yeast::NodeRef> {
if ctx.is_chained {
Some(ctx.literal("modifier", "chained_declaration"))
Some(ctx.literal("modifier", "chained_declaration").into())
} else {
None
}
@@ -63,10 +63,10 @@ fn chained_modifier(ctx: &mut yeast::build::BuildCtx<'_, SwiftContext>) -> Optio
/// condition.
fn and_chain(
ctx: &mut yeast::build::BuildCtx<'_, SwiftContext>,
conds: Vec<yeast::Id>,
conds: Vec<yeast::NodeRef>,
) -> yeast::Id {
conds
.into_iter()
conds.into_iter()
.map(yeast::Id::from)
.reduce(|acc, elem| {
tree!((binary_expr operator: (infix_operator "&&") left: {acc} right: {elem}))
})
@@ -79,7 +79,7 @@ fn and_chain(
/// guarantees at least one part.
fn member_chain(
ctx: &mut yeast::build::BuildCtx<'_, SwiftContext>,
parts: Vec<yeast::Id>,
parts: Vec<yeast::NodeRef>,
) -> yeast::Id {
let mut iter = parts.into_iter();
let first = iter
@@ -100,7 +100,7 @@ fn translation_rules() -> Vec<Rule<SwiftContext>> {
(source_file statement: _* @children)
=>
(top_level
body: (block stmt: {children})
body: (block stmt: {..children})
)
),
// Declarations may be wrapped in local/global wrapper nodes.
@@ -144,12 +144,12 @@ fn translation_rules() -> Vec<Rule<SwiftContext>> {
rule!(
(operator_declaration "prefix" (referenceable_operator _ @op) (simple_identifier)? @prec)
=>
(operator_syntax_declaration name: (identifier #{op}) fixity: (fixity "prefix") precedence: {prec})
(operator_syntax_declaration name: (identifier #{op}) fixity: (fixity "prefix") precedence: {..prec})
),
rule!(
(operator_declaration "postfix" (referenceable_operator _ @op) (simple_identifier)? @prec)
=>
(operator_syntax_declaration name: (identifier #{op}) fixity: (fixity "postfix") precedence: {prec})
(operator_syntax_declaration name: (identifier #{op}) fixity: (fixity "postfix") precedence: {..prec})
),
rule!(
(operator_declaration "infix" (referenceable_operator _ @op) (simple_identifier)? @prec)
@@ -157,7 +157,7 @@ fn translation_rules() -> Vec<Rule<SwiftContext>> {
(operator_syntax_declaration
name: (identifier #{op})
fixity: (fixity "infix")
precedence: {prec})
precedence: {..prec})
),
rule!((bitwise_operation lhs: @l op: @op rhs: @r) => (binary_expr left: {l} operator: (infix_operator #{op}) right: {r})),
rule!((nil_coalescing_expression value: @l if_nil: @r) => (binary_expr left: {l} operator: (infix_operator "??") right: {r})),
@@ -170,9 +170,9 @@ fn translation_rules() -> Vec<Rule<SwiftContext>> {
rule!((postfix_expression operation: @op target: @operand) => (unary_expr operator: (postfix_operator #{op}) operand: {operand})),
// TODO: Parenthesised single-value tuple is a grouping expression and should pass through.
// Multi-value tuples become tuple_expr.
rule!((tuple_expression value: _* @v) => (tuple_expr element: {v})),
rule!((tuple_expression value: _* @v) => (tuple_expr element: {..v})),
// Blocks contain statement* directly.
rule!((block statement: _+ @stmts) => (block stmt: {stmts})),
rule!((block statement: _+ @stmts) => (block stmt: {..stmts})),
rule!((block) => (block)),
// ---- Variables ----
// property_binding rules — these produce variable_declaration and/or accessor_declaration
@@ -198,8 +198,8 @@ fn translation_rules() -> Vec<Rule<SwiftContext>> {
type: _? @ty
computed_value: (computed_property accessor: _+ @@accessors))
=>
{{
ctx.property_name = Some(tree!((identifier #{pattern})));
{..{
ctx.property_name = Some(tree!((identifier #{pattern})).into());
ctx.property_type = ty;
let mut result = Vec::new();
@@ -223,13 +223,13 @@ fn translation_rules() -> Vec<Rule<SwiftContext>> {
computed_value: (computed_property statement: _* @body))
=>
(accessor_declaration
modifier: {ctx.binding_modifier}
modifier: {ctx.outer_modifiers.clone()}
modifier: {chained_modifier(&mut ctx)}
modifier: {..ctx.binding_modifier}
modifier: {..ctx.outer_modifiers.clone()}
modifier: {..chained_modifier(&mut ctx)}
name: (identifier #{name})
type: {ty}
type: {..ty}
accessor_kind: (accessor_kind "get")
body: (block stmt: {body}))
body: (block stmt: {..body}))
),
// Stored property with willSet/didSet observers (initializer
// optional) → a `variable_declaration` followed by one
@@ -249,19 +249,19 @@ fn translation_rules() -> Vec<Rule<SwiftContext>> {
value: _? @val
observers: (willset_didset_block willset: _? @@ws didset: _? @@ds))
=>
{{
{..{
let var_decl = tree!(
(variable_declaration
modifier: {ctx.binding_modifier}
modifier: {ctx.outer_modifiers.clone()}
modifier: {chained_modifier(&mut ctx)}
modifier: {..ctx.binding_modifier}
modifier: {..ctx.outer_modifiers.clone()}
modifier: {..chained_modifier(&mut ctx)}
pattern: (name_pattern identifier: (identifier #{name}))
type: {ty}
value: {val})
type: {..ty}
value: {..val})
);
// Publish the property name for the observer rules.
ctx.property_name = Some(tree!((identifier #{name})));
ctx.property_name = Some(tree!((identifier #{name})).into());
// Observers are subsequent outputs of this flattening
// rule, so they always get `chained_declaration`.
ctx.is_chained = true;
@@ -282,12 +282,12 @@ fn translation_rules() -> Vec<Rule<SwiftContext>> {
value: _? @val)
=>
(variable_declaration
modifier: {ctx.binding_modifier}
modifier: {ctx.outer_modifiers.clone()}
modifier: {chained_modifier(&mut ctx)}
modifier: {..ctx.binding_modifier}
modifier: {..ctx.outer_modifiers.clone()}
modifier: {..chained_modifier(&mut ctx)}
pattern: {pattern}
type: {ty}
value: {val})
type: {..ty}
value: {..val})
),
// property_declaration: flatten declarators (each may translate
// to multiple nodes — variable_declaration and/or
@@ -305,9 +305,9 @@ fn translation_rules() -> Vec<Rule<SwiftContext>> {
declarator: _* @@decls
(modifiers)* @mods)
=>
{{
let binding_text = ctx.ast.source_text(binding_kind);
ctx.binding_modifier = Some(ctx.literal("modifier", &binding_text));
{..{
let binding_text = ctx.ast.source_text(binding_kind.into());
ctx.binding_modifier = Some(ctx.literal("modifier", &binding_text).into());
ctx.outer_modifiers = mods;
let mut result = Vec::new();
@@ -342,19 +342,19 @@ fn translation_rules() -> Vec<Rule<SwiftContext>> {
data_contents: (enum_type_parameters parameter: _* @params))
=>
(class_like_declaration
modifier: {ctx.outer_modifiers.clone()}
modifier: {chained_modifier(&mut ctx)}
modifier: {..ctx.outer_modifiers.clone()}
modifier: {..chained_modifier(&mut ctx)}
modifier: (modifier "enum_case")
name: (identifier #{name})
member: (constructor_declaration parameter: {params} body: (block)))
member: (constructor_declaration parameter: {..params} body: (block)))
),
// enum_case_entry with explicit raw value → variable_declaration with that value.
rule!(
(enum_case_entry name: @name raw_value: @val)
=>
(variable_declaration
modifier: {ctx.outer_modifiers.clone()}
modifier: {chained_modifier(&mut ctx)}
modifier: {..ctx.outer_modifiers.clone()}
modifier: {..chained_modifier(&mut ctx)}
modifier: (modifier "enum_case")
pattern: (name_pattern identifier: (identifier #{name}))
value: {val})
@@ -364,8 +364,8 @@ fn translation_rules() -> Vec<Rule<SwiftContext>> {
(enum_case_entry name: @name)
=>
(variable_declaration
modifier: {ctx.outer_modifiers.clone()}
modifier: {chained_modifier(&mut ctx)}
modifier: {..ctx.outer_modifiers.clone()}
modifier: {..chained_modifier(&mut ctx)}
modifier: (modifier "enum_case")
pattern: (name_pattern identifier: (identifier #{name})))
),
@@ -376,7 +376,7 @@ fn translation_rules() -> Vec<Rule<SwiftContext>> {
rule!(
(enum_entry case: _+ @@cases (modifiers)* @mods)
=>
{{
{..{
ctx.outer_modifiers = mods;
let mut result = Vec::new();
@@ -418,7 +418,7 @@ fn translation_rules() -> Vec<Rule<SwiftContext>> {
=>
(constructor_pattern
constructor: (member_access_expr base: {typ} member: (identifier #{name}))
element: {items})
element: {..items})
),
// case .foo(x,y) pattern
rule!(
@@ -426,10 +426,10 @@ fn translation_rules() -> Vec<Rule<SwiftContext>> {
=>
(constructor_pattern
constructor: (member_access_expr base: (inferred_type_expr #{dot}) member: (identifier #{name}))
element: {items})
element: {..items})
),
// Tuple pattern and its (optionally named) items
rule!((pattern kind: (tuple_pattern item: _* @elems)) => (tuple_pattern element: {elems})),
rule!((pattern kind: (tuple_pattern item: _* @elems)) => (tuple_pattern element: {..elems})),
rule!((tuple_pattern_item name: @key pattern: @pat) => (pattern_element key: (identifier #{key}) pattern: {pat})),
rule!((tuple_pattern_item pattern: @pat) => (pattern_element pattern: {pat})),
// Type casting pattern (TODO)
@@ -452,9 +452,9 @@ fn translation_rules() -> Vec<Rule<SwiftContext>> {
=>
(function_declaration
name: (identifier #{name})
parameter: {params}
return_type: {ret}
body: (block stmt: {body_stmts}))
parameter: {..params}
return_type: {..ret}
body: (block stmt: {..body_stmts}))
),
// Parameters are wrapped in function_parameter, which also carries
// optional default values. Publishes the default value into `ctx`
@@ -463,7 +463,7 @@ fn translation_rules() -> Vec<Rule<SwiftContext>> {
rule!(
(function_parameter parameter: @@p default_value: _? @def)
=>
{{
{..{
ctx.default_value = def;
ctx.translate(p)?
}}
@@ -475,7 +475,7 @@ fn translation_rules() -> Vec<Rule<SwiftContext>> {
(parameter
external_name: (identifier #{ext})
pattern: (name_pattern identifier: (identifier #{name}))
default: {ctx.default_value})
default: {..ctx.default_value})
),
rule!(
(parameter external_name: @ext name: @name type: @ty)
@@ -484,7 +484,7 @@ fn translation_rules() -> Vec<Rule<SwiftContext>> {
external_name: (identifier #{ext})
pattern: (name_pattern identifier: (identifier #{name}))
type: {ty}
default: {ctx.default_value})
default: {..ctx.default_value})
),
// Parameter with just name and type (no external name)
rule!(
@@ -492,7 +492,7 @@ fn translation_rules() -> Vec<Rule<SwiftContext>> {
=>
(parameter
pattern: (name_pattern identifier: (identifier #{name}))
default: {ctx.default_value})
default: {..ctx.default_value})
),
rule!(
(parameter name: @name type: @ty)
@@ -500,7 +500,7 @@ fn translation_rules() -> Vec<Rule<SwiftContext>> {
(parameter
pattern: (name_pattern identifier: (identifier #{name}))
type: {ty}
default: {ctx.default_value})
default: {..ctx.default_value})
),
// Reference to a function, f(x:y:z:). This is parsed as a call with a single argument with multiple reference_specifier labels.
// We don't want downstream QL to try to handle this as a call_expr with a weird argument, so explicitly mark it as unsupported for now.
@@ -514,7 +514,7 @@ fn translation_rules() -> Vec<Rule<SwiftContext>> {
rule!(
(call_expression function: @func suffix: (call_suffix arguments: (value_arguments argument: (value_argument)* @args)))
=>
(call_expr callee: {func} argument: {args})
(call_expr callee: {func} argument: {..args})
),
// Value argument with label (value: _ matches both named nodes and anonymous tokens like nil)
rule!(
@@ -537,7 +537,7 @@ fn translation_rules() -> Vec<Rule<SwiftContext>> {
// Return / break / continue, one rule per keyword.
// The anonymous "return"/"break"/"continue" keywords are matched as
// string literals.
rule!((control_transfer_statement kind: "return" result: _? @val) => (return_expr value: {val})),
rule!((control_transfer_statement kind: "return" result: _? @val) => (return_expr value: {..val})),
rule!((control_transfer_statement kind: "break" result: @lbl) => (break_expr label: (identifier #{lbl}))),
rule!((control_transfer_statement kind: "break") => (break_expr)),
rule!((control_transfer_statement kind: "continue" result: @lbl) => (continue_expr label: (identifier #{lbl}))),
@@ -556,20 +556,20 @@ fn translation_rules() -> Vec<Rule<SwiftContext>> {
statement: _* @body)
=>
(function_expr
modifier: {attrs}
capture_declaration: {captures}
parameter: {params}
return_type: {ret}
body: (block stmt: {body}))
modifier: {..attrs}
capture_declaration: {..captures}
parameter: {..params}
return_type: {..ret}
body: (block stmt: {..body}))
),
// capture_list_item with ownership modifier (e.g. [weak self], [unowned x])
rule!(
(capture_list_item ownership: _? @ownership name: @name value: _? @val)
=>
(variable_declaration
modifier: {ownership}
modifier: {..ownership}
pattern: (name_pattern identifier: (identifier #{name}))
value: {val})
value: {..val})
),
// Lambda parameter with type and optional external name
rule!(
@@ -615,7 +615,7 @@ fn translation_rules() -> Vec<Rule<SwiftContext>> {
(if_expr
condition: {and_chain(&mut ctx, cond)}
then: {then_body}
else: {else_stmts})
else: {..else_stmts})
),
// Guard statement
rule!(
@@ -623,7 +623,7 @@ fn translation_rules() -> Vec<Rule<SwiftContext>> {
=>
(guard_if_stmt
condition: {and_chain(&mut ctx, cond)}
else: (block stmt: {else_stmts}))
else: (block stmt: {..else_stmts}))
),
// Ternary expression → if_expr
rule!(
@@ -635,7 +635,7 @@ fn translation_rules() -> Vec<Rule<SwiftContext>> {
rule!(
(switch_statement expr: @val entry: (switch_entry)* @cases)
=>
(switch_expr value: {val} case: {cases})
(switch_expr value: {val} case: {..cases})
),
// Switch entry with multiple patterns and body
rule!(
@@ -644,19 +644,19 @@ fn translation_rules() -> Vec<Rule<SwiftContext>> {
pattern: (switch_pattern pattern: @rest)+
statement: _* @body)
=>
(switch_case pattern: (or_pattern pattern: {first} pattern: {rest}) body: (block stmt: {body}))
(switch_case pattern: (or_pattern pattern: {first} pattern: {..rest}) body: (block stmt: {..body}))
),
// Switch entry with exactly one pattern and body
rule!(
(switch_entry pattern: (switch_pattern pattern: @pat) statement: _* @body)
=>
(switch_case pattern: {pat} body: (block stmt: {body}))
(switch_case pattern: {pat} body: (block stmt: {..body}))
),
// Switch entry: default case (no patterns)
rule!(
(switch_entry default: (default_keyword) statement: _* @body)
=>
(switch_case body: (block stmt: {body}))
(switch_case body: (block stmt: {..body}))
),
// if case PATTERN = expr — preserve the pattern directly (no Optional wrapping)
rule!(
@@ -702,8 +702,8 @@ fn translation_rules() -> Vec<Rule<SwiftContext>> {
(for_each_stmt
pattern: {pat}
iterable: {iter}
guard: {guard}
body: (block stmt: {body}))
guard: {..guard}
body: (block stmt: {..body}))
),
// While loop
rule!(
@@ -711,7 +711,7 @@ fn translation_rules() -> Vec<Rule<SwiftContext>> {
=>
(while_stmt
condition: {and_chain(&mut ctx, cond)}
body: (block stmt: {body}))
body: (block stmt: {..body}))
),
// Repeat-while loop
rule!(
@@ -719,28 +719,28 @@ fn translation_rules() -> Vec<Rule<SwiftContext>> {
=>
(do_while_stmt
condition: {and_chain(&mut ctx, cond)}
body: (block stmt: {body}))
body: (block stmt: {..body}))
),
// Labeled statement (e.g. `outer: for ...`). Strip the trailing ':' from the label token.
rule!((labeled_statement label: (statement_label) @lbl statement: @stmt) => {
let text = ctx.ast.source_text(lbl);
let text = ctx.ast.source_text(lbl.into());
let name = &text[..text.len() - 1];
tree!((labeled_stmt label: (identifier #{name}) stmt: {stmt}))
}),
// ---- Collections ----
// Array literal
rule!((array_literal element: _* @elems) => (array_literal element: {elems})),
rule!((array_literal element: _* @elems) => (array_literal element: {..elems})),
// Empty array literal
rule!((array_literal) => (array_literal)),
// Dictionary literal — zip keys and values into key_value_pairs
rule!(
(dictionary_literal key: _* @keys value: _* @vals)
=>
(map_literal element: {keys.into_iter().zip(vals).map(|(k, v)|
(map_literal element: {..keys.into_iter().zip(vals).map(|(k, v)|
tree!((key_value_pair key: {k} value: {v}))
)})
),
rule!((dictionary_literal element: _* @elems) => (map_literal element: {elems})),
rule!((dictionary_literal element: _* @elems) => (map_literal element: {..elems})),
rule!((dictionary_literal_item key: @k value: @v) => (key_value_pair key: {k} value: {v})),
// ---- Optionals and errors ----
// Optional chaining — unwrap the marker
@@ -753,8 +753,8 @@ fn translation_rules() -> Vec<Rule<SwiftContext>> {
(do_statement body: (block statement: _* @body) catch: (catch_block)* @catches)
=>
(try_expr
body: (block stmt: {body})
catch_clause: {catches})
body: (block stmt: {..body})
catch_clause: {..catches})
),
// Catch block with bound identifier; optional where-clause guard.
rule!(
@@ -766,14 +766,14 @@ fn translation_rules() -> Vec<Rule<SwiftContext>> {
=>
(catch_clause
pattern: {pattern}
guard: {guard}
body: (block stmt: {body}))
guard: {..guard}
body: (block stmt: {..body}))
),
// Catch block without error binding
rule!(
(catch_block keyword: (catch_keyword) body: (block statement: _* @body))
=>
(catch_clause body: (block stmt: {body}))
(catch_clause body: (block stmt: {..body}))
),
// Empty catch block: catch {}
rule!(
@@ -787,7 +787,7 @@ fn translation_rules() -> Vec<Rule<SwiftContext>> {
=>
(catch_clause
pattern: {pat}
body: (block stmt: {body}))
body: (block stmt: {..body}))
),
// As expression (type cast) — as?, as!
rule!((as_expression (as_operator) @op expr: @val type: @ty) => (type_cast_expr expr: {val} operator: (infix_operator #{op}) type: {ty})),
@@ -812,7 +812,7 @@ fn translation_rules() -> Vec<Rule<SwiftContext>> {
pattern: (name_pattern identifier: (identifier #{parts.last().unwrap()}))
imported_expr: {name}
modifier: (modifier #{kind})
modifier: {mods})
modifier: {..mods})
),
// Non-scoped import declaration (for example `import Foundation`):
// flatten the identifier parts into a member_access_expr and use a
@@ -823,7 +823,7 @@ fn translation_rules() -> Vec<Rule<SwiftContext>> {
(import_declaration
pattern: (bulk_importing_pattern)
imported_expr: {name}
modifier: {mods})
modifier: {..mods})
),
// ---- Types and classes ----
// Self expression → name_expr
@@ -831,7 +831,7 @@ fn translation_rules() -> Vec<Rule<SwiftContext>> {
// Super expression → super_expr
rule!((super_expression) => (super_expr)),
// Modifiers — unwrap to individual modifier children
rule!((modifiers _* @mods) => {mods}),
rule!((modifiers _* @mods) => {..mods}),
rule!((attribute) @m => (modifier #{m})),
rule!((visibility_modifier) @m => (modifier #{m})),
rule!((function_modifier) @m => (modifier #{m})),
@@ -848,7 +848,7 @@ fn translation_rules() -> Vec<Rule<SwiftContext>> {
// Keep a conservative textual fallback to avoid dropping type information.
rule!((user_type) @ty => (named_type_expr name: (identifier #{ty}))),
// Tuple type → tuple_type_expr
rule!((tuple_type element: _* @elems) => (tuple_type_expr element: {elems})),
rule!((tuple_type element: _* @elems) => (tuple_type_expr element: {..elems})),
rule!((tuple_type_item name: @name type: @ty) => (tuple_type_element name: (identifier #{name}) type: {ty})),
rule!((tuple_type_item type: @ty) => (tuple_type_element type: {ty})),
// Array type `[T]` → generic_type_expr with Array base
@@ -865,7 +865,7 @@ fn translation_rules() -> Vec<Rule<SwiftContext>> {
base: (named_type_expr name: (identifier "Optional"))
type_argument: {w})),
// Function type `(Params) -> Ret` → function_type_expr.
rule!((function_type parameter: _* @ps return_type: @ret) => (function_type_expr parameter: {ps} return_type: {ret})),
rule!((function_type parameter: _* @ps return_type: @ret) => (function_type_expr parameter: {..ps} return_type: {ret})),
rule!((function_type_parameter name: @name type: @ty) => (parameter external_name: (identifier #{name}) type: {ty})),
rule!((function_type_parameter type: @ty) => (parameter type: {ty})),
// Selector expression: `#selector(inner)` -- not yet supported
@@ -889,10 +889,10 @@ fn translation_rules() -> Vec<Rule<SwiftContext>> {
=>
(class_like_declaration
modifier: (modifier #{kind})
modifier: {mods}
modifier: {..mods}
name: (identifier #{name})
base_type: {bases}
member: {members})
base_type: {..bases}
member: {..members})
),
// Enum class declaration: same as a regular class but with an enum body.
rule!(
@@ -905,10 +905,10 @@ fn translation_rules() -> Vec<Rule<SwiftContext>> {
=>
(class_like_declaration
modifier: (modifier #{kind})
modifier: {mods}
modifier: {..mods}
name: (identifier #{name})
base_type: {bases}
member: {members})
base_type: {..bases}
member: {..members})
),
// Class declaration with empty body
rule!(
@@ -921,9 +921,9 @@ fn translation_rules() -> Vec<Rule<SwiftContext>> {
=>
(class_like_declaration
modifier: (modifier #{kind})
modifier: {mods}
modifier: {..mods}
name: (identifier #{name})
base_type: {bases})
base_type: {..bases})
),
// Protocol declaration
rule!(
@@ -935,10 +935,10 @@ fn translation_rules() -> Vec<Rule<SwiftContext>> {
=>
(class_like_declaration
modifier: (modifier "protocol")
modifier: {mods}
modifier: {..mods}
name: (identifier #{name})
base_type: {bases}
member: {members})
base_type: {..bases}
member: {..members})
),
// Protocol function — return type and body statements both optional.
rule!(
@@ -950,11 +950,11 @@ fn translation_rules() -> Vec<Rule<SwiftContext>> {
(modifiers)* @mods)
=>
(function_declaration
modifier: {mods}
modifier: {..mods}
name: (identifier #{name})
parameter: {params}
return_type: {ret}
body: (block stmt: {body_stmts}))
parameter: {..params}
return_type: {..ret}
body: (block stmt: {..body_stmts}))
),
// Init declaration → constructor_declaration. Body statements optional;
// body itself is also optional (protocol requirement).
@@ -965,9 +965,9 @@ fn translation_rules() -> Vec<Rule<SwiftContext>> {
(modifiers)* @mods)
=>
(constructor_declaration
modifier: {mods}
parameter: {params}
body: (block stmt: {body_stmts}))
modifier: {..mods}
parameter: {..params}
body: (block stmt: {..body_stmts}))
),
// Deinit declaration → destructor_declaration. Body statements optional.
rule!(
@@ -976,15 +976,15 @@ fn translation_rules() -> Vec<Rule<SwiftContext>> {
(modifiers)* @mods)
=>
(destructor_declaration
modifier: {mods}
body: (block stmt: {body_stmts}))
modifier: {..mods}
body: (block stmt: {..body_stmts}))
),
// Typealias declaration
rule!(
(typealias_declaration name: @name value: @val (modifiers)* @mods)
=>
(type_alias_declaration
modifier: {mods}
modifier: {..mods}
name: (identifier #{name})
r#type: {val})
),
@@ -999,9 +999,9 @@ fn translation_rules() -> Vec<Rule<SwiftContext>> {
(associatedtype_declaration name: @name inherits_from: _? @bound (modifiers)* @mods)
=>
(associated_type_declaration
modifier: {mods}
modifier: {..mods}
name: (identifier #{name})
bound: {bound})
bound: {..bound})
),
// Protocol property declaration: translate each accessor
// requirement to an `accessor_declaration` carrying the property
@@ -1018,8 +1018,8 @@ fn translation_rules() -> Vec<Rule<SwiftContext>> {
type: _? @ty
(modifiers)* @mods)
=>
{{
ctx.property_name = Some(tree!((identifier #{name})));
{..{
ctx.property_name = Some(tree!((identifier #{name})).into());
ctx.property_type = ty;
ctx.outer_modifiers = mods;
@@ -1040,23 +1040,23 @@ fn translation_rules() -> Vec<Rule<SwiftContext>> {
=>
(accessor_declaration
name: {ctx.property_name.ok_or("getter_specifier outside protocol_property_declaration context")?}
type: {ctx.property_type}
type: {..ctx.property_type}
accessor_kind: (accessor_kind "get")
modifier: {ctx.outer_modifiers.clone()}
modifier: {chained_modifier(&mut ctx)})
modifier: {..ctx.outer_modifiers.clone()}
modifier: {..chained_modifier(&mut ctx)})
),
rule!(
(setter_specifier)
=>
(accessor_declaration
name: {ctx.property_name.ok_or("setter_specifier outside protocol_property_declaration context")?}
type: {ctx.property_type}
type: {..ctx.property_type}
accessor_kind: (accessor_kind "set")
modifier: {ctx.outer_modifiers.clone()}
modifier: {chained_modifier(&mut ctx)})
modifier: {..ctx.outer_modifiers.clone()}
modifier: {..chained_modifier(&mut ctx)})
),
// protocol_property_requirements wrapper — should be consumed by above; fallback
rule!((protocol_property_requirements accessor: _* @accs) => {accs}),
rule!((protocol_property_requirements accessor: _* @accs) => {..accs}),
// Computed getter → accessor_declaration (body optional).
// Reads property name/type from the outer property_binding rule
// and binding/outer modifiers + chained tag from the outer
@@ -1065,58 +1065,58 @@ fn translation_rules() -> Vec<Rule<SwiftContext>> {
(computed_getter body: (block statement: _* @body)?)
=>
(accessor_declaration
modifier: {ctx.binding_modifier}
modifier: {ctx.outer_modifiers.clone()}
modifier: {chained_modifier(&mut ctx)}
modifier: {..ctx.binding_modifier}
modifier: {..ctx.outer_modifiers.clone()}
modifier: {..chained_modifier(&mut ctx)}
name: {ctx.property_name.ok_or("computed_getter outside property_binding context")?}
type: {ctx.property_type}
type: {..ctx.property_type}
accessor_kind: (accessor_kind "get")
body: (block stmt: {body}))
body: (block stmt: {..body}))
),
// Computed setter with explicit parameter name.
rule!(
(computed_setter parameter: @param body: (block statement: _* @body))
=>
(accessor_declaration
modifier: {ctx.binding_modifier}
modifier: {ctx.outer_modifiers.clone()}
modifier: {chained_modifier(&mut ctx)}
modifier: {..ctx.binding_modifier}
modifier: {..ctx.outer_modifiers.clone()}
modifier: {..chained_modifier(&mut ctx)}
name: {ctx.property_name.ok_or("computed_setter outside property_binding context")?}
type: {ctx.property_type}
type: {..ctx.property_type}
accessor_kind: (accessor_kind "set")
parameter: (parameter pattern: (name_pattern identifier: (identifier #{param})))
body: (block stmt: {body}))
body: (block stmt: {..body}))
),
// Computed setter without explicit parameter name; body optional.
rule!(
(computed_setter body: (block statement: _* @body)?)
=>
(accessor_declaration
modifier: {ctx.binding_modifier}
modifier: {ctx.outer_modifiers.clone()}
modifier: {chained_modifier(&mut ctx)}
modifier: {..ctx.binding_modifier}
modifier: {..ctx.outer_modifiers.clone()}
modifier: {..chained_modifier(&mut ctx)}
name: {ctx.property_name.ok_or("computed_setter outside property_binding context")?}
type: {ctx.property_type}
type: {..ctx.property_type}
accessor_kind: (accessor_kind "set")
body: (block stmt: {body}))
body: (block stmt: {..body}))
),
// Computed modify → accessor_declaration
rule!(
(computed_modify body: (block statement: _* @body))
=>
(accessor_declaration
modifier: {ctx.binding_modifier}
modifier: {ctx.outer_modifiers.clone()}
modifier: {chained_modifier(&mut ctx)}
modifier: {..ctx.binding_modifier}
modifier: {..ctx.outer_modifiers.clone()}
modifier: {..chained_modifier(&mut ctx)}
name: {ctx.property_name.ok_or("computed_modify outside property_binding context")?}
type: {ctx.property_type}
type: {..ctx.property_type}
accessor_kind: (accessor_kind "modify")
body: (block stmt: {body}))
body: (block stmt: {..body}))
),
// willset/didset block — spread to children (only reachable as a
// fallback; the outer property_binding manual rule normally
// captures the willset/didset clauses directly).
rule!((willset_didset_block _* @clauses) => {clauses}),
rule!((willset_didset_block _* @clauses) => {..clauses}),
// willset clause → accessor_declaration (body optional). Reads
// `ctx.property_name` set by the outer property_binding rule and
// binding/outer modifiers + chained tag from the outer
@@ -1125,24 +1125,24 @@ fn translation_rules() -> Vec<Rule<SwiftContext>> {
(willset_clause body: (block statement: _* @body)?)
=>
(accessor_declaration
modifier: {ctx.binding_modifier}
modifier: {ctx.outer_modifiers.clone()}
modifier: {chained_modifier(&mut ctx)}
modifier: {..ctx.binding_modifier}
modifier: {..ctx.outer_modifiers.clone()}
modifier: {..chained_modifier(&mut ctx)}
name: {ctx.property_name.ok_or("willset_clause outside property_binding context")?}
accessor_kind: (accessor_kind "willSet")
body: (block stmt: {body}))
body: (block stmt: {..body}))
),
// didset clause → accessor_declaration (body optional).
rule!(
(didset_clause body: (block statement: _* @body)?)
=>
(accessor_declaration
modifier: {ctx.binding_modifier}
modifier: {ctx.outer_modifiers.clone()}
modifier: {chained_modifier(&mut ctx)}
modifier: {..ctx.binding_modifier}
modifier: {..ctx.outer_modifiers.clone()}
modifier: {..chained_modifier(&mut ctx)}
name: {ctx.property_name.ok_or("didset_clause outside property_binding context")?}
accessor_kind: (accessor_kind "didSet")
body: (block stmt: {body}))
body: (block stmt: {..body}))
),
// Preprocessor conditionals — unsupported
rule!((diagnostic) => (unsupported_node)),