POSIX bracket expressions ([:name:], [.x.], [=x=]) are only valid nested
inside another character class in std::regex. The three predicates now
additionally require that at position `start` we are inside an outer
character class, approximated by requiring more non-escaped `[` than
non-escaped `]` before `start`. A well-formed POSIX bracket contributes
one `[` and one `]` at/after `start`, so the check is unaffected by
earlier POSIX brackets.
Also stop emitting individual charSetTokens for characters inside a
POSIX bracket (`inAnyPosixBracket` guard), which removes the spurious
`[RegExpCharacterRange] :]-z` on [[:alpha:]-z].
Post-fix, unnested `[:digit:]`, `[:alpha:]`, and `a[🅱️]c` are correctly
parsed as ordinary character classes / literal sequences rather than
`RegExpNamedCharacterProperty`; and the [[:alpha:]-z] range no longer
crosses the POSIX bracket boundary.
Bundle: github/codeql-action codeql-bundle-v2.26.1 (CodeQL CLI 2.26.1).
In default ECMAScript std::regex (no multiline flag; this parser cannot
detect construction-site flags), ^ and $ anchor to string start/end, not
line boundaries. Update RegExpCaret, RegExpDollar, and RegExpAnchor doc
comments to describe string-start/string-end semantics and note that
multiline is not modelled, mirroring PR #22200. Tokenization unchanged.
.expected files unchanged.
Bundle: github/codeql-action codeql-bundle-v2.26.1 (CodeQL CLI 2.26.1).
\U is a Ruby/PCRE-ism, not ECMAScript. Restrict
RegExpEscape.isUnicode()/getUnicode() to lowercase \uHHHH only. No \U
path exists in ParseRegExp.qll's escapedCharacter, and no corpus case
references \U. .expected files unchanged.
Bundle: github/codeql-action codeql-bundle-v2.26.1 (CodeQL CLI 2.26.1).
In ECMAScript std::regex, \0 matches NUL. The previous escapedCharacter
arms all rejected it: the final arm's `not exists(getChar(start+1).toInt())`
guard fails for "0", and the numbered-backref arm excludes 0. Add an
explicit \0 case (end=start+2) guarded so the following character is not
a digit, mirroring EcmaRegExp.escapedCharacter in PR #22200. Add corpus
case "a\\0b" to test.cpp; \0 now parses as RegExpEscape.
Bundle: github/codeql-action codeql-bundle-v2.26.1 (CodeQL CLI 2.26.1).
std::regex ECMAScript mode does not support \p{Name}; \p tokenizes as an
identity escape. Remove pStyleNamedCharacterProperty and its call sites;
namedCharacterProperty now covers only POSIX brackets ([:name:], [.x.],
[=x=]) and namedCharacterPropertyIsInverted keeps only the [[:^name:]]
case (fixing the offset from start+3 to start+2 for POSIX). Update
RegExpNamedCharacterProperty getName/isInverted docs to describe POSIX
brackets. Remove the four \p{...}/\P{...} corpus lines from test.cpp;
keep a plain [a-f\d]+ case for the class-with-escape shape.
Bundle: github/codeql-action codeql-bundle-v2.26.1 (CodeQL CLI 2.26.1).
Add posixCollatingSymbol([.x.]) and posixEquivalenceClass([=x=]) predicates
to ParseRegExp.qll, enabling correct tokenization of all three POSIX bracket
atom types inside character classes:
- [:name:] — already handled, unchanged
- [.x.] — new: posixCollatingSymbol predicate
- [=x=] — new: posixEquivalenceClass predicate
Refactored:
- charSetDelimiter: use inAnyPosixBracket helper (covers all three types)
- charSet closing: use inAnyPosixBracket helper
- inPosixBracket: updated to cover all three types
- simpleCharacter: updated to exclude all three types
- namedCharacterProperty: includes posixCollatingSymbol and posixEquivalenceClass
- inAnyPosixBracket: new private helper; uses pos in [x..y-1] for QL binding
Before this commit (commit 6):
[[.a.]] → RegExpCharacterClass([[.a.]) + stray ] (WRONG)
[[=a=]] → RegExpCharacterClass([[=a=]) + stray ] (WRONG)
After this commit:
[[.a.]] → RegExpCharacterClass containing RegExpNamedCharacterProperty [.a.]
[[=a=]] → RegExpCharacterClass containing RegExpNamedCharacterProperty [=a=]
Regenerated parse.expected and regexp.expected via codeql test run --learn
(CodeQL 2.26.1); all 2 tests pass.
Introduces cpp/ql/test/library-tests/regex/test.cpp with the Ruby regex
corpus mechanically translated to C++ string literals and wrapped in
std::regex construction calls using fully self-contained in-file stubs
(no #include of any external/standard header).
Stub surface:
std::basic_regex<CharT> constructor(const char*), constructor(const char*, int),
assign(const char*)
std::regex typedef for basic_regex<char>
std::regex_match / regex_search / regex_replace free functions
1:1 corpus mapping from regexp.rb:
- All Ruby /regex/ literals translated to C++ "string" literals
(each backslash doubled: /\d/ -> "\\d")
- Dropped: /#{A}bc/ (Ruby string interpolation, no string-literal form)
- Kept: a{,8}, .*m, \A, \z, \G, \h\H, (?'foo'...) — these are removed
in commits 4 and 5 (keeping them here preserves 1:1 mapping)
Also removes `abstract` from RegExp class in ParseRegExp.qll so that all
StringLiterals are regex candidates (trivial syntactic gate; no dataflow).
.expected generated by:
codeql test run --learn --search-path=. cpp/ql/test/library-tests/regex/
(CodeQL CLI 2.26.1, bundled C/C++ extractor)
All 2 tests passed.
This is a verbatim copy of the Ruby ParseRegExp.qll and RegExpTreeView.qll,
with only the file-header comments updated to reflect the new path.
The test files (parse.ql, regexp.ql, regexp.rb, *.expected) are also copied
verbatim from the Ruby test suite as a baseline reference.
This commit does NOT compile against C++: it still imports codeql.ruby.AST
and references Ruby AST types (RegExpLiteral, StringlikeLiteral, etc.).
The adaptation to compile against C++ is in the next commit.
Also adds the codeql/regex dependency to cpp/ql/lib/qlpack.yml and the
unreleased change-note at cpp/ql/lib/change-notes/2026-07-21-cpp-regex-tree-view.md.
