Add posixCollatingSymbol([.x.]) and posixEquivalenceClass([=x=]) predicates
to ParseRegExp.qll, enabling correct tokenization of all three POSIX bracket
atom types inside character classes:
- [:name:] — already handled, unchanged
- [.x.] — new: posixCollatingSymbol predicate
- [=x=] — new: posixEquivalenceClass predicate
Refactored:
- charSetDelimiter: use inAnyPosixBracket helper (covers all three types)
- charSet closing: use inAnyPosixBracket helper
- inPosixBracket: updated to cover all three types
- simpleCharacter: updated to exclude all three types
- namedCharacterProperty: includes posixCollatingSymbol and posixEquivalenceClass
- inAnyPosixBracket: new private helper; uses pos in [x..y-1] for QL binding
Before this commit (commit 6):
[[.a.]] → RegExpCharacterClass([[.a.]) + stray ] (WRONG)
[[=a=]] → RegExpCharacterClass([[=a=]) + stray ] (WRONG)
After this commit:
[[.a.]] → RegExpCharacterClass containing RegExpNamedCharacterProperty [.a.]
[[=a=]] → RegExpCharacterClass containing RegExpNamedCharacterProperty [=a=]
Regenerated parse.expected and regexp.expected via codeql test run --learn
(CodeQL 2.26.1); all 2 tests pass.
Add new test cases to test.cpp exercising POSIX bracket expressions inside
character classes as supported by std::regex ECMAScript mode:
- Single POSIX classes: [[:alpha:]], [[:digit:]], [[:space:]], [[:upper:]],
[[:lower:]], [[:alnum:]], [[:print:]], [[:punct:]]
- Negated outer bracket: [^[:space:]]
- Mixed: regular char + POSIX class [a[:space:]]
- POSIX collating symbol [[.a.]]
- POSIX equivalence class [[=a=]]
- Mixed POSIX + range [[:alpha:]0-9]
The existing r_posix1-r_posix4 cases remain untouched. New cases use distinct
names (r_posix_alpha, r_posix_digit, etc.) to avoid redeclaration.
Generated .expected via codeql test run --learn (CodeQL 2.26.1); all 2 tests
pass. The expected output captures the CURRENT (pre-fix) tokenization — commit 7
will change these rows when correct POSIX nesting is implemented.
Content-only edits to test.cpp to remove Ruby-only syntax that has no
C++ ECMAScript equivalent:
1. Removed: "a{,8}" — Ruby-only "{,n}" no-lower-bound quantifier
(ECMAScript requires an explicit lower bound in {n,m})
2. Removed: second ".*" for /.*/m — mode flags in C++ are construction-site
arguments (e.g. std::regex::multiline), not part of the pattern string.
This mode variant was the exact same pattern string as r_meta1 so it
added no coverage.
3. Removed: "(?'foo'fo+)" — Ruby single-quote named-group form.
ECMAScript only supports the angle-bracket form (?<name>...).
Left in place for now (removed with the parser in commit 5):
\\A, \\z, \\G, \\h, \\H — Ruby-only anchor/escape classes.
The POSIX-bracket cases are kept through commit 7.
.expected regenerated by:
codeql test run --learn --search-path=. cpp/ql/test/library-tests/regex/
(CodeQL CLI 2.26.1)
All 2 tests passed.
Introduces cpp/ql/test/library-tests/regex/test.cpp with the Ruby regex
corpus mechanically translated to C++ string literals and wrapped in
std::regex construction calls using fully self-contained in-file stubs
(no #include of any external/standard header).
Stub surface:
std::basic_regex<CharT> constructor(const char*), constructor(const char*, int),
assign(const char*)
std::regex typedef for basic_regex<char>
std::regex_match / regex_search / regex_replace free functions
1:1 corpus mapping from regexp.rb:
- All Ruby /regex/ literals translated to C++ "string" literals
(each backslash doubled: /\d/ -> "\\d")
- Dropped: /#{A}bc/ (Ruby string interpolation, no string-literal form)
- Kept: a{,8}, .*m, \A, \z, \G, \h\H, (?'foo'...) — these are removed
in commits 4 and 5 (keeping them here preserves 1:1 mapping)
Also removes `abstract` from RegExp class in ParseRegExp.qll so that all
StringLiterals are regex candidates (trivial syntactic gate; no dataflow).
.expected generated by:
codeql test run --learn --search-path=. cpp/ql/test/library-tests/regex/
(CodeQL CLI 2.26.1, bundled C/C++ extractor)
All 2 tests passed.
This is a verbatim copy of the Ruby ParseRegExp.qll and RegExpTreeView.qll,
with only the file-header comments updated to reflect the new path.
The test files (parse.ql, regexp.ql, regexp.rb, *.expected) are also copied
verbatim from the Ruby test suite as a baseline reference.
This commit does NOT compile against C++: it still imports codeql.ruby.AST
and references Ruby AST types (RegExpLiteral, StringlikeLiteral, etc.).
The adaptation to compile against C++ is in the next commit.
Also adds the codeql/regex dependency to cpp/ql/lib/qlpack.yml and the
unreleased change-note at cpp/ql/lib/change-notes/2026-07-21-cpp-regex-tree-view.md.
