An example (provided by @redsun82) is the string `f"{x:=^20}"`. Parsing
this (with unnamed nodes shown) illustrates the problem:
```
module [0, 0] - [2, 0]
expression_statement [0, 0] - [0, 11]
string [0, 0] - [0, 11]
string_start [0, 0] - [0, 2]
interpolation [0, 2] - [0, 10]
"{" [0, 2] - [0, 3]
expression: named_expression [0, 3] - [0, 9]
name: identifier [0, 3] - [0, 4]
":=" [0, 4] - [0, 6]
ERROR [0, 6] - [0, 7]
"^" [0, 6] - [0, 7]
value: integer [0, 7] - [0, 9]
"}" [0, 9] - [0, 10]
string_end [0, 10] - [0, 11]
```
Observe that we've managed to combine the format specifier token `:` and
the fill character `=` in a single token (which doesn't match the `:` we
expect in the grammar rule), and hence we get a syntax error.
If we change the `=` to some other character (e.g. a `-`), we instead
get
```
module [0, 0] - [2, 0]
expression_statement [0, 0] - [0, 11]
string [0, 0] - [0, 11]
string_start [0, 0] - [0, 2]
interpolation [0, 2] - [0, 10]
"{" [0, 2] - [0, 3]
expression: identifier [0, 3] - [0, 4]
format_specifier: format_specifier [0, 4] - [0, 9]
":" [0, 4] - [0, 5]
"}" [0, 9] - [0, 10]
string_end [0, 10] - [0, 11]
```
and in particular no syntax error.
To fix this, we want to ensure that the `:` is lexed on its own, and the
`token(prec(1, ...))` construction can be used to do exactly this.
Finally, you may wonder why `=` is special here. I think what's going on
is that the lexer knows that `:=` is a token on its own (because it's
used in the walrus operator), and so it greedily consumes the following
`=` with this in mind.
Uses the same trick as for `ExtractedArgumentNode`, wherein we postpone
the global restriction on the charpred to instead be in the `argumentOf`
predicate (which is global anyway).
In addition to this, we also converted `CapturedVariablesArgumentNode`
into a proper synthetic node, and added an explicit post-update node for
it. These nodes just act as wrappers for the function part of call
nodes. Thus, to make them work with the variable capture machinery, we
simply map them to the closure node for the corresponding control-flow
or post-update node.
Fixes the test failures that arose from making `ExtractedArgumentNode`
local.
For the consistency checks, we now explicitly exclude the
`ExtractedArgumentNode`s (now much more plentiful due to the
overapproximation) that don't have a corresponding `getCallArg` tuple.
For various queries/tests using `instanceof ArgumentNode`, we instead us
`isArgumentNode`, which explicitly filters out the ones for which
`isArgumentOf` doesn't hold (which, again, is the case for most of the
nodes in the overapproximation).
Explicitly adds a bunch of nodes that were previously (using a global
analysis) identified as `ExtractedArgumentNode`s. These are then
subsequently filtered out in `argumentOf` (which is global) by putting
the call to `getCallArg` there instead of in the charpred.
