Commit Graph

11 Commits

Author SHA1 Message Date
Taus
f75615b913 Merge pull request #17822 from github/tausbn/python-more-parser-fixes
Python: A few more parser fixes
2024-10-30 13:47:10 +01:00
Taus
5db601af3c Python: Allow comments in comprehensions
A somewhat complicated solution that necessitated adding a new custom
function to `tsg-python`. See the comments in `python.tsg` for why this
was necessary.
2024-10-23 14:24:47 +00:00
Taus
4f60494019 Python: Support assignments of the form [x,y,z] = w
Surprisingly, the new parser did not support these constructs (and the
relevant test was missing this case), so on files that required the new
parser we were unable to parse this construct.

To fix it, we add `list_pattern` (not to be confused with
`pattern_list`) as a `tree-sitter-python` node that results in a `List`
node in the AST.
2024-10-22 16:06:35 +00:00
Taus
9c913902c5 Python: Allow except* to be written as except *
Turns out, `except*` is actually not a token on its own according to the
Python grammar. This means it's legal to write `except *foo: ...`, which
we previously would consider a syntax error.

To fix it, we simply break up the `except*` into two separate tokens.
2024-10-22 15:39:29 +00:00
Taus
8053e0ed44 Python: Allow list_splats as type annotations
That is, the `*T` in `def foo(*args : *T): ...`.

This is apparently a piece of syntax we did not support correctly until
now.

In terms of the grammar, we simply add `list_splat` as a possible
alternative for `type` (which could previously only be an `expression`).
We also update `python.tsg` to not specify `expression` those places (as
the relevant stanzas will then not work for `list_splat`s).

This syntax is not supported by the old parser, hence we only add a new
parser test for it.
2024-10-22 15:17:12 +00:00
Taus
9803bbdc4b Python: Update class parser test 2024-10-21 15:35:48 +00:00
Taus
819b3d77ab Python: Update test expectations
Note that this still includes the somewhat puzzling parsing of
`Spam[**P2]` as an exponentiation with an empty left hand side. When we
fix that bug, we should also update this test to contain actually valid
syntax.
2024-10-15 11:22:33 +00:00
Taus
1ced5b44d7 Python: Add test for type parameter defaults 2024-10-15 11:22:30 +00:00
yoff
0b0e8a4bf5 Update python/extractor/tests/parser/.gitignore
As suggested by @tausbn
2024-10-09 12:22:17 +02:00
Rasmus Lerchedahl Petersen
ad630bc6ff Python: ignore some extractor test output
If you test the extractor locally, you want to ignore these files.
2024-10-09 11:34:58 +02:00
Taus
6dec323cfc Python: Copy Python extractor to codeql repo 2024-03-07 13:59:16 +00:00