Python: Improve parser logging/timing/customisability

Does a bunch of things, unfortunately all in the same place, so my
apologies in advance for a slightly complicated commit.

As for the changes themselves, this commit

- Adds timers for the old and new parsers. This means we get the overall
time spent on these parts of the extractor if the extractor is run with
`DEBUG` output shown.
- Adds logging information (at the `DEBUG` level) to show which
invocations of the parsers happen when, and whether they succeed or not.
- Adds support for using an environment variable named
`CODEQL_PYTHON_DISABLE_OLD_PARSER` to disable using the old parser
entirely. This makes it easier to test the new parser in isolation.
- Fixes a bug where we did not check whether a parse with the new parser
had already succeeded, and so would do a superfluous second parse.
This commit is contained in:
Taus
2024-10-30 13:58:46 +00:00
parent f75615b913
commit 2ef3ae9860
4 changed files with 24 additions and 6 deletions

View File

@@ -98,6 +98,7 @@ class ModuleImporter(object):
def get_import_nodes(self, loaded_module):
'Return list of AST nodes representing imports'
try:
self.logger.debug("Looking for imports in %s", loaded_module.path)
return imports_from_ast(loaded_module.py_ast)
except Exception as ex:
if isinstance(ex, SyntaxError):