Python: Improve parser logging/timing/customisability

Does a bunch of things, unfortunately all in the same place, so my
apologies in advance for a slightly complicated commit.

As for the changes themselves, this commit

- Adds timers for the old and new parsers. This means we get the overall
time spent on these parts of the extractor if the extractor is run with
`DEBUG` output shown.
- Adds logging information (at the `DEBUG` level) to show which
invocations of the parsers happen when, and whether they succeed or not.
- Adds support for using an environment variable named
`CODEQL_PYTHON_DISABLE_OLD_PARSER` to disable using the old parser
entirely. This makes it easier to test the new parser in isolation.
- Fixes a bug where we did not check whether a parse with the new parser
had already succeeded, and so would do a superfluous second parse.
This commit is contained in:
Taus
2024-10-30 13:58:46 +00:00
parent f75615b913
commit 2ef3ae9860
4 changed files with 24 additions and 6 deletions

View File

@@ -168,7 +168,7 @@ def read_tsg_python_output(path, logger):
p.stdout.close()
p.terminate()
p.wait()
logger.info("Read {} nodes and {} edges from TSG output".format(len(node_attr), len(edge_attr)))
logger.debug("Read {} nodes and {} edges from TSG output".format(len(node_attr), len(edge_attr)))
return node_attr, edge_attr
def evaluate_string(s):