hohn/codeql
1
0
Fork 0
mirror of https://github.com/github/codeql.git synced 2025-12-18 01:33:15 +01:00
Code Issues Packages Projects Releases Wiki Activity

Swift: skip QL code generation on untouched files

Browse Source 
This is a developer QoL improvement, where running codegen will skip
writing (and especially formatting) any files that were not changed.

**Why?** While code generation in itself was pretty much instant, QL
formatting of generated code was starting to take a long time. This made
unconditionally running codegen quite annoying, for example before each
test run as part of an IDE workflow or as part of the pre-commit hook.

**How?** This was not completely straightforward as we could not work
with the contents of the file prior to code generation as that was
already post-processed by the QL formatting, so we had no chance of
comparing the output of template rendering with that. We therefore store
the hashes of the files _prior_ to QL formatting in a checked-in file
(`swift/ql/.generated.list`). We can therefore load those hashes at
the beginning of code generation, use them to compare the template
rendering output and update them in this special registry file.

**What else?** We also extend this mechanism to detect accidental
modification of generated files in a more robust way. Before this patch,
we were doing it with a rough regexp based heuristic. Now, we just store
the hashes of the files _after_ QL formatting in the same checked file,
so we can check that and stop generation if a generated file was
modified, or a stub was modified without removing the `// generated`
header.
This commit is contained in:
Paolo Tranquilli
2022-11-18 16:29:19 +01:00
parent 07969260c8
commit 2cd58817d7
9 changed files with 1314 additions and 200 deletions
Download Patch File Download Diff File Expand all files Collapse all files

9
swift/codegen/codegen.py
View File

@@ -40,11 +40,13 @@ def _parse_args() -> argparse.Namespace:
p.add_argument("--codeql-binary", default="codeql", help="command to use for QL formatting (default %(default)s)")
p.add_argument("--cpp-output", type=_abspath,
help="output directory for generated C++ files, required if trap or cpp is provided to --generate")
p.add_argument("--generated-registry", type=_abspath, default=paths.swift_dir / "ql/.generated.list",
help="registry file containing information about checked-in generated code")
return p.parse_args()
def _abspath(x: str) -> pathlib.Path:
return pathlib.Path(x).resolve()
def _abspath(x: str) -> typing.Optional[pathlib.Path]:
return pathlib.Path(x).resolve() if x else None
def run():
@@ -56,9 +58,8 @@ def run():
else:
log_level = logging.INFO
logging.basicConfig(format="{levelname} {message}", style='{', level=log_level)
exe_path = paths.exe_file.relative_to(opts.swift_dir)
for target in opts.generate:
generate(target, opts, render.Renderer(exe_path))
generate(target, opts, render.Renderer(opts.swift_dir))
if __name__ == "__main__":