When join-ordering and evaluating this conjunction,
it is preferable to start with the relatively small set of
`sanitizer` calls, then compute the set of SSA variables accessed
as the arguments of those sanitizer calls, then reason about how
those variables are used in phi nodes.
Use directional binding pragmas to encourage this join order
by picking `sanitizer` first, and discourage picking the
opposite join order starting with `phi`.
This impacts performance of the ATM XSS queries on large databases like Node,
where computing all variable accesses from phi nodes leads to 435M+ tuples.
One of the heuristics for test files looks for source files
of the form `base.ext`, then looks for sibling test files
of the form `base.test.ext` or `base.spec.ext`.
On large databases, the result join order computed all source files,
the containers of those files, then all other files within those
containers, before computing the test file names and filtering using
those names.
The product of all files with all other files in the same containers
is of the same order of magnitude as the product of the `files`
table with itself, which on large DBs like Node can be 12M+ tuples.
As a performance optimisation, factor out a helper predicate that
computes the likely test file names for each source file, so these
can be determined with a single join against the files table.
This results in much better join orders, such as computing the set
of files and their containers, then the test file names, then the
sibling files with those names.
This loses some flexibility because the set of 'test' extension names
is hardcoded in the library rather than provided by the caller predicate.
The original predicate remains to avoid breaking other callers, but could
eventually be deprecated.
Absent features are now represented implicitly by the absence of a row
in the `tokenFeatures` relation, rather than explicitly by an empty
string. This leads to improved runtime performance. To enable this
implicit representation, we pass the set of supported token features to
the `scoreEndpoints` HOP. Requires CodeQL CLI v2.7.4.
The refactoring to remove the `CodeToFeatures` AST reintroduced a
performance problem. This commit resolves it by pushing size
restrictions into intermediate predicates.
Pushing the restriction to 256 tokens into the `bodyTokens` predicate
means we avoid this predicate blowing up due to very large functions.
This results in a runtime improvement from 1800s+ to 294s as measured
on a problematic repo on my machine (I didn't wait for the query to
finish running).
A few change notes slipped through the cracks of my previous change. These are now in the proper locations: `old-change-notes` for older notes, and `<lang>\ql\[src|lib]\change-notes` for current change notes.
