Combine two regexes into a single one.
This saves up to 5s on large databases by reducing the number
of separate scans of the comments table before regex matching.
The combined regex is slightly more permissive than the
original two, since it allows a combination of the two
matched formats. A string that matches one of the original
regexes will match the combined regex.
Some subclasses of GeneratedCodeMarkerComment regex match against `getLine(_)`.
When evaluated, this results in multiple scans (one per subclass that uses it)
of all comment lines in the database, before regex matching against those lines.
To make these scans smaller, regex match against the entire comment text
without splitting them into lines.
This is achieved using `?m` (multiline) and line boundaries in the regexes.
Previously heuristic sinks were always included, to avoid us filtering
them out due to not being an argument to an external library call.
In this commit we move the argument to an external library call
filtering to the query-specific endpoint filters.
This lets us filter out heuristic sinks if they match one of the other
endpoint filters, reducing FPs.
0e31439 introduces some occasional duplicate tokens due to duplicate AST
node attributes. The long-term fix is to update `CodeToFeatures.qll`,
but for the short-term, we update the concatenation to concatenate
unique (location, token) pairs.
Use a `strictcount` to identify whether there is exactly one feature or not.
If so, we use it. If not, we use the empty string.
Add context to ensure we filter the set of data flow nodes down to only
the set of endpoint nodes.
This performance optimisation avoids calculating the Cartesian product
of data flow nodes and feature names, but it does not avoid calculating
the (slightly smaller) Cartesian product of endpoint nodes and feature names.
Product size = number of endpoint nodes * number of feature names.
At time of writing there are 8 feature names.
