Commit Graph

31407 Commits

Author SHA1 Message Date
Aditya Sharad
ee97acefcd JS: Minor simplication of ranked basic block calculation
We have to look up the  node index within the block anyway,
so include it as an aggregation variable.
2022-01-18 16:23:04 +00:00
Aditya Sharad
c553330ee3 JS: Fix possible typo in DominatingPaths::hasWrite
Should most likely refer to `AccessPathWrite` and look for write nodes.
This also improves the performance of `rankedAccessPath`, since the
set of candidate blocks is now limited to blocks with both a read and a write.
2022-01-18 16:23:04 +00:00
Aditya Sharad
1d507f1993 JS: Improve performance of StandardEndpointFilters::isNumeric
Factor the regex-independent logic of `isReadFrom` into its own predicate.

Call this predicate directly from `isNumeric`, which doesn't have much
restrictive context on the set of starting nodes.
Use a binding hint to discourage starting with all expr nodes in this case.

Other callers may have more restrictive context on the set of nodes,
so they are not changed.
2022-01-18 16:23:04 +00:00
Aditya Sharad
1b7088abde JS: Tweak performance of CorsOriginHeaderWithAssociatedCredentialHeader
On databases with a large number of Exprs, it can be better
to start with the set of route handlers, then find their
response headers, then find the expression values set in
those headers.
2022-01-18 16:23:03 +00:00
Aditya Sharad
eec7b926b0 JS: Add this reference for clarity
No behaviour change.
2022-01-18 16:23:03 +00:00
Aditya Sharad
0a003edaa7 JS: Improve performance of Xss::isOptionallySanitizedEdge
When join-ordering and evaluating this conjunction,
it is preferable to start with the relatively small set of
`sanitizer` calls, then compute the set of SSA variables accessed
as the arguments of those sanitizer calls, then reason about how
those variables are used in phi nodes.

Use directional binding pragmas to encourage this join order
by picking `sanitizer` first, and discourage picking the
opposite join order starting with `phi`.

This impacts performance of the ATM XSS queries on large databases like Node,
where computing all variable accesses from phi nodes leads to 435M+ tuples.
2022-01-18 16:23:03 +00:00
Aditya Sharad
b7852cec7a JS: Improve performance of ClassifyFiles::isTestFile
One of the heuristics for test files looks for source files
of the form `base.ext`, then looks for sibling test files
of the form `base.test.ext` or `base.spec.ext`.

On large databases, the result join order computed all source files,
the containers of those files, then all other files within those
containers, before computing the test file names and filtering using
those names.

The product of all files with all other files in the same containers
is of the same order of magnitude as the product of the `files`
table with itself, which on large DBs like Node can be 12M+ tuples.

As a performance optimisation, factor out a helper predicate that
computes the likely test file names for each source file, so these
can be determined with a single join against the files table.
This results in much better join orders, such as computing the set
of files and their containers, then the test file names, then the
sibling files with those names.

This loses some flexibility because the set of 'test' extension names
is hardcoded in the library rather than provided by the caller predicate.
The original predicate remains to avoid breaking other callers, but could
eventually be deprecated.
2022-01-18 16:23:03 +00:00
Henry Mercer
03bb3cce73 JS: Update featurization for absent features optimization
Absent features are now represented implicitly by the absence of a row
in the `tokenFeatures` relation, rather than explicitly by an empty
string. This leads to improved runtime performance. To enable this
implicit representation, we pass the set of supported token features to
the `scoreEndpoints` HOP. Requires CodeQL CLI v2.7.4.
2022-01-13 16:29:39 +00:00
Henry Mercer
92d6fecc73 Optimize performance of body tokens
The refactoring to remove the `CodeToFeatures` AST reintroduced a
performance problem. This commit resolves it by pushing size
restrictions into intermediate predicates.
2022-01-13 16:29:04 +00:00
Henry Mercer
7f61738a23 Use US English spelling 2022-01-12 13:07:09 +00:00
Henry Mercer
6e37a65e84 Remove CodeToFeatures AST library 2022-01-12 12:47:28 +00:00
Henry Mercer
957e34d8a7 Make function body features library independent of CodeToFeatures AST 2022-01-12 12:47:28 +00:00
Henry Mercer
9e50ce873d Move function body features into their own file 2022-01-12 12:47:28 +00:00
Henry Mercer
865fb5d0ef Migrate representative entity -> representative function 2022-01-12 12:47:27 +00:00
Henry Mercer
0e5b493d0e Remove CodeToFeatures AST consistency checks
We no longer use the `CodeToFeatures` AST, therefore these checks are
defunct.
2022-01-12 12:47:27 +00:00
Henry Mercer
387829bbb4 Extract body tokens from the JS AST, not the CodeToFeatures AST 2022-01-12 12:47:25 +00:00
Henry Mercer
3ef69763a7 Merge pull request #7567 from github/henrymercer/atm-body-tokens-perf-opt
ATM: Optimize body tokens by pushing in size restriction
2022-01-12 12:45:27 +00:00
Tamás Vajk
9065a7f320 Merge pull request #7573 from tamasvajk/fix/java-field-decl-tostr
Java: Fix toString on field declarations with single field
2022-01-12 13:03:16 +01:00
Tony Torralba
8a80e02861 Merge pull request #7574 from pwntester/improve_strings_qll
Add models for AbstractStringBuilder.substring,subsequence,getChars
2022-01-12 12:01:28 +01:00
Tony Torralba
c2105e506b Added test cases 2022-01-12 11:06:58 +01:00
Alvaro Muñoz Sanchez
715d372572 Add models for AbstractStringBuilder.substring,subsequence,getChars 2022-01-12 10:54:27 +01:00
Anders Schack-Mulligen
c6a9b2b6ff Merge pull request #7572 from github/workflow/coverage/update
Update CSV framework coverage reports
2022-01-12 09:39:14 +01:00
Tamas Vajk
b9e0310aa2 Java: Fix toString on field declarations with single field 2022-01-12 09:22:16 +01:00
Michael Nebel
f17c110f51 Merge pull request #7562 from michaelnebel/csharp/record-seal-tostring
C#: Record types are allowed to seal ToString (test only).
2022-01-12 08:08:32 +01:00
github-actions[bot]
c79e8ab440 Add changed framework coverage reports 2022-01-12 00:10:48 +00:00
Andrew Eisenberg
da4f1d86aa Merge pull request #7355 from github/aeisenberg/remove-upgrades
Move upgrades into standard library packs
2022-01-11 14:09:10 -08:00
Andrew Eisenberg
07228672df Merge branch 'main' into aeisenberg/remove-upgrades 2022-01-11 11:25:27 -08:00
Mathias Vorreiter Pedersen
c45127fdd6 Merge pull request #7541 from github/rdmarsh2/dataflow-ipa-params
C++: Use an IPA type rather than negative indexes for argument/parameter matching in data flow
2022-01-11 16:52:13 +00:00
Tony Torralba
7b0d9ea525 Merge pull request #7054 from atorralba/atorralba/promote-log-injection
Java: Promote Log Injection from experimental
2022-01-11 17:26:18 +01:00
Henry Mercer
3f70476c87 ATM: Optimize body tokens by pushing in size limit
Pushing the restriction to 256 tokens into the `bodyTokens` predicate
means we avoid this predicate blowing up due to very large functions.

This results in a runtime improvement from 1800s+ to 294s as measured
on a problematic repo on my machine (I didn't wait for the query to
finish running).
2022-01-11 16:16:54 +00:00
Tony Torralba
1030ff7063 Update java/ql/src/Security/CWE/CWE-117/LogInjection.ql 2022-01-11 16:25:32 +01:00
Tony Torralba
4aacba8594 Merge pull request #6468 from atorralba/atorralba/promote-cleartext-sharedprefs
Java: Promote Cleartext storage of sensitive information using SharedPreferences from experimental
2022-01-11 16:23:53 +01:00
Tony Torralba
394c4a9ee0 Remove unused code 2022-01-11 14:50:48 +01:00
Michael Nebel
77763d7ee5 Merge pull request #7559 from michaelnebel/csharp/const-interpolatedstring
C#: Constant string interpolation (test only).
2022-01-11 14:01:55 +01:00
Michael Nebel
56bc3db46a C#: Add test case for sealed ToString modifier on a record type. 2022-01-11 13:58:43 +01:00
Michael Nebel
ae5d3a1ccb C#: Add example of sealing ToString on a record type. 2022-01-11 13:57:29 +01:00
Tony Torralba
50caf7d8dc Move change note to new location and remove import
Co-authored-by: Anders Schack-Mulligen <aschackmull@users.noreply.github.com>
2022-01-11 12:24:44 +01:00
Tony Torralba
b9e32208ee Move change note to new location 2022-01-11 12:23:16 +01:00
Michael Nebel
1d8f8f79bb C#: Add const interpolated string test case. 2022-01-11 12:02:07 +01:00
Michael Nebel
5b89f0e0b8 C#: Add example of const interpolated string. 2022-01-11 12:01:40 +01:00
Anders Schack-Mulligen
2a36744deb Merge pull request #7552 from smowton/smowton/fix/local-parameterized-classes
Note that parameterizations of local classes are themselves local
2022-01-11 09:36:15 +01:00
Alex Ford
b9ed8ed416 Merge pull request #7553 from github/revert-7498-dependabot/cargo/ruby/generator/clap-3.0
Ruby: Revert "Update clap requirement from 2.33 to 3.0 in /ruby/generator"
2022-01-10 19:36:40 +00:00
Alex Ford
17e5b9cffa Revert "Update clap requirement from 2.33 to 3.0 in /ruby/generator" 2022-01-10 18:21:04 +00:00
Chris Smowton
e352a4b994 Note that parameterizations of local classes are themselves local
Previously `LocalClass` itself would match `.isLocal()` whereas `LocalClass<Param>` would not. Rather than require each individual user to check for `.getSourceDeclaration().isLocal()`, let's note that the specializations themselves are local.
2022-01-10 18:19:31 +00:00
Tony Torralba
fbebf5e953 Move change note to new location 2022-01-10 17:27:02 +01:00
Tony Torralba
0e738622df Merge branch 'main' into atorralba/promote-log-injection 2022-01-10 17:24:25 +01:00
Tony Torralba
cc92ce2754 Fix QLDoc 2022-01-10 17:13:13 +01:00
Tony Torralba
e1e5e78464 Apply suggestions from code review
- Update CleartextStorage library to latest refactor
- Move change note to new location
2022-01-10 17:10:55 +01:00
Tony Torralba
d17e973b6b Apply suggestions from code review
Co-authored-by: Ethan Palm <56270045+ethanpalm@users.noreply.github.com>
2022-01-10 17:09:41 +01:00
Tony Torralba
ec8c234872 Fix predicate name 2022-01-10 17:09:41 +01:00