While we're here we're also adding a test for the `exportResultsToGist`
method, as there were no tests for the `export-results.ts` file.
We initially attempted to add the test to the pure-tests folder, but the
`export-results.ts` file imports some components from `vscode`, which
meant we needed to set up the test in an environment where VSCode
dependencies are available.
We chose to add the test to `vscode-tests/no-workspace` for convenience,
as there are already other unit tests there.
We've also had to import our own query and analysis result to be able
to work with data closer to reality for exported results.
Since we've introduced functionality to build a gist title, let's check
that the `exportResultsToGist` method will forward the correct title to
the GitHub Actions API.
Co-authored-by: Shati Patel <shati-patel@github.com>
We'd like to re-use this to test the `exportResultsToGist` method in
`export-results.ts`.
So let's move it to a shared folder in the `vscode-tests/no-workspace` folder.
Since there's no `helper.ts` file in this folder and to avoid any confusion with
the `helpers.test.ts` file, I've opted to put this shared method into `index.ts`.
Happy to be told there's a better pattern for this as it doesn't feel very nice!
Let's handle this case gracefully and skip displaying the number of repositories
when they're not available.
Similarly let's add a check to see if we should pluralize the `repository` noun
or not.
Co-authored-by: Shati Patel <shati-patel@github.com>
All exported MRVA gists are given the name `CodeQL variant analysis
results', which makes it hard to work out what it contains at a glance.
We're adding more information in the gist title to make it more useful.
Example of new title:
`Empty Block (Go) x results (y repositories)`
This translates to:
`<query name> (<query language>) <number of results> results (<number of repositories> repositories)`
Co-authored-by: Shati Patel <shati-patel@github.com>
We'd like to improve MRVA query gists by giving them more descriptive
titles that contain useful information about the query.
Let's add the number of query results to the title of the gist.
To do this we'll first need to count all the results provided to us in
the `analysisResults` array. There is an item in this array for each of
the repositories we've queried, so we're introducing a method to sum up
results for all the items in the array.
Co-authored-by: Shati Patel <shati-patel@github.com>
Previously, there was a bug where quick eval queries would crash when
the eval snippet is in a library file.
The problem was that the `codeql resolve queries` command fails when
passed a library file. The fix is to avoid passing the library file at
all. Instead, pass the directory. This is safe because the resolve
queries command only needs to know which query pack the file is
contained in. Passing in the parent directory is the same as passing in
a file in this particular case.
Currently `resolve ml-models` only supports queryspecs, i.e. .ql, .qls,
directory, and query pack specifications. Therefore quick evaluation within
a library isn't
supported.
In order to run our cli-integration tests, we're required to have a
local copy of the codeql CLI repo. We can then run the tests by running
the `Launch Integration Tests - With CLI` task from inside VS Code.
(See CONTRIBUTING.md for details.)
If we don't have the CLI repo cloned locally or we're not pointing to it
in `launch.json`, we don't get a clear indication of what the problem is.
The tests will still attempt to run.
Let's fail fast instead and add an actionable error message to the output.
The controller repo is set via the `codeQL.variantAnalysis.controllerRepo`
setting in VSCode.
While we have validation to check that the repo is not null and the
format of the controller repo is correct: `<owner>/<repo>`, we still
allow you to provide a non-existent repo (e.g. a mispelled one).
When the MRVA request is sent over to the API, it will verify that the
repo exists and return a very generic "Not Found" response.
This will then be logged out in the "Output" tab for VSCode.
We'd like to give users a better indication of what has gone wrong in
this case so we're making the error message more verbose.
Co-authored-by: Charis Kyriakou <charisk@github.com>
Co-authored-by: Shati Patel <shati-patel@github.com>
How did this ever work? It was using an old variant of the
qlpack name.
Also, this commit makes the unhandledRejection handler less
verbose. This gets hit when the tests end and there is a cancellation.
this is not an error.
