The data extension editor was only using the default data extensions
found in the `ql` submodule to find external API calls. This will add
support for using data extensions found in the workspace.
Rather than using the `codeQL.runningQueries.useExtensionPacks` setting,
this will always include data extensions since the editor doesn't make
sense to use without data extensions. We will also forbid the user from
opening this view unless they are using a CLI which supports data
extension packs.
We were using a single-use class for generating the flow model, while we
are actually able to do it using two functions. This is more in line
with our existing codebase.
This fixes the "Webview is disposed" error which occurs when the user
closes the variant analysis webview while the extension is still
activating. We will now check whether the webview is disposed before
restoring the view.
This was pointed out by CodeQL: when calling `setState` and using
`this.props`, it may not be up-to-date because `setState` may run
concurrently. Therefore, we should use the `setState` callback variant
to ensure we get the latest props.
This refactors the code a bit to ensure we're not using `this.props`
anywhere, including in the `getResultSets` function which is called
in the `setState` callback.
When updating to React 18, we removed the loading step from the updating
of the state of the result view since React would batch the updates
anyway. However, this caused a bug where the result view would be empty
when switching between queries. This is because the result view would
retain the old selected result set name. This would not happen
previously because React would re-render the view at least once, which
would cause the result view to be unmounted and re-created.
This fixes it by resetting the selected result set if we can't find the
result set in the new result sets.
The default Storybook Babel config did not recognize the `public`
keyword in our custom errors (e.g. `ExhaustivityCheckingError`
and `RedactableError`). To fix this, we can use Storybook's V7 mode to
supply a custom Babel config. This fixes the compilation error.
See: https://storybook.js.org/docs/react/configure/babel
This adds tests for the external API query and retrieving of results. It
does not use the "real" CLI integration, but instead mocks the CLI
server and query runner.
To make mocking easier and require less type casting, I've narrowed some
of the arguments of some other functions. They now use `Pick` to only
require the properties they need.
The version of ts-jest we were using was given a warning that it did
not support TypeScript 5, even though it was working fine. The latest
version of ts-jest adds official support for TypeScript and removes the
warning.
This loads in the existing data extension YAML file for the selected
database. It only supports the filename we save it to, and will not load
it from any other data extension YAML files.
This adds the ability to save the modeled methods in the data extensions
editor to a YAML file named after the database name. It will save it to
the `ql` submodule for now. Support for data extension packs will be
added later.
This updates the view of the data extensions editor to show a table of
possible sources/sinks/flow summaries that can be edited. It's not yet
possible to save the changes or load the existing file.
This upgrades Storybook to an alpha version to ensure we can start
Storybook when using TypeScript 5.0. This is temporary until we can
upgrade to Storybook 7 or to a released version of 6.5.17.
This adds a really simple regression unit test for the results view
which checks that the results view can render a SARIF file. This is
in preparation for the upgrade to React 18 to ensure that we don't
break the basic functionality of the results view.
This will improve the error message shown when monitoring fails. Instead
of showing "Error while monitoring variant analysis: Not Found", this
will now show "Error while monitoring variant analysis "Empty block
(javascript) [29/3/2023 10:45:10]: Not Found". This should make it
easier for the user to figure out which query history item is
problematic.
We're not using the full query history item label here because that
would require access to the query history item, which we don't yet have
here. Adding it here would add a dependency on the query history, which
seems undesirable.
We were ignoring errors coming from `vsce publish` and this was causing
the workflow to succeed even when the publish failed. This will remove
the `||` and let the workflow fail if the publish fails.
The `glob` package now uses promises in version 9, so we don't need the
separate `glob-promise` package anymore.
This also updates one call site to use `cwd` instead of `join` to
avoid possible issues due to a breaking change in version 8 which treats
Windows path separators differently. By changing the `cwd`, we should
not run into these issues.
Removing the `resolutions` of `glob-parent` did not change the
`package-lock.json`, so this does not have any effect on the package
version we are using.
We introduced this change to help with reducing flakiness in CI [1].
This has a slightly different effect locally, where every failed test
will output three times.
This in turn makes it harder to read, especially when you have multiple
failing tests.
Since the original intent for this behaviour was to be used in CI, I'm
proposing we disable it when the CI env variable isn't set.
I've opted to set it for all jobs involving tests, just for consistency.
I'm happy to limit it to just the places where it's required.
[1]: https://github.com/github/vscode-codeql/pull/2059
This adds support for detecting the `CommandManager.execute` method in
the unique command use query.
This may not be the best way to implement this. There's a method
`hasUnderlyingType` on `this.getReceiver().getType()`, but I couldn't
really figure out how to get it recognize `CommandManager`. It might be
possible if we can construct the type of `CommandManager`, but this will
probably include the filepath to the `CommandManager` class, which might
not neccessarily be something we want: moving the `CommandManager` class
should not require updating the query. I'm very happy to hear other
suggestions.
The `commandRunner` name doesn't really make sense since it doesn't
"run" a command, but rather registers a command. This renames it
to `registerCommandWithErrorHandling` and moves it to the
`common/vscode` directory.
The `codeQL.checkForUpdatesToCLI` command is registered pre-activation,
and we don't really want to create the command manager before
activation, so this will just add the correct type without registering
it using the command manager.
The command manager types didn't fully support commands defined with
`Partial` because it deduced that the command function was `undefined`
when the function was not defined. However, if the command is not
present, the command registration will not be called. This fixes the
types by specifying that the command definition will never be
`undefined`.
This class seems to have been introduced at some point to reduce the
dependency on VS Code from the test UI service. However, none of its
methods are being used anymore, and by using typed commands we have
already reduced the dependency on VS Code. Therefore, we can simply
remove this class.
We attempted to specify exactly which URI we're expecting here.
However, `Uri.parse` behaves differently in the test than it does in
the code so we've inadvertently created a flakey test [1]. The URI we
generate in the test has a `scheme: 'c'` while the one in the code has
a `scheme: 'C'` property.
This only happens on windows, not ubuntu.
Let's narrow the comparison to just the path of the URI.
[1]: https://github.com/github/vscode-codeql/actions/runs/4478429334/jobs/7871178529#step:7:231
Some of the methods in the `DatabaseUI` were public because they were
used in the `extension.ts` file. We have moved these method calls into
this file, so they do not need to be public anymore. We can also get rid
of the separation between some of these methods, so I've moved them into
the function that calls them.
Now that we fixed our expectation in the previous commit, we could see we
were stubbing this to false instead of true.
So now the test is checking the right scenario.
Our expectation was quite narrow: we expect to not call an `openFolder`
command. We didn't specify any params for it, which might mean this
expectation wasn't working like it should.
Let's just check that `executeCommand` isn't called at all.
The local query commands are using a separate logger, and this is not
supported by the command manager because it is quite specific to this
extension. Therefore, we create a separate command manager which uses
a different logger to separate the commands.
We're running this at the extension start-up. We don't want it to block the extension
from completing activation, so let's swallow any errors from the code tour and output
them, instead of letting this affect the rest of the extension activation.
When opening https://github.com/github/codespaces-codeql/ in a
codespace, it's easy to miss the prompt that tells you to open the
tutorial.code-workspace file.
In fact people actively dismiss the alert to get it out of the way.
If you miss that prompt, you end up with a single-rooted workspace,
which causes various other problems.
While there is an open issue to allow VS Code to open a default
workspace [1], there doesn't seem to have been any progress on it
in the last two years.
So we're taking matters into our own hands and forcing the extension
to open the tutorial workspace, if it detects it.
This will only happen if the following three conditions are met:
- the .tours folder exists
- the tutorial.code-workspace file exists
- the CODESPACES_TEMPLATE setting hasn't been set
NB: the `CODESPACES_TEMPLATE` setting can only be found if the
tutorial.code-workspace has already been opened. So it's a good
indicator that we're in the folder, but the user has ignored the prompt.
[1]: https://github.com/microsoft/vscode-remote-release/issues/3665
The local databases UI was essentially the only class which was defining
methods using assignment to a class property rather than using function
definitions and binding them. This switches it to use function
definitions and binding, which is more consistent with the rest of the
codebase.
This change allows the `codeQL.runningQueries.useExtensionPacks`
setting to be respected when running variant analysis queries. When
set to `all`, before uploading the generated query pack, all extension
packs in the workspace will be injected as dependencies into the qlpack
file.
In the results view, `setState` was used to set some state, and then
`loadResults` was called to set some other state. However, `setState`
is asynchronous, so the `this.state` in `loadResults` was not the state
that was set before the call.
This commit fixes it by combining the two `setState` calls into one. The
logic was hard to follow, so I'm not sure if this is the correct fix.
The `shouldKeepOldResultsWhileRendering` state seems to be unnecessary
now since everything is being done in `setState` call, but I'm not sure
about that.
The version of `@primer/react` we were using didn't have the correct
types for the `ThemeProvider`. By upgrading these packages to their
latest versions, all types are correct.
This enables React strict mode which will print extra warnings to the
console when we use certain constructs incorrectly. This does not affect
production builds.
See https://beta.reactjs.org/reference/react/StrictMode
The extension Webpack config does not use `babel-loader`, so we can
remove it as a direct dependency. `babel-loader` is still included in
our `node_modules` because Storybook depends on it, but the version is
now completely managed by Storybook rather than us.
This will make the progress options passed to `withProgress` optional by
moving it to be the second argument and setting a default value for the
`location`. This will make it much easier to use from a variety of
commands.
This removes the `args` from the `ProgressTask` passed to
`withProgress`. The `args` is only used by the
`commandRunnerWithProgress` and can easily be replaced by an anonymous
function that passes the `args` instead. This will simplify the
`ProgressTask` interface and make it easier to use.
The `commandRunnerWithProgress` implementation isn't actually any
different from `commandRunner`, except for the call to `withProgress`
and support for an `outputLogger` argument. Therefore, this will simply
make `commandRunnerWithProgress` a wrapper around `commandRunner`,
removing quite some duplication in the process.
If the user requests that extension packs be included in their MRVA run,
then do the following:
1. Search the workspace for all extension packs
2. Add each extension pack as an explicit and direct dependency on
the generated pack.
It is ok to use `*` as a dependency since we are guaranteed that
exactly one version of each injected extension pack dependency is
available when the pack is being compiled.
If we find multiple paths to an extension pack of the same name, this
is an error since it is ambiguous which path to use.
This will add a check to ensure that `showAndLogExceptionWithTelemetry`
is not called when downloading packs. This expectation is placed before
the check for `showAndLogInformationMessage` so that when the test
fails, the error message will be shown.
We are no longer including our dependencies in the VSIX package, so we
can tell VSCE that we don't want it to look at dependencies using
`--no-dependencies`. If we do this, VSCE doesn't require the
`node_modules` directory anymore and we can skip that step, which will
make building significantly faster.
I've confirmed that there are no changes between the two options by
building the extension both without and with the change. This is the
diff of the two outputs (using `diff -r`):
```diff
diff --color -r vscode-codeql-1.8.0-dev.2023.3.8.15.10.13/extension/package.json vscode-codeql-old/extension/package.json
7c7
< "version": "1.8.0-dev.2023.3.8.15.10.13",
---
> "version": "1.8.0-dev.2023.3.8.15.6.51",
diff --color -r vscode-codeql-1.8.0-dev.2023.3.8.15.10.13/extension.vsixmanifest vscode-codeql-old/extension.vsixmanifest
4c4
< <Identity Language="en-US" Id="vscode-codeql" Version="1.8.0-dev.2023.3.8.15.10.13" Publisher="GitHub" />
---
> <Identity Language="en-US" Id="vscode-codeql" Version="1.8.0-dev.2023.3.8.15.6.51" Publisher="GitHub" />
```
The only difference is the version number, which is expected.
When deleting a query history item and the "next" query is still
running, the `completedQuery` is `undefined`. This commit fixes it by
using optional chaining to ensure that the `completedQuery` is defined
before accessing its `successful` property.
This is definitely not a perfect solution since we're essentially just
moving the place where we're casting. However, because we have manually
made the types similar, this provides some type assurances where there
were none before. This also has the cast in only one place, which makes
it easier to find and fix in the future.
I was initially trying to understand why this method was failing due
to an unrelated error [1] so I ended up over-engineering the path
parsing.
We can use the path from the first workspace folder, like we do in
other places in the extension.
[1]: https://github.com/github/vscode-codeql/pull/2104
When a user goes through the Code Tour, we select a dummy `csv` database
for them to get them up and running.
Once they complete the code tour and would like to continue writing
queries, they will need to add their own database.
After they do that, we check the language of their new database and
generate a skeleton QL pack for them so that they don't need to create
these files by hand. See [1] for details.
This skeleton pack folder will be called
`codeql-custom-queries-<language>` and it comes with its own example
query: `example.ql`.
When we try to run this example query, the query gets confused about
which `dbscheme` to pick, as it sees a `qlpack.yml` file in the new
skeleton pack folder, as well as one in the existing `tutorial-lib`
folder.
So we'll need to get rid of the `tutorial-lib` folder in order to make room
for new queries to be run once the tour is complete.
This commit introduces a `handleTourDependencies` step which will
trigger a `codeql pack install` command in order to install real library
dependencies for `tutorial-queries`, since we no longer have the dummy
library in `tutorial-lib`.
Unfortunately `Object.defineProperty` doesn't work on proxies, so I've
added an options object to `mockedObject` which allows passing in
methods that will return a value for a specific property.
In https://github.com/github/codespaces-codeql/pull/12 we moved
the source for our tutorial database into `.tours` in order to
avoid confusing the user when we load the database into the
workspace, since they'd see two databases.
Since this is just the source, we'd like to hide it.
We mark this query as `@kind problem`.
We'll need to change the query a bit to make it fit this type of ...
erm... kind.
This means the results view will be formatted to display the file name
next to each of the results.
We're also getting rid of any mentions of an empty block query since
that's no longer what it checks.
This will remove some instances where we're using `as unknown as T` and
replace them by a call to `mockedObject<T>()`. The `mockedObject`
function is a bit more explicit about what it does and has types which
ensure that the methods that are set on the object actually exist.
Unfortunately, we can't fully get rid of `as unknown as T` in the
`mockedObject` function. However, this construct is more localized and
does not need to be used in as many places. If we do enable an ESLint
rule to prevent the use of `as unknown as T`, I would feel comfortable
with disabling the rule for the `mockedObject` function.
This also removes the type for the `runTests` `options` argument since
it's only used in this definition and don't actually use the type
anywhere else.
It seems that when we added the CSP policy to the webview, we did not
take into account that `d3-graphviz` uses `@hpcc-js/wasm` to load
Graphviz as a WASM module. This commit adds `'wasm-unsafe-eval'` to the
CSP policy to allow this.
This is step 3 in the Code Tour. At this point we don't need to create
the skeleton pack so let's disable that functionality.
Co-authored-by: Shati Patel <shati-patel@github.com>
By default, this is added when we call `runJsonCodeQlCliCommandWithAuthentication`.
However, `codeql pack add` doesn't support this option so we need to turn it off.
Co-authored-by: Shati Patel <shati-patel@github.com>
On windows, the `Uri.path` will return an extra folder, as we can see in the tests:
```
ENOENT: no such file or directory, open 'D:\C:\Users\RUNNER~1\AppData\Local\Temp\tmp-4784XPDQPb5jM6IW\test-ql-pack-ruby\qlpack.yml'
```
Let's use `Uri.fsPath` instead.
We were initially always removing the last folder in the workspace as
we assumed that would be the directory we use.
Now that we've switched to using a temporary directory, this is no longer
the case so we need to find the index of the directory in the list of
workspace folders and then use that index to remove the directory.
`Uri.parse` will not work with Windows paths as it will consider `C:\path`
to indicate a file scheme (the "C:" part) and will complain about it.
With `Uri.file` we can build the URI without hitting this complication.
We're checking that the skeleton QL pack doesn't exist as a workspace
folder, so we should be creating this folder in the workspace as well.
Initially this was being created in VSCode's local storage.
This will receive a folder name and language.
It will generate:
- a `codeql-pack.yml` file
- an `example.ql` file
- a `codeql-pack.lock.yml` file
It will also install dependencies listed in `codeql-pack.lock.yml`
file.
We were initially planning to call the `packInstall` command once
we generate `codeql-pack.yml` in order to install dependencies.
However, the `packAdd` command does this for us, as well as
generating a lock file.
Rather than trying to craft the lock file by hand, we're opting
to use the cli command.
NB: We're introducing a new `QueryLanguage` type which is identical
to the `VariantAnalysisQueryLanguage`. In a subsequent PR we'll
unify these two types.
Similar to what we do with `codeql pack install`.
Tnis will simulate us running `codeql pack add codeql/<language>-all`.
We're going to need in order to:
- generate a lock file (codeql-pack.lock.yaml)
- install the correct packages for our skeleton QL pack based on the
lock file.
This removes the remote queries history item as a supported history
item. This allows us to delete almost all code related to remote
queries except for the React view code which will be removed separately.
In the query serialization code, we now ignore remote queries.
This would run the unit, view, integration and CLI integration tests in
parallel, which would cause problems with multiple VSCode instances and
use a lot of memory.
- Add a new config that toggles between using all data extensions or
none.
- If using all data extensions, then before a query evaluation, run a
`codeql resolve qlpacks` command with the new `--kind` option. This
will return a list of extension packs in the workspace.
- Pass these packs to the cli before evaluation/
- This will only work with CLI v2.12.3 (not yet released) or later.
- Also include some CLI tests to ensure this works.
This will add support for the `codeql-pack.yml` filename in all places
where we currently support `qlpack.yml`. It centralizes the list of the
supported filenames in a single place and a method that can figure out
the correct filename to use.
This download download the sourcemaps from the release asset if it is
available. Unfortunately, I'm not able to test this yet because we don't
yet have any releases with sourcemaps attached. As a fallback, it will
still try to download from the workflow run.
Most of the warnings that are currently being shown in CI are coming
from VSCode. We can hide the VSCode output to make the CI logs more
readable. This should not influence the tests; the output of tests
(in particular using `console.log`/`console.error` etc.) will still be
shown.
See: 0c98dc12ad/packages/jest-runner-vscode/src/public-types.ts (L58-L68)
We'd like to add test coverage for the openDatabase function (which is
public).
At the moment, this relies on `resolveDatabaseContents` which is just
a standalone function.
This means we're unable to mock it using Jest.
So let's move it into its own class.
This method in turn depends on a `resolveDatabase` function, which we've
also moved into the new class.
The only usages I could find for there functions were from within
the `databases.ts` file.
This is unrelated to the changes in this PR but it's causing CI to fail.
```
config listeners › CliConfigListener › should listen for changes to 'codeQL.runningTests.numberOfThreads'
expect(jest.fn()).toHaveBeenCalledTimes(expected)
Expected number of calls: 1
Received number of calls: 2
109 | const newValue = listener[setting.property as keyof typeof listener];
110 | expect(newValue).toEqual(setting.values[1]);
> 111 | expect(onDidChangeConfiguration).toHaveBeenCalledTimes(1);
| ^
112 | });
113 | });
114 | });
```
We don't need to check that the callback is triggered a certain number of times, just that it works
so we can change this test to be more permissive.
We'd like to make it easier for a user going through the CodeQL Tour to
write their queries.
To help them along, we can generate skeleton QL packs once we know which
database they're using, instead of expecting them to know how to create
this themselves.
We're then able to download the necessary dependencies for their CodeQL
queries.
This checks that we're running the CodeTour by looking for the
`codeQL.codespacesTemplate` setting.
This will upload the sourcemaps produced as part of the release as a
release asset. This allows the sourcemaps to be downloaded and used
for decoding stack traces beyond the 90 day artifact limit of GitHub
Actions.
We run `npm run lint` every time we do a `git push`.
This takes quite a long time, and the lint command has already been run
when we created the commit in the first place.
Could we instead skip this and rely on CI to tell us if we've failed
to address a linting issue?
It seems like the `onDidChangeConfiguration` is being called multiple
times. It doesn't actually matter that it's being called twice, so we
just need to ensure it's called at least once.
This is blocking us from merging new PRs so while we figure out
how to fix them, let's skip the tests that are failing on our
`main` branch.
For full context: the tests started failing when a new version of
VSCode was released (1.75.0).
This adds support for mapping full stacktraces in the source map
script. This allows you to pass a full stacktrace to the script and get
back a stacktrace with all original positions.
This adds a script that can be used for retrieving the original source
location when given a location in the released extension. It will
download the source map from the Actions workflow run of the release and
use the `source-map` library to extract the original location.
In our tests, we were writing settings files to disk because we were
using the VSCode configuration API which writes settings to files. This
results in flaky tests because concurrency can cause the VSCode API to
misbehave.
This will switch the tests to use a mocked API by default. For some
tests the real implementation is used, but the large majority of tests
is now using a mocked version which only keeps track of the
configuration in memory. This makes it easier to reset the state between
tests since we can just empty out the in-memory configuration.
We have a codespace template which houses our CodeQL tour:
https://github.com/github/codespaces-codeql
This contains a repo with a default databases already loaded
for the user so that they can start writing queries more quickly.
At the moment we're asking the user to manually right click on
the database folder ('codeql-tutorial-database') and set it as
the current database.
We can take this one step further by defining a command that gets
triggered when we arrive at the step for setting up the database.
The command ("codeQL.setDefaultTourDatabase") will build the URI
pointing to our preloaded database and set it as the current one.
We initially considered whether we can re-use the setCurrentDatabase
command and pass the URI of the database from the codespace itself,
but the URI would be hardcoded as:
```
file://0-62/workspaces/codespaces-codeql/codeql-tutorial-database
```
as we can only pass the codeTour extension a command and string
parameters.
This would have been brittle as the filepath for a codespace might
change in the future.
Instead we can define a custom tour command ("setDefaultTourDatabase")
to look at the current workspace folder and build the path to the
database in the CodeQL extension.
Co-authored-by: Shati Patel <shati-patel@github.com>
Instead of using the third party `peter-evans/create-pull-request`
action for creating a PR for releases, we can use the already present
`./.github/actions/create-pr` action which is also used for the PR for
bumping CLI versions.
Instead of having different ESLint configuration files in each
directory which don't seem to inherit the configuration correctly, this
will add `overrides` in the root file.
This will add a welcome view to the database panel which is shown when
the controller repository is not setup. This welcome view will show a
button which can be used to set up the controller repository.
3 seconds is a really long time to wait for downloads since a
significant percentage of downloads will complete within 3 seconds. This
changes the update delay to 500ms which should still give us good
performance, but also make the download feel more responsive.
This removes all CodeQL CLI version constraints for unsupported CLI
versions (< 2.7.6). The oldest supported CLI version is 2.7.6 since GHES
3.3 recommends using CodeQL CLI 2.7.6.
Tests whether we choose "Yes" / "No" in the new modal window.
"Yes" -> remove the item and show you a toast notification
"No" -> don't remove item
This only shows up for in progress items.
We now have special behaviour for removing an "in progress" query so the
tests will be different.
Let's have a separate section for "in progress" queries. We'll add extra
behaviour testing in the next commit.
When you attempt to delete a query that's still in progress from your
query history, you get a prompt asking if you're sure.
If you pick "Yes", the item is removed and you see a toast notification
with a link towards the GitHub Action.
This will supply the GitHub access token to certain CodeQL CLI commands
such that private packages can be resolved. It will only do so if the
user has an existing auth session. If they don't, they will now get a
prompt to login. However, this will only happen for commands which
actually use authentication, which is limited to packaging commands.
This will set the `mode` of Webpack to `production` for release builds.
It will also stop inlining the sourcemap and instead produce a separate
file which is excluded by `.vscodeignore`.
In terms of the bundled extension, this will add 1 file
(`out/webview.js.LICENSE.txt`). It decreases the size of the VSIX file
from 4.28MB to 1.77MB.
VSCode was not able to find the original source of the bundled
extension because it was looking for the source in the `out` directory.
By setting the `sourceRoot` to the `extensions/ql-vscode` directory
which is located at `..` from the `out` directory, VSCode is able to
find the original source and breakpoints are hit.
This will copy the WASM file from source-map to the output directory.
This makes the source-map package work. See the comment in the code for
more details.
This bundles the extension files using esbuild, as recommended by
VSCode. This reduces the size of the extension from 34MB to less than
5MB.
Gulp will still run TypeScript to check types, but will not use the
TypeScript compiler output in the bundle.
Tests are now run separately, outside of Gulp, so their data doesn't
need to be copied anymore.
See: https://code.visualstudio.com/api/working-with-extensions/bundling-extension
Some checkouts of the github/codeql repo, such as the
internal submodule, may be named `ql` rather than
`codeql`. Allow this folder name when running tests.
I am removing these assertions so that our internal integration tests
can pass. They are currently failing because the number of dependencies
of the `codeql/javascript-all` pack has changed. It no longer makes
sense to test this value as newer versions of this pack will have more
dependencies and we expect this value will continue to go up.
This was initially added [here][1] but wasn't quite in the right place
to have the intended effect.
Let's move it up to the root of the project.
[1]: f515663640
This moves our existing test plan under a "Required testing" section.
We're also adding the scenarios used for testing live results under an "Optional testing" section.
I believe this doesn't change the user-visible behaviour at all. The user
won't be prompted to log in any more or less often than they would have
done before.
One benefit of this is that we can remove the registerListeners method
because we no longer need to know if the cached octokit is still valid.
Instead we just call vscode.authentication.getSession every time and it
will return the current session, which might be different from the last
time we called it. This might prompt the user to log in, but that would
have happened anyway because when the session changed we would have
overwritten our cached octokit instance.
Another benefit is that we no longer need the extension context and this
removed a surprisingly large amount of code where we are passing this
parameter around because we need it for the credentials.
The only downside I can see is that we call getSession more often and
create more javascript objects in general. I believe the performance
impact of this will be negligible and not worth worrying about.
I argue that calling createOctokit(false) adds no benefit. If an
authenticated session already exists then this silently create an
octokit, which makes getOctokit() a no-op just returning the field.
However if there is already an authenticated session then getOctokit()
would already be able to create an octokit without prompting the user.
On the other hand if there isn't an authenticated session then we
won't be able to pre-populate an octokit, so getOctokit() will have
to prompt the user anyway.
Not calling createOctokit(false) in registerListeners also doesn't
change behaviour. If the user is authenticated in the new session then
we would be able to create an octokit instance wihtout prompting in
getOctokit anyway. If the user is not authenticated in the new session
then we won't be able to create an instance without prompting either way.
The only benefit I can think of is that it moves a tiny amount of
computation earlier in the pipeline, but the amount of computation is
tiny and it isn't any more async than it would be if it happened in
getOctokit(). I don't think this is worth making the code more complex.
This was only used from initializeWithToken and only added a completely
separate case to the start of the method, effectively turning it into
two separate implementations. Therefore we can make things simpler by
inlining this case in the one place it is used.
It is true by default and no place in the codebase sets it to false. We can
simplify the code by removing this case we aren't using. If we want this
behaviour in the future we can always implement it again, but I think it's
likely to be unnecessary and if you don't want authenticated requests then
you likely won't be initializing a Credentials object.
This cannot happen already, even before the other changes in this PR.
The Credentials.initialize method can never return undefined, so these
checks would never return true. The real place that checks that we are
authenticated is in the vscode.authentication.getSession method, and
it will reject the promise if the user declines to authenticate or
anything else means we can't get an authenticated session.
I feel justified in removing the tests for these cases because the
code was never actually called in production, and we are covered by the
vscode authentication library rejecting the promise. Any exception
thrown from Credentials.initialize would behave the same as the one I'm
deleting.
This will sort the files in an exported Gist by the user-defined sort
order. It does so by prefixing the files with `result-{index}-` where
the `index` is the 1-based index of the repository in the sort order.
It will automatically pad the index with leading zeros to ensure that
the files are sorted in the correct order.
Unfortunately, we can't just use `{index}-` because numbers sort before
the `_` character, which is used in the summary filename to place it
first.
There are also some changes in how we determine which repositories to
export since we need to know in advance how many zeroes we need to pad
the index with. There should be no functional changes in which
repositories are actually exported.
The `tsconfig.json` inside `gulpfile.ts` needs to match the root
`tsconfig.json`, so by making it extend the root `tsconfig.json` and
changing just the options which decide which files are included, we can
remove a lot of duplication.
Instead of deleting the complete `_VSCODE_NODE_MODULES` object, we now
use a `Proxy` to intercept the `_isMockFunction` property. This is safer
and will not delete a global variable that VSCode expects to exist.
This fixes the tests on VSCode 1.74.0. The issue is as follows:
1. Jest wil try reset all mocks after each test, as it should.
2. When Jest does this, it will loop over all global variables and check
if they are mocks.
3. One of the global variables it checks is _VSCODE_NODE_MODULES, which
is a proxy object.
4. When Jest checks whether it is a proxy by getting _isMockFunction on
it, the `get` function on the proxy object will be called.
5. This will in turn call require, which will try to load
the non-existing `_isMockFunction` module. This throws the error we are
seeing.
By removing the `_VSCODE_NODE_MODULES` property from the global object
in the Jest environment, Jest will not try to reset it, and the tests
should work again.
See: 41bf230089/packages/jest-runtime/src/index.ts (L1173-L1186)
See: ed442a9e99/src/bootstrap-amd.js (L15)
This commit adds a new step to the CI workflow that runs type checking
on all directories containing `tsconfig.json` files, using `find` and
`xargs`. Unfortunately, this does not work on Windows, so on Windows
it's not possible to run all of these type checks locally.
This adds the environment variables necessary for running the date test
in all of these cases:
- When running the npm script outside of VSCode (using `cross-env`)
- When using the Jest Runner "Run" option (`terminal.integrated.env.*`)
- When using the Jest Runner "Debug" option
This integration test will check that the monitor will actually make
multiple requests to the API and that it will trigger a download
extension command for each repo that has finished scanning.
Unfortunately, one of the tests we have for local queries doesn't seem
to be working for variant analyses. I'm not sure why it isn't
working, but I think it's better to get the rest of the integration
tests in and then figure out what's going on with that one.
When rehydrating remote queries, we were awaiting the monitoring
command. Since this command may take minutes to hours to complete, it
seems like this would block the extension from loading. This is the same
issue as in https://github.com/github/vscode-codeql/pull/1698, but for
remote queries instead of variant analyses.
The repository selection was structured such that you would get in the
`else` case if there was nothing selected, but this case would also be
used if for some other reason the selected item was not valid.
This restructures the conditions to first check whether the user
cancelled out of the operation and will silently return in that case. In
other cases where it cannot determine the repositories, it will now show
a user-visible error.
This will hide the "Analyzed" panel when there are no scanned repos and
it's completely empty.
When all three panels are empty, this will also hide the search bar and
filters, and will skip rendering anything for the panels.
This restructures the variant analysis manager tests to follow this
pattern:
- Class
- Method
- Context
- Context
- ...
- Test
Before, we were only using this pattern for some of the tests and this
made it confusing were which method was being tested.
By splitting this off, it will also be easier to move some of these
tests out of the cli-integration tests and into the no-workspace or
minimal-workspace tests.
This will add a spinner to each repo row when the results for a
particular repo are loading. It will also disable the row to make clear
that it is loading and not clickable.
This adds a new filtering on SARIF code snippets for very large code
snippets (defined as 8MB or more). If less than 1% of such a snippet
is highlighted, it will not include the code snippet in the analysed
results, and it will thus not be shown in the UI.
This is to avoid very large SARIF files that can cause the extension
host to crash when the analysis results are send to the UI. I don't
think any of these snippets would ever be useful to show, so it should
be fine to just not include them.
The extension doesn't actually use anything regarding the language of
variant analyses, so this just updates some types.
The actual Swift support is done in the CLI, which is also used for
determining which languages are actually supported. So, the environment
variable is already used by the CLI for showing supported languages.
We were using two different implementations for opening the query file
and query text between the query history and the results view. This
moves the better implementation in the view to a command and uses these
commands for opening the query text/file in the query history and view.
This results in consistent error messages and behaviour between the two
different views.
This will add error handling to the retrieval of variant analyses in the
monitor by catching the error. It will show a warning to the user and
log it. Then, it will simply sleep for 5 seconds and try again.
I'm not sure if we want to show all of these errors to the user since
this can result in many warnings popping up if many variant analyses are
being monitored, but this is probably that the user should be made aware
of.
This will add a progress notification to exporting results to give users
feedback about what's happening.
Unfortunately, we need to change some things in how we handle the
actions on completion notifications since we want the progress
notification to disappear when that notification shows. This results in
us having to remove the `await` on the
`showInformationMessageWithAction` calls.
The Gist title in the result export didn't take into account the actual
number of exported repositories, it only used the scanned, unfiltered,
repositories in the variant analysis. This switches it to use the actual
exported repositories for determining the result and repository counts.
This is somewhat more complicated than we'd expect it to be since the
results are being read in async, so we need to switch the order of
operations and store some additional information for being able to
compute this information. However, this also makes the code somewhat
easier to understand since the summary file is now being created in only
1 location, rather than being split between a method and a for-loop.
When deserializing a webview, it could happen that a view was already
manually opened by the user before the webview was deserialized. This
would result in duplicate webview tabs, which is not supported by the
manager.
This will close the webview that is being deserialized and focus on the
existing view. This should ensure that we never have duplicate loaded
webview tabs. There could still be duplicate webview tabs if there are
non-deserialized tabs, but once it is opened it should be closed
automatically.
This will disable the export and copy buttons when no results would be
exported by executing the command. In contrast to the "normal" filtering
in the view, this will also take into account the checkboxes since those
are also used in the extension host.
This will allow exporting results for a variant analysis which is
cancelled or in-progress. Repositories for which the results are not yet
available or which have not yet been downloaded will not be exported.
The header of the summary file is incorrect, but this will be fixed in
a follow-up PR.
During the build process, we now get errors pointing out that we import two different
items and call them `QueryServerClient`. One is actually the legacy one and has been
superseded by the new one so far.
Let's fix this and appease the linter.
Full errors:
```
[gulp-typescript]
/home/runner/work/vscode-codeql/vscode-codeql/extensions/ql-vscode/src/extension.ts(88,9):
error TS2300: Duplicate identifier 'QueryServerClient'.
[gulp-typescript]
/home/runner/work/vscode-codeql/vscode-codeql/extensions/ql-vscode/src/extension.ts(89,9):
error TS2300: Duplicate identifier 'QueryServerClient'.
[gulp-typescript]
/home/runner/work/vscode-codeql/vscode-codeql/extensions/ql-vscode/src/extension.ts(1596,30):
error TS2345: Argument of type
'import("/home/runner/work/vscode-codeql/vscode-codeql/extensions/ql-vscode/src/legacy-query-server/queryserver-client").QueryServerClient'
is not assignable to parameter of type
'import("/home/runner/work/vscode-codeql/vscode-codeql/extensions/ql-vscode/src/query-server/queryserver-client").QueryServerClient'.
```
In the next commits we'll turn these rules on, one-by-one, and then
autofix the offenses.
At the end we'll be left with the rules that require manual attention.
This adds tests for the `loadResults` method of the variant analysis
results manager. It tests that SARIF results can be successfully
loaded and that the `onResultLoaded` event is fired.
The monitor return value was only used in tests, but we can also assert
the correct behavior using the calls it makes, rather than using the
result of the monitor.
The header color of a stat item was using the badge foreground color,
but badges can have a different background color than the editor. For
some themes, this would result in unreadable text. By using the editor
foreground color, the header should be readable in many more themes.
This adds four new VSCode themes to Storybook which will allow us to
more easily test these themes in Storybook. These themes were chosen
because they are either used for accessibility (the high contrast
themes) or are currently not compatible with the variant analysis UI
(there are items that are not visible).
When results were already cached in memory and the view requested the
result, it would not be loaded because the event would not be fired.
This fires the event when the result is loaded from cache as well, to
ensure that the view always receives the result.
This will stop the variant analysis monitor from monitoring when a
variant analysis is removed from the query history. Since the variant
analysis monitor cannot depend on the variant analysis manager (this
would create a circular dependency), a function is passed into the
variant analysis monitor for checking whether the variant analysis
should be cancelled.
This commit will also ensure that even if a variant analysis comes in
through the `onVariantAnalysisChange` callback, it won't be added to
the variant analysis map of the manager.
On Windows, we were showing the full path to the query, rather than just
the filename. This is because the `path` package being imported was
actually `path-browserify` which only claims support for POSIX. Since
Windows uses backslashes rather than forward slashes for paths, this
resulted in the full path being shown.
This creates a new `basename` function that works on both POSIX and
Windows by detecting whether a POSIX or Windows path is given. This
ensures that the correct path is shown on Windows, and will also ensure
that we show the correct path on Linux if the user has opened a variant
analysis that was originally created on Windows.
This will update the `jest-runner-vscode` patch to retry tests that fail
due to no test result being returned from the test runner.
This will also add some retries to the `minimal-workspace` and
`no-workspace` tests to help with flakiness.
Multiple VSCode instances were being launched when a second instance of
VSCode was being spawned with the same user data directory. This is
probably because VSCode restores the windows from the previous session,
even when `-n`/`--new-window` is passed.
This fixes it by patching `jest-runner-vscode` to always create a new
temporary user data directory, rather than re-using the same one for
all test suites.
This will patch `jest-runner-vscode` to retry tests. This is a temporary
test to see if this will help with the flakiness of the CLI integration
tests.
The biggest problem with this is that it will launch multiple VSCode
instances on every retry:
- First try (not a retry): 1 instance
- Second try: 2 instances
- Third try: 3 instances
- etc.
I'm not sure why this is happening and can't really narrow it down to a
specific cause. Even if I change the `runVSCode` call for the retry by
a simple `cp.spawn` call, it still launches multiple instances.
This adds debugging support to jest-runner for the integration tests
when they are run from the `out` directory. Unfortunately, this removes
the ability to debug the non-integration tests, such as the pure tests.
Instead of running the integration tests from the `out` directory, this
will run the integration tests from the `src` directory using `ts-jest`.
Unfortunately, we are not able to use TypeScript files for the
`jest-runner-vscode` configuration since `cosmiconfig` (the package that
handles the configuration loading for `jest-runner-vscode`) doesn't
support loading TypeScript files by default.
Since we are launching a completely different process for the extension
tests than the process that is launched by VSCode, we need to add some
special handling for the debugging.
This will let the extension host/VSCode expose a debugging port, which
VSCode will then connect to. This is "less desirable than letting the
bootloader do its thing", but a packaged VSCode application does not
allow using the bootloader (`NODE_OPTIONS`=`--require=...`). Therefore,
we have to fallback to this option.
See: 47c60558ec/src/configuration.ts (L405-L411)
Apparently, we're not importing the same `config` file as is used by the
actual extension, so mocking methods in this file does not do anything.
However, we can mock the `vscode` module, so we can use this for
returning different values in the configuration.
We also need to mock the authentication session since we don't have one.
This will ensure all mocks are restored after every test. This required
a significant amount of changes in the tests since `jest.spyOn` now
needs to be called in `beforeEach`, rather than in the `describe` block.
There were some things that were breaking due to version checks. Since
we aren't testing on these CLI versions (2.7.2 and 2.7.4 or older)
anymore, we can remove these checks and simplify the tests.
For `to.contain`, `jest-codemods` seems to have converted these to be
`expect.arrayContaining`, even for strings. This will make the correct
change for strings.
Jest does not support skipping tests when the test has already started
(which could also be in a before hook), so we need to manually return
from the tests when the CLI version does not support a tested feature.
Instead of calling `fail`, we can just let the error be caught by Jest,
which will automatically fail the tests. For other instances where we're
calling `fail` in case an error was not thrown, we will instead use
`.rejects.toThrow`.
This is a first pass on converting the cli-integration tests to Jest. It
uses a custom Jest runner (based on the jest-runner-vscode) to install
required extensions. It will also necessary to move some code for
installating the CLI to ensure it was possible to install the CLI from
outside VSCode.
Most of the conversion has been done by `npx jest-codemods`, but all
migration of Sinon has been done manually.
The filter and sort tests were located inside the React tests since they
were already using Jest. Now that the pure tests have been switched to
Jest, these tests can finally be moved to the "normal" pure tests.
The timeouts need to be set either on a per-file basis, or per test by
using the parameter in `it`. Since we have both Mocha and Jest types, we
need to declare in the test file which one we're using.
This migrates all no-workspace VSCode integration tests to Jest by
running `npx jest-codemods`, followed by manual fixes (mostly for Sinon
and Proxyquire).
When `jest-codemods` was run, it replaced the error message `array.join`
by a comment for the error message. Since Jest does not support custom
error messages out-of-the-box, this will instead do an equality check
with an empty array, which will ensure that the received array is
printed.
The config store was not being disposed in tests, resulting in Chokidar
watchers being left open. This was causing tests to not exit since there
were still open file descriptors.
This commit also fixes the `DbConfigStore` to make the correct `super`
call in its `dispose` method.
This converts all pure tests to Jest. This was done by first running
`npx jest-codemods` with the Mocha transformation, then manually fixing
any places where it hadn't automatically converted the correct thing
or had missed things (mostly Sinon).
This also sets up VSCode correctly for running Jest.
This moves the view Jest config to the view directory and adds a new
jest.config.js file which only references the view directory as a
project. This will make it easier to add multiple Jest configs for
separate projects.
`${workspace}` references are new in CLI version 2.11.3. These mean that
the version depended upon in a pack must be the version available in the
current codeql workspace.
When generating a variant analysis pack, however, we copy the target
query and generate a synthetic pack with the original dependencies.
This breaks workspace references since the synthetic pack is no longer
in the same workspace.
A simple workaround is to replace `${workspace}` with `*` references.
This adds Prettier and makes it replace tsfmt. VSCode is set to use
Prettier for formatting TypeScript/TSX files and format on save since
Prettier is very fast and does not cause any noticeable delay.
This adds a new options argument to the `loadResults` method which
allows the caller to specify that the results should not be saved to the
cache. This exposes a smaller API surface and makes it harder to misuse
the methods.
Each variant analysis export can be different due to different filters,
so there are two options:
- We need to clean up the directory before each export to ensure no old
files are left
- We need to use a separate directory for each export
This implements the second option, which is more flexible and allows the
user to retain different result exports.
This adds filtering (based on search and selected repositories) and
sorting to exporting results. This is done in the same way as for
copying the repository list, so the changes are fairly minimal.
This will use the selected repositories to limit which repositories are
included in the copied repo list. If there are both selected
repositories and a search filter (on the full name), the search filter
will be ignored and the selected repositories will be used in full.
This will pass the filter and sort parameters in the export repo list
message so it can be used by the command to filter and sort the
repositories which are placed in the repo list.
These functions can be re-used by the sorting and filtering code for
exporting results and copying repository lists, so these should not be
in the view directory.
The tests have been kept in the same place for now, but they should be
moved to the pure tests directory once those have been switched to Jest.
I figured it wasn't worth it to convert these to Mocha, and convert them
back to Jest in a week.
This will add a new `useState` call on the top-level to keep track of
the checkbox state. It will allow all downloaded repositories to be
selected. This will allow us to make the copy repository list and export
results button dependent on the selected repositories.
This moves some of the code that is specific to remote queries out of
the `run-remote-query.ts` file and instead places it in separate files
that only deal with remote queries, rather than also dealing with
variant analyses.
The `runRemoteQuery` and `runVariantAnalysis` were returning values
which were only used in tests. This removes them and replaces the tests
by expectations on the commands called by the methods.
This adds the export of variant analysis results. This is unfortunately
a larger change than I would have liked because there are many
differences in the types and I think further unification of the code
might make it less clear and would actually make this code harder to
read when the remote queries code is removed.
In general, the idea for the export of a variant analysis follows the
same process as the export of remote queries, with the difference being
that variant analysis results are loaded on-the-fly from dis, rather
than only loading from memory. This means it should use less memory, but
it also means that the export is slower.
There was only a single command for exporting variant analysis results,
which would either export the selected result or a given result. From
the query history, the command was always calculating the exported
result, while we can just give a query ID to export.
This will create two separate commands for exporting results, one for
exporting the selected results (user-visible) and one for exporting a
specific remote query result. This will make it easier to add support
for exporting variant analysis results.
I'm not sure if there will be impact from renaming the command. I expect
the only impact to be that the command history might not show the
command in the correct place (i.e. it disappears from recently used
commands), but please check if that is the only impact.
This removes the `runRemoteQuery` method and instead moves all logic
specific to remote queries/variant analysis to the remote queries
manager and variant analysis manager respectively. This will make it
easier to completely remove the remote queries manager in the future.
Now that we do not have a dry run mode, we can create and clean up the
temporary directory in the same function. This allows us to remove the
complete try..finally block inside `runRemoteQuery` and move it to a
much more local spot.
The remote queries tests were testing the data on the filesystem, rather
than the data submitted to the server. This required using a `dryRun`
parameter to prevent deleting the temporary directory, while we can
actually just test against the submitted data.
This will create an in-memory filesystem of the submitted query pack by
un-tar-gz'ing the query pack into memory and using that to test the
existence of certain files.
There is some common logic between remote queries and the variant
analysis flows which deals with parsing the query and asking the user
how to run the query. This extracts that part of the logic to a separate
method such that the only logic left in the actual `runRemoteQuery`
method is related to submitting the query.
This also changes the failure reason alert component to remove the logs
button since it's not used by any failure reason. Instead, a link is
added into the message for a failed Actions workflow using which the
Actions workflow run may be opened.
Shared by the AST viewer, jump to def, and find references
contextual queries.
This allows contextual queries to have their dependencies
resolved and be run whether the library pack is in the
workspace or in the package cache.
Clear the CLI server's pack cache before installing packs,
to avoid race conditions where the new lock file is not
detected during query running.
Adjust some helper methods.
If the library pack containing the AST query does not have
a lock file, it is likely to be in the package cache, not
a checkout of the CodeQL repo.
In this case, use `codeql pack resolve-dependencies`
to create a temporary lock file, and `codeql pack install`
to install the dependencies of this library pack.
This allows the CLI to resolve the library path and
dependencies for the AST query before running it.
We were not yet showing any errors when a result download had failed.
This adds a warning icon to any repositories for which the download has
failed and allow expanding the item to show an alert.
This will show a message for the failure reason in the variant analysis
view when the variant analysis has failed. There don't seem to be
designs for these alerts, but we will need to do a full design review of
the view at some point anyway, so I don't think the exact text is
important.
The variant analysis view was missing an alert when the variant
analysis was canceled. This adds it, and also adds a story for checking
what the view of a canceled variant analysis looks like.
Version checks are re-enabled whenever the version of vscode changes.
This is because the user would have needed to manually update their
vscode version in order to get this new version. And another failing
version check would mean there is a newer version that needs to be
downloaded.
It seems like the expansion of the test files pattern is different
between Windows and Linux/macOS. This fixes it by allowing Mocha to
expand the glob pattern rather than the shell which should fix the
inconsistency.
This implements the "Stop query" button on the view. It moves some of
the logic of actually cancelling the variant analysis to the manager
instead of being in the query history to allow better re-use of the
code.
This also adds tests for cancelling a local query and a remote query.
NB: We only cancel queries that are in progress, so the tests check
the behaviour both for in progress and not in progress items.
This removes all usages of the `gh-api` types from the variant analysis
code by replacing it by the same types defined in `shared`.
This is a breaking change for the query history since the files
serialized to disk now also change. However, since this is still behind
a feature flag the change should be safe to make now.
This adds sorting to the variant analysis repositories on the outcome
panels. The sort state is shared between all panels, so unlike the
design this doesn't disable the sort when you are on e.g. the no access
panel.
We're making a number of changes:
1. We're changing the userSpecifiedLabel value to be
`user-specified-name` instead of `xxx`
2. For local queries, we're changing `in progress` to `finished in 0
seconds` when the query has results. The previous version was
contradictory because any query still in progress wouldn't have results.
3. Similarly, for remote queries, we're changing `in progress` to
`completed` when the query has results. Here we actually set a `status`
property which means `in progress` becomes `completed`.
One factory method to rule them all!
There were a number of problems with these methods:
1. We were previously using two different factory methods to generate
a fake local queries. Ideally we'd just have one.
2. We weren't really creating a real LocalQueryInfo object, which
blocked us [1] from being able to correctly understand which fields we
need in our tests and how they interact together.
3. We stubbed a bunch of methods on the original object to get our tests
to work. We can now use a real object with all the trimmings.
[1]: https://github.com/github/vscode-codeql/pull/1697#discussion_r1011990685
Again, we'll need these for sorting.
We also want to be able to set/unset a userSpecifiedLabel. Since this factory
method is used in `history-item-label-provider.test.ts`, we have tests there
that count on this custom label being defined/undefined.
This adds a new textbox to the outcome panels that allows filtering by
the repository full name (e.g. `github/vscode-codeql`). The filtering
uses the same logic as the existing remote queries filter, i.e. by
converting the input and the repository full name to lower case and
checking the the latter includes the former.
Both `copyNoWorkspaceData` and `copyCliIntegrationData` return
promises. Since file copy-ing is quite fast at the moment, this
hasn't been a problem, but it might become a problem in the future
if we start copying larger files.
Let's wait for the operations to finish.
Now that we have a watch command to check when our test files
need updating, we don't need to do this step during the setup.
Co-authored-by: Andrew Eisenberg <aeisenberg@github.com>
Because we're no longer running `gulp` when we run our test command,
we're going to need a way to update our test files when they change.
This will watch for any changes in our test files and copy the new
version over.
Co-authored-by: Andrew Eisenberg <aeisenberg@github.com>
This "config store" creates a `dbconfig.json` file (if it doesn't yet exist),
and reads the file to load the database panel state.
Only the database config store should be able to modify the config
— the config cannot be modified externally.
We previously attempted to speed up no-workspace tests [1] but realised
we still needed to run some setup steps to get the latest files [2].
Given that we already have `npm run watch` running in the background
when we run our tests, we should be able to regenerate files on the fly.
This means we can drop `gulp` from our setup steps when running integration
tests.
While there's still a danger that you forget to run `npm run watch` in
the background, we think the massive speed up (10s -> 1s) is worth it
as we add more and more tests to this extension.
[1]: https://github.com/github/vscode-codeql/pull/1694
[2]: https://github.com/github/vscode-codeql/pull/1696
The tests were expecting the wrong results, except for the case where
the time was less than a second. For less than a second ago, it makes
sense to return "this minute". For times that are 2.001 minutes ago, it
makes sense to return "2 minutes ago" rather then the previous behaviour
of "3 minutes ago".
The `not_found_repo_nwos` field doesn't actually exist (anymore?) on the
GitHub API. The correct name is `not_found_repos`, so this renames the
field on the type and in the scenarios.
This uses a script to add the new `stargazers_count` and `updated_at` to
the scenario files. This is done by using the GitHub API to get the
information for each repo and then updating the scenario file.
The `updated_at` values are not completely representative since they are
the `updated_at` at time of running the script, rather than at the time
the variant analysis was run. However, this should not really matter in
practice. An alternative for scanned repositories might be getting the
creation time of the `database_commit_sha` commit.
Paired with @robertbrignull on debugging why having all types of
query history items isn't playing nicely when we try to remove an item.
We've tracked down the issue it the handleRemoveHistoryItem method
not correctly setting the `current` item after a deletion.
However, it's unclear whether the test setup is to blame or this is a
real bug.
I'm going to leave the tests for `handleRemoveHistoryItem` to test just
local queries for now (as they were originally) and will come back to
this in a different PR.
This will add the star count and last updated fields to the repository
row. We are able to re-use some components from remote queries, but we
cannot re-use `LastUpdated` since it requires a numeric duration, while
we are dealing with an ISO8601 date.
It seems like the Storybook stories were not being type-checked by CI
and got out-of-sync with the required types. This fixes the types and
also uses the factories to reduce the chance of this happening with
future changes.
We were expecting all three types to behave the same when clicked /
double clicked.
In fact local & remote queries only allow you to open the results view
when they're complete, while variant analyses always allow you to open
the results view no matter what their status.
Let's break down these tests per history item type and test the
expected behaviour more granularly.
NB: I tried moving some of the setup in beforeEach blocks, but alas
queryHistoryManager can be undefined so rather than adding `?` to
every method call, I'm just gonna leave the setup inside the tests.
In an ideal world, we'd stop declaring `queryHistoryManager` as
`undefined`:
```
let queryHistoryManager: QueryHistoryManager | undefined;
```
Baby steps!
In [1] we changed our factory methods to actually use QueryStatus when
creating remote query & variant analysis history items.
Previously we were just setting the value to `in progress`...
... which made the tests for history-item-label-provider.test.ts pass...
... but that value did not reflect reality ...
What we actually need to do is introduce a method to map different
query statuses to human readable strings, e.g.
QueryStatus.InProgress becomes 'in progress'
[1]: 4b9db6a298 (diff-217b085c45cd008d938c3da4714b5782db6ad31438b27b07e969254feec8298aL28)
We've introduced a new `local-query-history-item.ts` factory method [1]
which includes a cancellation token. The factory will need to import the
CancellationTokenSource from `vscode`.
We already had a factory method but it didn't quite map with the setup
we needed. For example we need to call `.completeQuery` rather than
providing a dummy `completedQuery` object.
The previous factory method was used in the tests for
`query-history-info.test.ts`. Because that factory omitted the
cancellation token, we could get away with having these tests in the
`tests/pure-tests` folder.
With the addition of the second factory method, the tests for
`query-history-info` blow up because they can't find `vscode`.
Now that we need to add more fields to local query history items, it's
becoming clearer that these `query-history-info` tests should live next
to the `query-history` tests in `vscode-tests/no-workspace`.
Granted, in an ideal situation we'd only have one factory method to
generate a local query history item, but combining these two methods
is actually quite painful. So for now let's at least have the query
history tests next to each other and appease Typescript.
This adds the new `stargazers_count` and `updated_at` fields in the
repositories to the appropriate `gh-api` and `shared` types.
To make testing easier this also moves the
`variant-analysis-processor.test.ts` to the pure tests since it doesn't
and shouldn't depend on any `vscode` APIs.
We're adding both remote query history items and variant analysis history
items to the query history.
We've introduced a little method to shuffle the query history list
before we run our tests so that we don't accidentally write tests that
depend on a fixed order.
The query history now has increased test coverage for:
- handling an item being clicked
- removing and selecting the next item in query history
- handling single / multi selection
- showing the item results
While we're here we're also:
1. Adding a factory to generate variant analysis history items
2. Providing all fields for remote query history items and ordering them
according to their type definition order. At least one field (`queryId`)
was missing from our factory, which we will need to make the tests work
with remote queries.
There are a couple of tests that check whether we can correctly
compare two local queries.
These shouldn't be applied to remote queries [1] so let's just
make that a bit clearer by moving them into a local queries describe
block and using the `localHistory` array to choose items to compare
instead of the `allHistory` array.
[1]: bf1e3c10db/extensions/ql-vscode/src/query-history.ts (L1311-L1314)
At the moment our query history tests are set up to only check
local queries.
Let's prepare the ground to introduce remote query history items
and variant analysis history items.
This will allow us to expand test coverage for these other types
of items.
The `createGist` functionw was part of `gh-actions-api-client`, while it
didn't actually involve anything related to the GitHub Actions API. This
moves it to the non-Actions-specific `gh-api-client` module.
Another candidate for moving to `gh-api-client` is
`getRepositoriesMetadata`, but that one is a bit more involved since it
uses `showAndLogErrorMessage`, so depends on the `vscode` module. This
means it would not be possible to test in the "pure" tests and we would
need to move all our `gh-actions-api` tests to the integration tests. It
will not be used for variant analysis queries anymore, so I don't think
it's worth moving or refactoring to not depend on `vscode`.
This will hook up the "View logs" link to make it open the variant
analysis actions workflow run. The method for creating the actions
workflow run URL has been extracted from the query history to make it
callable without a history item.
This adds the `controllerRepo` field to the `VariantAnalysis` shared
type. This is technically a breaking change since the old history won't
have this field and all calls on this will fail. However, the feature
is not available so this should be fine.
The variant analysis view would allow expanding the results when the
repo task was completed. However, it did not take into account whether
the results were actually downloaded. This will that by usign the
download status when the repo task was succeeded and sending the repo
states to the view on load.
This adds a new file `repo_states.json` which tracks the download status
of all repositories of a variant analysis. We will write this file when
a download has completed and skip a repository download if the repo
state is marked as `succeeded`. This should prevent duplicate downloads.
This will still queue all repositories, even those which have already
been downloaded. However, I expect the actual cost in the download
method to be negligible since it's just an in-memory check.
This wil remove the discrepancy between the files on which ESLint is run
when `lint-staged` is used and the files that are checked using
`npm run lint` and `npm run format`.
It will now also include the `.storybook` directory which was previously
excluded from the ESLint configuration.
This will make the creation of a webview panel async to allow the
`getPanelConfig` method to be an async function. This will allow us to
do some work (like retrieving the variant analysis) in the
`getPanelConfig` method.
This will close the variant analysis view when the corresponding variant
analysis history item is deleted from the query history. This required
some extra code to handle `dispose` being called on the view to ensure
this actually disposes the panel, but we can now call `dispose()` on the
view to close it.
This will change the pure tests Mocha setup to actually use the
`tsconfig.json` located in the `test` directory. Before, it was using
the root-level `tsconfig.json`. To ensure we are still using mostly the
same settings, this will extend the `test/tsconfig.json` from the
root-level `tsconfig.json`.
This splits the mock GitHub API server class into two parts: one for the
interactive, VSCode parts and one for the non-VSCode parts. This allows
us to use the non-VSCode part in tests.
This adds some basic integration tests for MRVA using the GitHub mock
API server. It only does basic assertions and still needs to stub some
things because it is quite hard to properly test things since VSCode
does not expose an API to e.g. answer quick pick pop-ups.
I'm not sure how useful these integration tests will actually be in
practice, but they do at least ensure that we are able to successfully
submit a variant analysis.
We've merged https://github.com/github/vscode-codeql/pull/1656
which actually implements item removal. We'll need to change our
tests to account for this.
We've also merged https://github.com/github/vscode-codeql/pull/1654
which implements opening the view when we click on a variant analysis
history item. So we've changed our tests to take into account that
there's now a `showView` method being called.
We will need to set up some VariantAnalysisHistoryItem types in order
to use them in our tests.
We're repeating what we've done for RemoteQueryHistoryItem for now.
Separately we'll think about setting up tests that check for both
remote queries and variant analysis in the query history.
At the moment we'd like to focus on just adding some test coverage
for variant analysis history items.
Co-authored-by: Nora Scheuch <norascheuch@github.com>
At the moment we create the results manager as a private property on the `VariantAnalysisManager`.
If we instead created it at the extension level and passed it to the `VariantAnalysisManager`, we would have more freedom to write unit tests for the `VariantAnalysisManager` without needing to reach into a private results manager property.
We had previously added a no-op placeholder for when we attempt
to remove a variant analysis from our query history.
This adds the implementation:
- removes the item from the query history
- cleans up any existing result files attached to the variant analysis
NB: The remote queries would store all their results in a single folder.
For variant analysis, we store results per repo. The folder names are build
using a cache key and are stored in `cachedResults`. The cache key is
built from the variant analysis id and the repo name.
In order to delete the results, we've had to pass in the full variant analysis
object to the manager and call `cacheResults.delete()` for each of its scanned
repos.
Co-authored-by: Charis Kyriakou <charisk@github.com>
Co-authored-by: Nora Scheuch <norascheuch@github.com>
msw doesn't seem to support binary responses because it decodes them to
a UTF-8 string. To work around that, we will do a separate fetch of the
file and save that.
When the mock GitHub API server setting was moved to the top-level, we
forgot the comamnds in the `package.json`. This updates the commands to
have the correct visibility.
See: https://github.com/github/vscode-codeql/pull/1643
This adds a linter for JSON scenario files which will validate the JSON
files in the scenarios directory against the TypeScript types. It will
convert the TypeScript types to JSON schema to simplify this process.
Unfortunately, this will not currently allow adding scenarios with
failing requests since the types do not allow this. Rather than removing
this validation, we should fix the types. This can be done in a follow-up
PR.
The command lint expects all command palette commands to have a common
prefix which these violated. So, I've moved them to being a scoped
command so we can have different lints.
If the workspace is restarted while databases are being loaded, this
change prevents any from being lost.
The bug was that each time a database was added when rehydrating a db
from persisted state on startup, the persisted db list
was being updated. Instead of updating the list each time we add a db,
on restart, instead update the persisted list only after all are added.
Note that we need to update the persisted list after reading it in since
the act of rehydrating a database _may_ change its persisted state.
For example, the primary language of the database may be initialized
if it was not able to be determined originally.
* Fix missing DIL for new query server
* Fix DIL error message when QLO was not expected.
* Update extensions/ql-vscode/src/run-queries-shared.ts
Co-authored-by: Andrew Eisenberg <aeisenberg@github.com>
Co-authored-by: Andrew Eisenberg <aeisenberg@github.com>
This adds a new class which will setup the MSW server to record requests,
save them to memory and save them to files when calling a separate save
method.
This adds a Storybook add-on that allows you to switch between VSCode
theme. It follows the pattern of the [outline](https://github.com/storybookjs/storybook/tree/v6.5.12/addons/outline/src)
and [backgrounds](https://github.com/storybookjs/storybook/tree/v6.5.12/addons/backgrounds)
add-ons.
Unfortunately, it doesn't apply the CSS to just the elements it should
be applied to, but globally to the complete preview. This is a limitation
of using CSS files rather than setting inline styles on the elements. We
might be able to resolve this in the future by extracting the CSS
variables from the CSS files, but this is somewhat more involved.
- Avoid Installing `xvfb` since it is already available.
- Ensure `supportsNewQueryServer()` takes the CLI version into account
- Always run the new query server tests on v2.11.1 and later
- Avoid printing directory contents in `run-remote-query-tests`
- Run tests with `--disable-workspace-trust` to avoid a non-fatal error
being thrown from the dialog service.
- Ensure the exit code of the extension host while running integration
tests is the exit code of the actual process. Otherwise, there is
a possibility that an error exit code is swallowed up and ignored.
- Remove a duplicate unhandledRejection handler.
- Handle Exit code 7 from windows. This appears to be a failure on
exiting and unrelated to the tests.
- Fix handling of configuration in tests:
1. It is not possible to update a configuration setting for internal
settings like `codeql.canary`.
2. On windows CI, updating configuration after global teardown. So,
on tests avoid resetting test configuration when tests are over.
Also, I tried to remove all those pesky errors in the logs like:
> [2094:1017/235357.424002:ERROR:bus.cc(398)] Failed to connect to the bus: Could not parse server address: Unknown address type (examples of valid types are "tcp" and on UNIX "unix")
I was following advice from here, but I can't get it working.
- https://github.com/microsoft/vscode-test/issues/127
- https://github.com/electron/electron/issues/31981
This adds a documented way to change the theme in Storybook from the
VSCode Dark+ theme to the VSCode Light+ theme. It requires multiple
changes to two files, but these are all quite simple and it has been
documented on the "Overview" page.
Previously we were only checking whether we're triggering the download
command in the extension.
Now we're mocking `autoDownloadVariantAnalysisResult` on the
variantAnalysisManager and checking that it's being called for all repos
that have available results.
Before we make any changes, let's extract some of the monitor code into
smaller methods.
Since we have test coverage, we're able to do this quite comfortably.
We added a `successful` property to serialized local queries. But, this
property does not exist on older serialized queries. This change ensures
older queries get a `successful` property when deserialized.
The threshold at which the bad join order detection reports a warning was previously hard-coded to 50. Initial feedback from internal QL developers suggests that this is too high, and should be configurable in any case. I've made it configurable via the `codeQL.logInsights.joinOrderWarningThreshold` setting, leaving the default at 50. Once we get more feedback about what a better default value is, I'll update the default.
This adds tests for the duration calculation and moves it down a
component to make this easier. Adding tests for the
`VariantAnalysisHeader` would require constructing a complete variant
analysis object, while this is now just a simple unit test.
This will add some new date fields that have been added in the API to
the variant analysis types and factories. They are stored as strings
since storing them as `Date` would make the types inconsistent if they
are serialized to JSON (`JSON.stringify` -> `JSON.parse` would result
in strings rather than dates).
The `text` property is already nested under `query`, so it's redundant
to prefix it with `query`. This also makes it consistent with the other
properties.
This will implement the final step of opening the query text. Inside
the webview, this will send the message to the extension host to open
the query text.
This will add a new text document content provider for showing variant
analyses. This is separate from the remote queries content provider
to allow this to evolve separately. It also retrieves the query text
from the manager rather than passing the text directly to prevent the
webview from opening a tab with arbitrary content.
See: 4c527a3573/extensions/ql-vscode/src/extension.ts (L1242-L1257)
This will add a new query text field to the variant analysis submission,
which will also propagate to the variant analysis itself. This will
allow us to show the query text on the variant analysis page.
This will register all settings for which a `Setting` instance is
created as settings which will be reset. This should make it less
error-prone to change settings in tests.
The `vscode-test` package was renamed to `@vscode/test-electron` in
December of last year. This commit updates the extension to use the new
package name.
The reason for this change is that the `vscode-test` package was
somewhat flaky in actually starting VSCode to run the tests from the
command line. The new package also has some bugfixes and other
improvements which would normally have been part of a new version of the
`vscode-test` package.
This makes it possible to open the query file in the editor when
clicking on the query filename.
This is a slightly different implementation from the remote queries
implementation. The remote queries implementation will send the file
path to open to the extension host, and the extension host will simply
open the given file path. If someone is able to inject JavaScript into
the webview, this would allow them to open an arbitrary file in VSCode.
By moving the file path logic to the extension host, we can ensure that
we only allow opening the actual query file.
* QueryServer: Add support for new query-server
* Add a new canary flag to enable new query server support
* Add evaluation results to query object
Ensures better backwards compatibility with legacy query objects.
* Fix query server command name
* Add log message for new query server
* Use only legacy results
Co-authored-by: alexet <alexet@semmle.com>
This will implement ebba9949a8
and d18e3dd40e
for the `Compare` and `RemoteQueries` views. These should not be
impacted in the same way as the `VariantAnalysis` view, but this will
make them consistent and more resilient to future changes.
This cleanup function would never be called in normal operation, but if
we do decide to add a dependency to this `useEffect`, this will ensure
that only one listener is registered at a time.
When the variant analysis view was being rerendered, we were also
reregistering the message listeners, while not deregistering the old
ones. This resulted in a loop of message listeners being registered,
and the variant analysis being rerendered every time a message was
received by one of the listeners. This will ensure that the listener
is only registered once to prevent this from happening.
To make debugging the view easier and prevent needing to run a variant
analysis for each change, this will add a simple command which opens a
variant analysis by its ID. This it not intended to be visible to users
at any point.
Now that we're unzipping results, we also have to use something closer
to a zip file when testing download functionality for the
`variantAnalysisManager`.
The `variantAnalysisManager` has access to the
`variantAnalysisResultsManager` so we could've stubbed the result
manager's `download` method instead of going as far as using a zip
fixture.
However, since the results manager is private it seems bad to make it
public in order to stub one of its methods.
So using realistic data in the setup seems like a good compromise.
This will:
- download a zip file as an ArrayBuffer
- save the file as `results.zip`
- unzip the contents into a `results/` folder
For the tests:
- In order to check whether we're saving the correct files in the tests,
we've had to make the `getRepoStorageDirectory` method public.
Unfortunately the temporary file path generated for tests is random so
we're not able to hardcode it.
- Now that we have a real zip file to use in our tests, we're first
converting this file into an ArrayBuffer, then stubbing the API to
return it. We then check that it's saved and unzipped correctly.
This matches what type of file we'd expect in real life: a zip file
containing a sarif file.
We've copied an example `results.sarif` file from other tests in the
`no-workspace` folder.
We expect this method to return a zip file which can be typed to an
`ArrayBuffer`. In the following commits we'll read this buffer and save it
as a zip file.
This class will be used to set test config values for the tests. It is
able to set the config value to a specified value for every test and
restore the value to the original value after the test.
Instead of using the `glob` library and a custom promise, this will use
`glob-promise` which is used by other parts of the codebase already.
This reduces the amount of code which manually needs to call `reject`
and makes it easier to read.
When we first submit the variant analysis for processing, we'd like to update
the query history panel.
At the moment we're just adding the setup for triggering the event. In a future
PR we'll consume this event and change the query history panel accordingly.
In order for this to happen we will need to introduce a new `VariantAnalysisHistoryItem`
type which will massage the data we get from the API into a type which the Query
History panel can consume.
Co-authored-by: Shati Patel <shati-patel@github.com>
When the `viewLoaded` message is received by the view, it will now
retrieve the variant analysis from the manager and send it to the
view. This will allow the view to display the variant analysis.
This will change tests that are using a mocked `CancellationTokenSource`
to use a real `CancellationTokenSource` instead. Tests are run inside
VSCode, so we can use these without mocking.
To create the interpreted and raw results from the SARIF/BQRS files, we
need some information from the repo task object. This will store the
repo task object to the filesystem as JSON so we can read them when
loading results.
In most cases, we will not have access to the full repo task object
since this needs to be retrieved from the API. Since we are only using
the full name from the repo task object, we can just use the full name
instead.
This will store all variant analysis that are run in the manager. Right
now, it only stores the variant analyses in memory. In the future, these
will be loaded from the query history and can be restored after a
restart.
We register a handler for the old command ID, but do not mention it in package.json.
This seems to be backward compatible without polluting the command palette.
This adds a new variant analysis results manager which is responsible
for downloading and loading variant analysis results to/from the
filesystem. It is essentially the `AnalysesResultsManager` modified to
suit the variant analysis results.
All fields in the variant analysis skipped repositories are optional,
but this was not properly defined in the API types. This will correct
the types and the functions processing the data such that they handle
non-existing fields.
To be able to send messages to the open view for a variant analysis, we
need to have a reference to the view. This is done by keeping track of
all open views in a dictionary indexed by their variant analysis ID.
We currently only allow one view per variant analysis, but do allow
multiple variant analysis views to be open at a time. In the future, we
may want to allow multiple views per variant analysis (such that e.g.
"Split right" works), but this is not supported yet.
The reason for the indirection through the interfaces is to prevent
circular dependencies between the variant analysis view and the manager.
This won't have an `id` field. We initially generated this the same
way we did for all other skipped repos, but this one is special because
it's only providing the fullName field, while the others also provide
`id` and `private`.
This introduces a new `autoDownloadVariantAnalysisResult` command which
will be called by the VariantAnalysisMonitor every time it detects a new
repo has been scanned.
In turn, this will use the `autoDownloadVariantAnalysisResult` method
which we defined in an earlier commit on the VariantAnalysisManager.
This method will be called from the VariantAnalysisMonitor once
a new repo has been scanned.
It will then perform an API request to get the repo task for it,
which will contain an `artifact_url`.
Finally it will use the API method we introduced in the previous commit
to download the result for the repo and then save it on disk.
In a previous PR [1] we introduced factories for generating variant analyses
(and their associated objects) that were returned from the API.
Let's also introduce factories for generating their VSCode equivalent.
We can immediately use them for generating a VariantAnalysis object for the
monitor tests.
[1]: https://github.com/github/vscode-codeql/pull/1545
This is a follow-up to clean up the skipped and analyzed repository
component duplication. The rows in both tabs are very similar, so this
will combine them to use a single component.
This will open the variant analysis view after the variant analysis has
been submitted. It will also show a notification that the analysis has
been submitted, which includes the query name.
This implements persistence for the variant analysis webview, allowing
the webview panel to be restored when VSCode is restarted. It's probably
easier to add this now than to try to add it later.
The basic idea is that there are no real differences when opening the
webview for the first time. However, when VSCode is restarted it will
use the `VariantAnalysisViewSerializer` to restore the webview panel.
In our case this means recreating the `VariantAnalysisView`.
To fully test this, I've also added a mock variant analysis ID as the
state of the webview. This value is now randomly generated when calling
the `codeQL.mockVariantAnalysisView` command. This allows us to test
opening multiple webviews and that the webviews are restored with the
correct state.
See: https://code.visualstudio.com/api/extension-guides/webview#persistence
At the moment we're only able to send one of:
- repositories
- repositoryLists
- repositoryOwners
In the future, we intend to be able to send a combination of these
but at the moment the API will only ever allow you to send one.
So let's be consistent and just send `repositories` here.
Currently, when running a query which produces raw results, we will show
all repositories, even if they do not have any results. This change will
ensure that we are only showing repositories which have results. This
matches the behavior for queries which produce interpreted results.
The `controllerRepo` parameter was being encoded/escaped by Octokit,
resulting in a URL like
`repos/dsp-testing%2Fqc-controller/code-scanning/codeql/queries` rather
than `repos/dsp-testing/qc-controller/code-scanning/codeql/queries`.
This switches it to use the ID instead, since we already have the ID
and do not have access to the owner and repo separately anymore.
Now that we have a monitor, we expect the variant analysis to return
a list of scanned repos.
Let's re-use our previous factory for creating mocked responses to
get a dummy variant analysis with scanned repos.
In a previous commit we were submitting a variant analysis to the API
and then triggering a `monitorVariantAnalysis` command.
Here we're hooking up the command to the VariantAnalysisMonitor class.
This will poll the API every 5 seconds for changes to the variant
analysis. By default it will continue to run for a maximum of 2 days,
or when the user closes VSCode.
The monitor will receive a variantAnalysis summary from the API that
will contain an up-to-date list of scanned repos.
The monitor will then return a list of scanned repo ids.
In a future PR we'll add the functionality to:
- update the UI for in progress/completed states
- raise error on timeout
- download the results
So that we're able to:
- set the status value
- build scanned and skipped repos by default
For previous tests, we needed to perform checks on scanned & skipped
repos so we needed to build them outside of this method. When we re-use
this method for the VariantAnalysisMonitor, we will just need a generic
ApiResponse so we can create these repos inside the method.
We're going to need some of these methods to generate a valid VariantAnalysis.
We might as well extract them from the tests for the VariantAnalysisProcessor.
Once we submit a variant analysis and get our response from the API,
we'd like to set up a way to monitor the variant analysis as it starts
producing live results.
Here we're using a VSCode command to trigger a monitoring process which
will poll the API for changes.
The `RawResultsTable` was using inline styles, while we should prefer
to use styled components. This refactors it to use styled components and
also improves some other miscelleanous things (extracting the props to
a separate type and moving the `Cell` above the `Row` since the latter
uses the former).
This adds the analyzed repositories component for showing within the
"Analyzed" tab. I wasn't completely sure whether there should be a
difference between "Pending" and "In progress", but pending will now not
show an icon, while in progress will show a spinner.
For the collapsible items, it does not reuse the `CollapsibleItem`
component because that component is tightly coupled with the styles
of the remote queries component.
This creates the component for showing the outcome panels. It does not
implement the content of each individual panel; it only implements the
tabs, panel views, and the general warnings.
I came across this when I had a query that threw an error while running
for unrelated reasons. At this point, the query results were in a bad
state, but this caused `safeMax` to be called with `undefined` and
it prevented the extension from starting. This changed fixed the error.
This involved changing a few different methods to take a Repository object
instead of taking owner and repo separately. Overall I think this is a good change.
This will add Storybook stories for the error, success, and warning
icons, as well as for the generic `Codicon` component.
To show the available icons for the `Codicon` component, a static JSON
list is generated from the contents of a CSV file included as part of
the `@vscode/codicons` npm package. The command to regenerate the file
is included in the story.
This will change the VariantAnalysisHeader to take the VariantAnalysis
domain model instead of a large amount of props.
It also adds the `canceled` status to the `VariantAnalysisStatus` to
represent a stopped variant analysis.
The "Export All" button was always exporting the selected query, while a
different query could be open in a VSCode panel. This will ensure that
the query ID is passed to the export function, so that the correct query
is exported.
This refactors the CodePaths and FileCodeSnippet components to be more
readable and in style with the rest of the "new" components. It does the
following:
- Remove uses of the `style` and `sx` props; replace it by using
`styled-components` instead
- Remove uses of Primer icons
- Split out the components into multiple files
- Change the colors of the severity to match VSCode colors (and make
them themable)
I haven't removed the use of the Primer `Overlay` component yet, since
this component seems to do quite a lot and the VSCode WebView UI Toolkit
doesn't have a replacement for it.
The `runUpgrade` query server command is mistakenly caching the old
dbscheme in memory after running the upgrade. The problem is in the
CLI. The workaround is to restart the query server after running an
upgrade. This is not a great solution, but considering that explicit
upgrades are now very rare. I do not think it is worth putting in
too much effort for a proper fix.
The working directory of ESLint was not set directly, so ESLint warnings
did not show up in VSCode. This sets the working directory properly such
that ESLint warnings are shown in VSCode.
See: https://github.com/Microsoft/vscode-eslint#settings-options
This will make a distinction between cancelled local and remote results,
allowing us to hide the *Open Variant Analysis on GitHub* item from
local failed/cancelled items. It also hides the *Show Evaluator Log*
items for cancelled/failed remote queries.
This sets up Storybook for testing of React components. It adds stories
for some of the MRVA components. It does not add stories for the main
MRVA views since those are not independent of VSCode and need to be run
from within VSCode.
It seems like the result-index artifact may not be available immediately
after the workflow run has finished. This adds a retry mechanism to wait
for the result-index to be available. It will retry at most 10 times
with a wait of 1 second between each retry.
The original code that logged the human-readable log summary generated the log asynchronously, which was a reasonable choice. When I added support for viewing and scanning logs, I didn't notice that the summary was being generated asynchronously, and wrote my code assuming that the summary was already on disk when I opened it to find where each relation's log started. The effect was that, depending on timing, the evaluation sometimes failed with an error popup complaining about not being able to open the log summary file.
The fix is to _generate_ the log summary synchronously, but continue to _log_ it asynchronously.
This does not remove the previously added mechanism of not requesting
credentials, but using them when they are available. I expect this to be
used in the future.
This makes authentication for download GitHub CodeQL databases optional.
If you are already authenticated, your token will be used. If you are
not authenticated, an anonymous request will be made.
If the canary flag is enabled, you will be prompted for credentials when
downloading a database and you are not yet logged in.
This will include the Codicons inside the webview bundle, reducing the
number of files that need to be loaded and the resource roots that need
to be included.
This will add a new abstract class that implements the creation of the
panel and webview to reduce duplication across the different interface
managers.
This will move all webviews into a single Webpack bundle. This will make
it easier to add new webviews since we don't need to add a new bundle,
but just need to add a new directory with an `index.tsx` file.
It also moves the CSS processing to Webpack so that we don't need to
specify the CSS files to use separately, but can simply do so in the
TypeScript files.
When the MRVA results panel is closed (so the panel gets disposed) and
opened again, it would not load the MRVA data (such as whether a query
has already been downloaded). This fixes it by also resetting the
internal state of whether the panel is loaded when the panel is
disposed.
We'd like to remove duplicate whitespace in these labels in order
to make it less likely that we introduce extra space.
We initially also tried trimming whitespaces at the start and end
of these labels but that had no effect.
We'd like to be able to add tests for when the result count exists and
when it's missing.
Let's change the createMockRemoteQueryInfo method so that we can pass
in parameters by name, e.g.
```
createMockRemoteQueryInfo(undefined, 2)
```
becomes
```
createMockRemoteQueryInfo({ repositoryCount: 2 }
```
We were splitting JSONL records based on the current OS newline sequence. In order to handle reading of logs from the opposite OS, I've switched our split to handle both flavors of line ending. This originally showed up as log parser unit tests failing on Windows (the checked-in log used Unix line endings), but could affect real world usage as well.
`undefined`, `null` and 0 will evaluate to `false` so if we only want to
display the repository count when these values are not present we can
check for a truthy value:
```
query.repositoryCount ? `(${pluralize(...)})` : '';
```
instead of checking explicitly:
```
query.repositoryCount !== undefined && query.repositoryCount !== null && query.repositoryCount != 0 ? `(${pluralize(...)})` : '';
```
We were splitting JSONL records based on the current OS newline sequence. In order to handle reading of logs from the opposite OS, I've switched our split to handle both flavors of line ending. This originally showed up as log parser unit tests failing on Windows (the checked-in log used Unix line endings), but could affect real world usage as well.
When a queryResult is created, it comes with an array for AnalysisSummaries.
There is one summary per repository.
We've had to calculate the total number of results for all summaries in multiple
places, so let's extract a method for this as well.
There are at least 4 different files where this method could DRY things up,
so let's extract it.
I've chosen to move it to src/helpers.ts but happy to be told there's a better
place for shared utility methods like this one.
Previously we would set all remote query results to -1 when someone
attempted to sort queries.
We would then only sort local queries as those had access to the number
of results.
Let's include number of results for remote queries in the sorting.
Co-authored-by: Shati Patel <shati-patel@github.com>
In the previous commit we're now displaying number of results for remote
queries.
Previously we could only do this for local queries.
Let's make the format match for both types of queries by displaying
number of results in parentheses: `(x results)`.
Co-authored-by: Shati Patel <shati-patel@github.com>
When you run a remote query, we'd like to display more information about
it in the Query History panel.
At the moment we've improved this [1] by adding the language and number of repositories.
In this commit we're also adding the number of results for a remote query.
So the final format of the query history item will change from:
`<query_name> - <query_status>`
to
`<query_name> (<language>) on x repositories (y results) - <query_status>`
[1]: https://github.com/github/vscode-codeql/pull/1427
Co-authored-by: Charis Kyriakou <charisk@github.com>
Co-authored-by: Shati Patel <shati-patel@github.com>
While we're here we're also adding a test for the `exportResultsToGist`
method, as there were no tests for the `export-results.ts` file.
We initially attempted to add the test to the pure-tests folder, but the
`export-results.ts` file imports some components from `vscode`, which
meant we needed to set up the test in an environment where VSCode
dependencies are available.
We chose to add the test to `vscode-tests/no-workspace` for convenience,
as there are already other unit tests there.
We've also had to import our own query and analysis result to be able
to work with data closer to reality for exported results.
Since we've introduced functionality to build a gist title, let's check
that the `exportResultsToGist` method will forward the correct title to
the GitHub Actions API.
Co-authored-by: Shati Patel <shati-patel@github.com>
We'd like to re-use this to test the `exportResultsToGist` method in
`export-results.ts`.
So let's move it to a shared folder in the `vscode-tests/no-workspace` folder.
Since there's no `helper.ts` file in this folder and to avoid any confusion with
the `helpers.test.ts` file, I've opted to put this shared method into `index.ts`.
Happy to be told there's a better pattern for this as it doesn't feel very nice!
Let's handle this case gracefully and skip displaying the number of repositories
when they're not available.
Similarly let's add a check to see if we should pluralize the `repository` noun
or not.
Co-authored-by: Shati Patel <shati-patel@github.com>
All exported MRVA gists are given the name `CodeQL variant analysis
results', which makes it hard to work out what it contains at a glance.
We're adding more information in the gist title to make it more useful.
Example of new title:
`Empty Block (Go) x results (y repositories)`
This translates to:
`<query name> (<query language>) <number of results> results (<number of repositories> repositories)`
Co-authored-by: Shati Patel <shati-patel@github.com>
We'd like to improve MRVA query gists by giving them more descriptive
titles that contain useful information about the query.
Let's add the number of query results to the title of the gist.
To do this we'll first need to count all the results provided to us in
the `analysisResults` array. There is an item in this array for each of
the repositories we've queried, so we're introducing a method to sum up
results for all the items in the array.
Co-authored-by: Shati Patel <shati-patel@github.com>
Previously, there was a bug where quick eval queries would crash when
the eval snippet is in a library file.
The problem was that the `codeql resolve queries` command fails when
passed a library file. The fix is to avoid passing the library file at
all. Instead, pass the directory. This is safe because the resolve
queries command only needs to know which query pack the file is
contained in. Passing in the parent directory is the same as passing in
a file in this particular case.
Currently `resolve ml-models` only supports queryspecs, i.e. .ql, .qls,
directory, and query pack specifications. Therefore quick evaluation within
a library isn't
supported.
In order to run our cli-integration tests, we're required to have a
local copy of the codeql CLI repo. We can then run the tests by running
the `Launch Integration Tests - With CLI` task from inside VS Code.
(See CONTRIBUTING.md for details.)
If we don't have the CLI repo cloned locally or we're not pointing to it
in `launch.json`, we don't get a clear indication of what the problem is.
The tests will still attempt to run.
Let's fail fast instead and add an actionable error message to the output.
The controller repo is set via the `codeQL.variantAnalysis.controllerRepo`
setting in VSCode.
While we have validation to check that the repo is not null and the
format of the controller repo is correct: `<owner>/<repo>`, we still
allow you to provide a non-existent repo (e.g. a mispelled one).
When the MRVA request is sent over to the API, it will verify that the
repo exists and return a very generic "Not Found" response.
This will then be logged out in the "Output" tab for VSCode.
We'd like to give users a better indication of what has gone wrong in
this case so we're making the error message more verbose.
Co-authored-by: Charis Kyriakou <charisk@github.com>
Co-authored-by: Shati Patel <shati-patel@github.com>
How did this ever work? It was using an old variant of the
qlpack name.
Also, this commit makes the unhandledRejection handler less
verbose. This gets hit when the tests end and there is a cancellation.
this is not an error.
For an inexplicable reason, the first time the selection
occurs, the value is incorrect. We often miss this error
in our tests if the expectation is reached before the
selection changed event fires.
It seems that the _second_ time the selection changed
event fires, the value is correct.
This change ensures we wait for the second selection change.
And we avoid running expectations until then.e
We're upgrading the minimum version of webpack from 5.28.0 to 5.62.2
since this version doesn't rely on OpenSSL for its hashing algorithm so
it wouldn't need legacy OpenSSL support when we decide to upgrade to
Node 18.
This allows us to build our extension on Node 18:
https://github.com/github/vscode-codeql/runs/6904100934?check_suite_focus=true
Happily, this also works fine with our current version of Node (16.13.0).
Adding two things:
- A bit more detail on how to add a tag and how to delete a badly named one
- Switch to the official way of sharing tags according to the git docs[^1]
[^1]: https://git-scm.com/book/en/v2/Git-Basics-Tagging
It's been pointed out that MRVA results are hard to read as the font is
small and narrowly spaced. It also doesn't match the font type normally
used in source files.
We can instead switch to using the font-family used by VS Code itself and
increase the font size from `x-small` to `small` for code snippets.
As recommended here https://github.com/github/vscode-codeql/pull/1369#issuecomment-1142418037, we want to stay in sync with the current node version shipped with
VSCode (v16.13.0):
32d40cf44e/remote/.yarnrc (L2)
For this we can add a `.nvmrc` file to alert nvm to switch to the preferred version automatically.
It will also help prevent builds from failing when setting up the project for the first time, as building the extension currently fails in Node v18: https://github.com/github/vscode-codeql/issues/1373
We're also updating the docs to mention using `nvm` to manage node versions and point to the right place to check for current supported versions.
1. `exportCsvResults` now no longer requires an `onFinish` callback.
2. The test adds a generic framework for creating a mock cli server.
This should be used in future tests.
If the `#select` resultset doesn't exist, arbitrarily choose the first
result set when viewing csv results. This will almost certainly be the
correct result set.
In the future, we could offer a popup if there are multiple result sets
available, but let's wait on that until someone actually asks for it.
Create the `time.ts` module as a place to put fime functions.
Move two time functions there and create tests for them.
The `humanizeUnit` function now uses ECMAscript apis. This ensures
that pluralization happens appropriately.
Also, fix a small bug in the results view to enure `repository`
is correctly pluralized.
The `lastUpdated` value is now the duration between timestamp of the
last time the repo was updated and time the file was downloaded.
This fixes the duration and it won't change over time.
1. Refactor references of `Stargazers` to `RepositoryMetadata` since
the query is now more generic.
2. Update the graphql query to request last updated as well as stars
3. Update web view to display last updated
4. Update sort mechanism for last updated
A few notes:
1. I used `Intl.RelativeTimeFormat` to humanize the times. It wasn't as
simple as I had hoped since I need to also make a guess as to which
unit to use.
2. The icon used by last updated is not quite what is in the wireframes.
But, I wanted to stick with primer icons and I used the closest I can
get.
3. The last updated time is retrieved when the query is first loaded
into vscode and then never changes. However, this time is always
compared with `Date.now()`. So, opening the query up a week from now,
all of the last updated times would be one week older (even if the
repository has been updated since then).
I don't want to re-retrieve the last updated time each time we open
the query, so this timestamp will get out of date eventually.
Is this confusing as it is?
The new support will be available in the next
release of the CLI, most likely 2.9.3,
This change requires the query to be run to be
passed in to the call to resolve ml-models.
The main problem this commit fixes is with vscode 1.67.0, an error is
thrown when inside of integration tests and a dialog box is opened. We
were opening the telemetry dialog box. Now, an env variable is set
during cli-integration tests that prevents the dialog from being
opened.
There are also other cleanups and improvements with cli-integration
tests that assist with running locally:
- `vscode-test` dependency has been renamed to `@vscode/test-electron`,
so use that instead and make the small API changes to support it.
- Commit the codeql-pack.lock.yml file so it isn't recreated on each
test run.
- Ensure all databases are removed before _and after_ each test run
that manipulates the set of installed databases
- Similarly, for quick query files, delete them before and after each
test.
- Change some async `forEach` blocks to for loops in order to support
sequential operations more easily.
This dependency was only used to parse package.json and
this can be just as easily parsed by regular JSON object.
jsonc can also parse JSON with comments, but there are no
comments in package.json.
This fixes a bug where an open results view will accumulate results from
other queries who have their results downloaded while this view is open.
The fix is to ensure that the results view for the query is open when
some results are downloaded.
This will now name databases downloaded from github based on their nwo.
Also, this adds a new button to suggest downloading from github in an
empty databases view.
Two scenarios handled:
1. no database for existing repo
2. repo does not exits (or no access rights for current user)
In either case, an error message is sent to the logs, with a notificaiton
in a popup.
Also, change interpolation of result count. For Remote queries, this
value will be empty. For local queries, use the label `X results`, where
`X` is the number of results for this query.
The labels for remote query items are interpolated using the same
strategy as local queries with two caveats:
1. There is no easy way to get the result count without reading files,
so, this value is kept empty.
2. There is no database name for remote queries. Instead, use the
nwo of the controller repo.
Also, adds tests for the history item label provider.
The label provider is the instance that performs the logic for
generating labels for history items, using string interpolation when
necessary.
This commit creates the label provider and uses it with local queries.
Remote queries will be changed in the next commit.
(Only in recently introduced locations. More work still needs to be
done.)
Also:
- Change error to info
- Create credentials directly, don't use a callback.
This change issues a cancel request when the user clicks on "cancel" for
a remote query.
The cancel can take quite a while to complete, so a message is popped up
to let the user know.
The labels for remote query items are interpolated using the same
strategy as local queries with two caveats:
1. There is no easy way to get the result count without reading files,
so, this value is kept empty.
2. There is no database name for remote queries. Instead, use the
nwo of the controller repo.
The extension ms-vscode.test-adapter-converter is now available on
open-vsx, but under a different name.
Fixes https://github.com/github/vscode-codeql/issues/1085
I have verified that I can publish and install the extension by
manually publishing v1.6.4.
@@ -67,10 +67,7 @@ members of the project's leadership.
## Attribution
This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4,
available at https://www.contributor-covenant.org/version/1/4/code-of-conduct.html
[homepage]: https://www.contributor-covenant.org
This Code of Conduct is adapted from the [Contributor Covenant, version 1.4](https://www.contributor-covenant.org/version/1/4/code-of-conduct.html).
For answers to common questions about this code of conduct, see
https://www.contributor-covenant.org/faq
[the Contributor Covenant FAQ](https://www.contributor-covenant.org/faq). For more about Contributor Covenant, see [the Contributor Covenant website](https://www.contributor-covenant.org).
Hi there! We're thrilled that you'd like to contribute to this project. Your help is essential for keeping it great.
@@ -23,13 +23,17 @@ Please note that this project is released with a [Contributor Code of Conduct][c
Here are a few things you can do that will increase the likelihood of your pull request being accepted:
* Follow the [style guide][style].
* Write tests. Tests that don't require the VS Code API are located [here](extensions/ql-vscode/test). Integration tests that do require the VS Code API are located [here](extensions/ql-vscode/src/vscode-tests).
* Write tests:
* [Tests that don't require the VS Code API are located here](extensions/ql-vscode/test).
* [Integration tests that do require the VS Code API are located here](extensions/ql-vscode/src/vscode-tests).
* Keep your change as focused as possible. If there are multiple changes you would like to make that are not dependent upon each other, consider submitting them as separate pull requests.
* Write a [good commit message](https://tbaggery.com/2008/04/19/a-note-about-git-commit-messages.html).
## Setting up a local build
Make sure you have installed recent versions of vscode (>= v1.52), node (>=12.16), and npm (>= 7.5.2). Earlier versions will probably work, but we no longer test against them.
Make sure you have installed recent versions of vscode, node, and npm. Check the `engines` block in [`package.json`](https://github.com/github/vscode-codeql/blob/main/extensions/ql-vscode/package.json) file for compatible versions. Earlier versions may work, but we no longer test against them.
To automatically switch to the correct version of node, we recommend using [nvm](https://github.com/nvm-sh/nvm), which will pick-up the node version from `.nvmrc`.
### Installing all packages
@@ -52,12 +56,10 @@ Alternatively, you can build the extension within VS Code via `Terminal > Run Bu
Before running any of the launch commands, be sure to have run the `build` command to ensure that the JavaScript is compiled and the resources are copied to the proper location.
We recommend that you keep `npm run watch` running in the backgound and you only need to re-run `npm run build` in the following situations:
We recommend that you keep `npm run watch` running in the background and you only need to re-run `npm run build` in the following situations:
1. on first checkout
2. whenever any of the non-TypeScript resources have changed
3. on any change to files included in one of the webviews
- **Important**: This is easy to forget. You must explicitly run `npm run build` whenever one of the files in the webview is changed. These are the files in the `src/view` and `src/compare/view` folders.
### Installing the extension
@@ -77,74 +79,23 @@ $ vscode/scripts/code-cli.sh --install-extension dist/vscode-codeql-*.vsix # if
You can use VS Code to debug the extension without explicitly installing it. Just open this directory as a workspace in VS Code, and hit `F5` to start a debugging session.
### Running the unit tests and integration tests that do not require a CLI instance
### Storybook
Unit tests and many integration tests do not require a copy of the CodeQL CLI.
Outside of vscode, in the `extensions/ql-vscode` directory, run:
You can use [Storybook](https://storybook.js.org/) to preview React components outside VSCode. Inside the `extensions/ql-vscode` directory, run:
```shell
npm run test&& npm run integration
npm run storybook
```
Alternatively, you can run the tests inside of vscode. There are several vscode launch configurations defined that run the unit and integration tests. They can all be found in the debug view.
Your browser should automatically open to the Storybook UI. Stories live in the `src/stories` directory.
Only the _With CLI_ tests require a CLI instance to run. See below on how to do that.
Alternatively, you can start Storybook inside of VSCode. There is a VSCode launch configuration for starting Storybook. It can be found in the debug view.
Running from a terminal, you _must_ set the `TEST_CODEQL_PATH` variable to point to a checkout of the `github/codeql` repository. The appropriate CLI version will be downloaded as part of the test.
More information about Storybook can be found inside the **Overview** page once you have launched Storybook.
### Running the integration tests
### Testing
The _Launch Integration Tests - With CLI_ tests require a CLI instance in order to run. There are several environment variables you can use to configure this.
From inside of VSCode, open the `launch.json` file and in the _Launch Integration Tests - With CLI_ uncomment and change the environment variables appropriate for your purpose.
## Releasing (write access required)
1. Double-check the `CHANGELOG.md` contains all desired change comments and has the version to be released with date at the top.
* Go through all recent PRs and make sure they are properly accounted for.
* Make sure all changelog entries have links back to their PR(s) if appropriate.
1. Double-check that the extension `package.json` and `package-lock.json` have the version you intend to release. If you are doing a patch release (as opposed to minor or major version) this should already be correct.
1. Create a PR for this release:
* This PR will contain any missing bits from steps 1 and 2. Most of the time, this will just be updating `CHANGELOG.md` with today's date.
* Create a new branch for the release named after the new version. For example: `v1.3.6`
* Create a new commit with a message the same as the branch name.
* Create a PR for this branch.
* Wait for the PR to be merged into `main`
1. Trigger a release build on Actions by adding a new tag on branch `main` named after the release, as above. Note that when you push to upstream, you will need to fully qualify the ref. A command like this will work:
```bash
git push upstream refs/tags/v1.3.6
```
* **IMPORTANT** Make sure you are on the `main` branch and your local checkout is fully updated when you add the tag.
* If you accidentally add the tag to the wrong ref, you can just force push it to the right one later.
1. Monitor the status of the release build in the `Release` workflow in the Actions tab.
1. Download the VSIX from the draft GitHub release at the top of [the releases page](https://github.com/github/vscode-codeql/releases) that is created when the release build finishes.
1. Unzip the `.vsix` and inspect its `package.json` to make sure the version is what you expect,
or look at the source if there's any doubt the right code is being shipped.
1. Install the `.vsix` file into your vscode IDE and ensure the extension can load properly. Run a single command (like run query, or add database).
1. Go to the actions tab of the vscode-codeql repository and select the [Release workflow](https://github.com/github/vscode-codeql/actions?query=workflow%3ARelease).
- If there is an authentication failure when publishing, be sure to check that the authentication keys haven't expired. See below.
1. Approve the deployments of the correct Release workflow. This will automatically publish to Open VSX and VS Code Marketplace.
1. Go to the draft GitHub release in [the releases tab of the repository](https://github.com/github/vscode-codeql/releases), click 'Edit', add some summary description, and publish it.
1. Confirm the new release is marked as the latest release at <https://github.com/github/vscode-codeql/releases>.
1. If documentation changes need to be published, notify documentation team that release has been made.
1. Review and merge the version bump PR that is automatically created by Actions.
## Secrets and authentication for publishing
Repository administrators, will need to manage the authentication keys for publishing to the VS Code marketplace and Open VSX. Each requires an authentication token. The VS Code marketplace token expires yearly.
To regenerate the Open VSX token:
1. Log in to the [user settings page on Open VSX](https://open-vsx.org/user-settings/namespaces).
1. Make sure you are a member of the GitHub namespace.
1. Go to the [Access Tokens](https://open-vsx.org/user-settings/tokens) page and generate a new token.
1. Update the secret in the `publish-open-vsx` environment in the project settings.
To regenerate the VSCode Marketplace token, please see our internal documentation. Note that Azure DevOps PATs expire every 90 days and must be regenerated.
[Information about testing can be found here](./docs/testing.md).
@@ -7,7 +7,7 @@ The extension is released. You can download it from the [Visual Studio Marketpla
To see what has changed in the last few versions of the extension, see the [Changelog](https://github.com/github/vscode-codeql/blob/main/extensions/ql-vscode/CHANGELOG.md).
[](https://github.com/github/vscode-codeql/actions?query=workflow%3A%22Build+Extension%22+branch%3Amaster)
@@ -15,11 +15,19 @@ To see what has changed in the last few versions of the extension, see the [Chan
* Shows the flow of data through the results of path queries, which is essential for triaging security results.
* Provides an easy way to run queries from the large, open source repository of [CodeQL security queries](https://github.com/github/codeql).
* Adds IntelliSense to support you writing and editing your own CodeQL query and library files.
* Supports you running CodeQL queries against thousands of repositories on GitHub using multi-repository variant analysis.
## Project goals and scope
This project will track new feature development in CodeQL and, whenever appropriate, bring that functionality to the Visual Studio Code experience.
## Dependencies
This extension depends on the following two extensions for required functionality. They will be installed automatically when you install VS Code CodeQL.
1. Double-check the `CHANGELOG.md` contains all desired change comments and has the version to be released with date at the top.
* Go through all recent PRs and make sure they are properly accounted for.
* Make sure all changelog entries have links back to their PR(s) if appropriate.
* For picking the new version number, we default to increasing the patch version number, but make our own judgement about whether a change is big enough to warrant a minor version bump. Common reasons for a minor bump could include:
* Making substantial new features available to all users. This can include lifting a feature flag.
* Breakage in compatibility with recent versions of the CLI.
* Minimum required version of VS Code is increased.
* New telemetry events are added.
* Deprecation or removal of commands.
* Accumulation of many changes, none of which are individually big enough to warrant a minor bump, but which together are. This does not include changes which are purely internal to the extension, such as refactoring, or which are only available behind a feature flag.
1. Double-check that the node version we're using matches the one used for VS Code. If it doesn't, you will then need to update the node version in the following files:
*`.nvmrc` - this will enable `nvm` to automatically switch to the correct node version when you're in the project folder
*`.github/workflows/main.yml` - all the "node-version: '[VERSION]'" settings
*`.github/workflows/release.yml` - the "node-version: '[VERSION]'" setting
1. Double-check that the extension `package.json` and `package-lock.json` have the version you intend to release. If you are doing a patch release (as opposed to minor or major version) this should already be correct.
1. Create a PR for this release:
* This PR will contain any missing bits from steps 1, 2 and 3. Most of the time, this will just be updating `CHANGELOG.md` with today's date.
* Create a new branch for the release named after the new version. For example: `v1.3.6`
* Create a new commit with a message the same as the branch name.
* Create a PR for this branch.
* Wait for the PR to be merged into `main`
1. Switch to `main` branch and pull latest changes
1. Lock the `main` branch.
* Go to the [branch protection rules for the `main` branch](https://github.com/github/vscode-codeql/settings/branch_protection_rules/16447115)
* Select "Lock branch"
* Click "Save changes"
1. Ensure that no PRs have been merged since the release PR that you merged. If there were, you might need to unlock `main` temporarily and update the CHANGELOG again.
1. Build the extension `npm run build` and install it on your VS Code using "Install from VSIX".
1. Go through [our test plan](./test-plan.md) to ensure that the extension is working as expected.
1. Switch to `main` and add a new tag on the `main` branch with your new version (named after the release), e.g.
```bash
git checkout main
git tag v1.3.6
```
If you've accidentally created a badly named tag, you can delete it via
```bash
git tag -d badly-named-tag
```
1. Unlock the main branch
* Go to the [branch protection rules for the `main` branch](https://github.com/github/vscode-codeql/settings/branch_protection_rules/16447115)
* Deselect "Lock branch"
* Click "Save changes"
1. Push the new tag up:
a. If you're using a fork of the repo:
```bash
git push upstream refs/tags/v1.3.6
```
b. If you're working straight in this repo:
```bash
git push origin refs/tags/v1.3.6
```
This will trigger [a release build](https://github.com/github/vscode-codeql/releases) on Actions.
* **IMPORTANT** Make sure you are on the `main` branch and your local checkout is fully updated when you add the tag.
* If you accidentally add the tag to the wrong ref, you can just force push it to the right one later.
1. Monitor the status of the release build in the `Release` workflow in the Actions tab.
* DO NOT approve the "publish" stages of the workflow yet.
1. Download the VSIX from the draft GitHub release at the top of [the releases page](https://github.com/github/vscode-codeql/releases) that is created when the release build finishes.
1. Unzip the `.vsix` and inspect its `package.json` to make sure the version is what you expect,
or look at the source if there's any doubt the right code is being shipped.
1. Install the `.vsix` file into your vscode IDE and ensure the extension can load properly. Run a single command (like run query, or add database).
1. Go to the actions tab of the vscode-codeql repository and select the [Release workflow](https://github.com/github/vscode-codeql/actions?query=workflow%3ARelease).
* If there is an authentication failure when publishing, be sure to check that the authentication keys haven't expired. See below.
1. Approve the deployments of the correct Release workflow. This will automatically publish to Open VSX and VS Code Marketplace.
1. Go to the draft GitHub release in [the releases tab of the repository](https://github.com/github/vscode-codeql/releases), click 'Edit', add some summary description, and publish it.
1. Confirm the new release is marked as the latest release at <https://github.com/github/vscode-codeql/releases>.
1. If documentation changes need to be published, notify documentation team that release has been made.
1. Review and merge the version bump PR that is automatically created by Actions.
## Secrets and authentication for publishing
Repository administrators, will need to manage the authentication keys for publishing to the VS Code marketplace and Open VSX. Each requires an authentication token. The VS Code marketplace token expires yearly.
To regenerate the Open VSX token:
1. Log in to the [user settings page on Open VSX](https://open-vsx.org/user-settings/namespaces).
1. Make sure you are a member of the GitHub namespace.
1. Go to the [Access Tokens](https://open-vsx.org/user-settings/tokens) page and generate a new token.
1. Update the secret in the `publish-open-vsx` environment in the project settings.
To regenerate the VSCode Marketplace token, please see our internal documentation. Note that Azure DevOps PATs expire every 90 days and must be regenerated.
* Unit tests: these live in the `tests/unit-tests/` directory
* View tests: these live in `src/view/variant-analysis/__tests__/`
* VSCode integration tests:
*`test/vscode-tests/activated-extension` tests: These are intended to cover functionality that require the full extension to be activated but don't require the CLI. This suite is not run against multiple versions of the CLI in CI.
*`test/vscode-tests/no-workspace` tests: These are intended to cover functionality around not having a workspace. The extension is not activated in these tests.
*`test/vscode-tests/minimal-workspace` tests: These are intended to cover functionality that need a workspace but don't require the full extension to be activated.
* CLI integration tests: these live in `test/vscode-tests/cli-integration`
* These tests are intended to cover functionality that is related to the integration between the CodeQL CLI and the extension. These tests are run against each supported versions of the CLI in CI.
The CLI integration tests require an instance of the CodeQL CLI to run so they will require some extra setup steps. When adding new tests to our test suite, please be mindful of whether they need to be in the cli-integration folder. If the tests don't depend on the CLI, they are better suited to being a VSCode integration test.
Any test data you're using (sample projects, config files, etc.) must go in a `test/vscode-tests/*/data` directory. When you run the tests, the test runner will copy the data directory to `out/vscode-tests/*/data`.
## Running the tests
Pre-requisites:
1. Run `npm run build`.
2. You will need to have `npm run watch` running in the background.
### 1. From the terminal
Then, from the `extensions/ql-vscode` directory, use the appropriate command to run the tests:
* Unit tests: `npm run test:unit`
* View Tests: `npm test:view`
* VSCode integration tests: `npm run test:vscode-integration`
#### Running CLI integration tests from the terminal
The CLI integration tests require the CodeQL standard libraries in order to run so you will need to clone a local copy of the `github/codeql` repository.
1. Set the `TEST_CODEQL_PATH` environment variable: running from a terminal, you _must_ set the `TEST_CODEQL_PATH` variable to point to a checkout of the `github/codeql` repository. The appropriate CLI version will be downloaded as part of the test.
2. Run your test command:
```shell
cd extensions/ql-vscode && npm run test:cli-integration
```
### 2. From VSCode
Alternatively, you can run the tests inside of VSCode. There are several VSCode launch configurations defined that run the unit and integration tests.
You will need to run tests using a task from inside of VS Code, under the "Run and Debug" view:
* Unit tests: run the _Launch Unit Tests - React_ task
* View Tests: run the _Launch Unit Tests_ task
* VSCode integration tests: run the _Launch Unit Tests - No Workspace_ and _Launch Unit Tests - Minimal Workspace_ tasks
#### Running CLI integration tests from VSCode
The CLI integration tests require the CodeQL standard libraries in order to run so you will need to clone a local copy of the `github/codeql` repository.
1. Set the `TEST_CODEQL_PATH` environment variable: running from a terminal, you _must_ set the `TEST_CODEQL_PATH` variable to point to a checkout of the `github/codeql` repository. The appropriate CLI version will be downloaded as part of the test.
2. Set the codeql path in VSCode's launch configuration: open `launch.json` and under the _Launch Integration Tests - With CLI_ section, uncomment the `"${workspaceRoot}/../codeql"` line. If you've cloned the `github/codeql` repo to a different path, replace the value with the correct path.
3. Run the VSCode task from the "Run and Debug" view called _Launch Integration Tests - With CLI_.
## Running a single test
### 1. Running a single test from the terminal
The easiest way to run a single test is to change the `it` of the test to `it.only` and then run the test command with some additional options
to only run tests for this specific file. For example, to run the test `test/vscode-tests/cli-integration/run-queries.test.ts`:
```shell
npm run test:cli-integration -- --runTestsByPath test/vscode-tests/cli-integration/run-queries.test.ts
```
You can also use the `--testNamePattern` option to run a specific test within a file. For example, to run the test `test/vscode-tests/cli-integration/run-queries.test.ts`:
```shell
npm run test:cli-integration -- --runTestsByPath test/vscode-tests/cli-integration/run-queries.test.ts --testNamePattern "should create a QueryEvaluationInfo"
```
### 2. Running a single test from VSCode
Alternatively, you can run a single test inside VSCode. To do so, install the [Jest Runner](https://marketplace.visualstudio.com/items?itemName=firsttris.vscode-jest-runner) extension. Then,
you will have quicklinks to run a single test from within test files. To run a single unit or integration test, click the "Run" button. Debugging a single test is currently only supported
for unit tests by default. To debug integration tests, open the `.vscode/settings.json` file and uncomment the `jestrunner.debugOptions` lines. This will allow you to debug integration tests.
Please make sure to revert this change before committing; with this setting enabled, it is not possible to debug unit tests.
Without the Jest Runner extension, you can also use the "Launch Selected Unit Test (vscode-codeql)" launch configuration to run a single unit test.
## Using a mock GitHub API server
Multi-Repo Variant Analyses (MRVA) rely on the GitHub API. In order to make development and testing easy, we have functionality that allows us to intercept requests to the GitHub API and provide mock responses.
### Using a pre-recorded test scenario
To run a mock MRVA scenario, follow these steps:
1. Enable the mock GitHub API server by adding the following in your VS Code user settings (which can be found by running the `Preferences: Open User Settings (JSON)` VS Code command):
```json
"codeQL.mockGitHubApiServer":{
"enabled":true
}
```
1. Run the `CodeQL: Mock GitHub API Server: Load Scenario` command from the command pallet, and choose one of the scenarios to load.
1. Execute a normal MRVA. At this point you should see the scenario being played out, rather than an actual MRVA running.
1. Once you're done, you can stop using the mock scenario with `CodeQL: Mock GitHub API Server: Unload Scenario`
If you want to replay the same scenario you should unload and reload it so requests are replayed from the start.
### Recording a new test scenario
To record a new mock MRVA scenario, follow these steps:
1. Enable the mock GitHub API server by adding the following in your VS Code user settings (which can be found by running the `Preferences: Open User Settings (JSON)` VS Code command):
```json
"codeQL.mockGitHubApiServer":{
"enabled":true
}
```
1. Run the `CodeQL: Mock GitHub API Server: Start Scenario Recording` VS Code command from the command pallet.
1. Execute a normal MRVA.
1. Once what you wanted to record is done (e.g. the MRVA has finished), then run the `CodeQL: Mock GitHub API Server: Save Scenario` command from the command pallet.
1. The scenario should then be available for replaying.
If you want to cancel recording, run the `CodeQL: Mock GitHub API Server: Cancel Scenario Recording` command.
Once the scenario has been recorded, it's often useful to remove some of the requests to speed up the replay, particularly ones that fetch the variant analysis status. Once some of the request files have manually been removed, the [fix-scenario-file-numbering script](../extensions/ql-vscode/scripts/fix-scenario-file-numbering.ts) can be used to update the number of the files. See the script file for details on how to use.
### Scenario data location
Pre-recorded scenarios are stored in `./src/mocks/scenarios`. However, it's possible to configure the location, by setting the `codeQL.mockGitHubApiServer.scenariosPath` configuration property in the VS Code user settings.
- Fix bug where users could end up with the mananged CodeQL CLI getting uninstalled during upgrades and not reinstalled. [#2294](https://github.com/github/vscode-codeql/pull/2294)
- Fix bug that was causing code flows to not get updated when switching between results. [#2288](https://github.com/github/vscode-codeql/pull/2288)
- Restart the CodeQL language server whenever the _CodeQL: Restart Query Server_ command is invoked. This avoids bugs where the CLI version changes to support new language features, but the language server is not updated. [#2238](https://github.com/github/vscode-codeql/pull/2238)
## 1.8.1 - 23 March 2023
- Show data flow paths of a variant analysis in a new tab. [#2172](https://github.com/github/vscode-codeql/pull/2172) & [#2182](https://github.com/github/vscode-codeql/pull/2182)
- Show labels of entities in exported CSV results. [#2170](https://github.com/github/vscode-codeql/pull/2170)
## 1.8.0 - 9 March 2023
- Send telemetry about unhandled errors happening within the extension. [#2125](https://github.com/github/vscode-codeql/pull/2125)
- Enable collection of telemetry concerning interactions with UI elements, including buttons, links, and other inputs. [#2114](https://github.com/github/vscode-codeql/pull/2114)
- Prevent the automatic installation of CodeQL CLI version 2.12.3 to avoid a bug in the language server. CodeQL CLI 2.12.2 will be used instead. [#2126](https://github.com/github/vscode-codeql/pull/2126)
## 1.7.10 - 23 February 2023
- Fix bug that was causing unwanted error notifications.
## 1.7.9 - 20 February 2023
No user facing changes.
## 1.7.8 - 2 February 2023
- Renamed command "CodeQL: Run Query" to "CodeQL: Run Query on Selected Database". [#1962](https://github.com/github/vscode-codeql/pull/1962)
- Remove support for CodeQL CLI versions older than 2.7.6. [#1788](https://github.com/github/vscode-codeql/pull/1788)
## 1.7.7 - 13 December 2022
- Increase the required version of VS Code to 1.67.0. [#1662](https://github.com/github/vscode-codeql/pull/1662)
## 1.7.6 - 21 November 2022
- Warn users when their VS Code version is too old to support all features in the vscode-codeql extension. [#1674](https://github.com/github/vscode-codeql/pull/1674)
## 1.7.5 - 8 November 2022
- Fix a bug where the AST Viewer was not working unless the associated CodeQL library pack is in the workspace. [#1735](https://github.com/github/vscode-codeql/pull/1735)
## 1.7.4 - 29 October 2022
No user facing changes.
## 1.7.3 - 28 October 2022
- Fix a bug where databases may be lost if VS Code is restarted while the extension is being started up. [#1638](https://github.com/github/vscode-codeql/pull/1638)
- Add commands for navigating up, down, left, or right in the result viewer. Previously there were only commands for moving up and down the currently-selected path. We suggest binding keyboard shortcuts to these commands, for navigating the result viewer using the keyboard. [#1568](https://github.com/github/vscode-codeql/pull/1568)
## 1.7.2 - 14 October 2022
- Fix a bug where results created in older versions were thought to be unsuccessful. [#1605](https://github.com/github/vscode-codeql/pull/1605)
## 1.7.1 - 12 October 2022
- Fix a bug where it was not possible to add a database folder if the folder name starts with `db-`. [#1565](https://github.com/github/vscode-codeql/pull/1565)
- Ensure the results view opens in an editor column beside the currently active editor. [#1557](https://github.com/github/vscode-codeql/pull/1557)
## 1.7.0 - 20 September 2022
- Remove ability to download databases from LGTM. [#1467](https://github.com/github/vscode-codeql/pull/1467)
- Remove the ability to manually upgrade databases from the context menu on databases. Databases are non-destructively upgraded automatically so for most users this was not needed. For advanced users this is still available in the Command Palette. [#1501](https://github.com/github/vscode-codeql/pull/1501)
- Always restart the query server after a manual database upgrade. This avoids a bug in the query server where an invalid dbscheme was being retained in memory after an upgrade. [#1519](https://github.com/github/vscode-codeql/pull/1519)
## 1.6.12 - 1 September 2022
- Add ability for users to download databases directly from GitHub. [#1485](https://github.com/github/vscode-codeql/pull/1485)
- Fix a race condition that could cause a failure to open the evaluator log when running a query. [#1490](https://github.com/github/vscode-codeql/pull/1490)
- Fix an error when running a query with an older version of the CodeQL CLI. [#1490](https://github.com/github/vscode-codeql/pull/1490)
## 1.6.11 - 25 August 2022
No user facing changes.
## 1.6.10 - 9 August 2022
No user facing changes.
## 1.6.9 - 20 July 2022
No user facing changes.
## 1.6.8 - 29 June 2022
- Fix a bug where quick queries cannot be compiled if the core libraries are not in the workspace. [#1411](https://github.com/github/vscode-codeql/pull/1411)
- Fix a bug where quick evaluation of library files would display an error message when using CodeQL CLI v2.10.0. [#1412](https://github.com/github/vscode-codeql/pull/1412)
## 1.6.7 - 15 June 2022
- Prints end-of-query evaluator log summaries to the Query Log. [#1349](https://github.com/github/vscode-codeql/pull/1349)
- Be consistent about casing in Query History menu. [#1369](https://github.com/github/vscode-codeql/pull/1369)
- Fix quoting string columns in exported CSV results. [#1379](https://github.com/github/vscode-codeql/pull/1379)
## 1.6.6 - 17 May 2022
No user facing changes.
## 1.6.5 - 25 April 2022
- Re-enable publishing to open-vsx. [#1285](https://github.com/github/vscode-codeql/pull/1285)
@@ -16,17 +16,19 @@ For information about other configurations, see the separate [CodeQL help](https
### Quick start: Installing and configuring the extension
1. [Install the extension](#installing-the-extension).
1. [Install the extension](#installing-the-extension).
*Note: vscode-codeql installs the following dependencies for required functionality: [Test Adapter Converter](https://marketplace.visualstudio.com/items?itemName=ms-vscode.test-adapter-converter), [Test Explorer UI](https://marketplace.visualstudio.com/items?itemName=hbenl.vscode-test-explorer).*
1. [Check access to the CodeQL CLI](#checking-access-to-the-codeql-cli).
1. [Clone the CodeQL starter workspace](#cloning-the-codeql-starter-workspace).
### Quick start: Using CodeQL
1. [Import a database from LGTM](#importing-a-database-from-lgtm).
1. [Import a database from GitHub](#importing-a-database-from-github).
1. [Run a query](#running-a-query).
---
<!-- markdownlint-disable-next-line MD024 -->
## Quick start: Installing and configuring the extension
### Installing the extension
@@ -69,35 +71,41 @@ in the starter workspace directory.
If you're using your own clone of the CodeQL standard libraries, you can do a `git pull` from where you have the libraries checked out.
<!-- markdownlint-disable-next-line MD024 -->
## Quick start: Using CodeQL
You can find all the commands contributed by the extension in the Command Palette (**Ctrl+Shift+P** or **Cmd+Shift+P**) by typing `CodeQL`, many of them are also accessible through the interface, and via keyboard shortcuts.
### Importing a database from LGTM
### Importing a database from GitHub
While you can use the [CodeQL CLI to create your own databases](https://codeql.github.com/docs/codeql-cli/creating-codeql-databases/), the simplest way to start is by downloading a database from LGTM.com.
While you can use the [CodeQL CLI to create your own databases](https://codeql.github.com/docs/codeql-cli/creating-codeql-databases/), the simplest way to start is by downloading a database from GitHub.com.
1.Open [LGTM.com](https://lgtm.com/#explore) in your browser.
1.Search for a project you're interested in, for example [Apache Kafka](https://lgtm.com/projects/g/apache/kafka).
1.Copy the link to that project, for example `https://lgtm.com/projects/g/apache/kafka`.
1. In VS Code, open the Command Palette and choose the **CodeQL: Download Database from LGTM** command.
1.Find a project that you're interested in on GitHub.com, for example [Apache Kafka](https://github.com/apache/kafka).
1.Copy the link to that project, for example `https://github.com/apache/kafka`.
1.In VS Code, open the Command Palette and choose the **CodeQL: Download Database from GitHub** command.
1. Paste the link you copied earlier.
1. Select the language for the database you want to download (only required if the project has databases for multiple languages).
1. Once the CodeQL database has been imported, it is displayed in the Databases view.
For more information, see [Choosing a database](https://codeql.github.com/docs/codeql-for-visual-studio-code/analyzing-your-projects/#choosing-a-database) on codeql.github.com.
### Running a query
The instructions below assume that you're using the CodeQL starter workspace, or that you've added the CodeQL libraries and queries repository to your workspace.
1. Expand the `ql` folder and locate a query to run. The standard queries are grouped by target language and then type, for example: `ql/java/ql/src/Likely Bugs`.
1. Open a query (`.ql`) file.
1. Right-click in the query window and select **CodeQL: Run Query**. Alternatively, open the Command Palette (**Ctrl+Shift+P** or **Cmd+Shift+P**), type `Run Query`, then select **CodeQL: Run Query**.
1. Right-click in the query window and select **CodeQL: Run Query on Selected Database**. Alternatively, open the Command Palette (**Ctrl+Shift+P** or **Cmd+Shift+P**), type `Run Query`, then select **CodeQL: Run Query on Selected Database**.
The CodeQL extension runs the query on the current database using the CLI and reports progress in the bottom right corner of the application.
When the results are ready, they're displayed in the CodeQL Query Results view. Use the dropdown menu to choose between different forms of result output.
If there are any problems running a query, a notification is displayed in the bottom right corner of the application. In addition to the error message, the notification includes details of how to fix the problem.
### Keyboard navigation
If you wish to navigate the query results from your keyboard, you can bind shortcuts to the **CodeQL: Navigate Up/Down/Left/Right in Result Viewer** commands.
## What next?
For more information about the CodeQL extension, [see the documentation](https://codeql.github.com/docs/codeql-for-visual-studio-code/). Otherwise, you could:
<pathd="M 35.300905,316.97546 H 93.308718 V 116.76062 L 30.203249,129.41687 V 97.07312 L 92.957155,84.41687 h 35.507815 v 232.55859 h 58.00781 v 29.88282 H 35.300905 Z"fill="#C5C5C5"/>
<pathd="M 35.300905,316.97546 H 93.308718 V 116.76062 L 30.203249,129.41687 V 97.07312 L 92.957155,84.41687 h 35.507815 v 232.55859 h 58.00781 v 29.88282 H 35.300905 Z"/>
// Ignore the error here, since we catch failures at a higher level.
return;
}
}
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
