Merge pull request #2129 from github/v1.7.11
Some checks failed
Release / Release (push) Has been cancelled
Release / Publish to VS Code Marketplace (push) Has been cancelled
Release / Publish to Open VSX Registry (push) Has been cancelled

v1.7.11
This commit is contained in:
Andrew Eisenberg
2023-03-01 10:57:50 -08:00
committed by GitHub
2 changed files with 79 additions and 55 deletions

View File

@@ -1,10 +1,11 @@
# CodeQL for Visual Studio Code: Changelog # CodeQL for Visual Studio Code: Changelog
## [UNRELEASED] ## 1.7.11 - 1 March 2023
- Enable collection of telemetry concerning interactions with UI elements, including buttons, links, and other inputs. [#2114](https://github.com/github/vscode-codeql/pull/2114) - Enable collection of telemetry concerning interactions with UI elements, including buttons, links, and other inputs. [#2114](https://github.com/github/vscode-codeql/pull/2114)
- Prevent the automatic installation of CodeQL CLI version 2.12.3 to avoid a bug in the language server. CodeQL CLI 2.12.2 will be used instead. [#2126](https://github.com/github/vscode-codeql/pull/2126)
# 1.7.10 - 23 February 2023 ## 1.7.10 - 23 February 2023
- Fix bug that was causing unwanted error notifications. - Fix bug that was causing unwanted error notifications.

View File

@@ -2,15 +2,17 @@
This document describes the manual test plan for the QL extension for Visual Studio Code. This document describes the manual test plan for the QL extension for Visual Studio Code.
The plan will be executed manually to start with but the goal is to eventually automate parts of the process (based on The plan will be executed manually to start with but the goal is to eventually automate parts of the process (based on
effort vs value basis). effort vs value basis).
#### What this doesn't cover ## What this doesn't cover
We don't need to test features (and permutations of features) that are covered by automated tests. We don't need to test features (and permutations of features) that are covered by automated tests.
### Before releasing the VS Code extension ## Before releasing the VS Code extension
- Go through the required test cases listed below - Go through the required test cases listed below
- Check major PRs since the previous release for specific one-off things to test. Based on that, you might want to - Check major PRs since the previous release for specific one-off things to test. Based on that, you might want to
choose to go through some of the Optional Test Cases. choose to go through some of the Optional Test Cases.
- Run a query using the existing version of the extension (to generate an "old" query history item) - Run a query using the existing version of the extension (to generate an "old" query history item)
@@ -24,23 +26,25 @@ choose to go through some of the Optional Test Cases.
1. Open the [UnsafeJQueryPlugin query](https://github.com/github/codeql/blob/main/javascript/ql/src/Security/CWE-079/UnsafeJQueryPlugin.ql). 1. Open the [UnsafeJQueryPlugin query](https://github.com/github/codeql/blob/main/javascript/ql/src/Security/CWE-079/UnsafeJQueryPlugin.ql).
2. Run a MRVA against the following repo list: 2. Run a MRVA against the following repo list:
```
{ ```json
"name": "test-repo-list", {
"repositories": [ "name": "test-repo-list",
"angular-cn/ng-nice", "repositories": [
"apache/hadoop", "angular-cn/ng-nice",
"apache/hive" "apache/hadoop",
] "apache/hive"
} ]
``` }
```
3. Check that a notification message pops up and the results view is opened. 3. Check that a notification message pops up and the results view is opened.
4. Check the query history. It should: 4. Check the query history. It should:
- Show that an item has been added to the query history - Show that an item has been added to the query history
- The item should be marked as "in progress". - The item should be marked as "in progress".
5. Once the query starts: 5. Once the query starts:
- Check the results view - Check the results view
- Check the code paths view, including the code paths drop down menu. - Check the code paths view, including the code paths drop down menu.
- Check that the repository filter box works - Check that the repository filter box works
- Click links to files/locations on GitHub - Click links to files/locations on GitHub
- Check that the query history item is updated to show the number of results - Check that the query history item is updated to show the number of results
@@ -74,7 +78,7 @@ choose to go through some of the Optional Test Cases.
1. Click a history item (for MRVA): 1. Click a history item (for MRVA):
- Check that exporting results works - Check that exporting results works
- Check that sorting results works - Check that sorting results works
- Check that copying repo lists works - Check that copying repo lists works
2. Open the query results directory: 2. Open the query results directory:
- Check that the correct directory is opened and there are results in it - Check that the correct directory is opened and there are results in it
3. View logs 3. View logs
@@ -84,12 +88,12 @@ choose to go through some of the Optional Test Cases.
Run one of the above MRVAs, but cancel it from within VS Code: Run one of the above MRVAs, but cancel it from within VS Code:
- Check that the query is canceled and the query history item is updated. - Check that the query is canceled and the query history item is updated.
- Check that the workflow run is also canceled. - Check that the workflow run is also canceled.
- Check that any available results are visible in VS Code. - Check that any available results are visible in VS Code.
### Test Case 6: MRVA - Change to a different colour theme ### Test Case 6: MRVA - Change to a different colour theme
Open one of the above MRVAs, try changing to a different colour theme and check that everything looks sensible. Open one of the above MRVAs, try changing to a different colour theme and check that everything looks sensible.
Are there any components that are not showing up? Are there any components that are not showing up?
## Optional Test Cases ## Optional Test Cases
@@ -99,9 +103,10 @@ These are mostly aimed at MRVA, but some of them are also applicable to non-MRVA
### Selecting repositories to run on ### Selecting repositories to run on
#### Test case 1: Running a query on a single repository #### Test case 1: Running a query on a single repository
1. When the repository exists and is public
1. Has a CodeQL database for the correct language 1. When the repository exists and is public
2. Has a CodeQL database for another language 1. Has a CodeQL database for the correct language
2. Has a CodeQL database for another language
3. Does not have any CodeQL databases 3. Does not have any CodeQL databases
2. When the repository exists and is private 2. When the repository exists and is private
1. Is accessible and has a CodeQL database 1. Is accessible and has a CodeQL database
@@ -109,14 +114,16 @@ These are mostly aimed at MRVA, but some of them are also applicable to non-MRVA
3. When the repository does not exist 3. When the repository does not exist
#### Test case 2: Running a query on a custom repository list #### Test case 2: Running a query on a custom repository list
1. The repository list is non-empty 1. The repository list is non-empty
1. All repositories in the list have a CodeQL database 1. All repositories in the list have a CodeQL database
2. Some but not all repositories in the list have a CodeQL database 2. Some but not all repositories in the list have a CodeQL database
3. No repositories in the list have a CodeQL database 3. No repositories in the list have a CodeQL database
2. The repository list is empty 2. The repository list is empty
#### Test case 3: Running a query on all repositories in an organization #### Test case 3: Running a query on all repositories in an organization
1. The org exists
1. The org exists
1. The org contains repositories that have CodeQL databases 1. The org contains repositories that have CodeQL databases
2. The org contains repositories of the right language but without CodeQL databases 2. The org contains repositories of the right language but without CodeQL databases
3. The org contains repositories not of the right language 3. The org contains repositories not of the right language
@@ -126,20 +133,25 @@ These are mostly aimed at MRVA, but some of them are also applicable to non-MRVA
### Using different types of controller repos ### Using different types of controller repos
#### Test case 1: Running a query when the controller repository is public #### Test case 1: Running a query when the controller repository is public
1. Can run queries on public repositories 1. Can run queries on public repositories
2. Can not run queries on private repositories 2. Can not run queries on private repositories
#### Test case 2: Running a query when the controller repository is private #### Test case 2: Running a query when the controller repository is private
1. Can run queries on public repositories 1. Can run queries on public repositories
2. Can run queries on private repositories 2. Can run queries on private repositories
#### Test case 3: Running a query when the controller repo exists but you do not have write access #### Test case 3: Running a query when the controller repo exists but you do not have write access
1. Cannot run queries 1. Cannot run queries
#### Test case 4: Running a query when the controller repo doesnt exist #### Test case 4: Running a query when the controller repo doesnt exist
1. Cannot run queries 1. Cannot run queries
#### Test case 5: Running a query when the "config field" for the controller repo is not set #### Test case 5: Running a query when the "config field" for the controller repo is not set
1. Cannot run queries 1. Cannot run queries
### Query History ### Query History
@@ -150,6 +162,7 @@ The first test case specifies actions that you can do when the query is first ru
with this since it has quite a limited number of actions you can do. with this since it has quite a limited number of actions you can do.
#### Test case 1: When variant analysis state is "pending" #### Test case 1: When variant analysis state is "pending"
1. Starts monitoring variant analysis 1. Starts monitoring variant analysis
2. Cannot open query history item 2. Cannot open query history item
3. Can delete a query history item 3. Can delete a query history item
@@ -160,8 +173,8 @@ with this since it has quite a limited number of actions you can do.
2. By query date 2. By query date
3. By result count 3. By result count
5. Cannot open query directory 5. Cannot open query directory
6. Can open query that produced these results 6. Can open query that produced these results
1. When the file still exists and has not moved 1. When the file still exists and has not moved
2. When the file does not exist 2. When the file does not exist
7. Cannot view logs 7. Cannot view logs
8. Cannot copy repository list 8. Cannot copy repository list
@@ -171,6 +184,7 @@ with this since it has quite a limited number of actions you can do.
12. Cannot cancel analysis 12. Cannot cancel analysis
#### Test case 2: When the variant analysis state is not "pending" #### Test case 2: When the variant analysis state is not "pending"
1. Query history is loaded when VSCode starts 1. Query history is loaded when VSCode starts
2. Handles when action workflow was canceled while VSCode was closed 2. Handles when action workflow was canceled while VSCode was closed
3. Can open query history item 3. Can open query history item
@@ -204,12 +218,14 @@ with this since it has quite a limited number of actions you can do.
4. A popup allows you to open the directory 4. A popup allows you to open the directory
#### Test case 3: When variant analysis state is "in_progress" #### Test case 3: When variant analysis state is "in_progress"
1. Starts monitoring variant analysis 1. Starts monitoring variant analysis
1. Ready results are downloaded 1. Ready results are downloaded
2. Can cancel analysis 2. Can cancel analysis
1. Causes the actions run to be canceled 1. Causes the actions run to be canceled
#### Test case 4: When variant analysis state is in final state ("succeeded"/"failed"/"canceled") #### Test case 4: When variant analysis state is in final state ("succeeded"/"failed"/"canceled")
1. Stops monitoring variant analysis 1. Stops monitoring variant analysis
1. All results are downloaded if state is succeeded 1. All results are downloaded if state is succeeded
2. Otherwise, ready results are downloaded, if any are available 2. Otherwise, ready results are downloaded, if any are available
@@ -220,6 +236,7 @@ with this since it has quite a limited number of actions you can do.
This requires running a MRVA query and seeing the results view. This requires running a MRVA query and seeing the results view.
#### Test case 1: When variant analysis state is "pending" #### Test case 1: When variant analysis state is "pending"
1. Can open a results view 1. Can open a results view
2. Results view opens automatically 2. Results view opens automatically
- When starting variant analysis run - When starting variant analysis run
@@ -227,9 +244,10 @@ This requires running a MRVA query and seeing the results view.
3. Results view is empty 3. Results view is empty
#### Test case 2: When variant analysis state is not "pending" #### Test case 2: When variant analysis state is not "pending"
1. Can open a results view 1. Can open a results view
2. Results view opens automatically 2. Results view opens automatically
1. When starting variant analysis run 1. When starting variant analysis run
2. When VSCode opens (if view was open when VSCode was closed) 2. When VSCode opens (if view was open when VSCode was closed)
3. Can copy repository list 3. Can copy repository list
1. Text is copied to clipboard 1. Text is copied to clipboard
@@ -240,43 +258,45 @@ This requires running a MRVA query and seeing the results view.
6. Can open query file 6. Can open query file
1. When the file still exists and has not moved 1. When the file still exists and has not moved
2. When the file does not exist 2. When the file does not exist
7. Can open query text 7. Can open query text
8. Can sort repos 8. Can sort repos
1. By name 1. By name
2. By results 2. By results
3. By stars 3. By stars
4. By last updated 4. By last updated
9. Can filter repos 9. Can filter repos
10. Shows correct statistics 10. Shows correct statistics
1. Total number of results 1. Total number of results
2. Total number of repositories 2. Total number of repositories
3. Duration 3. Duration
11. Can see live results 11. Can see live results
1. Results appear in extension as soon as each query is completed 1. Results appear in extension as soon as each query is completed
12. Can view interpreted results (i.e. for a "problem" query) 12. Can view interpreted results (i.e. for a "problem" query)
1. Can view non-path results 1. Can view non-path results
2. Can view code paths for "path-problem" queries 2. Can view code paths for "path-problem" queries
13. Can view raw results (i.e. for a non "problem" query) 13. Can view raw results (i.e. for a non "problem" query)
1. Renders a table 1. Renders a table
14. Can see skipped repositories 14. Can see skipped repositories
1. Can see repos with no db in a tab 1. Can see repos with no db in a tab
1. Shown warning that explains the tab 1. Shown warning that explains the tab
2. Can see repos with no access in a tab 2. Can see repos with no access in a tab
1. Shown warning that explains the tab 1. Shown warning that explains the tab
3. Only shows tab when there are skipped repos 3. Only shows tab when there are skipped repos
15. Result downloads 15. Result downloads
1. All results are downloaded automatically 1. All results are downloaded automatically
2. Download status is indicated by a spinner (Not currently any indication of progress beyond "downloading" and "not downloading") 2. Download status is indicated by a spinner (Not currently any indication of progress beyond "downloading" and "not downloading")
3. Only 3 items are downloaded at a time 3. Only 3 items are downloaded at a time
4. Results for completed queries are still downloaded when 4. Results for completed queries are still downloaded when
1. Some but not all queries failed 1. Some but not all queries failed
2. The variant analysis was canceled after some queries completed 2. The variant analysis was canceled after some queries completed
#### Test case 3: When variant analysis state is in "succeeded" state #### Test case 3: When variant analysis state is in "succeeded" state
1. Can view logs 1. Can view logs
2. All results are downloaded 2. All results are downloaded
#### Test case 4: When variant analysis is in "failed" or "canceled" state #### Test case 4: When variant analysis is in "failed" or "canceled" state
1. Can view logs 1. Can view logs
1. Results for finished queries are still downloaded. 1. Results for finished queries are still downloaded.
@@ -305,14 +325,17 @@ This requires running a MRVA query and seeing the results view.
1. Collapse/expand tree nodes 1. Collapse/expand tree nodes
Error cases that trigger an error notification: Error cases that trigger an error notification:
1. Try to add a list with a name that already exists
1. Try to add a list with a name that already exists
1. Try to add a top-level database that already exists 1. Try to add a top-level database that already exists
1. Try to add a database in a list that already exists in the list 1. Try to add a database in a list that already exists in the list
Error cases that show an error in the panel (and only the edit button should be visible): Error cases that show an error in the panel (and only the edit button should be visible):
1. Edit the db config file directly and save invalid JSON 1. Edit the db config file directly and save invalid JSON
1. Edit the db config file directly and save valid JSON but invalid config (e.g. add an unknown property) 1. Edit the db config file directly and save valid JSON but invalid config (e.g. add an unknown property)
1. Edit the db config file directly and save two lists with the same name 1. Edit the db config file directly and save two lists with the same name
Cases where there the welcome view is shown: Cases where there the welcome view is shown:
1. No controller repo is set in the user's settings JSON.
1. No controller repo is set in the user's settings JSON.