Commit Graph

4090 Commits

Author SHA1 Message Date
Elena Tanasoiu
c9f7860621 Move try/catch into execute command
So that we're able to catch any errors when this fails.
2023-04-12 16:52:04 +00:00
Elena Tanasoiu
eff5f1f197 Fix regex expression 2023-04-12 16:52:04 +00:00
Elena Tanasoiu
9f04c712e0 Use progress correctly
By setting maxSteps to the correct number of steps.
2023-04-12 16:52:04 +00:00
Elena Tanasoiu
e298f2ba09 Take storage path off App instead of extension context 2023-04-12 16:39:14 +00:00
Elena Tanasoiu
b4468d612a Download databases into extension storage path
Now that we've figured out how to set the storage path for both Mac &
Windows, we also want to make sure we're consistent with the location
where we save databases.

At the moment, our change will download databases directly in the
workspace folder.

When we call `downloadGitHubDatabase()` in other places outside the
wizard, we provide `ctx.storageUri.fsPath` as the location. [1] [2] [3]

Let's do the same here.

I've tested this on Mac for the codespaces-codeql & starter workspaces.

[1]: c7bb22c312/extensions/ql-vscode/src/local-databases-ui.ts (L476)
[2]: c7bb22c312/extensions/ql-vscode/src/extension.ts (L710)
[3]: c7bb22c312/extensions/ql-vscode/src/extension.ts (L1120)
2023-04-12 16:30:41 +00:00
Elena Tanasoiu
785c8c1f22 Rename storagePath -> qlPackStoragePath 2023-04-12 14:25:43 +00:00
shati-patel
4b01374a6f Use path.join in folder paths 2023-04-12 14:12:33 +01:00
shati-patel
a968aab209 Remove explicit Windows test
This should be covered by running the general test suite in CI (on windows-latest)
2023-04-12 13:26:23 +01:00
shati-patel
1e1594b2a7 Use fsPath instead of path everywhere 2023-04-12 13:14:20 +01:00
shati-patel
636be05f2b Fix "invalid characters" bug on Windows
We now use `fsPath` instead of `path`.

Note: I haven't yet fixed the tests, nor checked manually on mac/linux

Tangential change: we now use the `dirname` method, instead of manually splitting paths to get a parent folder.
2023-04-12 12:51:13 +01:00
Elena Tanasoiu
3087886400 Fix problem with detecting storage folder on windows
The `getFirstStoragePath()` method would break on windows:

```
Path contains invalid characters: /c:/git-repo/codespaces-codeql (codeQL.createSkeletonQuery)
```

This makes sense, since we're looking to get the parent folder by splitting for `/`.

In windows, paths use `\` instead of `/`.

So let's detect the platform and add a test for this case.
2023-04-12 09:47:00 +00:00
Elena Tanasoiu
a6ffb6b020 Move back mockCli into beforeEach
This was an attempt to make the tests run in a shorter timeframe, but
we've gone with increasing the timeout to 40s.
2023-04-12 09:38:43 +00:00
Elena Tanasoiu
72b2080b02 Rename search methods
To be consistent with other database item search methods, we're renaming
ours:

`digForDatabaseItem` -> `findDatabaseItemByNwo`
and
`digForDatabaseItemSameLanguage` -> `findDatabaseItemByLanguage`
2023-04-12 09:38:43 +00:00
Elena Tanasoiu
6b26323ee0 Move database Item search methods into skeleton wizard
And the tests for it. We can now re-use the factory from the previous commit.
2023-04-12 09:38:43 +00:00
Elena Tanasoiu
0ff2e83fde Move db mocking into factory file
So we can re-use it.
2023-04-12 09:38:43 +00:00
Elena Tanasoiu
de4afae0a0 Don't track when user doesn't provide repo 2023-04-12 09:17:01 +00:00
Elena Tanasoiu
389d8f04d9 Don't show progress when choosing repo 2023-04-12 09:17:01 +00:00
Elena Tanasoiu
919219c084 Show the user a selection box before downloading database
At the moment, we're always deciding which database to download for the
user for an example query.

We'd like to give them a chance to change the database, so here we're
adding a step where we're showing the user a selection box with the
suggested database pre-filled.

They can choose to type in a different database before continuing the
skeleton generation process.
2023-04-11 16:54:09 +00:00
Elena Tanasoiu
3e3eb0da0a Re-use existing database if one for the same language is already downloaded
We'd like to select an existing database for our query, if on is
already downloaded and matches the query language.

Previously we were re-using the database if the language and name
matched (e.g. the name would be `github/codeql`).
2023-04-11 16:53:00 +00:00
Elena Tanasoiu
460da1ea08 Add language to list of params 2023-04-11 15:00:00 +00:00
Elena Tanasoiu
f4a8de0f6b Don't count files that are not example<number>.ql
When we try to determine the next file name for our example query,
we only look at `example<n>.ql` files.

e.g. if the files in the folder are:
- `example.ql`
- `example2.ql`
- `MyQuery.ql`

we will create an `example3.ql` file.

Previously we were counting all existing `.ql` files.
2023-04-11 15:00:00 +00:00
Elena Tanasoiu
c7d9407a4c Rename workoutNextFileName -> determineNextFileName 2023-04-11 15:00:00 +00:00
Elena Tanasoiu
913942625c Increase timeout for skeleton wizard tests
We've now added more tests and pushed the total duration over 5 seconds
for all the tests in this file.

This limitation seems to be a recent development where files with tests
that last longer than 5 seconds start failing in jest.

We're bumping the timeout limit to 40 seconds for now.
2023-04-11 14:59:59 +00:00
Elena Tanasoiu
62bebc02bd Set storage path after the user selects language
And add tests for getFirstStoragePath method
2023-04-11 14:59:59 +00:00
Elena Tanasoiu
16a828938f Shorten logic for deciding when to ask for language
Let's only ask for the language when the language is not provided OR it's invalid.

Let's also add tests for these cases.
2023-04-11 14:59:59 +00:00
Elena Tanasoiu
59909e20b0 Convert folderName property into getter method 2023-04-11 14:59:59 +00:00
Elena Tanasoiu
b794427778 Remove unnecessary casting to DatabaseItem 2023-04-11 14:59:59 +00:00
Elena Tanasoiu
97fb4ea2ed Use Record 2023-04-11 14:59:59 +00:00
Elena Tanasoiu
911c3af225 Use a regular const instead of storing this on the class 2023-04-11 14:59:59 +00:00
Elena Tanasoiu
f949eda5ff Fix error message 2023-04-11 14:59:59 +00:00
Elena Tanasoiu
9c0deaec96 Make it clear we're checking the filename
Replaces `file[0]` with a more meaningful `filename`.
2023-04-11 14:59:58 +00:00
Elena Tanasoiu
fe3e9a713b Get rid of unnecessary ternary 2023-04-11 14:59:58 +00:00
Elena Tanasoiu
30011aaf65 Fail gracefully if we can't open the new query file 2023-04-11 14:59:58 +00:00
Elena Tanasoiu
061f347ea0 Don't obfuscate assignment when we choose language
Instead of assigning this property in a method, let's make the method
return a value and assign it more visibly.
2023-04-11 14:59:58 +00:00
Elena Tanasoiu
95ed076047 Shorten beforeEach to decrease chances of timeout 2023-04-11 14:59:58 +00:00
Elena Tanasoiu
c23b0bf6c8 Replace github/codeql with top databases for querying
We initially defined the default database to download as one from the
`github/codeql` repo as it was convenient.

However, this repo doesn't have a lot of vulnerabilities to discover.

Let's use repos that are in our MRVA top 10 list to allow users to
write more interesting queries.
2023-04-11 14:59:58 +00:00
Elena Tanasoiu
121ebc24a6 skeleton-query.ts -> skeleton-query-wizard.ts 2023-04-11 14:59:58 +00:00
Elena Tanasoiu
f212804634 Make "Create Query" command work with vscode-codeql-starter
We set up the "Create Query" command with the assumption that
the first folder in the workspace is the parent folder.

This is true for the `codespaces-codeql` repo where we expect
to use this command.

However, for the `vscode-codeql-starter` repo, the top level
folders are QL packs:
- codeql-custom-queries-cpp
- codeql-custom-queries-ruby
... etc.

In order to make the command work for people using the starter
repo, we'll need to introduce a check for these QL packs when
we decide the storage path.

The end goal is to replace the starter workspace completely
with the codespaces-codeql repo, so this code can be removed
in the future when we retire the repo.

Until then, the command will need this to be able to work in
both starter workspaces.
2023-04-11 14:59:57 +00:00
Elena Tanasoiu
b9decd8f80 Create QL pack in workspace instead of global storage
We were initially using the storage path of the extension.

We'd like to create these QL packs in the workspace instead.
2023-04-11 14:59:57 +00:00
Elena Tanasoiu
22d9487ab8 Be able to specify language when downloading github database
We offer `github/codeql` as a repo to use for downloading databases
for our skeleton pack.

Once the repo is specified, the user is prompted to choose a language.

At this point, we already know what language the user wants, so let's
change the `downloadGitHubDatabase` and `convertGithubNwoToDatabaseUrl`
methods to accept a language parameter.

We check if the language is in the list of languages received in the
response. If it isn't, we still prompt the user.
2023-04-11 14:59:57 +00:00
Elena Tanasoiu
91c4c9189f Introduce "Create Query" command 2023-04-11 14:59:57 +00:00
Elena Tanasoiu
ddd00d16b0 Introduce SkeletonWizard class
This will be triggered by a "Create Query" command.

It will:
- prompt the user for a language
- create a skeleton pack based on the language chosen
- download a database for the QL pack
- open the new query file

If the skeleton pack already exists, we just create a new query file
in the existing folder.

If the database is already downloaded, we just re-use it.
2023-04-11 14:59:57 +00:00
Elena Tanasoiu
e63f0fc0af Be able to specify the name of the skeleton query file
We introduced this QlPackGenerator a while ago. It always creates an `example.ql` query file as part of the skeleton pack.

We'd like to set the name of the query file, since we'll allow the user to create queries multiple times in the same skeleton pack folder.

The folder will be named `codeql-custom-queries-${language}` and will first receive an `example.ql` file.

If the user then tries to create a new query for the same language, we'll just create an `example2.ql`, `example3.ql` etc. file in the existing folder.
2023-04-11 14:59:57 +00:00
Elena Tanasoiu
2995b0120d Provide a way to search for database items by name and language
We'll use this to check whether a database for our ql pack already exists.

While there are other methods that search for a database item by URI, we
only have a language chosen by the user and an nwo ("github/codeql").

So let's introduce a way to search for the db based on the information we
have.
2023-04-11 14:59:57 +00:00
Elena Tanasoiu
053a1800a9 Export existing language prompt
We plan to ask the user to choose a language, before attempting
to download a corresponding database for them.

The functionality already exists, so let's re-use it.
2023-04-11 14:59:56 +00:00
Elena Tanasoiu
1a2e5d90df Extract functionality to download GitHub database
This was nested in a method that included prompting the user for a
github repo.

We'd like to re-use this to download a database of our choice from
GitHub, based on which language a user chooses.
2023-04-11 14:59:56 +00:00
Koen Vlaswinkel
957f71c9d4 Merge pull request #2293 from github/koesie10/use-data-extensions-in-editor
Use data extensions for finding external API calls
2023-04-11 16:01:26 +02:00
Koen Vlaswinkel
ae08a1b598 Use data extensions for finding external API calls
The data extension editor was only using the default data extensions
found in the `ql` submodule to find external API calls. This will add
support for using data extensions found in the workspace.

Rather than using the `codeQL.runningQueries.useExtensionPacks` setting,
this will always include data extensions since the editor doesn't make
sense to use without data extensions. We will also forbid the user from
opening this view unless they are using a CLI which supports data
extension packs.
2023-04-11 13:31:51 +02:00
Charis Kyriakou
deb2b83642 Update selected code flow when data flow paths change (#2288) 2023-04-11 11:02:03 +01:00
Koen Vlaswinkel
0115259778 Merge pull request #2271 from github/koesie10/data-extension-editor-cli-tests
Split and add tests for external API usages query
2023-04-11 11:46:29 +02:00