From fc86347b0fbce9ed8a4d4c0d22835b2b7c3064ba Mon Sep 17 00:00:00 2001 From: james Date: Fri, 9 Sep 2022 15:40:10 +0100 Subject: [PATCH] Add section about using the API to download dbs --- .../codeql-cli/creating-codeql-databases.rst | 18 +++++------- .../getting-started-with-the-codeql-cli.rst | 29 +++---------------- .../analyzing-your-projects.rst | 2 ++ .../reusables/download-github-database.rst | 5 ++++ .../reusables/download-lgtm-database.rst | 6 ---- 5 files changed, 19 insertions(+), 41 deletions(-) create mode 100644 docs/codeql/reusables/download-github-database.rst delete mode 100644 docs/codeql/reusables/download-lgtm-database.rst diff --git a/docs/codeql/codeql-cli/creating-codeql-databases.rst b/docs/codeql/codeql-cli/creating-codeql-databases.rst index 62ed161e9b9..c3d29972cde 100644 --- a/docs/codeql/codeql-cli/creating-codeql-databases.rst +++ b/docs/codeql/codeql-cli/creating-codeql-databases.rst @@ -379,17 +379,15 @@ The following example shows how you could use indirect build tracing in an Azure # `codeql database analyze` # then `codeql github upload-results` ... -Obtaining databases from LGTM.com ---------------------------------- - -`LGTM.com `__ analyzes thousands of open-source projects using -CodeQL. For each project on LGTM.com, you can download an archived CodeQL -database corresponding to the most recently analyzed revision of the code. These -databases can also be analyzed using the CodeQL CLI or used with the CodeQL -extension for Visual Studio Code. - -.. include:: ../reusables/download-lgtm-database.rst +Obtaining databases from GitHub.com +----------------------------------- +.. include:: ../reusables/download-github-database.rst + +Before running an analysis, unzip the databases and try :doc:`upgrading ` the +unzipped databases to ensure they are compatible with your local copy of the +CodeQL queries and libraries. + .. pull-quote:: Note diff --git a/docs/codeql/codeql-cli/getting-started-with-the-codeql-cli.rst b/docs/codeql/codeql-cli/getting-started-with-the-codeql-cli.rst index 0fd8e6b8fa0..976aa302fc6 100644 --- a/docs/codeql/codeql-cli/getting-started-with-the-codeql-cli.rst +++ b/docs/codeql/codeql-cli/getting-started-with-the-codeql-cli.rst @@ -47,22 +47,9 @@ Conditions `__. .. pull-quote:: Important - There are different versions of the CLI available to download, depending - on your use case: - - - If you want to use the most up to date CodeQL tools and features, download the - version tagged ``latest``. - - - If you want to create CodeQL databases to upload to LGTM Enterprise, download - the version that is compatible with the relevant LGTM Enterprise version - number. Compatibility information is included in the description for each - release on the `CodeQL CLI releases page - `__ on GitHub. Using the - correct version of the CLI ensures that your CodeQL databases are - compatible with your version of LGTM Enterprise. For more information, - see `Preparing CodeQL databases to upload to LGTM - `__ - in the LGTM admin help. + There are several different versions of the CLI available to download, depending + on your use case. If you want to use the most up to date CodeQL tools and features, download the + version tagged ``latest``. If you use Linux, Windows, or macOS version 10.14 ("Mojave") or earlier, simply `download the zip archive @@ -232,15 +219,7 @@ see ":doc:`About CodeQL packs `." - For the most up to date CodeQL queries, check out the ``main`` branch. This branch represents the very latest version of CodeQL's analysis. - - For the queries used in a particular LGTM Enterprise release, check out the - branch tagged with the relevant release number. For example, the branch - tagged ``v1.27.0`` corresponds to LGTM Enterprise 1.27. You must use this - version if you want to upload data to LGTM Enterprise. For further - information, see `Preparing CodeQL databases to upload to LGTM - `__ - in the LGTM admin help. - -1. Extract the zip archive +4. Extract the zip archive ~~~~~~~~~~~~~~~~~~~~~~~~~~ For Linux, Windows, and macOS users (version 10.14 "Mojave", and earlier) diff --git a/docs/codeql/codeql-for-visual-studio-code/analyzing-your-projects.rst b/docs/codeql/codeql-for-visual-studio-code/analyzing-your-projects.rst index 2b4e26b405a..6a59754eaa5 100644 --- a/docs/codeql/codeql-for-visual-studio-code/analyzing-your-projects.rst +++ b/docs/codeql/codeql-for-visual-studio-code/analyzing-your-projects.rst @@ -37,6 +37,8 @@ If you have a CodeQL database saved locally, as an unarchived folder or as a ZIP For more information about running query tests, see ":ref:`Testing custom queries `" in the CodeQL CLI help. +.. include:: ../reusables/download-github-database.rst + Running a query ------------------------ diff --git a/docs/codeql/reusables/download-github-database.rst b/docs/codeql/reusables/download-github-database.rst new file mode 100644 index 00000000000..c95482fa6a5 --- /dev/null +++ b/docs/codeql/reusables/download-github-database.rst @@ -0,0 +1,5 @@ +GitHub also stores CodeQL databases for over 200,000 repos on GitHub.com, which you can download using the API. The list of repos is constantly growing and evolving to make sure that it includes the most interesting codebases for security research. + +To download a database from GitHub.com using the `GitHub CLI `__, use the following command:: + + gh api /repos///code-scanning/codeql/databases/ -H 'Accept: application/zip' > path/to/local/database.zip diff --git a/docs/codeql/reusables/download-lgtm-database.rst b/docs/codeql/reusables/download-lgtm-database.rst deleted file mode 100644 index 11652d6512a..00000000000 --- a/docs/codeql/reusables/download-lgtm-database.rst +++ /dev/null @@ -1,6 +0,0 @@ -To download a database from LGTM.com: - -#. Log in to `LGTM.com `__. -#. Find a project you're interested in and display the Integrations tab (for example, `Apache Kafka `__). -#. Scroll to the **CodeQL databases for local analysis** section at the bottom of the page. -#. Download databases for the languages that you want to explore. \ No newline at end of file