From 75066813eeab9862cef5d7a49ebcd2608e6ea366 Mon Sep 17 00:00:00 2001 From: hubwriter Date: Thu, 14 Oct 2021 15:51:28 +0100 Subject: [PATCH 1/5] Add more Ruby refs as per CD plan --- docs/codeql/codeql-cli/about-ql-packs.rst | 2 +- .../codeql-cli/creating-codeql-databases.rst | 21 ++++++++++++++----- .../getting-started-with-the-codeql-cli.rst | 2 +- ...etting-up-codeql-in-visual-studio-code.rst | 2 +- .../codeql/query-help/codeql-cwe-coverage.rst | 2 ++ docs/codeql/query-help/index.rst | 3 ++- docs/codeql/reusables/extractors.rst | 4 +++- docs/codeql/reusables/ruby-beta-note.rst | 4 ++++ .../creating-path-queries.rst | 2 +- 9 files changed, 31 insertions(+), 11 deletions(-) create mode 100644 docs/codeql/reusables/ruby-beta-note.rst diff --git a/docs/codeql/codeql-cli/about-ql-packs.rst b/docs/codeql/codeql-cli/about-ql-packs.rst index dc47f2a023b..fb5a1d8a8a0 100644 --- a/docs/codeql/codeql-cli/about-ql-packs.rst +++ b/docs/codeql/codeql-cli/about-ql-packs.rst @@ -7,7 +7,7 @@ QL packs are used to organize the files used in CodeQL analysis. They contain queries, library files, query suites, and important metadata. The `CodeQL repository `__ contains QL packs for -C/C++, C#, Java, JavaScript, and Python. The `CodeQL for Go +C/C++, C#, Java, JavaScript, Python, and Ruby. The `CodeQL for Go `__ repository contains a QL pack for Go analysis. You can also make custom QL packs to contain your own queries and libraries. diff --git a/docs/codeql/codeql-cli/creating-codeql-databases.rst b/docs/codeql/codeql-cli/creating-codeql-databases.rst index 52bd076e636..93a2471793a 100644 --- a/docs/codeql/codeql-cli/creating-codeql-databases.rst +++ b/docs/codeql/codeql-cli/creating-codeql-databases.rst @@ -88,15 +88,15 @@ Creating databases for non-compiled languages --------------------------------------------- The CodeQL CLI includes extractors to create databases for non-compiled -languages---specifically, JavaScript (and TypeScript) and Python. These -extractors are automatically invoked when you specify JavaScript or Python as +languages---specifically, JavaScript (and TypeScript), Python, and Ruby. These +extractors are automatically invoked when you specify JavaScript, Python, or Ruby as the ``--language`` option when executing ``database create``. When creating databases for these languages you must ensure that all additional dependencies are available. .. pull-quote:: Important - When you run ``database create`` for JavaScript, TypeScript, and Python, you should not + When you run ``database create`` for JavaScript, TypeScript, Python, and Ruby, you should not specify a ``--command`` option. Otherwise this overrides the normal extractor invocation, which will create an empty database. If you create databases for multiple languages and one of them is a compiled language, @@ -127,14 +127,25 @@ When creating databases for Python you must ensure: packages that the codebase depends on. - You have installed the `virtualenv `__ pip module. -In the command line you must specify ``--language=python``. For example +In the command line you must specify ``--language=python``. For example:: :: codeql database create --language=python /python-database -executes the ``database create`` subcommand from the code's checkout root, +This executes the ``database create`` subcommand from the code's checkout root, generating a new Python database at ``/python-database``. +Ruby +~~~~ + +Creating databases for Ruby requires no additional dependencies. +In the command line you must specify ``--language=ruby``. For example:: + + codeql database create --language=ruby --source-root /ruby-database + +Here, we have specified a ``--source-root`` path, which is the location where +database creation is executed, but is not necessarily the checkout root of the +codebase. Creating databases for compiled languages ----------------------------------------- diff --git a/docs/codeql/codeql-cli/getting-started-with-the-codeql-cli.rst b/docs/codeql/codeql-cli/getting-started-with-the-codeql-cli.rst index 1144c1c3e6c..a8ef822a628 100644 --- a/docs/codeql/codeql-cli/getting-started-with-the-codeql-cli.rst +++ b/docs/codeql/codeql-cli/getting-started-with-the-codeql-cli.rst @@ -100,7 +100,7 @@ further options on the command line. The `CodeQL repository `__ contains the queries and libraries required for CodeQL analysis of C/C++, C#, Java, -JavaScript/TypeScript, and Python. +JavaScript/TypeScript, Python, and Ruby. Clone a copy of this repository into ``codeql-home``. By default, the root of the cloned repository will be called ``codeql``. diff --git a/docs/codeql/codeql-for-visual-studio-code/setting-up-codeql-in-visual-studio-code.rst b/docs/codeql/codeql-for-visual-studio-code/setting-up-codeql-in-visual-studio-code.rst index 0075826d421..a43c69edecb 100644 --- a/docs/codeql/codeql-for-visual-studio-code/setting-up-codeql-in-visual-studio-code.rst +++ b/docs/codeql/codeql-for-visual-studio-code/setting-up-codeql-in-visual-studio-code.rst @@ -78,7 +78,7 @@ Using the starter workspace ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The starter workspace is a Git repository. It contains: -* The `repository of CodeQL libraries and queries `__ for C/C++, C#, Java, JavaScript, and Python. This is included as a submodule, so it can be updated without affecting your custom queries. +* The `repository of CodeQL libraries and queries `__ for C/C++, C#, Java, JavaScript, Python, and Ruby. This is included as a submodule, so it can be updated without affecting your custom queries. * The `repository of CodeQL libraries and queries `__ for Go. This is also included as a submodule. * A series of folders named ``codeql-custom-queries-``. These are ready for you to start developing your own custom queries for each language, using the standard libraries. There are some example queries to get you started. diff --git a/docs/codeql/query-help/codeql-cwe-coverage.rst b/docs/codeql/query-help/codeql-cwe-coverage.rst index b333053a1c8..cc538196b4f 100644 --- a/docs/codeql/query-help/codeql-cwe-coverage.rst +++ b/docs/codeql/query-help/codeql-cwe-coverage.rst @@ -33,3 +33,5 @@ Note that the CWE coverage includes both "`supported queries ` - :doc:`CodeQL query help for Python ` - .. pull-quote:: Information Each query help article includes: @@ -23,6 +22,8 @@ View the query help for the queries included in the ``code-scanning``, ``securit For a full list of the CWEs covered by these queries, see ":doc:`CodeQL CWE coverage `." +.. include:: ../reusables/ruby-beta-note.rst + .. toctree:: :hidden: :titlesonly: diff --git a/docs/codeql/reusables/extractors.rst b/docs/codeql/reusables/extractors.rst index 9076f7a768d..a3a4952811d 100644 --- a/docs/codeql/reusables/extractors.rst +++ b/docs/codeql/reusables/extractors.rst @@ -15,4 +15,6 @@ * - JavaScript/TypeScript - ``javascript`` * - Python - - ``python`` \ No newline at end of file + - ``python`` + * - Ruby + - ``ruby`` \ No newline at end of file diff --git a/docs/codeql/reusables/ruby-beta-note.rst b/docs/codeql/reusables/ruby-beta-note.rst new file mode 100644 index 00000000000..761381777c0 --- /dev/null +++ b/docs/codeql/reusables/ruby-beta-note.rst @@ -0,0 +1,4 @@ + .. pull-quote:: Note + + CodeQL analysis for Ruby is currently in beta. During the beta, analysis of Ruby code, + and the accompanying documentation, will not be as comprehensive as for other languages. diff --git a/docs/codeql/writing-codeql-queries/creating-path-queries.rst b/docs/codeql/writing-codeql-queries/creating-path-queries.rst index 60723f488e1..3eb67207ccd 100644 --- a/docs/codeql/writing-codeql-queries/creating-path-queries.rst +++ b/docs/codeql/writing-codeql-queries/creating-path-queries.rst @@ -116,7 +116,7 @@ Declaring sources and sinks You must provide information about the ``source`` and ``sink`` in your path query. These are objects that correspond to the nodes of the paths that you are exploring. The name and the type of the ``source`` and the ``sink`` must be declared in the ``from`` statement of the query, and the types must be compatible with the nodes of the graph computed by the ``edges`` predicate. -If you are querying C/C++, C#, Java, or JavaScript code (and you have used ``import DataFlow::PathGraph`` in your query), the definitions of the ``source`` and ``sink`` are accessed via the ``Configuration`` class in the data flow library. You should declare all three of these objects in the ``from`` statement. +If you are querying C/C++, C#, Java, JavaScript, or Ruby code (and you have used ``import DataFlow::PathGraph`` in your query), the definitions of the ``source`` and ``sink`` are accessed via the ``Configuration`` class in the data flow library. You should declare all three of these objects in the ``from`` statement. For example: .. code-block:: ql From 4f247bab4e221f5a23fb5872b3be47ef28671b03 Mon Sep 17 00:00:00 2001 From: hubwriter Date: Thu, 14 Oct 2021 17:13:42 +0100 Subject: [PATCH 2/5] Add Ruby to language table --- docs/codeql/support/reusables/versions-compilers.rst | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/codeql/support/reusables/versions-compilers.rst b/docs/codeql/support/reusables/versions-compilers.rst index 42c830ea665..8726d878bf9 100644 --- a/docs/codeql/support/reusables/versions-compilers.rst +++ b/docs/codeql/support/reusables/versions-compilers.rst @@ -22,6 +22,7 @@ Eclipse compiler for Java (ECJ) [5]_",``.java`` JavaScript,ECMAScript 2021 or lower,Not applicable,"``.js``, ``.jsx``, ``.mjs``, ``.es``, ``.es6``, ``.htm``, ``.html``, ``.xhm``, ``.xhtml``, ``.vue``, ``.json``, ``.yaml``, ``.yml``, ``.raml``, ``.xml`` [6]_" Python,"2.7, 3.5, 3.6, 3.7, 3.8, 3.9",Not applicable,``.py`` + Ruby,"up to 3.02",Not applicable,"``.rb``, ``.erb``, ``.gemspec``, ``Gemfile``" TypeScript [7]_,"2.6-4.4",Standard TypeScript compiler,"``.ts``, ``.tsx``" .. container:: footnote-group From 6f34735f64d44844ec2b387a459d26fa1017d9e6 Mon Sep 17 00:00:00 2001 From: hubwriter Date: Tue, 19 Oct 2021 09:27:54 +0100 Subject: [PATCH 3/5] Update docs/codeql/writing-codeql-queries/creating-path-queries.rst --- docs/codeql/writing-codeql-queries/creating-path-queries.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/codeql/writing-codeql-queries/creating-path-queries.rst b/docs/codeql/writing-codeql-queries/creating-path-queries.rst index 3eb67207ccd..4eec766d488 100644 --- a/docs/codeql/writing-codeql-queries/creating-path-queries.rst +++ b/docs/codeql/writing-codeql-queries/creating-path-queries.rst @@ -116,7 +116,7 @@ Declaring sources and sinks You must provide information about the ``source`` and ``sink`` in your path query. These are objects that correspond to the nodes of the paths that you are exploring. The name and the type of the ``source`` and the ``sink`` must be declared in the ``from`` statement of the query, and the types must be compatible with the nodes of the graph computed by the ``edges`` predicate. -If you are querying C/C++, C#, Java, JavaScript, or Ruby code (and you have used ``import DataFlow::PathGraph`` in your query), the definitions of the ``source`` and ``sink`` are accessed via the ``Configuration`` class in the data flow library. You should declare all three of these objects in the ``from`` statement. +If you are querying C/C++, C#, Java, JavaScript, Python, or Ruby code (and you have used ``import DataFlow::PathGraph`` in your query), the definitions of the ``source`` and ``sink`` are accessed via the ``Configuration`` class in the data flow library. You should declare all three of these objects in the ``from`` statement. For example: .. code-block:: ql From aaa5046533cf6abb59e8fb9edc44bd579fe17929 Mon Sep 17 00:00:00 2001 From: hubwriter Date: Wed, 20 Oct 2021 11:17:38 +0100 Subject: [PATCH 4/5] Add beta note to page Calum added --- docs/codeql/codeql-language-guides/codeql-for-ruby.rst | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/codeql/codeql-language-guides/codeql-for-ruby.rst b/docs/codeql/codeql-language-guides/codeql-for-ruby.rst index eaf5e8c94e1..bfb29a012ef 100644 --- a/docs/codeql/codeql-language-guides/codeql-for-ruby.rst +++ b/docs/codeql/codeql-language-guides/codeql-for-ruby.rst @@ -14,3 +14,5 @@ Experiment and learn how to write effective and efficient queries for CodeQL dat - :doc:`Basic query for Ruby code `: Learn to write and run a simple CodeQL query using LGTM. - :doc:`CodeQL library for Ruby `: When you're analyzing a Ruby program, you can make use of the large collection of classes in the CodeQL library for Ruby. + +.. include:: ../reusables/ruby-beta-note.rst From 8f15dc4bd03c4f9ecde974f40c715715d1778ada Mon Sep 17 00:00:00 2001 From: hubwriter Date: Wed, 20 Oct 2021 12:48:20 +0100 Subject: [PATCH 5/5] Add 'requires glibc 2.17' in supported languages table --- docs/codeql/support/reusables/versions-compilers.rst | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/docs/codeql/support/reusables/versions-compilers.rst b/docs/codeql/support/reusables/versions-compilers.rst index 8726d878bf9..62678e16f05 100644 --- a/docs/codeql/support/reusables/versions-compilers.rst +++ b/docs/codeql/support/reusables/versions-compilers.rst @@ -22,8 +22,8 @@ Eclipse compiler for Java (ECJ) [5]_",``.java`` JavaScript,ECMAScript 2021 or lower,Not applicable,"``.js``, ``.jsx``, ``.mjs``, ``.es``, ``.es6``, ``.htm``, ``.html``, ``.xhm``, ``.xhtml``, ``.vue``, ``.json``, ``.yaml``, ``.yml``, ``.raml``, ``.xml`` [6]_" Python,"2.7, 3.5, 3.6, 3.7, 3.8, 3.9",Not applicable,``.py`` - Ruby,"up to 3.02",Not applicable,"``.rb``, ``.erb``, ``.gemspec``, ``Gemfile``" - TypeScript [7]_,"2.6-4.4",Standard TypeScript compiler,"``.ts``, ``.tsx``" + Ruby [7]_,"up to 3.02",Not applicable,"``.rb``, ``.erb``, ``.gemspec``, ``Gemfile``" + TypeScript [8]_,"2.6-4.4",Standard TypeScript compiler,"``.ts``, ``.tsx``" .. container:: footnote-group @@ -33,4 +33,5 @@ .. [4] Builds that execute on Java 7 to 16 can be analyzed. The analysis understands Java 16 standard language features. .. [5] ECJ is supported when the build invokes it via the Maven Compiler plugin or the Takari Lifecycle plugin. .. [6] JSX and Flow code, YAML, JSON, HTML, and XML files may also be analyzed with JavaScript files. - .. [7] TypeScript analysis is performed by running the JavaScript extractor with TypeScript enabled. This is the default for LGTM. + .. [7] Requires glibc 2.17. + .. [8] TypeScript analysis is performed by running the JavaScript extractor with TypeScript enabled. This is the default for LGTM.