From 43bcd5b26c1cb9e6813c2db7c9afc3ecb770789c Mon Sep 17 00:00:00 2001 From: yo-h <55373593+yo-h@users.noreply.github.com> Date: Mon, 24 Feb 2020 15:08:31 -0500 Subject: [PATCH 1/5] Add guidelines for experimental CodeQL queries and libraries --- CONTRIBUTING.md | 2 ++ cpp/ql/src/experimental/README.md | 1 + cpp/ql/test/experimental/README.md | 1 + csharp/ql/src/experimental/README.md | 1 + csharp/ql/test/experimental/README.md | 1 + docs/experimental.md | 42 +++++++++++++++++++++++ java/ql/src/experimental/README.md | 1 + java/ql/test/experimental/README.md | 1 + javascript/ql/src/experimental/README.md | 1 + javascript/ql/test/experimental/README.md | 1 + python/ql/src/experimental/README.md | 1 + python/ql/test/experimental/README.md | 1 + 12 files changed, 54 insertions(+) create mode 100644 cpp/ql/src/experimental/README.md create mode 100644 cpp/ql/test/experimental/README.md create mode 100644 csharp/ql/src/experimental/README.md create mode 100644 csharp/ql/test/experimental/README.md create mode 100644 docs/experimental.md create mode 100644 java/ql/src/experimental/README.md create mode 100644 java/ql/test/experimental/README.md create mode 100644 javascript/ql/src/experimental/README.md create mode 100644 javascript/ql/test/experimental/README.md create mode 100644 python/ql/src/experimental/README.md create mode 100644 python/ql/test/experimental/README.md diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index f22bc016c19..0583374555b 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -46,6 +46,8 @@ Follow the steps below to help other users understand what your query does, and Query help files explain the purpose of your query to other users. Write your query help in a `.qhelp` file and save it in the same directory as your new query. For more information on writing query help, see the [Query help style guide](https://github.com/Semmle/ql/blob/master/docs/query-help-style-guide.md). +In addition to contributions to our standard queries and libraries, we also welcome contributions of a more experimental nature, which do not need to fulfill all the requirements listed above. See the guidelines for [experimental queries and libraries](docs/experimental.md) for details. + ## Using your personal data If you contribute to this project, we will record your name and email diff --git a/cpp/ql/src/experimental/README.md b/cpp/ql/src/experimental/README.md new file mode 100644 index 00000000000..fc07363b24f --- /dev/null +++ b/cpp/ql/src/experimental/README.md @@ -0,0 +1 @@ +This directory contains [experimental](../../../../docs/experimental.md) CodeQL queries and libraries. diff --git a/cpp/ql/test/experimental/README.md b/cpp/ql/test/experimental/README.md new file mode 100644 index 00000000000..2a9db304b9f --- /dev/null +++ b/cpp/ql/test/experimental/README.md @@ -0,0 +1 @@ +This directory contains tests for [experimental](../../../../docs/experimental.md) CodeQL queries and libraries. diff --git a/csharp/ql/src/experimental/README.md b/csharp/ql/src/experimental/README.md new file mode 100644 index 00000000000..fc07363b24f --- /dev/null +++ b/csharp/ql/src/experimental/README.md @@ -0,0 +1 @@ +This directory contains [experimental](../../../../docs/experimental.md) CodeQL queries and libraries. diff --git a/csharp/ql/test/experimental/README.md b/csharp/ql/test/experimental/README.md new file mode 100644 index 00000000000..2a9db304b9f --- /dev/null +++ b/csharp/ql/test/experimental/README.md @@ -0,0 +1 @@ +This directory contains tests for [experimental](../../../../docs/experimental.md) CodeQL queries and libraries. diff --git a/docs/experimental.md b/docs/experimental.md new file mode 100644 index 00000000000..5a15cfa3779 --- /dev/null +++ b/docs/experimental.md @@ -0,0 +1,42 @@ +# Experimental CodeQL queries and libraries + +In addition to our standard CodeQL queries and libraries, this repository may also contain queries and libraries of a more experimental nature. Experimental queries and libraries can be improved incrementally and may eventually reach a sufficient maturity to be included in our standard libraries and queries. + +## Requirements + +1. **Directory structure** + + - Experimental queries and libraries are stored in the `experimental` subdirectory within each language-specific directory in the [CodeQL repository](https://github.com/Semmle/ql). For example, experimental Java queries and libraries are stored in `ql/java/ql/src/experimental` and any corresponding tests in `ql/java/ql/test/experimental`. + - The structure of an `experimental` subdirectory mirrors the structure of standard queries and libraries (or tests) in the parent directory. + +2. **Query metadata** + + - The query `@id` must not clash with any other queries in the repository. + - The query must have a `@name` and `@description` to explain its purpose. + - The query must have a `@kind` and `@problem.severity` as required by CodeQL tools. + + For details, see the [guide on query metadata](https://github.com/Semmle/ql/blob/master/docs/query-metadata-style-guide.md). + +3. **Formatting** + + - The queries and libraries must be [autoformatted](https://help.semmle.com/codeql/codeql-for-vscode/reference/editor.html#autoformatting). + +4. **Compilation** + + - Compilation of the query and any associated libraries and tests must be resilient to future development of the standard libraries. This means that the functionality cannot depend on the output of `getAQlClass` and it cannot use internal APIs. + - The query and any associated libraries and tests must not cause any compiler warnings to be emitted (such as use of deprecated functionality or missing `override` annotations). + +5. **Results** + + - The query must have at least one true positive result on some revision of a real project. + +6. **Contributor License Agreement** + + - The contributor can satisfy the [CLA](CONTRIBUTING.md#contributor-license-agreement). + +## Non-requirements + +Other criteria typically required for our standard queries and libraries are not required for experimental queries and libraries. In particular, fully disciplined query [metadata](docs/query-metadata-style-guide.md), query [help](docs/query-help-style-guide.md), tests, a low false positive rate and performance tuning are not required (but nonetheless recommended). + +Experimental queries and libraries may not be actively maintained as the standard libraries evolve and may be removed in the future. + diff --git a/java/ql/src/experimental/README.md b/java/ql/src/experimental/README.md new file mode 100644 index 00000000000..fc07363b24f --- /dev/null +++ b/java/ql/src/experimental/README.md @@ -0,0 +1 @@ +This directory contains [experimental](../../../../docs/experimental.md) CodeQL queries and libraries. diff --git a/java/ql/test/experimental/README.md b/java/ql/test/experimental/README.md new file mode 100644 index 00000000000..2a9db304b9f --- /dev/null +++ b/java/ql/test/experimental/README.md @@ -0,0 +1 @@ +This directory contains tests for [experimental](../../../../docs/experimental.md) CodeQL queries and libraries. diff --git a/javascript/ql/src/experimental/README.md b/javascript/ql/src/experimental/README.md new file mode 100644 index 00000000000..fc07363b24f --- /dev/null +++ b/javascript/ql/src/experimental/README.md @@ -0,0 +1 @@ +This directory contains [experimental](../../../../docs/experimental.md) CodeQL queries and libraries. diff --git a/javascript/ql/test/experimental/README.md b/javascript/ql/test/experimental/README.md new file mode 100644 index 00000000000..2a9db304b9f --- /dev/null +++ b/javascript/ql/test/experimental/README.md @@ -0,0 +1 @@ +This directory contains tests for [experimental](../../../../docs/experimental.md) CodeQL queries and libraries. diff --git a/python/ql/src/experimental/README.md b/python/ql/src/experimental/README.md new file mode 100644 index 00000000000..fc07363b24f --- /dev/null +++ b/python/ql/src/experimental/README.md @@ -0,0 +1 @@ +This directory contains [experimental](../../../../docs/experimental.md) CodeQL queries and libraries. diff --git a/python/ql/test/experimental/README.md b/python/ql/test/experimental/README.md new file mode 100644 index 00000000000..2a9db304b9f --- /dev/null +++ b/python/ql/test/experimental/README.md @@ -0,0 +1 @@ +This directory contains tests for [experimental](../../../../docs/experimental.md) CodeQL queries and libraries. From d06caefd8e9872f138bd8eeb3c8c5ea7686ac295 Mon Sep 17 00:00:00 2001 From: yo-h <55373593+yo-h@users.noreply.github.com> Date: Tue, 25 Feb 2020 11:17:42 -0500 Subject: [PATCH 2/5] Address code review comments for `experimental.md` --- docs/experimental.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/experimental.md b/docs/experimental.md index 5a15cfa3779..fef4729cfd2 100644 --- a/docs/experimental.md +++ b/docs/experimental.md @@ -2,6 +2,8 @@ In addition to our standard CodeQL queries and libraries, this repository may also contain queries and libraries of a more experimental nature. Experimental queries and libraries can be improved incrementally and may eventually reach a sufficient maturity to be included in our standard libraries and queries. +Experimental queries and libraries may not be actively maintained as the standard libraries evolve, may be changed in backwards-incompatible ways or may be removed entirely in the future without deprecation warnings. + ## Requirements 1. **Directory structure** @@ -23,7 +25,7 @@ In addition to our standard CodeQL queries and libraries, this repository may al 4. **Compilation** - - Compilation of the query and any associated libraries and tests must be resilient to future development of the standard libraries. This means that the functionality cannot depend on the output of `getAQlClass` and it cannot use internal APIs. + - Compilation of the query and any associated libraries and tests must be resilient to future development of the standard libraries. This means that the functionality cannot use internal APIs, cannot depend on the output of `getAQlClass` and cannot make use of regexp matching on `toString`. - The query and any associated libraries and tests must not cause any compiler warnings to be emitted (such as use of deprecated functionality or missing `override` annotations). 5. **Results** @@ -38,5 +40,3 @@ In addition to our standard CodeQL queries and libraries, this repository may al Other criteria typically required for our standard queries and libraries are not required for experimental queries and libraries. In particular, fully disciplined query [metadata](docs/query-metadata-style-guide.md), query [help](docs/query-help-style-guide.md), tests, a low false positive rate and performance tuning are not required (but nonetheless recommended). -Experimental queries and libraries may not be actively maintained as the standard libraries evolve and may be removed in the future. - From 21dd8757dd8b7d54dde9f0a5afd3ad6e3b9da16e Mon Sep 17 00:00:00 2001 From: yo-h <55373593+yo-h@users.noreply.github.com> Date: Tue, 25 Feb 2020 23:11:29 -0500 Subject: [PATCH 3/5] Update docs/experimental.md Co-Authored-By: Felicity Chapman --- docs/experimental.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/docs/experimental.md b/docs/experimental.md index fef4729cfd2..b606a791d9d 100644 --- a/docs/experimental.md +++ b/docs/experimental.md @@ -25,7 +25,7 @@ Experimental queries and libraries may not be actively maintained as the standar 4. **Compilation** - - Compilation of the query and any associated libraries and tests must be resilient to future development of the standard libraries. This means that the functionality cannot use internal APIs, cannot depend on the output of `getAQlClass` and cannot make use of regexp matching on `toString`. + - Compilation of the query and any associated libraries and tests must be resilient to future development of the standard libraries. This means that the functionality cannot use internal APIs, cannot depend on the output of `getAQlClass`, and cannot make use of regexp matching on `toString`. - The query and any associated libraries and tests must not cause any compiler warnings to be emitted (such as use of deprecated functionality or missing `override` annotations). 5. **Results** @@ -39,4 +39,3 @@ Experimental queries and libraries may not be actively maintained as the standar ## Non-requirements Other criteria typically required for our standard queries and libraries are not required for experimental queries and libraries. In particular, fully disciplined query [metadata](docs/query-metadata-style-guide.md), query [help](docs/query-help-style-guide.md), tests, a low false positive rate and performance tuning are not required (but nonetheless recommended). - From aeb87931976ba36417c10ffb454260de90e64359 Mon Sep 17 00:00:00 2001 From: yo-h <55373593+yo-h@users.noreply.github.com> Date: Wed, 26 Feb 2020 18:38:42 -0500 Subject: [PATCH 4/5] Update docs/experimental.md Break sentence down into shorter ones, as per review comment. --- docs/experimental.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/experimental.md b/docs/experimental.md index b606a791d9d..e1f56bcf7d4 100644 --- a/docs/experimental.md +++ b/docs/experimental.md @@ -2,7 +2,7 @@ In addition to our standard CodeQL queries and libraries, this repository may also contain queries and libraries of a more experimental nature. Experimental queries and libraries can be improved incrementally and may eventually reach a sufficient maturity to be included in our standard libraries and queries. -Experimental queries and libraries may not be actively maintained as the standard libraries evolve, may be changed in backwards-incompatible ways or may be removed entirely in the future without deprecation warnings. +Experimental queries and libraries may not be actively maintained as the standard libraries evolve. They may also be changed in backwards-incompatible ways or may be removed entirely in the future without deprecation warnings. ## Requirements From 63adc635971e63b6f20540dd2dd276d9cb536a08 Mon Sep 17 00:00:00 2001 From: yo-h <55373593+yo-h@users.noreply.github.com> Date: Wed, 26 Feb 2020 18:35:16 -0500 Subject: [PATCH 5/5] `CONTRIBUTING.md`: add paragraph on maintaining backwards compatibility --- CONTRIBUTING.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 0583374555b..649af08b996 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -46,6 +46,10 @@ Follow the steps below to help other users understand what your query does, and Query help files explain the purpose of your query to other users. Write your query help in a `.qhelp` file and save it in the same directory as your new query. For more information on writing query help, see the [Query help style guide](https://github.com/Semmle/ql/blob/master/docs/query-help-style-guide.md). +7. **Maintain backwards compatibility** + +The standard CodeQL libraries must evolve in a backwards compatible manner. If any backwards incompatible changes need to be made, the existing API must first be marked as deprecated. This is done by adding a `deprecated` annotation along with a QLDoc reference to the replacement API. Only after at least one full release cycle has elapsed may the old API be removed. + In addition to contributions to our standard queries and libraries, we also welcome contributions of a more experimental nature, which do not need to fulfill all the requirements listed above. See the guidelines for [experimental queries and libraries](docs/experimental.md) for details. ## Using your personal data