mirror of
https://github.com/github/codeql.git
synced 2025-12-16 08:43:11 +01:00
113 lines
4.9 KiB
Markdown
113 lines
4.9 KiB
Markdown
# Rust on CodeQL
|
|
|
|
## Development
|
|
|
|
### Dependencies
|
|
|
|
If you don't have the `semmle-code` repo you may need to install Bazel manually, e.g. from https://github.com/bazelbuild/bazelisk.
|
|
|
|
### Building the Rust Extractor
|
|
|
|
This approach uses a released `codeql` version and is simpler to use for QL development. From anywhere under your `semmle-code` or `codeql` directory you can run:
|
|
```bash
|
|
bazel run @codeql//rust:install
|
|
```
|
|
|
|
You can use shorter versions of the above command:
|
|
```bash
|
|
bazel run //rust:install # if under the `codeql` checkout
|
|
bazel run rust:install # if at the root of the `codeql` checkout
|
|
bazel run :install # if at the `rust` directory of the `codeql` checkout
|
|
```
|
|
|
|
You now need to create a [per-user CodeQL configuration file](https://docs.github.com/en/code-security/codeql-cli/using-the-advanced-functionality-of-the-codeql-cli/specifying-command-options-in-a-codeql-configuration-file#using-a-codeql-configuration-file) and specify the option:
|
|
```
|
|
--search-path PATH/TO/semmle-code/ql
|
|
```
|
|
(wherever the `codeql` checkout is on your system)
|
|
|
|
You can now use the Rust extractor e.g. to run Rust tests from the command line or in VSCode.
|
|
|
|
### Building the Rust Extractor (as a sembuild target)
|
|
|
|
This approach allows you to build a Rust extractor with a CLI built from source. From your `semmle-code` directory run:
|
|
```bash
|
|
./build target/intree/codeql-rust
|
|
```
|
|
You can now invoke it directly, for example to run some tests:
|
|
```bash
|
|
./target/intree/codeql-rust/codeql test run ql/rust/ql/test/PATH/TO/TEST/
|
|
```
|
|
|
|
### Building a Database
|
|
|
|
TODO
|
|
|
|
### Code Generation
|
|
|
|
If you make changes to either
|
|
* `ast-generator/`, or
|
|
* `schema/*.py`
|
|
|
|
you'll need to regenerate code. You can do so running
|
|
```sh
|
|
bazel run @codeql//rust/codegen
|
|
```
|
|
|
|
Sometimes, especially if resolving conflicts on generated files, you might need to run
|
|
```sh
|
|
bazel run @codeql//rust/codegen -- --force
|
|
```
|
|
for code generation to succeed.
|
|
|
|
### Updating `rust-analyzer`
|
|
|
|
Here's a rundown of the typical actions to perform to do a rust-analyzer (and other dependencies) update. A one-time setup consists in
|
|
installing [`cargo-edit`](https://crates.io/crates/cargo-edit) with `cargo install cargo-edit`. On Ubuntu that also requires
|
|
`sudo apt install libssl-dev pkg-config`.
|
|
|
|
1. From the root of the `codeql` repo checkout, run a Cargo upgrade:
|
|
```
|
|
cargo upgrade --incompatible --pinned
|
|
```
|
|
2. Look at a diff of the `Cargo.toml` files: if all `ra_ap_` prefixed dependencies have been updated to the same number, go on to the next step.
|
|
Otherwise, it means the latest `rust-analyzer` update has not been fully rolled out to all its crates in `crates.io`.
|
|
_All `ra_ap_` versions must agree!_
|
|
Downgrade by hand to the minimum one you see, and run a `cargo update` after that to fix the `Cargo.lock` file.
|
|
3. Commit the changes, skipping `pre-commit` hooks if you have them enabled:
|
|
```
|
|
git commit -am 'Cargo: upgrade dependencies' --no-verify
|
|
```
|
|
4. Regenerate vendored bazel files (these allow faster builds, particularly on CI where it has to start from scratch each time), commit the changes:
|
|
```
|
|
misc/bazel/3rdparty/update_tree_sitter_extractors_deps.sh
|
|
git add .
|
|
git commit -am 'Bazel: regenerate vendored cargo dependencies' --no-verify
|
|
```
|
|
5. Run codegen
|
|
```
|
|
bazel run //rust/codegen
|
|
```
|
|
Take note whether `rust/schema/ast.py` was changed. That might need tweaks, new tests and/or downgrade/upgrade scripts down the line.
|
|
6. Try compiling
|
|
```
|
|
bazel run //rust:install
|
|
```
|
|
* if it succeeds: good! You can move on to the next step.
|
|
* if it fails while compiling rust-analyzer dependencies, you need to update the rust toolchain. Sometimes the error will tell you
|
|
so explicitly, but it may happen that the error is more obscure. To update the rust toolchain:
|
|
* you will need to open a PR on the internal repo updating `RUST_VERSION` in `MODULE.bazel`. In general you can have this merged
|
|
independently of the changes in `codeql`.
|
|
* in `codeql`, update both `RUST_VERSION` in `MODULE.bazel` _and_ `rust-toolchain.toml` files. You may want to also update the
|
|
nightly toolchain in `rust/extractor/src/nightly-toolchain/rust-toolchain.toml` to a more recent date while you're at it.
|
|
* if it fails while compiling rust extractor code, you will need to adapt it to the new library version.
|
|
* for example updating annotations in `annotations.py`, adding / removing generated tests.
|
|
|
|
If you had to do any changes, commit them. If you updated the rust toolchain, running `rust/lint.py` might reformat or apply new
|
|
lints to the code.
|
|
7. Check with CI if everything is in order.
|
|
8. Run DCA with database caching disabled. Iterate on the code if needed.
|
|
9. If in step 5 the schema was updated, add upgrade/downgrade scripts and a change note. This is best done last to reduce the chance of
|
|
merge conflicts (none of the other testing depends on having upgrade and downgrade scripts in place). See
|
|
[Upgrading a language database schema](docs/prepare-db-upgrade.md).
|