2877835899
- [ ] List file names in log when request/response bodies are base64 encoded
gzipped tar file
: base64 -d < foo1 | gunzip | tar t| head -20
9.8 KiB
9.8 KiB
Using MRVA
This repository has several additions to illustrate a full MRVA workflow.
Set up controller repo
Following the instructions, start with manually creating the controller repository
gh repo create mirva-controller --public -d 'Controller for MRVA'This avoids
An error occurred while setting up the controller repository: Controller
repository "hohn/mirva-controller" not found.Populate the controller repository
mkdir -p ~/local/mirva-controller && cd ~/local/mirva-controller
echo "* mirva-controller" >> README.org
git init
git add README.org
git commit -m "first commit"
git branch -M master
git remote add origin git@github.com:hohn/mirva-controller.git
git push -u origin masterThis avoids
Variant analysis failed because the controller repository hohn/mirva-controller
does not have a branch 'master'. Please create a 'master' branch by clicking here
and re-run the variant analysis query.Use the codeql extension to run MRVA
Following the instructions and running ./FlatBuffersFunc.ql, the entry
google/flatbuffers has one result. Others have none.
Use custom list with target repos in VS Code
The json file is here:
/Users/hohn/Library/Application Support/Code/User/workspaceStorage/bced2e4aa1a5f78ca07cf9e09151b1af/GitHub.vscode-codeql/databases.json
It can be edited in VS Code using the {} button.
It's saved in the workspace, but not in the current git repository.
Here are two snapshots for reference:
{
"version": 1,
"databases": {
"variantAnalysis": {
"repositoryLists": [
{
"name": "mirva-list",
"repositories": [
"google/flatbuffers"
]
}
],
"owners": [],
"repositories": []
}
},
"selected": {
"kind": "variantAnalysisSystemDefinedList",
"listName": "top_10"
}
}or
{
"version": 1,
"databases": {
"variantAnalysis": {
"repositoryLists": [
{
"name": "mirva-list",
"repositories": [
"google/flatbuffers"
]
}
],
"owners": [],
"repositories": []
}
},
"selected": {
"kind": "variantAnalysisUserDefinedList",
"listName": "mirva-list"
}
}Run MRVA from command line
-
Install mrva cli
cd ~/local/gh-mrva # Build it go mod edit -replace="github.com/GitHubSecurityLab/gh-mrva=/Users/hohn/local/gh-mrva" go build . # Install gh extension remove mrva gh extension install . # Sanity check gh mrva -h -
Set up the configuration
cd ~/local/gh-mrva cat > ~/.config/gh-mrva/config.yml <<eof # The following options are supported # codeql_path: Path to CodeQL distribution (checkout of codeql repo) # controller: NWO of the MRVA controller to use # list_file: Path to the JSON file containing the target repos # git checkout codeql-cli/v2.15.5 codeql_path: /Users/hohn/local/codeql-lib controller: hohn/mirva-controller list_file: /Users/hohn/local/gh-mrva/databases.json eof -
Submit the mrva job
gh mrva submit --help gh mrva submit --language cpp --session mirva-session-4 \ --list mirva-list \ --query /Users/hohn/local/gh-mrva/FlatBuffersFunc.ql -
Check the status and download the sarif files
cd ~/local/gh-mrva # Check the status gh mrva status --session mirva-session-1 # Download the sarif files when finished gh mrva download --session mirva-session-1 \ --output-dir mirva-session-1-sarif # Or download the sarif files and CodeQL dbs when finished gh mrva download --session mirva-session-1 \ --download-dbs \ --output-dir mirva-session-1-sarif
Miscellaneous Notes
Action logs on Controller Repository
The action logs are on the controller repository at https://github.com/hohn/mirva-controller/actions.
The action>google flatbuffers log references
github/codeql-variant-analysis-action
Run actions/checkout@v4
with:
repository: github/codeql-variant-analysis-action
ref: main
token: ***
ssh-strict: true
persist-credentials: true
clean: true
sparse-checkout-cone-mode: true
fetch-depth: 1
fetch-tags: false
show-progress: true
lfs: false
submodules: false
set-safe-directory: true
env:
CODEQL_ENABLE_EXPERIMENTAL_FEATURES_SWIFT: trueThis is https://github.com/github/codeql-variant-analysis-action
The workflow producing the logs: https://github.com/github/codeql-variant-analysis-action/blob/main/variant-analysis-workflow.yml
Compacted Edit-Run-Debug Cycle
With a full Using MRVA cycle done, only these steps are needed in a edit-run-debug cycle.
cd ~/local/gh-mrva
# Build it
go clean
go build -gcflags="all=-N -l" . # go build .
./gh-mrva -h
# In log-submit-the-mrva-job.log after edit
SN=41
./gh-mrva submit --language cpp --session mirva-session-$SN \
--list mirva-list \
--query /Users/hohn/local/gh-mrva/FlatBuffersFunc.ql >& log-$SN.out &
sleep 1 && em log-$SN.out
# Check the status
./gh-mrva status --session mirva-session-$SN |& tee log-$SN-status.out
# Download the sarif files and CodeQL dbs when finished
./gh-mrva download --session mirva-session-$SN \
--download-dbs \
--output-dir mirva-session-$SN-sarif \
>& log-download-$SN.log &
echo log-download-$SN.log
# 2024/02/08 15:33:39 >> Response body is
# Zip archive data, at least v1.0 to extract, compression method=deflate
# 0:$ unzip -v foo
# Archive: foo
# Length Method Size Cmpr Date Time CRC-32 Name
# -------- ------ ------- ---- ---------- ----- -------- ----
# 2297 Defl:N 980 57% 02-08-2024 22:54 9465f5ff results.sarif
# 148 Defl:N 121 18% 02-08-2024 22:54 dc8df445 results.bqrs
# -------- ------- --- -------
# 2445 1101 55% 2 files
# 0 4 Local file header signature = 0x04034b50 (PK♥♦ or "PK\3\4") PK
# 0x50, 0x4b, 0x03, 0x04
# byteArray := []byte{ 0x50, 0x4b, 0x03, 0x04 }Use the delve debugger to find sigsev
https://github.com/go-delve/delve/blob/master/Documentation/usage/dlv.md
# Use the delve debugger to find sigsev
# compile debugging binaries with -gcflags="all=-N -l" on Go 1.10 or later
go build -gcflags="all=-N -l" .
# Check the status
dlv debug -- status --session mirva-session-$SN
# Type 'help' for list of commands.
# (dlv) c
dlv debug -- download --session mirva-session-$SN \
--download-dbs \
--output-dir mirva-session-$SN-sarif \VS Code Debugger Configuration
launch.json for download
{
"version": "0.2.0",
"configurations": [
{
"name": "Launch Package",
"type": "go",
"request": "launch",
"mode": "auto",
"program": "${workspaceFolder}",
"buildFlags": [],
"args": ["download", "--session", "mirva-session-11", "--download-dbs", "--output-dir","mirva-session-11-sarif"]
}
]
}launch.json for submission
Matching
./gh-mrva submit --language cpp --session mirva-session-$SN \
--list mirva-list \
--query /Users/hohn/local/gh-mrva/FlatBuffersFunc.ql >& log-$SN.out & {
"version": "0.2.0",
"configurations": [
{
"name": "Launch Package",
"type": "go",
"request": "launch",
"mode": "auto",
"program": "${workspaceFolder}",
"buildFlags": [],
"args": ["submit",
"--language", "cpp",
"--session", "mirva-session-29",
"--list", "mirva-list",
"--query", "/Users/hohn/local/gh-mrva/FlatBuffersFunc.ql"]
}
]
}