mrva/mrva-docker
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

switch to pollling rabbitmq; update images

Browse Source
This commit is contained in:
Michael Hohn
2025-06-16 16:07:37 -07:00
committed by =Michael Hohn
parent af46dd6fa4
commit 4012470a6d
6 changed files with 74 additions and 60 deletions
Download Patch File Download Diff File Expand all files Collapse all files

49
README.org
View File

@@ -268,26 +268,33 @@
* Using the Containers
** Running the containers
1. Start the containers
1. Build local container images; follow [[./docker-compose-demo.yml]]
+ client-ghmrva: [[file:containers/ghmrva/README.org::*MRVA cli tools container][MRVA cli tools container]]
+ code-server: [[file:containers/vscode/README.org::*MRVA VS Code server container][MRVA VS Code server container]]
+ hepc: [[file:containers/hepc/README.org::*HEPC Container Image Build][HEPC Container Image Build]]
+ [[file:containers/server/README.org::*server container image build][server container image build]]
+ agent: [[file:containers/agent/README.org::*agent image build][agent image build]]
2. Start the containers
#+BEGIN_SRC sh
cd ~/work-gh/mrva/mrva-docker/
docker-compose -f docker-compose-demo.yml down
docker ps
docker-compose -f docker-compose-demo.yml up
#+END_SRC
2. View all logs
3. View all logs
: docker-compose logs
3. Follow all logs if started with =-d=
4. Follow all logs if started with =-d=
#+BEGIN_SRC sh
docker-compose logs -f
#+END_SRC
4. Follow single container, =server=, logging via
5. Follow single container, =server=, logging via
#+BEGIN_SRC sh
cd ~/work-gh/mrva/mrvacommander
docker-compose up -d
docker-compose logs -f server
#+END_SRC
5. Cleanup in case of obscure errors (network or other)
6. Cleanup in case of obscure errors (network or other)
#+BEGIN_SRC sh
docker-compose -f docker-compose-demo.yml down --volumes --remove-orphans
docker network prune
@@ -358,18 +365,18 @@
*** Set up gh-mrva configuration
#+BEGIN_SRC sh
# Create configuration directory and generate config file for gh-mrva
mkdir -p ~/.config/gh-mrva
cat > ~/.config/gh-mrva/config.yml <<eof
# Configuration file for the gh-mrva tool
# codeql_path: Path to the CodeQL distribution (not used in this setup)
# controller: Placeholder for a controller NWO (not relevant in this setup)
# list_file: Path to the repository selection JSON file
# Create configuration directory and generate config file for gh-mrva
mkdir -p ~/.config/gh-mrva
cat > ~/.config/gh-mrva/config.yml <<eof
# Configuration file for the gh-mrva tool
# codeql_path: Path to the CodeQL distribution (not used in this setup)
# controller: Placeholder for a controller NWO (not relevant in this setup)
# list_file: Path to the repository selection JSON file
codeql_path: not-used/codeql-path
controller: not-used/mirva-controller
list_file: $HOME/work-gh/mrva/gh-mrva/gh-mrva-selection.json
eof
codeql_path: not-used/codeql-path
controller: not-used/mirva-controller
list_file: $HOME/work-gh/mrva/gh-mrva/gh-mrva-selection.json
eof
#+END_SRC
@@ -470,7 +477,7 @@
# Submit a new MRVA job with the second query
cd ~/work-gh/mrva/gh-mrva/
gh-mrva submit \
--language cpp --session mirva-session-2161 \
--language cpp --session mirva-session-2261 \
--list mirva-list \
--query ~/work-gh/mrva/gh-mrva/Fprintf.ql
#+END_SRC
@@ -478,15 +485,15 @@
*** Check status and download results for the second session
#+BEGIN_SRC sh
# Check the status of the second session
gh-mrva status --session mirva-session-2161
gh-mrva status --session mirva-session-2261
# Download SARIF files and databases for the second query
cd ~/work-gh/mrva/gh-mrva/
gh-mrva download --session mirva-session-2161 \
gh-mrva download --session mirva-session-2261 \
--download-dbs \
--output-dir mirva-session-2161
--output-dir mirva-session-2261
ls -l mirva-session-2161
ls -l mirva-session-2261
#+END_SRC

15
containers/agent/README.org
View File

@@ -1,17 +1,16 @@
* agent image setup
* agent image build
- build container
#+BEGIN_SRC sh
# Build the container via
cd ~/work-gh/mrva/mrva-docker/containers/agent/
MAG_TARGET=mrva-agent:0.4.0
docker build --no-cache --network host -t ${MAG_TARGET} .
docker build --no-cache --network host -t mrva-agent:0.4.0 .
# Run bash in the container in standalone mode
cd ~/work-gh/mrva/mrva-docker/containers/agent/
docker run --env-file ../../.env.container --rm -it \
--entrypoint /bin/bash \
${MAG_TARGET}
mrva-agent:0.4.0
'
ls /usr/local/bin/
entrypoint.sh mrvaagent
@@ -20,20 +19,20 @@
- Tag the container. This is sufficient for further use on the local machine.
#+BEGIN_SRC sh
docker tag ${MAG_TARGET} ghcr.io/hohn/${MAG_TARGET}
docker tag ${MAG_TARGET} ghcr.io/hohn/mrva-agent:0.4.0
#+END_SRC
- Push this container
#+BEGIN_SRC sh
docker push ghcr.io/hohn/${MAG_TARGET}
docker push ghcr.io/hohn/mrva-agent:0.4.0
#+END_SRC
- Test the registry image
#+BEGIN_SRC sh
# Test pushed container
docker pull ghcr.io/hohn/${MAG_TARGET}
docker pull ghcr.io/hohn/mrva-agent:0.4.0
docker run --env-file ../../.env.container --rm -it \
--entrypoint /bin/bash \
${MAG_TARGET}
mrva-agent:0.4.0
#+END_SRC

17
containers/ghmrva/README.org
View File

@@ -1,16 +1,21 @@
* MRVA cli tools container
Set up / run:
#+BEGIN_SRC sh
# Build
cd ~/work-gh/mrva/mrvacommander/client/containers/ghmrva/
make ghm
cd ~/work-gh/mrva/mrva-docker/containers/ghmrva/
docker build -t client-ghmrva-container:0.4.0 .
# Run
docker run -ti client-ghmrva-container:0.1.24 /bin/bash
docker run -ti client-ghmrva-container:0.4.0 /bin/bash
# In the container
gh-mrva -h
codeql -h
# Push
make ghm-push
# tag
docker tag client-ghmrva-container:0.4.0 ghcr.io/hohn/client-ghmrva-container:0.4.0
# push
docker push ghcr.io/hohn/client-ghmrva-container:0.4.0
#+END_SRC

26
containers/hepc/README.org
View File

@@ -1,4 +1,4 @@
* Container Image Build
* HEPC Container Image Build
Follow the [[file:~/work-gh/mrva/mrvahepc/README.org::*Usage Sample][Usage Sample]] in mrvahepc to set up the repository and data there. Then:
** Build container
@@ -7,13 +7,13 @@
rm -fR ./mrvahepc && cp -r ../../../mrvahepc .
# Usual build
docker build -t mrva-hepc-container:0.1.24 -f Dockerfile .
docker build -t mrva-hepc-container:0.4.0 -f Dockerfile .
# Build with full output, e.g. RUN ls
docker build --progress=plain --no-cache -t mrva-hepc-container:0.1.24 -f Dockerfile .
docker build --progress=plain --no-cache -t mrva-hepc-container:0.4.0 -f Dockerfile .
# Run standalone
docker run -p 8070:8070 -ti mrva-hepc-container:0.1.24
docker run -p 8070:8070 -ti mrva-hepc-container:0.4.0
# Test server
curl 127.0.0.1:8070/index -o - 2>/dev/null | wc -l
@@ -29,7 +29,14 @@
wget $(echo $url|sed 's|http://hepc|http://127.0.0.1:8070|g;')
#+END_SRC
** Access this container from another
** Access this container
from host, when run by docker-compose
#+BEGIN_SRC sh
cd ~/work-gh/mrva/mrva-docker/containers/hepc
docker exec -it mrva-hepc bash
#+END_SRC
from another
#+BEGIN_SRC sh
# Enter container
docker exec -it mrva-docker-client-ghmrva-1 bash
@@ -42,20 +49,19 @@
** Push this container
#+BEGIN_SRC sh
# Push container
docker tag mrva-hepc-container:0.1.24 ghcr.io/hohn/mrva-hepc-container:0.1.24
docker push ghcr.io/hohn/mrva-hepc-container:0.1.24
# 0.1.24: digest: sha256:b089c6495b04ed6b4a4f467cd16d671b2404834c53ba020d3afd195a4ec870fb size: 2831
docker tag mrva-hepc-container:0.4.0 ghcr.io/hohn/mrva-hepc-container:0.4.0
docker push ghcr.io/hohn/mrva-hepc-container:0.4.0
#+END_SRC
** Test the registry image
#+BEGIN_SRC sh
# Test pushed container
docker pull ghcr.io/hohn/mrva-hepc-container:0.1.24
docker pull ghcr.io/hohn/mrva-hepc-container:0.4.0
docker run \
-p 8070:8070 \
--name test-mrva-hepc-container \
--rm -it ghcr.io/hohn/mrva-hepc-container:0.1.24
--rm -it ghcr.io/hohn/mrva-hepc-container:0.4.0
curl 127.0.0.1:8070/index -o - 2>/dev/null | wc -l

18
containers/server/README.org
View File

@@ -1,22 +1,20 @@
* server image setup
* server container image build
- build container
XX: use
: make all
#+BEGIN_SRC sh
# Build the container via
cd ~/work-gh/mrva/mrva-docker/containers/server/
docker build --no-cache --network host -t mrva-server:0.1.24 .
docker build --no-cache --network host -t mrva-server:0.4.0 .
# The --network host is needed when using ../aptcacher/
# Run the container in standalone mode via
cd ~/work-gh/mrva/mrva-docker/containers/server/
docker run -d mrva-server:0.1.24
docker run -d mrva-server:0.4.0
# Just run bash in the container
docker run --env-file ../../.env.container --rm -it \
--entrypoint /bin/bash \
mrva-server:0.1.24
mrva-server:0.4.0
# In the container:
'
ls /opt/codeql/codeql
@@ -27,22 +25,22 @@
- Tag the container. This is sufficient for further use on the local machine.
#+BEGIN_SRC sh
docker tag mrva-server:0.1.24 ghcr.io/hohn/mrva-server:0.1.24
docker tag mrva-server:0.4.0 ghcr.io/hohn/mrva-server:0.4.0
#+END_SRC
- Push this container
#+BEGIN_SRC sh
docker push ghcr.io/hohn/mrva-server:0.1.24
docker push ghcr.io/hohn/mrva-server:0.4.0
#+END_SRC
- Test the registry image
#+BEGIN_SRC sh
# Test pushed container
docker pull ghcr.io/hohn/mrva-server:0.1.24
docker pull ghcr.io/hohn/mrva-server:0.4.0
docker run \
--env-file ../../.env.container \
--name test-mrva-server \
--rm -it ghcr.io/hohn/mrva-server:0.1.24 \
--rm -it ghcr.io/hohn/mrva-server:0.4.0 \
--mode=container --loglevel=debug \
/bin/bash
#+END_SRC

9
containers/vscode/README.org
View File

@@ -32,7 +32,6 @@
/bin/code-server --force --install-extension vscode-codeql-*.vsix
#+END_SRC
- pre-container build
#+BEGIN_SRC sh
# Collect artifacts
@@ -41,14 +40,14 @@
# Build the container via
cd ~/work-gh/mrva/mrva-docker/containers/vscode/
docker build -t code-server-initialized:0.3.3 .
docker build -t code-server-initialized:0.4.0 .
# # For debugging and significant Dockerfile changes, build from scratch:
# docker build --no-cache -t code-server-initialized:0.3.3 .
# docker build --no-cache -t code-server-initialized:0.4.0 .
# Run the container in standalone mode via
cd ~/work-gh/mrva/mrva-docker/containers/vscode/
docker run -v ~/work-gh/mrva/vscode-codeql:/work-gh/mrva/vscode-codeql \
-d -p 9080:9080 code-server-initialized:0.3.3
-d -p 9080:9080 code-server-initialized:0.4.0
# Connect to it, password is mrva
open http://localhost:9080/?folder=/home/coder
@@ -178,7 +177,7 @@
# 0:$ docker ps --format "table {{.ID}}\t{{.Image}}\t{{.Names}}"
VSCID=$(docker ps --format "table {{.ID}}\t{{.Image}}\t{{.Names}}" |grep code-server |cut -f1 -d " ")
docker commit $VSCID mrva-client-vscode:0.3.3
docker commit $VSCID mrva-client-vscode:0.4.0
#+END_SRC