mrva/mrvacommander
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
70 Commits 3 Branches 4 Tags
ef7552c43fba537a60e8a175e37ed20466d73f27
Commit Graph

70 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Nicolas Will
ef7552c43f Update .gitignore 2024-06-17 13:16:06 +02:00
Nicolas Will
50da8eefe8 Fix ZipSlip vuln and integer conversion issue 2024-06-17 12:55:28 +02:00
Nicolas Will
fc9fcc7ae6 Add server queue logic and refactor 2024-06-17 11:30:46 +02:00
Michael Hohn
8b310e43ad Fix storage modules types and interfaces to compile server 2024-06-16 20:16:26 -07:00
Michael Hohn
6229c08900 Remove postgres and references to it 2024-06-16 19:43:29 -07:00
Michael Hohn
b756668e70 Fix merge so server compiles 2024-06-16 19:36:31 -07:00
Michael Hohn
2c5ecd3a1e Merge the agent-impl branch into the server branch 2024-06-16 19:21:42 -07:00
Michael Hohn
3be3229ecd Remove duplicate test DB 2024-06-16 19:09:32 -07:00
Michael Hohn
335a61c28e For testing: Add request capture script. Replay via curl 2024-06-16 19:09:32 -07:00
Michael Hohn
cd0647836e Combine New/Setup functions 2024-06-16 19:09:32 -07:00
Michael Hohn
f611f02d1c try git lfs again 2024-06-16 19:09:31 -07:00
Michael Hohn
252859ae18 Add Go access test for the DB store and update README 2024-06-16 19:09:31 -07:00
Michael Hohn
9270230f08 Set up separate minio container for query pack storage 2024-06-16 19:09:31 -07:00
Michael Hohn
7347779072 Change container name to dbstore 2024-06-16 19:09:31 -07:00
Michael Hohn
bbb54caaf3 Set up the minio ql db 
- Set up the minio ql db
- Use bind mounts in docker compose to get persistence
- Populate the ql db with the sample DBs
 2024-06-16 19:09:31 -07:00
Michael Hohn
8df9673897 wip: Mark update slots with XX:, add pkg/server/container.go 2024-06-16 19:09:30 -07:00
Nicolas Will
903ca5673e Add dynamic worker management 2024-06-16 17:07:13 +02:00
Nicolas Will
7ea45cb176 Separate queue and agent logic and refactor 2024-06-16 11:18:22 +02:00
Nicolas Will
e107f6cf80 Fix ENV CODEQL_JAVA_HOME in Dockerfile 2024-06-15 23:10:37 +02:00
Nicolas Will
1a574c2f7f Add RabbitMQ connect retry and healthcheck 2024-06-15 00:39:21 +02:00
Nicolas Will
ea10403f6c Bump server Ubuntu version to 24.10 2024-06-15 00:23:30 +02:00
Nicolas Will
3b06e2061f Add RabbitMQ agent and containers 2024-06-15 00:23:14 +02:00
Nicolas Will
ec4d2b3eac Move postgres-init-scripts to init/postgres 2024-06-15 00:19:24 +02:00
Nicolas Will
c29daab045 Standardize NameWithOwner and Visible naming 
Acronyms are now "NWO" and "Vis" respsectively
 2024-06-14 12:55:45 +02:00
Nicolas Will
3218f64bcf Move archive functions into utils package 2024-06-14 12:48:33 +02:00
Michael Hohn
5b9df73424 Merge pull request #12 from advanced-security/hohn-0.1.11 
Hohn 0.1.11  Make cross-module visibility explicit
 2024-06-12 11:41:17 -07:00
Michael Hohn
5730c330f4 Add codeql to server container for standalone testing 
For full test, we cannot have

       ERROR codeql database analyze failed: error="exec:
       \"codeql\": executable file not found in $PATH" job="{MirvaRequestID:0
       QueryPackId:54674 QueryLanguage:cpp ORepo:{Owner:psycopg Repo:psycopg2}}"

For linux/arm64, use a Dockerfile that:
       - uses ubuntu 22.04 base image
       - adds the 1.17 version of the codeql bundle
       - extracts the bundle
       - adds a recent version of the JRE
       - extracts it
       - sets the CODEQL_JAVA_HOME environment variable to point to the JRE

The instructions are updated
 2024-06-12 11:28:37 -07:00
Michael Hohn
765a76f75a Provide MRVA_SERVER_ROOT via environment variable 2024-06-11 20:13:13 -07:00
Michael Hohn
4b721b5969 Fix docker-compose, add postgres init script 2024-06-11 18:05:44 -07:00
Michael Hohn
9c0cdb1fe4 Simplify naming, don't restate package name 2024-06-11 16:55:10 -07:00
Michael Hohn
2d88b351ff Introduce structs/interfacess for new storage units 
This commit simply splits the interfaces but introduces no new structs

     - Introduce the QueryPackStore, mrvacommander/pkg/qpstore
     - Introduce the CodeQL database store, pkg/qldbstore/interfaces.go
 2024-06-11 14:16:41 -07:00
Michael Hohn
fc29fc5653 wip: update passing Queue to Commander 
- Add minio to docker-compose
     - Fix use of server.NewCommanderSingle
 2024-06-11 13:19:05 -07:00
Michael Hohn
7e0d6909da wip: Make cross-module visibility explicit via Visibles structs 
All access is/will be through interfaces accessed through these structs.

This introduces several distinct storage units:
+ DB for server state
+ DB for codeql databases
+ query pack store

The steps for manually creating needed databases are in the README
 2024-06-07 13:14:41 -07:00
Michael Hohn
25cab583c1 wip: storage using postgres / gorm using partial json 
Several approaches of normalizing json were tried and ultimately found
impractical at this point.

Using a hybrid of tables and json is the current approach; this may be
further normalized later.
 2024-06-06 13:19:00 -07:00
Michael Hohn
593644ca2e wip: rename ID to JobId 2024-06-04 13:04:51 -07:00
Michael Hohn
0349961360 wip: start container version of server 2024-06-04 12:24:42 -07:00
Michael Hohn
71838d3320 Add a minimal gorm example that takes a go struct, creates a postgres table, and writes the struct to the table. 2024-06-03 14:03:26 -07:00
Michael Hohn
e850a36943 Minimal docker-compose.yml for server development 2024-06-03 11:50:03 -07:00
Michael Hohn
776b980d3b Add container for server work 2024-05-31 13:26:24 -07:00
Michael Hohn
c6ebfcc4c4 Add VS Code workspace 2024-05-31 11:55:57 -07:00
Michael Hohn
b9081b1945 wip: convert run-analysis.sh to golang version 2024-05-31 08:24:09 -07:00
Michael Hohn
75d0d312db Add Files created by agents at run-time to .gitignore 2024-05-28 19:56:09 -07:00
Michael Hohn
d88f31f357 Add two sample DBs 2024-05-26 12:41:40 -07:00
Michael Hohn
ba44db04da wip: server is now fully functional, some FIXMEs remain 2024-05-26 12:22:36 -07:00
Michael Hohn
f7155eba50 wip: add analysis runner / agent, separate Server/Queue/Agent, use New* initializers 2024-05-23 15:46:55 -07:00
Michael Hohn
2ab596bf1d wip: Move all references to github.com/hohn/ghes-mirva-server 2024-05-22 14:39:12 -07:00
Michael Hohn
4269bacf2a wip: update store. references to storage. in server.go 2024-05-21 11:45:47 -07:00
Michael Hohn
873339ff06 wip: port submit_response() 2024-05-21 10:51:01 -07:00
Michael Hohn
8cd4f4d809 wip: port queue.StartAnalyses 2024-05-20 20:07:39 -07:00
Michael Hohn
cf595f338a wip: port FileDownload 2024-05-20 14:28:33 -07:00