20 Commits

Author SHA1 Message Date
Michael Hohn
9fe6aed357 fix: add polling reconnect 2025-06-17 15:59:24 -07:00
Michael Hohn
3762654ef2 switch to pollling rabbitmq; update images 2025-06-16 16:07:37 -07:00
Michael Hohn
a5bb232af2 Use full repository path name in place of mrvacommander 2024-12-13 10:54:35 -08:00
Michael Hohn
4d52176c5a Add Publisher Confirms and Consumer Acknowledgements to rabbitmq channels
Also updated the end-to-end workflow

The confirmation channel size is intentionally very large to prevent
blocking the server or agents.
2024-11-14 12:04:18 -08:00
Michael Hohn
3566f5169e Type checking fix: Restrict the keys / values for ArtifactLocation and centralize the common ones 2024-07-08 12:07:46 -07:00
Michael Hohn
b3cf7a4f65 Introduce explicit type QueryLanguage = string and update code to clarify
Previously:
- There is confusion between nameWithOwner and queryLanguage.  Both are strings.
  Between

        runResult, err := codeql.RunQuery(databasePath, job.QueryLanguage, queryPackPath, tempDir)
    (agent.go l205)

  and

        func RunQuery(database string, nwo string, queryPackPath string, tempDir string) (*RunQueryResult, error)

  QueryLanguage is suddenly name with owner in the code.

  Added some debugging, the value is the query language in the two places it gets used:

        server         | 2024/07/03 18:30:15 DEBUG Processed request info location="{Data:map[bucket:packs key:1]}" language=cpp
        ...
        agent          | 2024/07/03 18:30:15 DEBUG XX: is nwo a name/owner, or the original callers' queryLanguage? nwo=cpp
        ...
        agent          | 2024/07/03 18:30:19 DEBUG XX: 2: is nwo a name/owner, or the original callers' queryLanguage? nwo=cpp

Changes:
- Introduce explicit type QueryLanguage = string and update code to clarify
- inline trivial function
2024-07-03 13:30:02 -07:00
Nicolas Will
e0cbc01d21 Fully implement local and container MRVA 2024-06-24 01:31:28 -04:00
Nicolas Will
fc9fcc7ae6 Add server queue logic and refactor 2024-06-17 11:30:46 +02:00
Michael Hohn
2c5ecd3a1e Merge the agent-impl branch into the server branch 2024-06-16 19:21:42 -07:00
Michael Hohn
cd0647836e Combine New/Setup functions 2024-06-16 19:09:32 -07:00
Nicolas Will
903ca5673e Add dynamic worker management 2024-06-16 17:07:13 +02:00
Nicolas Will
7ea45cb176 Separate queue and agent logic and refactor 2024-06-16 11:18:22 +02:00
Nicolas Will
c29daab045 Standardize NameWithOwner and Visible naming
Acronyms are now "NWO" and "Vis" respsectively
2024-06-14 12:55:45 +02:00
Michael Hohn
9c0cdb1fe4 Simplify naming, don't restate package name 2024-06-11 16:55:10 -07:00
Michael Hohn
7e0d6909da wip: Make cross-module visibility explicit via Visibles structs
All access is/will be through interfaces accessed through these structs.

This introduces several distinct storage units:
+ DB for server state
+ DB for codeql databases
+ query pack store

The steps for manually creating needed databases are in the README
2024-06-07 13:14:41 -07:00
Michael Hohn
25cab583c1 wip: storage using postgres / gorm using partial json
Several approaches of normalizing json were tried and ultimately found
impractical at this point.

Using a hybrid of tables and json is the current approach; this may be
further normalized later.
2024-06-06 13:19:00 -07:00
Michael Hohn
f7155eba50 wip: add analysis runner / agent, separate Server/Queue/Agent, use New* initializers 2024-05-23 15:46:55 -07:00
Michael Hohn
2ab596bf1d wip: Move all references to github.com/hohn/ghes-mirva-server 2024-05-22 14:39:12 -07:00
Michael Hohn
8cd4f4d809 wip: port queue.StartAnalyses 2024-05-20 20:07:39 -07:00
Michael Hohn
9750eeab20 wip: reorganized to go pkg structure 2024-05-15 15:57:27 -07:00