provvedo/wiregui
2
0
Fork 0
Code Issues Pull requests Projects Releases 6 Packages 1 Wiki Activity Actions

feat: WireGuard metrics collector + integration test stack
Some checks failed
Dev / test (push) Failing after 2m43s
Details
Dev / docker (push) Has been skipped
Details

Browse source 
Metrics collector (wiregui/collector.py):
- Standalone process spawned by web app when WG_METRICS_ENABLED=true
- Polls wg show dump every WG_METRICS_POLL_INTERVAL seconds (default 5)
- Updates device stats in PostgreSQL
- Pushes Prometheus-format metrics to VictoriaMetrics (if configured)
- Graceful shutdown on SIGTERM

Integration test stack (compose.yml):
- Unified compose file for dev, test, and integration modes
- VictoriaMetrics single-node TSDB for metrics storage
- 3 mock WireGuard client containers generating ping traffic
- Automated setup script seeds server keypair, admin user, client devices
- make test-stack-up: one command to start everything
- make test-stack-verify: validates metrics flowing end-to-end

Infrastructure:
- Makefile with targets for dev, test, integration, and production
- Integration tests verify VictoriaMetrics has data for all 3 clients
- Fix Dockerfile to include img/ directory
- Separate TESTS.md for test tracking, clean TODO.md for features only
This commit is contained in:
Stefano Bertelli 2026-03-31 18:30:15 -05:00
parent 70eb9f6b12
commit c5b66349d6
16 changed files with 932 additions and 115 deletions
Download patch file Download diff file Expand all files Collapse all files

2
.forgejo/workflows/dev.yml
View file

@ -66,7 +66,7 @@ jobs:
run: uv run alembic upgrade head
- name: Run unit tests
run: uv run pytest tests/ --ignore=tests/e2e -v --tb=short
run: uv run pytest tests/ --ignore=tests/e2e --ignore=tests/integration -v --tb=short
- name: Run E2E tests
run: uv run pytest tests/e2e/ -v --tb=short

2
.forgejo/workflows/release.yml
View file

@ -67,7 +67,7 @@ jobs:
run: uv run alembic upgrade head
- name: Run unit tests
run: uv run pytest tests/ --ignore=tests/e2e -v --tb=short
run: uv run pytest tests/ --ignore=tests/e2e --ignore=tests/integration -v --tb=short
- name: Run E2E tests
run: uv run pytest tests/e2e/ -v --tb=short

1
.gitignore vendored
View file

@ -6,3 +6,4 @@ __pycache__/
logs/
.idea/
.coverage
docker/mock-clients/configs/

2
Dockerfile
View file

@ -20,6 +20,7 @@ RUN uv sync --no-dev --frozen 2>/dev/null || uv sync --no-dev
COPY wiregui/ wiregui/
COPY alembic/ alembic/
COPY alembic.ini ./
COPY img/ img/
FROM python:3.13-slim AS runner
@ -34,6 +35,7 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
COPY --from=builder /usr/local/bin/uv /usr/local/bin/uv
COPY --from=builder /app/.venv /app/.venv
COPY --from=builder /app/wiregui /app/wiregui
COPY --from=builder /app/img /app/img
COPY --from=builder /app/alembic /app/alembic
COPY --from=builder /app/alembic.ini /app/alembic.ini
COPY --from=builder /app/pyproject.toml /app/pyproject.toml

123
Makefile Normal file
View file

@ -0,0 +1,123 @@
.PHONY: help install migrate dev dev-up dev-down dev-logs \
test test-unit test-e2e test-e2e-headed \
test-stack-up test-stack-seed test-stack-down test-stack-logs test-stack-verify \
prod-build \
clean
# Default target
help:
@echo "WireGUI — available targets:"
@echo ""
@echo " Development (app runs on host, infra in Docker):"
@echo " make install Install dependencies (uv sync)"
@echo " make migrate Run database migrations"
@echo " make dev Start infra + mock IdPs, run app locally"
@echo " make dev-up Start infra only (Postgres, Valkey, mock IdPs)"
@echo " make dev-down Stop all containers"
@echo " make dev-logs Tail container logs"
@echo ""
@echo " Testing:"
@echo " make test Run unit + e2e tests"
@echo " make test-unit Run unit tests only"
@echo " make test-e2e Run e2e tests (headless)"
@echo " make test-e2e-headed Run e2e tests in headed mode (visible browser)"
@echo ""
@echo " Integration stack (containerized WireGUI + WG clients + VictoriaMetrics):"
@echo " make test-stack-up Seed DB, build, start everything"
@echo " make test-stack-down Stop and remove containers + volumes"
@echo " make test-stack-logs Tail logs"
@echo " make test-stack-verify Verify metrics flowing to VictoriaMetrics"
@echo ""
@echo " Production:"
@echo " make prod-build Build production Docker image"
@echo ""
@echo " Housekeeping:"
@echo " make clean Remove generated files, caches, volumes"
# ---------------------------------------------------------------------------
# Development
# ---------------------------------------------------------------------------
install:
uv sync
migrate:
uv run alembic upgrade head
dev-up:
docker compose up -d postgres valkey mock-oidc mock-saml
dev-down:
docker compose down
dev-logs:
docker compose logs -f
dev: dev-up migrate
uv run python -m wiregui.main
# ---------------------------------------------------------------------------
# Testing
# ---------------------------------------------------------------------------
test-unit:
uv run pytest tests/ --ignore=tests/e2e --ignore=tests/integration -v --tb=short
test-e2e:
uv run pytest tests/e2e/ -v --tb=short
test-e2e-headed:
uv run pytest tests/e2e/ --headed --slowmo 300 -v --tb=short
test: test-unit test-e2e
# ---------------------------------------------------------------------------
# Integration test stack (real WireGuard + mock clients + VictoriaMetrics)
# ---------------------------------------------------------------------------
test-stack-up: test-stack-seed
docker compose up -d --build wiregui client1 client2 client3
@echo ""
@echo "Integration stack running:"
@echo " WireGUI: http://localhost:13000 (admin@test.local / admin123)"
@echo " VictoriaMetrics: http://localhost:8428"
@echo " Mock clients: 3 peers generating traffic every 3s"
test-stack-seed:
@echo "[*] Starting infrastructure..."
docker compose up -d postgres valkey victoriametrics
@echo "[*] Waiting for Postgres..."
@until docker compose exec -T postgres pg_isready -U wiregui > /dev/null 2>&1; do sleep 1; done
@echo "[*] Running migrations..."
uv run alembic upgrade head
@echo "[*] Seeding server keypair, admin user, and client devices..."
PYTHONPATH=. uv run python docker/mock-clients/setup.py
test-stack-down:
docker compose down -v
test-stack-verify:
uv run pytest tests/integration/ -v --tb=short
test-stack-logs:
docker compose logs -f wiregui client1 client2 client3 victoriametrics
# ---------------------------------------------------------------------------
# Production
# ---------------------------------------------------------------------------
PROD_IMAGE ?= wiregui
PROD_TAG ?= latest
prod-build:
docker build --no-cache -t $(PROD_IMAGE):$(PROD_TAG) .
# ---------------------------------------------------------------------------
# Housekeeping
# ---------------------------------------------------------------------------
clean:
find . -type d -name __pycache__ -exec rm -rf {} + 2>/dev/null || true
rm -rf .pytest_cache .coverage htmlcov
rm -rf docker/mock-clients/configs/
rm -rf .nicegui/

77
TESTS.md Normal file
View file

@ -0,0 +1,77 @@
# WireGUI — Test Suite
**Test count: 271 (201 unit + 70 E2E) | Unit coverage: 36% | Effective: ~81% (incl. E2E)**
**Run:** `uv run pytest` (unit) / `uv run pytest tests/e2e/` (E2E via Playwright)
---
## Unit Tests — Coverage by Module
**Done:**
- [x] `wiregui/api/deps.py` (91%) — 11 tests: Bearer token auth, get_current_api_user, require_admin
- [x] `wiregui/services/wireguard.py` (98%) — 6 tests: ensure_interface, set_private_key, set_listen_port, configure_interface
- [x] `wiregui/services/firewall.py` (94%) — 17 tests: _nft/_nft_batch errors, jump rules, policies, get_ruleset
- [x] `wiregui/auth/api_token.py` (100%) — covered via test_api_deps.py
- [x] `wiregui/auth/saml.py` — full SAML flow tested via mock SimpleSAMLphp IdP (e2e)
- [x] `wiregui/utils/server_key.py` (100%) — 3 tests: returns key, raises when missing, raises when empty
**Remaining unit test gaps (by coverage):**
- [ ] `wiregui/auth/seed.py` (29%) — test seed_admin, seed_idp_providers with various YAML configs, ensure_server_keypair
- [ ] `wiregui/tasks/__init__.py` (35%) — test register_task, cancel_all
- [ ] `wiregui/tasks/oidc_refresh.py` (40%) — test successful refresh, failure with notification, disable_vpn_on_oidc_error
- [ ] `wiregui/api/v0/configuration.py` (55%) — test GET/PUT configuration endpoints
- [ ] `wiregui/api/v0/devices.py` (65%) — test CRUD device API endpoints
- [ ] `wiregui/api/v0/rules.py` (70%) — test CRUD rule API endpoints
- [ ] `wiregui/tasks/connectivity.py` (72%) — test connectivity check loop
- [ ] `wiregui/utils/network.py` (73%) — test IPv6 allocation, edge cases in CIDR validation
- [ ] `wiregui/tasks/stats.py` (74%) — test WG stats polling loop
- [ ] `wiregui/tasks/vpn_session.py` (77%) — test session expiry loop
- [ ] `wiregui/auth/webauthn.py` (87%) — test verify_registration, verify_authentication with mock credential data
- [ ] `wiregui/auth/middleware.py` (0%) — test NiceGUI auth middleware redirect logic
---
## E2E Tests (Playwright)
**Completed test suites:**
- [x] `tests/e2e/test_login.py` (6 tests) — valid login, invalid password, nonexistent email, disabled user, logout, unauthenticated redirect
- [x] `tests/e2e/test_devices.py` (2 tests) — add device full flow, name validation
- [x] `tests/e2e/test_account.py` (8 tests) — change password (success/wrong/mismatch/short), create API token, TOTP registration + invalid code, account deletion
- [x] `tests/e2e/test_admin_users.py` (10 tests) — page renders, create user, duplicate email, edit role/password, disable/enable, delete, cascade delete, self-delete guard
- [x] `tests/e2e/test_idp_seed.py` (9 tests) — IdP YAML seeding (noop/missing/invalid, OIDC/SAML add, upsert, preserve), OIDC button visible, full OIDC login flow via mock-oidc
- [x] `tests/e2e/test_mfa_login.py` (4 tests) — MFA redirect on login, valid TOTP completes login, invalid code error, cancel returns to login
- [x] `tests/e2e/test_magic_link_page.py` (4 tests) — page renders, success on submit, empty email error, back to login
- [x] `tests/e2e/test_admin_devices.py` (7 tests) — list all devices, filter by user, create with defaults, create with overrides, edit name/description, delete, config dialog with QR
- [x] `tests/e2e/test_admin_rules.py` (7 tests) — list rules table, create accept/drop/global rules, edit action/destination, delete rule (all verified in DB)
- [x] `tests/e2e/test_admin_settings.py` (9 tests) — client defaults save/reload, security toggles (local auth, VPN session, unprivileged), OIDC add/delete, SAML add/delete (all verified in DB)
- [x] `tests/e2e/test_saml_login.py` (4 tests) — SAML button visible, redirect to IdP, SP metadata endpoint, full SAML login flow via mock SimpleSAMLphp
**Remaining E2E test suites:**
`tests/e2e/test_admin_diagnostics.py` — Admin Diagnostics:
- [ ] Page renders WireGuard interface status
- [ ] Active peers table shows devices with handshakes
- [ ] Connectivity checks table shows recent results
- [ ] Notifications list shows system notifications
- [ ] Clear single notification → removed
- [ ] Clear all notifications → list empty
`tests/e2e/test_devices_user.py` — User Device Pages:
- [ ] Device list shows only own devices (not other users')
- [ ] Create device → shows in table with allocated IPs
- [ ] Device detail page shows public key, IPs, stats, active config
- [ ] Device detail: edit name → persists
- [ ] Device detail: toggle config overrides → custom values saved
- [ ] Device detail: delete with confirmation → redirects to /devices
- [ ] Auto-refresh: stats labels update after timer fires (mock timer)
`tests/e2e/test_account_extended.py` — Account Page (additional):
- [ ] SSO providers section shows connected providers
- [ ] SSO providers section shows "No SSO providers" when empty
- [ ] MFA: add security key (WebAuthn) → method appears in table (mock navigator.credentials)
- [ ] MFA: delete method with confirmation → removed from table
- [ ] API tokens: expired token shows "Expired" badge
- [ ] API tokens: delete token → removed from table
- [ ] API tokens: copy button calls clipboard API
- [ ] Danger zone: disabled when only admin
- [ ] Danger zone: wrong email in confirmation → shows error

161
TODO.md
View file

@ -1,130 +1,87 @@
# WireGUI — Pending Items
**Test count: 268 (198 unit + 70 E2E) | Coverage: 36% unit, ~63% effective (incl. E2E)**
# WireGUI — TODO
---
## Testing
## WireGuard Metrics Collector
# WireGUI Implementation TODO
### Overview
Migration of Wirezone (Elixir/Phoenix) to Python/NiceGUI.
Source: `/home/stefanob/PycharmProjects/personal/wirezone`
Separate Python process dedicated to high-frequency WireGuard stats collection, with optional VictoriaMetrics time-series storage. Replaces the current 60s in-process polling with a 5s external collector.
**Test count: 268 (198 unit + 70 E2E) | Coverage: 36% unit, ~63% effective (incl. E2E)**
**Run:** `uv run pytest` (unit) / `uv run pytest tests/e2e/` (E2E via Playwright)
### Current state
- `tasks/stats.py`: polls `wg show dump` every 60s inside the web process asyncio loop
- UI timers: 30s refresh on device pages
- Worst-case latency: ~90s before a stat change is visible
### Target state
- Collector process: polls every 5s, writes to DB + VictoriaMetrics
- UI timers: 10s refresh
- Worst-case latency: ~15s
## Phase 7: Admin UI ✅
### Phase 1: Configuration ✅
- [ ] **TODO:** SAML provider management in Authentication tab
- [x] Add settings to `config.py`:
- `WG_METRICS_ENABLED: bool = False`
- `WG_METRICS_POLL_INTERVAL: int = 5` (seconds)
- `WG_VICTORIAMETRICS_URL: str | None = None` (e.g. `http://localhost:8428`)
- [x] When `WG_METRICS_ENABLED=false`, keep existing `stats_loop` as fallback
- [x] When `WG_METRICS_ENABLED=true`, skip registering `stats_loop` in `main.py`
## Phase 10: Polish, Testing & Deployment
### Phase 2: Collector process ✅
### Testing (partially done)
- [ ] HTTP-level integration tests (OIDC redirect/callback flow with respx mocking)
- [x] `wiregui/api/deps.py` (11 tests) — resolve_bearer_token (valid/expired/invalid/disabled/no-expiry), get_current_api_user (missing header/bad scheme/invalid token/valid token), require_admin (admin/unprivileged)
- [x] `wiregui/services/wireguard.py` (6 tests) — ensure_interface (exists/creates new), set_private_key, set_listen_port, configure_interface (no config/sets key+port)
- [x] `wiregui/services/firewall.py` (17 tests) — _nft error/success, _nft_batch error/stdin, add_device_jump_rule (ipv4-only/ipv6-only/no-ips/both), setup_base_tables error handling, masquerade error, peer-to-peer/lan-to-peers policies, get_ruleset fallback
- [ ] `wiregui/tasks/oidc_refresh.py` — test successful refresh, failure with notification, disable_vpn_on_oidc_error
- [x] `wiregui/auth/saml.py` — full SAML flow tested via mock SimpleSAMLphp IdP (e2e)
- [ ] `wiregui/auth/webauthn.py` — test verify_registration, verify_authentication with mock credential data
- [ ] E2E tests for admin pages (users, devices, rules, settings)
- [x] Create `wiregui/collector.py` — standalone entry point (`python -m wiregui.collector`)
- [x] No NiceGUI dependency — only asyncio + asyncpg + httpx
- [x] Poll `wg show <iface> dump` every `WG_METRICS_POLL_INTERVAL` seconds
- [x] Update Device rows in PostgreSQL (same fields as current `stats_loop`)
- [x] Push metrics to VictoriaMetrics via `/api/v1/import/prometheus` (if URL configured)
- [x] Graceful shutdown on SIGTERM/SIGINT
- [x] Web app spawns collector as subprocess when `WG_METRICS_ENABLED=true`
- [x] Web app terminates collector on shutdown
**E2E page tests (Playwright async API in `tests/e2e/`):**
- [x] `tests/e2e/test_login.py` (6 tests) — valid login, invalid password, nonexistent email, disabled user, logout, unauthenticated redirect
- [x] `tests/e2e/test_devices.py` (2 tests) — add device full flow, name validation
- [x] `tests/e2e/test_account.py` (8 tests) — change password (success/wrong/mismatch/short), create API token, TOTP registration + invalid code, account deletion
- [x] `tests/e2e/test_admin_users.py` (10 tests) — page renders, create user, duplicate email, edit role/password, disable/enable, delete, cascade delete, self-delete guard
- [x] `tests/e2e/test_idp_seed.py` (9 tests) — IdP YAML seeding (noop/missing/invalid, OIDC/SAML add, upsert, preserve), OIDC button visible, full OIDC login flow via mock-oidc
- [x] `tests/e2e/test_mfa_login.py` (4 tests) — MFA redirect on login, valid TOTP completes login, invalid code error, cancel returns to login
- [x] `tests/e2e/test_magic_link_page.py` (4 tests) — page renders, success on submit, empty email error, back to login
- [x] `tests/e2e/test_admin_devices.py` (7 tests) — list all devices, filter by user, create with defaults, create with overrides, edit name/description, delete, config dialog with QR
- [x] `tests/e2e/test_admin_rules.py` (7 tests) — list rules table, create accept/drop/global rules, edit action/destination, delete rule (all verified in DB)
- [x] `tests/e2e/test_admin_settings.py` (9 tests) — client defaults save/reload, security toggles (local auth, VPN session, unprivileged), OIDC add/delete, SAML add/delete (all verified in DB)
- [x] `tests/e2e/test_saml_login.py` (4 tests) — SAML button visible, redirect to IdP, SP metadata endpoint, full SAML login flow via mock SimpleSAMLphp
### Phase 3: VictoriaMetrics metrics
**E2E tests still needed:**
Metrics to push (Prometheus exposition format):
- [ ] `wiregui_peer_rx_bytes{public_key, user_email, device_name}` — counter
- [ ] `wiregui_peer_tx_bytes{public_key, user_email, device_name}` — counter
- [ ] `wiregui_peer_latest_handshake_seconds{public_key, user_email, device_name}` — gauge
- [ ] `wiregui_peer_connected{public_key, user_email, device_name}` — 1 if handshake < 180s, else 0
- [ ] `wiregui_peers_total` — gauge, count of active peers
`tests/e2e/test_login.py` — Login & Auth flows (remaining):
- [x] Login with MFA → redirects to /mfa challenge page
- [x] MFA challenge: valid TOTP code → completes login
- [x] MFA challenge: invalid code → shows error, stays on /mfa
- [x] MFA challenge: cancel → returns to /login
- [x] Magic link request page renders, shows success on submit
### Phase 4: UI improvements
`tests/e2e/test_admin_devices.py` — Admin Device Management:
- [x] List all devices across users
- [x] Filter by user → shows only that user's devices
- [x] Create device with full config overrides (DNS, endpoint, MTU, keepalive, allowed IPs)
- [x] Create device with defaults → use_default flags all True
- [x] Edit device name and description → persists
- [x] Edit device config overrides (toggle use_default off, set custom values)
- [x] Delete device → removed from table
- [x] Config dialog shows valid WireGuard config with real server public key
- [x] QR code renders in config dialog
- [ ] Reduce UI timer from 30s to 10s on device pages (devices.py, admin/devices.py)
- [ ] Add connection status indicator (green/yellow/red dot) based on handshake age
- Green: handshake < 2 min
- Yellow: handshake < 5 min
- Red: no recent handshake or never connected
- [ ] Add traffic rate display (bytes/sec computed from delta between polls)
- [ ] Device detail page: mini traffic chart (query VictoriaMetrics if available, else show last-known values)
`tests/e2e/test_admin_rules.py` — Admin Firewall Rules:
- [x] List rules → table shows action, destination, protocol, port, user
- [x] Create accept rule with CIDR → appears in table
- [x] Create drop rule with TCP port range → appears correctly
- [x] Create global rule (no user) → shows "Global"
- [x] Edit rule action (accept → drop) → persists
- [x] Edit rule destination → persists
- [x] Delete rule → removed from table
### Phase 5: Infrastructure ✅
`tests/e2e/test_admin_settings.py` — Admin Settings:
- [x] Client defaults: save endpoint, DNS, MTU, keepalive, allowed IPs → persists in DB
- [x] Client defaults: saved values reflected on page reload
- [x] Security: toggle local auth → persists
- [x] Security: change VPN session duration → persists
- [x] Security: toggle unprivileged device management/configuration → persists
- [x] OIDC: add provider → appears in table
- [x] OIDC: delete provider → removed from table
- [x] SAML: add provider → appears in table
- [x] SAML: delete provider → removed from table
- [x] Create `compose.test.yml` — full integration stack with real WG
- [x] Add VictoriaMetrics (single-node, port 8428, 7d retention)
- [x] Add 3 mock WG client containers (alpine + wireguard-tools)
- [x] Clients generate traffic by pinging each other through the tunnel every 3s
- [x] Setup script (`docker/mock-clients/setup.py`) generates keypairs and configs
- [x] Collector runs as subprocess inside the WireGUI container (shares network namespace)
- [ ] Add VictoriaMetrics to dev `compose.yml` (optional, for local testing)
`tests/e2e/test_admin_diagnostics.py` — Admin Diagnostics:
- [ ] Page renders WireGuard interface status
- [ ] Active peers table shows devices with handshakes
- [ ] Connectivity checks table shows recent results
- [ ] Notifications list shows system notifications
- [ ] Clear single notification → removed
- [ ] Clear all notifications → list empty
### Design notes
`tests/e2e/test_devices_user.py` — User Device Pages:
- [ ] Device list shows only own devices (not other users')
- [ ] Create device → shows in table with allocated IPs
- [ ] Device detail page shows public key, IPs, stats, active config
- [ ] Device detail: edit name → persists
- [ ] Device detail: toggle config overrides → custom values saved
- [ ] Device detail: delete with confirmation → redirects to /devices
- [ ] Auto-refresh: stats labels update after timer fires (mock timer)
- **Why a separate process?** The `wg show` subprocess call and DB writes at 5s intervals shouldn't share the asyncio loop with the web app. A separate process ensures UI responsiveness isn't affected by stats collection.
- **Why not `run.cpu_bound`?** That uses `ProcessPoolExecutor` for one-shot CPU tasks inside request handling — not suitable for a long-running daemon. A separate entry point is cleaner.
- **VictoriaMetrics push model:** Use the Prometheus remote write API. No scrape config needed — the collector pushes directly. VictoriaMetrics is optional; the collector works fine with just PostgreSQL.
- **Backward compatible:** When `WG_METRICS_ENABLED=false` (default), everything works exactly as it does today.
---
## UI
- [ ] SAML provider management in Authentication tab (admin settings)
- [ ] SSO Providers on account page: add Status column, "Disconnect" action
- [ ] Admin pages (users, devices, rules): apply same card-based styling as account/settings/diagnostics
`tests/e2e/test_account_extended.py` — Account Page (additional):
- [ ] SSO providers section shows connected providers
- [ ] SSO providers section shows "No SSO providers" when empty
- [ ] MFA: add security key (WebAuthn) → method appears in table (mock navigator.credentials)
- [ ] MFA: delete method with confirmation → removed from table
- [ ] API tokens: expired token shows "Expired" badge
- [ ] API tokens: delete token → removed from table
- [ ] API tokens: copy button calls clipboard API
- [ ] Danger zone: disabled when only admin
- [ ] Danger zone: wrong email in confirmation → shows error
## Features
### Deployment ✅
- [ ] First-run CLI setup command
---
### Remaining
- [ ] SSO Providers: add Status column, "Disconnect" action
- [ ] Admin pages (users, devices, rules): apply same card-based styling

127
compose.yml
View file

@ -1,12 +1,29 @@
# WireGUI — unified compose stack
#
# Dev mode (app runs on host):
# make dev — starts infra + mock IdPs, runs app locally
# make dev-up — starts infra only
#
# Integration test mode (real WireGuard + mock clients + metrics):
# make test-stack-up — seeds DB, builds, starts everything
# make test-stack-down — tears down and removes volumes
#
# Services are opt-in: only start what you need.
services:
# ---------------------------------------------------------------------------
# Core infrastructure (always needed)
# ---------------------------------------------------------------------------
postgres:
image: postgres:17
ports:
- "5432:5432"
environment:
POSTGRES_USER: wiregui
POSTGRES_PASSWORD: wiregui
POSTGRES_DB: wiregui
ports:
- "5432:5432"
volumes:
- postgres_data:/var/lib/postgresql/data
@ -17,9 +34,12 @@ services:
volumes:
- valkey_data:/data
# Test OIDC Identity Provider — accepts any login, issues real JWTs
# ---------------------------------------------------------------------------
# Mock identity providers (dev + e2e tests)
# ---------------------------------------------------------------------------
# OIDC — accepts any login, issues real JWTs
# Discovery: http://localhost:9000/test-idp/.well-known/openid-configuration
# Login: enter any username/password, it will issue a token
mock-oidc:
image: ghcr.io/navikt/mock-oauth2-server:2.1.10
ports:
@ -49,10 +69,10 @@ services:
]
}
# Test SAML Identity Provider — SimpleSAMLphp as IdP
# IdP Metadata: http://localhost:8080/simplesaml/saml2/idp/metadata.php
# Admin UI: http://localhost:8080/simplesaml (admin / secret)
# Test users: user1/password, user2/password
# SAML — SimpleSAMLphp as IdP
# Metadata: http://localhost:8080/simplesaml/saml2/idp/metadata.php
# Admin: http://localhost:8080/simplesaml (admin / secret)
# Users: user1/password, user2/password
mock-saml:
image: kenchan0130/simplesamlphp
ports:
@ -64,6 +84,97 @@ services:
volumes:
- ./docker/mock-saml/saml20-sp-remote.php:/var/www/simplesamlphp/metadata/saml20-sp-remote.php:ro
# ---------------------------------------------------------------------------
# WireGUI server (integration test mode — containerized with real WG)
# ---------------------------------------------------------------------------
wiregui:
build: .
ports:
- "13000:13000"
# 51820/udp exposed inside Docker network only — clients connect via service name
# Uncomment to expose to host: - "51820:51820/udp"
environment:
WG_DATABASE_URL: postgresql+asyncpg://wiregui:wiregui@postgres/wiregui
WG_REDIS_URL: redis://valkey:6379/0
WG_WG_ENABLED: "true"
WG_EXTERNAL_URL: http://localhost:13000
WG_ENDPOINT_HOST: wiregui
WG_METRICS_ENABLED: "true"
WG_METRICS_POLL_INTERVAL: "5"
WG_VICTORIAMETRICS_URL: http://victoriametrics:8428
WG_ADMIN_EMAIL: admin@test.local
WG_ADMIN_PASSWORD: admin123
WG_LOG_TO_FILE: "false"
WG_SECRET_KEY: test-secret-key-for-integration
cap_add:
- NET_ADMIN
sysctls:
- net.ipv4.ip_forward=1
- net.ipv6.conf.all.forwarding=1
depends_on:
- postgres
- valkey
# ---------------------------------------------------------------------------
# Metrics (integration test mode)
# ---------------------------------------------------------------------------
victoriametrics:
image: victoriametrics/victoria-metrics:v1.108.1
ports:
- "8428:8428"
command:
- "-retentionPeriod=7d"
- "-httpListenAddr=:8428"
volumes:
- vm_data:/victoria-metrics-data
# ---------------------------------------------------------------------------
# Mock WireGuard clients (integration test mode)
# Configs generated by: make test-stack-seed
# ---------------------------------------------------------------------------
client1:
build: docker/mock-clients
environment:
CLIENT_IP: ${CLIENT1_IP:-10.3.2.101}
PEER_IPS: ${CLIENT1_PEERS:-10.3.2.102 10.3.2.103}
PING_INTERVAL: "3"
volumes:
- ./docker/mock-clients/configs/client1.conf:/etc/wireguard/wg0.conf:ro
cap_add:
- NET_ADMIN
depends_on:
- wiregui
client2:
build: docker/mock-clients
environment:
CLIENT_IP: ${CLIENT2_IP:-10.3.2.102}
PEER_IPS: ${CLIENT2_PEERS:-10.3.2.101 10.3.2.103}
PING_INTERVAL: "3"
volumes:
- ./docker/mock-clients/configs/client2.conf:/etc/wireguard/wg0.conf:ro
cap_add:
- NET_ADMIN
depends_on:
- wiregui
client3:
build: docker/mock-clients
environment:
CLIENT_IP: ${CLIENT3_IP:-10.3.2.103}
PEER_IPS: ${CLIENT3_PEERS:-10.3.2.101 10.3.2.102}
PING_INTERVAL: "3"
volumes:
- ./docker/mock-clients/configs/client3.conf:/etc/wireguard/wg0.conf:ro
cap_add:
- NET_ADMIN
depends_on:
- wiregui
volumes:
postgres_data:
valkey_data:
vm_data:

4
docker/mock-clients/Dockerfile Normal file
View file

@ -0,0 +1,4 @@
FROM alpine:3.20
RUN apk add --no-cache wireguard-tools iproute2 iputils-ping
COPY entrypoint.sh /entrypoint.sh
ENTRYPOINT ["/entrypoint.sh"]

27
docker/mock-clients/entrypoint.sh Executable file
View file

@ -0,0 +1,27 @@
#!/bin/sh
# WireGuard mock client — configures interface and generates traffic
set -e
echo "[*] Configuring WireGuard interface..."
ip link add wg0 type wireguard
wg setconf wg0 /etc/wireguard/wg0.conf
ip address add "${CLIENT_IP}/32" dev wg0
ip link set wg0 up
# Route traffic to the VPN subnet through the tunnel
ip route add 10.3.2.0/24 dev wg0
echo "[*] WireGuard client up: ${CLIENT_IP}"
echo "[*] Generating traffic to peers every ${PING_INTERVAL:-5}s..."
while true; do
# Ping the server (first host in the subnet)
ping -c 1 -W 1 10.3.2.1 > /dev/null 2>&1 || true
# Ping other peers if specified
for peer_ip in $PEER_IPS; do
ping -c 1 -W 1 "$peer_ip" > /dev/null 2>&1 || true
done
sleep "${PING_INTERVAL:-5}"
done

147
docker/mock-clients/setup.py Normal file
View file

@ -0,0 +1,147 @@
#!/usr/bin/env python3
"""Seed the test stack: generate server + client keypairs, write client WG configs,
and insert devices into the database.
Usage: uv run python docker/mock-clients/setup.py
Requires: Postgres running and migrations applied (alembic upgrade head).
"""
import asyncio
import sys
from pathlib import Path
from uuid import uuid4
from sqlmodel import select
from wiregui.auth.passwords import hash_password
from wiregui.db import async_session, engine
from wiregui.models.configuration import Configuration
from wiregui.models.device import Device
from wiregui.models.user import User
from wiregui.utils.crypto import generate_keypair, generate_preshared_key
NUM_CLIENTS = 3
SUBNET = "10.3.2"
SERVER_ENDPOINT = "wiregui:51820"
CONFIG_DIR = Path(__file__).parent / "configs"
# Test admin user
ADMIN_EMAIL = "admin@test.local"
ADMIN_PASSWORD = "admin123"
# Client definitions
CLIENTS = [
{"name": f"test-client-{i}", "ip": f"{SUBNET}.{100 + i}"}
for i in range(1, NUM_CLIENTS + 1)
]
async def seed():
CONFIG_DIR.mkdir(parents=True, exist_ok=True)
async with async_session() as session:
# --- Server keypair ---
config = (await session.execute(select(Configuration).limit(1))).scalar_one_or_none()
if config is None:
config = Configuration()
session.add(config)
await session.flush()
if not config.server_private_key or not config.server_public_key:
server_priv, server_pub = generate_keypair()
config.server_private_key = server_priv
config.server_public_key = server_pub
session.add(config)
print(f" Server keypair generated: {server_pub[:20]}...")
else:
server_pub = config.server_public_key
print(f" Server keypair already exists: {server_pub[:20]}...")
# --- Admin user ---
admin = (await session.execute(
select(User).where(User.email == ADMIN_EMAIL)
)).scalar_one_or_none()
if admin is None:
admin = User(
email=ADMIN_EMAIL,
password_hash=hash_password(ADMIN_PASSWORD),
role="admin",
)
session.add(admin)
await session.flush()
print(f" Admin user created: {ADMIN_EMAIL}")
else:
print(f" Admin user already exists: {ADMIN_EMAIL}")
# --- Client devices (delete + recreate for clean state) ---
client_names = [c["name"] for c in CLIENTS]
client_ips = [c["ip"] for c in CLIENTS]
stale = (await session.execute(
select(Device).where(
Device.name.in_(client_names) | Device.ipv4.in_(client_ips)
)
)).scalars().all()
for d in stale:
await session.delete(d)
if stale:
await session.flush()
print(f" Cleaned up {len(stale)} stale device(s)")
for client in CLIENTS:
client_priv, client_pub = generate_keypair()
psk = generate_preshared_key()
device = Device(
name=client["name"],
public_key=client_pub,
preshared_key=psk,
ipv4=client["ip"],
user_id=admin.id,
)
session.add(device)
client["privkey"] = client_priv
client["pubkey"] = client_pub
client["psk"] = psk
print(f" Device '{client['name']}' created ({client['ip']})")
await session.commit()
# --- Write client WG configs ---
for i, client in enumerate(CLIENTS):
conf = f"""[Interface]
PrivateKey = {client["privkey"]}
[Peer]
PublicKey = {server_pub}
PresharedKey = {client["psk"]}
Endpoint = {SERVER_ENDPOINT}
AllowedIPs = {SUBNET}.0/24
PersistentKeepalive = 5
"""
conf_path = CONFIG_DIR / f"client{i + 1}.conf"
conf_path.write_text(conf)
print(f" Config written: {conf_path}")
# --- Write env vars for compose ---
env_lines = []
for i, client in enumerate(CLIENTS):
other_ips = " ".join(c["ip"] for c in CLIENTS if c["ip"] != client["ip"])
env_lines.append(f"CLIENT{i + 1}_IP={client['ip']}")
env_lines.append(f"CLIENT{i + 1}_PEERS={other_ips}")
env_path = CONFIG_DIR / "clients.env"
env_path.write_text("\n".join(env_lines) + "\n")
await engine.dispose()
def main():
print("[*] Seeding test stack...")
asyncio.run(seed())
print("\n[*] Done. Start the stack with:")
print(" make test-stack-up")
if __name__ == "__main__":
main()

0
tests/integration/__init__.py Normal file
View file

156
tests/integration/test_metrics_pipeline.py Normal file
View file

@ -0,0 +1,156 @@
"""Integration test: verify metrics flow from WG clients → collector → VictoriaMetrics.
Requires the full integration stack running: make test-stack-up
Run with: make test-stack-verify (or: uv run pytest tests/integration/ -v)
"""
import os
import time
import httpx
import pytest
VM_URL = os.environ.get("WG_VICTORIAMETRICS_URL", "http://localhost:8428")
WIREGUI_URL = os.environ.get("WG_EXTERNAL_URL", "http://localhost:13000")
EXPECTED_CLIENTS = ["test-client-1", "test-client-2", "test-client-3"]
# Wait up to this long for metrics to appear (collector runs every 5s)
MAX_WAIT = 60
POLL_INTERVAL = 5
def _vm_query(query: str) -> dict:
"""Execute an instant query against VictoriaMetrics."""
resp = httpx.get(f"{VM_URL}/api/v1/query", params={"query": query}, timeout=5)
resp.raise_for_status()
return resp.json()
def _vm_series(metric: str) -> list[dict]:
"""Get all series for a metric from VictoriaMetrics."""
resp = httpx.get(f"{VM_URL}/api/v1/series", params={"match[]": metric}, timeout=5)
resp.raise_for_status()
return resp.json().get("data", [])
@pytest.fixture(scope="module", autouse=True)
def check_stack_running():
"""Skip all tests if the integration stack isn't running."""
try:
r = httpx.get(f"{WIREGUI_URL}/api/health", timeout=3)
if r.status_code != 200:
pytest.skip("WireGUI not running")
except httpx.HTTPError:
pytest.skip("WireGUI not running — start with: make test-stack-up")
try:
r = httpx.get(f"{VM_URL}/health", timeout=3)
if r.status_code != 200:
pytest.skip("VictoriaMetrics not running")
except httpx.HTTPError:
pytest.skip("VictoriaMetrics not running — start with: make test-stack-up")
@pytest.fixture(scope="module")
def wait_for_metrics():
"""Wait until at least one peer metric appears in VictoriaMetrics."""
deadline = time.time() + MAX_WAIT
while time.time() < deadline:
result = _vm_query("wiregui_peers_total")
data = result.get("data", {}).get("result", [])
if data and float(data[0].get("value", [0, "0"])[1]) > 0:
return
time.sleep(POLL_INTERVAL)
pytest.fail(f"No metrics appeared in VictoriaMetrics after {MAX_WAIT}s")
def test_peers_total(wait_for_metrics):
"""wiregui_peers_total reports at least 1 active peer."""
result = _vm_query("wiregui_peers_total")
data = result["data"]["result"]
assert len(data) > 0
value = float(data[0]["value"][1])
assert value >= 1, f"Expected at least 1 peer, got {value}"
def test_rx_bytes_per_client(wait_for_metrics):
"""Each client has wiregui_peer_rx_bytes > 0."""
series = _vm_series("wiregui_peer_rx_bytes")
device_names = {s.get("device_name") for s in series}
for client in EXPECTED_CLIENTS:
assert client in device_names, (
f"Missing rx_bytes metric for '{client}'. "
f"Found: {device_names}"
)
# Verify values are non-zero (traffic is flowing)
for client in EXPECTED_CLIENTS:
result = _vm_query(f'wiregui_peer_rx_bytes{{device_name="{client}"}}')
data = result["data"]["result"]
assert len(data) > 0, f"No rx_bytes data for {client}"
value = float(data[0]["value"][1])
assert value > 0, f"rx_bytes for {client} is 0 — no traffic?"
def test_tx_bytes_per_client(wait_for_metrics):
"""Each client has wiregui_peer_tx_bytes > 0."""
for client in EXPECTED_CLIENTS:
result = _vm_query(f'wiregui_peer_tx_bytes{{device_name="{client}"}}')
data = result["data"]["result"]
assert len(data) > 0, f"No tx_bytes data for {client}"
value = float(data[0]["value"][1])
assert value > 0, f"tx_bytes for {client} is 0 — no traffic?"
def test_handshake_per_client(wait_for_metrics):
"""Each client has a recent handshake timestamp."""
now = time.time()
for client in EXPECTED_CLIENTS:
result = _vm_query(f'wiregui_peer_latest_handshake_seconds{{device_name="{client}"}}')
data = result["data"]["result"]
assert len(data) > 0, f"No handshake data for {client}"
ts = float(data[0]["value"][1])
assert ts > 0, f"Handshake timestamp for {client} is 0"
age = now - ts
assert age < 300, f"Handshake for {client} is {age:.0f}s old (stale?)"
def test_connected_status_per_client(wait_for_metrics):
"""Each client reports wiregui_peer_connected = 1."""
for client in EXPECTED_CLIENTS:
result = _vm_query(f'wiregui_peer_connected{{device_name="{client}"}}')
data = result["data"]["result"]
assert len(data) > 0, f"No connected status for {client}"
value = int(float(data[0]["value"][1]))
assert value == 1, f"Client {client} not connected (wiregui_peer_connected={value})"
def test_db_devices_have_stats():
"""Verify device rows in PostgreSQL also have updated stats."""
import asyncio
from sqlmodel import select
from wiregui.db import async_session, engine
from wiregui.models.device import Device
async def check():
async with async_session() as session:
result = await session.execute(
select(Device).where(Device.name.in_(EXPECTED_CLIENTS))
)
devices = result.scalars().all()
assert len(devices) == len(EXPECTED_CLIENTS), (
f"Expected {len(EXPECTED_CLIENTS)} devices, found {len(devices)}"
)
for device in devices:
assert device.latest_handshake is not None, (
f"Device {device.name} has no handshake in DB"
)
assert device.rx_bytes is not None and device.rx_bytes > 0, (
f"Device {device.name} has no rx_bytes in DB"
)
await engine.dispose()
asyncio.run(check())

176
wiregui/collector.py Normal file
View file

@ -0,0 +1,176 @@
"""WireGuard metrics collector — standalone process for high-frequency stats polling.
Run as: python -m wiregui.collector
Polls `wg show <iface> dump` at a configurable interval, updates device rows
in PostgreSQL, and optionally pushes Prometheus-format metrics to VictoriaMetrics.
"""
import asyncio
import signal
import time
import httpx
from loguru import logger
from sqlmodel import select
from wiregui.config import get_settings
from wiregui.db import async_session, engine
from wiregui.log_config import setup_logging
from wiregui.models.device import Device
from wiregui.models.user import User
from wiregui.services.wireguard import PeerInfo, get_peers
_shutdown = asyncio.Event()
def _handle_signal() -> None:
logger.info("Shutdown signal received")
_shutdown.set()
async def _update_db(peers: list[PeerInfo]) -> dict[str, dict]:
"""Update device rows in DB and return metadata for metrics labels.
Returns: {public_key: {"device_name": ..., "user_email": ...}}
"""
if not peers:
return {}
peer_map = {p.public_key: p for p in peers}
async with async_session() as session:
result = await session.execute(
select(Device, User.email).join(User).where(
Device.public_key.in_(list(peer_map.keys()))
)
)
rows = result.all()
labels = {}
updated = 0
for device, user_email in rows:
peer = peer_map.get(device.public_key)
if peer is None:
continue
# Only write connection status to PostgreSQL — traffic metrics go to VictoriaMetrics
device.latest_handshake = peer.latest_handshake
device.remote_ip = peer.endpoint.split(":")[0] if peer.endpoint else None
device.rx_bytes = peer.rx_bytes
device.tx_bytes = peer.tx_bytes
session.add(device)
updated += 1
labels[device.public_key] = {
"device_name": device.name,
"user_email": user_email,
}
if updated:
await session.commit()
logger.debug("Updated stats for {} devices", updated)
return labels
def _build_prometheus_payload(peers: list[PeerInfo], labels: dict[str, dict]) -> str:
"""Build Prometheus exposition format text for VictoriaMetrics import."""
now_ms = int(time.time() * 1000)
lines = []
active_count = 0
for peer in peers:
meta = labels.get(peer.public_key)
if not meta:
continue
tag = (
f'public_key="{peer.public_key[:16]}",'
f'device_name="{meta["device_name"]}",'
f'user_email="{meta["user_email"]}"'
)
lines.append(f"wiregui_peer_rx_bytes{{{tag}}} {peer.rx_bytes} {now_ms}")
lines.append(f"wiregui_peer_tx_bytes{{{tag}}} {peer.tx_bytes} {now_ms}")
handshake_ts = int(peer.latest_handshake.timestamp()) if peer.latest_handshake else 0
lines.append(f"wiregui_peer_latest_handshake_seconds{{{tag}}} {handshake_ts} {now_ms}")
connected = 1 if (handshake_ts and (time.time() - handshake_ts) < 180) else 0
lines.append(f"wiregui_peer_connected{{{tag}}} {connected} {now_ms}")
if connected:
active_count += 1
lines.append(f"wiregui_peers_total {active_count} {now_ms}")
return "\n".join(lines) + "\n"
async def _push_metrics(client: httpx.AsyncClient, url: str, payload: str) -> None:
"""Push Prometheus-format metrics to VictoriaMetrics."""
try:
resp = await client.post(
f"{url}/api/v1/import/prometheus",
content=payload,
headers={"Content-Type": "text/plain"},
)
if resp.status_code >= 400:
logger.warning("VictoriaMetrics push failed (HTTP {}): {}", resp.status_code, resp.text[:200])
except httpx.HTTPError as e:
logger.warning("VictoriaMetrics push error: {}", e)
async def run() -> None:
"""Main collector loop."""
settings = get_settings()
interval = settings.metrics_poll_interval
vm_url = settings.victoriametrics_url
logger.info(
"Collector started: interval={}s, victoriametrics={}",
interval, vm_url or "disabled",
)
client = httpx.AsyncClient(timeout=5) if vm_url else None
try:
while not _shutdown.is_set():
try:
peers = await get_peers()
labels = await _update_db(peers)
if client and vm_url and peers:
payload = _build_prometheus_payload(peers, labels)
await _push_metrics(client, vm_url, payload)
except asyncio.CancelledError:
break
except Exception as e:
logger.error("Collector poll failed: {}", e)
try:
await asyncio.wait_for(_shutdown.wait(), timeout=interval)
break # shutdown signalled
except asyncio.TimeoutError:
pass # normal — interval elapsed, loop again
finally:
if client:
await client.aclose()
await engine.dispose()
logger.info("Collector stopped")
def main() -> None:
setup_logging(log_to_file=get_settings().log_to_file)
loop = asyncio.new_event_loop()
for sig in (signal.SIGTERM, signal.SIGINT):
loop.add_signal_handler(sig, _handle_signal)
try:
loop.run_until_complete(run())
finally:
loop.close()
if __name__ == "__main__":
main()

5
wiregui/config.py
View file

@ -41,6 +41,11 @@ class Settings(BaseSettings):
smtp_password: str | None = None
smtp_from: str = "wiregui@localhost"
# Metrics collector
metrics_enabled: bool = False # run separate collector process for high-frequency stats
metrics_poll_interval: int = 5 # seconds between wg show polls (collector process)
victoriametrics_url: str | None = None # e.g. http://localhost:8428
# IdP provisioning
idp_config_file: str | None = None # path to YAML file with IdP definitions

33
wiregui/main.py
View file

@ -62,13 +62,17 @@ async def startup() -> None:
from wiregui.services.firewall import setup_base_tables, setup_masquerade
from wiregui.services.wireguard import configure_interface, ensure_interface
from wiregui.tasks.reconcile import reconcile
from wiregui.tasks.stats import stats_loop
await ensure_interface()
await configure_interface()
await setup_base_tables()
await setup_masquerade()
await reconcile()
if settings.metrics_enabled:
_start_collector()
else:
from wiregui.tasks.stats import stats_loop
register_task(stats_loop(), name="wg-stats")
register_task(vpn_session_loop(), name="vpn-session-expiry")
else:
@ -77,10 +81,37 @@ async def startup() -> None:
logger.info("WireGUI ready")
_collector_proc = None
def _start_collector() -> None:
"""Spawn the metrics collector as a subprocess sharing our network namespace."""
import subprocess
import sys
global _collector_proc
_collector_proc = subprocess.Popen(
[sys.executable, "-m", "wiregui.collector"],
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT,
)
logger.info("Metrics collector started (pid={})", _collector_proc.pid)
async def shutdown() -> None:
from wiregui.tasks import cancel_all
await cancel_all()
global _collector_proc
if _collector_proc and _collector_proc.poll() is None:
logger.info("Stopping metrics collector (pid={})", _collector_proc.pid)
_collector_proc.terminate()
try:
_collector_proc.wait(timeout=5)
except Exception:
_collector_proc.kill()
_collector_proc = None
app.on_startup(startup)
app.on_shutdown(shutdown)