# External Integrations

**Analysis Date:** 2026-03-12

## APIs & External Services

**MikroTik RouterOS:**
- Binary API (TLS port 8729) - Device polling and command execution
  - SDK/Client: go-routeros/v3 (Go poller)
  - Protocol: Binary encoded commands, TLS mutual authentication
  - Used in: `poller/cmd/poller/main.go`, `poller/internal/poller/`

**SMTP (Transactional Email):**
- System email service (password reset, alerts, notifications)
  - SDK/Client: aiosmtplib (async SMTP library)
  - Configuration: `SMTP_HOST`, `SMTP_PORT`, `SMTP_USER`, `SMTP_PASSWORD`, `SMTP_USE_TLS`
  - From address: `SMTP_FROM_ADDRESS`
  - Implementation: `app/services/email_service.py`
  - Supports TLS, STARTTLS, plain auth

**WebSocket/SSH Tunneling:**
- Browser-based SSH terminal for remote device access
  - SDK/Client: asyncssh (Python), xterm.js (frontend)
  - Protocol: SSH protocol with port forwarding
  - Implementation: `app/routers/remote_access.py`, `poller/internal/sshrelay/`
  - Features: Session auditing, command logging to NATS

## Data Storage

**Databases:**
- PostgreSQL 17 (TimescaleDB extension in production)
  - Async driver: asyncpg 0.30.0+ (Python backend)
  - Sync driver: pgx/v5 (Go poller)
  - ORM: SQLAlchemy 2.0+ async
  - Migrations: Alembic 1.14.0+
  - RLS: Row-Level Security policies for multi-tenant isolation
  - Models: `app/models/` (17+ model files)
  - Connection: `DATABASE_URL`, `APP_USER_DATABASE_URL`, `POLLER_DATABASE_URL`
  - Admin role: postgres (migrations only)
  - App role: app_user (enforces RLS)
  - Poller role: poller_user (direct access, no RLS)

**File Storage:**
- Local filesystem only - No cloud storage integration
  - Git store (bare repos): `/data/git-store` or `./git-store` (RWX PVC in production)
    - Implementation: `app/services/git_store.py`
    - Purpose: Version control for device configurations (one repo per tenant)
  - Firmware cache: `/data/firmware-cache`
    - Purpose: Downloaded RouterOS firmware images
    - Service: `app/services/firmware_service.py`
  - WireGuard config: `/data/wireguard`
    - Purpose: VPN peer and configuration management

**Caching:**
- Redis 7+
  - Async driver: redis 5.0.0+ (Python)
  - Sync driver: redis/go-redis/v9 (Go)
  - Use cases:
    - Session storage for SRP auth flows: `app/routers/auth.py` (key: `srp:session:{session_id}`)
    - Distributed locks: poller uses `bsm/redislock` to prevent duplicate polls across replicas
  - Connection: `REDIS_URL`

## Authentication & Identity

**Auth Provider:**
- Custom SRP-6a implementation (zero-knowledge auth)
  - Flow: SRP-6a password hash registration → no plaintext password stored
  - Implementation: `app/services/srp_service.py`, `app/routers/auth.py`
  - JWT tokens: HS256 signed with `JWT_SECRET_KEY`
  - Token storage: httpOnly cookies (frontend sends via credentials)
  - Refresh: 15-minute access tokens, 7-day refresh tokens
  - Fallback: Legacy bcrypt password support during upgrade phase

**User Roles:**
- Four role levels with RBAC:
  - super_admin - Cross-tenant access, user/billing management
  - admin - Full tenant management (invite users, config push, firmware)
  - operator - Limited: config push, monitoring, alerts
  - viewer - Read-only: dashboard, reports, audit logs

**Credential Encryption:**
- Per-tenant envelope encryption via OpenBao Transit
  - Service: `app/services/openbao_service.py`
  - Cipher: AES-256-GCM via OpenBao Transit engine
  - Key naming: `tenant_{uuid}` (created on tenant creation)
  - Fallback: Legacy Fernet decryption for credentials created before Transit migration

## Monitoring & Observability

**Error Tracking:**
- Not integrated - No Sentry, DataDog, or equivalent
- Local structured logging only

**Logs:**
- Structured logging via structlog (Python backend)
  - Format: JSON (production), human-readable (dev)
  - Configuration: `app/logging_config.py`
  - Log level: Configurable via `LOG_LEVEL` env var
- Structured logging via slog (Go poller)
  - Format: JSON with service name and instance hostname
  - Configuration: `poller/cmd/poller/main.go`

**Metrics:**
- Prometheus metrics export
  - Library: prometheus-fastapi-instrumentator 7.0.0+
  - Setup: `app/observability.py`
  - Endpoint: Exposed metrics in Prometheus text format
  - Not scraped by default - requires external Prometheus instance

**OpenTelemetry:**
- Minimal OTEL instrumentation in Go poller
  - SDK: `go.opentelemetry.io/otel` 1.39.0+
  - Not actively used in Python backend

## CI/CD & Deployment

**Hosting:**
- Self-hosted (Docker Compose for local, Kubernetes for production)
- No cloud provider dependency
- Reverse proxy: Caddy (reference: user memory notes)

**CI Pipeline:**
- GitHub Actions (`.github/workflows/`)
- Not fully analyzed - check workflows for details

**Containers:**
- Docker multi-stage builds for all three services
- Images: `api` (FastAPI), `poller` (Go binary), `frontend` (Vite SPA)
- Profiles: `full` (all services), `mail-testing` (adds Mailpit)

## Environment Configuration

**Required env vars:**
- `DATABASE_URL` - PostgreSQL admin connection
- `SYNC_DATABASE_URL` - Alembic migrations connection
- `APP_USER_DATABASE_URL` - App-scoped RLS connection
- `POLLER_DATABASE_URL` - Poller service connection
- `REDIS_URL` - Redis connection
- `NATS_URL` - NATS JetStream connection
- `JWT_SECRET_KEY` - HS256 signing key (MUST be unique in production)
- `CREDENTIAL_ENCRYPTION_KEY` - Base64-encoded 32-byte AES key
- `OPENBAO_ADDR` - OpenBao server address
- `OPENBAO_TOKEN` - OpenBao authentication token
- `CORS_ORIGINS` - Frontend origins (comma-separated)
- `SMTP_HOST`, `SMTP_PORT` - Email server
- `FIRST_ADMIN_EMAIL`, `FIRST_ADMIN_PASSWORD` - Bootstrap account (dev only)

**Secrets location:**
- `.env` file (git-ignored) - Development
- Environment variables in production (Kubernetes secrets, docker compose .env)
- OpenBao - Stores Transit encryption keys (not key material, only key references)

**Security defaults validation:**
- `app/config.py` rejects known-insecure values in non-dev environments:
  - `JWT_SECRET_KEY` hard-coded defaults
  - `CREDENTIAL_ENCRYPTION_KEY` hard-coded defaults
  - `OPENBAO_TOKEN` hard-coded defaults
- Fails startup with clear error message if production uses dev secrets

## Webhooks & Callbacks

**Incoming:**
- None detected - No external webhook subscriptions

**Outgoing:**
- Slack notifications - Alert firing/resolution (planned/partial implementation)
  - Router: `app/routers/alerts.py`
  - Implementation status: Check alert evaluation service
- Email notifications - Alert notifications, password reset
  - Service: `app/services/email_service.py`
- Custom webhooks - Extensible via notification service
  - Service: `app/services/notification_service.py`

## NATS JetStream Event Bus

**Message Bus:**
- NATS 2.0+ with JetStream persistence
  - Python client: nats-py 2.7.0+
  - Go client: nats.go 1.38.0+
  - Connection: `NATS_URL`

**Event Topics (Python publisher → Go/Python subscribers):**
- `device.status.>` - Device online/offline status from Go poller
  - Subscriber: `app/services/nats_subscriber.py`
  - Payload: device_id, tenant_id, status, routeros_version, board_name, uptime
  - Usage: Real-time device fleet updates

- `firmware.progress.{tenant_id}.{device_id}` - Firmware upgrade progress
  - Subscriber: `app/services/firmware_subscriber.py`
  - Publisher: Firmware upgrade service
  - Payload: stage (downloading, verifying, upgrading), progress %, message
  - Usage: Live firmware upgrade tracking (SSE to frontend)

- `config.push.{tenant_id}.{device_id}` - Configuration push progress
  - Subscriber: `app/services/push_rollback_subscriber.py`
  - Publisher: `app/services/restore_service.py`
  - Payload: phase (pre-validate, backup, push, commit), status, errors
  - Usage: Live config deployment tracking with rollback support

- `alert.fired.{tenant_id}`, `alert.resolved.{tenant_id}` - Alert events
  - Subscriber: `app/services/sse_manager.py`
  - Publisher: `app/services/alert_evaluator.py`
  - Payload: alert_id, device_id, rule_name, condition, value, timestamp
  - Usage: Real-time alert notifications (SSE to frontend)

- `audit.session.end` - SSH session audit events
  - Subscriber: `app/services/session_audit_subscriber.py`
  - Publisher: Go SSH relay (`poller/internal/sshrelay/`)
  - Payload: session_id, user_id, device_id, start_time, end_time, command_log
  - Usage: Session auditing and compliance logging

- `config.change.{tenant_id}.{device_id}` - Device config change detection
  - Subscriber: `app/services/config_change_subscriber.py`
  - Payload: device_id, change_type, affected_subsystems, timestamp
  - Usage: Track unapproved config changes

- `metrics.sample.{tenant_id}.{device_id}` - Real-time CPU/memory/traffic samples
  - Subscriber: `app/services/metrics_subscriber.py`
  - Publisher: Go poller
  - Payload: timestamp, cpu_percent, memory_percent, disk_percent, interfaces{name, rx_bytes, tx_bytes}
  - Usage: Live metric streaming (SSE to frontend)

**Server-Sent Events (SSE):**
- Frontend subscribes to per-tenant SSE streams
  - Endpoint: `GET /api/sse/subscribe?tenant_id={tenant_id}`
  - Connection: Long-lived HTTP persistent stream
  - Implementation: `app/routers/sse.py`, `app/services/sse_manager.py`
  - Payload format: SSE (text/event-stream)
  - Events forwarded from NATS to frontend browser in real-time
  - Used for: firmware progress, alerts, config push status, metrics

## Git Integration

**Version Control:**
- Bare git repositories stored per-tenant
  - Library: pygit2 1.14.0+
  - Location: `{GIT_STORE_PATH}/tenant_{tenant_id}/`
  - Purpose: Store device configuration history
  - Commits created on: successful config push, manual save
  - Restore: One-click revert to any previous commit
  - Implementation: `app/services/git_store.py`

---

*Integration audit: 2026-03-12*