diff --git a/.planning/REQUIREMENTS.md b/.planning/REQUIREMENTS.md new file mode 100644 index 0000000..31ffeb5 --- /dev/null +++ b/.planning/REQUIREMENTS.md @@ -0,0 +1,165 @@ +# Requirements: TOD v9.8 — SNMP Device Integration + +**Defined:** 2026-03-21 +**Core Value:** Operators can monitor their entire network fleet — MikroTik and SNMP devices side by side — from a single pane of glass +**Design Spec:** `docs/superpowers/specs/2026-03-21-v98-snmp-integration-design.md` + +## v9.8 Requirements + +### Foundation + +- [x] **FOUND-01**: Device model supports a device_type discriminator (routeros vs snmp) with backward-compatible defaults +- [x] **FOUND-02**: Database schema includes snmp_profiles table with system-shipped profiles and tenant-custom profiles +- [x] **FOUND-03**: Database schema includes unified credential_profiles table supporting routeros, snmp_v2c, and snmp_v3 credential types +- [x] **FOUND-04**: Database schema includes snmp_metrics hypertable for custom OID time-series data with 90-day retention +- [x] **FOUND-05**: Poller Collector interface abstracts device-type-specific collection (RouterOS and SNMP implementations) +- [x] **FOUND-06**: Existing PollDevice logic refactored into RouterOSCollector without behavior changes + +### Credential Management + +- [x] **CRED-01**: Operator can create a credential profile (RouterOS or SNMP) with encrypted storage via OpenBao Transit +- [x] **CRED-02**: Operator can assign a credential profile to one or many devices instead of per-device credentials +- [x] **CRED-03**: Updating a credential profile propagates new credentials to all linked devices on next poll cycle +- [x] **CRED-04**: Poller resolves credentials via fallback: per-device credentials first, then credential profile +- [x] **CRED-05**: CredentialCache refactored to GetRawCredentials with type-specific parsers (RouterOS, SNMPv2c, SNMPv3) + +### SNMP Polling + +- [x] **POLL-01**: Poller can poll SNMP devices using gosnmp with SNMPv1, v2c, and v3 support +- [x] **POLL-02**: SNMP collection is profile-driven — device profile defines which OIDs to collect per poll group +- [x] **POLL-03**: Standard SNMP metrics (ifXTable, hrStorageTable, hrProcessorLoad) map to existing hypertables (interface_metrics, health_metrics) +- [x] **POLL-04**: Custom OID data publishes as SNMPMetricsEvent and inserts into snmp_metrics hypertable +- [x] **POLL-05**: Counter32/Counter64 delta computation with Redis cache, including wraparound detection and sanity threshold +- [x] **POLL-06**: Profile cache refreshes from database periodically without per-device DB queries +- [x] **POLL-07**: SNMP devices use same scheduler, circuit breaker, Redis locks, and NATS pipeline as RouterOS devices + +### Device Profiles + +- [x] **PROF-01**: TOD ships 6 system default profiles (generic-snmp, network-switch, network-router, wireless-ap, ups-device, mikrotik-snmp) +- [x] **PROF-02**: Auto-detection probes sysObjectID via NATS request-reply and suggests matching profile +- [ ] **PROF-03**: Operator can create custom SNMP profiles with arbitrary OID collections grouped by poll group +- [ ] **PROF-04**: Operator can upload vendor MIB files and browse parsed OID tree to select collection targets +- [ ] **PROF-05**: Operator can test a profile against a live device before saving + +### Device Management + +- [x] **MGMT-01**: Operator can add a single SNMP device with IP, SNMP version, credential (profile or manual), and device profile +- [x] **MGMT-02**: Operator can bulk-add RouterOS devices using a credential profile + IP list (one per line, CIDR, or range) +- [x] **MGMT-03**: Operator can bulk-add SNMP devices using a credential profile + IP list with auto-detected profiles +- [ ] **MGMT-04**: Subnet scan discovers both RouterOS and SNMP devices with protocol-specific credential profiles +- [x] **MGMT-05**: Bulk add returns per-device results (success/failure with reason) and supports partial success + +### Fleet UI + +- [ ] **UI-01**: Fleet table shows SNMP devices alongside MikroTik devices with type icon, status, CPU, memory, uptime +- [ ] **UI-02**: Fleet table supports filtering by device type (All / RouterOS / SNMP) +- [ ] **UI-03**: Device detail page conditionally renders sections based on device_type (no RouterOS-only sections for SNMP devices) +- [ ] **UI-04**: SNMP device detail shows system info, interface metrics, health metrics, and custom OID charts +- [x] **UI-05**: Add Device dialog has tabs for RouterOS, SNMP, and VPN with credential profile selectors +- [x] **UI-06**: Credential profile management page lists, creates, edits, deletes profiles for both types +- [ ] **UI-07**: SNMP profile editor with OID tree browser, MIB upload, poll group configuration + +### Metrics & Data + +- [x] **DATA-01**: SNMP interface metrics (rx_bytes, tx_bytes, rx_bps, tx_bps) stored in existing interface_metrics hypertable +- [x] **DATA-02**: SNMP health metrics (CPU, memory, disk) stored in existing health_metrics hypertable +- [x] **DATA-03**: Custom SNMP metrics stored in snmp_metrics hypertable with metric_name, metric_group, oid, and value +- [x] **DATA-04**: SNMP metrics API returns time-bucketed data in same format as existing metrics endpoints +- [ ] **DATA-05**: Frontend charts for interface traffic and health work identically for SNMP and RouterOS devices + +### Backward Compatibility + +- [x] **COMPAT-01**: All existing RouterOS device functionality works unchanged after v9.8 migration +- [x] **COMPAT-02**: Existing API responses maintain shape (new fields are additive only) +- [x] **COMPAT-03**: Existing NATS event types and subjects are unchanged +- [x] **COMPAT-04**: 500+ mixed MikroTik/SNMP devices can be polled without performance degradation + +## Future Requirements (v9.9+) + +### SNMP Traps + +- **TRAP-01**: TOD receives and processes SNMP traps/informs +- **TRAP-02**: Trap events surface in the UI alongside polled metrics +- **TRAP-03**: Trap deduplication and rate limiting per device + +### Extended Monitoring + +- **EXT-01**: SNMP SET operations for device configuration +- **EXT-02**: sFlow/NetFlow/IPFIX collection +- **EXT-03**: Multi-protocol devices (RouterOS API + SNMP on same device) + +### Extensibility + +- **EXTENS-01**: External check executor (run scripts, parse output) +- **EXTENS-02**: Nagios plugin output format support +- **EXTENS-03**: Telegraf input plugin execution + +## Out of Scope + +| Feature | Reason | +|---------|--------| +| SNMP SET operations | Read-only monitoring for v9.8; write operations are vendor-specific | +| SNMP trap/inform reception | Requires listening server, firewall changes — deferred to v9.9 | +| SNMP device config backup | No standard SNMP mechanism exists | +| Nagios plugin execution | Trivially simple but not core to SNMP milestone — v10 extensibility | +| sFlow/NetFlow/IPFIX | Separate monitoring domain, v10+ | +| Multi-protocol devices | One device_type per device for v9.8 simplicity | +| SNMP device firmware management | No standard mechanism, vendor-specific | + +## Traceability + +| Requirement | Phase | Status | +|-------------|-------|--------| +| FOUND-01 | Phase 16 | Complete | +| FOUND-02 | Phase 16 | Complete | +| FOUND-03 | Phase 16 | Complete | +| FOUND-04 | Phase 16 | Complete | +| FOUND-05 | Phase 16 | Complete | +| FOUND-06 | Phase 16 | Complete | +| CRED-01 | Phase 17 | Complete | +| CRED-02 | Phase 17 | Complete | +| CRED-03 | Phase 17 | Complete | +| CRED-04 | Phase 16 | Complete | +| CRED-05 | Phase 16 | Complete | +| POLL-01 | Phase 18 | Complete | +| POLL-02 | Phase 18 | Complete | +| POLL-03 | Phase 18 | Complete | +| POLL-04 | Phase 18 | Complete | +| POLL-05 | Phase 18 | Complete | +| POLL-06 | Phase 18 | Complete | +| POLL-07 | Phase 18 | Complete | +| PROF-01 | Phase 18 | Complete | +| PROF-02 | Phase 18 | Complete | +| PROF-03 | Phase 20 | Pending | +| PROF-04 | Phase 20 | Pending | +| PROF-05 | Phase 20 | Pending | +| MGMT-01 | Phase 19 | Complete | +| MGMT-02 | Phase 19 | Complete | +| MGMT-03 | Phase 19 | Complete | +| MGMT-04 | Phase 19 | Pending | +| MGMT-05 | Phase 19 | Complete | +| UI-01 | Phase 19 | Pending | +| UI-02 | Phase 19 | Pending | +| UI-03 | Phase 19 | Pending | +| UI-04 | Phase 19 | Pending | +| UI-05 | Phase 19 | Complete | +| UI-06 | Phase 19 | Complete | +| UI-07 | Phase 20 | Pending | +| DATA-01 | Phase 18 | Complete | +| DATA-02 | Phase 18 | Complete | +| DATA-03 | Phase 18 | Complete | +| DATA-04 | Phase 17 | Complete | +| DATA-05 | Phase 19 | Pending | +| COMPAT-01 | Phase 16 | Complete | +| COMPAT-02 | Phase 16 | Complete | +| COMPAT-03 | Phase 16 | Complete | +| COMPAT-04 | Phase 18 | Complete | + +**Coverage:** +- v9.8 requirements: 44 total +- Mapped to phases: 44 +- Unmapped: 0 + +--- +*Requirements defined: 2026-03-21* +*Last updated: 2026-03-21 after roadmap creation* diff --git a/.planning/ROADMAP.md b/.planning/ROADMAP.md new file mode 100644 index 0000000..d279444 --- /dev/null +++ b/.planning/ROADMAP.md @@ -0,0 +1,168 @@ +# Roadmap: TOD v9.8 — SNMP Device Integration + +## Overview + +v9.8 extends TOD from a MikroTik-only fleet manager into a multi-vendor NMS by adding SNMP device monitoring alongside the existing RouterOS API path. The build follows a strict dependency chain: schema foundation and credential refactor first (must be backward-compatible with existing RouterOS flow), then backend API and NATS subscriber extension (must deploy before poller publishes SNMP events), then the SNMP collector in the Go poller, then frontend integration with bulk add, and finally the advanced custom profile builder with MIB upload. Each phase delivers a coherent capability that unblocks the next. + +## Milestones + +- v9.6 Config Backup - Phases 1-10 (shipped) +- v9.7 Tower & Site Management - Phases 11-15 (shipped 2026-03-19) +- v9.8 SNMP Device Integration - Phases 16-20 (in progress) + +## Phases + +
+v9.6 Config Backup & Change Tracking (Phases 1-10) + +- [x] **Phase 1: Database Schema** - Config snapshot, diff, and change tables with encryption and RLS +- [x] **Phase 2: Poller Config Collection** - SSH export, normalization, and NATS publishing from Go poller +- [x] **Phase 3: Snapshot Ingestion** - Backend NATS subscriber stores snapshots with SHA256 deduplication +- [x] **Phase 4: Manual Backup Trigger** - API endpoint for on-demand config backup via poller +- [x] **Phase 5: Diff Engine** - Unified diff generation and structured change parsing +- [x] **Phase 6: History API** - REST endpoints for timeline, snapshot view, and diff retrieval with RBAC +- [x] **Phase 7: Config History UI** - Timeline section on device page with change summaries +- [x] **Phase 8: Diff Viewer & Download** - Unified diff display with syntax highlighting and .rsc download +- [x] **Phase 9: Retention & Cleanup** - 90-day retention policy with automatic snapshot deletion +- [x] **Phase 10: Audit & Observability** - Audit event logging for all config backup operations + +
+ +
+v9.7 Tower & Site Management (Phases 11-15) - SHIPPED 2026-03-19 + +- [x] **Phase 11: Site Data Model + Foundation** - Sites CRUD, device assignment, site list with health rollup +- [x] **Phase 12: Per-Client Wireless Collection** - Poller extension to collect registration table and per-interface RF stats +- [x] **Phase 13: Link Discovery + Registration Ingestion** - Backend NATS consumer, MAC resolution, AP-CPE link state machine +- [x] **Phase 14: Site Dashboard + Sector Views + Wireless UI** - Site detail page, sector-centric view, per-station wireless tables +- [x] **Phase 15: Signal Trending + Site Alerting** - Signal history charts, degradation detection, site/sector alert rules + +
+ +### v9.8 SNMP Device Integration (Phases 16-20) + +- [x] **Phase 16: Schema Foundation + Credential Refactor** - Database migrations, Collector interface, credential cache backward-compatible refactor (completed 2026-03-21) +- [x] **Phase 17: Backend API + Subscriber Extension** - Credential profile and SNMP profile CRUD APIs, snmp_custom subscriber handler, NAK safety net (completed 2026-03-22) +- [x] **Phase 18: SNMP Collector Core** - gosnmp polling, profile-driven OID collection, counter delta computation, auto-detection (completed 2026-03-22) +- [ ] **Phase 19: Fleet UI + Bulk Add** - SNMP devices in fleet table, device detail, add device dialog, bulk add, credential profile management +- [ ] **Phase 20: Custom Profile Builder + MIB Upload** - MIB file upload, OID tree browser, profile editor, test profile against live device + +## Phase Details + +### Phase 16: Schema Foundation + Credential Refactor +**Goal**: Database schema supports SNMP devices, credential profiles, and device profiles; poller Collector interface enables protocol dispatch; credential cache refactor is backward-compatible with all existing RouterOS polling +**Depends on**: Nothing (first phase of v9.8; existing v9.7 foundation) +**Requirements**: FOUND-01, FOUND-02, FOUND-03, FOUND-04, FOUND-05, FOUND-06, CRED-04, CRED-05, COMPAT-01, COMPAT-02, COMPAT-03 +**Success Criteria** (what must be TRUE): + 1. Database has credential_profiles, snmp_profiles, and snmp_metrics tables; devices table has device_type, snmp_profile_id, and credential_profile_id columns + 2. All existing RouterOS devices continue to poll, store metrics, and display in the UI identically to before the migration (zero regression) + 3. Poller dispatches to RouterOSCollector or SNMPCollector based on device_type, with RouterOSCollector wrapping existing PollDevice logic without behavior changes + 4. Credential cache resolves credentials via fallback chain (per-device first, then credential profile) and handles legacy credentials without a type field as routeros + 5. Six system-shipped SNMP profiles (generic-snmp, network-switch, network-router, wireless-ap, ups-device, mikrotik-snmp) exist in the snmp_profiles table +**Plans:** 4/4 plans complete + +Plans: +- [x] 16-01-PLAN.md -- Database migrations (credential_profiles, snmp_profiles with seeds, devices columns, snmp_metrics hypertable) +- [x] 16-02-PLAN.md -- Go store.Device struct + FetchDevices query update with credential profile JOIN +- [x] 16-03-PLAN.md -- Credential cache refactor (GetRawCredentials, type parsers, backward-compat wrapper) +- [x] 16-04-PLAN.md -- Collector interface, RouterOSCollector wrapper, Scheduler dispatch by device_type + +### Phase 17: Backend API + Subscriber Extension +**Goal**: Python backend exposes credential profile and SNMP profile CRUD APIs with encrypted storage; NATS subscriber handles snmp_custom events; backend is fully deployed and ready before poller ships SNMP code +**Depends on**: Phase 16 (schema tables must exist; credential_profiles and snmp_profiles tables are FK dependencies) +**Requirements**: CRED-01, CRED-02, CRED-03, DATA-04 +**Success Criteria** (what must be TRUE): + 1. Operator can create, list, edit, and delete credential profiles (RouterOS and SNMP types) with credentials encrypted via OpenBao Transit + 2. Operator can assign a credential profile to devices, and updating the profile propagates new credentials to all linked devices on next poll cycle + 3. Deleting a credential profile that has linked devices returns HTTP 409 with a count of affected devices (no silent orphaning) + 4. NATS metrics_subscriber processes snmp_custom events and inserts rows into snmp_metrics hypertable; unknown event types are NAKed instead of ACKed + 5. SNMP metrics API returns time-bucketed data in the same format as existing metrics endpoints +**Plans:** 3/3 plans complete + +Plans: +- [x] 17-01-PLAN.md -- Credential profile CRUD API with OpenBao Transit encryption, deletion protection, device assignment +- [x] 17-02-PLAN.md -- SNMP profile CRUD API, metrics_subscriber snmp_custom handler, NAK safety net, SNMP metrics query endpoint +- [x] 17-03-PLAN.md -- Bulk device add API with credential profile support for RouterOS and SNMP devices + +### Phase 18: SNMP Collector Core +**Goal**: Poller polls SNMP devices end-to-end -- standard metrics flow into existing hypertables, custom metrics flow into snmp_metrics, and auto-detection identifies device profiles via sysObjectID +**Depends on**: Phase 17 (backend subscriber must be deployed and processing snmp_custom events before poller publishes them; credential profile and SNMP profile APIs must be live) +**Requirements**: POLL-01, POLL-02, POLL-03, POLL-04, POLL-05, POLL-06, POLL-07, DATA-01, DATA-02, DATA-03, PROF-01, PROF-02, COMPAT-04 +**Success Criteria** (what must be TRUE): + 1. Poller polls SNMP devices using SNMPv1, v2c, and v3 with correct authentication and encryption + 2. Standard SNMP interface metrics (rx_bytes, tx_bytes, rx_bps, tx_bps from ifXTable) appear in the existing interface_metrics hypertable alongside RouterOS interface data + 3. Standard SNMP health metrics (CPU, memory, disk from HOST-RESOURCES-MIB) appear in the existing health_metrics hypertable alongside RouterOS health data + 4. Custom OID data from device profiles publishes as snmp_custom events and lands in the snmp_metrics hypertable with correct metric_name, metric_group, oid, and value + 5. Auto-detection probes a device's sysObjectID via NATS request-reply and suggests a matching system profile (or generic-snmp fallback) +**Plans:** 5/5 plans complete + +Plans: +- [ ] 18-01-PLAN.md -- gosnmp dependency, SNMP client builder, counter cache, SNMPMetricsEvent struct +- [ ] 18-02-PLAN.md -- ProfileCache with DB loading, JSONB compilation, sysObjectID prefix matching +- [ ] 18-03-PLAN.md -- SNMPCollector.Collect with profile-driven OID collection, mappers, event publishing +- [ ] 18-04-PLAN.md -- DiscoveryResponder for SNMP auto-detection via NATS request-reply +- [ ] 18-05-PLAN.md -- Scheduler registration, main.go wiring, SoftwareVersion field + +### Phase 19: Fleet UI + Bulk Add +**Goal**: SNMP devices appear alongside MikroTik devices as first-class citizens in the UI; operators can add devices individually or in bulk using credential profiles +**Depends on**: Phase 18 (SNMP data must be flowing into the database for the UI to display; Phase 17 APIs for credential profiles and SNMP profiles must be live) +**Requirements**: MGMT-01, MGMT-02, MGMT-03, MGMT-04, MGMT-05, UI-01, UI-02, UI-03, UI-04, UI-05, UI-06, DATA-05 +**Success Criteria** (what must be TRUE): + 1. Fleet table shows SNMP devices alongside MikroTik devices with a type icon, and operators can filter by device type (All / RouterOS / SNMP) + 2. SNMP device detail page shows system info, interface traffic charts, health metrics charts, and custom OID charts -- with no RouterOS-only sections visible + 3. Add Device dialog has tabs for RouterOS, SNMP, and VPN with credential profile selectors filtered by device type + 4. Operator can bulk-add RouterOS or SNMP devices using a credential profile + IP list (one per line, CIDR, or range) and receives per-device results with success/failure reasons + 5. Credential profile management page lists, creates, edits, and deletes profiles for both RouterOS and SNMP types +**Plans:** 3/4 plans executed + +Plans: +- [ ] 19-01-PLAN.md -- API client SNMP types + fleet table type icon + device type filter +- [ ] 19-02-PLAN.md -- Add Device dialog redesign (RouterOS/SNMP/VPN tabs, credential profiles, bulk add) +- [ ] 19-03-PLAN.md -- Credential profile management page (CRUD, Settings route) +- [ ] 19-04-PLAN.md -- Device detail conditional rendering + SNMP metrics section + +### Phase 20: Custom Profile Builder + MIB Upload +**Goal**: Power users can upload vendor MIB files, browse OID trees, build custom SNMP profiles, and test profiles against live devices before saving +**Depends on**: Phase 19 (standard SNMP monitoring path must be solid and the SNMP profile editor page must exist in navigation; Phase 18 SNMP collector must support custom profiles) +**Requirements**: PROF-03, PROF-04, PROF-05, UI-07 +**Success Criteria** (what must be TRUE): + 1. Operator can upload vendor MIB files and the system parses them into a browsable OID tree with descriptions, types, and access modes + 2. OID tree browser lets operators expand/collapse MIB nodes and select OIDs to add to a custom profile's collection targets + 3. Operator can create custom SNMP profiles with arbitrary OID collections organized by poll group (e.g., fast 60s, standard 5m, slow 30m) + 4. Operator can test a custom profile against a live device and see actual OID values returned before committing the profile +**Plans**: TBD + +## Coverage + +| Category | Requirements | Phase | Count | +|----------|-------------|-------|-------| +| Foundation | FOUND-01, FOUND-02, FOUND-03, FOUND-04, FOUND-05, FOUND-06 | 16 | 6 | +| Credentials | CRED-01, CRED-02, CRED-03 | 17 | 3 | +| Credentials | CRED-04, CRED-05 | 16 | 2 | +| SNMP Polling | POLL-01, POLL-02, POLL-03, POLL-04, POLL-05, POLL-06, POLL-07 | 18 | 5/5 | Complete | 2026-03-22 | PROF-01, PROF-02 | 18 | 2 | +| Device Profiles | PROF-03, PROF-04, PROF-05 | 20 | 3 | +| Device Management | MGMT-01, MGMT-02, MGMT-03, MGMT-04, MGMT-05 | 19 | 3/4 | In Progress| | UI-01, UI-02, UI-03, UI-04, UI-05, UI-06 | 19 | 6 | +| Fleet UI | UI-07 | 20 | 1 | +| Metrics & Data | DATA-01, DATA-02, DATA-03 | 18 | 3 | +| Metrics & Data | DATA-04 | 17 | 1 | +| Metrics & Data | DATA-05 | 19 | 1 | +| Backward Compat | COMPAT-01, COMPAT-02, COMPAT-03 | 16 | 3 | +| Backward Compat | COMPAT-04 | 18 | 1 | +| **Total** | | | **44** | + +## Progress + +**Execution Order:** +Phases execute in numeric order: 16 -> 16.x -> 17 -> 17.x -> 18 -> 18.x -> 19 -> 19.x -> 20 + +| Phase | Plans Complete | Status | Completed | +|-------|----------------|--------|-----------| +| 16. Schema Foundation + Credential Refactor | 4/4 | Complete | 2026-03-21 | +| 17. Backend API + Subscriber Extension | 3/3 | Complete | 2026-03-22 | +| 18. SNMP Collector Core | 0/5 | Not started | - | +| 19. Fleet UI + Bulk Add | 0/4 | Not started | - | +| 20. Custom Profile Builder + MIB Upload | 0/? | Not started | - | + +--- +*Roadmap created: 2026-03-21* +*Last updated: 2026-03-22* diff --git a/.planning/STATE.md b/.planning/STATE.md new file mode 100644 index 0000000..2aa450b --- /dev/null +++ b/.planning/STATE.md @@ -0,0 +1,96 @@ +--- +gsd_state_version: 1.0 +milestone: v9.8 +milestone_name: SNMP Device Integration +status: unknown +stopped_at: Completed 19-02-PLAN.md (Add Device dialog + Bulk Add) +last_updated: "2026-03-22T01:01:06.013Z" +progress: + total_phases: 5 + completed_phases: 3 + total_plans: 16 + completed_plans: 15 +--- + +# Project State + +## Project Reference + +See: .planning/PROJECT.md (updated 2026-03-21) + +**Core value:** Operators can monitor their entire network fleet -- MikroTik and SNMP devices side by side -- from a single pane of glass +**Current focus:** Phase 19 — Fleet UI + Bulk Add + +## Current Position + +Phase: 19 (Fleet UI + Bulk Add) — EXECUTING +Plan: 4 of 4 + +## Performance Metrics + +**Velocity:** (from v9.7) + +- Total plans completed: 14 +- Average duration: 3.4 min +- Total execution time: ~0.8 hours + +## Accumulated Context + +### Decisions + +Decisions are logged in PROJECT.md Key Decisions table. + +- [v9.8] gosnmp in existing poller (not sidecar) -- unified device lifecycle +- [v9.8] Unified credential_profiles table for RouterOS + SNMP +- [v9.8] Standard SNMP metrics reuse existing hypertables (zero frontend changes for standard data) +- [v9.8] Counter delta computed in poller (not SQL LAG) for wraparound handling +- [v9.8] MIB parser as Go CLI binary (gosmi), not Python PySMI +- [Phase 16]: Raw SQL migrations via sa.text() for tables needing RLS + GRANT + partial unique indexes +- [Phase 16]: SNMP profile seed data as Python dicts with shared group constants to avoid OID duplication +- [Phase 16]: GetDevice also updated with new columns so interactive commands have device_type for conditional behavior +- [Phase 16]: Raw bytes cache separate from parsed creds, keyed with source prefix to prevent device/profile poisoning +- [Phase 16]: Legacy no-type-field JSON treated as RouterOS for backward compat with all existing credentials +- [Phase 16]: RouterOSCollector delegates to PollDevice (no body move) for minimal diff and preserved test surface +- [Phase 16]: RouterOSCollector registered inside NewScheduler (no main.go changes) +- [Phase 16]: Empty DeviceType defaults to "routeros" for backward compat with existing devices +- [Phase 17]: NAK unknown metric types instead of ACK -- prevents permanent data loss during deployment ordering mismatches +- [Phase 17]: Exclude profile_data JSONB from list response -- separate detail endpoint for full profile data +- [Phase 17]: New credential writes always use OpenBao Transit (never legacy AES-GCM) +- [Phase 17]: Credential fields are write-only -- accepted on create/update, encrypted, never returned in responses +- [Phase 17]: Separate /devices/bulk endpoint from legacy /devices/bulk-add for backward compatibility +- [Phase 17]: Credential profile type must match device type (routeros for routeros, snmp_v* for snmp) +- [Phase 17]: TCP reachability check only for RouterOS devices; SNMP (UDP) skips it +- [Phase 18]: MaxRepetitions=10 (not gosnmp default 50) for embedded device safety +- [Phase 18]: Counter sanity threshold at 90% of max value to distinguish reset from wrap +- [Phase 18]: Counter state in Redis with 600s TTL, MGET/MSET pipelining for efficiency +- [Phase 18]: sysOIDMap sorted by prefix length descending at load time for O(n) longest-prefix matching +- [Phase 18]: Invalid profile_data rows logged and skipped rather than failing entire cache load +- [Phase 18]: Inline gosnmp client construction in DiscoveryResponder to avoid snmp->bus->snmp import cycle +- [Phase 18]: Local withTimeout generic helper in snmp package (poller.withTimeout is unexported) +- [Phase 18]: walkTable safety valve at 10,000 PDUs to prevent memory exhaustion from misbehaving devices +- [Phase 18]: Poll groups collect independently -- partial SNMP collection failures do not abort the cycle +- [Phase 18]: RegisterCollector method on Scheduler for external collector registration (minimal invasiveness) +- [Phase 18]: ProfileCache.Load failure non-fatal at startup (profiles refresh on next 5-min cycle) +- [Phase 18]: DiscoveryResponder.Start failure non-fatal (discovery is convenience, not required for polling) +- [Phase 19]: Dot-notation route (settings.credentials.tsx) matching existing api-keys pattern +- [Phase 19]: credentialProfilesApi types added to api.ts as forward-compatible stub (plan 19-01 backend not yet executed) +- [Phase 19]: Always-visible three-tab layout (RouterOS, SNMP, VPN) instead of conditional two-tab +- [Phase 19]: SNMP tab requires credential profile (no manual SNMP credential entry) for operational security +- [Phase 19]: IP parsing v1 handles one-per-line only; CIDR and range expansion deferred with TODO + +### Pending Todos + +None yet. + +### Blockers/Concerns + +- Backend metrics_subscriber must handle snmp_custom events BEFORE poller starts publishing them (deployment ordering -- Phase 17 before Phase 18) +- Credential cache shape change (GetRawCredentials) must be backward-compatible or breaks all RouterOS polling +- Counter32 wraparound on high-speed interfaces produces silently wrong rate data if not handled from day one +- gosnmp BulkWalk can hang indefinitely on misbehaving devices without explicit timeout wrapping + +## Session Continuity + +Last session: 2026-03-22T01:00:58.297Z +Stopped at: Completed 19-02-PLAN.md (Add Device dialog + Bulk Add) +Resume file: None diff --git a/.planning/phases/19-fleet-ui-bulk-add/19-02-SUMMARY.md b/.planning/phases/19-fleet-ui-bulk-add/19-02-SUMMARY.md new file mode 100644 index 0000000..055e261 --- /dev/null +++ b/.planning/phases/19-fleet-ui-bulk-add/19-02-SUMMARY.md @@ -0,0 +1,128 @@ +--- +phase: 19-fleet-ui-bulk-add +plan: 02 +subsystem: ui +tags: [react, tabs, snmp, routeros, bulk-add, credential-profiles, tanstack-query] + +# Dependency graph +requires: + - phase: 19-fleet-ui-bulk-add/01 + provides: API types for credential profiles, SNMP profiles, bulk add with profile + - phase: 17-snmp-api + provides: Backend credential profile and bulk add endpoints +provides: + - Three-tab Add Device dialog (RouterOS, SNMP, VPN) with credential profile support + - Reusable BulkAddForm component for IP list bulk operations + - SNMP single-device add form with version selector and device profile +affects: [19-fleet-ui-bulk-add/03, 19-fleet-ui-bulk-add/04] + +# Tech tracking +tech-stack: + added: [] + patterns: + - "Multi-tab device dialog with conditional VPN tab" + - "Credential profile selector pattern for both RouterOS and SNMP" + - "BulkAddForm reusable component with deviceType prop" + - "IP list textarea parser with deduplication" + +key-files: + created: + - frontend/src/components/fleet/BulkAddForm.tsx + modified: + - frontend/src/components/fleet/AddDeviceForm.tsx + +key-decisions: + - "Always-visible tabs (RouterOS, SNMP, VPN) instead of conditional two-tab layout" + - "SNMP credential profile required (no manual SNMP credential entry) for security" + - "RouterOS tab retains manual credential fallback for backward compatibility" + - "IP parsing v1 handles one-per-line only; CIDR and range expansion deferred as TODO" + - "snmpProfilesApi.list returns array or object with profiles field -- handled both shapes" + +patterns-established: + - "BulkAddForm accepts deviceType prop and adapts its fields (SNMP port/profile vs API ports)" + - "Credential profile dropdowns filter by credential_type matching device type" + - "Status banner pattern shared across both single-add tabs" + +requirements-completed: [MGMT-01, MGMT-02, MGMT-03, MGMT-05, UI-05] + +# Metrics +duration: 5min +completed: 2026-03-22 +--- + +# Phase 19 Plan 02: Add Device Dialog + Bulk Add Summary + +**Three-tab Add Device dialog (RouterOS/SNMP/VPN) with credential profile selectors and reusable BulkAddForm for IP list bulk operations** + +## Performance + +- **Duration:** 5 min +- **Started:** 2026-03-22T00:54:28Z +- **Completed:** 2026-03-22T00:59:49Z +- **Tasks:** 2 +- **Files modified:** 2 + +## Accomplishments +- Redesigned Add Device dialog from conditional two-tab to always-visible three-tab layout (RouterOS, SNMP, VPN) +- RouterOS tab supports both credential profile mode and manual credential entry with "Add Multiple" toggle +- SNMP tab with version selector (v2c/v3), credential profile, device profile, and port configuration +- Created reusable BulkAddForm component for pasting IP lists with per-device result feedback + +## Task Commits + +Each task was committed atomically: + +1. **Task 1: Redesign AddDeviceForm with three tabs and credential profile selectors** - `74ddaad` (feat) +2. **Task 2: Create BulkAddForm component for IP list bulk operations** - `caf1435` (feat) + +## Files Created/Modified +- `frontend/src/components/fleet/AddDeviceForm.tsx` - Three-tab dialog with RouterOS, SNMP, VPN tabs and credential profile support +- `frontend/src/components/fleet/BulkAddForm.tsx` - Reusable bulk-add component with IP textarea, credential profile, and per-device results + +## Decisions Made +- Always-visible tabs instead of conditional layout -- simpler UX, consistent with three device types +- SNMP tab requires a credential profile (no manual SNMP credential entry) for operational security +- RouterOS tab retains manual credential fallback for backward compatibility with existing workflows +- IP parsing v1 supports one-per-line only; CIDR and range expansion deferred with TODO comments +- BulkAddForm handles both array and object shapes from snmpProfilesApi.list for resilience + +## Deviations from Plan + +### Auto-fixed Issues + +**1. [Rule 3 - Blocking] API types already existed from 19-01 commits** +- **Found during:** Task 1 (reading api.ts for types) +- **Issue:** Plan expected types might not exist from 19-01, but they were already committed +- **Fix:** Used existing types directly instead of adding placeholder comments +- **Files modified:** None (types already present) +- **Verification:** TypeScript compiles cleanly +- **Committed in:** N/A (no changes needed) + +--- + +**Total deviations:** 1 auto-acknowledged (1 blocking -- resolved by prior plan) +**Impact on plan:** No scope creep. Prior plan completion simplified this plan's execution. + +## Issues Encountered +None + +## User Setup Required +None - no external service configuration required. + +## Next Phase Readiness +- Add Device dialog fully functional with three tabs +- BulkAddForm ready to be used from both RouterOS and SNMP tabs +- Credential profile management UI (from 19-01/19-03) provides the profiles these forms consume +- Ready for 19-03 (credential profile management page) and 19-04 (device list filtering) + +## Self-Check: PASSED + +- [x] AddDeviceForm.tsx exists +- [x] BulkAddForm.tsx exists +- [x] Commit 74ddaad found +- [x] Commit caf1435 found +- [x] TypeScript compiles with no errors + +--- +*Phase: 19-fleet-ui-bulk-add* +*Completed: 2026-03-22*