homelab-monitor/CLAUDE.md

Homelab Monitor — Project Memory

Unified health dashboard for the homelab (infra docs: jhodgkin/homelab repo, docs/infrastructure.md). One login instead of logging into Proxmox, Zabbix, OMV, and every service separately.

How to resume work in a new session

1. List open issues: GET /api/v1/repos/jhodgkin/homelab-monitor/issues?state=open&limit=30 on git.jerodrigged.com (same pattern as the main homelab backlog, see global ~/.claude/CLAUDE.md). Milestone v1-dashboard is the current focus; everything else is backlog.
2. Pick an issue, comment that you're starting it, do the work, comment what you did, close it.
3. Commit and push after every meaningful step — don't let work sit uncommitted.

Architecture

• apps/api — Fastify + TypeScript. Polls collectors on an interval (POLL_INTERVAL_SECONDS, default 30s), writes snapshots to SQLite (apps/api/src/db), serves /api/*.
• apps/web — React + Vite + TypeScript. Polls /api/hosts every 15s, renders host/container cards.
• config/hosts.yaml — declares what to monitor (no secrets, committed). Credentials live in .env (gitignored) — see .env.example for the full list.
• Collectors implement the Collector interface (apps/api/src/collectors/types.ts) and are registered in apps/api/src/index.ts. Adding a new data source = new collector + config entry, no other wiring.
  • ProxmoxCollector (done): one API call to pve returns CPU/mem/disk and PSI pressure (pressurecpusome, pressurememoryfull) for the host + every LXC. Covers ~22 of ~24 machines.
  • SshHostCollector (done): covers .180 (omv) and .171 (ripper), the bare-metal boxes Proxmox can't see. Dedicated SSH key with a forced command= in the remote authorized_keys — the server always runs a fixed read-only script regardless of what's exec'd, so a leaked key is still contained. See docs/ssh-collector-key-setup.md, including the gotcha that diskPaths in hosts.yaml must be kept in sync by hand with the remote script's DISK_<label>= lines.
  • No Zabbix collector — see "Zabbix: intentionally skipped" below.
• Historical sparklines (done): /api/hosts embeds the last ~40 samples per host (one windowed query, ROW_NUMBER() OVER (PARTITION BY host_id ...)), rendered as small hand-rolled SVG sparklines in each card — no charting library. Fixed a real bug while building this: the Proxmox host node's own mem/disk had been null the whole time (/nodes/{node}/status nests memory/rootfs objects; code assumed the flat shape the LXC listing endpoint uses). LXC entries were never affected.
• Hot-reload (done): sshHosts/knownDevices in config/hosts.yaml are re-read on the next poll cycle (mtime check) — no container restart needed. Proxmox hosts already needed no config (auto- discovered every poll). See docs/hot-reload.md — includes a real single-file-bind-mount inode bug that silently broke this on first deploy, fixed by mounting ./config as a directory.
• Auth: local (bcrypt + signed session cookie) is always available — it's not an exclusive mode switch. OIDC via Authentik is an additional sign-in button, live when OIDC_ENABLED=true. See docs/oidc-setup.md. Deliberately additive so OIDC config problems can never lock out the admin login. Two real bugs found only once someone actually clicked the button (fixed, see docs/oidc-setup.md and e2e/README.md): OIDC_ISSUER_URL pointed at Authentik's LAN IP, which got baked into the browser-facing authorization_endpoint; and the callback hardcoded http:// when reconstructing the current URL (Fastify never sees HTTPS, it terminates upstream), sending the wrong scheme as redirect_uri in the token exchange. Both are exactly the class of bug curl-based verification cannot catch — real redirect chains and a real IdP login form need a real browser.
• Sign-out ends the full Authentik session, not just the app session (issue #19, done): RP-Initiated Logout via openid-client's buildEndSessionUrl (apps/api/src/auth/oidc.ts), full-page navigation (not fetch) so Authentik's browser cookie actually clears. Two more bugs found only via full click-through testing, same pattern as above: the provider's invalidation_flow was initially app-scoped (default-provider-invalidation-flow), which silently left Authentik's own session valid — fixed by switching to default-invalidation-flow; and the provider had no property_mappings, so the ID token never carried preferred_username/email and the UI showed the raw sub hash (with sub_mode: hashed_user_id) next to the sign-out button, looking like a leaked session token — fixed by attaching Authentik's default openid/profile/email scope mappings, no app code change needed. Full writeup in docs/oidc-setup.md. The Authentik blueprint is now version-controlled at deploy/authentik/homelab-monitor-oidc.yaml (source of truth — previously only lived on CT121 via ad-hoc scp/pct push); redeploy steps are in docs/oidc-setup.md.
• e2e/ (Playwright, done): drives a real browser against the live deployment, not a local dev server. local-login.spec.ts and oidc-login.spec.ts — the latter uses a dedicated Authentik test account (playwright-test, blueprint-provisioned, path users/service-accounts, never a real personal login) so the full OIDC flow can run unattended. Not wired into CI (hits production, needs secrets) — run by hand after auth-related changes. See e2e/README.md for the bugs it already caught.
• Device discovery (done, separate from the collector system above — it's inventory, not per-host metrics): scripts/discover-devices.sh runs via systemd timer on the CT122 host, not in Docker (real ARP entries live in the host's network namespace, not Docker's bridge network). API reads the resulting JSON each poll cycle. Sortable by every column in the UI (click a header, click again to reverse). Full reasoning in docs/device-discovery.md.
• Identifying unknown devices (done): five layers — MAC OUI vendor lookup (mac-oui-lookup, computed on read), mDNS hostname (added to the discovery sweep), manual labels (new device_labels table keyed by MAC, inline-editable in the dashboard, takes priority), an admin-triggered on-demand deep check per unknown device (mDNS + targeted SSDP/UPnP + curated port scan + HTTP title grab, runs on the CT122 host via a parameterized SSH forced-command reading $SSH_ORIGINAL_COMMAND, strictly validated), and optional Fingerbank enrichment folded into that same button (MAC + SSDP SERVER header, gated behind FINGERBANK_API_KEY, runs directly from the API container — no host-level access needed, unlike the other deep-check steps). Full writeup in docs/device-discovery.md. Already resolved ~59/71 devices including "Ring LLC" and multiple "Amazon Technologies Inc." entries. Fingerbank's honest ceiling without DHCP fingerprint data (which we don't have — not the DHCP server) is manufacturer-level confidence, same as free OUI — documented plainly, not oversold; UI shows the confidence band so a low-confidence guess isn't mistaken for a confirmed ID. Two real bugs hit while building this: mac-oui-lookup's named export crashed the whole process at startup under Node ESM/CJS interop (works fine under require(), fails under import { x } from, confirmed by running the built output with node, not just tsc); and SQLite's ALTER TABLE ADD COLUMN has no IF NOT EXISTS clause (conflated with CREATE TABLE's support for it) — check pragma table_info first instead.
• New-device alerting (done): seen_macs table (permanent, MAC-keyed, insert-only) tracks the first time each device was ever seen; bootstrap-safe so turning this on doesn't alert on the whole existing population. Pushes a Home Assistant persistent_notification when a genuinely new MAC appears (HOME_ASSISTANT_TOKEN + homeAssistant.url in hosts.yaml — token not yet provided by the user, so notifications aren't live yet even though detection and the dashboard badge are). Also shows a blue "new" badge in the dashboard for anything first seen in the last 24h, independent of HA config. Real bug caught via actual deployment (not just the unit test): the notification path was correctly bootstrap-safe, but the UI badge wasn't bootstrap-aware at all — 65/71 devices showed as new on first deploy. Fixed with a seen_macs.is_bootstrap column + a migration that backfills it correctly for already-deployed data. Verified with a full synthetic end-to-end test (injected a fake device into the discovery pipeline, confirmed only it got flagged, cleaned up after).

Infrastructure this project owns

• Gitea repo: jhodgkin/homelab-monitor
• Proxmox API token: monitor@pve!dashboard, role PVEAuditor (read-only), created via pveum on pve. Secret lives only in CT122's .env.
• Deployment: CT122 (homelab-monitor, 192.168.1.103), unprivileged LXC on pve, Docker + Compose. SSH alias homelab-monitor in ~/.ssh/config (root, key-based). ⚠️ CT122 ID conflict: the main homelab repo's jhodgkin/homelab#15 (Grafana/Loki/Prometheus migration) also planned to use CT122. That issue predates this dashboard's CT122 but wasn't checked before provisioning. Flagged in a comment on that issue — when #15 is picked up, it needs a different CT ID (123 or next free).
• SSH collector key: monitor_ed25519 keypair lives only at /opt/homelab-monitor/ssh/ on CT122 (gitignored, not in the repo). Public key installed on omv and ripper with a forced command — see docs/ssh-collector-key-setup.md.
• Authentik OIDC provider: set up via an Authentik blueprint on CT121 (not a manual UI step) — see docs/oidc-setup.md. Purely additive, didn't touch any existing Authentik user/credential. Issuer URL is the public https://auth.jerodrigged.com (a pre-existing NPM proxy host, predates this project) — never point this at the LAN IP, see the bug writeup above.
• Authentik test account: playwright-test (blueprint-provisioned, authentik_core.user, path users/service-accounts), used only by e2e/tests/oidc-login.spec.ts. Not a real person's login.
• Public exposure (done, issue #13 closed): https://monitor.jerodrigged.com is live. Chain is Cloudflare Tunnel (existing tunnel ac3a4440-..., added a Public Hostname pointing at NPM, not directly at the dashboard) → NPM proxy host id 14 (192.168.1.185:80 → 192.168.1.103:8090) → app. NPM cert is Let's Encrypt (id 18). ssl_forced must stay false on this proxy host — Cloudflare is in Flexible SSL mode (terminates TLS at the edge, talks plain HTTP to the tunnel origin), so ssl_forced: true causes an infinite redirect loop. Hit this exact bug once already; don't re-enable it. COOKIE_SECURE also stays false for the same underlying reason — the Fastify process never sees an HTTPS connection even when the browser is on HTTPS, since TLS terminates before it.
• Authentik redirect URIs: both the public (https://monitor.jerodrigged.com/..., now primary in .env) and LAN (http://192.168.1.103:8090/...) callback URLs are registered in the blueprint, so OIDC login still works if the tunnel/DNS is ever down.
• Vaultwarden: three items saved — "Homelab Monitor (dashboard admin)" (local admin login), "Homelab Monitor - Authentik OIDC client (homelab-monitor provider)" (client_id/secret, secure note), and "Homelab Monitor - Authentik test account (playwright-test)" (the e2e test account's login).
• Deep-check SSH key entry: monitor_ed25519's public key also has a second, separate authorized_keys entry on CT122's own root user (not just omv/ripper) — forced command deep-check-device.sh, parameterized via $SSH_ORIGINAL_COMMAND instead of the fixed-script pattern used elsewhere. deepCheck: in config/hosts.yaml declares the target (CT122's own LAN IP).
• Host packages (one-time, not in any deploy script — a from-scratch CT122 re-provision would need to redo this): avahi-utils and miniupnpc, installed via apt-get for mDNS/SSDP support.

Zabbix: intentionally skipped, not blocked

Investigated CT109 while working issue #11 and found Zabbix was never actually finished being set up (frontend not wired into Apache, setup wizard never run, no real hosts monitored, likely still on default Admin/zabbix credentials). Turned out this is expected: jhodgkin/homelab#15 already plans to decommission Zabbix entirely in favor of Prometheus+Loki+Grafana. User's call: skip Zabbix integration entirely rather than finish configuring a service that's being replaced. homelab-monitor issue #11 is closed as won't-do — if a metrics-alerting panel is wanted later, it should target Grafana/Prometheus once homelab#15 lands, as a new issue, not a reopening of #11.

Status: original backlog complete, OIDC verified end-to-end

All 14 issues filed at project start are closed (10 shipped, #11/Zabbix closed as won't-do, no others skipped). GET .../issues?state=open on this repo should return empty. First real user click-through of "Sign in with Authentik" surfaced two bugs (LAN-IP issuer, wrong-scheme redirect_uri in token exchange — both above), fixed, and now covered by e2e/oidc-login.spec.ts as regression coverage. Both local and OIDC login are Playwright-verified passing (5 consecutive runs) as of this writing.

e2e coverage expanded beyond the happy path (invalid credentials, session persistence, unauthenticated API rejection, device labeling, deep-check, Fingerbank, new-device badge) — 16 tests across 9 spec files, stable across repeated full-suite runs with parallel workers. All credentials this project generated are in Vaultwarden except the Home Assistant token (see below, not provided yet).

Unknown-device identification (issue #15, then #16 Fingerbank, then #17 new-device alerting) shipped after the OIDC work: OUI vendor lookup, mDNS, manual labels, on-demand deep check + Fingerbank, and alerting on genuinely-new MACs. See the "Identifying unknown devices" and "New-device alerting" architecture notes above.

If picking this up again with no specific ask from the user, there's no queued work — check in with them for what's next rather than inventing scope. Remaining known gaps, none blocking:

• HOME_ASSISTANT_TOKEN not yet set — new-device detection and the dashboard badge both work now, but the actual push notification to Home Assistant is silently skipped until this is provided (a long-lived access token from HA's Profile > Security).
• diskPaths in hosts.yaml vs. the remote forced-command scripts on omv/ripper is a manual-sync point (documented, not automated — only 2 hosts, hasn't been worth it).
• e2e/ isn't wired into CI — run by hand (cd e2e && npm test) after any auth-related change.
• The two avahi-utils/miniupnpc host packages on CT122 aren't in any deploy script — see "Infrastructure this project owns" above.
• User is evaluating buying a self-administrable router (UniFi/OPNsense) instead of the locked-down GFiber one — if that happens, knownDevices/deepCheck IPs in hosts.yaml may need updating, and a self-administered router might expose a real local API/DHCP data worth integrating (see conversation history around 2026-07-13 for the GFiber investigation — no usable local API found for it specifically).