jhodgkin
ce232c5a53
Cloudflare tunnel route -> NPM -> dashboard, Let's Encrypt cert via NPM's API, both public and LAN OIDC redirect URIs registered in Authentik. Hit and fixed a Flexible-SSL redirect loop (ssl_forced must stay false since Cloudflare terminates TLS at the edge and talks plain HTTP to the origin) -- documented clearly so it doesn't get "fixed" by accident later. Closes #13. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
87 lines
6.5 KiB
Markdown
87 lines
6.5 KiB
Markdown
# Homelab Monitor — Project Memory
|
|
|
|
Unified health dashboard for the homelab (infra docs: `jhodgkin/homelab` repo, `docs/infrastructure.md`).
|
|
One login instead of logging into Proxmox, Zabbix, OMV, and every service separately.
|
|
|
|
## How to resume work in a new session
|
|
|
|
1. List open issues: `GET /api/v1/repos/jhodgkin/homelab-monitor/issues?state=open&limit=30` on
|
|
`git.jerodrigged.com` (same pattern as the main homelab backlog, see global `~/.claude/CLAUDE.md`).
|
|
Milestone `v1-dashboard` is the current focus; everything else is backlog.
|
|
2. Pick an issue, comment that you're starting it, do the work, comment what you did, close it.
|
|
3. Commit and push after every meaningful step — don't let work sit uncommitted.
|
|
|
|
## Architecture
|
|
|
|
- `apps/api` — Fastify + TypeScript. Polls **collectors** on an interval (`POLL_INTERVAL_SECONDS`,
|
|
default 30s), writes snapshots to SQLite (`apps/api/src/db`), serves `/api/*`.
|
|
- `apps/web` — React + Vite + TypeScript. Polls `/api/hosts` every 15s, renders host/container cards.
|
|
- `config/hosts.yaml` — declares *what* to monitor (no secrets, committed). Credentials live in `.env`
|
|
(gitignored) — see `.env.example` for the full list.
|
|
- Collectors implement the `Collector` interface (`apps/api/src/collectors/types.ts`) and are registered
|
|
in `apps/api/src/index.ts`. Adding a new data source = new collector + config entry, no other wiring.
|
|
- `ProxmoxCollector` (done): one API call to `pve` returns CPU/mem/disk **and PSI pressure**
|
|
(`pressurecpusome`, `pressurememoryfull`) for the host + every LXC. Covers ~22 of ~24 machines.
|
|
- `SshHostCollector` (done): covers `.180` (omv) and `.171` (ripper), the bare-metal boxes Proxmox
|
|
can't see. Dedicated SSH key with a **forced `command=`** in the remote `authorized_keys` — the
|
|
server always runs a fixed read-only script regardless of what's exec'd, so a leaked key is still
|
|
contained. See `docs/ssh-collector-key-setup.md`, including the gotcha that `diskPaths` in
|
|
`hosts.yaml` must be kept in sync **by hand** with the remote script's `DISK_<label>=` lines.
|
|
- No Zabbix collector — see "Zabbix: intentionally skipped" below.
|
|
- Auth: local (bcrypt + signed session cookie) is **always** available — it's not an exclusive mode
|
|
switch. OIDC via Authentik is an *additional* sign-in button, live when `OIDC_ENABLED=true`. See
|
|
`docs/oidc-setup.md`. Deliberately additive so OIDC config problems can never lock out the admin login.
|
|
- **Device discovery** (done, separate from the collector system above — it's inventory, not per-host
|
|
metrics): `scripts/discover-devices.sh` runs via systemd timer **on the CT122 host**, not in Docker
|
|
(real ARP entries live in the host's network namespace, not Docker's bridge network). API reads the
|
|
resulting JSON each poll cycle. Full reasoning in `docs/device-discovery.md`.
|
|
|
|
## Infrastructure this project owns
|
|
|
|
- **Gitea repo**: `jhodgkin/homelab-monitor`
|
|
- **Proxmox API token**: `monitor@pve!dashboard`, role `PVEAuditor` (read-only), created via `pveum` on
|
|
`pve`. Secret lives only in CT122's `.env`.
|
|
- **Deployment**: CT122 (`homelab-monitor`, 192.168.1.103), unprivileged LXC on `pve`, Docker + Compose.
|
|
SSH alias `homelab-monitor` in `~/.ssh/config` (root, key-based).
|
|
⚠️ **CT122 ID conflict**: the main homelab repo's `jhodgkin/homelab#15` (Grafana/Loki/Prometheus
|
|
migration) also planned to use CT122. That issue predates this dashboard's CT122 but wasn't checked
|
|
before provisioning. Flagged in a comment on that issue — when #15 is picked up, it needs a different
|
|
CT ID (123 or next free).
|
|
- **SSH collector key**: `monitor_ed25519` keypair lives only at `/opt/homelab-monitor/ssh/` on CT122
|
|
(gitignored, not in the repo). Public key installed on `omv` and `ripper` with a forced command —
|
|
see `docs/ssh-collector-key-setup.md`.
|
|
- **Authentik OIDC provider**: set up via an Authentik blueprint on CT121 (not a manual UI step) — see
|
|
`docs/oidc-setup.md`. Purely additive, didn't touch any existing Authentik user/credential.
|
|
- **Public exposure (done, issue #13 closed)**: `https://monitor.jerodrigged.com` is live. Chain is
|
|
Cloudflare Tunnel (existing tunnel `ac3a4440-...`, added a Public Hostname pointing at NPM, not
|
|
directly at the dashboard) → NPM proxy host id 14 (`192.168.1.185:80` → `192.168.1.103:8090`) → app.
|
|
NPM cert is Let's Encrypt (id 18). **`ssl_forced` must stay `false`** on this proxy host — Cloudflare
|
|
is in Flexible SSL mode (terminates TLS at the edge, talks plain HTTP to the tunnel origin), so
|
|
`ssl_forced: true` causes an infinite redirect loop. Hit this exact bug once already; don't re-enable it.
|
|
`COOKIE_SECURE` also stays `false` for the same underlying reason — the Fastify process never sees an
|
|
HTTPS connection even when the *browser* is on HTTPS, since TLS terminates before it.
|
|
- **Authentik redirect URIs**: both the public (`https://monitor.jerodrigged.com/...`, now primary in
|
|
`.env`) and LAN (`http://192.168.1.103:8090/...`) callback URLs are registered in the blueprint, so
|
|
OIDC login still works if the tunnel/DNS is ever down.
|
|
- **Vaultwarden**: dashboard admin credentials saved as "Homelab Monitor (dashboard admin)". OIDC client
|
|
credentials not yet saved there (pending a `BW_SESSION` from the user).
|
|
- **Not yet verified**: nobody has actually clicked "Sign in with Authentik" through a real browser —
|
|
the redirect chain and params are confirmed correct via curl, but the final interactive consent/login
|
|
step needs a human.
|
|
|
|
## Zabbix: intentionally skipped, not blocked
|
|
|
|
Investigated CT109 while working issue #11 and found Zabbix was never actually finished being set up
|
|
(frontend not wired into Apache, setup wizard never run, no real hosts monitored, likely still on
|
|
default `Admin`/`zabbix` credentials). Turned out this is expected: `jhodgkin/homelab#15` already plans
|
|
to decommission Zabbix entirely in favor of Prometheus+Loki+Grafana. User's call: skip Zabbix
|
|
integration entirely rather than finish configuring a service that's being replaced. homelab-monitor
|
|
issue #11 is closed as won't-do — if a metrics-alerting panel is wanted later, it should target
|
|
Grafana/Prometheus once `homelab#15` lands, as a new issue, not a reopening of #11.
|
|
|
|
## Known gaps / explicitly deferred (see issues for detail)
|
|
|
|
- No historical charts yet — only the latest snapshot is shown (24h retention in SQLite unused by the UI so far).
|
|
- Public exposure (issue #13) is half-done: NPM proxy host exists, Cloudflare DNS/tunnel route does not.
|
|
Needs a Cloudflare API token (Zero Trust: Edit) from the user to finish.
|