homelab-monitor/docs/device-discovery.md

# LAN device discovery

Shows every device currently on the LAN, labeled known/unknown, in the dashboard's
"Network Devices" section.

## Why this isn't pure passive ARP reading

The original idea was to just read the ARP table (no active network traffic). In
practice that produces a nearly empty list: a host's ARP cache only contains entries
for peers it has actually exchanged traffic with, and the dashboard container has no
reason to talk to most LAN devices on its own. So discovery does a lightweight ICMP
ping sweep (no port scanning) first, to populate the cache, then reads it.

## Why this runs outside the Docker container

Real ARP entries live in the network namespace of whichever host is directly on the
LAN interface. The `api` container sits behind Docker's own bridge network — pinging
from inside it and reading `/proc/net/arp` there would show Docker's internal network,
not the actual LAN. Rather than switch the containers to `network_mode: host` (loses
container network isolation), discovery runs as a small script directly on the CT122
host via a systemd timer — the same pattern the homelab already uses for host-level
periodic jobs like the rclone backups on CT105.

## Pieces

- `scripts/discover-devices.sh` — ping sweep + `ip neigh show` parse, then an
  `avahi-resolve` mDNS reverse lookup per discovered IP (parallel, 2s timeout each so
  one non-mDNS device can't stall the run). Writes
  `/opt/homelab-monitor/data/devices-raw.json` (`[{ip, mac, state, mdnsHostname?}]`).
- `deploy/systemd/homelab-monitor-discover.{service,timer}` — runs the script every
  5 minutes on the CT122 host (not in Docker).
- `apps/api/src/discovery/index.ts` — reads that JSON file each poll cycle, labels
  entries against `knownDevices` in `config/hosts.yaml` (matched by IP), upserts into
  the `devices` SQLite table.
- `GET /api/devices` — serves devices seen in the last 24h (older entries are dropped
  rather than shown as stale-known, since there's no per-device up/down tracking).

## Identifying unknown devices

Three passive/config-driven layers, plus one on-demand active one:

1. **MAC OUI vendor lookup** (`mac-oui-lookup` npm package, `apps/api/src/routes/devices.ts`)
   — computed on every read from the MAC prefix, no storage needed. Already resolves
   most smart-home gear to a vendor (e.g. "Amazon Technologies Inc.", "Ring LLC",
   "Tuya Smart Inc." — a very common IoT chipset vendor).
2. **mDNS hostname** — see `discover-devices.sh` above, stored in `devices.mdns_hostname`.
3. **Manual labels** — `device_labels` table, keyed by **MAC** (survives DHCP IP
   changes, unlike `knownDevices` in `config/hosts.yaml` — see "Known limitation"
   below). `PUT`/`DELETE /api/devices/:mac/label`, inline-editable in the dashboard's
   Name column. Manual label > `knownDevices` config name > mDNS hostname (shown as an
   *italic hint*, not treated as "known" — nobody's actually confirmed it yet).
4. **On-demand deep check** — admin-triggered, single device, not automatic (avoids the
   noise/risk of doing this for the whole subnet on every poll). See below.

### Deep check

`scripts/deep-check-device.sh` runs, on request, against one target IP: mDNS resolve,
a targeted SSDP/UPnP query (many smart-home devices announce a `friendlyName` /
`manufacturer` / `modelName` this way — confirmed working against Home Assistant),
reverse DNS, and a small curated TCP port scan (21,22,23,80,443,554,5000,8000,8008,
8009,8060,8080,8443,9100,32400,62078) with an HTTP title/server grab on anything open.
Bounded timeouts throughout, finishes in well under 15s.

Same multicast-needs-real-network-access constraint as the ping sweep applies, so this
also runs on the CT122 **host**, not in the container — but unlike the scheduled
discovery script, this is triggered on demand from `POST /api/devices/:ip/deep-check`.
The API reaches it via SSH into CT122's own LAN IP (192.168.1.103), using the same
`monitor_ed25519` key as `SshHostCollector` but a **separate `authorized_keys` entry on
CT122 itself**. That entry's forced command is *parameterized*: forced commands ignore
whatever the client literally requests, but OpenSSH still exposes it via
`$SSH_ORIGINAL_COMMAND`, which the script reads and validates strictly (regex-anchored
`192.168.1.<1-254>`, never passed to a shell) before using it — confirmed a shell
injection attempt (`; rm -rf / #`) and an out-of-subnet IP both get rejected cleanly.

`deepCheck:` in `config/hosts.yaml` declares the target host (CT122 itself);
`apps/api/src/discovery/deepCheck.ts` does the SSH round-trip and JSON parsing.

Requires `avahi-utils` and `miniupnpc` installed on the CT122 host (`apt-get install
avahi-utils miniupnpc` — a one-time host package install, not part of any deploy
script, so re-provisioning CT122 from scratch would need to redo this step).

### Fingerbank enrichment (optional)

When `FINGERBANK_API_KEY` is set, the deep-check route (`apps/api/src/routes/devices.ts`)
also queries [Fingerbank](https://fingerbank.org)'s `/api/v2/combinations/interrogate`
with the device's MAC plus, if found, the SSDP `SERVER` header as an `upnp_user_agents`
signal (`apps/api/src/discovery/fingerbank.ts`). Unlike the other deep-check steps this
runs directly from the API container — no multicast/raw-socket access needed, just a
normal outbound HTTPS call — so no host-level or SSH changes were needed for this part.

**Honest limitation, confirmed by testing against a real device**: without a DHCP
fingerprint (which requires being the DHCP server — we're not, and have no way to
intercept that traffic from CT122), MAC-only or MAC+UPnP-signal queries often can't get
past manufacturer-level identification. A Nintendo device on this LAN queried as
`{"mac": "..."}` returned `device_name: "Hardware Manufacturer/Nintendo"` at
`score: 29` ("very little confidence" per Fingerbank's own bands) — no more specific
than the free OUI lookup already gives for free. The UI shows the confidence band
alongside the result (`confidenceLabel()` in `DeviceTable.tsx`) precisely so a
manufacturer-only guess at low confidence isn't mistaken for a confirmed ID. Still worth
having as opt-in enrichment — some devices *do* expose richer signals (a real DHCP
fingerprint, a distinctive UPnP string) that push the score meaningfully higher — but
don't expect it to reliably answer "what specific model is this" on its own.

Optional and gated: missing key, a Fingerbank API error, or no match all degrade
gracefully — the rest of deep-check's local findings (mDNS/SSDP/ports) are unaffected.

## Deploying/updating

```bash
scp scripts/discover-devices.sh scripts/deep-check-device.sh homelab-monitor:/opt/homelab-monitor/scripts/
scp deploy/systemd/homelab-monitor-discover.* homelab-monitor:/etc/systemd/system/
ssh homelab-monitor "chmod +x /opt/homelab-monitor/scripts/*.sh && \
  systemctl daemon-reload && systemctl enable --now homelab-monitor-discover.timer"
```

(In practice, `git pull` on CT122 already updates the script files at their deployed
path — the above is only needed for the systemd units or a from-scratch setup.)

## New-device alerting

Separate from known/unknown labeling: whenever a MAC address is seen on the network for
the **very first time ever**, the dashboard pushes a `persistent_notification` to Home
Assistant (config: `homeAssistant.url` in `config/hosts.yaml` + `HOME_ASSISTANT_TOKEN`
in `.env`, a long-lived access token from HA's Profile > Security). The idea: "unknown"
by itself isn't a useful alert signal (plenty of legitimate IoT gear stays permanently
unknown), but a MAC nobody's ever seen before showing up is worth a heads-up.

- `seen_macs` table (`apps/api/src/db/index.ts`) is a permanent, insert-only, MAC-keyed
  record — deliberately **not** derived from `devices.first_seen`, which is keyed by
  **IP** and would generate a false "new device" alert every time an existing device's
  DHCP lease just happened to change.
- **Bootstrap-safe**: the very first call (empty `seen_macs` table) seeds the baseline
  from whatever's currently on the network and reports nothing as new — otherwise
  turning this on for the first time would alert on the entire existing device
  population (71+ devices) all at once.
- `apps/api/src/discovery/homeAssistant.ts` posts to
  `{url}/api/services/persistent_notification/create` — chosen over a specific
  mobile-push `notify.*` service since it's guaranteed to work regardless of which
  notify integrations happen to be configured in the user's Home Assistant instance.
- Also surfaced in the dashboard itself: a blue "new" badge next to the status badge
  for any device first seen within the last 24h (`isNew` in the `/api/devices`
  response), independent of whether Home Assistant notifications are configured.
- Optional and gated: missing config, an HA API error, etc. don't affect discovery
  itself — only whether a push notification goes out.

## Known limitations

- `knownDevices` in `config/hosts.yaml` is matched by **IP**, not MAC — fine as long as
  DHCP reservations don't change, but a device losing its reservation would show up as
  "unknown" until the config is updated by hand. (Manual labels don't have this
  problem — they're keyed by MAC.)
- Deep check's SSDP/port-scan often finds nothing for cloud-connected devices (Ring,
  Echo) that deliberately minimize their LAN footprint — OUI vendor + mDNS are the
  primary identification layers for those; deep check helps most for devices that run
  a local web UI or SSDP responder (smart TVs, media devices, printers, Home Assistant).