homelab-monitor/docs/device-discovery.md

LAN device discovery

Shows every device currently on the LAN, labeled known/unknown, in the dashboard's "Network Devices" section.

Why this isn't pure passive ARP reading

The original idea was to just read the ARP table (no active network traffic). In practice that produces a nearly empty list: a host's ARP cache only contains entries for peers it has actually exchanged traffic with, and the dashboard container has no reason to talk to most LAN devices on its own. So discovery does a lightweight ICMP ping sweep (no port scanning) first, to populate the cache, then reads it.

Why this runs outside the Docker container

Real ARP entries live in the network namespace of whichever host is directly on the LAN interface. The api container sits behind Docker's own bridge network — pinging from inside it and reading /proc/net/arp there would show Docker's internal network, not the actual LAN. Rather than switch the containers to network_mode: host (loses container network isolation), discovery runs as a small script directly on the CT122 host via a systemd timer — the same pattern the homelab already uses for host-level periodic jobs like the rclone backups on CT105.

Pieces

• scripts/discover-devices.sh — ping sweep + ip neigh show parse, then an avahi-resolve mDNS reverse lookup per discovered IP (parallel, 2s timeout each so one non-mDNS device can't stall the run). Writes /opt/homelab-monitor/data/devices-raw.json ([{ip, mac, state, mdnsHostname?}]).
• deploy/systemd/homelab-monitor-discover.{service,timer} — runs the script every 5 minutes on the CT122 host (not in Docker).
• apps/api/src/discovery/index.ts — reads that JSON file each poll cycle, labels entries against knownDevices in config/hosts.yaml (matched by IP), upserts into the devices SQLite table.
• GET /api/devices — serves devices seen in the last 24h (older entries are dropped rather than shown as stale-known, since there's no per-device up/down tracking).

Identifying unknown devices

Three passive/config-driven layers, plus one on-demand active one:

1. MAC OUI vendor lookup (mac-oui-lookup npm package, apps/api/src/routes/devices.ts) — computed on every read from the MAC prefix, no storage needed. Already resolves most smart-home gear to a vendor (e.g. "Amazon Technologies Inc.", "Ring LLC", "Tuya Smart Inc." — a very common IoT chipset vendor).
2. mDNS hostname — see discover-devices.sh above, stored in devices.mdns_hostname.
3. Manual labels — device_labels table, keyed by MAC (survives DHCP IP changes, unlike knownDevices in config/hosts.yaml — see "Known limitation" below). PUT/DELETE /api/devices/:mac/label, inline-editable in the dashboard's Name column. Manual label > knownDevices config name > mDNS hostname (shown as an italic hint, not treated as "known" — nobody's actually confirmed it yet).
4. On-demand deep check — admin-triggered, single device, not automatic (avoids the noise/risk of doing this for the whole subnet on every poll). See below.

Deep check

scripts/deep-check-device.sh runs, on request, against one target IP: mDNS resolve, a targeted SSDP/UPnP query (many smart-home devices announce a friendlyName / manufacturer / modelName this way — confirmed working against Home Assistant), reverse DNS, and a small curated TCP port scan (21,22,23,80,443,554,5000,8000,8008, 8009,8060,8080,8443,9100,32400,62078) with an HTTP title/server grab on anything open. Bounded timeouts throughout, finishes in well under 15s.

Same multicast-needs-real-network-access constraint as the ping sweep applies, so this also runs on the CT122 host, not in the container — but unlike the scheduled discovery script, this is triggered on demand from POST /api/devices/:ip/deep-check. The API reaches it via SSH into CT122's own LAN IP (192.168.1.103), using the same monitor_ed25519 key as SshHostCollector but a separate authorized_keys entry on CT122 itself. That entry's forced command is parameterized: forced commands ignore whatever the client literally requests, but OpenSSH still exposes it via $SSH_ORIGINAL_COMMAND, which the script reads and validates strictly (regex-anchored 192.168.1.<1-254>, never passed to a shell) before using it — confirmed a shell injection attempt (; rm -rf / #) and an out-of-subnet IP both get rejected cleanly.

deepCheck: in config/hosts.yaml declares the target host (CT122 itself); apps/api/src/discovery/deepCheck.ts does the SSH round-trip and JSON parsing.

Requires avahi-utils and miniupnpc installed on the CT122 host (apt-get install avahi-utils miniupnpc — a one-time host package install, not part of any deploy script, so re-provisioning CT122 from scratch would need to redo this step).

Fingerbank enrichment (optional)

When FINGERBANK_API_KEY is set, the deep-check route (apps/api/src/routes/devices.ts) also queries Fingerbank's /api/v2/combinations/interrogate with the device's MAC plus, if found, the SSDP SERVER header as an upnp_user_agents signal (apps/api/src/discovery/fingerbank.ts). Unlike the other deep-check steps this runs directly from the API container — no multicast/raw-socket access needed, just a normal outbound HTTPS call — so no host-level or SSH changes were needed for this part.

Honest limitation, confirmed by testing against a real device: without a DHCP fingerprint (which requires being the DHCP server — we're not, and have no way to intercept that traffic from CT122), MAC-only or MAC+UPnP-signal queries often can't get past manufacturer-level identification. A Nintendo device on this LAN queried as {"mac": "..."} returned device_name: "Hardware Manufacturer/Nintendo" at score: 29 ("very little confidence" per Fingerbank's own bands) — no more specific than the free OUI lookup already gives for free. The UI shows the confidence band alongside the result (confidenceLabel() in DeviceTable.tsx) precisely so a manufacturer-only guess at low confidence isn't mistaken for a confirmed ID. Still worth having as opt-in enrichment — some devices do expose richer signals (a real DHCP fingerprint, a distinctive UPnP string) that push the score meaningfully higher — but don't expect it to reliably answer "what specific model is this" on its own.

Optional and gated: missing key, a Fingerbank API error, or no match all degrade gracefully — the rest of deep-check's local findings (mDNS/SSDP/ports) are unaffected.

Deploying/updating

scp scripts/discover-devices.sh scripts/deep-check-device.sh homelab-monitor:/opt/homelab-monitor/scripts/
scp deploy/systemd/homelab-monitor-discover.* homelab-monitor:/etc/systemd/system/
ssh homelab-monitor "chmod +x /opt/homelab-monitor/scripts/*.sh && \
  systemctl daemon-reload && systemctl enable --now homelab-monitor-discover.timer"

(In practice, git pull on CT122 already updates the script files at their deployed path — the above is only needed for the systemd units or a from-scratch setup.)

New-device alerting

Separate from known/unknown labeling: whenever a MAC address is seen on the network for the very first time ever, the dashboard pushes a persistent_notification to Home Assistant (config: homeAssistant.url in config/hosts.yaml + HOME_ASSISTANT_TOKEN in .env, a long-lived access token from HA's Profile > Security). The idea: "unknown" by itself isn't a useful alert signal (plenty of legitimate IoT gear stays permanently unknown), but a MAC nobody's ever seen before showing up is worth a heads-up.

• seen_macs table (apps/api/src/db/index.ts) is a permanent, insert-only, MAC-keyed record — deliberately not derived from devices.first_seen, which is keyed by IP and would generate a false "new device" alert every time an existing device's DHCP lease just happened to change.
• Bootstrap-safe: the very first call (empty seen_macs table) seeds the baseline from whatever's currently on the network and reports nothing as new — otherwise turning this on for the first time would alert on the entire existing device population (71+ devices) all at once.
• apps/api/src/discovery/homeAssistant.ts posts to {url}/api/services/persistent_notification/create — chosen over a specific mobile-push notify.* service since it's guaranteed to work regardless of which notify integrations happen to be configured in the user's Home Assistant instance.
• Also surfaced in the dashboard itself: a blue "new" badge next to the status badge for any device first seen within the last 24h (isNew in the /api/devices response), independent of whether Home Assistant notifications are configured.
• Optional and gated: missing config, an HA API error, etc. don't affect discovery itself — only whether a push notification goes out.

Known limitations

• knownDevices in config/hosts.yaml is matched by IP, not MAC — fine as long as DHCP reservations don't change, but a device losing its reservation would show up as "unknown" until the config is updated by hand. (Manual labels don't have this problem — they're keyed by MAC.)
• Deep check's SSDP/port-scan often finds nothing for cloud-connected devices (Ring, Echo) that deliberately minimize their LAN footprint — OUI vendor + mDNS are the primary identification layers for those; deep check helps most for devices that run a local web UI or SSDP responder (smart TVs, media devices, printers, Home Assistant).