I built a conversational assistant that reads and acts across a self-hosted app fleet (streaming chat with a multi-hop tool loop, every destructive action gated behind an explicit confirmation handshake, plus persistent memory, scheduled tasks, and daily briefings), then diagnosed and fixed a boot-ordering race that only it could hit.
The challenge
My self-hosted fleet had grown a set of standalone apps (contacts, notes, email, calendar), each with its own UI and data store, but no single conversational surface across them. The experience was "open the right app and click around." I wanted one assistant that could read across all of those systems and take actions on my behalf, safely, without a runaway agent doing something destructive unprompted.
Approach
The design principle: stream the reasoning, gate the writes, resume statelessly.
- Streaming chat with named events. The backend streams a turn as a sequence of named server-sent events (token, tool call, tool result, confirm, done) so the UI renders reasoning, tool activity, and confirmations live. Every message persists with its token counts and cost.
- A bounded multi-hop tool loop. The agent runs up to six tool-execution hops per turn, fanning out to the fleet's apps (contacts, notes, email, calendar) and to notification services over HTTP through a tools registry: enough to chain real work, bounded so it can't spin.
- Write-confirmation gating with a stateless resume contract. When the model calls a write tool, the backend does not execute it. It emits a
confirm event carrying the pending call IDs and halts. The turn resumes only when the client re-POSTs to the same endpoint with an approvals array and no new message; the server then executes or skips each call per its approved flag and signals done. Because the resume is stateless and endpoint-identical, any client, web PWA or native mobile, implements the handshake the same way. - Distilled and pinned memory, scheduled tasks with recurrence, and a morning-briefing trigger, all backed by the app's own schema with full-text search over message history.
Results
- Live in production, private-VPN-only by design, never public. The full agentic feature set shipped with zero TODO/FIXME markers in the source at audit: six-hop tool loop, write-confirmation gate, title auto-generation, fork/archive, full-text search, and an in-process scheduler for tasks and briefings.
- An 11-table schema with full-text search over messages; seven launch tools seeded at boot; dual-provider AI (a local model runtime plus a hosted LLM API) verified live at promotion, including cost tracking.
- A boot-ordering race, diagnosed and durably fixed. After a power outage, the service crash-looped in production: it failed to resolve the hostname of the service it fetches its configuration from, because it started before the private-VPN daemon and its DNS were up, and the systemd restart throttle then gave up before the VPN finished coming online. A pattern audit across every production app found this was the only one with a boot-time network dependency of that shape, which is exactly why it alone hit the race. The fix ordered the unit after the VPN daemon with a
network-online want and spaced out the restart retries. The crash-loop no longer recurs on cold boot. - Extended to a native client. A from-scratch mobile client later implemented the same streaming and confirmation handshake, proving the stateless resume contract was genuinely client-agnostic.
- First consumer of a shared AI gateway. When the fleet's AI gateway shipped its governance layer, this was the first app migrated onto it, routing all of its AI functions through the gateway, streaming verified end-to-end through the proxy, becoming the reference implementation the rest of the fleet followed.
What this demonstrates
- Guarded tool-use needs a stateless resume contract. Gating writes behind a confirm event that the client answers by re-POSTing approvals to the same endpoint keeps the handshake simple and lets any client, web or native, implement it identically.
- A backend that fetches configuration at boot creates a hidden ordering dependency. A loud crash on a transient boot-time DNS failure, combined with a restart throttle that gives up in a second, turns a five-second dependency delay into a dead service. The fix is to order the unit after its network dependency, not to loosen the throttle blindly.
- Most promotion failures are operational, not code. The production promotion surfaced only operational issues: a credential pasted with literal placeholder brackets, a config file missing from the build output, runner file-ownership, a stale local edit blocking a pull, each caught and resolved during the smoke test.
- A safety contract can be simple. One endpoint drives both the initial turn and the approved resume, so read tools execute inline while writes always pause for human approval, without duplicating logic per client.