Hardening a Mandatory AI Gateway Without Breaking a Single Consumer

The one service every app routes its AI traffic through was itself missing authentication on its most sensitive surface. So I hardened its admin and destructive surface and closed two info-leak paths, all without breaking a single consumer, by matching the protection mechanism to each endpoint class rather than blanket-locking a shared gateway.

The challenge

The AI gateway is critical infrastructure: every app that makes AI inference calls (to a local model server, a cloud LLM provider, or any future provider) routes through it, so all token usage, latency, and cost land in one place. It's a Node/Express + SQLite service exposing an admin dashboard and dozens of endpoints across several routers, including a destructive surface that can pull, delete, load, and unload models.

The problem: the gateway everything depended on was itself unprotected: no auth on any endpoint, including that destructive admin surface, and the dashboard sent no auth at all. It was the last open Critical item from a fleet-wide security audit.

The naive framing was wrong in two ways, both surfaced during read-only scoping:

1. The assumed "just bind to loopback" fix would have broken real consumers: not every caller of a shared gateway lives on the same host, so a loopback bind severs them.
2. A blanket /api/* auth would have broken the admin dashboard, which sent no auth, and the service had no inbound operator token to authenticate with in the first place.

Approach: split the surface; match the boundary to the endpoint class

• Read-only scoping first, execution later. The first pass produced a consumer map and a phased plan, deliberately deferring execution because it touches the production node. That scoping is what surfaced that the "obvious" loopback fix would break the Docker and mobile consumers.
• Split the surface; don't blanket-lock it. This is the central design insight. Auth the admin and destructive endpoints, since those are the ones that can change or delete something. The pass-through consumer paths can't change system state, so they belong in a different protection class than the operator surface. The security boundary differs by endpoint class.
• Phase by risk. Phase 1 = low-risk input/header hardening with zero consumer impact; Phase 2 = bearer-auth the admin surface; Phase 3 = a review of the existing network scoping.
• Reuse proven patterns. The timing-safe bearer middleware was lifted from a sibling app, not reinvented.

Phase 1: input/header hardening. Forwarded request headers were flipped from "everything but host" to a strict allow-list; the generic proxy now returns a bare 502 instead of leaking the upstream error message; the destructive model-management endpoints gained a model-name validator where before there was only an existence check. The cleverest bit: flipping the cloud-provider proxy's response-header handling from a strip-list to an allow-list, which can never re-introduce a previously-fixed stale-encoding bug, and swept up a deferred cosmetic header leak for free.

Phase 2: bearer-auth the admin surface. A timing-safe requireBearer was mounted on the admin routers. The token reads from a new secrets key, and the service is fatal on startup if that key is absent: it never runs the admin surface unprotected. On the dashboard, rather than editing every inline fetch site individually, a thin shim wrapped window.fetch to attach the bearer header to every admin request and clear the token on a 401: one change, prompt-once plus local storage. A pre-deploy grep confirmed no consumer app depended on the admin path, which is what made this zero-impact.

Phase 3: a review of the existing network posture, and a decision to change nothing. A live check confirmed the existing network scoping already matched the plan; the originally-considered change would have broken working consumers for no security gain. So the correct move was to leave it. Knowing when not to write code is the point of this phase.

Ordering discipline. The new bearer token was minted in the secrets store before the Phase-2 deploy, because the fatal guard crashes the service if the key is missing. Every production deploy went through a manual pull and restart; only read-only verification ran on the production node.

Results

• All three phases shipped to production and verified live; the last open Critical closed, emptying the Critical backlog tier.
• Verified live (read-only checks): unauthenticated and wrong-token admin requests → 401; consumer traffic unaffected; cloud-provider proxy headers confirmed clean; malicious model names (../../etc/passwd, bad name!) → 400 invalid model name; and the dashboard prompts once for the token, loads, and doesn't re-prompt on refresh.
• Zero consumer breakage: the whole point. Because the boundary was matched to the endpoint class, every AI consumer kept working while the admin surface got locked down.
• Honest about limits: the generic-502 path and the valid-model positive path were code-verified and negative-tested but not exercised against a live dead upstream; one downstream mobile re-test remained pending. Both were recorded rather than glossed.

What this demonstrates

• Match the security boundary to the endpoint class. Token-auth the operator and destructive surface; protect the machine-to-machine paths that legitimately can't carry a token by other means. Blanket-locking a shared gateway breaks its consumers.
• Scope read-only before touching production, and verify assumptions with a grep, not a hunch. Proving no app consumed the admin path is what de-risked the auth change to zero impact.
• Fail closed on missing secrets. Fatal-on-startup if the token is absent guarantees the admin surface is never accidentally served unprotected, paired with secret-before-deploy ordering.
• Allow-list beats strip-list, and one wrapper beats N edits. Flipping to an allow-list structurally prevents header regressions; shimming window.fetch covered ~24 call sites without touching each.

A companion piece covers building this same gateway.