May 2026
Published: July 2026
Every AI model you deploy creates a new attack surface. Not from the model itself — from the connection between the model and everything it touches.
Two years ago I had one Ollama instance. Now I have four hosts, a RAG pipeline, browser automation, and file shares that need to stay in sync. The agents work. The models work. But the thing nobody talks about is what sits between them — and why that's the layer that matters most.
Every time I spin up an AI model, I have to make a decision: which model handles this request, where does the data go, and who gets to see it? Today that decision is manual. I configure it once and hope it holds up for the entire run. That can be simple if you are chunking data into a vector database or scanning PDFs for a research project, but when your development spans multiple teams, multiple components, and has a plethora of reporting, compliance, and audit requirements, it's not as simple as saying one size fits all.
Placing a general purpose LLM or agent in front of a customer and hoping for a positive outcome is not how enterprise AI works. But in the rapid rise of AI, your CFO is running Copilot, your sales team is on ChatGPT, and your developers are vibe coding with Claude. Each one is a separate subscription, a separate data flow, a separate access control profile. And none of them are talking to each other.
The missing piece isn't a better model. It's a broker.
I need a router that sits between my agents and the world. Anytime and anywhere I have an AI model running, I want to tether it to my personal data and access control profile. This isn't just a workplace problem — it's a privacy and security issue too.
Here's what that looks like in practice:
Request comes in. The router inspects it — not just the prompt, but the metadata. Who's asking? What consent level? What tools are available? What's the data classification? And if you haven't classified your data yet, that's step zero. Most organizations haven't, and that's the real problem.
Policy decides. Based on rules I've authored, the router makes the call: this stays local, that goes to the cloud, this one gets blocked entirely. The decision is logged with a full trace — who routed what, why, and what happened next.
Response goes back. The user never sees the routing layer. They just get the answer. But behind the scenes, the router enforced every policy I set: privacy, cost, access control, data residency.
I have used the term guardrails in a number of articles and in this case the router is part of that package. The router is the appropriate place to enforce basic restrictions. These routers would look much like early firewall or proxy servers, or maybe a more modern reference would be DNS blocking. No one in my home knows there is a Pi-hole behind the scenes and they never really need to. The router should be invisible to the people it protects.
Lemonade, an AMD-sponsored project, just shipped exactly this in v11.5.0. Their Router is a JSON policy that defines candidates (local and cloud models), classifiers (PII detection, semantic similarity), and rules (keyword matching, metadata inspection). A request comes in, the policy evaluates it, and the router picks the model. If I look at my proxy example above, this feels a great deal like early firewall and proxy policy.
The killer feature: metadata-based privacy routing. If a request carries `consent: denied`, it routes to a local model — no data leaves the machine. Fail-open to local by default. Privacy isn't an afterthought; it's the fallback.
They also added MCP client hosting — their server connects to external tool servers and exposes their tools through admin-gated endpoints. Same pattern as my Hermes agents with Skills and MCP connections, but built into the inference layer.
This isn't just a Lemonade thing. The pattern is showing up everywhere — OpenRouter, Cloudflare's AI gateway, any project that sits between a user and a model and makes a policy decision. The question isn't whether you need a router. It's whether you're building one before or after the incident.
Lemonade is an inference optimizer. It routes requests to models. But enterprise AI isn't just about model selection — it's about the full agent lifecycle.
The router needs to be agent-aware, not just request-aware. It needs to know that this agent is running a job search, that one is managing a knowledge base, and this one is posting to LinkedIn. Each agent has different data access needs, different consent levels, different blast radius requirements.
The router also needs to handle state. My RAG pipeline runs against a vector database. My knowledge base syncs across hosts. My file shares need to stay consistent. A router that only looks at the request misses the operational reality that AI agents operate on persistent state — and that state needs to be managed, not just routed.
So the router needs to know location, context, state, and it needs to understand the financial implications of the request. Token maxing is thankfully over, but the notion that I don't have to burn money for a simple review of say our monthly utilization in PS — that's a repetitive task that can be pushed to a frontier model but would be better served by a cheap model supporting an internal skill.
And there's the API gateway question. As more organizations deploy multiple AI services — chat, code generation, image analysis, document processing — the router becomes the unified policy enforcement point. Not just for model selection, but for rate limiting, cost tracking, and audit compliance. The API gateway and the AI router are converging, and most organizations haven't started that conversation yet.
The router is the new perimeter. It's where access control, privacy, and cost management live. Every AI deployment needs one, and most don't have it.
The architecture that wins: Agent → Router → Model → Knowledge. The router enforces policy. The agent executes. The model generates. The knowledge persists. Miss any layer and you're building on sand.
Local-first isn't optional. It's the default. Cloud is the escalation path for complexity, not the starting point. If your architecture doesn't default to local, you've already lost the privacy argument.
The organizations that figure this out early will own the next five years. The ones that don't will be writing the incident reports.
What's sitting between your AI models and the world?
#EnterpriseAI #AIRouter #PrivacyByDesign #AgenticAI #Infrastructure
Built on a home lab, powered by local models, and owned by Andrew Katana.