May 2026
Every AI project has a moment where the demo works, the stakeholders nod, and someone says "great, now operationalize it."
That's where things fall apart.
Demos work because they're small. A single task, a single input, a single output. No state to manage, no files to sync, no approvals to enforce. You're watching it work in a controlled environment where nothing else touches the result.
Production is the opposite of that.
I've been running AI agents on my own infrastructure for two years. Not demos. Not POCs. Production systems that scrape data, manage knowledge bases, and run autonomously across multiple hosts. And the pattern I keep hitting is the same one every organization will face: the gap between "it works" and "it runs" is where AI projects go to die.
Here's what I mean.
AI changed a network device name during a bridge setup. Not a bug — a hallucination. It decided the name should be different, made the change, and moved on. I caught it by accident. If I hadn't been watching the logs, I would have spent hours debugging a problem that didn't exist until the agent invented it. The lesson: you need to be an active approver of every change, not a passive observer. If you're not in the loop, things go south.
I asked AI to automate file transfers between hosts. It requested SSH access to a system already sharing via NFS and SMB. It reached for a working solution, not the simplest one. When I pointed out the logic flaw, it used the share correctly. But I had to be there to catch it.
I was tailoring job applications. My agent altered the master CV and committed it to one repo without syncing the others. Two repos, two versions. My CV targeted "sales" roles for weeks before I noticed. That incident generated a new guardrail: every AI agent now has an "update all repos" rule. File sync between sandboxes isn't optional — it's a requirement you have to enforce.
I captured a database state during a VM migration and never refreshed it. Two weeks of stale responses from my RAG pipeline before I caught it. AI did its part with the migration, but the data was in use by other agents and processes in flight. Testing isn't just up/down — state matters.
AI consolidated 3TB of media from four hosts. Almost everything transferred. But it skipped an entire directory. No error. No warning. When I asked directly, the answer was: "I thought it had already been copied." A hallucination — it invented a completion that never happened. No way to know without checking every folder manually.
These aren't hypotheticals. They're Tuesday.
The pattern is clear: AI is good at the plan, good at the execution, but bad at the details that only matter after it ships. And when you introduce hallucinations into a production environment with a loose approval process, the failure modes multiply. File paths, hostnames, states, sync — the operational reality that separates a demo from a production system.
So what do we do?
Three things I've learned:
1. Sandboxes aren't optional. Every AI agent needs a controlled blast radius. File system, container, or VM — the framework changes with use cases, but the principle doesn't. No AI model has ever been granted root access to bare metal on my infrastructure.
2. State matters more than uptime. The VM came up. The migration succeeded. But the database was stale, the files were out of sync, and the agents were working from different versions of the truth. Testing needs to validate state, not just function.
3. Human-in-the-loop isn't a checkbox — it's a breakpoint. When a switch drops a connection mid-deployment, AI asks me to pull up the console. The break-glass scenario always involves direct human interaction. Plan for it. Be an active approver, not a passive observer.
The organizations that will win with AI aren't the ones that build fastest. They're the ones that figure out the post-delivery gap before it becomes a production incident.
What's the one operational detail your AI stack isn't telling you about?
*Andrew Katana is a cloud transformation and AI infrastructure leader. He writes about the operational reality behind AI adoption — from RAG pipelines to production failures to what actually works. Follow at [atkatana.com/blog](https://atkatana.com/blog).*
Built on a home lab, powered by local models, and owned by Andrew Katana.