We've built a lot of AI agents. Some of them work remarkably well — triaging 4,200 leads in 90 days, deflecting 68% of support tickets without a human touch, reviewing 31,000 listings overnight. Others, from past engagements and client post-mortems we've inherited, never made it to production. Or they shipped and failed quietly for months before anyone noticed.
The failures are rarely random. They follow patterns. Here are the six we see most often.
1. The demo doesn't test what production throws at it.
An agent that works on clean, curated data can still behave badly in production. The system prompt was written with optimistic examples. The retrieval pipeline was tested with well-formed queries from the engineering team. The tool calls were demonstrated with happy-path responses.
Real users are noisier. They ask questions the system wasn't designed for. They upload malformed files. They trigger edge cases at 2am when no one is watching. The demo succeeded because it controlled all the variables. Production removes that control, and the agent's actual robustness becomes apparent immediately.
The fix is deliberate: test with production-representative data before you ship. Pull real logs if you can. Use adversarial prompts from users who don't care about making the demo look good. Build the agent against reality, not the happy path.
2. No eval framework means no visibility into quality.
Most teams shipping AI systems don't have evals. This means they have no mechanism for knowing whether the agent is getting better or worse over time. A model update, a prompt change, a new retrieval index — any of these can move quality in either direction with no signal until a user complains.
Evals don't have to be complex. Start with a set of 50–100 representative inputs and expected outputs. Define what "correct" looks like for each. Run the eval suite before every model change, every prompt update, every deployment. If quality drops, you'll know immediately. If it improves, you'll know that too.
Without this, you're flying blind. The agent might be hallucinating 10% of the time and you'd have no way of knowing until the error rate shows up in your support tickets.
3. Retrieval quality is treated as solved when it isn't.
For agents that rely on retrieval-augmented generation (RAG), the quality of retrieved context determines the quality of the answer. A model that reasons well with bad context will still give a bad answer — confidently. This is worse than simply saying "I don't know."
The typical mistake: embed everything, build the index, ship it, call it done. What actually needs to happen: measure retrieval quality independently before it reaches the model. Does the top-5 retrieved context actually contain the information needed to answer the question? If it doesn't, the downstream generation step can't compensate.
Retrieval quality depends on chunking strategy, embedding model choice, metadata filtering, query rewriting, and reranking — each of which needs to be benchmarked for your specific data and query distribution. This is not a one-time setup task. It degrades as the corpus changes.
4. Tool design is underestimated as an engineering discipline.
Agents with access to tools — APIs, database queries, external services — are only as reliable as those tools. If the tool interface is ambiguous, the model will hallucinate valid-looking calls that return errors. If the tool response is verbose and unstructured, the model will misparse it. If the tool has no error handling, a transient failure becomes an agent failure.
Treat tool design the same way you'd treat API design for a software system. Clear input schemas. Explicit error types with actionable descriptions. Structured, predictable responses. Timeouts and fallbacks for external dependencies. Document what each tool is for and when to use it — not because the model needs documentation, but because writing it forces clarity on what the tool should and shouldn't do.
5. Hallucination handling is left as an afterthought.
Language models hallucinate. This is not a bug to be fixed; it's a property of the technology to be managed. An agent without explicit guardrails for hallucination will produce confident-sounding wrong answers, and users will act on them.
The countermeasures depend on the use case. For factual tasks, ground the model with retrieved context and instruct it to cite sources. For high-stakes decisions, build a human-in-the-loop step before any irreversible action. For customer-facing responses, add a confidence threshold below which the agent escalates rather than guesses. The specific approach matters less than the fact of having one.
6. There's no monitoring after launch.
Shipping without monitoring means you discover failures from angry customers, not alerts. By the time an issue surfaces in support tickets, it's been happening for days. The business impact has already occurred.
At minimum: log every input and output. Track latency and error rate. Alert on anomalies. Review a random sample of real conversations weekly. Set up a user feedback mechanism that's trivially easy to trigger ("Was this helpful?"). If you're not watching the system, you're not managing it — you're just hoping.
What actually works.
The agents that make it to production and stay there share a few things in common:
They were built eval-first. Before any integration work, someone defined what a good response looked like and built a test suite around it. This slows down the first two weeks and dramatically speeds up every subsequent change.
They were scoped narrowly. The first version did one thing well. It wasn't the full customer service replacement — it was the order-status lookup that handled 35% of ticket volume. Once that was solid, scope expanded. Agents that try to do everything on day one rarely do anything reliably.
They had explicit failure paths. Every agent has cases where it shouldn't try to answer. Building in a graceful handoff — to a human, to a search page, to a contact form — is not a sign of weakness. It's what makes the system trustworthy.
The model is a component, not the system. The most common mistake we see is treating the language model as the product. It isn't. The product is the system: the evals, the retrieval pipeline, the tool interfaces, the monitoring, the human oversight layer. The model is one part of that system, and not the most fragile part.