Most AI agent products don't struggle because the model is weak. They struggle because the architecture around the model splits the user's work into pieces the agent can't see across.
Ours did too. Our production agent ran as five separate contexts. Each one handled a slice of the job, and each slice worked. The problem showed up between the slices: context lost at every handoff, work repeated because one part couldn't see what another had already done, and users who had to understand our internal structure just to predict what the agent would do next.
We migrated the whole product to one unified architecture. I led that migration, and it changed how I think about agent design.
Boundaries are where agents fail
Every internal boundary is a place where context gets summarized, and summaries lose exactly the detail that turns out to matter. When we cut the boundaries, a whole class of "the agent forgot what it was doing" bugs disappeared without any model change.
One context makes memory possible
With five contexts there's no clean answer to "what does the agent remember about this user?" Each piece remembered its own slice. The unified architecture is what made a persistent, per-user memory system practical: one place to write, one place to recall from. That memory layer shipped as a direct result of the migration.
Users think in one agent, so the architecture should too
Nobody asks "which of your five contexts handled my request?" They ask "why did it forget?" The internal structure was our problem. We had made it theirs.
The migration is the easy part to underestimate
Moving a live product with real users onto a new architecture is mostly not an AI problem. It's a migration-strategy, scalability, and integration problem, the same discipline as any production system change. My five years of backend migrations mattered more here than anything agent-specific.
The takeaway
The lesson I'd hand to anyone building agent products: before adding capability, remove boundaries. A model that sees the whole job beats a smarter model that sees a fifth of it.