What It Takes to Build an AI Research Assistant for Enterprise Data

In early 2024, AI demos were everywhere. Every week brought a new video of an AI system answering complex questions, synthesizing research, writing code that worked on the first try. The demos were impressive. Then you tried to use one of these systems on your actual data — the database with the awkward schema, the PDF archive with inconsistent formatting, the spreadsheets that made sense to the person who built them and to no one else — and the gap became apparent quickly.

The demo gap in AI is not primarily a model capability problem. The underlying models are genuinely capable of sophisticated reasoning. The gap is a systems problem: connecting those models to real enterprise data in ways that produce reliable, accurate, auditable results requires solving a set of engineering problems that don't appear in demos.

The Data Access Problem

Enterprise data exists in many forms: relational databases, document stores, file systems, APIs, spreadsheets, email archives, wikis, code repositories. A research assistant that can only access one of these sources has limited utility. One that can access all of them — translating natural language queries into the appropriate access pattern for each — is genuinely useful.

Building this multi-source access layer was the first significant challenge in building Agentica. Each data source type requires a different access strategy: SQL translation for relational databases, semantic search for document collections, API calls for live service data. The challenge isn't implementing any one of these — it's building an agent that can reason about which sources to query, in what order, and how to synthesize results from heterogeneous sources into a coherent answer.

The Accuracy Problem

Demo AI systems are evaluated on whether the output looks right. Production AI systems need to actually be right. The difference is significant: an output that looks right but contains subtle inaccuracies — a number that's slightly off, a date that's wrong, a condition that's stated as absolute when it has exceptions — can cause real harm in enterprise decision-making contexts.

Improving accuracy required solving three sub-problems: improving retrieval (making sure the relevant information is available to the model), improving faithfulness (making sure the model uses the retrieved information rather than its parametric knowledge), and improving verification (checking the output before returning it). The hybrid retrieval pipeline described in our other posts addresses the first. Structured prompting with explicit grounding instructions addresses the second. And a lightweight critic pass — asking the model to identify claims in its own output that are not supported by the provided sources — addresses the third.

The Transparency Problem

When a human analyst answers a research question, you can ask them how they arrived at the answer. They can show you the sources they consulted, explain which sources they weighted most heavily, and justify the reasoning that connected the sources to the conclusion. This auditability is essential in enterprise settings: an answer you can't verify is an answer you can't act on confidently.

Building transparency into Agentica required making the reasoning process inspectable at every step. Every tool call is logged with its inputs and outputs. Every retrieval operation records which documents were retrieved and their relevance scores. Every reasoning step that contributes to the final answer is tracked and can be surfaced on request. This operational transparency — not just the final answer but the full chain of evidence and reasoning that produced it — is what makes the system usable for consequential decisions.

The Safety Problem

A research assistant that can only read data is relatively safe to operate autonomously. A research assistant that can also take actions — write to databases, send notifications, update records, trigger workflows — requires explicit safety controls. The capability to act makes the system far more useful; it also introduces a class of failure modes that pure read access does not.

Agentica's HITL system was designed around this expanded capability surface. The key insight was that safety controls need to be embedded in the tool infrastructure, not in the model prompting. Prompting a model not to do something is a soft constraint that can fail under adversarial inputs or in edge cases the prompt author didn't anticipate. Tool-level authorization — requiring explicit human approval before any tool with write or execute access is called — is a hard constraint that doesn't fail for prompt-level reasons.

Two Years Later

Building Agentica has involved solving a long sequence of problems that weren't visible from the outside at the start. Each solution exposed the next problem. The retrieval pipeline evolved through three major architectures before reaching its current form. The memory system went through two complete redesigns. The safety controls were added reactively, then rebuilt proactively.

The system we have today is substantially better than the system we had a year ago, and the system we'll have a year from now will be substantially better than today's. This is normal for hard engineering problems. The gap between demo-grade and production-grade AI is real, it's closeable, and closing it requires treating AI integration as a serious engineering discipline — not a matter of finding the right prompt.