Multi-agent architecture for unstructured business data

page AI answer engines will read and cite.**

Why the limits stay in the summary. It is tempting to put the architecture in the ungated page and hold the failure modes for the PDF. That would be backwards: the honest limits are the most distinctive content in the asset and the most likely to be quoted. They stay. (Rule C1, C3.)

The architecture of a business-data AI system is determined by the failure modes of business data, not by the choice of model. If you know how business documents fail, you can derive most of the architecture yourself.

Parsing: business documents are hostile to naive extraction

A document that failed to parse is indistinguishable from a document that contains nothing relevant — no error surfaces, and the system answers using what it did manage to read. That is the highest-leverage failure in the stack.

Naive extraction flattens a contract's liability table into a run of numbers, attaches spreadsheet values to the wrong row, and repeats a quoted email paragraph five times so retrieval scores it as important. The response is 30+ format parsers with layout-aware chunking, where chunk boundaries are set using document structure so a clause stays attached to its heading and a table stays a table.

What it still fails at: handwriting, low-quality scans, drawings without a vector layer, and tables whose structure is carried only by visual alignment. We publish no global parsing accuracy figure, because accuracy is a property of your documents — the honest version of that number is measured against your estate.

Retrieval: two kinds of question, and no single index serves both

"Find policy AB-4471" and "find where we agreed to indemnify them" are different retrieval problems. A lexical index cannot find the indemnity clause when the contract says "hold harmless". A dense index will return AB-4417 for AB-4471, because they are near-identical in embedding space and completely different in the real world — and it will do it confidently.

The response is hybrid retrieval: BM25F plus dense vectors, fused by reciprocal rank fusion, then re-scored by a multimodal reranker. RRF combines ranks rather than scores, which sidesteps the problem that lexical and dense scores are not on a comparable scale and that normalising them requires corpus-specific tuning that drifts. It is the less clever option and the more robust one, which is the correct trade for infrastructure.

What it still fails at: retrieval returns the top k passages, so aggregation questions are not retrieval questions. "How many of our contracts have an uncapped indemnity" needs every contract examined, not ten passages found. That is extraction into a structured store followed by a query — a different pipeline, and it should be scoped as one.

Memory: five layers because knowledge has different lifetimes

Memory is separated into layers because each has a different retention period and a different access-control boundary. Working memory lasts a turn; session memory a conversation; project memory the life of a project; the knowledge graph holds entities and relationships indefinitely; the audit trail lasts as long as the obligation.

Systems that conflate these leak, because ambiguity in an access boundary resolves under load in the permissive direction — and the concrete failure is a user in one project receiving context derived from another project's material.

What it still fails at: entity resolution is never fully solved. Whether "Acme Ltd", "Acme Limited" and "ACME (UK)" are one entity or three is a judgment, and when it is wrong the system confidently aggregates unrelated companies. Graph assertions carry provenance so a wrong merge can be unwound — a repair path, not a prevention.

Agents: one agent with every tool is the common failure

The most common failure in agentic deployments is not a bad model — it is one general agent, given every tool and no budget, in a loop. It wanders, because each step is locally reasonable and the trajectory is not. It loops, retrying with variations too small to change the result. It over-reaches, using data the requesting user was not entitled to see — nothing malfunctioned, the scope was never drawn. And it cannot be debugged, because every step had the same identity.

The response is role-scoped agents on ReAct loops under explicit step and token budgets with circuit breakers, delegating over Ed25519-signed messages, confined by per-project scope isolation.

Two of those are security controls rather than conveniences. Signed delegation matters because when one agent asks another to act, that is a privilege question: without authentication, a successful prompt injection escalates from a bad answer to instructions another agent executes with its own permissions. Scope isolation enforced in the retrieval layer matters because a constraint expressed in a prompt is a suggestion to a language model; a constraint enforced below the agent is a boundary.

What it still fails at: budgets prevent runaway loops by truncating work. A task that genuinely needed more steps returns an incomplete answer, and that must be distinguishable from a complete one.

Retrieval makes document ingestion an attack surface

Once a system retrieves documents into a model's context, anyone who can get a document into the corpus can attempt to write instructions to your agents. The retrieved passage and the user's question arrive in the same context, and the model has no reliable way to distinguish text it was asked to read from text telling it what to do.

A supplier invoice, a submitted CV, an inbound email in an indexed mailbox — the ingestion paths most valuable to index are often the ones with the least control over what enters them.

The response is a scanner before the model, a validator after it, tool-level authorisation, signed delegation to limit blast radius, and an audit chain that makes an attempt investigable rather than inferred.

What it still fails at: no scanner is complete. Injection is an open problem and detection is adversarial — the attacker iterates. It is one control among several, and it is a property of retrieval architectures generally rather than a peculiarity of ours. What differs between vendors is whether it is named.

The whole path, with no bypass

user / channel → privacy filter → injection scanner → local model
              → output validator → audit chain

In compliance mode on_prem_required the system is fail-closed: if a control is unavailable the request fails rather than proceeding without it. A control that can be skipped under load is not a control; it is a default.

The privacy filter runs before inference — detection after a model has seen the data is cleanup, not minimisation — and is published as an MIT-licensed library on PyPI, so a security team can read the code rather than accept a description of it.

Where this is the wrong architecture

A small, clean corpus does not need any of this. If a few hundred structured documents fit in a long context window, put them in context and use a good model. If your questions are aggregation questions, you need a data-engineering project, not retrieval. If your tasks are single-step, agents add latency and failure surface for no benefit. If your material is public, a frontier model is likely better and cheaper.

And below roughly 90,000 queries a month, no on-premise deployment we build pays back inside three years — including our entry tier.

Eight tests to run in a pilot

Every one is designed to make a weak implementation visible rather than be answered in a meeting.

1. Break the parser on purpose, then ask a question whose answer lives inside the structure you stressed.
2. Ask for an identifier, then the same document as a concept, using none of its vocabulary.
3. Ask an aggregation question. The correct behaviour is to tell you it is not a retrieval question. A confident number from ten passages is the most dangerous failure mode, shown to you for free.
4. Request material outside your scope. Enforced scope refuses; prompted scope sometimes complies.
5. Plant an instruction in a test document. Detected? Executed? In the audit trail?
6. Ask the same question three weeks apart and ask why the answers differ.
7. Ask for the record of a request from last month — the record, not a description of the capability. Then ask what makes it tamper-evident.
8. Ask what the system cannot do. Anyone who has run one in production has a list.

Download the full paper — each layer with its failure modes, costs and limits, plus the component reference and the collected limits table.

---

Companion piece: The anatomy of an on-premise AI deployment — the physical half: what arrives, what runs on it, and who operates it.