Enterprise AI agents are only as reliable as the messiest documents behind them

Enterprise agent reliability is decided by the document layer. A support agent reading a 40-page PDF, where the refund policy lives in a footnote on page 31 and a contradicting version sits in an old email, will confidently act on the wrong one. The work that makes agents dependable is unglamorous cleanup of the source documents and contracts nobody has owned in years, and that is the exact project no team volunteers for and no vendor sells. If nobody in the building can say which document governs, the agent cannot either.

Everyone is hyping agent autonomy. The bottleneck sits one layer below the model, in the paper the model reads.

The model already reads better than your docs are written

Frontier models parse a clean policy PDF fine. Give GPT-5 or Claude a well-structured, single-source refund policy and it will quote it correctly every time.

The failures come from the inputs. Three versions of the same SOP live in three folders. A pricing table splits across two pages, so the extraction grabs half of it. A scanned contract runs through OCR and 'net 30' becomes 'net3O', which reads as garbage or as a random number depending on the day.

The bottleneck moved off the model and onto the paper behind it. I build these systems for clients, and the model is almost never the thing that fails in a pilot. The source set is. A model that reads perfectly still reads the wrong document perfectly if that is the document you handed it.

A wrong answer costs more than no answer

An agent that quotes a superseded pricing tier, or approves a refund the current contract forbids, creates a liability a human never triggers. A human would have paused and asked which document is current. The agent runs straight through.

Price it out. One confident wrong action on a live contract clause can cost more than the whole automation saved that quarter. Approve a refund the 2024 contract disallows on a mid-six-figure account and you have handed back the savings from six months of ticket deflection in a single call. The math is asymmetric: no answer costs you a slow ticket, a wrong answer costs you the clause.

This is why 'the agent got it 90% right' is the wrong frame. The 10% is not evenly distributed. It clusters exactly where two documents disagree and the machine picked the loud one over the current one.

The plumbing is the project nobody owns

Making agents reliable means someone reconciles the duplicate SOPs, deprecates the stale PDFs, decides which document is the source of truth per policy, and structures the extraction so a table survives a page break. That is weeks of discovery work, and it spans legal, ops, and support.

No single department owns it. Legal owns the contracts but does not run the support agent. Support owns the agent but cannot decide which contract governs. Ops owns the SOPs but has never been asked to declare one canonical version. So the reconciliation stays undone, the pilot quietly underperforms, and everyone in the room blames the model, because the model is the only thing in the room with a vendor logo on it.

Call it what it is: input-ownership work. Someone has to look at every document the agent will read and rule on which version is authoritative. That ruling is a human decision recorded once, and it is the exact decision the pilot skipped.

Clean the source before you buy the agent

Teams buy the agent first and discover the document mess in production. Reverse the order. Map and reconcile the documents your agent will read before it reads a single one of them.

Sequencing is the whole argument here. A garbage input layer turns the best model into a confident liability generator, and you find that out in front of a customer instead of in a review. I have watched pilots stall for a month on this while the team debated prompt tuning, when the real blocker was two SOPs from 2022 and 2024 that nobody had ever archived.

The mechanism is boring and it works. Pull the documents. Find the duplicates and contradictions. Assign one owner. Have that owner declare the source of truth per policy and archive the rest. Then structure the extraction so the numbers survive OCR and the tables survive page breaks. Only then do you point an agent at it.

Handling the obvious objections

The first objection: RAG and retrieval quality solve this, so manual cleanup is wasted effort. Retrieval ranks what you feed it. Give it two contradictory documents and it retrieves both, and the model has no signal that the 2022 contract was superseded unless a human marked it so. Better embeddings rank a mess more precisely. They still surface both versions, and the model still guesses which one is true.

The second objection: models are getting good enough to reason through contradictions. Reasoning over conflicting sources means the model guesses which one governs, and in a contract dispute a guess is the liability you were trying to remove. The fix is a decision recorded by a person about which document is authoritative. That decision is the ownership work everyone keeps skipping because it lives between three departments.

The third objection: this is just data hygiene, and every consultant says clean your data. True, and it usually lands as a slogan nobody acts on. The claim here is narrower and you can execute it this week: before enabling an agent on a document set, name the source of truth per policy and archive the rest. That is not a cultural program. It is a spreadsheet, an owner, and an afternoon of rulings.

What to do this week

Pick the one workflow you most want an agent to run. Support refunds, contract lookups, whichever one you keep pointing at in planning. Before you buy or enable anything, pull every document that agent would read into one place. Find the duplicates and the contradictions, and there will be more than you expect. Assign one named person as owner, and have them declare which version is the source of truth for each policy and archive the everything else this week.

Then book the 30 minutes it takes to check the ugliest scanned contract in the set: read what the OCR actually produced, because that is what your agent will read too. If nobody can say which document governs, do not turn the agent on. It will answer anyway, and it will be confident, and the wrong answer will cost you the clause.

If you have a problem, if no one else can help, and if you can find them, maybe you can hire Nyyon.