AI agents are just APIs with worse error handling

I've now wired language models into a few production surfaces — document extraction, a support triage flow, some internal tooling. The pitch decks make it sound like a new discipline. In practice about eighty percent of the work is the integration engineering we've always done, and the remaining twenty percent is one problem we genuinely haven't had before.

The eighty percent is boring, and that's good news

A model call is a network call to a third party that's slower than you'd like, occasionally rate limits you, sometimes returns something you didn't expect, and bills you per request. We have decades of institutional knowledge about how to handle that.

• Timeouts, retries with backoff, and a circuit breaker so one degraded provider doesn't take your app down
• Idempotency keys, so a retry doesn't double-charge or double-send
• A queue for anything that doesn't need to be synchronous, which is most things
• Cost tracking per feature, because a per-request bill is a per-request bill whether it's an LLM or a payment processor

None of that is AI engineering. It's the same discipline as integrating Direct Debit or an SMS gateway, and teams that already do it well tend to ship agents fine.

The twenty percent that's actually new

Here's the part that has no precedent: the failure isn't an error. It's a confident, well-formatted, entirely wrong answer.

A payment API that fails returns a 402. You catch it, you handle it, you move on. A model that fails returns a 200 with beautifully structured JSON containing an invoice total that doesn't exist. Your error handling never fires, because from the transport layer's point of view nothing went wrong.

The scariest response your system can get is a valid one that happens to be false.

So the engineering moves from 'handle errors' to 'assume plausible output is wrong until something cheap and deterministic says otherwise'.

Verify with code, not with another model

When we extract figures from a document, the model produces the numbers and ordinary code checks them: do the line items sum to the total, is the date within a sane range, does the reference match a pattern we recognise, does this supplier exist in our database.

Every one of those checks is a cheap, deterministic, testable function. None of them involve a second model call. When a check fails the item goes to a human queue rather than silently into the ledger.

const extracted = await model.extract(document);

const issues = [
  sumsMatch(extracted) ? null : 'line items do not sum to total',
  knownSupplier(extracted.supplierId) ? null : 'unknown supplier',
  dateWithinRange(extracted.issuedAt) ? null : 'implausible date',
].filter(Boolean);

if (issues.length) return queueForReview(extracted, issues);
return commit(extracted);

I keep seeing 'use a second model to check the first model' proposed as the answer. Sometimes it helps. But two systems that fail in correlated ways aren't independent checks, and you've now doubled your latency and your bill to get a verification you can't reason about.

Give the agent the smallest possible blast radius

The other lesson is about permissions. It's tempting to hand an agent broad database access because it makes the demo impressive. Don't.

Agents get the same treatment as any other untrusted integration: a narrow set of specific operations, scoped credentials, and destructive actions gated behind a human confirmation. Not because the model is malicious, but because it's non-deterministic, and non-deterministic plus DELETE is a combination you only get to regret once.

Log the whole exchange, not just the result

When a model call produces something wrong, 'it returned a bad total' is not enough to debug with. You need the exact input, the exact output, the model and version, and which of your checks tripped.

We store all of it. It's more data than a normal integration log and it has been worth it every single time — partly for debugging, partly because when a provider ships a new model version, the only way to know whether your accuracy moved is to have the old results to compare against.

It also makes an uncomfortable question answerable. When someone asks 'how often is this wrong?', the honest answer needs a number, and the number needs records. Teams that can't produce one usually find the real rate is worse than the vibe.

The teams shipping agents successfully aren't the ones with the cleverest prompts. They're the ones who treated a model like any other unreliable third-party dependency and built the boring scaffolding around it.

Keep reading