Why AI agents fail: what 328 failures in 1,346 real runs show

Why AI agents fail, in my own system, mostly has nothing to do with how clever the model is. I run my business on a team of AI agents, and between 12 September and 19 October 2026 they ran 1,346 times. 328 of those runs failed, and every one of them is still in the log.

This is that log, sorted by cause. If you are thinking about putting agents to work in your business, it is an honest picture of what goes wrong once the demo is over. If you would rather talk through your own setup, book a call and bring the job you want an agent to do.

What counts as a failure?

A run is one job an agent was given: answer a message, draft an article, review a task, check the day’s news. It fails when it ends without the thing it was asked for.

That second half matters. Early on, some runs finished without an error and still produced nothing. They looked green and did nothing. The rule now is that a run is judged by what it produced, not by whether it crashed, so those count as failures too. It is the first lesson in the Devian OS case study, and it is why the number below is higher than a dashboard of green lights would suggest.

What were the 328 failures?

Shares are of the 328 failures, rounded to the nearest whole number.

Read it from the top and a pattern shows. The first four causes, about two thirds of everything, are capacity, money and plumbing. The model was never reached, or never got to finish.

Devian OS · PerformanceReal screen

Why are busy models the biggest cause?

Because most of my agents run on free tiers, and a free tier is shared with everyone else using it. Of the 79 busy or rate-limited failures, 22 were a paid model telling me it was “experiencing high demand”. Nearly all the rest came from free routes.

The messages are blunt. A free model “temporarily rate-limited”. A free tier that “allows 1 request every 1 min”. On that one route, 30 of 31 runs failed, because an agent rarely makes just one call. I wrote up which free routes held up and which didn’t in running AI agents on free models.

What I built in response is a backup chain. Each model can name up to three backups, and when a run hits a usage limit before anything has come back, it moves to the next one. It never switches halfway through an answer, so you don’t get half a reply in one model’s voice and half in another’s. A daily health check also drops models that have left a free list before an agent calls them.

Why do connections drop?

66 runs ended with “fetch failed”: the request went out and nothing came back. Some were the provider, some were my own machine sleeping or the network going down in the middle of a long run.

There is no clever fix. Keep runs short enough to rerun, and count dropped connections as their own cause. If they hide inside “the AI failed”, you end up switching models to fix a network problem.

What happens when the system restarts?

35 runs were cut off because the system restarted while they were working, or were cancelled. They are marked “interrupted” in the log instead of sitting forever as “running”.

That sounds small. It matters because a job that is stuck as “running” looks like work in progress, and nobody picks it up again.

What happens when the money runs out?

34 runs hit a limit: prepaid credits depleted, a free allowance used up for the week, a subscription’s session or weekly limit. One provider’s prepaid credits ran dry in late September and stopped a feature until it moved to a free model.

The lesson is to give every paid route a fallback, and to make a limit show up as a limit. “You have used this week’s allowance” is a two-minute fix. “The AI is broken” is an afternoon of guessing.

Do models just fail to answer?

Sometimes. 19 runs, 6% of the failures, ended with the model returning no text and calling no tools. Most of those were reasoning models that spent their whole output budget thinking and had nothing left to write with.

Two changes cut that down: writing jobs now ask for low reasoning effort, and a run that comes back empty gets one more attempt with the tools withdrawn, so the model has to answer in words.

What goes wrong with tools?

18 runs failed because a model and its tools didn’t fit together. A model connected in a way that allows one turn and no tools was given work that needed tools. Another provider rejected the format of a tool call.

The fix is to test each model on the actual job, with the actual tools, before giving it that job. A model that chats well can still be the wrong worker.

What about setup?

26 runs failed on setup: an API key that wasn’t set, an app that wasn’t installed, a local model server that wasn’t running. Dull, and entirely fixable. The useful change was making each of these errors name the missing piece instead of returning a generic failure.

Is it getting better?

Yes, though not as neatly as a chart would like. The busiest week, 14 to 20 September, had 658 runs and 31% of them failed. Over the last 30 days there were 686 runs and 136 failures: 20%.

Part of that is the fixes above. Part is that I was experimenting less: the busiest weeks were the ones where I was trying every model I could find. Anthropic’s engineers give the same advice from the other side in building effective agents: use the simplest setup that works and add agents only where the task needs them. I would still plan for one run in five failing on free models, and build for it.

One more number from the same log. My agents read about 32.8 million tokens and wrote about 2.6 million: roughly twelve tokens in for every one out. Most of the work an agent does is reading context, which is why what you put in its memory matters as much as which model you pick.

What should you take from this?

If you are putting AI agents to work in a small business, five things from this log are worth stealing:

1. Expect failures, and count them. A system that hides its failures teaches you nothing. Mine keeps all 328.
2. Judge a run by what it produced. “No error” is not the same as “done”.
3. Give every agent a fallback model. Free tiers get busy, credits run out and models get renamed.
4. Sort failures by cause. A dropped connection, a used-up allowance and a confused model need three different fixes.
5. Keep a person on anything that leaves. Failures inside the system are cheap. A failure that reaches a customer is not. That is what an approval gate is for, and I explain how mine works in human in the loop: deploying AI agents without losing control.

If you are planning your own setup, how to build an agentic OS for a small business puts these lessons in the order I would build them, and the AI operating system guide explains the parts.

The short version

In 1,346 real runs, 328 failed, and most of the causes were capacity, money and plumbing rather than the model’s intelligence. Busy models, dropped connections, restarts and spent credits made up about two thirds of the failures. Backup models, a daily check on which models are still free, short runs and an honest log fixed most of what could be fixed.

If you want agents working in your business without the guesswork, book a call for a free 30-minute strategy session. The full method, failures included, is in my free book, The Agentic OS.

Frequently asked questions

Why do AI agents fail?

In my own log, mostly for plumbing reasons: a model that was busy or rate-limited, a connection that dropped, credits that ran out, or a model that was renamed. Only a small share were the model answering badly.

What failure rate should I expect from AI agents?

My system failed on 24% of runs over its first five weeks and about 20% over the last 30 days. Expect a high rate at first and plan for it: a backup model, short runs you can rerun, and a log you actually read.

How do you stop AI agents failing silently?

Judge every run by what it produced, not by whether it threw an error. Some of my early runs finished without an error and produced nothing. They now count as failures.

Are free AI models good enough for agents?

For a lot of work, yes, but they fail more often. Most of my busy and rate-limit failures were free tiers. Give every agent a backup model and check daily which free models are still free.