Running AI agents on free models: what 735 real runs taught me

Free AI models for agents can carry a lot of the work, and in my own system they do. Between 12 September and 19 October 2026, 735 of the 1,346 runs in my own agent system went through free model routes or a model running on my own machine. That is more than half the work, at no per-call cost.

It is not free of trouble, though. Taken together, the free routes failed on 27% of their runs, against 21% for everything else, and the ways they failed were predictable. This is what the log shows, and the three habits that keep a free setup usable. If you want to talk through which jobs in your business could run this way, book a call.

How often did each free route fail?

These are my own numbers, under my own workload. They describe how each route held up for me, not how good the service is for everyone.

For comparison, the whole system failed on 24% of runs over the same weeks. The full breakdown of every failure is in why AI agents fail.

The 97% is worth a sentence on its own. That tier allowed one request a minute per model. An agent rarely makes one call: it reads, thinks, uses a tool and answers. So a limit that sounds generous for a chat was useless for an agent.

Why do free models fail more often?

Because they are shared, and because they change. In my log the free-route failures came down to four things:

• Busy models. Messages like “temporarily rate-limited” and “all providers for this model are busy right now”. The model existed and was free. It just had no room for me.
• Allowances that run out. One route gives a free allowance per rolling seven days. When it was used up, 18 runs failed until the next period started.
• Models that get renamed or withdrawn. A model id that worked last week comes back as “does not exist”. Free models come and go faster than paid ones, and some stop being free without much notice.
• Reasoning models with nothing left to say. Some free models think before they write, and the thinking counts against the same output budget. A few spent all of it thinking and returned an empty answer.

None of these are about intelligence. When free AI models for agents answer at all, they are often as good as paid ones for drafting, sorting and research. The trouble is getting an answer at all.

Devian OS · Virtual OfficeReal screen

What keeps a free setup usable?

Three habits, each one built after the log showed me I needed it.

Check every day which models are still free

A job runs every morning at 05:40 and checks the models the agents use. On routes that publish which models are free, a model that has left the list is marked “not free” and is never called again, and the agents using it move to another one straight away. Without that check, the first sign of a model going paid would be a bill.

Give every agent a backup

Each model in the system can name up to three backups. When a run hits a usage limit before anything has come back, it moves to the next model in the chain. It never switches half-way through an answer, so a reply is never half one model and half another.

Match the model to the job

The same free model is good at one job and bad at another, so I pick per role, not per agent:

1. Conversation needs speed. My voice assistant needs a fast free model. A slow one, however clever, makes a conversation feel broken.
2. Writing needs a model that doesn’t overthink. Writing jobs ask for low reasoning effort, which stopped most of the empty answers.
3. Work with tools needs a model that supports tools. Some free routes refuse tool calls outright. Test the model on the real job, with the real tools, before giving it the role.
4. Private or overnight work can go local. A small model on my laptop, run with Ollama, is free and keeps data on the machine. It took over 90 seconds to answer with full context, so it does background jobs, never conversation.

When should you pay?

Not everything should run on free routes. My system also uses subscriptions I already pay for, and one paid route ran out of prepaid credits in late September, which is its own lesson about fallbacks.

My rule for a small business is simple. If a customer is waiting on the answer, or the job cannot be late, use a model you can rely on and give it a free backup. If it is research, drafting or a background job that can retry tomorrow, a free route is fine.

That split is also how I would start with clients. AI automation projects start from $3,000 with a fixed quote after a free strategy session, and the model bill afterwards depends a great deal on how much of the work can run on free or cheap routes.

What does this mean for an agentic OS?

An agentic OS is the layer that lets several agents work together: roles, a board, a shared memory, an approval gate and a choice of models. The choice of models is the part free routes stress-test. If you can swap the model behind a role with a setting, a free model leaving the free list costs you a minute. If the model is wired into everything, it costs you a day.

That is why I treat models as replaceable parts. The AI operating system guide explains the other parts, and how to build an agentic OS for a small business covers the order I would build them in.

The short version

More than half my agents’ work ran on free models, and it worked, with a higher failure rate than paid routes. The failures were busy models, used-up allowances, renamed models and empty answers. A daily check on what is still free, a backup for every agent and the right model for each job keep it usable.

If you want agents in your business without a surprise model bill, book a call for a free 30-minute strategy session. The whole system is described in my free book, The Agentic OS.

Frequently asked questions

Can you run AI agents on free models?

Yes, for a lot of work. More than half the runs in my own system went through free routes. Expect them to fail more often than paid models, and build in a backup for every agent.

Why do free AI models fail more often?

They are shared. In my log the failures were busy models, weekly free allowances running out, models renamed or withdrawn, and reasoning models that spent their output budget thinking.

Is a local model a free alternative?

It is free and private, but on an ordinary laptop it is slow. A small local model took over 90 seconds to answer me with full context, which is fine for an overnight job and useless for a conversation.

When should a small business pay for AI models?

When the work touches customers or cannot wait. Free routes suit research, drafting and background jobs. Anything a customer is waiting on deserves a model you can rely on.