The Chef Was Never the Problem: Why AI Pilots Fail in Production

You've seen the number. Ninety-five percent of enterprise AI pilots fail. It came out of an MIT study a year ago, Fortune put it at the top of a newsletter, and it's been recycled in roughly every AI keynote since.

Almost everyone quoting it has taken the wrong lesson from it.

The usual reading is that the technology disappointed — models oversold, vendors overpromising, reality catching up. That isn't what I've seen. In most of the failures I've watched up close, the pilot worked. It worked well enough that somebody signed a budget. Then it went to production, stopped working, and nobody could quite say why.

The tasting menu problem

Picture a chef cooking one plate for one critic.

She has all afternoon. The mise en place is immaculate. Every ingredient was picked that morning. She uses her best pan, plates it herself, carries it out to the table. The critic writes something glowing.

On the strength of that plate, the restaurant hires her and opens for dinner service.

Friday night. Three hundred covers. Half the prep is missing because a supplier didn't show. Two tables have allergies nobody flagged. The pass is backed up, one section is short-staffed, and the ticket printer keeps jamming.

The dish didn't get worse. The conditions did.

That's the gap between an AI pilot and an AI system in production, and it's why the 95% number keeps getting misread. Nobody's measuring whether the model can cook. They're measuring whether it can cook on a Friday.

What actually changes between the two

Look at it that way and the failure modes stop being mysterious.

In a pilot, the data has been cleaned by the same team that built the pilot. In production it arrives the way the business actually produces it — half-filled fields, three spellings of the same vendor, a free-text notes column carrying information that should have been structured a decade ago.

In a pilot there's a human reading every output, because seeing how it does is the entire point. In production that human is one person covering a queue of four hundred, and the checking quietly stops.

Scope moves too. One workflow with clean edges becomes four systems, two of which nobody currently owns, and one of which is a mainframe the vendor stopped supporting in 2019.

And when something breaks in a pilot, the team fixes it that afternoon. In production it joins a ticket queue behind a payroll bug.

None of that is a model problem. All of it is a kitchen problem.

The mistake underneath

A pilot isn't a small version of production. It's a different exercise answering a different question.

A pilot answers: could this work under good conditions? Worth knowing. Not worth very much.

Production answers: does this hold up under our conditions, with our data, at our staffing levels? Hardly anyone tests that before committing, because testing it is expensive, unglamorous, and demos badly to a board.

So the real failure isn't that 95% of pilots fail. It's that a pilot got treated as evidence when it was only ever a feasibility check. Then the number gets quoted as proof the technology isn't ready, and the actual lesson — we validated the wrong thing — goes unlearned.

What I'd do instead

Run the pilot on bad data deliberately. Not the curated extract — last quarter's real records, broken ones included. If it only works on clean input, you've learned something important while it's still cheap to learn.

Decide up front who owns it at 2am. If the answer is "we'll figure that out later," the project has already failed and just hasn't been told yet.

Choose the workflow by how messy it is, not how visible. The impressive demo and the durable deployment are rarely the same use case, and picking for the demo is how you end up in the 95%.

Instrument before you scale. If you can't watch the outputs drift, you'll hear about it from a customer, which is the most expensive monitoring system money can buy.

The number is fine. The conclusion isn't.

I've no quarrel with the MIT figure. Enterprise AI is hard and most attempts do stall.

But "95% fail" has turned into a way of ending conversations instead of starting them. It gets used to justify waiting, and waiting isn't free either.

The chef was never the problem. Nobody had run a dinner service.