Why pilots stall before they reach the P&L
A pilot proves the technology works. It proves almost nothing about the operating model that has to carry the work afterwards.
A pilot that works is a strange kind of evidence. It tells you the technology is capable. It tells you very little about whether your organisation can run it, because a pilot is deliberately constructed to remove every obstacle that production will reinstate.
This is not a criticism of pilots. Removing obstacles is the point — you are isolating one variable. The mistake is treating the result as a prediction.
What the pilot quietly suspended
Look at how a typical pilot is set up and you can list what it took off the table.
Data was prepared by hand, or a single well-behaved extract was used. In production, the data arrives continuously, from systems with their own release cycles, and it is wrong in ways that are seasonal.
Users were volunteers. They were interested, they had context, and they forgave the rough edges. In production, the users are everyone, including the people who were not consulted and who have a working method they like.
The team was senior and dedicated. In production, the work is handed to a team with four other priorities and no memory of why the design is the way it is.
Governance was a conversation. In production, it is a control, and the control has to be evidenced.
And nothing had to be switched off. This is the one that is most often missed. A pilot runs alongside the existing process. Production usually means replacing it, which means someone has to be willing to stop doing the old thing.
The gap is organisational, not technical
When a programme stalls after a successful pilot, the diagnosis offered is usually technical: the model did not generalise, the data was not ready, the integration was harder than expected.
Some of that is true. But in our experience the binding constraint is more often that nobody has the authority to make the changes production requires. The pilot sponsor could authorise an experiment. Rolling it out means changing a team's targets, retiring a report someone's bonus depends on, or asking a function to accept an output it did not commission. Those are different decisions, made by different people, and they were not part of the pilot's approval.
So the work does not fail. It waits. It waits for a governance forum, then for a budget cycle, then for a reorganisation, and by the time it is unblocked the sponsor has moved and the model is stale.
Run the pilot inside the constraints
The fix is not a better pilot. It is a pilot designed to answer the organisational question at the same time as the technical one.
Concretely, that means four changes.
Use production data from the first week, in whatever poor state it is in. A pilot on cleaned data answers a question you will never ask again.
Include at least one reluctant user. Not to be difficult — to find out what the objection is while it is still cheap to design around.
Name the thing that stops. Before the pilot starts, write down what process, report or activity the new capability replaces, and get the owner of that thing to agree in principle. If no one will, you have learned something important about whether this reaches production at all.
Agree the measure and the line. Decide, in advance, which line in the operating plan should move and by roughly how much, and instrument for it before go-live. Retrofitting measurement onto a live capability produces numbers nobody trusts, which is worse than no numbers.
None of this makes the pilot much longer. It makes it considerably more informative.
What a stalled programme costs
The direct cost is the sunk spend, which is usually modest. The real cost is the second one.
An organisation that has run three pilots and shipped none of them has learned something about itself, and what it has learned is that this kind of work does not land. The next proposal is met with more scepticism, needs a bigger business case, and gets a smaller budget. Enthusiasm is a finite resource and it is spent on the pilots that did not ship.
That is the reason to care about the sequence. Not tidiness — the fact that a programme gets a limited number of attempts at credibility, and the first one sets the terms for everything after it.
The test
Before a pilot starts, ask one question of whoever is sponsoring it: if this works, what changes on Monday, and who has to agree to it?
If the answer comes quickly and names people, the pilot is worth running. If it produces a pause, the work to do first is not technical.
All insights