The most expensive words in AI right now are "let's build it and see." We've watched companies commit serious budget to AI initiatives on the strength of a demo and a hunch — and we've taken the opposite path with our own clients deliberately. This post is the reasoning behind that choice.
Why AI projects fail differently
Normal software fails visibly: a feature doesn't work, you fix it. AI fails statistically: the model works in the demo, mostly works in testing, and quietly underperforms in production where the data is messier and the stakes are real. By the time you know, you've spent the budget.
The root cause is almost always the same: nobody established, before building, whether the data could actually support the prediction being asked of it.
What a feasibility study actually is
It's a small, fixed-cost engagement with one job: produce evidence. In practice that means four things:
- Your real data, not a benchmark. Public datasets prove nothing about your sensors, your customers, your noise.
- Success criteria agreed in writing, upfront. What accuracy, on what metric, over what horizon, counts as "working"? If you can't define it, you're not ready to build it.
- A genuine attempt to hit those criteria — a real model, built quickly but honestly, evaluated the way production would evaluate it.
- An honest verdict, either way. "Your data can't support this yet" is a successful study. It just saved you the full build.
A real example
A German industrial client came to us needing to predict two operational values — calorific and steam output — minutes into the future, from volatile real-world process data. We didn't open with a big proposal. We opened with a feasibility study against their live data, with quality gates agreed in advance — and a commercial structure where part of our fee depended on passing them.
The model hit 95% accuracy across a 5–60 minute prediction horizon, evaluated via live API. Only then did the engagement progress. The client never had to take our word for anything — the evidence arrived before the commitment did.
If a vendor's AI proposal doesn't include a way for the idea to fail cheaply, you're funding their learning curve.
Questions to ask before any AI build
- What exactly are we predicting or automating, and how will we measure "good"?
- Do we have enough historical data, and does it actually contain the signal?
- What happens operationally when the model is wrong — and how often can we tolerate that?
- What's the smallest experiment that would prove or kill this idea?
The takeaway
Buy evidence before you buy ambition. A feasibility study costs a fraction of a failed AI build — and a vendor willing to tie their fee to passing agreed quality gates is telling you something important about their confidence.