A pattern I keep seeing goes like this. A team builds an AI demo. It impresses people in a meeting. Then nothing happens, or a few months later it's quietly switched off. The technology was fine. The project had one of a handful of familiar problems.
The usual suspects
It started with the tool, not the problem. "We should use a language model somewhere" is a hunt for a use, and it ends up pointing at whatever is easiest to demo, which is rarely what's most valuable.
Nobody measured the starting point. If you don't know how long the task takes now, or how many errors it has, you can't say whether the AI version is better. You end up arguing about how it feels.
It worked on ten examples. Demos use the clean, typical cases. Real work is full of oddities: a half-empty field, two languages in one message, a customer who writes in all capitals. Testing on a few hundred real, messy cases shows you what you'll actually get.
No one owns it. Someone has to be responsible for checking the output, fixing the prompts or the data, and deciding when it's good enough. When that falls between teams, it falls.
It doesn't fit into how people work. A tool that needs a person to leave their usual screen, copy something across and paste the result back will lose to the old way within a week.
The checking costs more than the saving. If every answer must be verified in full by an expert, you've moved the effort, not removed it.
Features of a good first project
When I help a team choose, I look for a task with these qualities.
- Frequent. It happens daily or weekly, so a small saving per run adds up.
- Text-heavy or repetitive. Sorting, tagging, summarising, drafting, searching.
- Quick to check. A person can look at the result and judge it in seconds, ideally against the source.
- Forgiving. A wrong answer is an annoyance that's caught, not a disaster that's sent to a customer.
- Measurable. You can count hours before and after, or errors against a sample you've checked by hand.
Examples that tend to qualify: sorting inbound support tickets, tagging customer feedback by theme, drafting the first version of a weekly report from numbers already computed, or searching a library of internal documents.
Examples that usually don't: anything where an unchecked answer goes straight to a customer, decisions about individuals, or calculations that code would do better.
Run it like a small experiment
- Write down the current process, how long it takes and how good it is.
- Collect a set of real examples, a few hundred if you can, including awkward ones. Set aside a portion that you never touch while building.
- Build the simplest thing that could work, with a person reviewing results.
- Compare with the starting point on the held-back examples.
- Decide, with the numbers on the table: use it, improve it, or stop.
Stopping is a good outcome. A two-week experiment that saves you from a six-month build has paid for itself.
Before you scale
Think about what happens at ten times the volume, what it costs per run, what happens when the underlying model is updated and the answers shift, and where the data goes. Those are dull questions that decide whether something lives or dies in production.
Most useful first projects are small. They don't make headlines, but they give a team something they actually use, and some confidence for the second one. If you'd like help choosing and testing a first project, our AI solutions work is built around that shape.