By Simon Weiner, founder of AS Consulting, the London AI-automation consultancy.
You cannot improve what you do not measure — and most small-business AI projects fail for exactly that reason. The technology is rarely the problem. The problem is that owners switch a tool on, feel vaguely busier, and can never say whether it actually helped. This post is about the single discipline that separates an AI project that survives from one that quietly dies: recording a baseline before you automate anything, and measuring the same number afterwards.
Why measurement, not the model, decides the outcome
Capable AI tools are now cheap and widely available, so the constraint on value is no longer capability. It is whether you can demonstrate the value you created. An automation that genuinely saves five hours a week but was never measured looks, at the next budget review, exactly like one that did nothing — and it gets cancelled just the same. Measurement is what converts "AI feels useful" into "AI cut our first-response time from hours to seconds and lifted bookings," and only the second statement survives a sceptical question. If you take one idea from the full guide, take this: the baseline is not admin, it is the thing that lets the project live.
What to measure, and when
The right metric is task-specific and is chosen to capture the value you intend to create. For an enquiry-response automation, measure first-response latency and conversion rate. For a document-drafting automation, measure time per document and error rate. For a lead-routing automation, measure throughput and misrouting. In every case, record the number before you touch anything, observe it during the supervised pilot, and observe it again after the tool runs on its own. The comparison must be like-for-like, which is why you record the baseline first rather than reconstructing it afterwards from memory, which never works.
Two measurement traps
Two mistakes quietly ruin otherwise good projects. The first is counting activity instead of outcome: tracking how many messages the AI sent, rather than what those messages achieved. Activity always rises with usage and tells you nothing about value. The second is measuring efficiency while ignoring quality. A tool that halves the time a task takes but degrades the output can be a net loss that a time-only metric hides completely. The fix is to pair an efficiency metric with a quality or error metric for any customer-facing task, so a speed gain that comes at the cost of accuracy is visible rather than disguised.
A worked example
A UK service business measured its enquiry handling before changing anything and found first responses often issued the next morning. It piloted an assistant that replied instantly, kept a human approving the drafts, and tracked the same latency and conversion numbers throughout. When the figures held up, it let the tool respond on its own — and booked 47 jobs in four days, with no other change. The point is not the headline number, which is one business's experience and not a promise; it is that the business could attribute the result to the change because it had isolated and measured it. Without the baseline, those 47 jobs would have been a pleasant mystery instead of a repeatable method.
How measurement fits the wider method
Measurement is the second of five disciplines that make small-business AI work: pick one repetitive task, record a baseline, pilot supervised, go live on evidence, and finish one task before starting the next. They reinforce each other — the baseline is what makes "go live on evidence" possible, and finishing one task before the next is what lets you keep measuring cleanly rather than drowning in mixed signals from five simultaneous changes. The deeper treatment, including a thirty-day sequence and the five failure modes, is set out in the AS Consulting guide and the accompanying LinkedIn article.
Measurement is what earns the budget
There is a political dimension to measurement that owners overlook. Every recurring cost in a small business is, sooner or later, questioned — by you on a tight month, by a partner, by an accountant. An automation that cannot answer the question "what does this actually do for us?" with a number loses that argument every time, regardless of how useful it really is. A baseline and a measured improvement are not bureaucracy; they are the evidence that wins the argument and keeps the tool in place long enough to compound. The owners who treat the first ten minutes of recording current numbers as the most important step of the whole project are the ones whose automations are still running a year later, because they can always show what they are worth.
From one measured task to an automated business
Measurement also tells you when to move on. Once a task is reliably handed over and the numbers have settled at their improved level, the same recording discipline frees your attention for the next task — and gives you a clean before-and-after for that one too. This is how a single measured automation becomes a measured sequence, and a measured sequence becomes a business that quietly runs its repetitive work on evidence rather than hope. The tools change; the discipline does not. If you measure, you can improve, defend and expand; if you do not, you are guessing, and guessing is what gets AI projects cancelled. Start with the number, not the tool.
The cost of not measuring is invisible until it is not
The damage done by skipping measurement is hard to see precisely because no number is attached to it. A business runs an automation for six months, never records what it saved, and then on a difficult month cancels the subscription to trim costs, taking with it a tool that was quietly saving a day a week, because nobody could prove it. The loss is real but invisible, and it is the most common way genuinely useful automations die in small businesses. Measurement makes the invisible visible: it turns we think this helps into this returns this many hours and this much revenue, an argument no cost-cutting exercise can casually override. The ten minutes spent recording a baseline is cheap insurance against losing something valuable for want of evidence, and it is the difference between an automation that compounds for years and one that disappears the first time money is tight and decisions get made on impressions rather than numbers. If you do nothing else from this article, record the current number for one task before you change it.
Frequently asked questions
How long should I measure before deciding? Long enough to cover normal variation — usually the two-week supervised pilot plus a couple of weeks of live running. If nothing measurable moved in four weeks, you likely automated the wrong task.
What if I did not record a baseline? Start one now for the next phase; you cannot reconstruct the past reliably, but you can measure from today forward and compare.
Is measurement worth the effort for a tiny business? Especially for a tiny business, because your time is the scarcest resource and you cannot afford to keep paying for automations you cannot prove are working.
Automate smarter. By Simon Weiner, founder of AS Consulting (asconsulting.top), London.
No comments:
Post a Comment