How to Measure if an Automation Works:Outcome, Not Output
How to measure if an automation works: pick the outcome first, take the number before you switch it on, compare it weeks later, and stop counting messages sent
To measure if an automation works, pick the outcome it was built to change, not whether it ran. Take the number before you switch it on, take it again a few weeks later, and compare. If nothing moved, the workflow is running perfectly and achieving nothing.
How to measure if an automation works (and why "it ran" proves nothing)
Open your automation tool right now. It will tell you something reassuring. Runs completed. Messages sent. Steps executed.
Lovely. None of that is the point.
A workflow that runs is a workflow that runs. It says nothing about whether the thing you built it for is any better. You can send a thousand perfectly timed messages and leave your business exactly where it was. A perfectly executing workflow that does not move the outcome is overhead with a progress bar.
So ask a harder question. What was this supposed to change? Then go and see if it did.
Pick the outcome before you build, not after
This is where most people lose the argument. They build first, then look for something to prove it worked. Whatever number went up becomes the evidence.
Do it the other way round. Before you switch the workflow on, write one line: this exists to improve this. Then take the measurement as it stands today.
No measurement before means no comparison after. You will be guessing, and a guess dressed as a result is the most expensive kind.
The guide's own checklist puts it plainly. Know what outcome this should improve, and measure it before switching on. That is a design step, not a reporting step. If you can't name the outcome, you are not ready to build.
Free guide · 21 pagesAutomate the follow-through, not the relationship.The Automation Playbook: what to hand to a machine first and how to build it so it behaves.Get the free guide →Match the measure to the handoff it protects
Every workflow worth keeping protects a specific handoff. So the measure is the state of that handoff, not a count of activity.
Here is how the pairing looks in practice.
| What you built it to protect | Wrong measure | Right measure |
|---|---|---|
| A quick first reply to a new enquiry | Confirmation emails sent | The time between an enquiry arriving and a first reply |
| No conversation stalls without a next step | Tasks created | How many conversations are sitting with nothing planned |
| Quiet leads get chased | Follow-ups sent | How many quiet leads come back, and how many are simply forgotten |
| A smooth start for new customers | Onboarding messages delivered | How many new customers actually complete onboarding |
| Promises on a call become actions | Reminders fired | How many promises were kept on the day meant |
Read the middle column again before you measure if an automation works. Every item in it is easy to count and nearly worthless. Every item in the right column takes a moment of thought. That moment is the job.
Measure if an automation works: the before and after, step by step
The method is dull, which is why it works.
- Name the handoff. One workflow, one handoff, one sentence.
- Take the starting measure. Use the state of things as they are today, with no workflow.
- Switch it on and leave it alone. Resist tweaking in week one. You will muddy the comparison.
- Measure again after a few weeks. Same method, same definition, same place you looked before.
- Decide. Better, the same, or worse. Only three answers.
Notice the third step. If you keep changing the workflow while you measure it, you will never know which change did anything. Let it run long enough to mean something.
And be fair to your earlier self. If the number moved, ask whether the workflow moved it, or whether something else did. A new offer, a busier season, a better page. You can't always prove it. But you can ask.
A worked example, in sentences
Say you sell a productised audit and new enquiries arrive through a form. Your pain is that replies wait for the one evening you are free.
The workflow: when someone submits the form, tag them as a new enquiry, create a task for you to reply within a day, and send a short confirmation saying you will be in touch personally.
What was it built to change? The wait before a first reply. So that is what you measure. Before switching on, you look at your last batch of enquiries and note how long each waited. After a few weeks, you look at the next batch and note it again.
Maybe the wait fell. Maybe it did not, because the task is created but you still ignore your task list. In that case the workflow ran perfectly and the handoff is still dropped. The fix is not another email. It is a better place to see the task, or fewer things competing with it.
Counting confirmation emails would have told you everything was fine. Measuring the wait told you the truth.
What to do with the answer
Three answers, three actions.
It improved. Good. Leave it alone and look again at the next review. Do not build three more like it just because one worked. Check each one against its own outcome.
It did not move. This is the useful one. The workflow may be running perfectly and achieving nothing, and it is far better to learn that before you build more. Ask why. Was the handoff you protected not actually the one leaking? Is the process underneath it wrong? Remember that automation amplifies whatever it runs, including a bad process. If your follow-up is clumsy by hand, it is now clumsy on a schedule.
It got worse. Pause it, find out why, and decide what happens to the people already inside. Sometimes the cause is a trigger that fires for the wrong people, in which case everything after it is wrong at scale. Sometimes it is a message that reads as a rule, not a person.
If the answer is "it did not move" and you can see no way to improve it, that is a signal to switch the automation off. Retiring a workflow is maintenance, not failure.
Free assessmentHow trapped are you, really?Take the Corporate Suffocation Index, the free solopreneur readiness assessment.Take the assessment →Measure the feel as well as the figure
Some of what matters will never show up in a count.
Read the replies. Read the complaints. Are people thanking you, or asking whether a human is there? A workflow can improve every figure you track and still make people feel processed rather than helped. Automation is excellent at making sure the right thing happens. It is poor at making someone feel known.
That is why the monthly review includes a look at replies and complaints. It is a small, slightly uncomfortable habit, and it catches the damage no number records.
If the signs are bad, the problem is probably not the timing or the subject line. It is that you handed the relationship to a machine. What a solopreneur should never automate covers where that line sits.
Check that it still means what it meant
A good result today does not buy you a good result next year. Measures decay, because the world underneath them changes.
At each review, check two things alongside the outcome. First, does the trigger still mean what it meant when you built it? A stage renamed, a form reworked, a tag reused for something else: any of these can make the same trigger mean something different. Second, has the process underneath changed? A workflow that faithfully automates last year's process is faithfully doing the wrong thing.
If the numbers look good but one of those two has shifted, your measurement is flattering a workflow that has drifted. This is also where spotting a silent failure and measuring results overlap: both ask you to read what actually happened, not what you designed.
Free guide · 21 pagesAutomate the follow-through, not the relationship.The Automation Playbook: what to hand to a machine first and how to build it so it behaves.Get the free guide →Keep the number of workflows reviewable
One last discipline. Every workflow you run is a workflow you have to measure.
When you work alone, the owner of every workflow is you. That is only workable if you can still say, in one line, what each active workflow is for, and if the number you run is one you can review on your own.
If you can't do that, you do not have too little measurement. You have too many workflows. Cut before you measure more.
Free assessmentHow trapped are you, really?Take the Corporate Suffocation Index, the free solopreneur readiness assessment.Take the assessment →Frequently asked questions
How long should I wait before judging a workflow?
A few weeks is the guide's rule of thumb: measure the outcome before you switch on, and again a few weeks later. Leave the workflow alone in between so you can tell what changed the result.
What if I forgot to measure before switching it on?
Switch nothing off yet. Start measuring now and treat today as your baseline. You can also compare against how the handoff behaved when you did it by hand, if you remember honestly.
Are open rates or click counts enough?
They tell you the message ran and was seen. They do not tell you whether the handoff is now being kept. To measure if an automation works, measure the state of the handoff itself.
What if the numbers improved but people seem annoyed?
Believe the people. A count alone cannot measure if an automation works, so read replies and complaints for signs they feel processed rather than helped, and check whether the workflow has wandered into territory that belongs to you.
The full method, with the checklists, is in The Automation Playbook, a free 21-page playbook.
How trapped are you, really?
Take the Corporate Suffocation Index, the free solopreneur readiness assessment.