How to Audit a Lead Scoring Model in Four Boxes
How to audit a lead scoring model: keep the score at handoff, sort every lead into four boxes, check conversion rises with the score, then change one thing.
Audit a lead scoring model by comparing the score each lead had when it reached you with what happened next. Sort every lead into four boxes, check that conversion rises with the score, then change one or two things. Without the saved score, you cannot audit.
How to audit a lead scoring model: start with what you saved
Here's an uncomfortable test. Do you know whether your score is any good?
Not whether it feels right. Whether it's right.
Most solo operators can't answer. They built a lead scoring model once, felt clever for an afternoon, and never looked again. A score that has never been compared against what actually closed is an opinion with arithmetic attached. It might be a good opinion. Nobody knows, including you.
The audit is how you find out. And it depends entirely on one thing you had to do months ago: keep the score each lead had at the moment you picked it up.
Scores change every night. Behaviour fades, new activity arrives. If you only keep today's number, then when a deal closes in six months you have no idea what the score said when it reached you. And that is the only number that tells you whether the model worked.
If you didn't save it, start now. Your first audit will be thin. Your second will be useful. There's no shortcut around that.
Check whether you have enough to audit
Do not audit on three deals.
A handful of closed customers will have something in common by chance, and you'll see a pattern that isn't there. Fit your model to that and you've fitted it to noise.
Wait until you have a reasonable number of finished deals, ones you can link back to the contact who created them. Set the trigger in advance. Pick the number of closed deals at which you'll revisit the weights properly, and write it down. Until then, restrict yourself to two kinds of change: fixing evidently broken inputs, and adjusting the order of the tiers when a clear pattern shows up. Leave the exact numbers alone.
If you're still at the stage where every conversation is research, you may not need a score yet at all. Whether lead scoring is worth it for a solopreneur covers that honestly.
Free guide · 21 pagesRation the one thing you cannot scale.The Lead Scoring Playbook: deciding which leads deserve your time.Get the free guide →The four boxes for your lead scoring model
This is the heart of it. Take every lead that crossed the line and put it in exactly one box.
| Closed | Did not close | |
|---|---|---|
| High score at handoff | The lead scoring model was right. Keep what you did. | False positive. The model overvalues something cheap. |
| Low score at handoff | False negative. Study this box hardest. | Also right, and reassuring. |
Two of those boxes are applause. Two are homework.
Start with the false positives, high score and no sale. These are leads your model loved that never bought. The cause is nearly always the same: a cheap, repeatable behaviour without a tight enough cap. Someone opened a hundred emails or visited a page over and over, and the score treated enthusiasm as intent. Find the cheap signal behind most of your high-score losses. That's your first fix.
Then take the false negatives, low score and a sale. This is the box to study most carefully. Every customer who bought despite a low score did something on the way that your model didn't value, or arrived by a route your model can't see. Go and find out what. Read their history. Ask yourself what they did that you weren't counting.
That's where the next version of the model comes from. Not from theory. From the people who proved you wrong.
Does the score actually predict anything?
The boxes tell you about individual cases. This second test tells you whether the whole thing works.
Group your contacts into bands by the score they had when they reached you. Low, middle, high. Then look at the proportion of each band that became an opportunity.
The proportion should climb as the bands climb. High converts better than middle, middle better than low.
If it's roughly flat, you have a problem bigger than any point value. The highest band converting no better than the middle one means the score isn't predicting buying, however sophisticated it looks. Nudging weights won't fix that. You have to find what actually does predict buying, and rebuild around it.
That's a hard result to get. It's also a cheap one, because you found out before you spent another year calling the wrong people.
Read what you wrote down
Numbers are half the audit. The other half is text.
Open the reasons you recorded when you sent leads back to nurturing. Wrong fit. Too early. No budget. Just researching. If you've been writing them, you have a quarter's worth of notes in your own words. Read all of them in one sitting.
Patterns jump out. If "wrong fit" appears constantly, your fit gate is too loose. If "too early" dominates, your threshold may be fine but your impatience isn't. These notes are moments when you looked at the score and the human reality side by side. Few inputs are more valuable.
Then check three smaller things:
- Was your response time honoured? A model can look bad because you were slow, not because it was wrong.
- How many contacts sit above the line? The marketing qualified stage should be a minority. If it has swollen, the threshold has drifted.
- Which inputs score nothing? Keep a running list of tags, events and pages that feed no signal.
That last item catches a quiet killer. Scores depend on the names of things: the tag a form applies, the name of an event, the address of a page. Rename a tag or relaunch a form and a signal that was worth a great deal can start contributing nothing, with no error anywhere. When you're the one who both launches the form and maintains the score, you're the person least likely to notice, because you were thinking about the launch.
Free assessmentHow trapped are you, really?Take the Corporate Suffocation Index, the free solopreneur readiness assessment.Take the assessment →Change one thing, then write it down
Now the discipline part. You'll find five problems. Fix one or two.
If you change four things at once, the next audit can't tell you which change helped. You've destroyed your own experiment.
Fix in this order:
- Broken or renamed inputs first. Before any weights are touched.
- The cheap signal behind the false positives. Usually a missing cap.
- The missed signal behind the false negatives. Add it only if you actually record it.
- Weights last, and only after you've passed your closed-deal trigger.
Then write down what you changed and why. One line is enough. In a year, that log is the model's history, something you can read instead of something you have to remember.
If the cheap signal turns out to be a missing ceiling, the fix lives in how to cap lead scoring signals. If everything looks right on paper but the wrong people keep getting through, the common causes are collected in lead scoring mistakes to avoid.
A quarterly rhythm that fits a small week
You don't need a day. You need an unglamorous hour, four times a year.
Read the distribution and the share above the line. Read every recycle reason. Check the list of inputs that score nothing. Check your response time. Sort leads into the four boxes. Study the low-score customers. Run the band test. Change at most two things. Write them down.
Put it in your calendar as a recurring block. If it's a good intention, it will be skipped the first busy quarter. If it's an appointment, it won't.
Free guide · 21 pagesRation the one thing you cannot scale.The Lead Scoring Playbook: deciding which leads deserve your time.Get the free guide →What a bad result should not make you do
Your first audit will probably look bad. That's normal, and it's useful. The temptation is to tear everything up and start again with a cleverer formula.
Don't. A bad result is information about one or two specific things: a missing cap, a renamed tag, a threshold set too loose. Rebuilding from scratch throws away the parts that worked and teaches you nothing about the parts that didn't.
Keep the lead scoring model small enough that you can explain any score in a minute. A small model is easy to audit, because when something is wrong, the visible reasons point at it. A forty-signal monster hides its own errors. Audit is far easier on a simple thing.
Free assessmentHow trapped are you, really?Take the Corporate Suffocation Index, the free solopreneur readiness assessment.Take the assessment →Frequently asked questions
How often should I audit a lead scoring model?
Once a quarter is right for most small businesses. Read the distribution, run the four boxes, read your recycle reasons, check inputs that score nothing, and change at most one or two things.
What do I need before I can audit?
The score each lead had when it reached you, with its contributions, saved at every stage change. Today's score is no use for a deal that closes months later. You also need a reasonable number of finished deals.
What is a false negative in lead scoring?
A lead with a low score at handoff who went on to buy. It is the most valuable box to study, because the customer did something your model did not value or came by a route it cannot see.
Should I rebuild the weights after a few sales?
No. A few closed deals share things by chance, and you would be fitting the model to noise. Fix broken inputs and adjust tier order until you pass the closed-deal trigger you set in advance.
The full method, with the checklists, is in The Lead Scoring Playbook, a free 21-page playbook.
How trapped are you, really?
Take the Corporate Suffocation Index, the free solopreneur readiness assessment.