How to Choose Scorecard Pillars Without Writing Forever

Choose scorecard pillars by the advice you can give, not by the topics you can think of. Each pillar needs its own write-up and actions at every level, so fewer is better. Keep the ones where people genuinely differ, and merge any that measure the same thing.

How to choose scorecard pillars you can afford

Pillars are also called dimensions. Same thing: the separate areas your scorecard scores and reports on.

Everyone starts by brainstorming. What are all the parts of this subject? Eight come out. They all feel important. You keep all eight.

That's the mistake. A pillar isn't a topic. A pillar is a promise to write something useful at every level, about one area, for strangers. Eight pillars is eight promises.

So the question isn't "what matters?" Plenty matters. The question is "what can I give advice on, and where do my people actually differ?"

Count the cost of scorecard pillars before you pick them

Every pillar multiplies your writing. Pillars times levels is the number of cells, and every cell needs a write-up and roughly four actions.

Look at what adding two more scorecard pillars does. It doesn't make the scorecard better. It makes your job bigger, and the extra pillars are usually the ones you know least about and write worst.

If you're short of hours, and you are, start with fewer pillars than feels respectable. You can add one later, once you've seen real people take the first version. You can't easily undo eight.

Free guide · 28 pagesBuild a scorecard that tells the truth.The Scorecard Playbook: an assessment that separates people and gives you a reason to talk.Get the free guide →

Four tests for every candidate pillar

Run each idea through these. If it fails one, merge it or drop it.

Test one: can you give specific advice?

For this pillar, can you say what people at a low level usually get wrong, what the first move is, and roughly what it costs? If all you've got is "improve this area", the pillar will produce generic advice. Generic advice is what separates a report someone acts on from one they skim.

Test two: do your people actually differ here?

You can only find this out with real answers, but you can guess early. If you suspect everyone in your audience scores about the same on a pillar, it won't change anyone's advice. After launch the check is concrete: look at the middle half of your respondents on that pillar. If they span under about ten points, the pillar separates nobody. Merge it into a neighbour or write harder questions.

Test three: is it truly a separate thing?

Watch for one pillar that comes out weakest for nearly everyone. If the same pillar is the weakest for most of your respondents, your scorecard is a single-topic diagnostic wearing five hats. Usually four of the pillars are measuring the same underlying thing. Better to know now. Ask yourself whether someone could be strong on one and weak on the other. If not, they're one pillar.

Test four: would a person want to reach it?

This one is about the experience. Question order usually follows pillar order, with the pillar name printed above each question. So you're not ordering questions. You're ordering chapters.

Name every pillar as a chapter title someone would want to get to. "Operational maturity" is a department. A plain name for the thing they're actually struggling with is a chapter.

Order the pillars like chapters

Because they are chapters, the order matters more than most builders realise.

• Open with the easiest, most flattering pillar. Let people feel they're doing fine and get moving.
• Put the hardest one third or fourth. Past the point where they've invested, before they get tired.
• Never open with the pillar you are most proud of. Open with the one that gets people moving.

Rhythm matters too. Questions that move on by themselves keep people going; anything that needs a press slows them. How the asking feels is a design choice, and how to calibrate a scorecard goes into reading whether it's working.

Two verdicts, so each pillar needs a job

A scorecard gives two verdicts. The overall result answers "who are you". The per-pillar result answers "what do you do next".

The first gets shared. The second is the deliverable.

That tells you what a pillar is for. Each one needs to produce a different next step. If two pillars would send someone to the same action, they're one pillar. It's the same rule as for overall results: if two would get the same follow-up, you have one fewer than you thought.

The results page sorts pillars weakest first and flags the top one as the place to start. So a person's weakest pillar is the one they read most closely. Pick pillars where a low score points to something you can actually fix, not something you'd merely describe.

A pillar is not a weight

One trap before you build. Don't add questions to a pillar to make it matter more. The average is taken inside each pillar first, so a three-question pillar and a twelve-question one count equally if their weights match. Adding questions doesn't raise importance.

Importance is a separate lever, set between pillars, and how to weight scorecard questions explains how it works. For now, choose pillars because they're distinct, not because you want one to dominate.

Free assessmentHow trapped are you, really?Take the Corporate Suffocation Index, the free solopreneur readiness assessment.Take the assessment →

Pillars that can say something uncomfortable

There's a temptation to pick pillars where everyone does fine. It feels generous, and it avoids awkward results.

Don't. An instrument that flatters everyone is worthless. A result that can't be uncomfortable creates no urgency, no credibility and no reason to act, and the sophisticated people you want to reach spot the flattery in seconds and discount everything else.

So test each pillar with a blunt question: could someone score badly here and be told something true they didn't want to hear? If a pillar can only ever say "well done", it's a courtesy, not a pillar. Replace it with something people avoid looking at.

The same goes for the subject as a whole. A scorecard can't manufacture concern about a problem nobody has. If your people don't already worry about it, no scoring will change that, and you'll get completions without conversations. Pillars are only as good as the worry underneath them.

What to do when the data says you chose wrong

You will choose some wrong. Everybody does. The data tells you which, and it tells you in plain arithmetic.

If a pillar's middle half spans under about ten points, merge it into a neighbour or write harder questions for it. If the same pillar is weakest for most people, check whether the others are really separate. If one pillar's questions are mostly dead, with one option holding the bulk of the answers, cut the questions first and see whether the pillar survives.

Cutting is the highest-leverage edit you'll make. It shortens the assessment, which raises completion, and it removes noise, which sharpens every result.

But hold the line on one thing: don't change what the options are worth after launch. Pillar structure is flexible early. Scoring is not. Anything that alters option values or weights forks the instrument, and old and new results stop being comparable. Tune before you send traffic, then batch your changes and date them in your own notes.

Free guide · 28 pagesBuild a scorecard that tells the truth.The Scorecard Playbook: an assessment that separates people and gives you a reason to talk.Get the free guide →

Your pillar-picking pass

1. Brainstorm freely. Write every candidate area down. Ten is fine.
2. Cut to what you can write. Use the grid sum. Five is a sensible ceiling for a first scorecard.
3. Merge lookalikes. If someone could not be strong on one and weak on the other, they're one pillar.
4. Give each a different next step. Draft one action per pillar. If two read alike, merge.
5. Name them as chapters. Order easiest first, hardest third or fourth.
6. Plan to test it. After the first fifty or so completions, check which pillar is weakest for nearly everyone and which has a narrow spread.

You'll probably still cut one of your scorecard pillars after launch. That's normal. The whole structure is part of how to build a scorecard, and the pillars are the part you're allowed to get wrong the first time, as long as you're watching.

Free assessmentHow trapped are you, really?Take the Corporate Suffocation Index, the free solopreneur readiness assessment.Take the assessment →

Frequently asked questions

How many scorecard pillars should you have?

As few as you can write good advice for. Five is a sensible ceiling for a first one. Each added pillar multiplies the write-ups and actions you owe at every level.

Do more questions make a pillar count for more?

No. The average is taken inside the pillar first, so a three-question pillar and a twelve-question pillar with equal weights count equally. Importance is set between pillars, not by question count.

What if one pillar is weakest for almost everyone?

Then your scorecard is probably one topic wearing several hats. Several of your pillars are likely measuring the same underlying thing, so merge them.

How do I know a pillar is not working?

Look at the middle half of respondents on that pillar. If they span under about ten points, everyone scores the same there and it will never change anyone's advice.

The full method, with the checklists, is in The Scorecard Playbook, a free 28-page playbook.