How much evidence is enough? Part 1 of 2 about prioritising assumptions

Stay up to date with the latest insights

I bet you're sick of product coaches and the world saying "collect evidence before you make a bet on the risky ideas". In fact, I myself shared an article last week that's main message was: You don't need to test everything but make sure to test the very risky stuff.

But then the inevitable question in your head becomes: "Thank you, Mr. Obvious, but how much evidence is enough? And how do I get my stakeholders to stop saying 'We should build this because I know it's good? and turn the conversation into an objective debate about evidence?"

How can you tell if how much evidence is enough to be confident?

I'm a huge fan of David Bland's Assumptions Mapping. I use it myself for my own ideas, I train teams in my workshops, and I coach teams on continuously using it for their discovery activities, especially for 0-1 products.

In this 2-part series, I will share two practical ways to enhance your Assumptions Mapping that will transform these frustrating debates into productive decisions. I will show you how to combine Assumptions Mapping with two other methods to make assumption prioritisation more concrete.

And yes, you can use these techniques in your day to day conversations about "knowing enough to proceed with the idea" - without running a full Assumptions Mapping exercise and whether it's a 0-1 product or simply a new feature for your existing product. I will explain how at the end of each article.


Combination 1: Using the Confidence Meter to add precision to evidence

The standard Assumptions Map has two axes: Importance (vertical) and Evidence (horizontal). That horizontal axis is where teams get trapped in subjective arguments about what counts as evidence and how strong that piece of evidence is.


How can you tell if you have "enough" evidence?

That's where Itamar Gilad's Confidence Meter comes in.

The Confidence Meter provides a numerical scale from 0.01 to 10 that helps teams assess their confidence level based on the type and quality of evidence they have:

  • 0.01-0.1: Opinions (hunches, beliefs without data)

  • 0.2-0.3: Assessments (expert evaluations)

  • 0.5-3: Data (usage statistics, surveys)

  • 3-7: Test results (actual experiments)

  • 7-10: Validated in market

My most favourite version of the Confidence Meter is the infographic that Itamar shared in this LinkedIn post. Save it, share it, put it on a Miro board. It's gold!

And here’s a link to his confidence calculator as a spreadsheet.


Now, when you map the Confidence Meter to the Assumptions Map, it could look something like this:

When I use this with teams, I guide them to place each assumption along this spectrum rather than simply in the "evidence/no evidence" quadrants. This creates much more nuanced and productive conversations.

Why I wrote that it could look like that is because your specific thresholds may differ based on your industry and risk tolerance – the key is to define them explicitly rather than arguing abstractly about "enough evidence."

For example, at Doodle, in my team I established that a confidence level of 3 was enough evidence for low-risk assumptions, while higher-risk ones needed to be at least a 6 or 7. Doodle is a productivity SaaS that doesn't destroy lives when something goes wrong and can be iterated on very quickly. Under these circumstances speed is more important exact knowledge of the truth. Therefore, a 3 (sometimes even less than 3 🤫) can be absolutely sufficient while in a general high-risk industry or difficult to iterate environment a 7 might be the minimum.


Confidence Meter without Assumptions Map

Not every debate needs a full blown Assumptions Map. Some debates start with the simple question "What is the worst that can happen that would ruin our idea?" Or sometimes they are even completely unguided, just a debate about opinions instead of data and evidence.

In those situations, there are two potential questions you can throw in, depending on your environment:


Option 1: "Is this a known, presumption or assumption?"

"Wait. Your argument, is this a known, presumption or assumption?".

Then give them the definition of each. I define:

  • a known as something that is a fact by nature's or market’s laws or something that we have lots of strong evidence for.

  • a presumption as something that we take as a fact (even though it's not) because we have lots of indication or some strong evidence of its truth. But increasing the confidence even further would be too costly or impossible.

  • an assumption as anything that we guess or think that we don't have any evidence for, or only some light indication.

Whatever they say, you can stark asking for the concrete evidence according to the Confidence Meter. Either by asking directly: "Ok, how many people out of how many said they want it? Did they show any action that proves they'd really use it if we built it? Are they already paying for something that serves the same JTBD?" etc. Or by applying your best customer interview techniques to your internal conversations: "It's a presumption? Oh that sounds great. Could you talk me through what happened in that regard that made you learn that this is a presumption. I want to make sure I get it right."


Option 2: "Which data points do we have?"

Depending on the openness of your colleagues and ability to cope with direct feedback, you can either ask directly for the data points and evidence, and how they collected it. A bit like "How do we know that it's more than an assumption?

Or you can directly mention evidence pieces from the Confidence Meter and move from the most suitable high evidence-points evidence list to the low evidence-points evidence list. For example, in a low-data environment asking for A/B test results is ridiculous and will damage trust in your judgment, but asking "Do all competitors have it?" or even "is it a top customer request" or "is it a major loss-reason in our win-loss-analysis" are suitable.

From the evidence they share in the debate, you can understand the evidence strength and suggest suitable next steps.


When to use this combination

Use the Confidence Meter blended Map when:

  • Your team is open to learning

  • People are looking for a structure to guide evaluation

  • You want to establish consistent standards across multiple sessions


Disclaimer

Keep in mind that the level of directness and openness in your debates, and whether you will be able to continue with further experiments or not always depends on your work environment. Don't try it out blindly and when people get mad say "But Büşra said I should ask." No, please. Don't risk your job only to do thing "by the book". These are inputs that hopefully spark ideas on how to proceed when you're stuck, but using the tone of voice and patience that is required in your context.


2nd part next time

This was part 1 of bringing clarity to the evidence debate.

In the next article, I will share Combination 2 - the "Pain-Based" Assumptions Map - which you can use when:

  • Discussions have become stubborn or circular

  • You need to break through analysis paralysis quickly

  • Team members need a wake-up call about the real-world impacts of their assumptions

Can you guess which technique I blend with Assumptions Mapping for that? 😉


Product management insights, delivered to your inbox

Sign up for weekly product insights. No spam.