A rant on ‘active users’ as a North Star Metric, and how to fix it.

This one is dedicated to Tim Herbig who inspired me to write this up after I ranted on LinedIn about WAUs as fluffy North Star Metrics. Thank you for making me turn my rants into words, my friend.

Let's go!

In 2019, I researched “North Star Metric”. There was barely any information available.

The only relevant search result was “what Sean Ellis said”. 

Sean Ellis: A North Star Metric is a single metric that “best captures the core value that your product delivers to [its] customers.”

His content taught me what a North Star is, what it stands for, how to find it, and what a good NSM looks like, as opposed to a bad one. 

But that was all there was to it. No step-by-step guide, no practical examples, no other support material. So, at Doodle, where I was Lead Product Manager at the time, we tried different ways and different metrics until we found our NSM.

Today, it’s a whole different scenario.

When you google the term “North Star Metric” (in quotes), you get 126,000 results. Searching for “North Star Metric examples” gives you many pages listing NSMs of several companies, especially well-known ones. As an example, here is one of those lists that Lenny Rachitsky collected in June 2021.

On this list, you can clearly see two patterns around NSM: they’re related to revenue and ACTIVE users.

What you can NOT see is a further definition of what “active” means, and a reason why some of these companies have chosen a business KPI as a North Star Metric.  Nor can you see on those lists detailed definitions for each NSM. Instead, they are superficial and don’t show the nuance that makes the NSM actionable.

“So what, Büşra?!”

So, the problem is that readers believe that's all it takes to define a North Star Metric. But if you can’t go deeper and define what “active” means, you can’t capture the core value that your customers experience when they use your product.

Let’s see a few examples:

1. You are an online shop for furniture.
   Does any kind of “user activity” represent the core value of your product?
   No. Therefore, this isn’t relevant to you.
   
   

2. You are an online newspaper.
   Does any kind of “user activity” represent the core value of your product?
   Somehow. When a user reads an article, this is engagement, it's “activity”. So maybe it’s less the activity itself but the depth and frequency of engagement. Regardless, this is important for you. You should keep on reading.
   
   

3. You are an online scheduling tool.
   Does any kind of “user activity” represent the core value of your product?
   Absolutely! Absence of activity stands for “nobody uses your product”, which means no product success, which in turn means no business success.
   So, especially if this is your case, read on!

The North Star Dilemma

The NSM has its roots in Growth Hacking but it is a very valuable framework in product management and business alignment, too.

Now, let’s look at it from the product management perspective.

→ What is the NSM framework good for?

According to Amplitude, it serves three critical purposes at a company:

1. It helps prioritise and accelerate informed but decentralised decision-making. 

2. It helps teams align and communicate. 

3. It enables teams to focus on impact and sustainable, product-led growth.

Going back to the ambiguity around just saying “activity”: does this mysterious, undefined word “activity” help you prioritise and make decisions, align teams, and focus on impact and sustainable, product-led growth?

No. That’s why I consider just saying “activity” without defining it further as a vanity metric.

Vanity metrics are “‘good for feeling awesome, bad for action.’ [...] ‘[A vanity metric] makes you feel good but doesn’t mean anything to your business’” - Eric Ries

Now we have a dilemma.

According to Sean Ellis, you want a metric that always goes up and to the right. Because it should be a growth metric, and growth shouldn’t be tanked. With the NSM, you want a metric that correlates to business growth through quantifying the product’s core value. Yes, this sounds quite like a growth metric.

A cumulative metric always goes up and to the right.

HOWEVER…

1. Small Concern: 

In the analytics context, you don’t want to look at cumulative charts, as they always go up and to the right. They tell you that the metric is growing, but:

a) Especially for the people who are not so experienced in data, it’s not easy to spot when there’s a problem;

b) It’s not immediately obvious where the problem is. 

Cumulative charts? They're a celebration theater. They make everything look great, as long as the line keeps climbing. But is it really growing as expected? Or are problems hiding in plain sight?

When you don’t know if you have a problem, you can’t know which actions to take to ensure you’re on track. To me, this meets the definition of a vanity metric. 

It’s only a small concern though because, by now, the majority of companies know that more important than the NSM alone, are its Input Metrics. Especially in combination with a further breakdown into a driver tree, these metrics help you understand what went wrong. Plus a different visualisation of the NSM-progress and this point is fixed.

2. Big Concern:

Words like “relevant”, “active”, “good”, “bad”, and so on are shortcuts to keep things crisp but they are not precise. 

When a metric is open for interpretation, you can’t trust the numbers. Therefore, any discussion about priorities and decisions will include someone saying “yeah, but this is YOUR interpretation of the data, we could also interpret it this or that way”. Been there, seen that. This will sabotage your prioritisation and decision-making, rather than help it.

3. Huge Concern: 

If the definition of “activity” is not clear and any team can interpret any activity as value creation, then you’ll align your team based on what? 

This has a cascading effect.

The cascading effect of having different definitions of a metric.

The consequences of both the big and the huge concerns are that the NSM becomes a vanity metric in itself. When teams cannot work towards the NSM, they cannot influence product and business success appropriately. In turn, this means they can’t use the NSM to focus on impact and sustainable, product-led growth as much as they could if it was clearly defined.

So What Is The Solution?

I suggest you address this in two ways:

1. Define The Fluffy Words in Your NSM

Amplitude does a great job on this! 

That's Amplitude’s North Star Metric

Their NSM is Weekly Learning Users (WLU). Did you notice the difference? It’s not just Weekly ACTIVE Users. They go a step further and define what a “learning user” looks like: “a user who shares a learning that is consumed by at least 2 other people in the previous 7 days”. Wow! This is powerful! This is precise! This is actionable!  

Many companies define their NSM better but the problem is that you can barely find these definitions on the internet. That’s why many teams, who use fluffy NSMs without further definition, struggle with the practicability of it.

Here are some more examples where “activity” is defined:

• Miro: Number of collaborative boards within a business

• Twitch: Five-minute plays

• Spotify: Time spent listening to music

• Netflix: Median view hours per month

• Slack platform team: Number of automation interactions

• Quora: Number of questions a user answers

• Patagonia: Time period of clothes worn

• Amazon: Number of purchases per month

2. Understand Your Product's Core Value

Technically, you need to understand your product’s core value BEFORE you can define “activity” in your NSM. 

Therefore, look at your product’s core action. What do you want your users to do to generate business value? What is the core action that reflects the value your product delivers?

For example:

• If you’re Spotify, it’s the time users spend listening to music.

• If you’re Amazon, it’s the number of purchases per month.

This action should be SO DAMN GOOD, so valuable and accessible, that it turns your users into buyers. It’s not about engagement, it’s about true, repeatable value.

From there, you can derive your NSM. Once you’ve nailed this, you’ll have a metric that doesn’t just measure activity but ties directly to your product’s success.

There are different ways to find your product’s North Star Metric, which I’ll explore in a future article. For now, start by focusing on your product’s core action and use that as your NSM.

Conclusion

A strong North Star Metric is essential to align teams, prioritize effectively, and drive sustainable growth.

To get it right, you must avoid the fluff and vanity metrics like the plague. Otherwise, you’ll end up in the hell of vanity metrics that make you feel good but give you no relevant information about what is happening and how you can improve.

You should also have a deep understanding of your product’s core value and make it clear across the company - that’s the only way to empower teams to drive real change.

I’d love to know: how do you define the NSM in your organisation?

—
This article was edited by Diana Bernardo.