How to choose and organize KPIs & Metrics for your product?
Why do we need metrics and KPIs?
Metrics and KPI help us to make decisions.
Why do we need to make decisions and actions?
To achieve the goal. So your metrics and KPI should start from the goal.
When you have the goal, you can find a top-line metric that measures success and work to increase your KPI.
Three most essential questions about metrics that you can ask now:
Does this metric show us to progress to the product/business goals?
Does it tell us that the product is getting better and users are getting value?
What actions can we make if this metric will change?
To define KPIs, you can use one of the measurement frameworks for your product.
Measurement frameworks are a way of structuring metrics, and those all-important key performance indicators around the business's goals, strategy, and objectives.
Below are some of the frameworks that you can check, try, adapt for your product:
1. AARRR - widely accepted as the five most important metrics for a startup to focus on. That is because these metrics effectively measure your company's growth while at the same time being simple and actionable.
2. HEART - set of user-centered metrics. It was developed to evaluate the quality of the user experience and help teams measure UX changes' impact.
3. Unit Economics - profit/loss calculation per customer. This is an aggregate indicator that determines whether there is a financial sense in scaling the project and where it has a break-even point.
4. North Star Metric - is the key measure of success for a company's product team. It defines the relationship between the customer problems that the product team is trying to solve and the revenue that the business aims to generate.
5. Pyramid of Metrics - it is a hierarchy and classification in one framework. It helps to tune your metrics: review, systemize, find a relationship, focus the team on metrics of different levels.
These measurement frameworks satisfy different needs (startups, UX, economics, product), so choose that fit your situation (product stage, goals, tasks).
How to choose metrics and KPIs for your product?
For example, we will choose to try the Metrics Hierarchy framework.
It is a tree structure or diagram led by the primary product/business metric.
On the top, you choose your metric, and below, you decompose it (make formulas to calculate the metric).
To build it, you need to analyze the data and understand the real relationships between the metrics.
But before choosing the metric, you need to select the goal that you want to measure. It can appear in different places. Let's check an example when it comes from a product strategy. When you know the goal, you can create KPIs & Metrics.
Benefits of Metrics hierarchy:
Assists in anomalies investigation. If one metric decreases, you can check the metrics on lower levels. For example, if we see a drop in profit margin, we should check ARPPU(Average Revenue Per Paying User), the number of paying users, or changes in costs.
Prevents roll out of changes that might harm the whole company. If you launch experiments to improve the metric on lower levels (ex. Paying users), you need to check if it doesn't harm metrics on higher levels. For example, You can increase the number of paying users, but ARPPU will decrease, and the profit margin may also decrease.
Gives more context to the team. You can show the metric that we need to work on, and the team sees all components and leverages on which they can impact to move this metric.
Estimates the impact of lower-level changes on the KPI. You can create a model that will estimate changes on lower-levels on KPI. For example, if we increase Retention by 10%, we can estimate the impact on the Profit margin metric.
What's next? How to understand the metric?
There are a bunch of things that you want to check before defining the metric.
1. Check Seasonality / Dynamics to choose how you will compare the data: w/w, m/m, y/y, etc.
2. Check Sensitivity to see if your metric reflects the changes in the product.
3. Check data By Segments to see if you need to look at them separately.
4. Check Distribution to choose the right measure (AVG, Mean, etc.).
5. Check the Time component to choose the right measure.
6. Check Correlation to approve the importance/value of the metric.
Review your metrics based on Lean Analytics book recommendations:
This is another example that I like:
After you've created a metric, you can start to work on it.
Think with the team about goals, focus, and how we will achieve them.
Involve all team in the process of metrics creation.
Look at the business from all angles, understand the relationship between metrics, and make the system.
Based on the frameworks, you can build dashboards with all metrics & segments to quickly investigate KPIs changes.
Not every framework will fit your business. Adapt and conquer.
Hierarchy allows you to see all leverages on your KPI and manage them, prioritize and estimate the potential of new features.
List of great questions to your KPIs & Metrics:
Does this metric show our progress to the goal?
Does it tell us that product is getting better and users getting value?
What actions can we make if this metric will change?
Is this metric a leading indicator or lagging?
Does it correlate with business metrics?
How can we improve this metric but make the product worse?
How will we monitor/compare this metric?
Do we need to look at separate segments regularly or alerts are enough?
Is it easy to explain it?
Is this metric sensitive? Is it change if we will add additional features or other product changes?
Does it have a time element to it?