Metrics can serve a powerful purpose to create alignment, objective decision making, and velocity within an organization. However, too often I see metrics being misused that lose sight of the big picture, optimize in the wrong direction, or aren’t actually usable. Here are common mistakes to watch for out for when setting KPIs:
The perfect metric… that can’t be measured - There are often big picture goals that an organization aims to drive - revenue or time savings as examples. However, sometimes these are attributable to individual product changes (due to scale or noise, for example). In this case, it’s important to consider proxy metric and leading indicators (with proof that they ladder to the big picture).
Topline metric that occurs too late - While a purchase is a common and great ultimate aim, in the case of products with long consideration lifecycles, it may be a metric that takes too long to enable data-driven decision making. Similarly, retention is important strategically, but making decisions on 12 month customer life time values will lose too much time. Proxy metrics and leading indicators can again be powerful here.
Gameable metric that guarantees success - We want to make sure that fewer people call help support - this is very easy, we can make the contact number completely hidden! Watch out for metrics that can be achieved while harming customer outcomes. They can still be used, but at least ensure there is a counter metric to balance unintended consequences.
Operational metric focused on a local maximum - Teams often want to set metrics where they feel in control of their fate. However, sometimes when doing this, they lose sight of the big picture. For example, the Video Ad team that goals on Video Ad Revenue… and then proceeds to cannibalize Static Post Ad Revenue with their product changes, not driving the organizational goal of overall Revenue. Keep an eye on if the metric created ladders towards the broader organization’s aspirations or goals, or can come at the expense of them.
Alphabet soup of too many metrics - Every team is empowered to create a metric they own and are fully able to control and measure! Their metric can closely measure what is most important to their focus! This is great, right? Nope! In this scenario, it becomes impossible to compare across different teams and organizations to make tradeoff and resourcing decisions.
To ensure you avoid these common pitfalls above, set principles upfront around your metrics. Then, as you consider new metrics, be sure they meet the following principles:
Strategy - metrics are a proxy of a strategy, meant to serve as a measurable, objective indicator if the changes the team is making serve towards the overall strategy. Strategy must come first, and metrics must connect to it.
Measurability - metrics are only good in use. The metric must actually exist. It must be something the team can measure granularly and sensitively enough for the changes being made. This includes considering statistical power for experimentation and enabling cuts for optimization decisions.
Consistency - enabling a common language can create empowered and predictable decision making. It enables comparison and prioritization across different efforts and teams. Having consistent metrics shared across an organization unlocks the power that metrics can create for velocity and scaling.
Motivating and mission oriented - a great metric that captures customer benefit can motivate an organization to move towards their common goal. Don’t just consider the benefit to the company - what is the customer benefit for which the company is aiming? Realizing customer benefit will be critical for long term company success.
Right metric for right purpose - No metric is inherently bad, but requires consideration in how it is used. A counter metric, for example, can help balance out a metric that focuses too much on one part of the equation. An operational metric focused too granularly on a particular product feature can enable excellent optimization decisions, even if it’s not the goal metric.
Feel like your product metrics could use some help? Reach out to learn how Arc Growth can help!