DashboardTV · DashboardTV

Good Metrics are Hard

The right metrics, measures and broadcast appropriately can transform a business. The wrong metrics can sink your company. It's not always obvious which is which, even in hindsight.

The most important thing you can remember is that while metrics are just numbers, they are numbers that have the power to change the behavior of people who see them.

Time

Metrics are effective on a few different axis.

One of the most important axis is time. Metrics usually fall into one of two categories: Leading or Lagging.

A Leading indicator tells you what might or probably will happen. A Laggging indicator tells you what has happend.

Sales volume is an example of a lagging indicator, telling you what buisness closed over the past week, month, quarter, etc. Sales demo calls is typically a leading indicator, and used in conjuction with Conversion Rates can accurately predict future sales volume.

On a recruting team, interview volume is typically a leading indicator of how fast you can hire for a particular role. Whereas support volume is an lagging indicator of the quality of your product, its documentation or surrounding ecosystem.

It's important to understand the benefits of each type. A Lagging indicator on your hard drive doesn't help. you don't need to know when it's full, you need to know before it fills. Similarly, A Leading indicator is never a guaruntee. There's always the chance of the world shifting underneath you. This may be a new product by a competitor or even something extreme like the COVID19 pandemic.

Who is the audience?

Since the goal of a good metric is to change behavior, it's important to understand the audience whose behavior you want to change.

Leadership and C-suite need high level rollup metrics. The kind of numbers you would report to the board of directors.

A Customer Support center would have very relevent tactical metrics.

Beware Metrics that Mislead

One of the worst types of dashboards are those that are filled with nothing but numbers because the numbers are available. You need to carefully curate the metrics you publish.

There is a story (myth?) of a customer support division who wanted to reduce call time so that more customers were serviced. By measuring and reporting call time, they drove the opposite behavior. Support agents would answer the phone then immediately hang up. Average call times were down and managers were praised, but clearly hanging up on customers who need you isn't the way to win over them.

Another phrase you hear a lot is vanity metrics. If your website gets a ton of visitors and you track and publish those numbers sometimes odd things happen. A Silicon Valley company noticed that even tho their vistors were going up, they weren't making any more money. People would come to the site but not purchase anything. The company was optimizing the wrong behavior.

Measure what matters. Figuring out what matters is hard.

In the CS case above, a company would be better off measuring NPS or some kind of cusomter satisfaction survey result. Instead of measuring visitors, start measuring and reporting on metrics such as customer conversion rates or signups.

Most companies aim to make money and often tying metrics to core business goals such as sales volume rather than antecedent activity makes sense.

This is not always obvious however. Finding good metrics is a process. You are very rarely "done". There is a period of experimentation and refinement. Whether your employees or your customers or any cohort, changing people's behavior is difficult and often filled with unintended consequences.

Gaming the System or Abusing Metrics

Pubslihing the wrong metric can lead to the Cobra Effect:

The British government was concerned about the number of venomous cobras in Delhi.[3] The government therefore offered a bounty for every dead cobra. Initially this was a successful strategy as large numbers of snakes were killed for the reward. Eventually, however, enterprising people began to breed cobras for the income. When the government became aware of this, the reward program was scrapped, causing the cobra breeders to set the worthless snakes free; the wild cobra population further increased.

It is often the case that focusing on one metric too heavily can distort behaviors and outcomes. Sometimes you need a second metrics to cover the weaknesses of the first.

If a sales organization is measured on deals closed, this seems like an fairly striaghtforward metric. However, if those customers tend to churn fast and leave after a year or so, something has gone wrong. It's possible the sales team is going after the types of customers who don't use the product, don't derive value from it or otherwise don't fit a profile.

Adding a covering metric of LTV (lifetime value) or churn rate could potentially help improve the average customer profile, espcially if your product is subscription based in some way.

How to expose the metrics to the people who need to see them.

Metrics that aren't communicated don't help. We're obviously big fans of metrics on a TV on the wall here at DashboardTV.com. You have lots of other options however. Daily emails are common, but you have to make sure the information is interesting or useful. You don't want the emails to end up filted away. Periodic, automated slack messages can work well. Slack isn't usually an informal todo list in the way that email is. Similar to email, if the company isn't engaging with the metrics, you might find the channel muted.

We don't usually recommend relying on your people having to go login and look up the metrics. You typically want to push it to them in some manner.

You can't reach every employee every time. Sometimes it works best if the metrics you push are more relevent to the audience. Instead of forcing company wide metrics to everyone. consider department or team level metrics. Sales teams often live and die by ladders or pace against accelerators. Support teams typically need their NPS high. Good metrics are important, But not every metric is personally relevent to every employee.

Ideally, employees are seeing and acting on metrics they can affect in some way. In general, things that an employee has control over, should be looked at more often (real-time, daily, weekly). Things that an employee doesn’t have control over, look at less often (monthly, quarterly, yearly). For the latter, monthly, or quarterly reports, company-wide all-hands and other broad status meetings work great.