The utility of an unwatched dashboard
The utility of an unwatched dashboard
It's not ALL waste. Just mostly waste
A much shorter post this week due to the holiday break in the US. I hope everyone got some rest!
The most common reason I’ve heard to justify the creation of a dashboard is some variation of “we need this because [people] can then look at the metrics whenever they need it”. The theory is that now that the numbers and charts are up on a self-serve dashboard, that the analyst is free to do other work, while teams will have a finger on the pulse of the business like never before.
But Thanksgiving break just passed in the US, and more broadly, much of the world is entering the wintery part of the year where business and activities slow down due to multiple major holidays being in close proximity to each other. The number of people interested in looking at any dashboard and keeping tabs on a metric is at its lowest point of the year.
And yet, business and life itself keeps continuing on just fine. Moreover if you have access to dashboard pageview data, or just notice that the data cache is always stale, it’s pretty easy to discover that the vast majority of dashboards don’t get looked at. Once, I watched a breaking change get shipped to the data warehouse, and over half the dashboards of a couple of teams broke, and no one seemed to notice.
Armed with this knowledge, most people take the position that such a dashboard should not have existed in the first place. After all, what value can be generated if the tool is never used by the intended audience? To make things worse, if the data person needs spend time and effort to maintain the dashboard, that’s just adding insult to injury.
On average, I agree with the sentiment. It never feels good to put in effort to build something that doesn’t get used. But most things aren’t cut and dry, so I’ve been pondering on this recently — is there’s any redeeming value at all for having a bunch of rarely (or never) used dashboards lying around?
Why do most dashboards go overlooked?
There’s many factors that go into why people learn to ignore dashboards. The information contained can be repetitive over days and people get bored. It’s not relevant to what they’re working on. It’s somewhere hard to locate and they forget where it is. Etc.
But out of everything, I think the biggest one is simply because the numbers and charts on a dashboard very rarely have any direct personal meaning to the people using it. There’s tons of other work to do, and unless that dashboard is directly tied to your performance or compensation, there are probably more important things to look at. People are more likely to check stock prices when they actually own (and thus benefit from) the stock.
Even if people mean well and know that a metric is important, like say, revenue of the company/division/local group, it is still very rare to find an individual that successfully turns checking a dashboard, or reading an automated email, into a habit that sticks. Humans are just like that. Without an external source of motivation, it’s the default state of dashboards and other self-serve tools to be forgotten.
So bring on the external motivation
What’s the best way to provide external motivation to look at a dashboard? In my experience, shove a meeting onto people’s calendars to check and discuss the metrics on the dashboard. This creates a powerful forcing function to look at the thing, and also creates a situation where you can’t free-load and pretend to look at the dashboard — there are witness all around. Yes, this is like sharing your homework in school.
The value is that now discussions are happening about the data. It means that we occasionally must face the fact that maybe we don’t care if a metric goes up or down. It means we might have to admit that our metrics aren’t telling us anything and we need to switch. The dashboard just happens to be the starting point of all this discussion. This is how plans and decisions get made.
But, Randy, surely you’re not suggesting throwing up a hugely expensive meeting involving multiple stakeholders, just to justify the existence of your dashboard?
Why, yes. I am.
Because if such a dashboard were important enough to create and commit to maintaining, then it should be generating enough value that such a meeting is worth it. If we honestly, truly, think that a team needs to monitor the conversion rate of the sign-up funnel for users who are using the site from airplanes in flight, then we should set aside time to follow up on the effort. Otherwise we don’t probably don’t need to build the dashboard, and helps keep the overall number in check.
Obviously, this is an extreme example, but I’ve had a number of such meetings go pretty well. Eventually, when it becomes obvious the dashboard isn’t relevant anymore, you now have a clear signal to turn the dashboard down.
The other, quieter use case — dashboards as canned food
I can think of one other, less exciting, reason for letting certain dashboards exist. They act like magical preserved analyses for analysts to reference in the future. It’s like that can of baked beans you have in the back of the pantry, it’s probably good and useful to have, it might’ve gone bad over the past 3 years there, but it can save your butt in unexpected ways.
Even when no one else is ever going to put a dashboard to use, the analyst that created it is most likely going to remember that such a dashboard exists, and what questions it’s capable of answering. Then, when a relevant question pops up (as they inevitably do), the analyst will often be the first person to remember and pull out the dashboard for use.
Even if the dashboard has been broken by various data source changes, very often some bits of the methodology can be salvaged and updated to fit the modern systems. Even with having to fix things, it’s probably a bigger time save than having to redo an analysis from scratch.
Should we be in the business of making esoteric canned food? Not really. The general advice of refusing to make dashboards is typically correct.
But despite following the advice, pushing back, and trying to minimize dashboard creation, it’s inevitable that we wind up making a couple of throwaway dashboards every year. Whether it’s because there’s too much organizational pressure to make one, or because it actually seemed like a good idea at the time, we’re going to do it. I want to say that even those efforts have some value inside them. It’s not all complete waste.
So my suggestion is that whenever we do make a dashboard that falls by the wayside, we take a minute and intentionally package it up nicely for our future selves. It doesn’t take too much effort to stick a couple of comments in the background somewhere, just in case we ever revisit things.
It might not be gourmet tasty data, but it does get the job done.
About this newsletter
I’m Randy Au, currently a Quantitative UX researcher, former data analyst, and general-purpose data and tech nerd. The Counting Stuff newsletter is a weekly data/tech blog about the less-than-sexy aspects about data science, UX research and tech. With occasional excursions into other fun topics.
All photos/drawings used are taken/created by Randy unless otherwise noted.
Curated archive of evergreen posts can be found at randyau.com
Supporting this newsletter:
This newsletter is free, share it with your friends without guilt! But if you like the content and want to send some love, here’s some options:
Tweet me - Comments and questions are always welcome, they often inspire new posts
A small one-time donation at Ko-fi - Thanks to everyone who’s supported me!!! <3