Why statistics are both powerful and dangerous


Photo by Lite

Having dinner with a friend last night, one of the topics that came up was our shared obsession with statistics. It reminded me how looking at the wrong numbers has tripped me up. As an old teacher told me "You start off measuring what you value, and end up valuing what you measure".

As a concrete example, I have no idea how much I weigh. I stopped looking at scales years ago because  watching that number fluctuate day to day made me stressed out and demotivated. What I do watch is whether I can fit into my pants! That skips the stress of worrying about a few pounds, but warns me if I'm drifting from my normal range.

Another topical example is the stock market. I don't believe in stock-picking, so I'm purely invested with a low-fee S&P 500 index fund. I desperately try to avoid seeing the day-to-day level, because my primate brain will kick into flight mode when it drops and I'll be tempted to sell. As Behavior Gap explains very well, that's why individual investors sell low and buy high, getting dramatically worse returns than institutions. Again, that rapidly fluctuating number isn't really what I care about – it's how much I'll get years from now when I sell.

Once you start tracking a number, it becomes a priority. I'm fanatical about measuring user engagement with Mailana, because improving people's experience with the service is the only way I can make progress. I have a daily list of statistics for visits, bugs reported and twitter mentions, and seeing that every morning really helps me focus on what's important. These are good metrics because I actually care about exactly what they're measuring.

Another quote I remember is "Give me your daily routine and I'll tell you what your priorities are" : what you spend most time on is what you value most. It's the same for measurements; decide what's important to you and then pick the statistics. Don't just pick what's convenient or you'll find yourself making decisions that improve the numbers but destroy your business.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: