Previous Submission

Vanity Metrics: The Easiest Way to Fail Without Realizing It

Next Submission

We’re all guilty of it. Impressed by the sound of ten million registered users? 100k visitors per day? 100% month over month growth? Founders are trained to optimize for these numbers – to get traction at all costs. It sounds great to investors, it gives employees conviction, and it plays well at cocktail parties.

But these metrics don’t actually mean anything. They aren’t metrics that show the true health of the business. They’re easily gameable and simple to show on graph that’s up and to the right. At it’s most docile, it’s a storytelling device. At it’s most dangerous, it’s a recipe for failure.

They’re called vanity metrics for a reason. They make you look good, and if you’re not careful, you convince yourself that they matter. What’s more, once you start on the path of optimizing vanity metrics, it’s hard to turn back. You’ve told your investors that these stats are the most important to track – are you really going to stop investing in them and suffer a nasty hit?

I’ve seen several startups get caught up in this vicious cycle. I remember hearing a founder brag about how he had found the cheapest way to get a click. StumbleUpon Ads had $0.10 CPCs, and he could buy heaps of them to catch up on a bad month. “Gotta keep that hockey stick growth!” he said. The problem is, you couldn’t ask for a leakier bucket than StumbleUpon. I bet his bounce rate on those clicks were north of 90% and his retention on those visitors was even worse. It’s the equivalent of putting a bunch cash in a box and throwing a lit match into it. Even worse, it can create the illusion that you’re doing something right and make you ignore what’s actually important: building an awesome product that customers love and stick to.

This problem has gotten better as industry trends are correcting previous bad behavior. Visitors and app downloads have been replaced by monthly / daily active users. The best startups maniacally track their performance on these types of metrics, and optimize around the engagement, retention, and referral parts of the funnel. They understand that if you try to grow before you reach product market fit or before you have positive unit economics, you are just exacerbating a fundamental problem.

But startups that are still using these stats as a measure of success are walking a dangerous path. And without even realizing it, they may be heading straight to failure.

Previous Submission Failing at Hardware
Next Submission Wounded by Failure

3 thoughts on “Vanity Metrics: The Easiest Way to Fail Without Realizing It

  1. Eric, thanks for the post. I agree that optimizing for these vanity metrics could easily create a situation where the feedback mechanism of meeting or failing to meet KPI targets is ineffective. If a business is tracking the wrong metrics or failing to set appropriate targets (like in Dinr), it could waste significant resources by scaling before it has achieved product market fit. The challenge I see is not in physical products/services where unit economics provide hard data to point to, but rather in social media plays where the get big fast mantra has dominated the dialogue. In social media, scale often drives revenue. Accordingly, scale is very important to track, but is dangerous because you are right, the product-market-fit and health of the business are difficult to point to. These businesses could easily scale before they are ready because the relevant metrics of true business health are difficult to identify/hard to track in the early stages. Other than tracking user engagement (e.g., DAUs), any tips on how social media ventures should think about their metrics and target settings?

  2. Eric thank you for your post! Your point rings especially true after today’s case on Pixamo/HitBliss. The founders experienced ‘record’ growth within a few months and modeled financials after the success and apparent virality of their product. However, after a closer look at their customer acquisition strategy (essentially having users tag pictures of non-users, who would inevitably sign up in order to access photographs they are in), it was clear that the desired growth numbers would be fairly easy to hit-the bigger question and threat that could undermine the business was actually the competitive landscape. Pixamo’s competitive advantage was the tagging feature – although the growth in the use of the product may have indicated market potential, Facebook already had the customer base and could easily introduce the tagging features, essentially rendering Pixamo worthless. Sharon Peyer and Andrew Prihodko were lucky to have sold Pixamo, but they could have easily winded up in a situation where seemingly positive metrics could have dictated the decisions they took, when in fact, the basis of their decision should have been dictated by other factors.

  3. I absolutely agree. Even as people start looking into engagement, the metrics can tell a misleading story. For example, one of the metrics that we’ve been tracking is average time per session to get a sense of how engaged our users are. However, average time on page actually increased when our site load was heavy and load times increased. This would result in decreased conversion rates but a higher average time on page. The conversion rate is what actually matters for the business model, and time on page was just another vanity metric.

Leave a comment