A prospective client came to me, and they were very excited about building a new feature. They said to me, “Panashe, this is going to change everything for us. Nobody is doing this!”

“Great,” I replied. “What would success look like?” Silence.

See, they didn’t have any data on what their users engage with in the product. This means they had no way to quantify the impact of new features. In fact, they weren’t tracking their sales funnel, or their uptime, or how much they were spending per client. This was a data poor organization.

Sriram Narayan defines measurement debt thusly:

An organization takes on measurement debt when it implements initiatives without investing in the measurement infrastructure required to validate the benefits delivered by those initiatives.

This founder thought he had a technical debt problem. His inexperienced engineers were stuck, his codebase was poorly documented, and he needed someone senior to deliver the project.

But that wasn’t his biggest problem.

See, measurement debt is even worse than technical debt. While technical debt slows you down, measurement debt can make you go in the wrong direction entirely. Without measurement, how do you know that what you’re doing is having a positive impact on your product and your customers? You can refactor your code, but you’ll never recover the data that you didn’t measure. And the costs of tech debt are tangible and visceral - you can see it in your developers’ pain. Often measurement debt is hidden until you start measuring, and it’s impossible to reconstruct lost data.

The result of your measurement debt is that you’re flying blind. You’re unable to identify the true bottlenecks holding your business back. And if you’re seeking outside investment, it will be difficult to demonstrate what’s valuable about your product and your business.

The solution isn’t complicated, but it requires changing how you think about product development:

• Start with basic metrics before adding new features. This sets a good foundation to understand what works now, and what works in the future.
• Measure what matters to the business. Are your metrics ultimately tied to underlying business value?
• Build data collection into every new feature. New features give you the opportunity to understand a portion of your product from the outset.
• Create feedback loops with the data that you gather. For example, tracking user engagement and feature adoption on the launch of a feature, and feeding that into the decision of what feature to build next.

When was the last time you launched a feature without knowing how to measure its success? What would you do differently now? Hit reply and let me know, I’d love to hear from you.