In any organization, informed decision-making relies heavily on the use of analytics. Analytics is the systematic analysis of metrics to uncover patterns, correlations, and insights that inform decision-making and strategy.. Years ago, I read a book (Working Knowledge) that defined intuition as compressed experience. This phrase was a revelation; it means that one can develop an intuition for numbers.
Despite having rather severe dyscalculia (dyslexia for numbers) I’ve built an instinct for metrics and learned to use data to navigate complex business problems. You don’t have to be a math major to do it, and it’s invaluable for setting OKRs. Creating effective Key Results requires knowledge of leading and lagging indicators, an ability to identify proxies for unknown metrics, and skill in balancing insights from qualitative and quantitative research. Here’s how I think about metrics and how you can develop a strong sense for them in your work.
Predicting Results and Learning from Outcomes
There is a distinct difference between metrics, analytics, and data:
- Data refers to raw, unprocessed facts and figures collected from various sources, which may include numbers, text, images, or other types of information.
- Metrics are specific, quantifiable measures derived from data, used to track and assess the performance, progress, or impact of certain activities or processes.
- Analytics involves the systematic analysis of data and metrics to uncover patterns, correlations, and insights that inform decision-making and strategy.
In essence, data is the raw input, metrics are the refined measurements, and analytics is the process of interpreting this information to derive meaningful insights.
The journey to mastering metrics begins with regular short-cycle collection of data and a practice of predictions. Predicting the correct answer to a question is a popular and effective instructional strategy for retention because it actively engages learners, making them process and internalize the material more deeply. Predicting metrics results provides a benchmark to compare actual results against, enabling you to identify patterns, refine your understanding, and improve future decisions.
Before launching any project or initiative, it’s crucial to make a guess about it will perform — even if it is a terrible guess. These predictions are your hypotheses. As the project progresses, you’ll gather data to compare against your predictions. The comparison of theory and reality is where the learning happens. By consistently predicting outcomes and reviewing the actual results, you start to understand patterns and nuances in your data, helping you refine your instincts over time.
Finding Proxies for Unknown Numbers
In an ideal world, we’d have all the data we need to make perfect decisions. However, we often deal with unknowns and incomplete information. This is where proxies come in. Proxies are indirect measures that stand in for the data we can’t directly obtain. For example, if you can’t measure customer satisfaction directly, you might look at repeat purchase rates or net promoter scores as proxies.
Developing the skill to identify and use proxies requires creativity and a deep understanding of your business. Over time, as you test and validate these proxies, your ability to make informed decisions in the face of uncertainty improves.
Leading vs. Lagging Indicators
A key aspect of effective metrics work is knowing the difference between leading and lagging indicators. Lagging indicators, such as sales revenue or customer satisfaction scores, reflect past performance. They’re important for understanding the outcomes of your efforts, but don’t offer insights into future performance.
Leading indicators, on the other hand, are metrics that predict future success. For instance, the number of leads generated or website traffic can be leading indicators of future sales. Identifying and monitoring leading indicators allows you to make proactive adjustments to your strategy before problems manifest in your lagging indicators.
Finding the right leading indicators requires experimentation and analysis. Start by identifying what activities or behaviors typically precede successful outcomes in your business. Test different metrics to see which ones most reliably predict your key results, and focus on these as your leading indicators.
Balancing Qualitative and Quantitative Research
Metrics aren’t just about numbers. Qualitative research plays a vital role in understanding the context behind the data. While quantitative metrics tell you what is happening, qualitative insights explain why it’s happening. Balancing both types of research ensures a more holistic understanding of your performance.
Quantitative research involves numerical data, statistical analysis, and measurable outcomes. It’s essential for tracking performance over time and identifying trends. However, relying solely on quantitative data can lead to missing the nuances of customer behavior.
Qualitative research, such as customer interviews, site visits, and open-ended survey questions, provides rich, detailed insights. These methods help you understand the motivations, emotions, and pain points of your customers. Use qualitative research to inform your quantitative metrics and vice versa. For example, if your quantitative data shows a drop in user engagement, qualitative research can help you uncover the reasons behind it.
In my experience, the best way to begin your understanding of qualitative research are the books Don’t Make Me think and Rocket Surgery Made Easy. They will teach you the why and how to test interfaces with users. For quanititative, no book beats Lean Analytics. But your best teacher will always be practicing with predictions.
Practical Steps to Develop Your Instinct for Metrics
- Start with Clear Objectives: Before diving into metrics, define what success looks like. Establish clear, measurable objectives that align with your business goals.
- Make Predictions: For every initiative, predict the outcomes you expect. Document these predictions and use them as a benchmark for evaluating actual results.
- Identify Proxies: When you can’t measure something directly, think creatively about what other metrics might serve as useful proxies. Test these proxies to ensure they correlate with your desired outcomes.
- Differentiate Indicators: Learn to distinguish between leading and lagging indicators. Focus on finding leading indicators that can help you make proactive adjustments.
- Integrate Research Methods: Use both quantitative and qualitative research to gain a comprehensive understanding of your performance. Allow insights from one method to inform the other.
- Review and Adjust: Regularly review your metrics and compare them against your predictions. Adjust your strategies based on what you learn, and continuously refine your understanding of what drives success.
Conclusion
Building an instinct for metrics is an ongoing process of prediction, measurement, and experimentation. By understanding the role of proxies, distinguishing between leading and lagging indicators, and balancing qualitative and quantitative research, you can develop a more nuanced and effective approach to metrics. This skill not only enhances decision-making but also drives continuous improvement and long-term success. Embrace the journey of learning from your data, and over time, you’ll cultivate a powerful intuition for what metrics matter most.