Data-Driven vs. Data-Informed: Finding the Right Balance for Your Product Team
Data has undeniably emerged as a powerful force in today's product management practices, transforming the way organizations make decisions and strategize for growth. But with this newfound power comes a pressing need to understand the best ways to harness it, leading to a critical discussion around data-driven and data-informed approaches. (Yes yes, with great power comes great responsibility!)
In this blog post, we will explore these two distinct methodologies, shedding light on their differences, strengths, and potential pitfalls. By understanding the nuances of each approach, you can make more informed choices that ultimately benefit your organization and help it thrive.
Understanding the data-driven approach
A data-driven approach is defined as letting data guide your decision-making process. This approach is popular due to its ability to provide objective insights for making informed decisions. Let's explore its advantages and limitations.
Advantages of a data-driven approach
Objective decision-making: By relying on data, businesses can reduce the influence of personal biases and emotions in decision-making. This allows for more objective and rational decisions, which can lead to better results.
Identifying trends and patterns: A data-driven approach enables businesses to identify trends and patterns, which can help them make strategic decisions and stay ahead of the competition.
However, the two statements above aren’t entirely true. Here’s the thing - while data may provide you with hard, cold facts about trends and patterns, they don’t actually tell the whole story. You’re also not building a humanless, emotionless product! You’re building a product that will not only impact the daily lives of its users but also have an impact on their habits and behaviors. If you solely rely on numbers, how can you truly build a human-centered product?
Limitations of the data-driven approach
With that in mind, it’s important to understand the limitations of a data-driven approach.
Incomplete information: Relying solely on data can lead to decisions based on incomplete information, which may not yield the best results. Numbers can only give you a high-level insight into behaviors (just because I log into Twitter every day, doesn’t mean I am actively engaging with it anymore. The data says I am logging in, but if Elon took the time to ask me how I’m enjoying it, I’d probably give it quite a low score on that front.)
Risk of over-optimization: Focusing too much on data-driven decisions can lead to over-optimization of small aspects of the product while neglecting the bigger picture and overall vision.
Exploring a data-informed approach
A data-informed approach involves using data alongside qualitative insights to make decisions. By striking a balance between leveraging data and incorporating real human input, this method ensures a more well-rounded decision-making process.
Data-informed decision-making combines the best of both worlds—data and human insights. This approach recognizes that data is valuable, but also acknowledges the importance of experience, intuition, and judgment in making decisions. It goes beyond just the numbers, taking into consideration the human elements that contribute to the overall context and success of the product.
A data-informed approach takes into account the possibility of biases in data collection and analysis, as well as the context in which data is collected. This ensures a more comprehensive understanding of the situation, allowing for more effective decision-making. By considering multiple perspectives and sources of information, a data-informed approach can provide more nuanced insights that can better align with the company's goals and strategies.
I have heard many people say “This is just semantics, everybody knows this!” - and yet… not everybody does. However, the choice of terminology can have a significant impact on the behavior and understanding of stakeholders. By adopting the term "data-informed," product managers can create a culture that values both data and human input, leading to better-informed and more effective decision-making.
When comparing a data-driven approach to a data-informed one, it's essential to consider the unique needs and goals of your organization. While both approaches can provide valuable insights, a data-informed approach is often better suited to situations where qualitative factors and context play a significant role in decision-making. By integrating both data and human intuition into this process, organizations can create more holistic and effective strategies that consider a wide range of factors and perspectives. Remember, we aren’t building products for ourselves, but for others.
Case Studies: data-driven vs data-informed in practice
Facebook's "7 Friends in 10 Days" insight
Facebook's early growth team discovered that users who made 7 friends in their first 10 days were more likely to remain loyal to the platform. This data-driven insight was valuable, but it was not the whole story. The team's intuition and understanding of the product, as well as their ability to empathize with users, allowed them to design experiments, test hypotheses, and refine the user experience in a more informed way. By incorporating qualitative insights and their deep understanding of the product, the Facebook team was able to optimize the platform in a way that fostered user engagement and loyalty.
You can read more about it here.
Airbnb's transformation from a struggling startup to a billion-dollar business
In 2009, Airbnb was on the verge of failure, with a flat revenue of $200 per week and founders maxing out their credit cards. A non-scalable, non-technical solution turned the company's fortunes around and set the stage for Airbnb's massive growth.
While part of Y Combinator, co-founder Joe Gebbia and his team worked with Paul Graham to identify why Airbnb wasn't growing. Gebbia and his team initially believed they had to solve problems through scalable code. However, the success of the photo upgrade experiment showed them the importance of non-scalable solutions and meeting customers' needs in the real world. This customer-centric design approach, influenced by Gebbia's early design school experiences, became a core value for Airbnb.
Airbnb actively encourages employees to take measured risks and develop new features, even if the initial idea isn't data-driven. This "pirate" culture allows the company to move quickly and explore new opportunities. Airbnb also fosters an environment where employees learn to ship new features from day one, exemplified by the simple yet impactful change of the "star" function to a "heart" which increased engagement by over 30% (sounds crazy - but so it is!)
More about Airbnb here.
How to transition from data-driven to data-informed
Transitioning from a data-driven to a data-informed approach can provide your team with a more comprehensive perspective on decision-making.
Here are some steps to help you make this shift:
Encourage a culture of data literacy
Train your team: Invest in training and development programs to ensure that your team members are proficient in using data and understanding its limitations.
Promote discussions: Encourage team members to share insights from data and question assumptions during meetings and brainstorming sessions.
Combine qualitative and quantitative data
Gather customer feedback: Collect qualitative data from customer interviews, surveys, and support interactions to complement quantitative data.
Perform competitor analysis: Study your competitors' strategies, products, and marketing campaigns to gain a broader understanding of the market. (This does not mean copy them, but remain informed!)
Test and iterate
Formulate hypotheses: Use both intuition and data to develop hypotheses you can test.
Measure the results: Track the outcomes of your hypotheses and use data to validate or refute them.
Adjust and improve: Based on the data and insights you've gathered, refine your strategies and continue iterating on improvements.
Conclusion
The debate between data-driven and data-informed approaches highlights the importance of understanding the strengths and limitations of relying solely on data to make decisions. By adopting a data-informed approach, organizations can strike a balance between leveraging the power of data and incorporating valuable human insights. This holistic approach to decision-making allows businesses to create more effective strategies, optimize their products, and ultimately create growth in an increasingly data-centric world. Remember, the key to success in today's data-driven world is not merely the ability to crunch numbers but to thoughtfully integrate data with human intuition and experience to create products and services that resonate with users and meet their needs.
Andrea Saez
Read also
Experience the new way of doing product management
Experience the new way of doing product management