Here's your Playbook
Download now
Oops! Something went wrong while submitting the form.
Learn More
TABLE OF CONTENT
This piece is a community contribution from Ryan Duffy, a Data Strategy Leader with over a decade of experience! He has been part of the inner workings of what it takes to effectively deliver enterprise data, including establishing the Chief Data Offices for some of the largest financial institutions – some of which are probably represented in your wallet right now. We’re thrilled to feature Ryan’s unique insights as a Strategy Leader on Modern Data 101!
We actively collaborate with data experts to bring the best resources to a 9000+ strong community of data practitioners. If you have something to say on Modern Data practices & innovations, feel free to reach out! Note: All submissions are vetted for quality & relevance. We keep it information-first and do not support any promotions, paid or otherwise!
Feature Your Ideas!
Introduction
The subject of data products continues to gain traction across the industry, with some sectors leading the way while some are stepping back to learn from other’s missteps. "Data Product" has become a ubiquitous term, yet the definition varies significantly across the industry.
While numerous perspectives exist on what constitutes a data product, this article presents a practical framework for viewing data products across a spectrum from foundational to analytical methods.
In my experience as a consultant, conversations quickly get into the weeds, so I've aimed to frame the concepts of data products and the ways they can be measured to drive value into a simple construct that can then be applied and expanded further.
It is important to remember that the goal of a data product organization is not only to build data products but also to measure, manage, and understand...
…how their Data Product Portfolio is being used to achieve top-line business goals.
The Data Product Spectrum
Foundational Data Products
At the base of the data product spectrum are foundational data products - the bedrock of enterprise data architecture. These domain-oriented products serve as the authoritative source of master and reference data.
Think of a financial institution's customer master data product containing the golden source of customer information or a product master maintaining definitive product specifications and hierarchies.
These foundational products are made available through data marketplaces (i.e., internal platforms that enable controlled discovery, access, and distribution of data products across the organization), enabling broad enterprise consumption while maintaining data governance standards (i.e., the policy and procedures used as organizational rules and processes for data management).
The foundational products are often the first to be created as they are more discrete, and many other products and services use them as clean, trusted data sources.
Note: While I am using the term “Foundational Data Product” this has also been referenced using similar terms within the industry such as “Source Data Products” or “Entity Data Products” as cited by other community contributors (e.g., referenced in articles [1], [2], [3])
Integrated Data Products
Moving along the spectrum, the next bucket of data products can be described as integrated and combine data across domains to serve specific business needs.
For instance, a "Customer 360" data product might merge customer master data with transaction history, product holdings, and interaction data.
These products are purpose-built but maintain flexibility for various use cases. They serve as building blocks for more sophisticated analytical solutions while providing immediate value through their integrated view of cross-domain datasets.
Note: Similar to multiple terms referencing foundational data products, “Integrated Data Products” have also been referenced using similar terms within the industry such as “Aggregated Data Products”, “Derived Data Products” or “Business Data Products” as cited by other community contributors (e.g., referenced as Aggregated here.)
Analytical Data Products
At the far end of the spectrum are analytical data products. These are less ‘data as a product’ type dataset products, but rather purely consumable assets that are designed to drive action with specific insights, answer questions, or solve use cases using the other data products. These take various forms:
- Dashboards and reporting tools designed for a specific business purpose
- Predictive models (e.g., credit risk scoring engines)
- Automated decision systems (e.g., trade execution algorithms)
- AI/ML solutions (e.g., fraud detection systems)
Note: As with other data products, other Modern Data 101 contributors have offered insights on this topic as well. Learn more here!
With this understanding of the data product spectrum, organizations need to evaluate and improve their data products systematically in a way that maintains high value and relevance.
Having data products for the sake of having data products is a losing strategy. The following proposed measurement framework provides the structure needed to assess and enhance data products across all levels of the spectrum.
Measuring Data Products: A Comprehensive Framework
“Do we have a consistent way to measure and identify tangible business value?”
Inventorying data products, classifying them by domain or business, and knowing counts and distribution is a great start to begin managing the portfolio, but measuring data products isn't just about tracking metrics—it's about driving and understanding value. While organizations continue to gain traction in building data products, particularly on the analytical side of the spectrum, many often struggle to measure their effectiveness systematically.
In nearly every organization I've been part of or consulted for, countless dashboards and curated data sets exist in varying stages of maturity, use, and age. The exponential growth is a result of treating data solutions as 'projects' rather than 'products' that continually evolve to meet new demands and maintain relevancy.
📝 Related Reads
Building your Sausage Machine for Data Products 🌭: Less Tech, More Strategy | Issue #37
This challenge stems from the complexity of tracking not just technical performance but also business value and user satisfaction across different types of products along the spectrum. Consistent measures must be in place to monitor and evolve your data product portfolio.
A comprehensive measurement framework serves multiple purposes.
- Provides transparency into product performance and value delivery.
- Enables data product teams to identify improvement opportunities and prioritize enhancements.
- Helps organizations make informed decisions about product lifecycle management, from investment in new features to retirement of outdated products.
To create an effective measurement system, organizations need to think holistically about their data products. This means moving beyond simple usage statistics - which, again, is a great start and better than nothing - to understand the full picture of product health, adoption, and performance.
By integrating these measurements into a comprehensive scorecard, organizations can assess their data product portfolio at multiple levels moving from individual products to domain-specific collections to enterprise-wide analytics.
A scorecard approach enables teams to identify under-performing products quickly, celebrate successes, and make data-driven decisions about product investments.
For instance, a data domain owner might discover that while their foundational data products have high-quality scores, adoption rates are low due to poor documentation or accessibility issues. Similarly, an enterprise view might reveal overlapping products across domains, presenting opportunities for consolidation and cost savings.
📝 Related Reads
Metrics-Focused Data Strategy with Model-First Data Products | Issue #48
The Go-To Metrics for Data Products: Start with these Few | Issue #12
How does a Data Product Strategy Impact the Day-to-Days of Your CMO, CDO, or CFO | Issue #34
Data Product Health
"Is the product trusted and described correctly?"
The cornerstone of any successful data product or program lies in its trustworthiness and clarity of purpose. This measurement bucket focuses on the fundamental quality and documentation aspects that determine whether a data product can be reliably used as a source of truth.
By monitoring health metrics, organizations can ensure their data products maintain the highest standards of data integrity and usability.
Quantitative Measures
- Data Quality Score: Composite measure of completeness, accuracy, and timeliness.
Example: A customer master data product maintaining a 98% quality score based on required field completion and address validation. - Metadata Compliance Rate: Percentage adherence to metadata standards.
Example: 95% compliance with required business glossary terms and data lineage documentation.
Qualitative Measures
- Subject Matter Expert Reviews: Regular assessments or attestations by domain experts, product owners, and active consumers.
- Documentation Completeness: Evaluation of supporting materials and guides.
Adoption & Usage
"Are customers using the product, and is it relevant broadly?"
While having a healthy data product is essential, its true value can only be realized through active use across the organization. Imagine having a sportscar but keeping it in the garage and walking to work. Having a pristine product is useless if it stays on the shelf.
This bucket examines how widely and effectively the data product is being utilized, helping teams understand whether their product meets real business needs and identify opportunities for expansion or improvement.
Quantitative Measures
- Active User Count: Number of unique users/applications accessing the product Example: A risk data product supporting 15 different applications across risk, finance, and regulatory reporting.
- Usage Frequency: Patterns of access and consumption Example: Daily access patterns showing consistent workday usage with peaks during month-end closing.
Qualitative Measures
- Use Case Coverage: Documentation of supported business processes and use cases across the organization.
- User Feedback Sessions: Structured feedback on product utility and gaps.
📝 Related Reads
How Data Developer Platforms enable data product adoption and development tracks in parallel.
Building Data Platforms: The Mistake Organisations Make | Issue #54
Performance & Reliability
"Is the product achieving set business goals?"
The technical excellence of a data product directly impacts its ability to deliver business value. This measurement category evaluates both the technical performance and business impact of the data product, ensuring it meets service-level agreements (SLA) while delivering tangible benefits to the organization.
Quantitative Measures
- System Performance: Response times and availability metrics Example: 99.9% availability with sub-second query response times
- Business Impact Metrics: Revenue generated or costs saved Example: $2M annual savings from automated credit decisioning
Qualitative Measures
- Customer Satisfaction Score (CSAT)
- Net Promoter Score (NPS)
Implementation Guidance & Considerations
The Cultural Shift: Beyond Technology
While the technical aspects of data products are crucial, equally important is the cultural transformation required to succeed in this space. Many organizations have historically approached data management from a purely technical perspective, focusing on delivery for specific use cases rather than thinking about data as a product that serves multiple stakeholders across the enterprise.
This shift from technical data delivery to product thinking requires fundamental changes in how organizations approach data management. It demands new roles, new processes, and most importantly, a new mindset.
Data Product Owners must think like traditional product managers, considering user experience, market demand, and competitive advantage. Data engineers need to think beyond pipelines to consider how their work enables broader business value. Business analysts must expand their view from individual reports to understanding how their requirements fit into the larger data product ecosystem.
The challenge isn't just in building and measuring data products—it's in transforming how organizations think about and interact with data. While many organizations talk about treating data as a product, few have successfully made this cultural shift. Truly embracing data products and shifting from concept to reality requires:
- Leadership commitment to product thinking and delivery
- Investment in product management capabilities and resources
- Cross-functional collaboration between technical and business teams
- User-centric design approaches
- Continuous feedback loops with stakeholders on usage and value
- Clear ownership and accountability structures over data products and path to efficiently resolve issues when they arise
Data Product Creation & Maintenance Risks
The proliferation of data products presents a significant challenge for organizations. Just as dashboard sprawl has plagued BI teams, unchecked data product growth can lead to maintenance overhead and reduced value.
Key Risks
- Product Proliferation: Redundant or overlapping products
- Technical Debt: Aging technology and architectural misalignment
- Value Degradation: Declining relevance and usage
Leading Practices for Sustainable Management
- Regular Value Assessments via ongoing feedback loops, quarterly reviews and attestations, and usage pattern analysis.
- Product portfolio rationalization via identifying product consolidation opportunities, clear product retirement criteria, and processes.
- Data product governance framework with clear ownership and accountability over the full data product lifecycle.
Next Steps to Measure Your Own Data Products
Remember, the journey to effective data product management is as much about cultural transformation as it is about technical implementation. By understanding your products' position on the spectrum and maintaining rigorous measurement practices, you can ensure your data product portfolio continues to drive business value while avoiding the pitfalls of unchecked growth.
So, I have a data product, now what?
In addition to the categorization, frameworks, and example metrics we've outlined, here are 3 immediate next steps to accelerate your data product journey and drive sustained value.
- Map your current data products along the spectrum
- Implement the measurement framework across all product types and tailor it to your specific business needs
- Establish regular review cycles for data products aligned to the end-to-end lifecycle management
By applying the principles and measurements outlined in this article, organizations can begin shifting from traditional data delivery to true product thinking, ultimately driving greater value from their data assets.
Thanks for reading Modern Data 101! Subscribe for free to receive new posts and support our work.
MD101 Support ☎️
If you have any queries about the piece, feel free to connect with the author(s). Or feel free to connect with the MD101 team directly at community@moderndata101.com 🧡
Author Connect 🖋️
Find me on LinkedIn 🙌🏻
From The MD101 Team
Bonus for Sticking With Us to the End!
🧡 The Data Product Playbook
Here’s your own copy of the Actionable Data Product Playbook. With 1000+ downloads so far and quality feedback, we are thrilled with the response to this 6-week guide we’ve built with industry experts and practitioners. Stay tuned on moderndata101.com for more actionable resources from us!
