Get weekly insights on modern data delivered to your inbox, straight from our hand-picked curations!
The Data Product precipice sits right at the intersection of Approach and Technology. Just like particles have a dual nature: wave and matter.
With the product mindset, organizations are focusing on understanding the value of treating data as a product for improved outcomes. This article walks through the fundamental anatomy and lifecycle of data products, justifying it as an approach and a technology to assess how this product mindset in data space helps data and metric-driven orgs address their business objectives/purposes.
An approach is a PROCESS- a way to do things. Data Products establish that process through the Data Product Lifecycle.
One might argue Data Products is more an approach than technology, much like Agile for Software. Agile went through its share of pushback from sticky processes and emerged as a must-have. Post-adoption, Agile became a game changer for software teams and products.
Interestingly, the Data Product approach has changed that. It works for the core indefinite element of any data stack absent in the software stack: Data. And it needs to be actively enforced through a Data Product Platform that integrates with existing data stacks.
Agile tools, the likes of JIRA and Asana, enable Agile Software Development, but establishing a cyclic and accountable cycle for data development is slightly more challenging since the main ingredient, data, is volatile and subject to constant change.
Data Developer Platforms (a data product infrastructure standard) solves this problem by enabling the Data Product Approach for data development. It establishes a repetitive and reliable cycle despite the dynamic nature of data.
Technology refers to a TANGIBLE MACHINE/mechanism. Data Products are tangible products that users can “purchase” and use. The lifecycle or approach discussed above is enabled through this tangible product - the product’s structure or anatomy.
The benefit of the data product lies in its anatomy itself. The secret is tightly coupling data with infrastructure, code, and metadata instead of loosely processing and projecting data for various applications.
How does that help?
When data freely floats across your infra, there’s hardly any scope for reproducible cycles for incremental evolution.
Evolving lifecycles are the best bet to optimize the value of data that is intrinsically dynamic. Coupling data with infrastructure, code, and metadata allows it. The tight coupling fosters quick changes while maintaining high stability and interoperability with a plethora of tools, sources, and consumption endpoints that the data stack necessitates.
Every Data Product serves a specific purpose or drives a set of business metrics which can be organized as a metric tree.
We explain this ability through a compact metric tree model for the office of CDO and CMO, where every low-level to high-level metric across tech and business layers is interconnected and traceable due to the anatomy of the data product (data + metadata + code + infra).
As a consequence, decision-makers can roll out informed decisions across solutions, operations, and development tracks without having to know the ins and outs of every vertical - only the factors that move the needle.
As a consequence, decision-makers can roll out informed decisions across solutions, operations, and development tracks without having to know the ins and outs of every vertical - only the factors that move the needle 🪡
If you’re looking for more ideas and insights on similar lines, here’s an interesting piece that talks about an agile-like approach specifically in the context of data instead of a direct translation of Agile in the context of software. The article also delves into a high-level strategy on essentials you must acquire to get going.
Erik Lenhard, partner and director at BCG, also talks about the necessity of this convergence between Agile and Data tracks through Data Mesh (or data products as core building blocks).
The article is crisp and highlights high-value points directly without beating around the bush. An excerpt that’s a personal favourite:
Once again, thanks for sticking to the end! Here’s a bonus meme 🤡
datadeveloperplatform.org - a community influenced repo specifying the fundamentals of a data platform infrastructure standard, ideal for data product development.
