In this session, Dominik addresses some critical questions about using legacy data systems, the benefits of using data products, especially to the analysts, and how to consume these data products effectively with the right strategies.

Michael Toland chats with Chandan Gadodia, where they delve into the critical role of data product managers. They discuss how data product managers differ from IT and functional product managers, emphasizing their impact on product success. The episode highlights the evolving responsibilities in data management and the unique contributions of data product managers.

Jordi Puig explores how the Lean Startup principles can streamline data product development. By adapting concepts like the Value Hypothesis, MVP, and iterative design, Puig advocates for a minimal-effort approach to validate data product viability. He emphasizes the importance of checking if the product is used, valuable, and resource-feasible before heavy investment.

Karin Håkansson explores the idea of treating data as a social platform to address persistent issues in data governance, such as data quality, ownership, & consumer satisfaction. Karin suggests that creating a social platform for data can improve adoption, foster community, & incentivize teams to prioritize consumer needs and compliance standards. Karin explains the concept leaning on a LinkedIn analogy.

Alex Merced explores the significance of structuring data around two core elements—entities and events. Entities represent the current state of objects, such as "customer" or "product," while events capture actions over time, like "customer purchase" or "order shipped."

Jonas Dieckmann explores how GenAI can improve data quality—a critical foundation for effective analytics. Dieckmann outlines three key applications of GenAI: automating data harmonization, enabling dynamic data labeling, & generating synthetic data. By integrating GenAI into data pipelines through APIs, companies can automate quality improvements, and unlocking deeper insights.

Dr. Irina Steenbeek explores the complexities of harmonizing data and AI governance, focusing on whether integration is essential. She proposes O.R.A.N.G.E. as a framework for balancing data and AI governance, compliance, and risk.

In this article, Aurimas elaborates on how agent memory enables contextual decision-making, combining short-term memory for immediate tasks and long-term memory (episodic, semantic, procedural) for continuity and grounding.

Ugo Ciracì explores FinOps architecture for cost management in data platforms, focusing on establishing a reference model to allocate costs in data product-based systems. Ciracì proposes the Bounded Ownership Design Principle (BODP) to assign costs based on data, metadata, infrastructure, and software ownership.

Oscar Pulido introduces the Analytics Engineering Framework (AEF), a standardized design for building scalable, cost-effective data platforms on Google Cloud Platform (GCP). The AEF simplifies complex data operations by adopting a multi-repository approach, allowing independent pipeline development and seamless deployment.

Bart Vandekerckhove 🔐 and Charlotte Ledoux talk about the rise of Data Governance Engineering role, its responsibilities, team structures, and more. They talked about the need of data governance engineering and how data governance used to be a decade ago.

In the latest episode of Drill to Detail podcast, Host Mark Rittman interacts with Christian Steinert, Founder Steinert Analytics, a data analytics consultancy. They explore topics like understanding some the general business problems he deals with when helping a company.

Stephen Bailey explores the pervasive issue of clutter in digital environments, using a therapist's office as a metaphor. Bailey compares the accumulation of digital "stuff" — from data catalogs to wikis — to physical clutter, which hinders productivity and clarity.

Eric Sandosham, Ph.D. discusses the limitations of data democratisation and self-service in organizations. He argues that merely providing access to data does not equate to improved decision-making. Instead, he emphasizes the need for pre-processed information tailored to business users’ needs.

Lindsay Murphy indulges in an insightful conversation with Sol Rashidi, Head of Technology at Amazon's Startup Division. With over 25 years in the data industry, Sol shares her leadership journey, including roles at Estee Lauder, Merck, and IBM Watson. They explored the topics like: knowing your career's "shelf-life," the value of networking, and transitioning from technical roles to executive positions. Sol also introduces her new course, "Transition from a Tech Practitioner to an Executive," and her book Your AI Survival Guide.

Christopher Chin highlights the fusion of science and the humanities, showing how music's intuition and data's precision together foster human-centered insights. Drawing from his experience in tech, Christopher discusses the communication struggles faced by talented professionals and how he addresses this through his consultancy.

In this episode, David Savage sits down with Cindi Howson, Chief Data Strategy Officer at ThoughtSpot, to explore the transformative potential of Generative AI in maximizing return on investment.

Karthik Ravindran discusses the growing significance of data products in data transformation. He addresses the recurring question of why data products matter, introducing the CAP framework—Consumer centric, Audience oriented, and Purpose driven. By applying CAP, organizations can shape data as products, improving their value, durability, and accessibility.

Olivier Wulveryck explains how businesses can activate a "value flywheel effect" by integrating data into their strategic and operational models. Inspired by the concept from The Value Flywheel Effect book, Wulveryck outlines a four-phase framework for aligning data with business goals.

Dorian Drost discusses the use of path models, also known as Structural Equation Models (SEMs), to explore and test relationships between variables. Drost introduces key concepts like mediators and moderators and demonstrates how the Python package semopy can be used to model these relationships.

Eric Sandosham, Ph.D., explores the differences between "Data-First" and "Data-Driven" approaches. "Data-First" focuses on designing systems and processes with relevance and quality of data at their core, while "Data-Driven" emphasizes a cultural shift towards using data to inform decision-making. He highlights that while these approaches complement each other, they require strong organizational enablers, particularly data fluency, to be effective.

Zach Morris Wilson explores the transformative role of GenAI in data engineering, highlighting its impact on pipeline development, maintenance, and the evolving skillset of data engineers. He emphasizes how tools like ChatGPT and LLMs streamline code generation, data quality checks, and troubleshooting, reducing burnout.

Annie Phan explores the critical role of analytics translators in bridging the gap between data teams and business stakeholders. Annie emphasizes that simply being data literate isn't enough; professionals must also communicate insights in a way that aligns with business goals. Drawing on her experience as a business analyst, she highlights the importance of technical expertise, business acumen, and interpersonal skills in making data-driven decisions actionable.

Kristof Martens discusses how the traditional medallion architecture—categorizing data into bronze, silver, and gold tiers—may limit self-service data teams by imposing rigid structures. He suggests adopting a data product thinking approach, where teams independently manage and share data products with greater autonomy and flexibility. Martens introduces the open-source Data Product Portal as a solution to enable self-service.

Ergest Xheblati discusses the limitations of medallion architecture. He highlights how, despite its structured approach (bronze, silver, and gold layers), medallion architecture often leads to messy data pipelines, unreliable metrics, and high costs due to insufficient logical organization.

Learn why medallion architecture is not enough!

Barbara Galiza discusses 5 common marketing data issues: measurement, quality, accessibility, literacy, & activation. She breaks down the symptoms of each problem & offers potential solutions, emphasizing that different challenges require tailored approaches.

hirshanka Das recaps insights gained from the Open Source Data Summit on the evolution of data catalogs. Emphasizing open-source benefits—adaptability, community innovation, and transparency—Das advocates for open solutions to enable robust data management and governance across platforms.

Frannie Helforoush interviews Katy Pusch, focusing on her expertise in data product management, especially with decision-support products. Katy discusses how machine learning & analytics can empower stakeholders to make informed decisions.

Jon Su discusses the shift from using packaged analytics tools like Google Analytics & Adobe Analytics to a warehouse-first approach. This approach sends behavioral data directly to a company’s internal data warehouse (e.g., Snowflake, BigQuery) rather than third-party platforms.

Zach Wilson examines common pitfalls in data modeling that can hinder effective analytics. He highlights issues such as inadequate logging practices, failure to achieve 3rd Normal Form in production environments, schema mismatches between production & analytical systems, delivering based on requirements rather than actual needs, & constructing point solutions that lack scalability.

In this session, you'll learn to identify common data issues such as duplicates and inconsistencies, implement the COAT method for normalization and classification, grasp the importance of data quality for AI success, and explore Susan's upcoming data framework.

This article highlights how dirty data, characterized by inaccuracies and inconsistencies, significantly hampers decision-making in organizations & to combat this teams should employ the COAT framework through practices like supplier normalization.

In Episode 251 of this Podcast, Aurimas shares insights into LLM ops and its challenges and the importance of branching experiments and fine-tuning models to further scale the outcomes of LLM ops.

Robert S. Seiner proposes "Data Investor" as a more dynamic alternative to "Data Owner" in governance. Unlike Data Owners, who focus on safeguarding data, Data Investors actively engage with data to maximize its value and ROI.

Anne Nasato, P.Eng draws parallels between data engineering platforms & refineries, highlighting how both prioritize backend processes over end-user experience. However, the rise of generative AI & customer-centric products is shifting this focus, bringing users closer to the data lifecycle.

Dan Miller highlights Airbnb's internal platform, Sandcastle, which enables data scientists, engineers, & product managers to quickly prototype & share data/AI ideas as web applications. The platform overcomes challenges in showcasing interactive AI-powered concepts to non-technical stakeholders.

Brett A. Hurt emhasizes on "bottoms-up" data governance, inspired by Agile Development, for faster ROI. He argues that traditional top-down frameworks stifle innovation, while Agile Data Governance focuses on quick wins, collaboration, automation, and aligning with business goals.

Jens Linden, PhD highlights the importance of data strategy for organizations to become data-driven. He discusses the challenges of implementing data strategy and introduces the "Playing to Win" framework as a suitable approach. The framework includes tools like the Strategy Process Map and Strategy Choice Cascade to guide the strategy design process.

In this podcast episode, Susan dive into her unexpected journey into the data world, starting with a spend analytics job, and how that led to her founding her own business focused on dirty data.

Peter Baumann discusses the importance of defining a vision for Data & Analytics in organizations. He explains that while many companies are not yet data-driven, creating a clear vision—tailored to their specific needs and business goals—provides direction like a "north star" for future decisions. He emphasizes that a vision isn't a measurable goal but a long-term target that guides the organization's use of data.

In this article, Cindi expains how early adopters of Gen AI will get a competitive advantage in analytics over others.

A strong data strategy is key for start-ups and scale-ups to boost valuation and ROI by validating market fit, optimizing operations, fostering innovation, and managing risks—all of which make them more attractive to investors and set the stage for sustainable growth.

Tim Wilson explores the recurring challenges faced by analytics teams and business users, based on responses from a survey conducted by facts and feeling. Key issues include unclear goals leading to ineffective reporting, messy data, and the failure of self-service analytics platforms. Analysts struggle with explaining data complexity and ensuring actionable insights, while business users face difficulties accessing data. Both groups share frustration with data quality, though analysts perceive it as a greater issue.

Dr. Irina Steenbeek discusses the concept of critical data elements, tracing its evolution from various experts and its practical application in data management. She talks about the definition, objectives, and challenges of critical data, emphasizing the need for prioritization in data-related initiatives and more! Dr. Irina explores data management capabilities that contribute to or benefit from critical data, along with a high-level methodology for implementation.

In this discussion, Madison talks about the skills that aspiring analytics engineers should prioritize and shares insights into the importance of networking and job search, along with tips on interview processes.

Avoiding Compounding Costs, Multiplying Impact of AI Initiatives, Mitigating Fatal or Hidden Risks, and Always Putting Data First 🥇

Vu Trinh details Discord's evolution from using in-house data orchestration solutions to adopting modern tools like Dagster and dbt. Initially, Discord used their internal system, Derived, to manage complex data transformations. However, as their needs grew, they found limitations in Derived's flexibility.

Harish Govindaraju discusses the challenges data organizations face in transitioning from delivering traditional data outputs to creating true data products. He emphasizes that a data product includes not just data but also essential metadata ensuring usability, reliability, and governance. He also explores the importance of clear ownership, data quality, lineage, security, and interoperability.

In this interview, David discusses AI’s evolving role in healthcare, highlighting its potential for improved diagnosis and treatment, enhanced clinician efficiency, and its ability to address staff shortages and cut costs through human-AI collaboration.

Robert S. Seiner emphasizes the critical importance of understanding and managing organizational data through a comprehensive Data Asset Inventory. This inventory is more than just a list; it’s a detailed catalog that includes internal and external data, metadata, and contextual information.

In this episode, Michele talks about her experiences of using the ABC Framework and shares strategies to help product marketers understand the importance of identifying market reaction, clearly communicate with customers and offer customised solution like custom pricing to reduce churn and demonstrate commitment towards cusotmer satisfaction during price adjustments.

Brittany discusses the recurring issue of companies frequently switching Business Intelligence (BI) tools without addressing the root causes of their data challenges. She argues that the high costs and complexities involved in implementing new BI platforms often don't yield long-term benefits, as the core problem lies in the inability of these tools to scale with evolving data models.

Ananth Packkildurai provides a comprehensive guide to evaluating Change Data Capture (CDC) tools, which are crucial for real-time data integration and analytics in modern data architectures. He emphasizes CDC's relevance in unified data environments, its role in modernizing legacy systems, and its importance for real-time business demands.

Marc Delbaere explores the integration of data lineage and data value to maximize business impact. He draws a parallel to the Japanese term "Tsundoku" to highlight the wasted potential of unused data catalogues. Delbaere introduces "Data Value Lineage" as a process of ensuring that all data activities are linked to business value.

Pavel Samsonov critiques Nike's shift to a data-driven strategy, orchestrated by CEO John Donahue on McKinsey's advice, which led to a $25B market cap loss. This approach favored easily measurable data, ignoring the qualitative insights critical for effective marketing. Samsonov argues that relying solely on quantitative data can obscure important nuances, leading to misguided decisions.

This blog differentiates between a Data Governance Clinic, an intensive workshop offering personalized advice, and a Data Governance Mastermind, a peer-to-peer session for ongoing support and collaborative problem-solving to help audiences understand what they need.

In his latest blog, Seattle Data Guy emphasizes the importance of foundational skills beyond specific technologies in the data field. He highlights the value of thinking in systems, developing data intuition, fostering team growth, and maintaining constant curiosity.

Torsten Walbaum explores how perfectionism can hinder careers in high-growth companies, particularly for Data Scientists. He argues that delivering results quickly is often more valuable than striving for perfection, as businesses prioritize speed over flawless work.

Bernd Wessely challenges the current practices in data engineering, highlighting confusion over its definition and the problematic integration of business logic. He argues that data engineering should focus solely on the movement, manipulation, and management of data, leaving business logic to application developers.

Tobias Macey, in conversation with Tom Baeyens, discussed the importance of data contracts in ensuring data reliability in the latest episode of Data Engineering Podcast. They compare data observability, which is reactive and focuses on diagnosing issues, with data testing, which is preventive and ensures data quality across handover points.

Mikkel Dengsøe explores the composition of data teams across 40 top companies in the US and Europe. He emphasizes that there’s no universal ideal ratio for data roles, as different companies have distinct needs and structures.

Kris Peeters discusses the complexity of modern data platforms and the challenges of using a "bag of tools" approach, which requires developers to master multiple tools and scripts. He advocates for creating a unified interface that simplifies the user experience, reduces cognitive load, and supports diverse use cases, such as data warehousing and data science.

Host Richie Cotton discusses with Gunther Hagleitner how large language models (LLMs) can significantly boost data teams' productivity through LLM-assisted coding. The conversation focuses on the technology, talent, and processes needed to harness these productivity gains. Key insights include adjusting workflows to integrate LLMs and understanding the evolving skillsets required for data practitioners in this new landscape.

This article talks about key trends like the rise of real-time data processing, increased adoption of data warehouses and lakes, the importance of data observability for pipeline reliability, and a shift toward modular and reusable data architectures.

Barr Moses says that many data quality initiatives fail due to poor incentive structures and lack of focus on what truly matters. Effective data qulaity scorecards need to prioritize critical data, standardize quality measurement, and incentivize both producers and consumers. Successful initiatives ensure both data producers and consumers trust and adopt the data provided, leveraging modern tools like data observability solutions and data catalogs to streamline and sustain these efforts.

Sandeep Bose discusses the pitfalls of traditional data lake strategies, hilighting their high costs, long timelines, and fragmented data organization. Instead of achieving a unified data repository, many enterprises end up with hard-to-navigate data silos. Bose argues for a shift towards domain-driven data architectures and modern technologies like Generative AI (GenAI) and Machine Learning (ML).

Dan McKone emphasizes the importance of using data and analytics not just to support but to directly derive business strategy, especially during critical decision-making moments. Despite the widespread availability of data, many leaders rely too much on intuition, leading to less effective decisions. He highlights examples like HP's strategic split, Spotify's subscription model, and Starbucks' location strategy as successes of data-driven decision-making.

In the latest episode of the Data Pancakes, the host explores the complexities of defining data products due to varying interpretation and lack of standardization. He discusses three perspectives on data products: business, technical, and user.

In this session Susan offers insights on data quality, including her COAT methodology and the Dirty Data Maturity Model, along with tips for data cleansing, common management mistakes, and her vision for future data tools. She also offers advice for aspiring women in the field based on her experiences.

Eric Sandosham advises smaller organizations on building data analytics capabilities effectively. Instead of focusing on efficiency, they should tackle flawed decision-making by examining assumptions about revenue, risk, and customer experience. The recommended approach is to start with Diagnostic Analytics to identify data gaps, then move to Predictive Analytics for tool standardization, and finally to Descriptive Analytics to enhance data use and storytelling.

Cassie Kozyrkov discusses teh evolving role of data scientists in the age of AI, emphasizing their critical importance in enterprise-scale automation. She argues that while AI tools evolve, the foundational skills of data scientists remain essential for ensuring effective, safe and reliable AI systems.

Learn if your data team really need gen AI to begin with!

A simple and jargon-free framework for data modeling.

Dylan Anderson emphasizes the critical importance of data modelling in today's rush towards new technologies. He argues that neglecting data modelling leads to short-term fixes and inefficiencies. Data modelling, spanning conceptual, logical, and physical phases, aligns business processes with data structures, enabling better decision-making and operational efficiency.

In the latest Data Engineering podcast episode, host Tobias Macey discussed the evolution and challenges of product management for data platforms with Praveen Gujar, LinkedIn’s Director of Product. They explored the importance of clean, reliable data and the role of product managers in bridging engineering and business teams.

The article highlights the confusion caused by the rise of multiple C-level technology roles and how some organizations are consolidating them into "SuperTech" leadership positions, typically held by CIOs, to streamline data and tech management, improve clarity, and drive digital transformation.

In this interview, Tom Davenport reflects on the evolution of business process management (BPM) from its rise in the 1990s to its current state amidst new technologies like AI and process mining, advocating for a blend of traditional process thinking with innovative tools to achieve business value.

Wannes Rosiers explores the distinction between data catalogs & data product catalogs within the context of modern data management. He highlights that while traditional data catalogs focus on data structure and definitions, data product catalogs expand this concept to include the entire user experience and functionality.

In this interesting read, Nicola talks about focusing on data governance strategies and how to align them with your organisation’s strategies.

Sarah Levy emphasizes the necessity for data teams to approach AI-driven analytics cautiously. With the launch of Tableau’s Einstein Copilot, the integration of generative AI promises enhanced data exploration. Sarah highlights that effective data modelling and governance are crucial for leveraging AI in analytics.

In this blog, Ananth Packkildurai explores the modern data stack's origins, rise, and fall. Initially emerging from various innovations and industry shifts, tools like Apache Yarn and S3 paved the way for scalable data processing.

Wasteman discusses best practices for creating accurate and reliable software-based financial systems. He covers fundamental financial definitions, goals of accounting systems, engineering principles, and best practices (use of integers for financial amounts, granularity for currency conversions, consistent rounding methodologies).

In this newsletter Susan shares highlights that the key to successful digital transformation is prioritizing people—engage users early, align technology with their needs, and plan for future data requirements to avoid resistance and costly failures.

This article explores how analytics engineering teams empower start-ups and scale-ups to make data-driven decisions, automate processes, and fuel sustainable growth.

Aurimas Griciūnas discusses observability in the LLMOps platform and pipeline, focusing on areas that benefit from infrastructure scale.

Théo Tortorici discusses how combining a semantic layer with AI data scientists facilitates easy data adoption. A semantic layer abstracts complex data into user-friendly terms, ensuring consistent metrics across an organization. He addresses business problems like consistent reporting, & reduced dependency on data teams.

The Seattle Data Guy emphasizes the challenges new data leaders face due to a lack of coaching and guidance. His blog offers advice on becoming successful in data leadership roles, such as understanding the business, communicating in business terms, and ensuring data quality.

In the latest episode of the Dataframed podcast, Shuang Li, Group Product Manager at Box, shared insights on building a scalable data platform with the host Adel Nehme. Shuang talked about cloud migration challenges, data ingestion, and processing pipelines.

Théo Tortorici discusses the benefits of integrating a semantic layer with AI data assistants for easy data adoption. He resolves business problems like inconsistent reporting and reliance on data teams, and technical issues such as data complexity and scalability.

Arup Nanda discusses various approaches to structuring data architecture teams based on evolving data landscapes. He contrasts traditional monolithic database designs with modern decoupled architectures like microservices and streaming systems.

Steven Macleod joins Joe to discuss the importance of operational data modeling, the flaws in current data modeling practices, and the challenges of data modeling at startups and in 3rd party tools.

Arslan Shahid guides through building an AI-powered “Auto-Analyst” system designed to assist data scientists and analysts by automating data analysis tasks. Arslan details the design and functionality of various agent, statistical analytics agent, machine learning agent, and code combiner agent.

Joe Reis compares the evolution of mixed martial arts (MMA) to the field of data modelling. He argues that MMA fighters must be proficient in multiple fighting styles, and data professionals must master various data modelling techniques.

Learn the two macro trends are forcing data modeling into a multi-disciplinary practice.

Daniel Lambert, M.Sc. discusses the critical importance of modern data architecture for successful AI projects. He highlights the exponential growth of data, necessitating agile and flexible architectures capable of handling vast amounts of information from diverse sources.

Mikkel Dengsøe discusses the critical role of data observability across all data pipeline stages. He emphasizes real-time issue detection in data ingestion, storage, transformation, and analysis to uphold data quality and reliability.

Hugo Lu draws attention to the emerging cracks in the Modern Data Stack (MDS), highlighting three key issues. Firstly, there’s a shift towards all-in-one data platforms like Databricks and Snowflake. Secondly, there’s a decline in support quality and increasing service bugs among MDS providers. Thirdly, competitive pressures are intensifying as major players like Snowflake encroach on their partners’ territories

Pedram Navid discusses the evolution of data roles, emphasizing the shift from data scientists to data engineers and now to data platform engineers. Initially, data scientists were key; then, data engineers emerged to handle data ingestion and transformation. Today, data platform engineers focus on creating frameworks allowing others to build custom pipelines.

Get a detailed perspective on the reasons to invest into semantic data layer with Olga.