Get weekly insights on modern data delivered to your inbox, straight from our hand-picked curations!
‍
If there is one thing they all agree on, it's the importance and power of Conway's Law. Important enough to affect every system I've come across, and powerful enough that you're doomed to defeat if you try to fight it.
The law is probably best stated, by its author, as: Any organization that designs a system (defined broadly) will produce a design whose structure is a copy of the organization's communication structure.-- Melvin Conway [1]
Conway's Law is essentially the observation that software architectures look remarkably similar to the organization of the dev team that built it. It was originally described to me by saying that if a single team writes a compiler, it will be a one-pass compiler, but if the team is divided into two, then it will be a two-pass compiler. Although we usually discuss it with respect to software, the observation applies broadly to systems in general.
Conway’s law also applies to designing data products, and it is largely harmful to try and fight it. It is, therefore, essential to note that every organisation might have a different design approach that best serves their interests and matches their communication patterns. Consequently, the design phase is spread across a wide band of combinations, except for a few common core functions.
It goes without saying that to enable a broad spectrum of design operations, which is also standardised, you need a flexible modular infrastructure - much like an AWS for data products - but with DataInfra-specific resources. Excellent platform developers are coming together to build the Data Developer Platform Standard that outlines some of these standard resources.
‍
Subtly introducing the concept of “Data Design Thinking” and the need for a deep understanding of data, user competence, and domain expertise in creating efficient data products for professional environments, Christoph Nieberding, Co-Founder, Chart Doktor-Data Product School, suggests ’Design Thinking’ as an effective approach to devising impactful solutions to tricky problems. He says Design Thinking revolves around discovering the environment, defining user needs, developing ideas, and delivering solutions while considering user feedback.
Zach Gemignani, Chris Gemignani, Dr. Richard Galentino, and Dr. Patrick Schuermann extensively discussed the principles and concepts of designing data products in their paper, Data Fluency: Empowering Your Organization with Effective Data Communication. In Appendix A, they specifically talked about Designing Data Products and curated a checklist of useful resources. A must-read for Design practitioners aiming to solve complex problems.
‍
‍
Let’s face it, we data engineers are building Data Products for our business partners. It is they who are supposed to derive immensely valuable insights using Data Products. But are these business personas technically as sound as data engineers? Are we building something that’s user-centric and user-friendly when it comes to design or overcomplicating with technical friction inducing cognitive overload?
Eric Weber, in his blog, Making Data Actionable: The Immense Challenge of Good Data Products, sheds light on this and suggests that design matters. However, this does not translate just to the “looks” but to ease of use.
In the same blog, Eric talked about a 10-second rule. He says if the user can quickly understand the concept and connect with the context, the probability of him sticking with it doubles. However, if the user lands in the swamp and finds it hard to relate, be assured of losing him.
We also found some very powerful UI fundamentals for complex products, as outlined by Ivan Seow:
1. Two-Tap Rule: Execute primary functions within a maximum of two taps.
2. The Five-Point Rule: Every new font, font size, or color on a page counts for one point. If your page contains more than five points, it’s probably time to redesign.
3. The 98% Rule: Design for how the product will be used 98% of the time.
‍
Simon O’Regan published an article Designing Data Products, which elaborates on the types of data products and describes 5 variants centered around Raw Data, Derived Data, Algorithms, Decision Support, and Automated Decision-Making. The article goes a step further and talks about interfaces to interact with data products.
In my experience the biggest problems that teams encounter with data products happen when they apply methodologies like human-centred design on more technical data products.
For technical data products, the product boundary is often artificially constrained by functional organisation considerations, and the product and UX team is often insufficiently technical to either a) understand the intricacies of technical user behaviour or b) insufficiently inclined to explore these intricacies.
To assume then that the Design-Thinking or Lean methodologies that we’ve been reading about should be applied out-of-the-box is naive.
‍
We have discussed the recipe for building Data Products in a recent blog that focuses specifically on design- the core capabilities and functions of the design stage. The blog shares steps and key considerations for developing effective data products.
The key to designing data products is to work backwards from the user problem. A product approach means to identify the consumer and their problems before kickstarting any effort. This implies mapping any and all efforts around data to specific business goals/metrics and challenges. Do check out the metrics-first approach.
‍
For more context, refer to: How to Build Data Products? | Design: Part 1/4
‍
Data 2030 Summit is an annual strategy event gathering the Data Management community in one place and platform to discuss ways of enabling faster Data Innovation and AI deployment across the enterprise by setting up a modern Data Management strategy and platform for the new decade.
The summit presents 40+ international speakers including Ashish Arora(SVP - Central Group), Eddie Lim(Head of Data & Analytics - Singtel), Monica Srinivasan(Executive Director - Standard Chartered Bank), Shamik Banerjee(Sr. mgr, Data Products - British American Tobacco – DBS), Manish Kumar(Vice President DFE - Great Eastern) and many more such experts from data space.
Event Date: 29 Sept. 2023 | Mode: Offline | Register
‍
The event is the showcase of next-generation technologies and strategies from the world of Artificial Intelligence & Big Data, an opportunity to explore and discover the practical and successful implementation of AI & Big Data in driving forward your business in 2023 and beyond.
Expo will present speakers from data and AI domain including Tarun Rana(Head of data and analytics - Henkel), Maarten van den Outenaar(CDO - Royal Schiphol Group), Timea Töltszéki (Head of Data and Platforms - Boehringer Ingelheim), Ashraf Ali K M(Director Data Science & Analytics - Footlocker), and many more.
Event Date: 26-27 Sept. 2023 | Mode: Offline | Register
‍
As usual, here’s a light breather for you for sticking till the end!
‍
Follow for more on LinkedIn and Twitter to get the latest updates on what's buzzing in the modern data space.
‍
Feel free to reach out to us on this email or reply with your feedback/queries regarding modern data landscapes. Don’t hesitate to share your much-valued input to the Editor.
