Article
Many companies want to give their employees access to data, but are overwhelmed by the size and complexity of the data, as well as security and privacy risks inherent with opening it up. One powerful way that companies are overcoming these challenges is by embracing the concept of a data product.
A data product is an application that’s created to enable users to access curated data or insights generated from data. Data products can be developed for an external audience, such as Netflix’s movie recommendation system, or they can be used internally, such as a sales data product for regional managers.
Enabling Data Exploration
The typical company stores vast amounts of data across a multitude of data silos, including databases, file systems, object stores, and even directly within applications. Knowing what’s contained in those data stores is a massive challenge in its own right, and is step one in the data product journey.
Many companies today are adopting data catalogs to help them explore structured and unstructured data in a controlled and predictable manner. Data catalog vendors like Alation and others use metadata to track data within an enterprise and use indexes and other methods to help customers find the data they need. In addition to catalogs, Alation helps control access to data through data governance, and supports the concept of a Data Products Marketplace, where users can browse a variety of data products their company exposes, including domain-specific data products created as part of a data mesh.
Ensuring Data Quality
One important aspect of a data product is the quality assurance it affords. Raw data often contains errors or needs a certain degree of shaping and transformation before it can be used. This is particularly true for derived data sets that are used as the source for downstream data products, as well as data that’s used for training AI models.
Companies can use various techniques for ensuring high-quality data in data products. Ataccama, for instance, enables data engineers to set up and enforce data quality rules that ensure that data meets minimum standards. That’s important considering that the vendor recently found that 41% of organizations report data quality as a major challenge.
Providing Data Governance
Another way that data products can empower data-driven decision-making is through rigorous data governance. By automating the processes that assure companies that correct procedures are being followed regarding the provenance, lineage, security, and privacy of data, companies can move more quickly with their data product rollouts without worrying whether shortcuts are being taken.
One of the vendors providing data governance capabilities for data product development is Collibra. The company, which was listed in the Leaders Quadrant in Gartner’s first ever Magic Quadrant for data governance, is a backer of data meshes as a way to process and share data as a product.
Conclusion
Data products have the potential to democratize access to data and accelerate adoption of analytics and AI to better position a company to compete. The best data products are custom-developed and are themselves products of various tools and techniques that companies can bring together. Specifically, the roles that data exploration, data quality, and data governance play in enabling data product development should not be overlooked by prospective data product users.
FAQs
Q: What is a data product?
A: A data product is an application that’s created to enable users to access curated data or insights generated from data.
Q: What are the benefits of using a data product?
A: Data products can help companies democratize access to data, accelerate adoption of analytics and AI, and better position a company to compete.
Q: How do data products address data governance challenges?
A: Data products can address data governance challenges by automating the processes that assure companies that correct procedures are being followed regarding the provenance, lineage, security, and privacy of data.
