Applying the 4 Principles of Data Mesh to Deliver the Promise of Industry 4.0

Leading manufacturers have begun to embrace the four principles of a data mesh to evolve their architecture into one that supports delivering Industry 4.0 projects at scale.

Monica McDonnell
Monica McDonnell
2023年3月3日 4 最小阅读
Data mesh and the promise of industry 4.0

Artificial Intelligence (AI) and Machine Learning (ML) can return significant value to factories, but there are still challenges to overcome. These challenges are not in the data science – the code usually works well. The primary challenges to delivering AI/ML projects at scale are rooted in architecture and data governance. Manufacturers need to find an efficient, and cost-effective way to find, understand, use & then reuse data across multiple projects.

Leading manufacturers have begun to embrace the four principles of a data mesh, even if it is not referred to as a data mesh. Their aim is to evolve their architecture into one that supports delivering Industry 4.0 projects at scale and avoiding the two most common modes of failure.

Initial failures were labelled ‘pilot purgatory’. Here the primary mode of failure was technology scaling:  what worked in the lab did not always scale to support the demands of the production environment.

Once the technology scaling issue was rectified, ‘use case purgatory’ emerged as a new failure mode. Here the challenge is resource scaling. Companies were delivering Industry 4.0 solutions using an Industry 3.0 architecture. In other words, creating a dedicated pipeline to feed an application which supports a single, or few use cases. In this purgatory, use cases are deployed, but at a high price. The development costs of a dedicated data pipeline coupled with support cost of maintaining isolated apps means this purgatory is not filled with failure, it is sparsely populated. Additional use cases are out of reach due to limited budget.

The clear path out of use case purgatory is a better use of resources – and in this case the key resources are data and data science skills. In a factory environment the amount and variety of data is large and growing.  However, it is also finite – and crucially multipurpose. For example, sensor data that comes from a machine can support maintenance use cases, provide insight into the quality of products produced and support process improvement. Machine data is both a large data set and needs very specific operator knowledge to provide context to measurements, alerts, and error codes. To gather and preparing this set of data multiple times is a waste of data science resources.  Why not collect, cleanse and pre-process it once for all three use cases, plus others?  

This is why the 4 principles of a data mesh should help make better use of limited resources. The first two principles describe a multipurpose approach to data cleansing.

Firstly, domain-driven ownership of data will ensure those people who really understand contextual data are given responsibility to prepare this for multiple use cases. As OT and IT have traditionally been separate in a manufacturing company, the ownership principle should be relatively easy to implement.

Secondly, data must be treated as a product. Creating data products is a new approach, but one that should be readily embraced given the use case purgatory experience. Data products can be as simple as data sets prepared for analytic purposes, or more complex such as the output of ML routines.
industry-4.png

These two principles do not require specific technology, the main implication is organizational. Distributed groups must accept responsibility for ownership of a specific set of data – and ensuring this is available to all authorized users. This has added benefits in terms of reducing cost and increasing security by reducing data movements and redundancies.

Thirdly, a self-serve data infrastructure as a platform needs to be implemented. This is crucial to allowing a variety of teams to build on the work of others to create their own insights from data, as well as additional data products.  

And fourthly, the federated computational governance requires a blend of mesh-oriented governance practice and the ability to automate some tasks such as schema or lineage creation.  

The final two principles do imply some technical capabilities or tools. At Teradata, we have ongoing investment in technical capabilities to find, understand, use and re-use data – regardless of where it is stored. This is perfectly aligned with implementing a data mesh. Instead of having to move data at a high cost, we can implement an open data ecosystem and bring analytics to the data.  

This open ecosystem will also help manufacturers achieve the ideal of good, quick, and cheap in their Industry 4.0 analytics projects:

Good:  Increased use case deployment based on trusted & secure data 

Quick: Speed of use case deployment & use case execution

Cheap: Limit data movements and redundancies, improve compute efficiency

Manufacturing operations, supported by both IT and OT are not too far away from implementing the data mesh principles. To reap the full potential of AI and ML within a factory setting only small adjustments are required: Data governance practice to adapt ownership and introduce data products, coupled with highly scalable compute power able to operate efficiently in a diverse data ecosystem are the next steps.
 

Tags

关于我们 Monica McDonnell

Monica McDonnell is a highly experienced consultant in the field of enterprise software, digital transformation and analytics. Her career has spanned Africa, the US and Europe with time spent on ERP and supply chain planning before focusing on delivering value from data. Monica advises on how to deliver business value by combining good data governance and advanced analytics technologies. Helping automotive companies understand how to release the full potential of Industry 4.0 technologies, and dramatically improve customer experience management as enabled by the connected vehicle is central to her role. Monica earned a BSc in Industrial Engineering from the University of Witwatersrand, and a MSc in Software Engineering from Oxford University. 

查看所有帖子 Monica McDonnell

随时了解情况

订阅 Teradata 的博客,获取每周向您提供的见解



我同意作为本网站提供商的Teradata天睿公司可能偶尔向我发送Teradata市场沟通电子邮件,其中包含有关产品、数据分析、活动和网络研讨会邀请的信息。我了解我可以随时通过点击我收到的任何电子邮件底部的取消订阅链接取消订阅。

您的隐私很重要。您的个人信息将根据Teradata全球隐私政策收集、存储和处理,您可以通过单击此隐私链接阅读和打印。