Corporate Data Quality

Boris Otto • Hubert Österle

Corporate Data Quality

Prerequisite for Successful Business Models

Boris Otto

Fraunhofer Institute for Material Flow and Logistics

Dortmund

Germany

Hubert Österle

CDQ AG

St. Gallen

Switzerland

ISBN 978-3-7375-7592-8

ISBN 978-3-7375-7593-5 (eBook)

Published in 2015

Printed and published by epubli GmbH, Prinzessinenstraße 20, 10969 Berlin

http://www.epubli.de

Published under Creative Commons CC BY-NC 4.0

http://creativecommons.org/licenses/by-nc/4.0/legalcode

The Deutsche Nationalbibliothek lists this publication in the Deutsche Nationalbiblio- graphie; detailed bibliographic data are available on the Internet at http://dnb.d-nb.de

Copyright: © 2015 The authors

Cover design: Andreas Karré

Cover image: Shutterstock Image ID 304478969, Copyright: Sergey Nivens

Translation: ZIS GmbH

Foreword

Digitization is causing upheaval for the economy as well as for society overall. Under these circumstances, even more than before, data is becoming a strategic resource for companies, for public organizations and for individuals. Only when high quality data about customers and products, and contextual information about their whereabouts, preferences and billing conditions exist will companies be able to provide digital services that will make life easier, open new business opportunities or make transactions between companies quicker and simpler.

Corporate data quality as a prerequisite for successful business models was and is the mission statement for the Competence Center Corporate Data Quality (CC CDQ). The CC CDQ is a consortium research project, in which more than one hundred employees from more than 30 major companies have been working with researchers from the University of St. Gallen and from Fraunhofer IML since the spring of 2006. We have been working on solutions and methods for corporate data quality in more than 40 two-day consortium workshops and with more than 200 project meetings. The content of this book has arisen almost exclusively from the CC CDQ research.

The book will address three groups of readers. Firstly, the book would like to provide support to the project and line managers for the construction and development of company-wide data quality management (DQM). Secondly, the book would like to inform students and teaching staff at colleges and universities about the foundations of data quality management as a corporate function and place a pool of cases studies in their hands. Thirdly, the book will address the significant concepts from research and practical experience for researchers interested in their application.

The contents of this book form the core of the results of the CC CDQ project. It will provide an overview of the most important issues about corporate data quality based on practical examples. The book will refer repeatedly to more detailed material provided for all questions.

This book would not have been possible without the combined capabilities and experiences of a number of people. We owe our thanks to the representatives of the companies that have participated in the CC CDQ for their active collaboration in the consortium research process. They openly discussed their companies’ problems, developed solutions together with the researchers, tested them in corporate practice and ensured that the research efforts were always enjoyable while doing all of this. Also, we would like to thank all of our scientific co-workers, who have contributed to the success of the CC CDQ with their passion and their efforts in their dissertational intents. Of these people, Rieke Bärenfänger, without whose care and determination this book would not exist, is owed special thanks.

Corporate data quality has been making many friends for us for more than eight years. We hope that the readers will also enjoy the results.

Boris Otto
Hubert Österle

Table of Contents

1 Data Quality – A Management Task.. 1

1.1 Trends in Digitization.. 3

1.1.1 Penetration into Every Area of Life and Economy. 3

1.1.2 Industry 4.0.. 5

1.1.3 Consumerization.. 7

1.1.4 Digital Business Models. 10

1.2 Data Quality Drivers. 11

1.2.1 A 360-degree View of the Customers. 12

1.2.2 Corporate Mergers and Acquisitions. 13

1.2.3 Compliance. 14

1.2.4 Reporting Systems. 15

1.2.5 Operational Excellence. 16

1.2.6 Data Protection and Privacy. 17

1.3 Challenges and Requirements of Data Quality Management. 18

1.3.1 Challenges in Handling Data. 18

1.3.2 Requirements on Data Quality Management. 21

1.4 The Framework for Corporate Data Quality Management. 23

1.4.1 An Overview of the Framework. 23

1.4.2 Strategic Level 23

1.4.3 Organizational Level 25

1.4.4 Information System Level 27

1.5 Definition of Terms and Foundations. 28

1.5.1 Data and Information.. 29

1.5.2 Master Data. 31

1.5.3 Data Quality. 32

1.5.4 Data Quality Management (DQM). 34

1.5.5 Business Rules. 35

1.5.6 Data Governance. 37

1.6 The Competence Center Corporate Data Quality. 38

2 Case Studies of Data Quality Management. 42

2.1 Allianz: Data Governance and Data Quality Management in the Insurance Sector 44

2.1.1 Overview of the Company. 44

2.1.2 Initial Situation and Rationale for Action.. 45

2.1.3 The Solvency II Project. 46

2.1.4 Data Quality Management at AGCS. 46

2.1.5 Insights. 52

2.1.6 Additional Reference Material 52

2.2 Bayer CropScience: Controlling Data Quality in the Agro-chemical Industry 53

2.2.1 Overview of the Company. 53

2.2.2 Initial Situation and Rationale for Action.. 54

2.2.3 Development of the Company-wide Data Quality Management 57

2.2.4 Insights. 64

2.2.5 Additional Reference Material 65

2.3 Beiersdorf: Product Data Quality in the Consumer Goods Supply Chain 65

2.3.1 Overview of the Company. 65

2.3.2 Initial Situation of Data Management and Rationale for Action 67

2.3.3 The Data Quality Measurement Project. 71

2.3.4 Insights. 77

2.3.5 Additional Reference Material 78

2.4 Bosch: Management of Data Architecture in a Diversified Technology Company 79

2.4.1 Overview of the Company. 79

2.4.2 Initial Situation and Rationale for Action.. 80

2.4.3 Data Architecture Patterns at Bosch.. 82

2.4.4 Insights. 87

2.4.5 Additional Reference Material 87

2.5 Festo: Company-wide Product Data Management in the Automation Industry 88

2.5.1 Overview of the Company. 88

2.5.2 Initial Situation and Rationale for Action regarding the Management of Product Data 90

2.5.3 Product Data Management Projects between 1990 and 2009 96

2.5.4 Current Activities and Prospects. 101

2.5.5 Insights. 102

2.5.6 Additional Reference Material 103

2.6 Hilti: Universal Management of Customer Data in the Tool and Fastener Industry 104

2.6.1 Overview of the Company. 104

2.6.2 Initial Customer Data Management Situation and Rationale for Action 105

2.6.3 The Customer Data Quality Tool Project. 106

2.6.4 Insights. 113

2.6.5 Additional Reference Material 114

2.7 Johnson & Johnson: Institutionalization of Master Data Management in the Consumer Goods Industry. 114

2.7.1 Overview of the Company. 114

2.7.2 Initial Data Management Situation in the Consumer Products Division and Activities up to 2008. 115

2.7.3 Introduction of Data Governance. 116

2.7.4 Current Situation.. 118

2.7.5 Insights. 122

2.7.6 Additional Reference Material 124

2.8 Lanxess: Business Intelligence and Master Data Management at a Specialty Chemicals Manufacturer. 125

2.8.1 Overview of the Company. 125

2.8.2 Initial Data Management Situation and Business Intelligence 2004 – 2011 126

2.8.3 Master Data Management at Lanxess since 2011. 126

2.8.4 Structure of the Strategic Reporting System since 2012. 129

2.8.5 Insights. 133

2.8.6 Additional Reference Material 135

2.9 Shell: Data Quality in the Product Lifecycle in the Mineral Oil Industry 135

2.9.1 Overview of the Company. 135

2.9.2 Initial Situation and Rationale for Action.. 136

2.9.3 Universal Management of Data in Product Lifecycle. 137

2.9.4 Challenges during Implementation.. 137

2.9.5 Using the New Solution.. 138

2.9.6 Insights. 139

2.9.7 Additional Reference Material 139

2.10 Syngenta: Outsourcing Data Management Tasks in the Crop Protection Industry 140

2.10.1 Overview of the Company. 140

2.10.2 Initial Situation and Goals of the Master Data Management Initiative 141

2.10.3 The Transformation Project and the MDM Design Principles 143

2.10.4 Master Data Management Organizational Structure. 145

2.10.5 The Data Maintenance Process and Decision-making Criteria for the Outsourcing Initiative. 149

2.10.6 Insights. 153

2.10.7 Additional Reference Material 153

3 Methods and Tools for Data Quality Management. 155

3.1 Method for DQM Strategy Development and Implementation 155

3.1.1 Structure of the Method. 156

3.1.2 Examples of the Techniques used by the Method. 157

3.2 Maturity Assessment and Benchmarking Platform for Data Quality Management 163

3.2.1 Initial Situation.. 163

3.2.2 Maturity Model and Benchmarking as Control Instruments 164

3.2.3 The EFQM Model of Excellence for the Management of Master Data Quality 166

3.2.4 Corporate Data Excellence: Control Tools for Managers of Data Quality 167

3.3 The Corporate Data League: One Approach for Cooperative Data Maintenance of Business Partner Data. 170

3.3.1 Challenges in Maintaining Business Partner Data. 170

3.3.2 The Cooperative Data Management Approach.. 171

3.3.3 The Corporate Data League. 172

3.4 Additional Methods and Tools from CC CDQ.. 176

4 Factors for Success and Immediate Measures. 178

4.1 Factors for the Success of Data Quality Management. 178

4.2 Immediate Measures on the Path to Successful Data Quality Management 179

5 Bibliography.. 181

6 Glossary.. 193

Table of Abbreviations

API

Application Programming Interface

BE

Business Engineering

CAD

Computer-aided Design

CC CDQ

Competence Center Corporate Data Quality

CDL

Corporate Data League

CDQM

Corporate Data Quality Management

CIQ

Customer Information Quality

COO

Chief Operating Officer

CRM

Customer Relationship Management

CRUD

Create, Read, Update, Delete (database operations)

DAMA

Data Management Association

DQM

Data Quality Management

DUNS

Data Universal Numbering System

EFQM

European Foundation for Quality Management

ERP

Enterprise Resource Planning

EU

European Union

GS1

Global Standards One

GTIN

Global Trade Item Number

IRR

Internal Rate of Return

IS

Information System

ISO

International Standards Organization

IT

Information Technology

LCC

Lifecycle Costing

MDM

Master Data Management

NPV

Net Present Value

OMG

Open Management Group

p.a.

per annum

PIM

Product Information Management

PLM

Product Lifecycle Management

ROI

Return on Investment

SBVR

Semantics of Business Vocabulary and Rules

SCM

Supply Chain Management

TCO

Total Cost of Ownership

TQM

Total Quality Management

XAL

Extensible Address Language

About the Authors

Prof. em. Dr. Dr. h.c. Hubert Österle was professor for Business Engineering and director of the Institute of Information Management at the University of St. Gallen (IWI-HSG) from 1980 to 2014. In 1989, he founded the Information Management Group and served in the company’s management and supervisory boards. In 2006, he founded the Business Engineering Institute St. Gallen AG and is presiding over its supervisory board. He is also member of the supervisory board of the CDQ AG. His main research areas are life engineering, corporate data quality, business networking, business engineering, and independent living.

Prof. Dr. Boris Otto holds the Audi-Endowed Chair of Supply Net Order Management at the Technical University of Dortmund and is director for Information Management and Engineering at the Fraunhofer Institute for Material Flow and Logistics. The focal points of his research and teaching fields are business and logistic networks, corporate data management as well as enterprise systems and electronic business. Boris Otto studied Industrial Engineering in Hamburg and received his doctor’s degree under the supervision of Prof. Hans-Jörg Bullinger at the University of Stuttgart. He habilitated at the University of St. Gallen under the supervision of Prof. Hubert Österle. Further research appointments were at the Fraunhofer Institute for Industrial Engineering in Stuttgart and at the Tuck School of Business at Dartmouth College in New Hampshire in the United States. He gained comprehensive practical experiences at PricewaterhouseCoopers and at SAP. Boris Otto is a member of the scientific advisory board of eCl@ss e.V., a leading standard-setting organization for the classification of articles and products. He also heads the Data Innovation Lab at the Fraunhofer Innovation Center for Logistics and IT and is president of the supervisory board of the CDQ AG.

1 Data Quality – A Management Task

Chapter Summary

Chapter 1 will introduce the role of data in the digitization of business and society and describes the most important business drivers for data quality. For companies, data represents a strategic resource that must be cultivated with a view towards the issues of time, expense and, naturally, quality. Data quality management is the corporate function for improving and assuring the quality of the company’s data in an enduring manner. This chapter will present the Framework for Corporate Data Quality Management and introduce essential terms and concepts. A section about the Competence Center Corporate Data Quality (CC CDQ)’s research efforts will provide an overview of the foundations for the research methods employed by this consortium.

Data is the foundation of the digital economy. The penetration of all areas of life and business with “digital services” supplies data as the fuel for new services, new access to customers, new pricing models, new economic systems and finally for a major percentage of the innovation decisive for business. All IT applications generate electronic data, which in turn creates a flood of data that has not been seen until now and which needs to be understood and used.

Ericsson, for example, is a leading provider of telecommunications products and services. With its headquarters in Stockholm, Sweden, this company provides solutions for the broadband mobile Internet, among other services. The use of these solutions creates new data. At the same time, Ericsson is re-positioning their services away from the field of network technologies into the field of digital services. Together with the Maersk container shipping company, Ericsson provides information transparency across the global supply chain (Ericsson 2012). Thus, for example, the maturity of bananas on trans-oceanic ships from South America to Europe can be continuously monitored and shipping speeds and losses at the destination port can be adjusted as needed. This leads to improved flow of goods at the port, optimization of fuel consumption by ships and, ultimately, to customer satisfaction at fruit stands in supermarkets.

An increasingly higher level of data quality is being demanded by corporate innovations as well as by the classic data quality drivers like business process harmonization. Because of the digital connectivity of entire value networks, data errors and misuse are having more significant effects than they did in the age of isolated IT applications. For example, organized groups of hackers (Dahlkamp and Schmitt 2014) are hacking into email traffic between companies, presenting themselves as creditors and redirecting payments for deliveries and services to fraudulent accounts. Often, this does not become obvious until the right creditor sends payment reminders for late payments. At that point, the transaction can no longer be reversed.

Data quality is not an “issue of hygiene”, but requires management. In the digital economy, companies must cultivate data like any other economic assets, especially with regard to cost, time and, of course, quality.

Structure of this Book

The first chapter of this book will review current data quality management drivers and introduce the Framework for Corporate Data Quality Management. In addition, this will be combined with the state of the science and practices regarding data quality management and will lead into the core concepts.

The case studies in Chapter 2 will show how important companies have made data quality a duty for all levels of management. The quality of the master data[1] cannot be guaranteed in one, central IT department, but rather must be ensured at the location of data acquisition and usage, meaning in the business divisions. The case studies document how ten companies from different sectors have anchored data quality management in everyday business routines.

Chapter 3 will present methods and tools that will provide support to companies constructing a successful master data quality management system. All of the methods have been tested several times in practice.

Chapter 4 will summarize the primary insights of the approaches described for solutions and present a list of immediate measures for improved data quality management.

1.1 Trends in Digitization

New forms of information technology are changing every area of the economy as well as of society overall, as researchers, such as Kagermann (2014), have analyzed them from the view of the Federal Republic of Germany. We have summarized the development into four trends (Figure 1-1).

Figure 1-1: Mega-trends in Digitization (authors’ illustration)

1.1.1 Penetration into every Area of Life and Economy

According to the International Telecommunications Union, 2.9 billion people used the Internet in 2014, meaning roughly 40% of the world population (ITU 2015). The technological innovations of the last 15 years are responsible for this penetration into both the private and business areas.

· Mobility: wireless networks and miniaturization of computers and other components, like sensors and cameras, are bringing digital services to the location of use, whether in private life, such as recording a hiking route, or in business, such as remote diagnosis of a machine.

· Usability: touch screens and many other improvements in details, like logging into digital services through a Facebook account or vocal input and output systems, have drastically reduced the threshold for usage. Other efforts to ease usage, like data glasses (such as Google Glass), control using gestures and detection of eye movements, have also been distinguishing themselves.

· Content and community: whether individually (such as through blogs and tweets) or in combination (such as Facebook), innumerable people have been producing a volume of content in the form of written words, pictures, audio and video files, which can only be reviewed by machines. YouTube recorded more than one billion requests for video clips per day in June 2014[2]. Facebook recorded roughly 1.3 billion active users in March 2014[3].

· Communication: this content is being exchanged synchronously, asynchronously, privately and for business. Accessing email, text messaging, and social networking are among the top four most-popular daily activities of smartphone owners in the United States in January 2014 (statista 2015). Visual communication has been increasingly supplementing more common audio telephony and instant messaging services (such as WhatsApp) are frequently used in addition to email messages.

· Big data: unexpectedly high volumes of data are the result of the penetration of the economy and society overall by digital services, while at the same time they are the foundation for the personalization of services, especially those based on providing location information (Figure 1-2).

Figure 1-2: Online Activities for Private Purposes over the Last Three Months in Swiss Households (Froideveaux 2012 p. 25)

Almost one quarter of the world population used smartphones in 2014 and in both North America and Western Europe, about 50 percent of total population used smartphones in one way or another (statista 2015). Digital networking has had an enormous impact on the formation of people’s opinions in their political, economic and private affairs. From the view of data management, the following aspects (among others) should be taken into consideration.

· Data security: until now, the Intranet was considered the perimeter, meaning the boundary where data had to be secured. This boundary has dissolved and companies must go beyond it to protect not only networks and application systems but also enable data objects, which themselves must know who should be permitted read access and who should not (O'Brien 2014).

· Data production: classically, companies have acquired data centrally (such as customer data collected through a central, internal marketing service). Due to the spread of social media and social networks however, data consumers are increasingly becoming data producers (Strong et al. 1997). Customer data can be acquired directly from the customer or from external agencies by smartphones or onsite tablets. Employees expect that the data will be accessible from everywhere.

· Streams instead of records: millions of users generate data flows in social networks and through social media. This represents new challenges to companies, because the traditional processing of data was oriented on transactions, meaning that individual records were written persistently to databases. However, the increasing usage of data streams from social networks, such as from the cyber-physical systems in Industry 4.0, can no longer be updated incrementally, but rather must be followed continuously (BITKOM 2014).

1.1.2 Industry 4.0

The term “Industry 4.0” stands for the Fourth Industrial Revolution, meaning the merger of the physical and virtual worlds through so-called “cyber-physical systems” (Bauernhansl et al. 2014). The data will be acquired more precisely and in more detail than previously without time delays or the help of people. Machines are becoming capable of working with the Internet, assuming the tasks of production and data distribution independently and the data, which has only been available in the factory for a long time, is becoming accessible to the entire company and its business partners (Figure 1-3).

Figure 1-3: Data Acquisition at the Interface between the Virtual and Physical Worlds (Fleisch 2010; Wahlster 2011 p. 5)

Industry 4.0 scenarios are changing the basic handling of data both within and between companies. Three issues make this clear.

· Decentralization of data management: things are becoming “smart”, meaning that they produce, use and have an increasing amount of data and rely less on central control systems. As a consequence, things are assuming increasing importance in the distribution of data without requiring central computers.

· From the class to the instance: the focus of electronic data processing has been on traditional classes of things, meaning articles with a certain Global Trade Item Number (GTIN) or products with certain material numbers. Industry 4.0 now means that each instance of a class of products can be identified, meaning individual hydraulic cylinders or the individual bottles of hydraulic fluid (Österle and Otto 2014).

· Continuous combination of the flow of information and goods: traditionally, industrial data processing has targeted the flow of information and goods to certain control points. One example is the goods receipt record in the central warehouse for the delivery of goods. Industry 4.0 scenarios use RFID technologies, for example, and enable access to the status and location information for the individual products at any time (Österle and Otto 2014).

The inBin intelligent container developed by the company SICK[4] together with the Fraunhofer Institut für Materialfluss und Logistik (Fraunhofer IML, Institute for Material Flow and Logistics) is one example of an Industry 4.0 application. The inBin knows its location, records its environmental temperature and arranges for its own pickup (Figure 1-4).

Figure 1-4: The inBin intelligent container (Fraunhofer IML 2015)

A powerful data management system that fulfills the following requirements is the prerequisite for the success of Industry 4.0 in individual companies as well as across supply chains.

· Mastery of the volume of data: the data management system in the company must be capable of processing and reasonably evaluating the amounts of data (Wrobel et al. 2014).

· Decentralized data processing: when machines, containers, freight and so on become intelligent, this means that they will have to assume the tasks of processing their own data. Data analysis, aggregation and provision therefore no longer occur centrally in the Enterprise Resource Planning (ERP) and data warehousing systems, but rather locally onsite. Central corporate data processing will be supplemented by a network of decentralized intelligent devices (Aggarwal et al. 2013).

· Determination of data standards: advantages in terms of time, expenses and quality through the use of cyber-physical systems and automated data interchange can only be realized when standards for the description and exchange of data have been established. These standards must apply internally to the company, at least, and their applicability across entire supply chains would be better (Otto et al. 2014). The MobiVoc initiative developed, for example, a data vocabulary for new mobile solutions[5].

1.1.3 Consumerization

Every one of us today uses a number of different consumer services that support various situations of our lives (Österle 2014). Figure 1-5 depicts ten areas of life in which people use digital services, from support for navigation to listening to music, from comparing prices to controlling the illumination of homes remotely. As an example, the area of communication has been expanded with two additional layers in order to provide an impression of the multitude of services. A more detailed, but still incomplete mind map of digital consumer services can be found at il.iwi.unisg.ch/appmap (Amiona 2014).

Figure 1-5: Ten Areas of Life and Examples of Digital Services supporting Them (Amiona 2014)

At the same time, consumers increasingly expect digital services to be customized to their individual needs. Companies are reacting to the consumerization of information technology by orienting their business processes on the consumers’ needs, thus the consumer process. This process consists of all activities that an individual accomplishes for the fulfillment of various needs (such as purchasing, athletics and traveling) in a certain situation of the consumer’s life.

Consumerization leads to a new role for consumers in economic life (“consumer centricity”). They are no longer the terminal or transitional points in the unidirectional flow of goods and information, but rather directly affect public opinions of products and companies through platforms like FoodWatch.org and are now acting both as consumer and producer of goods and services. Examples include the floods of indignation that descended on Nestlé because of the use of palm oil in KitKat chocolate bars and the crowd sourcing of programming services.

Figure 1-6: Network Analysis of the Flow of Product Information at Beiersdorf (Schierning 2012 p. 9)

Figure 1-6 depicts an example of how the flow of product information has changed over a period of five years at Beiersdorf, a manufacturer of consumer goods. On the one hand, the number of participants in the company network increased from 2007 to 2012, because companies like Apple and Google, as well as online retailers like Zalando use and distribute Nivea, for example. Borrowing a term from ecology, the expanded corporate network can also be viewed as an ecosystem. On the other hand, consumers have moved from the periphery to center with regard to control of the data, since nearly every company in the network interacts with consumers (Schierning 2012).

Nestlé not only maintain classic corporate data systems, but also consumer data. Nestlé had 94 million fans on Facebook and 16 million clicks to view their Contrex video on YouTube. Data from online shops, where Nespresso sold more than 50% of their coffee packets for example, should be added to these figures.

Consumer centricity means a rejection of the traditional corporate-centered view of the end customers for companies. The design and improvement of interaction with consumers is no longer the only focus from the view of the companies (Inside-out approach), but also the integral consumer process across the boundaries of individual companies (Outside-in approach).

Consumerization places new requirements on the management of data.

· Data ownership: who does the data belong to? This multi-facetted discussion about data protection and statements like those of Mark Zuckerberg of Facebook, that data security is no longer a social standard (Johnson 2010), indicate that the trend in consumerization has surpassed the traditional understanding of ownership and possession of immaterial goods. So-called “data brokers” collect personal Internet data in legal gray areas (Anthes 2015). For companies, this means that they must formulate a uniform legal position in regards to data protection. Legislatures are being asked to crate uniform frameworks.

· Data integration: people no longer use a single channel for communication in order to connect to a company, but rather use multiple channels. The Swiss retailer, Migros, identified nine different channels (offline and online) through which they communicate with consumers. This diversity includes traditional letters, online shops and email and text messages. Because consumers expect to be uniquely identified through all channels and to get the same prices and rebates on Migros products, the company had to provide consistent, current and complete data about their customers and products across all channels (Schemm 2012).

· Combinations of “structured” and “unstructured” data: as a consequence of consumerization, companies are no longer only providing information in traditional alphanumeric formats, such as descriptions, weights and prices about the products, but are more often providing product video clips, marketing information and lists of active ingredients. The differences between product data (which is generally stored in central ERP or Product Lifecycle Management (PLM) systems) and multimedia product information (which is frequently distributed using a number of internal application systems and external service providers, such as advertising agencies) can no longer be maintained (Österle and Otto 2014).

1.1.4 Digital Business Models

The penetration of digital services into the economy and society overall and in particular, of the industrial and consumer sectors, will lead to new types of business models outside of classic companies[6]. Examples from the area of consumer services include Google as well as Airbnb, idealo and many other companies that bring a large number of consumers and business customers together with a large number of providers. These companies have been assuming a role as brokers between the supply and demand for services from a variety of participants. From a more technical point of view, one frequently speaks of the “Internet of services”. Four developments characterize these business models.

· Focus on data: new business models for the Internet-based service economy use data as a strategic resource (see Figure 1-7). For example, Deutsche Post provides high-resolution geographic information for retailers, insurers, real estate agents and public administration and other customers (data as the product)[7] through the GEOVISTA service.

· Industrial convergence: traditional sector boundaries are losing their significance. Google is one of the innovation drivers for autonomous cars; classic vehicle manufacturers are potential licensees for this technology. Amazon has transformed itself from a book retailer into a fulfillment expert, who offers special capabilities like scalable IT infrastructure services or provides logistics service to companies from many sectors and even consumers.

· Hybrid services: often, digital business models combine digital services with classic offline services. One example involves the car-sharing models that combine digital rental and provision of cars including payment (generally supported by smartphone apps) with the classic services of mobility.

· Consumer process: the Internet of services is oriented on the individual, meaning the individual consumer, the patient, the service technician or the shopper. The goal is “end-to-end” support for life situations, such as purchasing, jobs, mobility, therapy and health care (Österle and Senger 2011).

Figure 1-7: Digital Business Models (Brenner and Herrmann 2012 p. 20)

1.2 Data Quality Drivers

Digital business models and the Internet of services are based on the resource of data. Data quality is therefore no longer a question of “hygiene” or even of internal use by line departments, but rather has become critical for operational excellence. Data quality is defined as a measure of the applicability of data for fulfilling certain requirements in business processes, where it is used (Otto et al. 2011). The following material will constantly treat “data management” with special consideration of data quality management.

The most important drivers for quality-oriented data management include:

· 360-degree view of the customers

· Corporate mergers and acquisitions

· Compliance

· Reporting systems

· Operational excellence

· Data protection and privacy

1.2.1 A 360-degree View of the Customers

Knowledge about the customers is the starting point for marketing and sales, as well as for the development of products and services. For this reason, companies must be capable of gaining access to all information about the customers’ needs. For consumers, such information includes Internet surfing behavior, purchases and peer groups in social networks; for business customers, their addresses, subsidiaries, contact information and the name of their contact people as well as data about purchased products and existing contracts.

Bühler is a globally active manufacturer of production systems specializing in the industry of food. They make digital customer profiles available to their employees in the Customer Service and Marketing departments. These profiles answer questions such as:

· How much revenue has been made with the customer (and all of its subsidiaries) in the current fiscal year?

· Which of our systems and services are used by the customer and at which locations?

· When will maintenance contracts expire?

· Which employees made contact with which customer employees in the last three months? What were the results of such contacts?

· How profitable is the customer relationship?

The 360-degree view of the customers places many requirements on quality-oriented data management.

· Data quality: customer data must be consistently, currently and completely available for all functional departments (Marketing, Services, etc.).

· Data lifecycle: how customer data is acquired by the company, where it was acquired and stored, who will modify and change the data and which business processes and systems use the data must be clearly defined.

· Data security: with consumer data, provisions must be made to ensure that data protection provisions will be maintained, including that customer data will be deleted upon request.

· Data governance: companies must clearly determine who will be responsible for which customer data in the company. Is the field staff responsible for customer addresses or the internal Marketing department? Can service employees change the customer status to active? Who will collect email messages with this customer or their Facebook pictures?

1.2.2 Corporate Mergers and Acquisitions

Corporate mergers and acquisitions are important tools for corporate strategies. In the chemical industry for example, BASF has taken over the electro-chemical division of Merck, the fine chemistry company, Orgamol, the catalyzer manufacturer, Engelhard, the building chemistry division of Degussa and the special chemical group, Ciba, since 2005. These acquisition have been integrated in the uniform application systems and business processes.

Nestlé represents another example of corporate integration. The company operates more than 2000 different brands, which are produced in almost 90 countries and sold in more than 190 countries[8]. Of their total revenue amounting to more than 92 billion Swiss franks in 2013, 93% are processed through the GLOBE central Enterprise Resource Planning system. Figure 1-8 depicts several pieces of important information about GLOBE.

Figure 1-8: Important Information about the GLOBE Central System at Nestlé (according to Muthreich 2013 p. 18)

The GLOBE program has pursued three goals since it began operations in 2001, specifically: the company-wide use of best practices based on shared business processes, the introduction of standardized application systems and the use of data as an asset. The prerequisite for this is a powerful data management system, which has integrated many corporate acquisitions over the last few years in particular.

· Data standards: binding specifications for the acquisition, maintenance and use of master data, such as customer, supplier and material and product information, must be applied.

· Data acquisition at the source: due to the size and complexity of the company, data cannot be acquired centrally, but rather as close as possible to its source.

· Data quality: the size of the GLOBE system does not allow contaminated data to be introduced into the system and then to be cleaned up afterwards. Instead, the data must be correct when it is entered the first time (first time right principle).

· Data integration: an integrated system like GLOBE does not allow for “data silos”, but rather must work with all business areas, functions and markets using an integrated database. However data integration can only be developed after the company has changed their approach to data from “my data” to “our data”.

1.2.3 Compliance

The increasing concentration of regulations is forcing companies to comply with a large and continuously increasing number of legal and official specifications and guidelines. The following measures are two examples of such regulations.

· The EU REACH (Registration, Evaluation, Authorization and Restriction of Chemicals) directive regulates the obligation of registering chemicals that have been imported into the EU and determines the data requirements for registration (no data, no market)(EC 2006). The data requirements include specifications about the manufacture and safe use of the chemicals, among other things. Companies must include these requirements in their master data records for materials and keep them current, consistent, complete and in the proper form for reporting purposes.

· With the keyword “Solvency II”, the European Commission has harmonized insurance oversight law, in particular with regard to so-called “solvency capital”. Components of the directive include specifications for risk management and obligations for creation of report for insurance companies. These specifications have resulted in demands to comply with a company-wide, uniform management system of markets, core business and financial data (Salchegger and Dewor 2008).

Novartis, the pharmaceutical company, must be able to provide complete, current and correct data about clinical studies and active ingredients in products based on official and legal requirements. As a prerequisite for this, the company has created a universal, company-wide regulatory submission process. The data management system plays a special role in that process.

· Data consistency: not only the data itself but also the metadata (definitions, lists of values and so on) must be consistent across systems, business processes and features.

· Data lifecycle: the entire lifecycle of the data, from its creation to archiving and deletion, must have been defined.

· Data governance: who has which rights for the definition and use of which data must be defined.

1.2.4 Reporting Systems

Companies expend between 1 and 5 percent of their revenue for the creation and operation of powerful corporate software, such as SAP Business Suite (Reynolds 2010; Equey et al. 2008), however often cannot answer basic questions. Examples of such questions include:

· How many products make up our inventory?

· What level of procurement comes from the largest ten suppliers?

· What level of revenue did we earn over last fiscal year from our most important customer?

The reason for this is not due to something such as inability to use the systems or a lack of maturity in IT operations, but rather the lack of a so-called “single source of truth”. Large companies consist of a number of divisions, locations and business process, which have developed their own images of reality (customers, materials, suppliers and so on) over the course of time. When, for example, the level of procurement for all locations, divisions and so on with a given supplier and all of their subsidiaries should be determined as part of a supplier development measure, the various depictions of reality do not fit together.

For example, Figure 1-9 depicts several data quality challenges using ZF Friedrichshafen AG as an example, because they had to master such challenges for a meaningful business partner reporting system.

For a trustworthy reporting system, the data management components include:

· Data model: the prerequisite for the single source of truth is tight management of the customer, product and supplier data, such that all objects that are important throughout the company can be uniquely identified.

· Data quality: use of the data in the reporting system specifies the requirements on data quality, meaning the level for actuality, completeness and consistency that must be fulfilled by certain attributes of customer, product and supplier data.

· Data architecture: on the one hand, data architecture defines the corporate data model, but on the other hand also determines which systems are single sources of truth for which data objects or attributes and the other systems into which that data will be distributed from there.

Figure 1-9: Data Quality Challenges for Reporting Customer Revenue at ZF Friedrichshafen (Möller 2012 p. 24)

1.2.5 Operational Excellence

Along with the standardization and automation of business processes, companies are taking advantage of the effects of scaling, while also reducing their complexity. The prerequisite for this is a consistent understanding of the data in the company that will be used in every area of business, because standardization of the business processes will not be possible if, for example, material master data has been defined differently in sub-processes or regions, and is generated or used differently.

In 2007, the consumer goods division of Johnson & Johnson in the US suffered from many problems related to the data quality of the business processes (see Figure 1-10 and Section 2.7).

Figure 1-10: Problems in Business Processes resulting from Poor Data Quality at Johnson & Johnson (Otto 2014 p. 20)

Less than 30 percent of the logistics data about articles, meaning information about the dimensions and weights of articles, complied with the allowed error tolerance of 5 percent or less. Put differently, more than 70 percent of the logistical data was incorrect. Johnson & Johnson re-generated their master data management and achieved a Six Sigma level with regards to their data quality in 2013[9]. The prerequisites for this included:

· Data governance: a central office in the company determines how data should be defined, created, used and deleted, and what level of quality it had to have so that the business processes could operate smoothly.

· Data quality measurement: measuring the quality of the data monthly was one of the most important attributes. A record that did not completely fulfill several hundred business rules was considered defective.

· Workflow-supported record creation: the acquisition and modification of data was regulated clearly and supported universally using a workflow management system.

1.2.6 Data Protection and Privacy

Private people are increasingly surrendering personal data voluntarily on the Internet in order to use offers such as social networks. While data protection is barely regulated in the US, the protection of personal data in the European Union has been defined as a basic right (EU 2010). Data may only be processed with the agreement of the people affected or on the basis of a legal justification. For companies, these specifications affect advertising measures and the analysis of customer behaviors. Of course, the legal foundation is not always clear in consideration of integration through the Internet and as a consequence of the outsourcing of IT activities to other countries, where other data protection provisions apply.

In the spirit of information security, companies also have the duty of protecting the personal data managed by them against unauthorized access by third parties. The data leaks of Internet companies are particularly damaging in this regard. For that reason, a telecommunications company attempted the legal prevention of the publication of an article in December 2013, which would have informed the public that information about the bank accounts of 7,500 customers as well as 5.6 million email subscriptions had been stolen. Through a court decree, the provisional measure was lifted in the meantime and the information was published (Schmid 2014).

1.3 Challenges and Requirements of Data Quality Management

Companies have been presented with the task of making the social trends of digitization (Section 1.1) useful on the one hand while also having to find answers for the primary drivers for data quality (Section 1.2) on the other hand. This results in a number of specific challenges and service requirements that companies must take into consideration for the management of master data oriented on quality (or data quality management or DQM, for short).

1.3.1 Challenges in Handling Data

The Competence Center Corporate Data Quality (CC CDQ) at the University of St. Gallen and the Fraunhofer Institut für Materialfluss und Logistik (Fraunhofer Institute for Material Flow and Logistics) in Dortmund has been addressing the major challenges of data management and been developing new approaches to solutions for them since the spring of 2006. As part of the CC CDQ research, data managers from various industries were surveyed about the major challenges for data management in April 2013 (Österle and Otto 2014). Table 1-1 depicts the prioritization of the challenges as a result of this focus group.

By far, focus group participants considered the improvement and safeguarding of the quality of data to be the most important challenge. Data quality is a measure for the extent to which the data is suitable for fulfilling the requirements of the business processes which will use said data (Otto et al. 2011). Data quality can be measured using a variety of data quality dimensions, such as consistency, actuality and completeness.

Table 1-1: The Ten Major Challenges of Data Management[10]

Focus group participants viewed the transparency of data usage to be the second most important challenge. In large companies with complex application system landscapes in particular, it is often unclear where and how data has been acquired by the company, in which systems it has been saved as primary data and what will be done with it after its distribution to local application systems. White and Radcliffe (2010) use the term “lack of downstream visibility” of the data in the context.

Redundant maintenance of data was viewed as the third most important challenge. One example is the acquisition and maintenance of supplier master data in various business areas for the same company or for various companies whose supplier base overlaps. Typically, each company acquires and maintains their data separately, even though they all need the same data. How many places maintain, for example, address information for IBM? How often must one consumer enter their address and other information? Wouldn’t there be an advantage in portals like Facebook attempting to record redundant identification information for the consumers and providing these identities to all interested, authorized parties?

Manual maintenance of data is a fourth challenge, which arises in the data processing for breaches in media (Fleisch and Österle 2004). One example of such a media breach is typing or scanning customer master data from a business card into a Customer Relationship Management (CRM) system. Manual maintenance of data is more susceptible to errors and thereby presents a danger to the quality of data.

The issue of limitations of central data architectures is viewed as the fifth important challenge, because an increasing amount of data will be procured from external sources in the future and made available to business processes at the time it is required. One example is the specification of CO₂ emissions in the production and distribution of consumer goods. Manufacturers of consumer goods who are obligated to provide such specifications, such as in France (AFNOR 2009), will not keep such data in central (ERP) systems, but rather access them from authorized databases provided by third parties.

The semantic integration of data is the sixth most important challenge of data management. In data integration, semantics are defined as the interpretation of data in a specific application case (Ziegler and Dittrich 2007). The term “customer”, which is understood to be an active customer by a company’s accounting department, but understood as a potential customer by the marketing department, is one example.

The separation of structured and unstructured data was indicated to be the sixth most important priority. Alphanumeric data that is often organized according to a relational database scheme is designated as structured data. Textual messages, audio files, video clips, images, tweets and drawings are considered to be unstructured data. The separation of these two types of data presents problems to companies, when reporting systems should analyze data from social networking platforms or consumer portals in addition to revenues (Baars and Kemper 2008).

Data security took eighth place among the important challenges of data management. In 2011, Sony became the victim of a hacker attack against their PlayStation network, through which users’ data was stolen. The company was sentenced to pay a penalty in the amount of 250,000 GBP in the UK at the beginning of 2013, because the incident “could have been prevented” (BBC 2013) in the view of the Office of the Information Commissioner (ICO). The loss of reputation however weighed even more heavily.

The separation of OLAP and OLTP was considered the ninth most important challenge of data management. Data is often acquired and maintained in OLTP systems, such as ERP systems, and subsequently extracted from them in order to be imported into OLAP systems, such as data warehouses and business intelligence applications, after steps for its transformation and cleanup have been taken. The challenge consists of storing the data redundantly thereby, which causes additional efforts and presents a danger to the data quality.

The management of not only classes of entities but also instances in the future was prioritized as tenth place among the challenges. A specialist machine builder can identify each individual replacement part by using RFID. This approach distinguishes itself from classic data processing in that a master data record (including part number) describes the sub-class and inventory data describes the respective stock of inventory. The data management system will have to be able to handle the increasing quantity of records.

1.3.2 Requirements on Data Quality Management

The examples in Sections 1.1 and 1.2 have shown that these requirements will be exacerbated by the process of digitization. Table 1-2 summarizes the most important requirements for a successful data quality management system, which can be derived from the Top 8 challenges.

Prio.

Challenge

Requirement

1

Data Quality

Definition and measurability: modern data quality management must determine the level of data quality that business processes will require for smooth operation. The concept of “only what has been measured can be managed” applies. The quality of the data must therefore be measured continuously. If there are deviations from expectations, data quality management must introduce measures to increase the quality of the data.

2

Transparency of data usage

Transparency and responsibility: the data lifecycle, starting with its creation in the company and its initial acquisition in an information system, continuing through its use up to its archiving and deletion, must be known and defined according to the requirements of the business processes. Data quality management must control and monitor this lifecycle.

Companies must clearly control the definition of data as well as its use. To accomplish this, responsibilities must be created and assigned within the company. For example, a central office in the Accounting or Marketing departments must define the customer master data for company-wide revenue reporting, so that the same customer can be identified and handled the same in all areas of business.

3

Redundant maintenance of data

Prevention: data quality management cannot first begin after the data is already defective, but rather must act preventively, as with other company assets (for example, preventive maintenance of the production systems or measures for the prevention of illness among employees).

4

Manual maintenance of data

Automation: the volume of data, its diversity and its rate of change are increasing. In order to master the resulting complexity, companies must automate data processing tasks (such as the creation of data, the measurement of its quality, its modification and provision) as much as possible, such as through workflows or business rules.

5

Limitations of central data architectures

Flexibilization and distribution: data architectures define a uniform, company-wide model for the corporate data (the corporate data model) and also determine the distribution of the data and maintenance of the data architecture. Traditionally, these structures have the disadvantage that they can only be created with high levels of bureaucracy and are seldom kept current. Modern data architectures must be able to adjust with sufficient flexibility to new requirements and include both classic intra-company, as well as external, data objects. The challenge exists in making this flexibility possible while also continuing to be authoritative for the primary business objects throughout the company.

6

Semantic integration

Uniformity: corporate data as company-wide master data must be clearly identified and be used uniformly. Data quality management must integrate corporate data to accomplish this. In principle, there are two architectural variations available for this: either the data will be gathered into one system or it will remain in various systems and those systems will be integrated with each other through interfaces and the exchange of data.

Separation of “structured” and “unstructured” data

Uniformity of the data types: In the context of big data, the terms “structured” and “unstructured” are often used in order to describe the uniformity of the types of data handled by the company. This means that, in addition to data that is stored in relational databases in ERP systems, data (such as video clips and images) as well as unconventional, external data (such as from Internet usage or social media channels) will be used increasingly. Such data may supply valuable insights about the market and consumer preferences. However, such unstructured data requires new tools for the analysis of the data and is normally not stored in relational databases.

Data protection

Currently, multi-national companies are being confronted with various specifications for data protection. The data quality management system must make sure that these rules will be maintained. The fact that guidelines are often not transparent, frequently change (which is also unknown) and that no one knows the extent to which they have been implemented in the various systems are sources of problems. For that reason, many companies view data protection as a hindrance, rather than an opportunity.

Table 1-2: Requirements on Data Quality Management

1.4 The Framework for Corporate Data Quality Management

In practice, the requirements indicated above must be considered with respect to the special needs and capabilities of each company, so that data quality management can be successfully established throughout the company. Data quality does not mean quality at any price, but rather quality in accordance with the corporate strategy, the business processes, the structure of the organization and the information systems.

1.4.1 An Overview of the Framework

The framework for the quality management of corporate (master) data offers a solution for these design tasks, by applying the approach of Business Engineering to the company-wide management of data quality (see Figure 1-11). In general, Business Engineering is a method-oriented, model-based approach for designing companies in the information age (Österle and Winter 2003).The “Framework for Corporate Data Quality Management” proposes six design areas for developing artifacts at the strategic, organizational and information system levels (Otto 2011b; Otto et al. 2011). Each design area will have its own types of results (documents).

1.4.2 Strategic Level

The strategy for the management of data quality will orient the management of data quality on the company goals (see Table 1-3).

Table 1-3: Results for the Data Quality Strategy

Figure 1-11: The Framework for Corporate Data Quality Management (Otto et al. 2011 p. 10)

One example for the connection between the management of data quality and the company’s goals can be found at DB Netz AG, which is responsible for the railway infrastructure in Germany. The railway infrastructure includes the network of rails, tunnels, bridges, train stations and so on. An agreement on performance and financing governs the allocation of resources from the Federal Republic of Germany to DB Netz AG in the sense of subsidization of the repair and maintenance work performed on the railway infrastructure.

The amount of the annual subsidy (within certain limits) depends directly on the quality of the land registry for the infrastructure, which includes the registration of the number, maintenance status and certain performance parameters (such as permitted speed) for all infrastructure systems. Therefore, a high level of consistency, actuality, completeness and availability of the master data for the infrastructure system will positively affect the funding of the entire company.

1.4.3 Organizational Level

The organizational level includes three design areas, specifically the Management System area for data quality management (also called “data quality controlling system” or “quality assurance”), the DQM Organization system and the Processes and Methods for DQM.

The management of data quality can only be targeted properly after the meaning of “good (master) data” has been quantified. To achieve this, the quality of the data must be measured. Key performance indicators (KPIs) for data quality are quantitative measures for data quality (Hüner 2011)[11]. The decisive factor when developing a key performance indicator system for data quality lies in ascertaining what should be measured and what can be measured. Key performance indicator systems for data quality must be oriented on the factual requirements and combined with the key performance indicators for business processes to the greatest extent possible. Table 1-4 depicts the results for the Management System design area.

Table 1-4: Results for the Management System

Because the management of master data is an interdisciplinary issue, the tasks of managing the data must be coordinated across the boundaries between the company’s individual divisions and business areas. The organization of the DQM system is intended to serve this purpose. It is a virtual organization in many companies, through which the employees remain in their original chains of command for purposes of discipline and are also associated with an additional technical chain of command (Table 1-5).

Organization of the management of data quality manifests itself in the DQM roles and the assignment of responsibilities to these roles. In practice, a variety of roles have evolved in order to allow the tasks associated with company-wide management of data quality to be taken seriously.

Table 1-5: Results for Organization

In addition to the identification and description of the roles, the responsibilities must be defined. Responsibilities indicate which task areas and rights (such as rights to give instructions, plan, make decisions and employee participation) should be assigned to a role in the management of the master data. For example, the development of a uniform data model for the business objects used interdepartmentally is one task area. The company data steward, who also is responsible for the development of the master data management system, bears primary responsibility for this. In addition to the company data steward, those people responsible for data, who have the necessary specialist and technical knowledge to approve the data model or request improvements, are often defined as additional roles. The data owner is responsible and accountable for certain data objects and generally a management representative (such as the manager of the Central Purchasing or Supply Chain Management departments) for a specialized area.

The fourth design area, DQM Processes and Methods, is based on the management of the lifecycle for master data as well as those processes according to which the employees manage the quality of the data (Table 1-6). One of the most important causes of poor data quality is the lack of overall cultivation of individual classes of master data. Companies are organized according to their roles (such as purchasing or marketing), countries or markets and business processes (such as order-to-cash or make-to-stock). For that reason, only a few companies have an office that maintains an overview of the acquisition, modification, use and deletion of individual pieces of master data.

The task of analyzing causes and effects of low quality master data is very complex for this reason. As a rule, causes are actions that are executed using data in application systems (such as creation, modification, supplementation or deletion). These actions in turn affect business processes whose quality can be quantified by key performance indicators.

Table 1-6: Results for Processes and Methods

1.4.4 Information System Level

The system level covers two design areas, specifically the DQM architecture and the application systems for the management of data quality. Table 1-7 summarizes the results for the DQM Architecture design area. The architecture for the distribution and retention of data describes which data will be stored in which systems and indicates how data will flow between those systems.

The core business object model is a central result of the management of data quality, because it is the prerequisite for a uniform understanding of the data and therewith for the intended use of the data as well. Its development and progress must include the specialist departments, because the knowledge about the significance of the master data in the business processes is only available in those departments. The core business object model will then be transformed into a corporate data model for implementation by the IT department.

Table 1-7: Results for the Architecture of Corporate Data

Ultimately, the landscape of application systems forms the sixth design area for the management of data quality. The results for this design area are presented in Table 1-8.

Table 1-8: Application System Results

This design area deals with the analysis, design, implementation and improvement of those application systems that will be required to support the management of data quality. This includes special master data management systems like SAP NetWeaver MDM on the one hand and software tools for the management of the core business object model on the other hand. When selecting these application systems for the management of master data, issues of data modeling, data quality management, security, usage interfaces, data distribution architecture and, in particular, integration; must be taken into consideration both in regards to the systems as well as to the information to be integrated into them. The Fraunhofer Institute for Industrial Engineering (Fraunhofer IAO) has provided an extensive comparison of the established systems (Kokemüller 2009).

1.5 Definition of Terms and Foundations

The management of master data and its quality is not a new issue. Its penetration into all areas of business through digital services has merely increased its special importance to a drastic extent. Both practice and research have been discussing this issue for as long as corporate information systems have been used to support business processes. Knowing which solutions are already available, which are predicted and which are not has special importance for both research and practical applications. For that reason, the central concepts and terms of data quality management must be defined first. Figure 1-12 depicts the most important terms and their relationships with each other.

Figure 1-12: Conceptual Model of the Terms for Data Quality Management (authors’ illustration)

1.5.1 Data and Information

Data describes the properties of business objects, meaning the material and immaterial object found in the real world (Boisot and Canals 2004). There are in fact many papers about the difference between data and information, however a clear understanding of this term has not yet been accepted (Boisot and Canals 2004; Badenoch et al. 1994). One school of thought understands information to be knowledge that is exchanged during human communication; another takes the perspective of the distribution of information, according to which data is the building block for information (Oppenheim et al. 2003). Accordingly, data must be processed in order to become information (Van den Hoven 1999; Holtham 1995; Wang 1998). According to ISO/IEC 2382-1, data is the formalized representation (meaning ready for further processing, interpretation and communication) of the properties of business objects (ISO/IEC 1993).

The logical organization of data distinguishes a number of levels of aggregation (Chen 1976; Levitin and Redman 1998; Yoon et al. 2000). Data elements form the lowest level of aggregation. Data elements are instantiations of the attributes of data objects (such as a customer’s family name). Records form the second level of aggregation. A record is the instantiation of a data object. For example, a customer master data record is constructed using all of the characteristics of the customer business object such that business processes (marketing, service and accounts receivable) will run smoothly for customers. At the third level of aggregation, tables (such as a customer master table) aggregate multiple records. Databases in turn aggregate several tables. A customer management database could contain all customer master data as well the associated marketing data. The totality of all company databases ultimately forms the corporate data resource. Figure 1-13 visualizes these relationships.

Figure 1-13: Overview of the Logical Organization of Data (authors’ illustration)

This depiction of the organization of data is oriented on the relational data model. In the ideal, semantically unique case, there will be a 1:1 relationship between the world of data and the real world, meaning that one data object represents precisely one business object. In reality however, there are often several data objects that simultaneously represent the same business object. In that case, management of the data quality is required and guidelines for processes and systems must be established, which will allow identification of the “right” data object for a certain context and will provide it for use in business processes.

1.5.2 Master Data

The ISO 8000 standard from the International Organization for Standardization (ISO) (ISO 2009) defines master data as information objects “…which are independent and fundamental for an organization. [Master data] needs to be referenced in order to perform transactions.” Within a company, this data must be identified uniquely and interpreted consistently across multiple organizational units. In this, corporate data is so-called “global” master data that applies to the entire company. In contrast with this, “local” master data applies only, for example, to one business division, one location or one company role. The focus of the methods and tools presented here is on the corporate data. However, because the term master data has entered the general language as well as commercial software solutions, this book will predominantly use this more popular term.

In contrast with transaction or inventory data, master data does not change very often. For that reason, this data is also sometimes called “static data”. In practice, it is not possible to define one generally valid list for all classes of master data. Thus for example, data about contracts is considered master data in the energy and insurance industries, however in contrast is considered transaction data by the telecommunications industry due to the comparably short terms of contracts and frequency of changes.

Bosch, for example, has classified the following data items as master data (Hatz 2008):

· Customers

· Customer hierarchies

· Materials

· Suppliers

· Employees

· Charts of accounts

· Organizational units

In practice, differentiating master data from transaction or inventory data is less relevant than the determination of which individual attributes of a class of master data must be managed by a central master data management department, because one central organizational unit alone cannot assume this task due to the complexity of the individual classes of master data. In principle, the question of which attributes of a class of master data fall within the scope of the central master data management department will be answered by the analysis of the strategic requirements of each individual company.

The following characteristics for differentiation may help with this issue (White 2010; White and Radcliffe 2007).

· Organizational scope: differentiation between local data and “global” data (meaning corporate data used throughout the company)

· Type of data: differentiation between “structured” data (which is typically managed in relational databases) and “unstructured” data like product information (such as images, advertising text and application video clips)

· Location of the definition of the metadata: differentiation between the internal definition of significance, formats and default values on the one hand and external definition on the other hand (such as the country and currency codes defined by the ISO or classification standards like eCl@ss and UN/SPSC)

Master data that has been defined externally is called reference data. Examples of such data include the country and currency codes noted above as well as geo-data. Metadata (literally data about data) describes and defines the properties of other types of data (DAMA 2008 p. 84). Modification data is one example of a sub-class of metadata. This data determines when, how and by whom a specific data item has been modified.

1.5.3 Data Quality

Data quality is a multi-dimensional concept that is dependent on the context (Wang and Strong 1996). Thus, a single characteristic that completely describes data quality does not exist. Rather, there is a variety of dimensions of data quality that describe the quality of data overall. Typical dimensions for the quality of data include[12]:

· Correctness: does the data factually agree with the properties of the real world object that it should represent?

· Consistency: do several versions of the data regarding the same real object to be stored in different information systems, for example, agree with each other?

· Completeness: are all of the values or attributes of a record completely present?

· Actuality[13]: does the data agree with the current status of the real object at each point in time and will the data be modified as the state changes?

· Availability: can data users access the data easily at the desired point in time?

Contextual dependency means that the data quality will be sufficient for a business transaction, but may be insufficient for a different transaction. For example, a customer’s correct delivery address, including the correct shipping ramp, is essential for an automobile supplier’s logistics department in order to ship the contract. For the marketing department in the same company, the correctness and consistency of the delivery address is insignificant, because the company name and country information suffices to allow that department to evaluate the revenue earned with that customer in the previous year. For this reason, data quality is defined as a measure for the applicability of data for certain requirements in those business processes where it is used (Otto et al. 2011).

Data quality will change over time, because the data ultimately represents a snapshot of reality, which will also change over time. For example, customers will move to new addresses and suppliers will change the form of their legal incorporation. Figure 1-14 represents the typical progress of the data quality over time, as found in many companies.

Figure 1-14: Typical Progress of Data Quality over Time in a Company (Otto 2014 p. 21)

Many companies first begin to solve problems with data quality when the quality of the data has fallen below a certain level that prevents business processes from running smoothly. Examples of problems in data quality, which suddenly become visible once the data quality has already fallen below such a level, include problems in migration, a high level of manual activity in the business processes and management reports with different values for the same key performance indicators.

1.5.4 Data Quality Management (DQM)

The mandate of Data Quality Management (DQM) is to analyze, improve and ensure the quality of the data. The Data Management Association (DAMA) defines that all activities, procedures and systems that measure, improve and ensure the suitability of data for use belong to DQM. This perspective again adopts quality management methods from the area of production management (DAMA 2008).

DQM generally differentiates between preventive and reactive measures. Preventive DQM measures target the avoidance of defects in the data with negative effects on the quality of the data. In contrast, reactive DQM measures target the discovery of existing defects in the data and their correction.

The reactive approach has several disadvantages.

· Resources for improving the quality of the data (such as software, expert knowledge and consulting activities) are not planned and budgeted, and may therefore not be available in cases of doubt.

· Purely reactive management of data quality is frequently associated with a lack of measurement of the data quality. In such cases, companies frequently do not have target values, meaning that the question of whether the (reactive) measures for the improvement of the data quality are sufficient, or extend well beyond their desired goals, cannot be measured or determined.

· The total quality management approach (an approach to the management of quality from operations management) has demonstrated that the total costs of all reactive measures in quality management exceed the costs of preventive quality management (Reid and Sanders 2005). This applies to material, as well as immaterial, resources (like data).

The use of automated validation rules (or “business rules”, see the following section) for manual data entry is one example of a preventive DQM measure. The Data Universal Numbering System (DUNS) number is a numeric code used for the identification of companies and is an important piece of master data used by many companies. For this case, services are available that check the validity of the number entered on a form supported by the system and prevent errors by doing so. An example of a reactive measure using the same case is the subsequent cleanup of duplicated data in the same, or various, databases.

The principle goal of companies is to detect data defects before they enter the system as much as possible, in order to avoid risks and costs as a consequence of defective data quality. However, defective data is not the only cause of costs; the DQM process itself causes costs. DQM costs arise both from preventive as well as reactive measures. Analogous to the management of quality in general (Campanella 1999), a disproportionate connection between data quality and DQM costs can be insinuated. Incremental costs for DQM therefore increase.

Figure 1-15: Costs of Data Quality (according to Eppler and Helfert 2004 p. 318)

In contrast to this, the follow-up costs of data defects fall as the quality of the data increases. The ideal total cost of data quality is the minimum of the total function of DQM costs and follow-up costs of data defects (Eppler and Helfert 2004) (Figure 1-15). The task of DQM therefore lies in finding the ideal combination of costs from preventive and reactive measures. In practice, difficulties often arise because accounting typically does account for many DQM costs. This applies in particular to the follow-up costs that originate in various corporate roles and can hardly be quantified in many areas.

1.5.5 Business Rules

The automated validation of business rules is an important tool both for proactive and reactive data quality management. Business rule define the execution and performance of business processes (Ross and Lam 2011). Thereby, business rules formalize the business policies at companies (OMG 2008). They frequently help to articulate the knowledge from employees’ heads so that this knowledge can then be retained for reuse and automation of the IT systems. Business rules are the connecting element between the business process and data management, because in addition to controlling business processes, business rules also serve to measure and control the quality of data. In practice, these rules are sometimes called “validation rules” or “testing rules”.

Simple business rules define the mandatory fields on data entry dialogs, for example. Thus, a simple rule for the creation of customer master data might mean that a customer data record will only be considered complete when it includes the legal form of incorporation (GmbH, AG, etc.). If this entry is not provided, the record cannot be processed further. In the Semantics of Business Vocabulary and Business Rules (SBVR) Standard from the Object Management Group, such rules are called “structural rules” (OMG 2008). A complex example might be a rule that controls the criteria according to which decisions for the further processing of a complaint in the Customer Service department are made. Such a rule is called an “operative rule” in the SBVR. Table 1-9 shows one example each for structural and operative rules.

Table 1-9: Examples of a Business Policy and Business Rules (Ofner 2013 p. 51)

Formal languages like SBVR or Business Process Model and Notation (BPMN) document business rules so that they can be easily implemented as program code and read directly by machines. A rule engine can manage the total quantity of business rules for a company. The task of data quality management is to identify, document, implement and maintain the relevant business rules to ensure a higher quality of data.

Collectively, business rules provide the following advantages for data quality management.

· Automation: they automate parts of the business processes and can accelerate them thereby while also reducing the manual efforts (such as entering default values into IT-supported forms in advance).

· Avoidance of errors: business rules help to avoid cursory and deliberate errors (validation rules can prevent mandatory entries from being forgotten). This is one measure for the proactive management of data.

· Measurement and control of the quality of the data: firstly, records can be checked continuously or periodically for conformance with rules, once business rules have been defined and implemented in IT systems. This is the only manner in which errors in records can be discovered automatically before they cause errors in business processes. This knowledge can initially be used to initiate measures for the improvement of the data quality (a reactive measure for the management of data quality). Secondly, the result of the measurement of data quality using business rules can be expressed as an index (or target value) for the quality of the data. With that value, the change in the quality of the data in a system can be measured over time and, for example, the development of the quality of the data can be compared between two different business units.

· Awareness of DQM: an initiative for the determination of business rules contributes to creating awareness of the importance of DQM in general for participating employees. Doing this increases the chances that employees will also note problems in data quality that cannot be tested using rules.

Naturally, the use of business rules is also associated with several challenges.

· Articulation: because the knowledge of the business rules generally is not found in documents (at least not in one place), but rather is hidden as knowledge in the minds of the employees, the biggest challenge exists in documenting this knowledge[14].

· Reduction of process flexibility: if the business rules are defined too restrictively, they can lead to increased expenditures or truly incorrect entries in seldom-occurring or unpredicted process variations, when employees have to work around the rules.

· Over-evaluation of the significance of the data quality index: a good index for data quality will ultimately indicate that the data measured by all known and implemented rules is correct. However, if these rules are incomplete or obsolete, the level of data quality will be overestimated. As a consequence, business rules must be actively managed. When the reality of business changes, the rules must also be changed. Furthermore, data quality managers must accept that automated validation rules can never detect all problems in data quality. For that reason, the training and continuing motivation of the employees to pay attention to the quality of the data using “healthy common sense” as part of the everyday routine may not be disregarded once the business rules have been implemented.

1.5.6 Data Governance

Data governance is distinguished from DQM. Data governance pursues the goal of maximizing the value of the data in the company (Otto 2011a). Consideration of the value of data as an asset also draws on the transposition of the concepts for the management of material goods to the treatment of data (Horne 1995). Today, both research and practice are discussing whether the value of data for companies can, and should, also be reflected in the companies’ financial statements (Atkinson and McGaughey 2006). In principle, data only has value when it is used. Its suitability for use in turn is defined as data quality (see above). Low data quality reduces the value of the data as an asset to the company, because its usability is also reduced (Even and Shankaranarayanan 2007). Companies are therefore striving to achieve a quality of data required by the business strategy using DQM.

The relationship between DQM and data governance follows the differentiation between governance and management suggested by the ISO (ISO/IEC 2008). In this sense, data governance represents the management role for DQM, because data governance determines which decisions must be made in the treatment of data and who will make them. The task of DQM is to make and implement the decisions relevant to data quality. Figure 1-16 represents this context graphically.

Figure 1-16: Data Quality Management and Data Governance (Otto 2011a p. 236)

1.6 The Competence Center Corporate Data Quality

The Competence Center Corporate Data Quality (CC CDQ) is a consortium research project that is researching, developing and testing solutions for the permanent assurance of the quality of data in large companies.

Consortium research is being conducted in association between research institutions and companies that are working together on an issue of mutual interest. Consortium research has several goals (Österle and Otto 2010).

· Partners in research and practice define the research goals together and evaluate the ongoing work and the results of the project.

· Several partner companies contribute their expertise and guarantee the researchers access to their knowledge.

· The results of the research are artifacts (for example, methods, models or prototypes) that contribute to the solution of the practical problem.

· The design process undergoes multiple iterations and encompasses the cycles of iteration over four phases and several partner companies.

· Partner companies test the artifacts in their operational environments.

· Partner companies finance parts of the project, at least.

· Partners in researchers and practice participate in the project over a significant period of time (two years, as a rule).

· The results of the research are made publicly accessible.

Consortium research proceeds cyclically according to similar principles like Action Design Research (Sein et al. 2011), meaning that the results are continuously measured, developed and tested against the research and practical requirements. Figure 1-17 depicts this process.

Figure 1-17: Consortium Research (Österle and Otto 2010 p. 278)

The members of the consortium meet regularly at workshops and exchange their experiences, discuss open issues and work together on methods, models and solutions. So-called “bilateral projects” subsequently validate the results in a manner individual to the company.

The CC CDQ was founded in November 2006 with the goal of finding answers to the following questions about data quality management.

· What contribution does master data quality make to company goals?

· How can the performance of data quality management be measured?

· What is the position of one’s own company in comparison with others?

· What are the costs and advantages of data quality?

· How do companies establish data governance in their organizations?

· What is the proper level of standards and regulation for the data?

· How can mutual understanding of the master data throughout the entire company be created?

· What is the proper data architecture?

· What contributions do innovative technologies like Semantic Web and In-Memory Computing provide?

Thirty companies have participated in the CC CDQ since 2006. On average, roughly ten to fourteen companies have been active at the same time and membership in the consortium has endured for two to more than seven years. Table 1-10 lists all of the current and former companies that have been partners in the Competence Center since the establishment of the CC CDQ in 2006.

Company

Duration of Membership in CC CDQ

ABB Ltd.

2014 – Present[15]

AO Foundation

2011 – 2012

AstraZeneca PLC

2012 – Present

Bayer AG

2006 – Present

Beiersdorf AG

2010 – Present

Corning Cable Systems GmbH