Improving Master Data Quality: A Data Quality Analyst’s perspective – Part 1 of 3

This article is part of a three-part series which addresses the end-to-end process of improving Master Data, from the perspective of a Data Analyst. This series covers the following topics across three articles:

Part 1: Roles required and identification of Data Quality issues

Part 2: Data profiling and assessment, and

Part 3: Data cleansing.

In general all initiatives with an objective of improving master data quality should tackle the problem from both a bottom up as well as a top down approach. Simply speaking, bottom up (correction) is all about correcting data quality issues in existing data, whilst top down (prevention) puts measures into place to prevent issues from re-occuring over and over again. It is pointless to spend time and effort on cleansing data if poorly designed or managed processes and systems continue to allow low quality data back in on an ongoing basis.

For the bottom up or corrective component, due to high volumes of data and/or complexity of data quality issues, it is frequently necessary to employ programmatic cleansing methods, sometimes using specialised data quality technology platforms. Of course there will always be those data quality issues that will not be able to be resolved without controlled human intervention, but the trick is to minimise this by ensuring that the programmatic approach resolves as muc as possible.

This series of three articles focuses primarily on the bottom up programmatic component.

In a Master Data Improvement (MDI) project, the Data Quality Analyst has a role to play in all stages of the Data Quality Improvement process.

A simple definition of Data Quality Improvement is:"...a process of measuring the data quality by assessing it and then correcting the data. By creating a repeatable process, trends can be monitored over time."

The typical roles needed for an MDI project are; Data Architects, Analysts, Developers, Project Managers and Business Data Stewards. Each role can have one or more people assigned to it, but generally, there would only be one Project Manager and one Architect. In smaller projects, however, it’s possible to have a person assigned to more than one role.

Let’s explain how these roles would work by using a building/construction project as an analogy.

The Data Architect:

Much like in a building project, the Architect draws up the plan with all the components and how they all fit together. This plan needs to be approved by all stakeholders (like the homeowner in a building project). The Architect then oversees the construction of the building to the required specifications.

The Data Quality Analyst:

The Data Quality Analyst methodically plans the work to be done through analysing, defining, documenting, quality controlling and reporting on all processes, much like the builder in a building project.

The Developer:

The Developer does the ‘hard work’ and labour, technically implementing through code the different aspects of the project per the documentation provided by the Data Quality Analyst and Architect. The Developer is like the Brick Layer, Plumber or Electrician in a building project.

Project Manager:

The Project Manager ensures that everything works according to plan, within a specified time and budget. He/she also ensures that the project team has what they need at any given time during the project. The Project Manager liaises with the project team and the business to resolve matters as they arise, resembling much of the same duties as a Project Manager in a building project.

Business Data Steward:

The Business Data Steward understands the data and works closely with the project team and Data Owners. He/she is ultimately responsible for ensuring quality data and therefore needs to approve all the rules defined by the Data Quality Analyst before the rules are implemented and also verifies the rules post implementation. The Business Data Steward monitors Data Quality trends and manages the ongoing improvement process after the project has completed, much like a Care-Taker that would look after the property once the building project is completed.

Identifying Data Quality issues

In a Master Data Quality Improvement project, the first step in the process involves identifying the primary data domains (e.g. customer, product, supplier) and then investigating each area. The subsequent process for each data domain is the same.

The steps involved in identifying and resolving data quality issues are:

1. Gathering requirements
2. Data profiling
3. Data quality assessment

• Gathering requirements
• Discover business pain points
  Companies choose to improve the quality of their data for a reason. For the Data Analyst, the trick is to decipher the business problem down to the data characteristics that contribute to the problem. For example, if the customer delivery address is incorrect, then goods would be delivered to the incorrect address. This low-quality data would impact cost, delivery times and customer satisfaction.
• Define data standards
  Companies typically have more than one source application where data is captured. It is, therefore, essential that the Data Quality Analyst defines and documents the data standards of all capture fields where Master Data is captured. This is done together with the Business Data Steward. Naturally, one needs to know for each data field on a system if the value is optional, mandatory or even not applicable. This process evolves as the project progresses, and more information becomes available. The data standards document is critically important for the Data Analyst; it helps to identify business rules and the related assessment and cleansing rules. This document is also useful to business in improving data capturing procedures.

Continue to read Part 2 of 3: Improving Master Data Quality from a Data Analyst’s perspective here.

Need to discover how we can help your organisation attain business success through well-managed data?

Please get in touch: infoblueprint.co.za/contact