Key Challenges
• Merge customer identities siloed in various databases.
• Create “golden records” allowing division-level databases to identify specific customers.
• Allow administrator access to override data conflicts.
Abundant Data Means Complexity and Redundancy
In changing business environments, systems and databases evolve to meet increased demands. Business growth leads to increased database complexity and requires infrastructure improvements. Data is then reentered into the new best in class system, leaving room for simple mistakes. As a result, records are often redundant and inconsistent across departmental databases. Inconsistent data affects customer service. Master data management (MDM) is a process that creates a master-level dataset of information. MDM allows inter-divisional communication without requiring the costly integration of division-level datasets.Our client, a large retailer, needed a way to merge siloed customer identities. Because customer identities were not tracked across business divisions, sales and marketing teams could not coordinate activities. For example, customers who purchased a product still received promotional emails to buy the same product. Product sales information was not used to refine sales and marketing efforts. In some cases, even the departments had duplicate customer records.
Our Approach
Our client needed to improve their data quality. We first created reporting copies of each dataset. Then we created “golden records”—master customer identities that enable division-level databases to identify unique customers.To create the golden records, we extracted each of the ten divisional databases. Then, we cleaned and profiled the databases. Next, we matched the various identities using customized rules, fuzzy lookups, and machine learning techniques. Next, we deleted duplicate identities and standardized customer identity values. Machine learning algorithms then identified similar identities and linked them. Now one customer identity acted as the master record. We then attributed the master record to the customer identities within the siloed divisional databases.
MDM is now automated and occurs on a recurring basis. Golden records are added and refined to keep pace with changes in the transactional data. The solution uses data about customers across various systems to improve understanding of the customer. Data security and privacy requirements are still observed. The solution is entirely compliant with the regulations established by the General Data Protection Regulation (GDPR).
We also built administrative modules to allow users to address cases where machine learning failed. Matching rules, thresholds, and other system parameters are managed within the administrative module.