Thursday, October 3, 2019

Data Dictionary for Dynamics 365

Business Scenario

Dynamics 365 is a cloud-based business applications platform. It combines enterprise resource planning (ERP) and customer relationship management (CRM) components with productivity applications and artificial intelligence (AI) features. Out of the box, Dynamics 365 offers diverse tools, but complex business environments often require custom components. These components, called solutions, enable businesses to customize and expand Dynamics 365 features to meet specific business needs.

Each solution is deployed as a single zip file. The zip file contains all the objects (entities, workflows, processes, roles, etc.) present in the environment. The file includes both pre-existing objects and objects that are newly added as part of a current development cycle.

We recently launched a Dynamics 365 implementation for a large software company. The project contains more than 200 entities and several processes and workflows associated with each entity. In Dynamics 365, entities are used to model and manage business data for Customer Engagement apps.

The software company’s Dynamics 365 implementation is still evolving. Sprint-over-sprint we are adding new features and incorporating change requests. These changes include adding new entities or updating or deleting existing entities and their attributes (e.g., field name changes, character length changes, or adding new workflows).

When solutions in large Dynamics 365 projects (such as the project detailed above) are updated, team members must track the changes. Tracking changes allows:

1. The internal team to review and monitor changes before deploying to production.
2. Administrators of downstream systems to anticipate changes that may affect their ETL (Extract-Truncate-Load) jobs.
3. Devs to collaborate on solution updating efforts.

Dynamics 365 does not maintain a subversion of a solution, which would help identify what changed in an updated release. Due to the lack of subversions, it becomes difficult to highlight release over release changes and share them with the larger audience.

We needed to develop a way to keep track of solution changes in a Dynamics 365 environment across multiple releases.

How We Did It: The Technical Implementation

We created a utility called Data Dictionary that accepts two Dynamics 365 solution files as inputs. Data Dictionary compares two versions of a solution (previous and current version). The utility processes the solution files and generates an output in the form of three Excel files. Figure 1 shows Data Dictionary’s comparison flow.

Figure 1: Utility comparison flow diagram

Excel Output Files:

1. Excel file (previous version) - Provides schema of the previously deployed solution.
2. Excel file (current version) - Provides schema of the current solution. This includes the objects that were there in the previous version and those introduced in the current version.
3. Excel file (delta) – This file is similar to the Excel file (current version) but with all the changes highlighted to clearly identify what has been added, modified, or removed from the system in the current version.

Key Highlights:

1. Helps the internal team review (at a glance) what has changed in a solution before deploying the solution to production.
2. Communicates changes between releases to downstream systems. Knowing the changes allows downstream users to update their ETL (Extract-Truncate-Load) jobs, ensuring that they report on the latest data points.
3. Improves dev team collaboration by allowing them to quickly spot differences between two solutions versions.

Business Outcomes:

Data Dictionary saved significant manual effort in identifying and communicating changes in solutions releases. We run Data Dictionary after each major Dynamics 365 deployment and share the output files with key stakeholders.