Thursday, November 19, 2020

Engineer Product Features in Real Time Based on Customer Feedback

Business Case:

In today’s consumer-centric environment, the best way to improve a product is by responding to customer feedback. A typical customer feedback cycle requires cross-team communication between support, sales, and engineering. The sales team offers insights into product features that receive recurring feedback; the engineering team has the technical expertise to implement changes and offer support. Responding quickly to feedback is critical to ensure customer satisfaction.

Our client, a global technology giant, stores support ticket information in Azure DevOps. Our client’s sales team receives thousands of support cases a day, but only converted 600 cases a year into engineering features. Without an effective classification and communication system, feedback went unresolved. Although the sales team was logging customer feedback, they couldn’t connect with the engineering team, who often didn’t know the feedback existed. In partnering with us, our client needed to bridge the gap between their sales team and engineering team.

Key Challenges:

  Define a feedback triage process for the engineering team 
  Effectively share feedback between teams  
  Create a reporting system where engineers can access all relevant feedback data  
  Reduce feedback response time 

Our Solution:

We developed a two-part solution to connect our client’s sales and engineering teams: 
1.  Establish a feedback triage process 
2.  Create a Power BI dashboard that pulls feedback information from the engineering DevOps 

Figure 1: Solution Design

The triage process begins by pulling customer feedback from two different sales portals into a single DevOps workspace. During level one triage, the engineering team verifies the feedback Work Item is complete with pre-defined metrics such as impact, customer scenario, and description. If the Item lacks key information, the team sends it back to the requestor.  

As part of level two triage, the engineering team meets weekly to assess feedback Items. If an Item needs additional information, the team sends it back to the requestor. Otherwise, the team tags the feedback as “Evidence” and attaches it to an engineering Feature. Engineers respond to the Features directly through DevOps. One Feature might contain dozens of Evidence Items based on repeat customer feedback. To ensure Features aren’t duplicated, we created a separate table in an Azure Databricks notebook that maps new feedback with existing Features.  

Figure 2: Engineering Feedback Dashboard Home Page (using sample data)

With a triage process in place, we then developed a dashboard where engineers could view and prioritize all open Features. The dashboard pulls data from DevOps using API calls in Azure Databricks. Our solution uses custom calculations to derive key metrics from the data. Once transformed, data is pushed to Azure SQL tables that populate the Power BI dashboard. The dashboard includes information such as revenue impact and customer profile so engineers can effectively prioritize feedback response. Engineers can group data by sector, product type, country of origin, and many other metrics using the dashboard’s multiple tabs.  

Figure 3: Feedback by Revenue Impact Drilldown (using sample data)

Business Outcomes:

Our engineering triage process and Power BI feedback dashboard enable our client to close the gap between sales and engineering. With an effective triage process, the engineering team can quickly access relevant data. To increase response speed, we group repeat queries under a single Feature. With numerous impact views, the dashboard enables engineers to effectively prioritize tasks. Customers receive faster responses to high-priority feedback.  

Increased accountability improves the team’s ability to collaborate on technically complex feedback. Our client can continually enhance products and services based on customer requirements. Effective feedback response enables our client to build long-lasting customer relationships and identify potential revenue opportunities.  

Since implementation, our client has created over 1K feedback Items, increasing DevOps usage by 66% in under five months. Over 300 of these items have been converted into Features for the engineering team to work on. The dashboard is used across the engineering team, with over 1K views each month.


    Developed an engineering triage process and Power BI dashboard that enable the engineering team to respond to customer feedback
    Increased use of the engineering DevOps workspace by 66%
    Enabled engineers to prioritize customer feedback with a dashboard that has over 1K monthly views
    Reduced feedback response time