Wednesday, November 7, 2018

Case Study: Improving Customer Support with Machine Learning



Key Challenges

   Create a machine learning model that recommends results from previously resolved customer support tickets.
   Supply the confidence rating of results related to the current support ticket.
   Enable newly onboarded support staff to quickly find answers to frequently asked customer questions.

Anticipating Customer Support Needs

In today’s era, customers increasingly expect companies to anticipate their needs. Being able to respond quickly and accurately to support requests is more important than ever. While loyalty is best won by exceeding expectations, strong customer support is necessary when something inevitably goes wrong.

Our client, a large software company, needed a revamped support system for their partner portal. The portal hosts 60,000 users each day. Before our work with the client, the partner portal had up to 100 active support tickets at any given time. Customer support struggled to keep up with the number of support requests. Because of the high number of support requests, however, certain questions were often repeated. To better address repeat questions, the support staff created a knowledge base of resolved tickets. At the time, the knowledge base simply consisted of a large text file where questions and answers were stored. The support staff was able to search for answers to active support tickets within the database, but they often had to wade through hundreds of results of varying relevance.

Our Process

The client asked us to develop a machine learning solution that would reduce the amount of time it took to get new support staff up to speed and help answer support questions quickly and accurately.

We immediately got to work developing a machine learning model. We knew that the quality of the model’s responses would depend on the quality of our data. We trained the machine learning model by asking support staff the following question: “To which category does this support ticket belong?” We used the feedback to categorize the 100 active support tickets and build a knowledge base. We also manually tagged the resolved issues in the support database.

Early results from the machine learning implementation were not optimal. Only 30% of results were relevant to the support questions. After some investigation, we concluded that the data needed additional cleansing. After making the necessary improvements, the success rate rose to above 80%.

Better Results for the Staff and Customers

The improved machine learning model yielded immediate results. The client’s active support ticket count now hovers around zero. But the project's lead programmer and analyst warns against celebrating purely based on these results. The analyst stresses that an improved user experience is a better measure of success than relying on numbers. “Our focus is always on the service that we provide to the end user. We evaluate how effective we are by measuring how many times a ‘resolved’ ticket bounces from a user back to support. The active support ticket count is not how we measure the effectiveness of our work.”


Our machine learning model also improved the customer support onboarding process. Before the system was implemented, there was a significant disparity between the time it took for an experienced employee to resolve a support query and the time it took a new employee. While a new employee will always be slower than an experienced employee—learning does take time, after all—the disparity between the two has significantly lessened.

Our client is now able to leverage their knowledge base to resolve support requests quickly and accurately. The machine learning model generates a list of related resolved issues, as well as confidence ratings, that the support staff can rely on. Our implementation eliminated the need for the support staff to manually search through large text files, allowing the staff to create a better customer experience.