Monday, August 26, 2019

Unsupervised Machine Learning Model Improves Customer Feedback Analysis



Business Scenario

Our client showcases a variety of technology products in their retail stores to customers. Product demonstrations highlight product capabilities to drive sales. Following product demonstrations, the voice of customer team collects customer feedback. Customer feedback provides a better understanding of how improved product demonstrations improve sales.

To capture customer feedback, the voice of customer team reviewed forms, written comments, reviews, ratings, and surveys using a supervised machine learning model. The team trained the supervised machine learning model using textual data from other teams they worked with. But the supervised machine learning model approach struggled whenever the voice of customer team wanted to add a new dataset. Because the model was based off the provided training materials, the model could not identify material themes that did not originate from the training materials. As a result, each time the voice of customer team worked with a new dataset, the learning model required retraining. To minimize training costs and improve scalability, we needed to create an unsupervised machine learning model.

How We Did It: The Technical Implementation


Figure 1: Unsupervised machine learning model
To create the unsupervised learning model, we researched three topics:

   Theme identification using supervised models
   Term frequency-inverse document frequency (TFIDF) methodologies
   Key phrase extraction

At the beginning of the project, we conducted research to determine how to implement an unsupervised machine learning approach for theme identification. Prior to the completion of our project, no such research had ever been conducted. We started our research with supervised models, even though our solution needed to eliminate manual theme identification. Studying supervised models enabled our team to better understand how human heuristics guided machine learning. To better understand how to track the statistical significance of themes, we studied TFIDF methodologies. By studying supervised model results and TFIDF methodologies, we eventually discovered the key insight we used to structure our unsupervised machine learning model: We could extract key phrases in English using a recurring pattern: (Adj/verb)*(Noun/Proper noun). After discovering the pattern, we worked on several proofs of concept using the pke and rake libraries and Microsoft’s Cognitive Services.
   
The original implementation of the project involved extracting key phrases from customer reviews and feedback comments. First, we divided feedback into different sentences using full stops and spaces. Then, we used several text preprocessing techniques to extract the core message without contractions, punctuation, stop words, conjugations, or spelling mistakes. We then employed text processing techniques to extract the key phrases. Key phrases were identified using nouns and adjectives. (In product feedback, comments typically used the product as the noun and the characteristics of the product as adjectives.) We then used the Word2Vec model to convert the key phrases to vectors. The resulting vectors were then passed through an unsupervised learning engine to cluster the key phrases in order to decipher feedback themes. Each cluster helped identify feedback themes. Finally, we performed a T-test to identify the significant themes. Our approach enabled us to identify newly emerging feedback themes without requiring model retraining.

We fed data to our machine learning model through Azure Databricks. Azure Databricks offered our client the speed and flexibility they were looking for. Azure Databricks allows users to run robust analytics algorithms and drive real-time business insights. Azure Databricks also offers one-click, autoscaling deployment that ensures enterprise users’ scaling and security requirements are suitably met. Azure Databricks also features optimized connectors, which we used to run Microsoft Cognitive Service APIs. These APIs allowed our team to quickly implement entity recognition and key phrase extraction. Because the Azure Databricks solution was managed from a single notebook, our teams could collaborate easily across office locations.

Business Outcomes:

The completed unsupervised machine learning model enabled the voice of customer team to scale the solution to new datasets immediately. Azure Databricks further enhanced scalability and improved deployment speed. With the finished solution, our client can identify themes not contained within training materials and receive significant themes without delay. The completed solution also reduced the manual effort required to identify and map themes.

Thursday, August 22, 2019

Infographic: 14 Best Practices for Power BI


Wednesday, August 14, 2019

Integrated Dynamics 365 Workflow Reduces Manual Effort and Expedites Communication



Business Scenario

Our client displays a variety of technology products to customers in their retail stores. Product demonstrations showcase product capabilities and drive sales. Following product demonstrations, the voice of customer team collects customer feedback. Customer feedback provides insights into how to improve product demonstrations. Improved product demonstrations, in turn, lead to increased sales.

Before we implemented our solution, retail customers provided feedback via phone calls. Our client used a SharePoint system to capture product demonstration feedback. But collecting feedback over the phone and using SharePoint presented a significant challenge. Our client conducted product demonstrations for two different teams without a standardized workflow. The first team, the customer success team, ran product demonstrations to determine how existing customers could use products and offerings. The second team, the enterprise business team, attended product demonstrations to better understand products and capabilities. Both teams needed to coordinate with retail store associates and customers to plan product demonstrations.

To set up product demonstrations, customer success teams and enterprise business teams independently called store associates. No centralized system existed for teams to plan product demonstrations and avoid double bookings. After product demonstrations, customer success team members or enterprise business team members called customers to receive feedback. Because the feedback system was not automated, eliciting customer feedback often required great effort. After receiving feedback, team members manually entered the feedback into SharePoint, which required significant time. Team members could then log in to see feedback for all product demonstrations. Our solution needed to streamline our client’s workflow. A streamlined workflow would deliver two benefits:

   Reduce manual effort
   Expedite communication

How We Did It: The Technical Implementation

Recently, Microsoft released a version of Forms Pro allowing Forms to integrate with Dynamics 365 and Microsoft Flow.  We were among the first to implement a solution leveraging integration support between Microsoft Forms Pro, Microsoft Flow, and Dynamics 365. We used the integration support to standardize retail store customer feedback.

We built the pipeline using the Power Platform and Forms Pro and stored customer feedback data in Dynamics 365. We created a Microsoft Flow architecture using the Power Platform. We used PowerApps to capture the details of the planned product demonstrations, generate invitations for the demo events, and capture the feedback in Dynamics 365. Using Microsoft Flow and Forms, we generated the documents to capture the feedback from the demo sessions. Responses from Microsoft Forms route to Dynamics 365, where they are stored under the respective event, customer, or sales leader. We used Microsoft Flow, which is integrated with Azure Active Directory, to create the product demonstration requests.

The completed workflow uses no code. The workflow leverages Microsoft Flow and Forms's service-based back end technology. As a result, the platform is easy to scale and maintain. Workflows are updated by editing or adding templates, avoiding complex code changes. Using the platform, sales leaders create requests for in-store product demonstrations. Store managers then either approve, modify, or reject the request. Forms Pro also enables store managers to share customer feedback. In addition, because customer feedback from Forms Pro is linked to the originating event in Dynamics 365, sales efforts and outcomes are directly attributed to customer feedback.

Business Outcomes:

   Reduction of manual effort
   Expedited communication

The integrated solution using Microsoft Forms Pro, Microsoft Flow, and Dynamics 365 improved our client’s workflow at two junctures. First, by changing product demonstration scheduling to an electronic process, we reduced the time needed to schedule product demonstrations. Second, by automating the creation and entry of surveys, we reduced the amount of effort needed to input survey data and improved communication speed. The completed integrated workflow also connected sales outcomes to customer feedback from product demonstrations. 

Tuesday, August 13, 2019

MAQ Software Releases Award Winning XPRIZE Apps for the Indian Subcontinent in English, Hindi, Urdu, and Bangla


Available in English, Hindi, Urdu, and Bangla, the free Android apps engage children with literacy and numeracy lessons

August 16, 2019, Redmond, Washington: Advancing foundational literacy and numeracy skills, MAQ Software today launched Pehla School, Pehla Class, and Simple Class based on three Global Learning XPRIZE award-winning apps. The highly engaging apps, available for Android tablets and phones, empower children ages 3-10 to improve reading, writing, and math skills.

The apps are available immediately on Google Play:

Pehla School (based on Kitkit School - Global Learning XPRIZE co-winner)
   Pehla School (bit.ly/pehlaschoolenglish)
   पहला स्कूल - Pehla School (bit.ly/pehlaschoolhindi)
   Pehla School - پہلا اسکول (bit.ly/pehlaschoolurdu)
   পহেলা স্কুল - Pehla School (bit.ly/pehlaschoolbangla)

Pehla Class (based on onecourse - Global Learning XPRIZE co-winner)
   Pehla Class (bit.ly/pehlaclassenglish)
   पहला क्लास - Pehla Class (bit.ly/pehlaclasshindi)

Simple Class (based on Chimple - Global Learning XPRIZE finalist)
   Simple Class (bit.ly/simpleclassenglish)
   सिम्पल क्लास - Simple Class (bit.ly/chimplehindi)

Our Mission Statement

Over 200 million students in the Indian subcontinent are underserved. While over 95% of children in India are enrolled in schools, many students face significant learning gaps. According to the ASER 2018 report, only half of Class 5 children in India can read at Class 2 level. Only 28% of Class 5 children can do basic Class 2 arithmetic.

With low cost internet access and advances in hardware and software, learning apps are an opportunity to improve learning outcomes worldwide. Google showcased the power of new technology, using advances in Natural Language Processing (NLP) for speech recognition in its free Bolo with Google application. Bolo with Google improves literacy skills in Hindi and English and has been downloaded more than 500,000 times from Google Play.

Based on the open source software developed by the winning teams of $15 million Global Learning XPRIZE, MAQ Software expands free learning app offerings for the Indian subcontinent. MAQ Software optimized the suite of XPRIZE-based learning apps for large scale deployments. Using advances in text to speech implementation in Android phones, MAQ Software will continue to improve the suite of learning apps to utilize new versions of Android-based software and hardware.

By supplementing existing digital tools, school systems and non-profits can improve learning outcomes dramatically through these apps. MAQ Software is providing technical expertise and grants to enable the non-profits to advance their mission.

About the Global Learning XPRIZE

In 2014, the Global Learning XPRIZE launched a $15 million education technology prize. Funded by Elon Musk, the Global Learning XPRIZE challenged participants to develop solutions that enable children to learn reading, writing, and arithmetic within 15 months. Out of the initial pool of nearly 200 applicants, XPRIZE selected five finalists (Kitkit School, onebillion, Chimple, CCI, and RoboTutor) to receive $1 million each.

In field testing, the apps generated remarkable learning outcomes. For example, students who used Kitkit School showed an average reading test improvement of 30–45% and an average math test improvement of 14–37%. Students who did not use the app showed much lower or no change in their test scores.

The Global Learning XPRIZE winning apps help kids: 

Learn on their own: Kids learn reading and math skills through thousands of fun activities.

Personalize instruction: Kids can choose from a variety of reading and math lessons. The apps include hundreds of literacy sessions, math sessions, multi-level books, and videos.

Read without distractions: All the apps work offline without ads, allowing children to focus on the current lesson.

Discover many topics: The apps cover a variety of engaging topics, including animals, science, friendship, and role models.

Showcase creativity: The apps include virtual instruments, a blackboard to practice writing, and drawing and coloring tools to create unique artwork.

About MAQ Software

MAQ Software delivers innovative software solutions to Fortune 500 companies. Using data analytics, cloud technology, and artificial intelligence, we accelerate software initiatives that enable our customers to transform their industries. We help our clients build stronger customer relationships, drive sales, identify new customers, and analyze marketing results from our three state-of-the-art engineering centers located in Washington state and India.

Thursday, August 8, 2019

Azure Services Improves Power BI User Experience



Business Scenario

Our client, a large international software company, adopted Power BI across their entire organization. As a result, many internal project managers and sales managers moved their reports to Power BI. The managers’ workspaces utilize hundreds of reports. Because the workspaces contain so many reports, organization is a challenge.

Our client asked us to create a solution for their Power BI reports to solve three challenges:

   Report organization
   Filter management
   PowerPoint creation

The first challenge involved report organization. Because of the way Power BI functions, our client could not divide reports into team subcategories. In addition, team members could not easily organize their own requirements. As a result, team members needed extra effort to locate relevant reports. For our solution to succeed, users needed to find report organization intuitive.

The second challenge involved filter management. Filters applied on one report were not automatically applied to another. Team members could not globally manage filters. As a result, team members who wanted to know about data from France, for example, needed to apply the France filter to each individual report. Only then could team members analyze the data. Before the implementation of our solution, our client used OneNote to document reports for role-specific users. Users then used the documentation to select the correct report and apply the right set of filters. To improve user experience, our solution needed to enable global filter application.

The third challenge revolved around creating PowerPoint decks. Each quarter, business leaders within our client’s organization met to discuss sales and marketing goals. To prepare for the meeting, the business leaders created a PowerPoint using visuals from Power BI reports. In the client’s old process, business leaders needed to:

1. Manually screenshot the report
2. Save the image
3. Move the image to the PowerPoint
4. Format the image correctly

Effort to create PowerPoint slide decks routinely required one month per quarter. The cost of running the VMs each month to generate the slide decks on demand regularly exceeded $10,000. Our solution needed to minimize the VM cost and the time spent creating the PowerPoint for the quarterly meeting. 

To address the three challenges, we implemented a three-part solution:

   Report collections
   Global filter panel
   Export to PowerPoint tool

To improve report organization, we created report collections driven by user roles. Admins can create the collections, and admins can also embed reports on the website. Admins also determine how collections are presented to users. Admins use business capability, focus area, area grouping, role category, and role to determine who sees the Power BI reports. Reports then appear in the appropriate workspace and act as a one-stop shop for the designated users.

To improve filter management, we created a global filter panel. The global filter panel enables central management of all filters across all reports. Individual users can use the global filter panel to customize the filters applied to their reports. As a result, users do not need to reapply filters each time they navigate to a report. Users can also favorite filters.

To reduce the time needed to export Power BI visuals to PowerPoint, we implemented our own Export to PowerPoint tool. Using the tool, users can browse Power BI reports as a gallery of visuals. Once users identify the relevant visuals, they place the visual in an online representation of the PowerPoint slide. Users can choose both out-of-box and custom visuals to include in their slides. When the slides are complete, the tool exports snapshots into a PowerPoint slide deck, complete with custom layout. Our Export to PowerPoint tool reduced the time needed to create the quarterly PowerPoints from one month’s effort to a half hour.

How We Did It: The Technical Implementation

To build the role selection capabilities, we created an Azure web application with embedded Power BI reports. The application connected to an Azure database that housed the report collection and role definitions. The setup allowed admins to add additional reports to the collection through the web interface.

We built the global filter panel using HTML5 controls and integrated the panel as part of the web application. Integrating the panel enabled us to filter selections through the user session and save favorite filter selections. Favorite filters empowered users to customize the default views of the reports to suit individual needs. Global filter selection is passed to the embedded reports, filtering the displayed data within each report.

To generate PowerPoint exports, we used Azure Kubernetes Services to orchestrate two containers. (See Figure 1.) One container loaded the reports in a headless browser (Puppeteer) and captured snapshots of the visuals in the reports. We saved the captured images to Blob storage. We used the other container to generate the PowerPoint file from the selected images and save the file to Blob storage. Once a user successfully downloads the PowerPoint file, the file and images are deleted from Blob storage. 

Figure 1: Export to PowerPoint tool architecture

Business Outcomes:

   Improved user experience by filtering reports based on role
   Empowered users to create custom report collections
   Reduced effort by allowing users to filter reports globally
   Decreased time required to produce PowerPoint slide decks from days to minutes
   Lowered costs by implementing a pay-per-use architecture

The completed Azure web application resulted in several benefits for our client. The custom report collections significantly improved user experience. Users no longer had to search to find relevant reports. The global filter also improved user experience. Using the global filter, users could apply the same filters across multiple reports, saving significant amounts of time.

The Export to PowerPoint tool resulted in the most influential benefits for our client. Using the Export to PowerPoint tool, our client reduced the time needed for creating PowerPoints from one month to a few hours. The actual time needed to export the PowerPoint decreased as well, from several hours to five minutes. Most importantly, because the completed Azure web application did not require VMs to constantly run, our client saved over $10,000 per month.

Wednesday, August 7, 2019

Power BI Dataflows Reduces Cost and Saves Time



In today’s business environment, executives require total visibility into company activity. In order to provide visibility, team members must either provide reports or personally convey the desired information. Both reports and meetings require significant time.

Business Scenario

Our client, the business development (BD) team of a global technology company, needed to provide the executives with immediate insight into sales leads at the geography level (Asia Pacific, Western Europe, United States, etc.). Prior to the implementation of our solution, the BD team used Trello to track sales leads and Dynamics 365 to convert sales leads into a deal. Throughout the sales process, the BD team manually entered notes about sales status. But because sales status was tracked on multiple platforms, providing visibility into sales status proved difficult. To present a consolidated view of sales status, a BD executive spent hours creating an Excel-based report for the executives.

To provide the executives with immediate insight into sales status, we created a Power BI dashboard. The Power BI dashboard shows current sales leads, ongoing sales discussions, and finalized sales. The dashboard also utilizes a tagging system, which converts unstructured text into actionable business KPIs. The dashboard also displays whether a sales lead is “in the red zone.” If a sales lead is “in the red zone,” no one has pursued the lead in over six months. The dashboard’s “red zone” feature allows the executives to track sales lead progress and follow up with the appropriate geography lead with minimal effort. The tagging system allows the executives to understand what business actions typically lead to successful sales. 

How We Did It: The Technical Implementation

Before we implemented our solution, the BD team used Microsoft Flow and the RESTful API to move sales data from Trello to Azure SQL Database. (For a visual representation of the previous architecture, please refer to Figure 1.) To track sales data from Dynamics 365, BD team exported the data to Excel. The BD team used Azure Blob Storage and SharePoint for data refreshes. Next, they  manually imported the Excel data into Azure SQL Database. Azure SQL Database functioned as a data storage location. Data was then pulled to create Power BI reports.

Figure 1: Old architecture

When we were approached to redesign the architecture, we challenged ourselves to address four drawbacks of the existing architecture:

   Azure Database, Azure Blob Storage, and SharePoint incurred a significant cost.
   Microsoft Flow and the RESTful API incurred significant maintenance costs.
   The existing architecture required security approval and environmental setup from a support team.
   Daily refreshes involved manual effort.

To address the four drawbacks, we redesigned the architecture almost exclusively using Power BI dataflows. (For a visual representation of the new architecture, refer to Figure 2). We used dataflows to ingest, transform, integrate and enrich the sales data by defining data source connections, ETL logic, and refresh schedules. By using Power BI dataflows, our client unified data processing for sales data from Trello and Dynamics 365. In addition, using Power BI dataflows resulted in an almost no-code architecture. Dataflows is a centralized service, resulting in uniform ETL logic across the company and alleviating the need for costly infrastructure and server maintenance. The completed Power BI dataflow architecture addressed the four drawbacks, resulting in the following benefits:

   Eliminated Azure Database, Azure Blob Storage, and SharePoint costs.
   Eliminated Microsoft Flow and RESTful API maintenance costs.
   Eliminated need for support team.
   Eliminated need for manual refreshes.

Figure 2: The current architecture

Business Outcomes:

The completed Power BI dataflow solution resulted in several business and technical benefits. Using the new Power BI dashboard enabled faster surfacing of business insights, greater executive visibility into sales activity, and eliminated the effort needed to create the Excel-based report for the CEO. With the Power BI dataflow architecture, our client dramatically improved cost and time savings.

Friday, August 2, 2019

Delivering Data Faster: The Five Phases of Data Migration to the Cloud



Migrating Data Systems

Data systems are integral to business success. Data systems generate insights that drive sales and marketing. Migrating data systems to the cloud offers many benefits. Cloud architectures improve refresh speed, consolidate resources, and improve data uniformity. But data migration requires transferring terabytes or even petabytes of data. Data migration also requires two production environments to run in parallel. Because data migration involves carefully navigating multiple logistical challenges, it requires a deeply considered plan.

In this article, we will detail our data migration process. We will examine data migration in five phases:

1.   Discovery Phase
2.   Analysis and Design Phase.
3.   Planning and Approval Phase.
4.   Execution Phase
5.   Verification Phase.

Discovery Phase

During the discovery phase, we evaluate our client’s existing data system. Every client has distinct needs. We look at sources, the data itself, size, scale, refresh frequencies, and source relationships.

In our most recent data migration project, we partnered with a global technology company to address the following four challenges:

1.   High data refresh and process times
2.   Server dependencies among multiple streams
3.   Difficulty managing and maintaining infrastructure
4.   Difficulty patching infrastructure

On average, our client’s data required 24 hours to pull and process. Our client’s existing solution to slow processing times was simply to upgrade servers. Continuously upgrading servers resulted in a significant infrastructure cost.

By migrating their data, our client hoped to meet four main objectives:

1.   Enable real-time reporting
2.   Facilitate independent refreshes of multiple streams
3.   Easily manage and maintain infrastructure
4.   Decrease costs

Analysis and Design Phase

We divide our analysis and design phase into three steps. First, we research potential applicable technologies. Second, we read documentation to determine which technologies we want to use. Finally, we perform a series of proofs of concept to determine technology suitability and cost-benefit.

In our most recent data migration, we created the following proofs of concept (for reference, please refer to Figure 1):

•   Azure Analysis Services
◦   We conducted a proof of concept using Azure Analysis Services to optimize processing through Azure Data Warehouse, Azure SQL Database, and Azure Data Lake Services. Specifically, we examined how multi-partitioning and file splitting affected optimization.
•   Azure Data Warehouse
◦   We processed multiple models simultaneously by cloning Azure Data Warehouse to parallelize processing and improve execution speed.
•   Azure Data Warehouse versus Azure SQL Database
◦   We conducted a cost-benefit analysis for Azure Data Warehouse and Azure SQL Database. Of the two technologies, Azure Data Warehouse offered greater scalability. Because our client’s data system transfers 50 terabytes of data per day, we chose Azure Data Warehouse.

Planning and Approval Phase

During the planning and approval phase, we propose our solution and receive feedback from the client. As mentioned, with our recent client, we needed to meet four objectives. We proposed five steps to address our client’s objectives:

1.   To ease infrastructure maintenance, leverage the serverless, scalable, and distributed architecture of the cloud.
2.   Implement a real-time job and asset monitoring framework.
3.   Implement real-time data refresh pipelines for real-time reporting.
4.   To improve processing speed, leverage the distributed processing power of the cloud.
5.   Enable data publishes based on source availability using an intelligent job monitoring and validation framework.

Figure 1: A high level representation of a large data system.

Execution Phase

During the execution phase, we implement the approved architecture. For our recent client, we used Azure Data Lake Storage for the storage layer. We used Azure Databricks for the processing layer. For the publishing layer, we used tabular models and SQL. We used Power BI and Excel for the visualization layer. The project’s architecture consisted of a seven-step process:

1.   Data is staged into Azure Data Lake Storage from upstream using Azure Data Factory.
2.   Data is transferred to Azure Databricks from Azure Data Lake Storage.
3.   Data is processed using Azure Databricks.
4.   Processed data is moved back into Azure Data Lake Storage for downstream users and into Azure Data Warehouse to create reporting views.
5.   The intelligent job monitoring and validation framework provides independent processing and refreshes.
6.   The tabular model is processed.
7.   Reports are visualized using Power BI or Excel.

Verification Phase

During the verification phase, we conduct user acceptance testing (UAT) through numerous user acceptance sessions. UAT allows technical users, business users, and operations team members to become familiar with the new data system as the old system is gradually moved offline.


Benefits of Data Migration

Our client’s data migration reduced data latency and improved data availability, resulting in the latest data infrastructure. Data migration also resulted in real-time reporting benefits.

•   Latency Benefits
◦   2X more efficient refresh cycle
◦   Source availability-based refresh schedule
•   Data Availability
◦   Single source of truth for all reporting layers
◦   Unlimited scalability with accelerated data processing
◦   Improved disaster recovery measures with geo-replication
•   Data Infrastructure
◦   Automatic start, termination, and scaling of local storage
◦   Reduced support costs
•   Real-Time Reporting
◦   Real-time data refresh pipelines for critical reports

Our data migration process balances overcoming logistical challenges with achieving business objectives.