Automating product feature categorization with AI for an automotive components manufacturer

  

 

About our client

Our client is a leading provider of innovative cockpit electronics and electrification solutions. With a global presence and a focus on cutting-edge technology, they design, manufacture, and sell multiple automobile components.

They required an automated system to streamline the classification of product feature descriptions provided by car manufacturers into specific primary and secondary disciplines. The need arose from the increasing complexity and volume of textual data. These factors made manual classification both time-consuming and prone to errors.

The issue at hand

·    Manual classification effort:
Previously, the client relied on a manual process to classify textual product features provided by manufacturers. This process was labor-intensive, prone to human error, and consumed a significant amount of time given the large volume of data.

·    Scalability issues:
As the volume of product feature data grew, the manual classification system became unsustainable. The client faced difficulties in scaling the process to accommodate increasing amounts of data without hiring additional resources.

·    Inconsistent accuracy:
Due to the complexity of the product features and the subjective nature of manual classification, the accuracy of the classification was inconsistent. This variability led to inefficiencies in product management and data analysis.

How we stepped in with an innovative AI solution

Our client needed to automate the classification of product features (textual statements from car manufacturers) into predefined primary and secondary disciplines. To address the client's needs, an advanced NLP and ML system was developed to categorize product features into disciplines using the following workflow:

1.      Data preparation:

·       Gathered historical textual data with corresponding labels.

·       Preprocessed text data by cleaning, tokenizing, and generating embeddings using OpenAI models to ensure a structured format for machine learning models.

·       Addressed imbalances in the dataset using data balancing techniques and by assigning appropriate class weights.

2.      Model development and training:

·       Developed and trained a machine learning model to classify product features, aiming for a high accuracy of 95%.

·       Azure Synapse was used to store and process the training data.

·       Text classification models were serialized for future use, ensuring that the trained model configurations could be easily deployed.

3.      Deployment and inference:

·       The trained model was stored in Blob Storage for future retrieval.

·       The model was used to classify new product features, and predictions were stored back in Blob Storage for analysis and further use.

·       The system allowed for easy retrieval of prediction results, stored in CSV format, using Azure Storage Explorer.

About the solution flow

Figure 1: Components of the AI-driven solution

Data collection and storage:

·       Historical data was collected and stored in Blob Storage using Azure Storage Explorer.

·       The storage system allowed for seamless retrieval of training data when required for model building.

Data retrieval and preprocessing:

·       Data was retrieved from Blob Storage to Azure Synapse using Jupyter Notebooks for preprocessing.

·       Preprocessing steps included cleaning, aggregation, and lemmatization of text data, ensuring it was suitable for machine learning models.

Data transformation:

·       Text data was transformed into tokenized vectors and numeric labels for input into machine learning models.

·       Embeddings were generated using advanced NLP techniques, enabling the system to understand and classify textual data effectively.

Model training and evaluation:

·       Various machine learning models were trained, tuned, and evaluated for performance.

·       The final model was serialized into a pickle file to ensure its reuse for future classifications.

Prediction and inference:

·       The system was designed to input new product data and classify it using the pre-trained model.

·       The inference results were stored back into Blob Storage and were available for download in CSV format for analysis.

Result storage and retrieval:

·       Prediction results were stored securely in Blob Storage.

·       Results were accessible through Azure Storage Explorer, ensuring ease of access for future analysis or reporting.

How the solution yielded business value

·       Improved efficiency: The automated system significantly reduced the time and effort required for product feature classification. It has reduced human dependency by around 90%.

·       High accuracy: The model achieved close to 90% accuracy, ensuring reliable classification of product features into predefined categories.

·       Scalability: The solution was scalable, allowing client to handle a larger volume of textual data without compromising on speed or accuracy.

·       Cost savings: Automation eliminated the need for manual classification, resulting in substantial cost savings over time.

These results underscore the success of the NLP solution in transforming the organization’s approach to data management. By using advanced AI technologies, the client not only optimized their internal processes but also positioned itself for greater efficiency and competitiveness in the market.

For any further inquiries, contact Sales@MAQSoftware.com to see how AI can transform your business, improve productivity, and accelerate your delivery.

Simplifying Power BI integration for the automotive industry using EmbedFAST

   

A leading company that is transforming the used car industry faced significant challenges maintaining traditional BI reports created by different groups, resulting in time-consuming management processes and redundant reports. 

 

The challenge

The client previously relied on their own traditional reporting solution. On top of it providing minimum flexibility and scalability, several challenges led them to seek a more efficient and scalable solution: 

·       Lack of administrative functionality: Their tool did not provide sufficient admin capabilities, limiting their control over report management and customization.

·       High licensing costs: The licensing costs for their tool were becoming prohibitively expensive, prompting the need for a more cost-effective alternative.

·       Customization constraints: There was difficulty in customizing reports and dashboards to meet evolving business needs.

·       Complex management: Cumbersome tenant and user management caused operational inefficiencies and scalability issues.

Our solution

EmbedFAST offers a comprehensive solution that addressed these challenges and transformed the client’s reporting capabilities:

·       Expanded administrative control: EmbedFAST delivers robust administrative functionality, allowing the client to manage reports, user roles, and permissions seamlessly within the platform.

·       Centralized reporting platform: EmbedFAST's platform seamlessly manages all the Power BI reports from one portal.

·       Cost-effective licensing: With EmbedFAST, the client is able to significantly reduce their licensing expenses, providing a more budget-friendly solution without compromising on features or scalability.

·       Highly customizable dashboards: EmbedFAST offers a flexible environment that allows the client to easily customize reports and dashboards, tailoring them to their specific business needs.

·       Simplified tenant and user management: With easy-to-use tools for managing tenants and users as well as streamlining operations, EmbedFAST enables the client to scale their reporting infrastructure with minimal effort.

Figure 1: Creating a subscription

Figure 2: Export history

Built with client insight and feedback

In addition to providing a base solution, we made several improvements to EmbedFAST based on the client’s specific needs:

·       New report and bookmark sharing functionality: Introduced the ability to share reports and bookmarks, improving collaboration across teams.

·       Authentication upgrade to Okta: Replaced the existing authentication system with an Okta-based setup, enhancing security and compliance.

·       Feature expansion and tool efficiency: Added new capabilities like recommended reports based on user’s department and favorites to improve the overall functionality and performance of the tool, boosting its efficiency and effectiveness for the client’s operations. Enhanced export to Excel and scheduling functionality.

Figure 3: Managing/sharing bookmarks

The impact

The client achieved faster business processes and improved operational efficiency. EmbedFAST provided a scalable and efficient way to embed dashboards, empowering their operations and enhancing workflow.

Ready to unlock the power of Power BI with EmbedFAST? Contact Sales@MAQSoftware.com today to learn more, request a demo, or get a personalized quote.

Transforming data management in manufacturing with an AI-driven chatbot solution

  

Generative AI (Gen AI) is transforming manufacturing by enabling advanced data analysis, automation, and faster decision-making. Tools like OpenAI's GPT-3.5 can help further streamline processes, improve data access, and drive innovation. Gen AI solutions—such as chatbots—boost efficiency by providing quick, accurate responses, reducing manual tasks, and increasing productivity. All in all, these factors combined can give manufacturers a competitive edge.

 

About our client

Our client is a globally recognized manufacturer specializing in industrial test, measurement, and diagnostic equipment. Renowned for their innovative solutions, they serve a broad range of industries, including electronics, industrial automation, and electrical engineering. Their commitment to precision and reliability has established them as a leader in their field.

The issue at hand

The client managed their project lifecycle management data and processes through multiple documents stored on a SharePoint website. While SharePoint also hosted the latest news in the form of .aspx pages, users found it challenging to locate specific documents and extract the necessary information. This lack of efficient data retrieval hindered productivity and slowed down decision-making processes. To address this, the client needed a generalized chatbot that could simplify data access and improve workflow efficiency.

How we stepped in with a Generative AI solution

To address the client’s data management challenges, a chatbot powered by advanced AI technologies was developed. The aim was to create an intuitive system that allowed employees to easily query information from a centralized repository of project-related documents. This AI-driven chatbot not only improved data accessibility but also ensured accurate responses from trusted sources. By integrating Azure AI services with OpenAI’s GPT-3.5 model and Azure Cognitive Search, the solution was built to improve operational efficiency. The chatbot became a critical tool in the client’s project management, enabling faster decision-making and reducing time spent on manual data searches.

The solution was implemented through a multi-step process that ensured seamless integration of various data sources and technologies:

1.     Data ingestion: Power Automate flows automated the copying of files from SharePoint and static Excel sheets to Blob storage, triggered by file additions or modifications. Data included PDFs, Excel files, and .aspx pages to cover the organization’s document repository.

2.     Data processing: Custom code was developed to decode .aspx pages and extract relevant text, saved as .txt files for inclusion in the data workflow. Doing this ensured all formats were accessible to the chatbot.

3.     Indexing: Azure Cognitive Search—a service within Azure AI that specializes in indexing large volumes of data—indexed data from Blob storage. The service processed various formats such as PDFs, Excel, and .txt files, creating a searchable index that served as the chatbot’s backbone.

4.     AI integration: With the data indexed and ready for querying, the next step involved deploying OpenAI’s GPT-3.5 model. The model was integrated with Azure Cognitive Search to generate precise, context-based responses using the indexed data. When a user submitted a query, GPT-3.5 analyzed the ranked results returned by Azure Cognitive Search and crafted a well-informed answer based on the indexed data.

5.     User interaction: The chatbot interface, hosted on an Azure WebApp, allowed for real-time interactions. Cosmos DB stored historical conversations for future reference and analysis.

6.     Additional features: To further optimize the solution, backend processes were implemented for data management and cost efficiency. These included converting old files from Hot to Cold storage and processed data from Cosmos DB into gold tables for reporting and analysis.

Figure 1: Components of the AI-driven chatbot solution

How the solution yielded business value

Implementation of the AI-driven chatbot solution yielded improvements across the organization’s operations. Through this solution, the organization was able to dramatically improve the accessibility and usability of their project lifecycle management data. The chatbot not only empowered employees to quickly find the information they needed but also facilitated more informed decision-making.

The post-deployment measures demonstrated the effectiveness of the solution:

·       Improved data retrieval time: The time required to locate and retrieve specific documents or information was reduced drastically, allowing employees to focus more on high-value tasks.

·       Increased employee productivity: With quick access to accurate information, overall employee productivity saw an increase. This was measured by the reduction in the time spent on non-productive activities like manual data searches.

·       Reduction in operational costs: By automating data retrieval and integrating backend processes like Hot to Cold storage conversion, the organization achieved a large reduction in operational costs related to data management.

·       Increased data accessibility: The centralization and indexing of diverse data formats made 100% of the project lifecycle management data accessible through a single interface. This meant that no critical information was overlooked.

These results underscore the success of the chatbot solution in transforming the organization’s approach to data management. By using advanced AI technologies, the client not only optimized their internal processes but also positioned itself for greater efficiency and competitiveness in the market.

For any further inquiries, contact Sales@MAQSoftware.com to see how chatbots powered by Gen AI can transform your business, improve productivity, and accelerate your delivery.

[OLD CONTENT] Everything You Need to Know About Migrating to Power BI

 

Table of Contents

1. What is Power BI?

2.  Power BI Migration Strategy

3.  Benefits of Migrating to Power BI

4.  Power BI Migration Case Studies

1.  Tableau to Power BI

2.  Qlik to Power BI

3.  SAP Business Objects (SAP BO) to Power BI

4.  MicroStrategy to Power BI

5.  Looker to Power BI

6.  Cognos to Power BI

What is Power BI?

Power BI is a business intelligence (BI) tool that enables users to easily transform their data into actionable insights. Business intelligence refers to the collection and analysis of business operation data. Insights from this data enable business leaders to identify growth opportunities and close operational gaps.

Previously, BI reporting platforms required developer expertise. Today, anyone can develop intuitive, insightful dashboards and reports with the right tool. Power BI is a powerful, easy-to-use tool that offers a wide range of storytelling visuals that help you understand your business opportunities. 

Power BI Migration Strategy

Large-enterprise companies rely on insights from BI platforms. When companies use unoptimized platforms, insights are slow, inaccurate, and siloed, delaying business-critical decisions for weeks. The most common issues our clients face with other BI platforms are:

1.    Slow-loading pages and reports

2.    Difficulty managing and maintaining multiple data sources

3.    Decentralized reporting

4.    High maintenance costs

Increasingly often, large-enterprise companies are turning towards Power BI. Power BI’s centralized, dynamic reporting better addresses their real-time business needs. However, migrating large volumes of data from enterprise systems can be challenging.
Managing terabytes of data and training thousands of team members in a new system requires meticulous planning. That’s where we come in. After leading over 100 Power BI migrations for large-enterprise companies, and implementing over 8,000 Power BI solutions, we’ve developed a simple six-step migration strategy. With our strategy, you can rest assured that we’ve covered all the bases, enabling you to migrate seamlessly to Power BI.

1. Requirement Gathering and Analysis

Before we start actually moving data, it’s important to understand your current landscape. This means evaluating the existing reporting platform to understand your current needs, key functionalities, and gaps. We examine reports, dashboard usage, UI/UX, audiences, data sources, and security to create a report inventory and data estate. This information determines your migration scope, performance requirements, and complexity.

2. Planning and Design

Now that we understand your existing landscape, it’s time to move onto developing a road map. This sets the stage for the migration’s success. As Antoine de Saint-Exupéry once said, “A goal without a plan is just a wish.”

Here, we propose a solution based on all the requirements gathered in step one. To ensure everyone agrees with the plan of action, we set up a proposal meeting that involves architects, data administrators (admins), infrastructure admins, legal and security teams, and the Power BI product team (if required).

In general, we divide planning and design into five sub-steps:

1. Perform a detailed gap analysis to identify the different features, visualization, and modeling challenges we need to address during migration

2.  Propose a Power BI architecture, including security implementation, refresh latency, and report performance

3.  Design report templates and prepare mock-ups

4.  Define the scope for automated validation

5.  Propose a deployment strategy and implementation timeframe

3. Execution

Now, it’s time to implement the approved solution architecture. Because we spend so much time on the planning stage, this step is straightforward. To optimize our workflow, we follow the agile methodology with the following steps:

1.  Sprint plan: Create a product backlog and define the scope and length of sprints

2.  Implementation: using best practices, reusable templates, and themes, start the report migration and provide incremental report builds

3.  Performance tuning: Refine the architecture and report layout to optimize the data model for performance.

4. Testing: Use a set of in-house performance analysis tools to automate testing, which tracks query performance and suggests visual layout and data validation optimizations

5.  Deployment: Close our sprint by automating report deployment and readying the build for user acceptance testing (UAT)

4. Deployment and Post-Production

During this step, we ensure the new reports are user-friendly and high-performing. First, we conduct numerous UAT sessions. UAT ensures the reports are optimized for their target audience. Once we receive sign-off for UAT and production, it’s time for deployment. We automate deployment, giving end users immediate access to the reports. To complete the transfer of ownership, we hand off the code, report, and workspace inventory to our client.

For many companies, Power BI migration ends here. However, we believe that successful adoption is a critical part of migrating to Power BI. That’s why we dedicate the next two steps to post-migration success.

5. Center of Excellence (CoE)

According to Microsoft, “A Center of Excellence (CoE) drives innovation and improvement and brings together like-minded people with similar business goals to share knowledge and success, while at the same time providing standards, consistency, and governance to the organization.”

During our CoE trainings, we enable our clients to become self-sufficient Power BI users. We run numerous CoE sessions that train team members across your organization in Power BI capabilities, governance, and best practices. These enable technical users, business users, and operations team members to become familiar with the new data system as the old system is gradually moved offline. Our custom trainings include regular office hours with certified engineers, an advanced curriculum, and pre-built solutions and frameworks. On average, our CoEs shorten the Power BI adoption timeframe from years to months.

If you are already at this migration stage, or need some help boosting Power BI adoption, check out our virtual CoE trainings, offered to any organization year-round:

•  Dashboard in a Day 

•  Data Visualization CoE 

•  Admin CoE 

6. Decommissioning

There’s nothing worse than a cluttered data system. To avoid redundancies, we systematically retire old reports. Here, our main goal is moving you onto the new system without impacting ongoing business operations. At MAQ Software, we believe migration to Power BI should be seamless.

Figure 1: Complete Process Overview

Benefits of Migrating to Power BI

By migrating to Power BI using our six-step approach, our clients have benefitted from:

Quicker Insights for Decisionmakers

•  Reduced latency between data sources and reports  

•  Increased scalability 

Self-Service BI

•  Business users can create reports and customize dashboards without developer expertise  

Centralized Reporting

•  Admins can easily manage and govern their organization’s reports with centralized administrative capabilities 

•  Users can combine data from different sources, such as Dynamics 365, Office 365, and hundreds of other relational databases

•  Increased accuracy by offering a single source of truth for all reporting layers through shared datasets 

Power BI Migration Case Studies

As the 2021 Microsoft Power BI Partner of the Year, we have experience migrating clients from a wide variety of data visualization platforms to Power BI. Our expertise enables us to easily manage large volumes of data and enable business continuity throughout the migration process. Here is a sample of how we’ve empowered our clients to migrate to Power BI.

Tableau to Power BI

Client: An international fast-moving consumer goods (FMCG) company.

Situation: Our client wanted to centralize their reporting platforms by migrating from Tableau to Power BI. As their existing Tableau reports were developed over time, it was complex to migrate them without compromising functionality.

How We Did It: We discussed each report in detail to understand its underlying business purpose. Then, we used our knowledge of Power BI to identify the best methods of achieving the same results in a new system. Spending time with the actual report users gave us insight into end user flow, enabling us to design an intuitive Power BI report.

Results: We migrated over 250 Tableau workbooks to Power BI. The new reports were better organized and decluttered. With easy navigation and optimized design, the new reports achieve the same functionalities as the old ones, with increased performance and accessibility. Our Center of Excellence trainings also helped increase post-migration adoption by 300%.

Qlik to Power BI

Client: EF Education First, a global education foundation with offices in 50 countries.

Situation: EF Education First needed a modern reporting platform with self-service analytics, easy scalability, and low operational costs.

How We Did It: We performed a gap analysis of the features and visualizations in Qlik and Power BI. Qlik supported 16 reports, with a data source consisting of 20+ SQL databases and 30 Excel sources. We ensured all required data could be transferred and visualized per the client’s needs.

Results: Power BI’s low-code architecture and cloud-based centralization, gave EF Education First access to self-service scalability.

Find out more about our QlikView to Power BI migration

SAP Business Objects (SAP BO) to Power BI

Client: A multinational food, snack, and beverage corporation with 250,000+ team members.

Situation: With our client’s high volume of data, their existing SAP BO reports took over five minutes to load. Running many slow-loading reports takes up the team’s valuable time, negatively impacting business operations.

How We Did It: We implemented a tabular model with Azure Analysis Services (AAS) to enable fast, efficient reporting in Power BI. Data loads from our client’s existing Teradata storage into AAS. For users with alternate view and calculation needs, reports can be exported directly from AAS to Excel. AAS is more equipped to store the huge models and pre-aggregated data needed for real-time visualization. AAS provides a dedicated central processing unit (CPU) and memory, independent of the load on premium capacity.

Results: Migrating from SAP BO to Power BI reduced reports run time by 90%. Previously, reports could take up to 5 minutes to load. With our solution’s back-end Azure Analysis Services (AAS), dense data now loads into Power BI in less than 20 seconds. Users can rapidly customize and run reports without the wait. AAS also has a built-in feature that provides time intelligence for KPIs on the fly.

MicroStrategy to Power BI

Client: A global Fortune 500 retailer.

Situation: Our client sends weekly and monthly report snapshots to subscribed internal and external users. MicroStrategy offers an easy and intuitive method to share reports like this. However, our client had recently migrated their other systems to Power BI as it offered better long-term scalability. To reduce costs, our client wanted to consolidate all functionalities to a single platform. We needed to implement a similar export/subscription functionality using Power Platform.

How We Did It: We used the existing subscription list and created a security model works with Power Automate schedules. Then, we converted data tables in MicroStrategy to paginated reports in Power BI. Using the Export API, the data can now be exported as an attachment to share with external and internal users.

Results: We helped our client retire their outdated MicroStrategy reports without losing their easy sharing capabilities. Because Power BI is part of the Power Platform, it integrates seamlessly with other powerful tools, such as Power Automate and Power Apps. Now, our client can view dashboards, manage reports, and share insights using a platform that is both scalable and sustainable.

Looker to Power BI

Client: A leading retail firm that provides office supplies. 

Situation: Our client sought a centralized (BI) platform that delivers low operational and maintenance costs while providing self-service analytics capabilities. They also required seamless migration from their on-premises data source to a cloud-based one. 

How We Did it: Our team established a centralized Power BI dataset by importing data from a cloud-based source. To optimize query performance and minimize costs, we implemented custom partitioning and incremental refresh policies in Power BI. By doing so, we reduced the overall number of queries fired to the cloud-based source. Our solution also met the customer's requirements for data refresh latency, ensuring that the dataset was always up-to-date and readily available for analysis. 

Results: We assisted the client in retiring their Looker reports and migrating to Power BI, empowering end-users with self-service reporting capabilities. With Power BI's user-friendly interface, users can easily customize their report views and gain valuable insights. Power BI's built-in export functionalities also enable users to seamlessly share their findings with others, making it a more collaborative and efficient tool for the client's reporting needs. 

Cognos to Power BI

Client: A global service provider in the Health, Tax & Accounting, Legal & Regulatory, and Finance industries. 

Situation: Due to our client's high volume of reports, their existing Cognos reporting system had a high cost per click—on top of having limited UI features. This drawback led to a downward impact on business operations. 

How We Did It: We implemented a tabular Microsoft SQL Server Analysis Services (SSAS) model that allowed for fast and efficient reporting in Power BI. The data from the client's existing data warehouse was loaded into SSAS, which is better equipped to store large models and pre-aggregated data needed for real-time visualization. With SSAS as the backend, reports can be generated directly from Excel for users with business priority and calculation needs. Additionally, SSAS provides a dedicated CPU and memory, which further optimizes the reporting process. Powerful features such as Export, Subscribe, and User Management (which can restrict users with lower privileges from publishing reports to the workspace) can easily be customized and managed using Power BI Report Server. 

Results: Migrating from Cognos to Power BI reduced the cost per click by ~50%, while the aesthetically appealing visuals also improved the usage of the reports. Our solution allows for dense data to load into Power BI in less than 3 seconds, allowing users to rapidly customize with a better UI and run reports without delay. With SSAS, there is a built-in feature that provides time intelligence for KPIs on the fly, which further enhances the reporting process. 

***

While our six-step migration strategy provides a general framework for success, each organization’s needs are different. Need help getting your Power BI migration on track? Partner with us by emailing Sales@MAQSoftware.com .

Up Next

To further improve your Power BI performance, check out our Power BI Best Practice Guide.