Thursday, June 16, 2022

Enhancing fintech analytics to provide millions of borrowers with better loan options

Share

Business Case

Our client, a leading fintech company, enables thousands of financial institutions to engage millions of borrowers with better loan options. Our client was on a mission to expand their analytics platform when they faced a critical block: Their existing platform architecture was at maximum data capacity. To onboard new customers, our client needed a more scalable analytics solution. In addition, our client wanted to enhance their platform’s reporting experience. Existing reporting was limited and required users to export data to Excel for manual analysis, delaying insights. To increase their product value and onboard more customers, our client needed a scalable architecture with embedded reporting.

Key Challenges

•  Enable analytics platform to scale to 1000+ customers 
•  Enable self-serve, near real-time analytics 
•  Enable AI/ML capabilities for future innovation 
•  Improve security of financial data 

Our Solution

We rebuilt our client’s analytics platform using Azure Synapse, Azure Data Lake Storage, Azure Data Factory, Azure Databricks, and Power BI. To ensure operational and technical excellence throughout the build, we followed the five pillars of the Azure Well-Architected Framework and leveraged migration strategies from Microsoft’s Cloud Adoption Framework.

Reliability: Implemented query replica within Azure Analysis Services (AAS) to ensure resource intensive queries do not impact ETL processing. Configured secondary and backup resources to ensure 100% resource availability.

Security: Enabled role-based access, disabled public access to storage accounts with PII data to ensure partner data is isolated within the ecosystem. In doing this, we greatly reduced the risk of security threats.

Cost Optimization: Implemented auto-scaling in lower environments, enabled Databricks to scale down when inactive, and deployed Power BI report on cost monitoring to scale services as needed.

Operational Excellence: Created Terraform automated scripts for Azure resources deployment. Implemented proactive monitoring for pipeline bottlenecks, ETL execution, and failures.

Performance Efficiency: Implemented parallel processing and concurrent querying of underlying data model for 1000+ customers using Azure Databricks.

In addition, our automated deployment framework uses continuous integration/ continuous delivery (CI/CD) pipelines to create Azure landing zones by focusing on identity, network, and resource management. To deploy Azure landing zones, we used a proprietary approach that combines the benefits of both the “start small and expand” and the “enterprise-scale”. Using industry-standard best practices and our center of excellence for Azure infrastructure setup, we ensured the right configuration to build a strong foundation and avoid rework in the future. This approach reassures our customers about our capabilities while creating a secure and reliable environment that is built to last.

Business Outcomes

With our Azure Synapse-based solution, our client’s platform now offers powerful self-service, near real-time analytics, enabling their customers to reach millions of borrowers faster. The platform now has the capacity to scale and support over 1000 customers. With Azure Synapse, our client can easily integrate machine learning models like fraud detection and recommendation engines without major architecture changes. To accelerate onboarding, we developed an automated deployment framework that onboards new customers in a single click, reducing setup time from days to hours.