Introduction
Data is power. According to McKinsey, “Banks own more customer data than companies in most other industries, but they are behind many other consumer-facing sectors when it comes to personalization at scale.” https://www.mckinsey.com/industries/financial-services/our-insights/the-state-of-retail-banking-profitability-and-growth-in-the-era-of-digital-and-ai. In banking, it’s about turning that power into actionable insights while carefully navigating data security risks. Financial institutions must balance protecting sensitive data and embracing data democratization to improve customer experiences, reduce risk, and drive innovation.
Sigma Computing, a next-generation analytics and business intelligence platform, provides a secure path, leveraging Databricks as the intelligent data warehouse at the core. This partnership enables banks to leverage insights from customer segmentation to risk assessment, all without compromising security. In this blog, we’ll show how Databricks and Sigma jointly provide a secure, exploratory workflow for banking analysts as they use consumer data to drive better personalization and new revenue streams.
Data Intelligence is a New Lifeline for Banking
Digital banks like Nubank and fintechs like Wise and Brex have outpaced traditional enterprise banking by offering customers streamlined digital experiences, real-time financial services, and transparent pricing. In contrast, conventional banks often lag behind due to legacy infrastructure. This technical debt leads to longer development cycles and, as a result, limits banks’ ability to develop new products and anticipate evolving customer expectations. More than ever, data is a strategic asset, and banks can leverage data as a competitive advantage.
Data intelligence is a key to unlocking the full potential of enterprise data. To align with our vision of full data democratization to any user, Databricks releases tools such as Genie spaces, a conversational interface where business users can interrogate their data. What differentiates Genie is that this service incorporates full enterprise context of business jargon and technical metadata, built into the space as a foundation for answering business questions via Unity Catalog, Databricks’ governance solution.
Why Banks Struggle with Attaining Data Democratization
Banks are highly regulated and risk-averse by nature. They handle massive volumes of sensitive data, from transaction histories to personal customer information, which makes them prime targets for cyberattacks and insider threats. Data exfiltration—the unauthorized transfer or theft of data—looms as a constant risk, driving stringent security protocols. But here’s the dilemma: while protecting data is essential, locking it down too tightly hampers innovation and slows decision-making.
In this domain, data democratization becomes a challenge. Democratization promises to unlock the potential of data, putting it in the hands of those who can act on insights—data analysts, product managers, and even executives. However, banks often find themselves torn between their need for open, flexible access to data and the strict security measures they must uphold.
Traditional data systems in banks are siloed, making it difficult for teams to share data seamlessly across departments. The fear of unauthorized access leads to a restrictive environment where only a select few have the privilege to access critical information. This bottleneck stifles the agility that data democratization aims to provide. Additionally, regulatory compliance adds another layer of complexity, as banks must ensure that they meet local and global standards like GDPR and CCPA while still fostering a data-driven culture. A prime example of a bank that solved these problems is HSBC, with their reinvention of payments, powered by Databricks. They were able to consolidate 14 databases into a single Lakehouse. Moreover, they were able to protect data by masking in real-time and modernized their customer segmentation strategy, resulting in the #1 payments app in Hong Kong.
In the next section, we’ll explore how customers can use Databricks and Sigma Computing to solve the challenges above.
Unleashing Data Intelligence While Adhering to Compliance
The Business Problem
Imagine a set of business analysts tasked with increasing loyalty for banking products in revolving credit. The internal IT team must provide these analysts with no-code, non-technical tooling. There are stringent requirements for restricting export functionality. On the other hand, analysts need to manipulate data with full access to lineage and metadata to explain any business result, such as a proactive credit line increase offer, to their stakeholders.
Market Tools for Self-service Analysis
End users require self-service analysis. In most cases, the primary delivery tool to enable this is Excel. However, providing data in Excel is a surefire way to increase data exfiltration risk and proliferate copies of data, leading to inefficiencies. Enter Sigma computing.
Sigma offers a path for end users to view data feeds and analytics without providing unbridled access to datasets. Sigma also brings familiarity with spreadsheet UX, which makes adoption among Excel users especially frictionless. Sigma offers platform teams a unique way to extend data to non-technical users using a construct known as ‘secure embeds’.
Allowing True Self-service with Sigma’s Secure Embed
Sigma allows you to create a secure embed self-service widget in the application of your choice. For example, developers can create a Flask app, embedded email, or Teams integrations that authenticate with JWT tokens and call the Sigma service. JWT (JSON Web Tokens) are compact, URL-safe tokens that securely transmit information between systems, ensuring data integrity and authentication. This format is foundational for modern, API-driven applications, allowing large enterprises to authenticate users securely without exposing sensitive information. In this Sigma Computing and Databricks integration, JWT ensures users are granted real-time, personalized access to workbooks and dashboards. JWT allows for secure client access to financial data, and this implementation protects dashboard access, maintaining both performance and security by embedding user-specific claims within each token.
Embed Users with No Workspace Entitlements
Users can now easily access data, and role-based access controls can be applied to enable workbook owners to provide restricted access in a simple one-time toggle. See the images below for reference. Here, the external user resides in the ‘Embed Banking Analysts’ team. The analytics workbook owners provide one-time access to the embed user team, and end users are ready to consume any dataset they are entitled to. All the fine-grained access controls defined as part of Unity Catalog extend to groups in Databricks, particularly row and column-level filtering.
Deploying Financial Analytics Apps on the Domain-Intelligence Lakehouse
Databricks now offers a brand new, streamlined way to deploy the application to serve secure embed users – Databricks Apps. Using Databricks Apps, data engineering teams can develop and sync code into the Databricks workspace and deploy an application within minutes, making data accessible to analysts and non-technical users just as they would in a standard custom portal. Whether you’re using Databricks Apps or a self-hosted application (we use Flask in our Apps demo), platform teams can onboard embed users to fully democratize data to non-technical users who are used to leveraging pivot tables and on-the-fly ratios for banking use cases.
In our sample app, we focus on customers with internal profiles and external data from a credit bureau (e.g., TransUnion) and financial aggregators such as Plaid and Yodlee, which will supply credit scores, credit lines, and trends on usage of financial products. Our initial summary (below in red) shows that most customers are interested in home ownership. On the other hand, engagement is low, and the bank we work for is interested in increasing loyalty. As a starting point, our embed users can quickly look at liquidity ratios and target customer segments for a proactive credit line increase.
The analytics we see are helpful, but how can we empower embed users with a better exploration experience without sacrificing data exfiltration risk?
As seen below, inside the Sigma environment, we can create an entirely customized role called ‘Explorer But Not Exporter’, preventing users from exporting data using download buttons. In the spirit of innovation, we give the analyst user rights to edit the workbook, for example, new column calculations and creations, which will allow for a much richer experience and adding calculations such as liquidity ratios for efficiently assessing opportunities for proactive credit line increases or investments for banking clients. Users are now fully empowered to add analytics personalized to their questions.
While the experience has allowed users to create new binning, heatmaps, and segmentation lists, how do we ensure trust in the data produced? Our sample tables in the application show a proactive credit line increase prediction, which we can now apply to our target segments. Analysts must understand where predictions originate to explain analytics and results to business stakeholders and executives. With the built-in lineage in Unity Catalog, which extends beyond data into models, dashboards, features, and functions, we can pinpoint exactly where our predictions came from. All of the derived source information below and tags are easily obtained from Unity Catalog system tables (AWS | Azure | GCP), providing a higher level of trust and explainability with the business.
Conclusion
The financial services industry is at a crossroads. Traditional banks can no longer afford to remain bound by outdated, siloed data systems, while fintech disruptors continue to redefine customer experiences through real-time data intelligence. In this high-stakes environment, secure data democratization isn’t just a luxury—it’s a lifeline.
Sigma Computing’s secure embed capabilities, combined with Databricks’ robust application deployment platform and data lakehouse, present a transformative solution for banks seeking to leverage data without compromising security. Sigma’s secure embeds provide a controlled experience that aligns perfectly with stringent regulatory requirements. With Sigma, banks can empower their analysts, investigators, product managers, and sales teams to access vital insights in a more secure and compliant environment.
By deploying applications on Databricks, financial institutions can swiftly and seamlessly extend secure data access to embed users, customizing experiences while maintaining an ironclad grip on data security. In an industry where the margin for error is razor-thin, the ability to deliver data-driven insights through secure, embedded applications becomes a competitive edge.
The message to financial institutions is clear: the time for passive data strategies is over. Those who embrace secure, self-service data intelligence will mitigate risks and unlock the agility needed to thrive in today’s digital market. Banks must evolve now, and leveraging tools like Sigma Computing and Databricks can activate a data-driven culture that turns raw information into actionable insights.
Sign up for Databricks and Sigma today to enable secure, self-service data intelligence.
Source link
lol
