(VERSUSstudio/Shutterstock)
With the latest update to its data unification and management software, Reltio is employing one-shot learning techniques using large language models (LLMs) to resolve identities of objects in organizations’ data, which will save them the pain and hassle of using manual techniques, the company says.
When you have 15,000 people named “Jim Smith” in your books, you need to take steps to figure out which particular Jim Smith is the one that lives on 123 Main St., owns two cars, has three kids, and is a potential candidate for an upgraded cellular plan. That’s the job of entity resolution, a sometimes complex but ultimately necessary step in the big data analyics journey.
Reltio previously relied upon human-created rules and fuzzy matching techniques in Reltio Customer 360 to resolve entities in customer data, says Venki Subramanian, the company’s senior vice president of product management. But since it discovered that software based on large language models (LLMs) can do this task so much better, customers will no longer need to labor over this critical but time-consuming task.
Reltio’s FERN automatically finds connections in entities (PopTika/Shutterstock)
“The basic reason why we went with large language models, and where we are seeing success, is LLMs are inherently really, really good at comparing strings with context awareness,” Subramanian says. “So I can actually compare names, I can compare phone numbers, I can compare email addresses, or physical addresses. And these LLMs are really good at comparing and telling me how similar they are with the context built into it.”
Reltio used open source LLMs like Llama3 and Flan-T5 to train its own one-short learning models, dubbed Flexible Entity Resolution Networks (FERNs). These FERNs come pre-trained using and don’t require any additional training on customer data to be able to resolve entities at a very reliable level, Subramanian says.
“This is something that our customers will be able to activate inside their tenants,” he says. “And as they load data or as they add data to it, it is operating continuously to identify ‘Is this new record, that we got a duplicate?’ or ‘Is it already something that we can identify as something else that exists?’”
Reltio created the FERNs by combining the publicly available LLMs with its own domain knowledge. Reltio recommends that customers use the FERNs and ditch the old rule-based and fuzzy matching methods, which didn’t work as well and took more time to operate, Subramanian says.
Reltio was designed to be an enterprise-wide unified data layer
“When it gets deployed to a customer, it’s as simple as turning it on and setting some thresholds that fits your data to say, ‘Hey, above 80%, I’m confident for the system to take action on it’ or ‘I want my data stewards to review all matches that are above 70%,’” he says.
Reltio’s software is designed to function as a enterprise-wide unified data layer that allows different parts of a company to have access to consistent and reliable data. The company’s solutions, including master data management (MDM) and Customer 360 offerings, utilize the same core data capabilities in the platform.
Primary users on the data team include data stewards who are in charge of ensuring the enterprise has access to clean, well-managed data, as well as employees on the business side, such as analysts and managers, who are using data as part of their jobs. The company has hundreds of customers across a range of industries, including life sciences, healthcare, financial services, insurance, retail, travel, and hospitality.
The update to Reltio Customer 360 includes a new dynamic segmentation capability that should make it easier for analysts and others to quickly get access to actionable data. The product now can automatically segment customer data based on customer attributes, such as annual income and ZIP Code, as well as behavior, such as purchases, claims, and interaction with the call center.
“I can combine my attributes of different entities along with their interactions or transactions or other additional information and relationships that exist between different entities,” Subramanian says. “And I can create segments, like ‘Give me all people who have over $100,000 annual income to purchase the product XYZ and who also expressed interest in an additional product.’ So I can create fairly complex queries, all of that inside Reltio without having to extract all of the data and then do this kind of analysis in a data warehouse system or in a CDP.”
Reltio is also hooking its offerings into data governance products. This release brings integration with Microsoft Purview Data Governance, the release of Collibra’s data catalog unveiled in August, and the Unity metadata catalog from Databricks, as the company’s announced last month. In the future, the company will be working to hook Reltio into other popular data catalogs, including those from Alation and Atlan, Subramanian says, as well as the Apache Polaris metadata catalog created by Snowflake.
Integrating with other big data ecosystem tools is important for giving Reltio customers as painless of a data access experience as possible, Subramanian says.
“We focus on supporting open standards wherever they are available or providing seamless integrations wherever they are possible,” he says. “When our customers buy Reltio, they’re not just buying a technology platform, they’re buying a solution that comes pre-seeded with semantic models of data for different data domains, depending on their specific industry verticals.”
Related Items:
Building a Trusted Data Foundation for AI/ML and Business Intelligence (BI)
The Rise and Fall of Data Governance (Again)
Unlocking the Full Potential of Data: The Crucial Role of Data Governance in Integrated Analysis
Source link
lol
