(Lidiia/Shutterstock)
Despite the hype around generative AI, studies show just a fraction of GenAI projects have made it into production. A big reason for this shortfall is the concern organizations have about the tendency for large language models (LLMs) to hallucinate and give inconsistent answers. One way organizations are responding to these concerns is by implementing trust layers for AI.
Generative models, such as LLMs, are powerful because they can be trained using large amounts of unstructured data, and then respond to questions based on what they have “learned” from said unstructured data (text, documents, recordings, pictures, and videos). Organizations are finding this generative capability incredibly useful for the creation of chatbots, co-pilots, and even semi-autonomous agents that can handle language-based tasks on their own.
However, an LLM user has little control over how the pre-trained model will respond to these questions, or prompts. And in some cases, the LLM will generate wild answers completely disconnected from reality. This tendency to hallucinate–or as NIST calls it, to confabulate—cannot be fully eliminated, as its inherent with how these types of non-deterministic, generative models are designed. Therefore, it must be monitored and controlled.
One of the ways organizations can keep LLMs from going off the rails is by implementing an AI trust layer. An AI trust layer can take several forms. Salesforce, for example, uses multiple approaches to reduce the odds that a customer has a poor experience with its Einstein AI models, including by using secure data retrieval, dynamic grounding, and data masking, toxicity detection, and zero retention during the prompting stage.
(Lightspring/Shutterstock)
While the Salesforce Einstein Trust Layer is gaining ground among Salesforce customers, other organizations are looking for AI trust layers that work with a range of different GenAI platforms and LLM models. One of the vendors building an independent AI trust layer that can work across a range of platforms, systems, and models is Galileo.
Voyage of AI Discovery
Before co-founding Galileo in 2021 with fellow engineers Atindriyo Sanyal and Vikram Chatterji, COO Yash Sheth spent a decade at Google, where he built LLMs for speech recognition. The early exposure to LLMs and experience working with them taught Sheth a lot about how these types of models work–or don’t work, as the case may be.
“We saw that LLMs are going to unlock 80% of the world’s information, which is unstructured data,” Sheth told BigDATAwire in an interview at re:Invent last month. “But it was extremely hard to adapt or to apply these models onto different applications because these are non-deterministic systems. Unlike any other AI that is predictive, that gives you the same answer every time, generative AI does not give you the same answer every time.”
Sheth and his Galileo co-founders recognized very early on that the non-deterministic nature of these models would make it very difficult to get them into production in enterprise accounts, which have less appetite for risk when it comes to privacy, security, and putting one’s reputation on the line than the move-fast-and-break-stuff Silicon Valley crowd. If these LLMs were going to be exposed to tens of millions of people and achieve the trillions of dollars in value that have been promised, this problem had to be solved.
“To actually mitigate the risk when it’s applied to mission critical tasks,” Sheth said, “you need to have a trust framework around it that can ensure that these models behave the way we want them to be, out there in the wild, in production.”
Starting in 2021, Galileo has taken a fundamentally different approach to solving this problem compared to many of the other vendors that have popped up since ChatGPT landed on us in late 2022, Sheth said. While some vendors were quick to apply frameworks for traditional machine learning, Galileo spent the better part of two years conducting research, publishing papers, and developing its first product built specifically for language models, Generative AI Studio, which it launched in August 2023.
“We want to be very thorough in our research because again, we are not building the tool–we are building the technology that works for everyone,” Sheth said.
Mitigating Bad Outcomes
At the core of the Galileo’s approach to building an AI trust layer is another foundation model, which the company uses to analyze the behavior of the LLM at issue. On top of that, the company has developed its own set of metrics for tracking the LLM behavior. When the metrics indicate bad behavior is occurring, they activate guardrails to block it.
“The way this works is we have our own evaluation foundation models that act, and these are dependable, reliable models that give you the same output every time,” Sheth explained. “And these are models that can run all the time in production at scale. Because of the non-deterministic nature, you want to set up these guardrails. These metrics that are computed each time in production and in real time, in low latency, block the hallucinations, block bad outcomes from happening.”
Galileo helps customers implement guard rails for GenAI (phoelixDE/Shutterstock)
There are three components of Galileo’s suite today: Evaluate, for conducting experiments across a customer’s GenAI stack; Observe which monitors LLM behavior to ensure a secure, performant, and positive user experience;, and Protect, which prevents LLMs from responding to harmful requests, leaking data, or sharing hallucinations.
Taken together, the Galileo suite enables customers to trust their GenAI applications the same way they trust their regular apps developed using deterministic methods, Sheth said. Plus, they can run Galileo wherever they like: on any platform, AI model, or system.
“Today software teams can ship or launch their applications almost on a daily basis. And why is that possible?” he asks. “Two decades ago, around the dot-com era, it used to take teams a quarter to launch the next version of their application. Now you get an update on your phone every like every few days. That’s because software now has a trust layer.”
The tooling involved in an AI trust layer look significantly different than what a standard DevOps team is used to, that’s because the technology is fundamentally different. But the end result is the same, according to Sheth–it gives development teams the peace of mind to know that, if something goes awry in production, it will be quickly detected and the system can be rolled back to a known good state.
Gaining GenAI Traction
Since launching its first product barely a year-and-a-half ago, Galileo has begun to generate some momentum. The company has a handful of customers in the Fortune 100, including Comcast, Twilio, and ServiceNow, and established a partnership with HPE in July. It raised $45 million in a Series B round in October, bringing its total venture funding to $68.1 million.
As 2025 kicks off, the need for AI trust layers is palpable. Enterprises are champing at the bit to release their GenAI experiments into production, but officers just can’t sign off until some of the rough edges are sanded down. Sheth is convinced that Galileo has the right approach to mitigating bad outcomes from non-deterministic AI systems, and giving enterprises the confidence they need to green light the GenAI.
“There are amazing use cases that I’ve never seen possible with traditional AI,” he said. “When mission critical software starts becoming infused by AI, what’s going to happen to the trust layer? You’re going to go back to the stone ages of software. That’s what is hindering all the POCs that are happening today from reaching production.”
Related Items:
EY Experts Provide Tips for Responsible GenAI Development
GenAI Adoption: Show Me the Numbers
LLMs and GenAI: When To Use Them
Source link
lol
