Early GenAI Adopters Seeing Big Returns for Analytics, Study Says

(Wright Studio/Shutterstock)

It’s become fashionable to question whether generative AI ultimately will generate positive returns on the massive investments that companies are making. Gartner, for example, said 30% of GenAI projects will end in failure by next year. But a new report commissioned by ThoughtSpot found that early adopters are seeing significant results when using GenAI for analytics.

ThoughtSpot commissioned MIT Sloan Management Review (SMR) Connections and its research partner, Kadence International, to survey 1,000 business leaders about their use of GenAI for analytics. The subjects were segmented into three groups based on the maturity level of their GenAI initiatives, with 67% classified as early adopters who have already put some GenAI apps into production, 26% who are planning to deploy it, and 7% who are still evaluating.

Among the early adopters, 47% expect a return on investment (ROI) for GenAI applications of 100% or more over three years, with 12% of that group expecting an ROI of more than 300% and 11% expecting an ROI of 200% to 299%. That’s significantly more than the planners cohort, of which 38% expect an ROI of 100% over three years, with 11% expecting an ROI of 200% to 299% and just 2% expecting an ROI of 300% or more.

Early adopters are expecting big returns from GenAI investments (Image source: “Generative AI for Data and Analytics: How Early Adopters Are Reaping the Rewards”)

The report, titled “Generative AI for Data and Analytics: How Early Adopters Are Reaping the Rewards,” also suggests that GenAI may be driving a competitive gap between those who effectively wield the technology and those who don’t.

Among early adopters, 37% report that their GenAI use is “far ahead of market and competitors,” compared to 11% for the planning cohort, while another 46% of early adopters say GenAI has put them “slightly ahead of market/competitors” versus 51% of the planning cohort.

Those heady numbers caught the attention of Cindi Howson, ThoughtSpot’s Chief Data Strategy Officer, who is optimistic about the potential of GenAI to positively impact the field of data and analytics.

“The value that we can derive from this in terms of productivity gains and whole new business models–we’re just getting started,” Howson said. “We are in the dial-up days of the Internet, and people are only just now starting to think of the potential here.”

Hard Benefits of GenAI for BI

There are many different ways to monetize GenAI, with chatbots and co-pilots being the two most prominent use cases since ChatGPT debuted in the fall of 2022, and agentic AI being the latest GenAI trend. But in ThoughtSpot’s case, the company sees GenAI being used a little differently–specifically, to improve its customers’ analytics and business intelligence programs.

When analytics and BI improves at a company, that can benefit them in a myriad of ways, from generating higher revenues and productivity due to making better and faster data-driven decisions, to greater business efficiency and even the creation of data products.

(TSViPhoto/Shutterstock)

“The benefits are either hard benefits, like creating new revenue streams, or improving the decision-making around those revenue streams, and then [improving] the operating efficiencies in that work process,” Howson said.

Studies have shown that only about 25% of employees in the typical organization have the capability to ask questions of the organizations data. In other words, BI and analytics is available only to a quarter of employees. ThoughtSpot’s goal is for 100% of workers to have access to analytics, and it sees GenAI helping to get there.

“That’s part of our mission,” Howson said. “We know that we have low data literacy, and that is an upskilling that everyone is going through. And generative AI, being able to explain the chart or the outlier on the page, is having an impact on that as well.”

GenAI in Analytics

ThoughtSpot is applying GenAI in a few different ways, chief among them by using natural language query (NLQ) to reduce the level of technical necessary to query data (although there are big limits to this; more on that in a bit). Other uses include using GenAI to automate the generation of dashboards and reports and to help spot anomalies in data.

Top reasons for using GenAI (Source: “Generative AI for Data and Analytics: How Early Adopters Are Reaping the Rewards”)

“For a dashboard author, it’s going to take away the doldrums and the silly work that they do and truly elevate them,” Howson said. “For the businesspeople, it will allow them to really ask better questions and become more analytical rather than flying blind…So generative AI, I believe will improve everyone’s work, but the ones that are not learning how to use it, they are the ones that risk being left behind or replaced.”

GenAI “can comb through internal and external databases and retrieve relevant information much faster than executives or knowledge workers could ever do on their own,” ThoughtSpot said in the report. “And it enables people to find the answers they need by asking questions in natural language and exploring results in a conversation, instead of downloading information created by data experts, who may have lacked the business knowledge to make it helpful in practical situations.”

Even before ChatGPT’s arrival, ThoughtSpot was striving to improve that figure through the use of NLQ technology. When ChatGPT demonstrated the awesome power of large language models (LLMs), many companies figured that LLMs could generate coherent SQL as well as it could generate Shakespearean sonnets in English or creating code segments in Java.

Unfortunately, that’s not the case, according to Howson.

“We know that straight text-to-SQL does not work. At best, you get 30% accuracy,” she told BigDATAwire. “What we’ve had in market for 10 years is a proven, patented semantic layer, as well as a number of ranking algorithms, as well as a RAG architecture, so that you’re improving the accuracy. And then lastly, human in the loop to, again, further improve the accuracy.”

Foundations for GenAI Success

You can’t just wake up one day and decide to overhaul your operations with GenAI. Just as companies found with the previous generation of traditional machine learning technology, there are precursor steps that companies typically must complete before they’re in a position to apply the latest, greatest learning tech.

ThoughtSpot Chief Data Strategy Officer Cindi Howson

MIT’s report bares this out. Among early adopters, the top five challenges to GenAI include security considerations, strategic challenges, model usage/quality concerns, data challenges, and implementation challenges. Data management and overall strategy remain big inhibitors, Howson said.

“You cannot do AI without a strong data foundation and you cannot have good impact unless you have aligned to business value,” she said. “There’s a difference between doing proofs of concepts…versus saying we can improve the customer experience, or we can reduce our dashboard backlog and improve analyst productivity and business user productivity. So having those two ingredients is one of the biggest differences.”

At BigDATAwire, we have covered the data management issues of GenAI ad nauseum. As Howson pointed out, getting the line of business and the IT department on the same page is another issue that should not be overlooked.

“There’s so much us versus them and frustration on both sides,” she said. “The data team is too slow. Business gets frustrated. They run off and do their own thing. And it [GenAI] is enabling them to have better conversations about the need and co-innovating.”

For all of the hype, it’s clear that GenAI presents real opportunities. While not all the use cases will pan out, it’s clear from MIT’s report that early adopters already are. The potential of GenAI seems poised to grow considerably over the next few years, making it critical for businesses to make investments today to put them on a path for success down the road.

“The value that we can derive from this in terms of productivity gains, whole new business models, where we’re just getting started,” Howson said. “We are in the dial-up days of the Internet [with GenAI], and people are only just now starting to think of the potential here.”

You can download MIT’s full report here.

Reality Check for GenAI: Deloitte Finds Enthusiasm Tempered by Adoption Hurdles

Google Cloud Research Shows Strong ROI for Early Adopters of GenAI

Source link
lol