Why Keeping Humans in the Loop Is Critical for Trustworthy AI

As the global generative AI rollout unfolds, companies are grappling with a host of ethical and governance concerns: Should my employees fear for their jobs? How do I ensure the AI models are adequately and transparently trained? What do I do about hallucinations and toxicity? While it’s not a silver bullet, keeping humans in the AI loop is a good way to address a decent cross-section of AI worries.

It’s remarkable how much progress has been made in generative AI since OpenAI shocked the world with the launch of ChatGPT just a year-and-a-half ago. While other AI trends have come and gone, large language models (LLMs) have caught the attention of technologists, business leaders, and consumers alike.

Companies collectively are investing trillions of dollars to get a leg up in GenAI, which is forecasted to create trillions in new value in just a matter of years. And while there has been a bit of a pullback lately, many are banking that we’ll see big returns on investment (ROI), such as the new Google Cloud study that found 86% of GenAI adopters are seeing a growth of 6% or more in annual company revenue.

So What’s the Hold Up?

We’re at an interesting point in the GenAI revolution. The technology has proved that it’s mostly ready, and early adopters are reporting some success. What’s holding up the big GenAI success celebrations, it would seem, are some of the knottier questions around things like ethics, governance, security, privacy, and regulations.

In other words, we can implement GenAI. But the big question is should we? If the answer to that questions is “yes,” the next one is: So how do we implement it while adhering to standards around ethics, governance, security, and privacy, to say nothing of new regulations, like the EU AI Act?

(amgun/Shutterstock)

For some insight into the matter, Datanami spoke to Cousineau, the vice president of data model and governance at Thomson Reuters. The Toronto, Ontario-based company has been in the information business for nearly a century, and last year, its 25,000-plus employs helped the company bring in about $6.8 billion in revenue across four divisions, including legal, tax and accounting, government, and the Reuters News Agency.

As the head of Thomson Reuters’ responsible AI practice, Cousineau has substantial influence on how the publicly traded company implements AI. When she first took the position in 2021, her first goal was to implement a company-wide program to centralize and standardize how it builds responsible and ethical AI.

As Cousineau explains, she started out by leading her team to establish a set of principles for AI and data. Once those principles were in place, they then devised a series of policies and procedures to guide how those principles would be implemented in practice, including with both new AI and data systems as well as legacy systems.

When ChatGPT landed on the world in late November 2022, Thomson Reuters was ready.

“We did have a good chunk of time [to build this] before generative AI took off,” she says. “But it allowed us to be able to react quicker because we had the foundational work done and the program function, so we didn’t have to start to try to create that. We actually just had to continuously refine those control points and implementations, and we still do as a result of generative AI.”

Building Responsible AI

Thomson Reuters is no stranger to AI, and the company has been working with some form of AI, machine learning, and natural language processing (NLP) for decades before Cousineau arrived. The company had “notoriously…great practices” in place” around AI, she says. What it was missing, however, was the centralization and standardization needed to get to the next level.

Carter Cousineau is the vice president of data model and governance at Thomson Reuters

Data impact assessments (DIAs) are a critical way the company stays on top of potential AI risk. Working in conjunction with Thomson Reuters lawyers, Cousineau’s team does an exhaustive analysis of the risks of a proposed AI use case, from the type of data that’s involved and the proposed algorithm, to the domain and of course the intended use.

“The landscape overall is different depending on the jurisdiction, from a legislative standpoint. That’s why we work so closely with the general counsel’s office as well,” Cousineau says. “But to build the practical implementation of ethical concept into AI systems, our sweet spot is working with teams to put the actual controls in place, in advance of what regulation is expecting us to do.”

Cousineau’s team build a handful of new internal tools to help the data and AI teams stay on the straight and narrow. For instance, it developed a centralized model repository, where a record of all of the company’s AI models was kept. In addition to boosting the productivity of Thomson Reuters’ 4,300 data scientists and AI engineers, who have an easier way to discover and re-use models, it also allowed Cousineau’s team to layer governance on top. “It’s a dual benefit that it served,” she says.

Another important tool is the Responsible AI Hub, where the specific risks associated with an AI use case are laid out and the different teams can work together to mitigate the challenges. Those mitigations could be a piece of code, a check, or even a new process, depending on the nature of the risk (such as privacy, copyright violation, etc.).

But for other types of AI applications, one of the best ways of ensuring responsible AI is by keeping humans in the loop.

Humans in the Loop

Thomson Reuters has a lot of great processes for mitigating AI risk, even in niche environments, Cousineau says. But when it comes to keeping humans in the loop, the company advocates taking a multi-pronged approach that ensures human participation at the design, development, and deployment stages, she says.

“One of the control points we have in model documentation is an actual human oversight description that the developers and product owners would put together,” she says. “Once it moves to deployment, there are [several] ways you can look at it.”

For instance, humans are in the loop when it comes to guiding how clients and customers use Thomson Reuters products. There are also teams at the company dedicated to providing human-in-the-loop training, she says. It also places disclaimers in some AI products reminding users that the system is only to be used for research purposes.

“Human in the loop is a very heavy concept that we integrate throughout,” Cousineau says. “And even once it’s out of deployment, we use [humans in the loop] to measure.”

Humans play a critical role in monitoring AI models and AI applications at Thomson Reuters, including things like tracking model drift, tracking the overall performance of models, including precision, recall, and confidence scores. Subject matter excerpts and lawyers also review the output of its AI systems, she says.

“Having human reviewers is a part of that system,” she says. “That’s the piece where a human in the loop aspect will continuously play a very important role for organizations, because you can get that user feedback in order to ensure that the model’s still performing the way in which you intended it to be. So humans are actively still in the loop there.”

The Engagement Factor

Having humans in the loop doesn’t just make the AI systems better, whether the measure is greater accuracy, fewer hallucinations, better recall, fewer privacy violations. It does all those things, but there’s one more important factor that business owners will want to keep in mind: It reminds workers that they are critical to the success of the company, and that AI won’t replace them.

“That’s the part that’s interesting about human in the loop, the invested interest to have that human active engagement and ultimately still have the control and ownership of that system. [That’s] where the majority of the comfort is.”

Cousineau recalls attending a recent roundtable on AI hosted by Snowflake and Cohere with executives from Thomson Reuters and other companies, where this question came up. “Irrespective of the sector…they are all comfortable with knowing that they have a human in the loop,” she says. “They do not want a human out of the loop, and I don’t see why they would want to, either.”

As companies chart their AI futures, business leaders will need to strike a balance between humanness and AI. That’s something they have had to do with every technological improvement over the past two thousand years.

“What a human in the loop will provide is the information of what the system can and can’t do, and then you have to optimize this to your advantage,” Cousineau says. “There are limitations in any technology. There are limitations in doing things completely manually, absolutely. There’s not enough time in the day. So it’s finding that balance and then being able to have a human in the loop approach, that would be something that everyone is ready for.”

Related Items:

Five Questions as the EU AI Act Goes Into Effect

What’s Holding Up the ROI for GenAI?

AI Ethics Still In Its Infancy