Drag queens the future of healthcare? At least Healthvana thinks so

How is Healthvana helping engage people about their health? 

In one of the most unexpected ways possible: Drag queens. 

“Drag queens are about acceptance and taking you as you are,” Gabriella Palmeri, VP of partnerships at Healthvana, said onstage this week at VB Transform. “You’re getting the unvarnished, non-documental empathetic truth. And I’ll also add that the persona is fun, it’s de-stigmatizing.”

Taking this unique approach, the patient engagement platform company is already serving tens of thousands of patients and has a lofty goal to reach one million by the end of 2024. 

“We help digitize the experience for patients,” Ramin Bastani, CEO of Healthvana, said at VB Transform. “You may never have heard of us, but the patients who use us actually love it.”

Reaching patients where they are

Bastani began the session by polling the audience:

“How many people in here use a patient portal?” (There was a show of hands.)

“How many people like their patient portal? Does anyone like their patient portal?” (Another show of hands.)

“Okay, so they suck for the most part,” the executive deduced.

Healthvana is aiming to upend this with its platform, which it says is one of the largest patient-facing AIs in the U.S. The bot is connected to patients’ health histories and past chats and provides instant responses.

“As a patient, it knows your health history, it has existing chats and conversations in any language you want, and the responses are instant,” said Bastani.

When patients log in, they have the option to have a typical human exchange, talk with a standard AI or interact with the drag queen AI.

Palmeri explained that patients are using the platform to have serious, open conversations, getting answers instantly. For example, they can ask urgent questions about medications. But the bot can also handle the playful ones as well (like ‘Should I be worried about it raining men?’ to which it might respond, ‘If it’s raining men, grab your most fabulous umbrella and let the blessing shower over you.’)

Silly as it may seem, 80% of patient conversations are happening with the drag queen persona, Palmeri noted.

Also importantly, 25% of messages are occurring outside of clinic hours. “So we’re engaging patients that are traditionally hard to reach, and we’re doing that on their terms,” she said.

Slow, iterative, deliberate build

Bastani noted that healthcare providers simply can’t keep up with all the messages patients send, and they are in desperate need of help from AI. “We didn’t build this technology in search of a problem,” he said. 

More than 100 clinicians tested the platform before it was provided to patients in a “very slow rollout,” Palmeri explained. This was not only due to regulatory, compliance and patient health and safety reasons, but because patients generally interact with AI in a fundamentally different way than they do with humans. 

Initially, she noted, just five patients were interacting with the AI; now, 50,000 patients across 15 states are in Healthvana’s early access program. 

Working with OpenAI, Healthvana “built this slowly, iterated and we didn’t move on until we were comfortable to do so,” said Palmeri. The company has a “very rigorous” testing and prompt generation process based on its 10 years of answering patient inquiries and data from more than one million patient-provider messages. 

“We think we’re several months ahead of a lot of folks on evaluation because we’re dealing with real patients and real information,” said Bastani. 

He emphasized, however, that Healthvana has a zero retention environment with OpenAI — meaning the AI company does not keep any of the patient-provider conversations.

Three categories of rigorous evaluation

One of the biggest reasons for Healthvana’s success is its evaluation process, Palmeri noted. Everything the AI sends is reviewed by two content moderators within 24 hours. It’s rated based on accuracy, comprehensiveness and contextual sensitivity. 

“So was the message correct, was it complete and did it make sense in the context of the conversation?,” said Palmeri. 

Anything that didn’t score a five — five being the best — is then put into a product feedback loop to fix.

“We have this human review process that has tens of thousands of message ratings to improve the message evaluation model,” Sam Warmuth, Healthvana CPO, said at VB Transform. “We’ve had multiple reviewers, human reviewers, reviewing every message. It’s still going on to this day for every single message.”

Warmuth ultimately detailed three categories of evaluation, all of which are critical. 

The first is the model evaluation or the initial investigation research for the best model(s). After that is single-turn evaluation — that is, ‘Does this agent actually solve the problem that the user is trying to solve?’ Finally, the multi-turn or multi-channel application involves actual back and forth between the AI and the user. 

“Multi-channel evaluation means you have an input and an output, but then that output goes back into the next input,” said Warmuth. “You constantly are going back and refining what you’re talking about with the AI.”

On the back end, multiple models are also interacting with each other, pulling out APIs, reading and entering information into databases, using various tools, then communicating with the simulated human for multi-turn workflows, he said. 

Healthvana also uses chain point detection, which identifies when something is off based on the last evaluation. For instance, there may be a workflow that is usually resolved by AI in 20 messages — then suddenly it takes five. 

“That could be good,” he noted. “That could mean that your AI is working better. But it also could be an indication that somehow the LLM is taking a shortcut and the patient’s not having the right experience.”

This means, of course, that they’re not being their most fabulous self.