Wellness tech company Tripp announced its AI meditation beta experience Kōkua XR is now available on the Tripp app on the Meta Quest virtual reality platform.
Trained across over 11 million global wellness sessions, the beta version of Kōkua XR leverages Tripp‘s mood data to deliver individualized meditative reflections based on user feedback in real-time.
Initially available on Tripp’s mobile app, the offering expands to VR to meet the demand for personalized wellness experiences in XR. Tripp itself is the name of company’s meditation app and it is available across VR, AR and mobile devices including Meta Quest, Apple Vision Pro, HTC Vive Flow, Pico 3 & 4, PSVR 2, PSVR and Rift/Rift S, iOS and Android.
But Kōkua XR is starting out as an AI experience on the Tripp app on Meta Quest and will move to other platforms later. When you enter the Tripp app there’s a catalog of offerings users can choose from. Kōkua XR is a new experience category.
Lil Snack & GamesBeat
GamesBeat is excited to partner with Lil Snack to have customized games just for our audience! We know as gamers ourselves, this is an exciting way to engage through play with the GamesBeat content you have already come to love. Start playing games now!
Named after the Hawaiian concept of selfless care and support, Kōkua XR leverages Tripp’s arsenal of mood data to generate hyper-personalized, interactive AI-powered reflections in real-time. Since its beta release on mobile in February, Kōkua XR has increased session engagement by over 100%–showcasing the robust appetite for tailored wellness experiences and the power of AI in unlocking hyper-personalization with real-time experience generation.
“This is rapid evolution. I am finding the team is actually now very energized by it. They all feel very proud of this work. I think for everyone in our company, it feels like one of the most revolutionary products they’ve ever worked on,” said Nanea Reeves, CEO of Tripp, in an interview with GamesBeat.
She added, “We’ve pumped it up with a lot of analytics, and are excited to see how people interact with it. Now, there will be a big beta tag all over it. But yeah, it’s exciting. We’ve had it live on mobile for a while in a rudimentary form. Our VR audiences are a lot more active. So for me, it’s a defining moment for us because I’ve always had this vision of this mood on demand. Being able to adapt to you in real time.”
How it works
With Tripp, you speak to it and give it a request. You can look at the response and create a conversation to center yourself and listen to the support. The app gives you calm and peaceful visualizations with colorful lighting. You can ask it to tell you a joke and it will do so.
The proprietary tech enables the app to be agnostic to large language models (LLMs) and so it is connecting to ChatGPT4, Gemini, Claude and more. There will be a Llama integration too. It responds in Reeves’ own voice. Over time, you may be able to modify the voice, perhaps including your own voice.
Tripp uses a scoring method based on the request to determine which LLM is going to return the best response for the user. OpenAI definitely has been winning more than the others, Reeves said.
It takes a few seconds to respond, given latency in the network. Reeves asked it for a meditation to help her go to sleep. It can also create a bedtime story.
“Imagine yourself in that screen for us. Surrounded by the towering trees and the gentle nature. Take a deep breath,” Kōkua XR responded.
The app has multiple environments for you to choose from.
“Without any hard coding, if I change the environment to an underwater environment, it will tell me to notice the fish or notice the coral,” Reeves said.
Tripp is working with GGWP to make sure the community is properly moderated and that the AI is behaving with the proper guard rails. It does so in a privacy preserving way, Reeves said.
“It’s a fascinating new world,” Reeves said. “It’s a really cool use case to use technology to help someone feel better. We don’t have to try to be everything to everyone. We just err on the side of support and how we support the user in the moment.”
Investor feedback
Tripp has been on this journey for years.
“From the very start of the company years ago, the Tripp team has been envisioning personalized, dynamic and immersive experiences that learn and adapt to each user’s state in realtime,” said Tim Chang, venture partner at Mayfield who led Tripp’s seed round in 2017, in a statement. “Tripp has always been methodical about building a data and technology foundation in anticipation of emerging AI capabilities to tailor and optimize mindfulness and consciousness expanding experiences to unprecedented levels.”
Chang added, “Tripp’s breath detection models now paired with their real-time AI generated meditations show the power of how hyper-personalized experiences can be used for positive impact as well as mental and emotional wellness.”
Tripp’s breath detection can be experienced today on Tripp’s “Breathe” category that is powered by AI and focuses solely on breathing exercises. It is currently available on the Tripp app across VR platforms. The breath detection technology uses a gyroscope from the headset to identify your breath pattern.
Built on 11 million sessions
Kōkua XR is a first-of-its-kind approach to wellness that uses AI to not only deliver individualized reflections, but continues to evolve with each session to better support its users. The offering is built on mood data from over 11 million Tripp sessions, where users are asked to self-report how they are feeling before and after each experience.
The result is Kōkua XR’s AI interface that is built from Tripp’s patented and proprietary wellness focused data environment designed to deliver experiences that benefit the people using Tripp’s apps on multiple platforms. Kōkua XR and Tripp’s AI platform is truly powered by the Tripp community.
Tripp’s technology is backed by a number of patents. In this release, the company is calling out two specific patents the first “adapting a virtual reality experience for a user based on a mood improvement score” (patent here) and “adapting media content to a sensed state of a user” (patent here) these patents are for adapting experiences customize wellness experiences based on a user’s mood and kinematic data like heart rate and breathing patterns.
With each interaction, Kōkua XR learns how to better support its users — individually and as collectively across the entire Tripp community. First available on mobile and the Meta Quest, Kōkua XR will be expanding across Tripp’s platforms in the coming months.
“The people who use Tripp have played a unique role in shaping this technology by sharing their feelings with us. Each session allows our members to embody the collaborative and selfless spirit of Kōkua, enhancing support for themselves and others,” said Reeves. “No path to wellness is the same, and with Kōkua we’re ensuring your journey to supporting your mental and emotional well-being is as unique as you are. It’s not just an upgrade; think of it as a whole new way to approach wellness…personal and able to be accessed and shared across multiple platforms.”
Tripp uses two established patents to dynamically customize wellness experiences, responding immediately to changes in a user’s mood and key physical indicators such as breathing patterns collected from the headset itself.
The system collects mood data, breath movement data and generates scores from user surveys before and after sessions to build a mood and emotions vector database, enabling AI to generate tailored meditations.
Thanks to pending patents, Tripp ensures accurate user input with a unique interaction system that allows corrections, cancellations, and interruptions, and delivers snappy, context-rich responses by treating voice as dialogue lines rather than an audio stream through a proprietary atomic delivery method.
The pending patents that are referred to are for ensuring accurate user input with a unique interaction system that allows corrections, cancellations, and interruptions, and delivers snappy, context-rich responses by treating voice as dialogue lines rather than an audio stream through a proprietary atomic delivery method. This unique approach allows Tripp to continually refine its offerings, providing users with the most effective and relevant content for their well-being.
This unique approach allows Tripp to continually refine its offerings, providing users with the most effective and relevant content for their well-being.
“It’s no secret that generative AI has profoundly changed our experience with everyday interactions and experiences, and our approach to wellbeing is no exception,” said Malte Barth, founding partner at Bitkraft who led Tripp’s Series A round in 2022, in a statement. “Nanea and the Tripp team have spent the last 5 years building one of the most powerful engines to deliver meditation and wellness experiences that is unparalleled in its personalized touch and commitment to users. Her vision has been lightyears ahead of most.”
The announcement comes on the heels of several milestones including being named among the Best in Spatial Computing on the Apple Vision Pro, and Founder and CEO Nanea Reeves recognized as a finalist in the Women that Build Awards 2024.
Tripp has also received over 2.6 million five-star ratings in-app on its experiences and is also the most reviewed mindfulness app on the Meta Quest platform.
Future improvements
Based on the feedback so far, users are creating prompts to deal with communication challenges and friendship Issues (makes up 20% of the prompts), anxiety and feelings of worry, positive feelings and social engagement, relaxation and mindfulness and feeling stressed and overwhelmed.
“Kōkua XR was built off of millions of sessions where people sharing their feelings, so I feel like our users have helped us create Kōkua XR, that they’re part of the creation of it,” Reeves said. “An individual’s contribution of just saying how they’re feeling in the moment actually helps Kōkua XR get better at supporting the whole community. And this is really unspoken about a lot of AI and AI powered engines. The AI is not a separate thing. It’s being co-created by the people using our app.”
Reeves said the team will figure out rewards for users in the future for helping the company evolve the technology. There will also be more gamified approaches for those rewards.
“I also want them to feel a part of it when we ask people in our Tripp VR app on the breathing data collection (for the Apple Vision Pro in the future).”
So far, there is about a 70% opt-in to share data to help train the models. AI has had some impact on Tripp itself. As an example, the sound engineers are freed up to work on music in the experiences, instead of cleaning up audio files. There’s a redirection of work that results from the AI assistance for devs.
“There was fear in the company around AI tooling. Now we’ve seen it really expand creativity in the company. We are doing some image generation, and it gives a foundational amount of work in just a couple of minutes rather than days. You can spend the time making the experience more interactive. We’re finding it is helping us produce higher quality audio.”
AI origins
Reeves said Kōkua has been in the works since the start of the company since the original goal has always been to create mood-altering experiences that are personalized to an individual. For the first phase, Tripp needed to collect a data environment to build the engine for Kōkua XR.
At the start of every session, Tripp asked people how they felt at the start of every session. Tripp consulted Walter Greenlee, an adviser and neuroscientist at Standford University, as well as the National Mental Health Innovation Center. This ultimately turned into the 11 million sessions for user feedback.
“Researchers have looked at that data and we realized that we now had a good foundation and could build an engine that would initially just start recommending content for the individual,” Reeves said. “And then we started working with some of the labs to generate our meditation content. And then, ultimately, we created the voice avatar,” using Reeves’ voice for the female avatar and a company executive’s for the male voice.
Eventually, it became clear that Tripp could hold a conversation in real time, taking semantic input from the user, translating that into sentiment, and then passing it through an LLM, and then producing text output. That could then get converted into a voice answer. In case one of the LLMs doesn’t work, it fails over to another LLM that provides the response. The collection of LLMs is dubbed “the council,” Reeves said. To reduce the latency, Tripp created an alternative solution that it patented with “atomic components” that can be cached so that they stream down the client bits at a time.
“It’s all happening in real time,” Reeves said. “We’ve just put in a big analytics layer so we can have themes percolate up to us without looking at the actual data submitted for privacy reasons. We can look at how is Kōkua XR responding and what are people asking it for. There may be edge cases that we didn’t anticipate.”
Tripp has some hard-coded data that can be triggered if someone says they want to harm themselves or harm others. Tripp can intercept those statements with a supportive message and a redirect the person to help lines. This has already been tested on the mobile app. The Kōkua XR app has been in testing since February with a customer group that is 59% male. The responses have evolved over time.
“Over time, the person’s trust builds up and the requests can get more personal,” Reeves said. “From the early data on mobile, we learned that many people are good at explaining how they feel in the moment. So just even having an interface where you can start to develop that language a little bit more and be encouraged and supported through the use of this technology, I think is going to benefit people in being able to say more succinctly, and clearly, how they’re feeling in the moment.”
Tripp has been very popular to date. It is the leading wellness app on the most popular list on Meta.
“This is a major evolution for us in having more adaptive experiences now show up in real time,” Reeves said. “One user uploaded images of his mother, whom he lost, into Tripp, and that has been helping him.. We found that people who personalize the app are 400%, more engaged than those who don’t.”
A lot of wellness apps see a lot of drop off in the first 30 days, but personalization has helped drive the success of Tripp, Reeves said. With Kōkua XR, Tripp has seen about 100% increase in session use, even without any marketing.
The hope is that computer vision will become useful at some point given the personal images that people can upload — again with a privacy layer for protection. If someone says something inappropriate, Tripp will guide that person back to meditation.
“There are ways we can use AI computer vision to integrate them into Tripp even better. It’s exciting and it’s a moving target. We can tune all this on the server side. That’s what’s really interesting. It does require client update to make these adjustments if we’re saying something in real time,” Reeves said.
Kōkua XR is available on the Tripp mobile app for free. In VR, you can experience Kōkua XR in the Tripp app which is available for a monthly or annual subscription: The annual subscription is $3.33/month, or $39.99 annual. The monthly subscription is $9.99/month.
The company has raised $26.3 million to date.
Source link lol
