Today I’m delighted to say we have registered the first known IRB-approved study of its kind on clinicaltrials.gov – a trial that seeks to explore how a reduced set of large language model (LLM)-augmented Woebot features compares with the same features in Woebot as it is today. We’ll soon start recruitment for this study, our 14th randomized control trial to date, to help us understand how LLMs can be applied to accelerate the delivery of safe, engaging, and potent digital solutions.
This milestone comes after we opened a conversation about how ready generative AI really is for mental health and noted that further work is required to build, test, and regulate any new technology that may be used in a healthcare setting. Here are some additional details regarding what we’ve been doing and thinking about over the last few months.
While there are still some challenges when it comes to their generative abilities, we have observed that prompts that combine LLMs for understanding with human-composed responses provide the most improved therapeutic delivery. We’ll share more about the engineering approaches we developed along the way in an upcoming technical blog post. What we learn in this study will help us identify the use cases that are ready to be scaled and studied in future versions of our products.
Science in the Loop
When people get sufficient access to CBT at the right time, they can make their own breakthroughs. But despite being the most widely recommended treatment for a range of common mental health problems, there has been minimal progress in making high-quality CBT radically accessible (engaging, personalized, available) or more potent in any systematic way. The last decade has seen a slew of digitized CBT products with outcomes rivaling those from traditional care, but we don’t seem to be able to move the needle on outcomes. Indeed, despite thousands of studies over the last 50 years, there has been no great advancement in the treatment and research of mental disorders.
As with most big advances in medicine, it is not just the technology or the tool but their combination with emerging basic science and experimental methods that accelerates the field. Similarly, at Woebot Health, we are advancing our work on LLMs but in combination with experimental methods and a mechanistic framework of mental health to improve the product, experience, and outcomes. We’re calling this methodology Science in the Loop™ and believe it will speed up the delivery of highly engaging, safe, and efficacious solutions.
Breakthroughs in oncology and other disease areas had similar histories, moving from one size fits all paradigms to ones where new tools allowed massive advances in understanding underlying mechanisms and the development of new products to target them. Access to targeted cancer treatments today seems normal, but 20 or 30 short years ago, medicine took a one-size-fits-all approach. Twenty percent of women with breast cancer used to follow treatments that failed. It wasn’t until the mechanism that characterized some of these tumors was discovered as having a genetic basis in human epidermal growth factor receptor 2 (HER-2) that a range of transformative therapies was launched. Mental health is now at its own inflection point for precision.
Because of the privileged position that Woebot has in people’s lives, the data we have gathered is a meaningful, rich, and ecologically valid reflection of users’ lived experience, i.e. a “real world” mapping of their experience. Our scientists can systematically identify factors—complex cognitive, emotional, behavioral, and social—that interfere with their therapeutic journey. Then our expert clinicians and translational scientists are the humans in the loop that create treatment adaptations to target those mechanisms. Using this approach, we recently found that baseline resilience was a significant predictor of outcome change in an 8-week study of our adult product (see the recently published paper here).
We’re honored to be in a position to take advantage of some of the greatest leaps in technology that the world has seen. As we continue to mature in our application of LLMs to drive improved engagement and outcomes, we are also excited to see what advances we can make within the field of psychotherapy itself, and to unravel novel mechanisms underpinned by new interactional capabilities. As we continue to infuse science in the loop of our utilization of LLMs, we’ll continue doing what science does best: designing rigorous studies and sharing those results with the broader community so we may build this new paradigm, together.
