Asking artificial intelligence for advice can be tempting. Powered by large language models (LLMs), AI chatbots are available 24/7, are often free to use, and draw on troves of data to answer questions. Now, people with mental health conditions are asking AI for advice when experiencing potential side effects of psychiatric medicines — a decidedly higher-risk situation than asking it to summarize a report.
One question puzzling the AI research community is how AI performs when asked about mental health emergencies. Globally, including in the U.S., there is a significant gap in mental health treatment, with many individuals having limited to no access to mental healthcare. It’s no surprise that people have started turning to AI chatbots with urgent health-related questions.
Now, researchers at the Georgia Institute of Technology have developed a new framework to evaluate how well AI chatbots can detect potential adverse drug reactions in chat conversations, and how closely their advice aligns with human experts. The study was led by Munmun De Choudhury, J.Z. Liang Associate Professor in the School of Interactive Computing, and Mohit Chandra, a third-year computer science Ph.D. student.
“People use AI chatbots for anything and everything,” said Chandra, the study’s first author. “When people have limited access to healthcare providers, they are increasingly likely to turn to AI agents to make sense of what’s happening to them and what they can do to address their problem. We were curious how these tools would fare, given that mental health scenarios can be very subjective and nuanced.”
De Choudhury, Chandra, and their colleagues introduced their new framework at the 2025 Annual Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics on April 29, 2025.
Putting AI to the Test
Going into their research, De Choudhury and Chandra wanted to answer two main questions: First, can AI chatbots accurately detect whether someone is having side effects or adverse reactions to medication? Second, if they can accurately detect these scenarios, can AI agents then recommend good strategies or action plans to mitigate or reduce harm?
The researchers collaborated with a team of psychiatrists and psychiatry students to establish clinically accurate answers from a human perspective and used those to analyze AI responses.
To build their dataset, they went to the internet’s public square, Reddit, where many have gone for years to ask questions about medication and side effects.
They evaluated nine LLMs, including general purpose models (such as GPT-4o and LLama-3.1), and specialized medical models trained on medical data. Using the evaluation criteria provided by the psychiatrists, they computed how precise the LLMs were in detecting adverse reactions and correctly categorizing the types of adverse reactions caused by psychiatric medications.
Additionally, they prompted LLMs to generate answers to queries posted on Reddit and compared the alignment of LLM answers with those provided by the clinicians over four criteria: (1) emotion and tone expressed, (2) answer readability, (3) proposed harm-reduction strategies, and (4) actionability of the proposed strategies.
The research team found that LLMs stumble when comprehending the nuances of an adverse drug reaction and distinguishing different types of side effects. They also discovered that while LLMs sounded like human psychiatrists in their tones and emotions — such as being helpful and polite — they had difficulty providing true, actionable advice aligned with the experts.
Better Bots, Better Outcomes
The team’s findings could help AI developers build safer, more effective chatbots. Chandra’s ultimate goals are to inform policymakers of the importance of accurate chatbots and help researchers and developers improve LLMs by making their advice more actionable and personalized.
Chandra notes that improving AI for psychiatric and mental health concerns would be particularly life-changing for communities that lack access to mental healthcare.
“When you look at populations with little or no access to mental healthcare, these models are incredible tools for people to use in their daily lives,” Chandra said. “They are always available, they can explain complex things in your native language, and they become a great option to go to for your queries.
“When the AI gives you incorrect information by mistake, it could have serious implications on real life,” Chandra added. “Studies like this are important, because they help reveal the shortcomings of LLMs and identify where we can improve.”
Citation: Lived Experience Not Found: LLMs Struggle to Align with Experts on Addressing Adverse Drug Reactions from Psychiatric Medication Use, (Chandra et al., NAACL 2025).
Funding: National Science Foundation (NSF), American Foundation for Suicide Prevention (AFSP), Microsoft Accelerate Foundation Models Research grant program. The findings, interpretations, and conclusions of this paper are those of the authors and do not represent the official views of NSF, AFSP, or Microsoft.
Biomedical engineers at Georgia Tech created a treatment that could one day unlock a universal strategy for treating some of the hardest-to-treat cancers — like those in the brain, breast, and colon — by teaching the immune system to see what it usually misses.
Their experimental approach worked against those kinds of cancers in lab tests and didn’t damage healthy tissues. Importantly, it also stopped cancer from returning.
While the therapy is still in early stages of development, it builds on well established, safe technologies, giving the treatment a clearer, quicker path to clinical trials and patient care.
Reported in May in the journal Nature Cancer, their technique is a one-two punch that flags tumor cells so they can be recognized and then eliminated by specially enhanced T cells from the patient’s own immune system.
Click here for the full story.
Researchers from Emory University and Georgia Tech developed a smartphone app to screen for anemia using images of a person’s fingernail beds instead of the conventional blood test.
Anemia, which is believed to affect over 2 billion people worldwide, occurs when someone does not have enough healthy red blood cells or hemoglobin to carry oxygen to the body’s tissues.
The color of the fingernail bed reflects the amount of hemoglobin in the blood. Hemoglobin is the protein in red blood cells that carries oxygen through the body and gives blood its red hue.
When hemoglobin levels drop, fingernail beds appear paler.
The AI-powered algorithm in the new app was tested on over 9,000 users, whose fingernail selfies were compared to their clinical blood test results.
Within seconds of submitting a fingernail photo, the tech estimated hemoglobin levels with remarkable accuracy.
Click here for the full story!
The Children’s Healthcare of Atlanta Pediatric Technology Center at Georgia Tech (PTC) has named Natasha Rishi-Bohra as Deputy Director, effective June 2. In this role, she will serve as a strategic leader and primary point of contact for the PTC.
Natasha brings more than 13 years of experience at the intersection of health strategy, innovation, and operations. She has led major transformation efforts across healthcare systems, focusing on advancing value-based care, improving access, and implementing scalable solutions across diverse populations.
Most recently, Natasha held leadership roles at Evolent, where she directed enterprise-level strategy and performance solutions in specialty and primary care. Her work included designing and operationalizing care models to close clinical gaps, reduce avoidable hospitalizations, and create value-driven partnerships with provider organizations.
Previously, she served as a senior strategy leader at the New York City Department of Health and Mental Hygiene, where she led programs to support the adoption of value-based payment models and integrated care delivery for high need primary care practices. She has also advised national and global health organizations through roles at Deloitte Consulting and Global Impact Advisors, driving strategic planning, policy implementation, and system-level innovation.
Natasha also brings a deep commitment to teaching and mentorship. She currently serves as adjunct faculty at the Emory University Rollins School of Public Health, where she guides students in applying health policy analysis to real-world challenges. Previously, she held a similar faculty role at NYU. Natasha holds a Master of Public Health from Boston University, with a concentration in Maternal and Child Health.
We are excited to welcome Natasha to this important role and look forward to working with her to further the work of the PTC.
Sincerely,
Stanislav Emelianov, PhD
Co-Director, Children's Pediatric Technology Center at Georgia Tech
Wilbur Lam, MD, PhD
Co-Director, Children's Pediatric Technology Center at Georgia Tech
The Children’s Healthcare of Atlanta Pediatric Technology Center at Georgia Tech (PTC) has named co-leads for its work in Technology and Devices (Pillar 3)—Steven Goudy, MD, and Omer Inan, PhD as co-leads for its work in Technology and Devices (Pillar 3).
Steven Goudy, MD, and Omer Inan, PhD, are co-leaders for Pillar 3. Dr. Goudy is Division Chief of Otolaryngology at Children’s and Emory. His clinical interests include cleft lip and palate, head and neck tumors, maxillary development, Pierre Robin sequence, vascular malformations, and velopharyngeal insufficiency. His research focuses on craniofacial bone regeneration and the basic biologic mechanisms that control facial bone and soft tissue regeneration. In addition to his clinical work and research, Dr. Goudy has experience in company formation and philanthropy.
Dr. Inan is Regents Entrepreneur and Linda J. and Mark C. Smith Chaired Professor of Electrical and Computer Engineering at Georgia Tech. Over the past 12 years, he has worked with multiple investigators at Children’s on dozens of projects. He is interested in designing clinically relevant medical devices and systems and translating them from the lab to patient care applications. Developing new technologies for monitoring chronic diseases, such as heart failure, at home is a focus of his research.
Please join us in welcoming Dr. Goudy and Dr. Inan to their new roles. We are inspired by the excitement they bring to the PTC, as we look forward to continuing the work to enhance the lives of children and young adults. This continued collaboration between clinical experts and scientists will bring discoveries to the clinic and the bedside.
Sincerely,
Stanislav Emelianov, PhD
Co-Director of the PTC, Georgia Institute of Technology
Wilbur Lam, MD, PhD
Co-Director of the PTC, Children’s Healthcare of Atlanta