ChatGPT Effectively Modifies Medical Notes to Patient-Friendly Formats

Credit: Unsplash

Several case studies are emerging for the use of artificial intelligence (AI) to help improve patient literacy and education, according to findings presented at the 2024 American College of Physicians Internal Medicine Meeting.^1,2

In one case study presented as a poster,¹ ChatGPT effectively and accurately modified chronic kidney disease (CKD) text into patient friendly and easily readable information, providing the potential to substantially improve patient literacy. In another presentation,² on applying augmented intelligence and machine learning, a presenter showed an effective case study of translating their discharge instructions from medical abbreviations and text into a patient friendly format.

In the poster experiment,¹ completed by Oscar Pena, MD, and Monika Mishra, MD, both at the Hartford HealthCare St. Vincent's Medical Center, researchers identified 33 questions and answers specific to CKD that were related to risk factors, disease, and forms of kidney replacement therapy. For each, they prompted ChatGPT to "make the text readable for someone at 6th grade or below." The output from the artificial intelligence (AI) tool were then tested against Flesh Kincaid grade level (FKGL) test to assess the final reading level and whether the process effectively converted the medical text into an easily readable format.

"Our study demonstrated that ChatGPT, an artificial intelligence tool, can make medical information on CKD more comprehensible and more accessible to everyone, regardless of their literacy levels," the authors wrote in their poster. "This shows that Al has the potential not only to convey information effectively but also to tailor it to specific audiences.”

The text that was used for the simplification process was from a recompilation of materials from the National Kidney Foundation website. The reading level varied on each of the 33 answers at baseline. The average original reading level for these answers was grade 9.1 (confidence interval [CI], 8.3-10, standard deviation [SD], ± 2.4). After the ChatGPT modifications, the reading level average was grade 7.0 (CI, 6.5-7.6, SD ± 1.6), per the FKGL test.

The original readability score varied significantly from answer to answer, with the most complicated reaching grade nearly to grade 17 in complexity. The most dramatic improvement was seen on a question originally scored at a reading level of grade 11.8. In this example, ChatGPT modified the answer to a grade 5.2 reading level. Each result represented a statistically significant improvement in readability (P <.05).

"By using Al to enhance readability, we can increase awareness and understanding of CKD among the general public, especially those who may find traditional medical information too complicated," the authors noted. "Our research indicates that incorporating Al into health communication strategies can improve public health literacy and enable individuals to make informed decisions about their well-being."

In another talk at the ACP meeting,² Ivana Jankovic, MD, Adjunct Clinical Assistant Professor, Division of Endocrinology, Diabetes and Clinical Nutrition Oregon Health & Sciences University, presented similar success translating medical jargon into patient friendly output. This case study was in diabetes and sought to change medical notes into a grade 8 reading level.

For the input, Jankovic included discharge instructions (abbreviated in the prompt as D/C). The input included several common abbreviations, such as BID, BG, qAC, QIDHS, and PCP and the prompt was again a simple "please write this following instructions at an eight grade level." The response not only simplified the instructions for a patient but also included addition tips, such as setting phone reminders and keeping a blood sugar log.

“It did a really good job of breaking this down for the patient. Not only did it do this but without me asking, it gave tips,” Jankovic said. “It can make your job faster and easier. It can take things to the level of your patient without you expending effort for it.”

References

1. _{Pena, O, Mishra M. ChatGPT in CKD: Using Artificial Intelligence for Health Literacy. Poster presented at: ACP Internal Medicine Meeting; April 19, 2024; Boston, MA. https://annualmeeting.acponline.org/ Accessed April 19, 2024.}
2. _{Pandita D, Chen J, Jankovic, I. Applying Augmented Intelligence and Machine Learning in Your Practice. Session presented at: ACP Internal Medicine Meeting; April 19, 2024; Boston, MA. https://annualmeeting.acponline.org/ Accessed April 19, 2024.}