Using GPT-4 to Automate the Generation of Lay Summaries for Cancer Publications

Abstract

Abstract Background Cancer research literature is often riddled with technical jargon that is not digestible to the average person. Individuals interested in research studies may want to contribute through patient partner engagement or sample donation but find the relevant literature overwhelming. Through the generation of lay summaries, previously inaccessible research papers become easier to comprehend, especially for patient partners or data donors. With large language models (LLMs) continuing to advance, so does their capability to summarize large texts. Objectives In this study, we examined whether LLMs can produce lay summaries of scientific literature at-scale, while maintaining readability and accuracy to their source texts. Methods We developed a tool to generate lay summaries of open-access article abstracts and their full texts with GPT-4-Turbo. Prompt development aimed for a target 8th grade reading level assessed with Flesch-Kincaid Grade Level. Human-review metrics were used to evaluate readability and accuracy when generated using abstracts versus full text articles. Results The average Flesch-Kincaid Grade Level Score was 7.13 for abstract-based summaries and 7.39 for full text-based summaries, indicating summaries at around 7th grade reading level. Human-review metrics showed these summaries were of similar readability and accuracy when generated using abstracts versus full text articles, with mean accuracy scores from human review of 7.09 vs 7.42 out of 10 respectively. Additionally, qualitative patient-based assessment indicated these summaries would encourage participation in research studies. Conclusion By generating lay summaries for complex and lengthy research papers, their scientific information becomes accessible to a larger audience, including patient partners interested in contributing to cancer research. Summaries that are easy to understand will allow participants to make informed decisions about their involvement and appreciate the impact of their contributions if and when their results are published. Lay Summary This study explores if artificial intelligence (AI) can help make hard to read cancer research papers easier to understand for members of the public. Problem When people donate cancer tissue samples or participate in research studies, they often want to know how their contributions are being used. However, scientific papers are full of technical language that’s hard for most people to grasp. People in past studies have said this can make them less willing to take part in research. Methods The study created a computer program using AI (GPT-4-Turbo) to turn complex kidney cancer research papers into simple summaries. They tested whether the AI could summarize both short abstracts and full-length papers effectively. They aimed for summaries at a 6th to 8th-grade reading level. This was to follow Canadian and U.S. health communication guidelines. Results The AI created 106 summaries. Computer measures showed the summaries were close to a 7th-grade reading level. Though, researchers had to tell the AI to write for a 2nd-grade audience to achieve this. Of note, summaries from short abstracts were just as accurate and readable as those from full papers. Eighteen volunteers, including five patient partners, reviewed the summaries and rated them for clarity and accuracy. They were rated at around 7 out of 10 points. All patient partners said these summaries would help them decide whether to join research studies and feel more informed about how their contributions matter. Why It Matters This tool could help patients and donors better understand research without needing a science degree. When people can see how studies work, they are more likely to participate in future research. While patient partners emphasized the need for summaries to be accurate and reliable, this approach shows promise as a unique strategy to better connect the public with research.

Ähnliche Arbeiten