RELIABILITY OF AI-GENERATED RESPONSES ON FREQUENTLY-POSED QUESTIONS BY PATIENTS WITH CHRONIC KIDNEY DISEASE

 

Certificate Output Instructions

For best output, select "Paper Size" as "A4" and "Margin" as "0" or "None".

To save or print to PDF, please select Print Destination > Save as PDF, enable Background Graphics under "More Settings", then click "Save".

 


 

Certificate Background

   

Presented the abstract " "
(Abstract co-author(s):  )

 

 

E-Poster Presentation

During the congress, E-Posters will be accessible to all participants on the congress website 24/7, as well as in the E-poster stations in the congress center. 

Preparing your E-Poster

Please review the E-Poster format requirements carefully when preparing your E-Poster. Should your E-Poster not meet the mentioned requirements, it may not be displayed as described above.

​E-Poster Submission Deadline

Please prepare and upload your E-Poster no later than March 14, 2026 11.59PM CET. After this date, you will no longer be able to prepare and upload your E-poster and it will not be displayed and accessible on the congress website.​

E-Poster Format Requirements
  • PDF file
  • Layout: Portrait (vertical orientation)
  • One page only (Dim A4: 210 x 297mm or PPT)
  • E-Poster can be prepared in PowerPoint (one (1) PowerPoint slide) but must be saved and submitted as PDF file.
  • File Size: Maximum file size is 2 Megabytes (2 MB)
  • No hyperlinks, animated images, animations, and slide transitions
  • Language: English
  • Include your abstract number
  • E-posters can include QR codes, tables and photos
https://storage.unitedwebnetwork.com/files/1099/7f3bcf84fad1ef29d724110b5cfbc548.pdf
RELIABILITY OF AI-GENERATED RESPONSES ON FREQUENTLY-POSED QUESTIONS BY PATIENTS WITH CHRONIC KIDNEY DISEASE

Please follow the instructions below to input your abstract title.

Abstract titles should be brief and reflect the content of the abstract.

  • The title will not be accepted if it exceeds 25 words.
  • Type in CAPITAL LETTERS.
  • Lowercase may be used for abbreviations only, for example, mRNA.
Emi
Furukawa
Emi Furukawa efurukawa-tho@umin.ac.jp The University of Tokyo Hospital University Hospital Medical Information Network Center Tokyo Japan *
Hiroko Okada okadahiroko-tky@umin.org The University of Tokyo Department of Health Communication Tokyo Japan -
Yuriko Nishiie yuriko.nishiie.728@gmail.com The University of Tokyo Department of Health Communication, Tokyo Japan -
Tsuyoshi Okuhara okuhara-ctr@umin.ac.jp The University of Tokyo Department of Health Communication Tokyo Japan -
 
 
 
 
 
 
 
 
 
 
 

Since the release of ChatGPT in November 2022, AI tools have gained increasing recognition for their potential to generate and disseminate healthcare information. As AI tools have become widely adopted among the general population, they have become primary information sources for patients with chronic kidney disease (CKD) as well. However, as AI tools sometimes generate factual or inaccurate information, known as hallucinations, the reliability of information must be evaluated. This study assesses the reliability of AI-generated information about CKD across major AI tools in Japan, identifying challenges and potential applications for patient education, with implications for the safe and effective integration of AI tools.

In August 2024, we conducted a cross-sectional study analyzing responses from three leading AI tools (ChatGPT-4o mini, Microsoft Copilot, and Gemini) to CKD-related keywords. We entered frequently asked questions related to CKD as prompts based on the keywords most frequently searched for on Google Trends. Each AI response was evaluated using the validated Quality Analysis of Medical Artificial Intelligence (QAMAI) tool, which assessed accuracy, clarity, relevance, completeness, references, and usefulness on a five-point scale ranging from 1 to 5 points. The threshold for each item was set at 4 points or higher. We first investigated which domains posed the greatest challenges in reliability, and then compared the domains that threatened the reliability of each AI tool.

We included 207 AI responses from 23 prompts. The AI tools generated reliable information, with a median QAMAI score of 23 (interquartile range (IQR): 7) out of 30. Relevance and completeness were strong, with 95.2% and 73.4% of responses meeting the threshold, respectively; however, the provision of sources was weakest, with only 48.3% of responses satisfying the criterion. However, information accuracy and resource availability varied (median (IQR): ChatGPT vs Copilot vs Gemini=18(2) vs. 25(3) vs. 24(5), P<0.01), and ChatGPT provided the least accurate information and did not provide any resources. In particular, three (4.3%) of the ChatGPT responses about CKD medications included references to fictitious drug names, and one response provided a false statement of the drug's mechanism (e.g., spironolactone is used to promote the excretion of potassium).

AI-generated responses to CKD-related queries were generally reliable and complete, but occasional critical inaccuracies and insufficient referencing limited accessibility. Ensuring the trustworthiness of information is crucial, underscoring the importance of healthcare professionals critically evaluating and guiding the safe use of AI in patient education.

Kewords