Back
Navigating the expansive literature on kidney transplantation poses a challenge for nephrology fellows. Emerging AI models like ChatGPT offer a novel avenue for educational resource curation. This study evaluates ChatGPT's efficacy in identifying accurate, relevant core reading resources in kidney transplantation.
Using the prompt "What are the core reading resources for nephrology fellows on kidney transplantation?", we queried ChatGPT 3.5. The top 20 results were scrutinized for existence, relevance, authorial accuracy, and link validity. Citations and Google Scholar indices were used as relevance metrics.
Of the 20 references suggested by ChatGPT in the field of kidney transplantation, a verification process confirmed the existence of 15 (75%), leaving a non-existence rate of 25% for the references provided. Examination of the existing references revealed a 100% accuracy rate in terms of author attribution. However, the digital object identifier (DOI) was inaccurate in 15 (75%) of the references. Additionally, when links were provided, only 2 (10%) of them were verified to be correct. The presence of a DOI was the most significant predictor for citations, although no strong correlations between the features were found.
While ChatGPT successfully identifies a majority of existing and author-accurate references, the frequent occurrence of incorrect DOIs and links constrains its utility as a standalone resource for transplant nephrology education. Further refinement is needed before ChatGPT can be reliably deployed for academic resource curation in kidney transplantation (Figure 1)