MEDICAL LEARNERS’ EXPERIENCES WITH AI TOOLS VERSUS CLINICAL TUTORS: COMPARING CHATGPT, CLAUDE, AND DEEPSEEK

Artificial intelligence (AI) and large language models (LLMs) are starting to play a role in medical education. They give quick access to explanations and can help with clinical reasoning, but we don’t yet know how well they work compared to traditional teaching with a tutor. Learning medicine isn’t just about memorizing facts — it also depends on clear communication, tailoring information to the learner’s level, and making sure knowledge sticks long-term. To decide how best to use AI in training, we need to understand how learners actually interact with these tools and how much they trust them compared with human tutors.

A cross-sectional survey was conducted among medical learners of different levels ( Two medical trainees who have just graduated medical school and doing they medicine rotations, two clinical fellows who have completed at least year in training in nephrology and a medical resident in his 5th year who have not had exposure to being in the dialysis unit yet). They were instructed round in the dialysis unit with a clinical tutor and ask 12 clinical questions related to Hemodialysis. They are then asked to divide the same questions into three groups and assign each group to a different LLMs (DeepSeek, Claude, and ChatGPT). Following this exercise the  participants were given a structured questionnaire designed to capture both quantitative ratings and qualitative reflections of their experience.

The questionnaire included:Background data: training level, frequency of AI use, and approach to prompt formulation.Comparative model ratings: clarity, level appropriateness, simplification of complexity, immediate retention, delayed recall, and accuracy confidence (5-point Likert). Fact-checking practices: whether information was immediately verified, planned for later review, or accepted without verification.Tutor comparison: direct rating of each LLM answer against tutor feedback.Perceptions and preferences: confidence, engagement, trust factors, and willingness to recommend integration of LLMs into curricula. Open reflections: advantages, limitations, and views on complementary use of tutors and LLMs.

All participants reported using LLMs either daily or weekly. While ChatGPT was noted for its accessibility, other models were rated higher in terms of providing more comprehensive and seemingly accurate information. Every respondent reported that they would verify LLM answers against conventional sources such as UpToDate, textbooks, journals, or clinical tutors. All participants were familiar with using prompts to obtain more tailored answers. In terms of clarity, the three models performed similarly and supported immediate recall; however, DeepSeek was rated lower when residents were asked whether they would still remember the information after a week. Although residents expressed concerns about the risk of misinformation, most believed that LLMs should nonetheless be incorporated into medical education.

Large Language Models are increasingly used among medical learners. Educating medical residents about possibility of misinformation and biases is becoming now more urgently needed. Future studies should take place to understand the role of these models in medical learning and decision making.