ASSESSING THE DIAGNOSTIC CAPABILITIES OF GPT-4O MINI IN HYPONATREMIA: POTENTIAL AND LIMITATIONS

7 Feb 2025 12 a.m. 12 a.m.
WCN25-AB-941, Poster Board= FRI-068

Introduction:

Hyponatremia is a complex diagnosis that requires precise identification of its underlying causes for effective treatment. Many early-stage providers struggle with this, leading to increased morbidity and prolonged hospital stays. As artificial intelligence (AI) gains traction in healthcare, we evaluated GPT-4o mini for its potential in diagnosing hyponatremia. GPT-4o mini stands out for its targeted, resource-efficient, and quick diagnostic capabilities, along with easy deployment and regulatory compliance, making it a promising tool for complex medical tasks in diverse clinical settings. If proven effective, it could significantly aid in identifying the etiology of hyponatremia, enabling more accurate and timely treatment.

Methods:

Objective: To evaluate the accuracy of GPT-4o mini in diagnosing the etiology of hyponatremia both with and without the provision of predefined answer options. Method: 14 different scenario's involving multiple causes of hyponatremia, iso-osmolar (hyperlipidemia and Hyperparaproteinemia), hyperosmolar (hyperglycemia, other solutes like mannitol and combination of hyperosmolar and hypo-osmolar) and hypo-osmolar (hypovolemic from renal and extrarenal source, SIADH, primary polydipsia, low solute diet, SIADH, hypothyroidism, adrenal insufficiency, heart failure, cirrhosis and renal failure) were provided. For each scenario, the following data were provided: serum sodium, osmolarity, glucose, TSH, cortisol, BUN, creatinine, urine osmolarity, urine sodium, and volume status.. Iso-osmolar scenarios were assessed both with and without lipid and free light chain levels. Data Collection: GPT-4o mini's responses were evaluated with and without predefined answer options provided for all cases on two separate occasions, 24 hours apart. The answer options included: A. Hyperlipidemia or Hyperparaproteinemia B. Hyperglycemia C. Mannitol use D. Diarrhea or prior diuretic use E. SIADH F. Current diuretic use with a thiazide diuretic G. Primary polydipsia or beer potomania H. Heart Failure or Cirrhosis I. Renal Failure J. Hypothyroidism K. Adrenal Insufficiency.

Results:

Evaluation

Correct Answers

Incorrect Answers

% Correct

First time with options

13

1

92.8%

First time without options

13

1

92.8%

Second time with options

13

1

92.8%

Second time without options

13

1

92.8%

GPT-4o mini consistently identified 13 out of 14 correct answers across all scenarios, regardless of whether predefined answer options were provided or whether the test was repeated after 24 hours. The model consistently failed to accurately diagnose iso-osmolar hyponatremia when lipid and free light chain levels were not provided, although it did suggest the possibility of pseudohyponatremia. Nevertheless, GPT-4o mini correctly identified the etiology in all other cases of hyponatremia, as described in the methodology.

Conclusions:

GPT-4o mini accurately identified the etiology of hyponatremia in most cases, though it occasionally lacked the correct scientific reasoning. The error in iso-osmolar hyponatremia highlights the need for deeper understanding and more detailed data input in complex cases. While not yet fully suitable for clinical use, GPT-4o mini shows potential as a future guiding tool with proper clinical correlation.

I have no potential conflict of interest to disclose.

I used generative AI and AI-assisted technologies in the writing process.
During the preparation of this work the authors used ChatGPT 4.omini in order to improve readability. AI was not used to draw inference and conclusion from the results. After using this tool/service, the authors reviewed and edited the content as needed and take full responsibility for the content of the publication.