THE PREDICTIVE PERFORMANCE AND USAGE OF ARTIFICIAL INTELLIGENCE (AI)-BASED FISTULA FAILURE MODEL IN SINGAPORE HEMODIALYSIS (HD) CLINICS

Introduction:

Arteriovenous fistulas (AVFs) are the preferred vascular access (VA) for Hemodialysis (HD) due to their longevity and lower risks of complications especially infection. AVF failure can lead to significant morbidity in HD patients, leading to setbacks in patient management. Thus, there is a substantial need to identify patients at risk for AVF failure [1]. Recently, Artificial Intelligence (AI) algorithms have been built using common clinical measurements to predict personalized Risk-based AVF failure. The Fistula Failure Model (FFM) represents a such clinical decision tool. The AI risk estimation tool was integrated into the EuCliD® system and provides a monthly estimate of Fistula Failure Risks (FFR) over the next 90 days. The objective of the current study was to evaluate the usage and efficacy of the FFM [2] in the Fresenius Medical Care (FMC) HD clinical setting in a Singapore HD cohort compared with other countries.

Methods:

This retrospective study analysed 83,126 electronic health records of adult patients receiving HD in 7 countries (Australia, Czech Republic, Italy, Portugal, Singapore, Slovakia and Spain) to the compare performance of FFM in Singapore with its global performance. The data was collected for a period of 12-month from February 2023-January 2024. All the patients were registered in the European Clinical Database (EuCliD, FMC) and consented to utilize their pseudo-anonymized data for secondary data analysis. The predictive accuracy compared to actual failure incidence was assessed using Area Under the Curve (AUC) of the Receiver Operating Characteristic (ROC). Physician’s degree of agreement with FFM prediction was assessed using a 5-point scale- Strongly Agree, Agree, Neither Agree or Disagree, Disagree, and Strongly Disagree.

Results:

For the Singapore cohort, 9,795 records from 1,031 patients were analysed. The ROC - AUC score for the overall FFM performance in Singapore was 0.71 with a fistula failure incidence of 8.1%. The global ROC-AUC score for FFM across the 7 countries was 78.1%, with a global failure incidence percentage of 6.2%. For the other 6 countries combined, 73,331 records from 9,275 patients were analysed. The ROC - AUC score for the overall FFM performance of these countries was 0.79 with a fistula failure incidence of 5.9%. Table 1 shows the number of records analysed, the percentage of fistula failure incidence, and ROC-AUC score for each country included in the analysis. From the period between July 2023 to June 2024, we observed an increase in the overall usage of FFM overtime. However, the model usage varied between different clinics in Singapore. Eleven out of 33 clinics (33.3%) demonstrated minimal utilization of the FFM predictions. Lastly, we also observed an increase in the physician’s degree of agreement for validated AVF risk scores over time (Figure 1).

Table 1: Number of records, fistula failure incidence, and ROC-AUC score for different countries:

Conclusions:

The AI-based FFM demonstrated a robust global predictive performance though with some variation across different clinical settings. The overall FFM performance in Singapore was acceptable, but localized adaptation is required to further improve the model’s performance. Fine-tuning the FFM to meet the needs of various clinical settings will help reduce time-consuming procedures and costs, while achieving better outcomes for patients.

I have potential conflict of interest to disclose.
This research is funded by Fresenius Medical Care.

I did not use generative AI and AI-assisted technologies in the writing process.