CAPUTRING WATER TESTING RESULTS: A PRACTICAL APPLICATION OF LARGE LANGUAGE MODELS IN NEPHROLOGY

Artificial intelligence (AI) has made a worldwide impact, but its translation to Nephrology remains nascent. We share an example of a use case applicable to nephrologists worldwide. Water testing for reverse osmosis (RO) water and dialysate is important for safe haemodialysis. This ensures patient safety and is mandated worldwide, and the Association for the Advancement of Medical Instrumentation (AAMI) guidelines are widely followed. Prior to this, our water testing reports were in PDF format (Figure 1), which is not computable. Thus, record keeping was paper-based previously without an electronic repository. 

An assistant was created in using PAIR, an AI chatbot developed by GovTech Singapore, powered by Claude Sonnet running in a local environment. Retrieval augmented generation was used, and documents which specified acceptable dialysate and RO water parameters were referenced. Prompts were crafted to instruct the assistant to extract the relevant information, and display this in table format. The input was one or multiple PDF files, and the output was a table of the water testing results, with each row representing a single result of a specific machine on a specific date (Figure 2). Microsoft Excel and Sharepoint were used, allowing data visibility to all stakeholders. We gradually rolled this out all clinical areas performing dialysis in the hospital.

There has been positive user feedback from all clinical areas for this, with benefits at user level and institutional level. At the user level, the solution saves time by enabling users to obtain quick feedback if testing meets the required standards and thus if repeat testing is required. Also, this facilitates compliance as all users are able to see at a glance if requisite tests have been sent off for every machine for that month. Prior to this, some clinical areas were manually keying in results into an excel file and time would be taken for data entry. Using an AI solution improves data accuracy by reducing human error. Overall, we estimate that at least 10 man-hours a month are saved in total.

The benefit for the institution is that data is not manually filed. Data is now captured in an institutional repository and can be queried years down the road if needed. Also, as the data is now stored in a relational database, the machine ‘tag ID’ and date of water testing may act as keys and allow data transformation to view the results of each machine sorted chronologically, which is useful during audits. There were no hallucinations noted as the primary function was for character recognition and data transformation.

There are potential improvements to further enhance the solution, such as adding a result acknowledgment module for doctors to electronically acknowledge results, improving the user interface, and using Python to automate some of the processes. We would like to share a use case of successful application of artificial intelligence in a Nephrology unit. This is potentially scalable as water and dialysate testing is ubiquitous for all haemodialysis units. We believe this approach may allow institutions to turn non-computable file formats to relational databases and useful knowledge.