Some electronic surveillance systems are equipped with natural language processing to help detect complications like catheter-associated urinary tract infections, but how well do they work?
NLP algorithms gather data recorded in clinical notes without manual review. To test their accuracy, researchers compared the results of an NLP-augmented algorithm with standard surveillance, including electronic and manual data extraction, at a large, urban tertiary care Veterans Affairs hospital.
All patients admitted to the acute care units and the intensive care unit of the hospital from March 1, 2013, through Nov. 30, 2013, were included in the study, which was recently published in Infection Control & Hospital Epidemiology.
Here are three findings from the study.
1. The NLP-augmented algorithm identified 27 percent more indwelling urinary catheter days in the acute care units and 28 percent fewer indwelling urinary catheter days in the intensive care unit than standard surveillance identified.
2. The algorithm flagged 24 CAUTIs versus standard surveillance methods, which uncovered 20 CAUTIs. The CAUTIs identified were overlapping but not the same.
3. Overall, the positive predictive value was 54.2 percent, and overall sensitivity was 65 percent.
According to the researchers, developing and implementing a NLP algorithm demanded considerable upfront effort of behalf of the clinicians and programmers to determine current language patterns.
Ultimately, the study revealed NLP algorithms were most useful for identifying simple clinical variables but not as effective as standard surveillance methods for detecting CAUTIs.