Robyn Alexandra Cairns MD, FRCPC, Presenter: Consultant, McKesson Corporation

Thomas Rosenal MSc, MD, Abstract Co-Author: Nothing to Disclose

Francis Y. Lau MSc, PhD, Abstract Co-Author: Nothing to Disclose

The performance of the NCBO Annotator for correctly extracting all clinically relevant terms and excluding irrelevant terms is limited but RadLex encoding of correctly extracted terms is more accurate. The NCBO Annotator shows promise as a viable alternative to time intensive manual processing for simply identifying and encoding existing RadLex preferred concepts in free text radiology documents.

Content from narrative medical imaging (MI) reports could be more efficiently applied to clinical decision support and other applications in electronic health records if coded relevant terms were generated from the MI reports. The National Center for Biomedical Ontology (NCBO) Annotator is an open source ontology-based web service that automatically identifies ontologic terms from free text and returns codified terms. The objective of this study was to determine if domain relevant terms can be accurately extracted and encoded from free text paediatric orthopaedic MI reports using the NCBO annotator with RadLex, a terminology for radiology, applied as the reference ontology.

The NCBO annotator results for extraction and RadLex encoding of retrospectively collected free text paediatric orthopaedic MI reports were evaluated separately for 51 reports. The overall extraction performance, recall (sensitivity) and precision (positive predictive value), of the NCBO Annotator was evaluated by comparing the automated extracted terms to a “gold standard list” of relevant terms manually generated for each report by a paediatric musculoskeletal radiologist. The contextual correctness of coding of the relevant NCBO Annotator extracted terms was also assessed by comparing the NCBO Annotator RadLex encoding results with manual RadLex encoding results.

A comparison of 1055 NCBO extracted terms with the 711 manually identified terms included in the 51 gold standard extraction lists demonstrated a recall rate of 50% and precision of 34% for the NCBO Annotator. Analysis of the encoding of the relevant NCBO extracted terms (n=353) found that 98% of the terms were RadLex exact match terms and these terms were encoded in correct clinical context 88% of the time.

Cairns, R, Rosenal, T, Lau, F, An Evaluation of Automated RadLex Encoding of Free Text Pediatric Orthopedic Medical Imaging Reports.  Radiological Society of North America 2014 Scientific Assembly and Annual Meeting, - ,Chicago IL. http://archive.rsna.org/2014/14004710.html