Lionel Tim-ee Cheng MBBS, Presenter: Nothing to Disclose

Guergana Savova PhD, Abstract Co-Author: Nothing to Disclose

Vinod Kaggal, Abstract Co-Author: Nothing to Disclose

Bradley James Erickson MD, PhD, Abstract Co-Author: Research collaboration, IBM Corporation

Natural Language Processing (NLP) is a rapidly emerging tool for automated clinical data retrieval from electronic records. While proven in the detection of disease, the utility of NLP in discerning the progression of disease from free-text radiology reports has not been studied. We aimed to test the hypothesis that NLP could be used to classify tumor status from unstructured follow-up brain MRI reports.

Consecutive follow-up brain tumor MRI reports (2000-2007; Mayo Clinic, Rochester MN) were manually annotated using consensus guidelines on tumor status (regression, stable, progression). Reports were stratified by tumor status and randomized to machine training (80%) or testing (20%) groups. The features used to train and test a Support Vector Machine model were linguistic markers for tumor status subject and tumor status along with information spread coefficients extracted via Mayo Clinic Information Extraction System. NLP outcomes were compared to manual classification by a radiologist (gold standard).

Of 16417 classification instances (1171 stable, 353 progressed, 165 regressed, 14728 irrelevant) in 772 reports, 13224 (970 stable, 302 progressed, 123 regressed, 11829 irrelevant) were used for machine training, and 3193 (201 stable, 51 progressed, 42 regressed, 2899 irrelevant) for testing. Overall NLP classification was accurate in 2971 of 3193 instances (93%). Compared to manual classification, machine NLP classification was 62%, 20% and 17% sensitive; as well as 98%, 99% and 100% specific, for stable, progressed and regressed disease respectively (NLP mean PPV=55%; mean NPV=98%).

Machine NLP showed good accuracy and high negative predictive value for discerning changes in tumor status from free-text MRI reports. Small numbers of positive (stable, progressed, regressed) instances may have contributed to low sensitivity in this preliminary series, and future algorithm refinements are warranted. These findings suggest that NLP may have novel application in the automated classification of clinical progression or response from electronic databases.

NLP may be able to interpret disease status from unstructured follow-up brain tumor MRI reports, allowing expedient retrieval of valuable therapeutic response data from radiology databases.

Cheng, L, Savova, G, Kaggal, V, Erickson, B, Utility of Automated Natural Language Processing to Discern Changes in Tumor Status from Unstructured MRI Reports.  Radiological Society of North America 2008 Scientific Assembly and Annual Meeting, February 18 - February 20, 2008 ,Chicago IL. http://archive.rsna.org/2008/6005953.html