Brian E. Chapman PhD, Presenter: Nothing to Disclose

James Y. Chen MD, Abstract Co-Author: Research Consultant, EBM Technologies, Inc Research Consultant, Banyan Biomarkers, Inc

Asako Miyakoshi MD, Abstract Co-Author: Nothing to Disclose

Wendy Chapman PhD, Abstract Co-Author: Nothing to Disclose

Amilcare Gentili MD, Abstract Co-Author: Nothing to Disclose

Showing radiologists the cues in context did not significantly change their probability assignments, overall. However, assertion cues changed more than negation cues. Evaluating probability assignments for lexical assertion, negation, and uncertainty cues may not require displaying the cues in context. 

Understanding how uncertainty is expressed in radiology reports is a critical task for natural language processing applications. pyConTextNLP is a natural language processing (NLP) package that uses predefined cues to determine whether a finding is negated, asserted, or uncertain. We measured how radiologists understanding of these cues differed when presented without context and with sentence-level context.

We created a set of 241 linguistic cues from pyConTextNLP and from translations from a Swedish corpus. Sentences containing the cues were identified in a separate corpus of 4727 de-identified CTPA reports. Focusing on the Impression section, we randomly selected up to five sentences containing the cue, resulting in 321 sentences containing cues modifying findings. Three radiologist assigned the probability of the finding’s existence based on the sentence. The radiologists had previously provided probabilities (single-point and ranges) for each cue isolated from contextual information. We measured inter-radiologist discordance scores between contextual and non-contextual probability assignments and between single-point probabilities and probability ranges.

There was an insignificant positive shift of 0.024 in probabilities when viewed in context (paired t-test, p=0.35). Cues that showed high disagreement among radiologists when viewed without context also showed high intra-radiologist inconsistency when viewed in context (Pearson’s R=0.36,p=0.0006). Assertion cues changed more than negation cues when seen in context (Pearson 0.26, p=0.016).

Chapman, B, Chen, J, Miyakoshi, A, Chapman, W, Gentili, A, Measuring How Perceived Meanings of Uncertainty Cues Differs with and without Sentence-level Context in Radiology Reports.  Radiological Society of North America 2013 Scientific Assembly and Annual Meeting, December 1 - December 6, 2013 ,Chicago IL. http://archive.rsna.org/2013/13026346.html