Akshay Kohli MBBS, Presenter: Nothing to Disclose

John William Robinson BSC, Abstract Co-Author: Nothing to Disclose

John Ryan PhD, Abstract Co-Author: Nothing to Disclose

Mike Evanoff PhD, Abstract Co-Author: Nothing to Disclose

Mark McEntee, Abstract Co-Author: Nothing to Disclose

Patrick C. Brennan PhD, Abstract Co-Author: Nothing to Disclose

Tamara Miner Haygood MD, PhD, Abstract Co-Author: Nothing to Disclose

Variability amongst radiologists may influence diagnostic performance. This study explores the relationship between reader characteristics and lesion detection ability using receiver operating characteristic (ROC) and JAFROC methodologies.

A test set of 40 postero-anterior chest radiographs was developed, of which 20 were normal whilst 20 contained up to 3 simulated nodules, of varying subtlety. Images were presented to 12 senior US radiologists of whom six were chest specialists. For the ROC analyses, observers were asked to rate their degree of suspicion for the presence of nodules by using a confidence rating scale (1-6). JAFROC analyses required the observers to locate and rate as many suspicious areas as they wished using the same scale. Unlimited time was provided to each reader and resultant data were used to generate Az and FOM scores for ROC and JAFROC analyses respectively. Using Pearson methods, scores of performance were correlated with 12 reader characteristics recorded using a radIQ questionnaire. A p- value of less than 0.05 was considered to be significant for all tests and the Pearson correlation coefficient (r) demonstrated the strength of the relationship.

JAFROC analyses showed that improved reader performance was significantly linked with chest specialty (p<0.03, r=0.56), hours per week reading chest radiographs (p<0.03, r=0.55) and chest readings per year (p<0.04, r=0.54). The relationship between CME credits in chest radiology and performance almost reached a significant level (p<0.072, r=0.45). ROC analyses demonstrated only one significant relationship, hours per week reading chest radiographs (p<0.02, r=0.67).

JAFROC analysis revealed correlations between reader characteristics and diagnostic performance for nodule detection that remained obscure with ROC analysis. By JAFROC methods, reader characteristics such as chest specialty, chest readings per year and hours per week reading chest radiographs are significantly related to improved reader performance.

In the US, 200,000 cases of lung cancer are diagnosed annually, yet 20-30% are missed by radiologists. Here we show how JAFROC analyses identify reader characteristics linked with nodule detection.

Kohli, A, Robinson, J, Ryan, J, Evanoff, M, McEntee, M, Brennan, P, Haygood, T, Jackknife Free-Response Receiver Operating Characteristic (JAFROC) Figures of Merit (FOM) Help Identify Key Radiologist Characteristics Linked to Diagnostic Efficacy in Chest Radiology.  Radiological Society of North America 2010 Scientific Assembly and Annual Meeting, November 28 - December 3, 2010 ,Chicago IL. http://archive.rsna.org/2010/9009915.html