Feng Li MD, PhD, Presenter: License agreement, Hologic, Inc (R2 Technology, Inc) License agreement, Deus Technologies, LLC License agreement, Riverain Medical License agreement, Mitsubishi Corporation License agreement, MEDIAN Technologies License agreement, General Electric Company License agreement, Toshiba Corporation

Roger Engelmann MS, Abstract Co-Author: Nothing to Disclose

Kunio Doi PhD, Abstract Co-Author: Shareholder, Hologic, Inc (R2 Technology, Inc) License Agreement, Hologic, Inc (R2 Technology, Inc) License Agreement, Deus Technologies, LLC License Agreement, Riverain Medical License Agreement, Mitsubishi Corporation License Agreement, MEDIAN Technologies License Agreement, General Electric Company License Agreement, Toshiba Corporation Research Support, Deus Technologies, LLC Research Support, DuPont Research Support, Elbit Medical Imaging Ltd Research Support, Fuji Photo Film Co, Ltd Research Support, General Electric Company Research Support, Hitachi, Ltd Research Support, Eastman Kodak Company Research Support, Konica Minolta Group Research Support, Mitaya Manufacturing Co, Ltd Research Support, Mitsubishi Corporation Research Support, Koninklijke Philips Electronics NV Research Support, Hologic, Inc (R2 Technology, Inc) Research Support, Riverain Medical Research Support, Seiko Corporation Research Support, Siemens AG Research Support, 3M Company Research Support, Toshiba Corporation

Heber M. MacMahon MD, Abstract Co-Author: Consultant, Riverain Medical Research support, MEDIAN Technologies Stockholder, Hologic, Inc (R2 Technology, Inc)

To evaluate the number of actual detections versus “accidental” markings by a CAD system for lung cancers on chest radiographs.

A Nodule Detection CAD program (Riverain 1.1.0) was applied to 34 PA digital chest radiographs (all showed a nodular lung cancer that was visible in retrospect, but had not been mentioned in the report). These “radiologist-missed” cancers (mean size, 14.8 mm; range, 7-23 mm) were identified by two radiologists and confirmed by surgery or biopsy. We used a criterion by which the center of the circle, used by the CAD to indicate a possible nodule, was required to be within the area of the lesion boundary for it to be counted as a true detection. However, the marks applied by the CAD system consist of 5 cm diameter circles. A marking was considered as “accidental” or “by chance” if the center point of the circle was not within the lesion boundary, although the lesion was located completely or partially within the 5 cm circle. The CAD sensitivities and the false positive marks were analyzed.

For the 34 chest radiographs, the total CAD marks were 132, with a sensitivity of 26% (9/34) and an average of 3.6 false positive marks per radiograph. If the criterion was expanded to include lesions located completely within the circles, the sensitivity for cancers marked would increase to 41% (14/34) with 3.4 false positive marks per image. If the criterion was again expanded to include lesions located at least partially within the circles, the sensitivity would increase to 53% (18/34) with 3.3 false positive marks per image.

The reported sensitivity of a CAD system can vary by as much as 100% and the specificity a smaller but substantial amount, depending on the precise definition of true versus false marks.

In evaluating CAD systems, it is important to understand how their apparent accuracy can be influenced by the definition of true (false) marks that is used in determining sensitivity (specificity).

Li, F, Engelmann, R, Doi, K, MacMahon, H, True Detection versus “Accidental” Marking of Cancers by CAD on Chest Radiographs.  Radiological Society of North America 2007 Scientific Assembly and Annual Meeting, November 25 - November 30, 2007 ,Chicago IL. http://archive.rsna.org/2007/5007125.html