Jessica Faruque MS, Presenter: Nothing to Disclose

Daniel L. Rubin MD, Abstract Co-Author: Grant, General Electric Company

Christopher Frederick Beaulieu MD, PhD, Abstract Co-Author: Nothing to Disclose

Aya Kamaya MD, Abstract Co-Author: Nothing to Disclose

Grace Anlon Tye MD, Abstract Co-Author: Nothing to Disclose

Sandy Napel PhD, Abstract Co-Author: Medical Advisory Board, Fovia, Inc Consultant, Carestream Health, Inc

Ronald M. Summers MD, PhD, Abstract Co-Author: Royalties, iCAD, Inc Grant, iCAD, Inc Stockholder, Johnson & Johnson Software grant, Viatronix, Inc

Developing an independent reference standard of similarity is a challenge for content-based image retrieval (CBIR). For a CBIR project aimed at retrieving similar liver lesions imaged with CT, we are developing an efficient method for creating a visual similarity standard by predicting all pair-wise similarity scores from a small subset of them.

Three radiologists reviewed all 171 pair-wise combinations of 19 portal venous CT images containing liver lesions in random order, and rated them for visual similarity. We first determined inter-reader agreement based on the Kappa statistic. Next, we determined if the overall similarity matrix, averaged over the readers, could be predicted from a subset of the pair-wise ratings. We removed a random subset of the ratings, and predicted the remaining ratings by computing the set of entries completing the matrix that minimized the maximum sum of the singular values of the matrix. We ran 180 iterations of this experiment using a variety of random subsets containing between 8% and 99% of the matrix, computed the root mean square (RMS) error between the calculated and actual entries for each iteration, binned the results into bins of width 10% of the actual entries included in the matrix, and computed the means and standard deviations of RMS error in each bin.

Kappa scores between raters 1 and 2, 1 and 3, and 2 and 3 were 0.72, 0.42, and 0.43 respectively. The mean RMS error ranged from 5.6±0.6 to 1.9±0.2 points on a 9 point scale for matrix completion with a range of 8% to 99% of the entries included, respectively. The mean RMS error showed good agreement to within 3 points on a 9 point scale when using as little as 30% of the entries in the matrix.

These pilot results suggest that we can predict pair-wise similarity ratings to reasonable accuracy from a fairly small set of human observations, representing approximately 30% of the total set. This may increase the feasibility of creating similarity reference standards for large databases of radiological images.

This method may make generation of a similarity reference standard feasible for large databases, which in turn may allow training and development of CBIR systems.

Faruque, J, Rubin, D, Beaulieu, C, Kamaya, A, Tye, G, Napel, S, Summers, R, A Scalable Reference Standard for Perceptual Similarity of CT Liver Lesions Using Matrix Completion Techniques.  Radiological Society of North America 2011 Scientific Assembly and Annual Meeting, November 26 - December 2, 2011 ,Chicago IL. http://archive.rsna.org/2011/11013067.html