Standardization in Quantitative Imaging: A Multi-Center Comparison of Radiomics Feature Values Obtained by Different Software Packages on Digital Reference Objects and Patient Datasets

Tuesday, Dec. 3 3:30PM - 3:40PM Room: N229

Michael F. McNitt-Gray, PhD, Los Angeles, CA (Presenter) Institutional research agreement, Siemens AG
Sandy Napel, PhD, Stanford, CA (Abstract Co-Author) Medical Advisory Board, Fovia, Inc; Scientific Advisor, EchoPixel, Inc; Scientific Advisor, RADLogics, Inc
Jayashree Kalpathy-Cramer, MS, PhD, Portland, OR (Abstract Co-Author) Research support, General Electric Company; Research support, F. Hoffmann-La Roche Ltd;
Akshay Jaggi, Stanford, CA (Abstract Co-Author) Nothing to Disclose
Nastaran Emaminejad, MSc, Los Angeles, CA (Abstract Co-Author) Nothing to Disclose
Mark Muzi, BS, Seattle, WA (Abstract Co-Author) Nothing to Disclose
Dmitry Goldgof, PhD, Tampa, FL (Abstract Co-Author) Nothing to Disclose
Hao Yang, New York, NY (Abstract Co-Author) Nothing to Disclose
Ella F. Jones, PhD, San Francisco, CA (Abstract Co-Author) Nothing to Disclose
Muhammad W. Wahi-Anwar, BS, Los Angeles, CA (Abstract Co-Author) Nothing to Disclose
Yoganand Balagurunathan, PhD, Tampa, FL (Abstract Co-Author) Nothing to Disclose
Mahmoud Abdalah, PhD, Tampa, FL (Abstract Co-Author) Nothing to Disclose
Binsheng Zhao, DSc, New York, NY (Abstract Co-Author) Royalties, Varian Medical Systems, Inc; License agreement, Keosys SAS; License agreement, Hinacom Software and Technology, Ltd;
Lubomir M. Hadjiiski, PhD, Ann Arbor, MI (Abstract Co-Author) Nothing to Disclose
Apurva Virkud, Ann Arbor, MI (Abstract Co-Author) Nothing to Disclose
Heang-Ping Chan, PhD, Ann Arbor, MI (Abstract Co-Author) Research collaboration, General Electric Company; Institutional Grant, General Electric Company ;
Larry A. Pierce II, PhD, Seattle, WA (Abstract Co-Author) Nothing to Disclose
Keyvan Farahani, PhD, Bethesda, MD (Abstract Co-Author) Nothing to Disclose


Radiomics features are being increasingly proposed for clinical applications such as predicting patient response to therapy or prognosis. The purpose of this work was to investigate the agreement among these features when computed by several groups utilizing different software packages with standardized feature definitions and common image datasets designed to identify possible differences.


Nine sites from the NCI's Quantitative Imaging Network PET-CT working group participated in this project. Nine common quantitative imaging features were selected for comparison including features that describe morphology, intensity, shape and texture. A standard lexicon developed by the International Biomarker Standardisation Initiative (IBSI) was adopted as the feature definition reference. The common image data sets were: (a) two sets of 3D Digital Reference Objects (DROs) developed specifically for this effort (200mm and 50 mm diameter objects): a uniform sphere, a sphere with intensity variations, and a complex shape object with uniform intensity; and (b) 10 patient image scans from the LIDC dataset using a specific lesion in each scan. To eliminate variation in feature values caused by segmentation differences, each object (DRO or lesion) was accompanied by a Volume of Interest (VOI), from which the features were calculated. Feature values for each object (DRO or lesion) were reported. The percent coefficient of variation (CV) was calculated across software packages for each feature on each object.


10 sets of results were obtained for the DROs. Six of the nine features demonstrated excellent agreement with CV < 1%. Larger variations (CV>= 13%) were observed for the remaining three features. Only 2 sets of results from patient datasets were obtained so far, but similar trends were observed with the exception being kurtosis, which showed higher CV than in the DROs.


By computing common radiomics features on a common set of objects using the same VOIs for each object, we have shown that while several features agree strongly across software packages, others do not. This highlights the value of feature definition standardization as well as the need to further clarify definitions for some features.


Remaining disagreement in the community as to radiomic feature definitions and implementation details should be resolved before radiomic analysis becomes part of routine practice.

Printed on: 10/20/20