Radiomics features are being increasingly proposed for clinical applications such as predicting patient response to therapy or prognosis. The purpose of this work was to investigate the agreement among these features when computed by several groups utilizing different software packages with standardized feature definitions and common image datasets designed to identify possible differences.

Nine sites from the NCI's Quantitative Imaging Network PET-CT working group participated in this project. Nine common quantitative imaging features were selected for comparison including features that describe morphology, intensity, shape and texture. A standard lexicon developed by the International Biomarker Standardisation Initiative (IBSI) was adopted as the feature definition reference. The common image data sets were: (a) two sets of 3D Digital Reference Objects (DROs) developed specifically for this effort (200mm and 50 mm diameter objects): a uniform sphere, a sphere with intensity variations, and a complex shape object with uniform intensity; and (b) 10 patient image scans from the LIDC dataset using a specific lesion in each scan. To eliminate variation in feature values caused by segmentation differences, each object (DRO or lesion) was accompanied by a Volume of Interest (VOI), from which the features were calculated. Feature values for each object (DRO or lesion) were reported. The percent coefficient of variation (CV) was calculated across software packages for each feature on each object.

10 sets of results were obtained for the DROs. Six of the nine features demonstrated excellent agreement with CV < 1%. Larger variations (CV>= 13%) were observed for the remaining three features. Only 2 sets of results from patient datasets were obtained so far, but similar trends were observed with the exception being kurtosis, which showed higher CV than in the DROs.

By computing common radiomics features on a common set of objects using the same VOIs for each object, we have shown that while several features agree strongly across software packages, others do not. This highlights the value of feature definition standardization as well as the need to further clarify definitions for some features.

Remaining disagreement in the community as to radiomic feature definitions and implementation details should be resolved before radiomic analysis becomes part of routine practice.