dyuing89@gmail.com

To evaluate the performance of a deep learning-based automatic detection (DLAD) algorithm in detecting malignant pulmonary nodules on chest radiographs (CPAs) and its comparison with human experts.

A DLAD algorithm was developed using a 25-layer deep convolutional network in a novel semi-supervised manner with 41,792 cases. For this observer performance test, 181 CPAs not used in the development of DLAD were included: 119 CPAs with 147 pathologically- or clinically-confirmed malignant lung nodules (mean size, 2.50 cm±1.60) and 62 normal CPAs. Reference for the nodules was established via CT taken within a week. Nineteen readers including 3 non-radiology physicians, 6 radiology residents, 5 board-certified radiologists, and 5 thoracic radiologists independently reviewed each CPA to detect lung nodules on a five-point confidence scale without DLAD (test 1). After test 1, each reader was allowed to change their decision by reviewing the results of test 1 and that of DLAD (test 2). The detection performances of DLAD, human experts (test 1), and human experts using DLAD (test 2) were evaluated and compared using jackknife free-response receiver operating characteristic (JAFROC) figure of merits (FOMs) on a per-nodule basis.

DLAD alone exhibited a FOM of 0.857, which was significantly higher than that of 16 of 19 readers (all Ps <0.05). The mean FOM of the 19 readers using DLAD were significantly higher than that without DLAD (0.825 vs. 0.713, P=0.002). All readers showed improved detection performances for malignant pulmonary nodules using DLAD (mean FOM increase of 0.044 [range, 0.007-0.193]) with significant differences in 15 readers (P<0.05). On subgroup analysis, FOMs of the four reader groups (non-radiology physicians, radiology residents, board-certified radiologists, and thoracic radiologists) were 0.678, 0.784, 0.808, and 0.820, respectively, and their detection performances significantly improved with DLAD (0.814, 0.817, 0.827, 0.841; all Ps <0.005).

DLAD showed better performance than most experts in detecting malignant pulmonary nodules on CPAs and enhanced the performance of human experts when used in conjunction.

The excellent performance of DLAD demonstrates its great potential in changing our daily clinical practice; DLAD may provide preliminary interpretation in pulmonary nodule detection on chest radiographs.

sarim.ather@ouh.nhs.uk

Lung nodule volumetry is currently the preferred method of following up small indeterminate lung nodules. This study assesses how CT texture analysis (CTTA) performs compared to volumetry in differentiating benign and malignant nodules.

Data was collected for 143 patients with lung nodules (4-15mm) on an initial CT scan, with a follow-up CT 29 to 2747 days later. All scans had a slice thickness below 1.5mm. Nodules were delineated in 3D using a thresholded region with manual edits and the contour was propagated to a latter scan using deformable image registration. Definitive nodule diagnoses were established through histology, 2-year stability or malignant growth. Initially, the set was biased with respect to size (large nodules more likely to be cancer), so a subset of 113 patients (58 benign, 55 malignant) was selected such that size at presentation was uninformative (AUC 0.50-0.51). From each CT, a suite of classical texture features (Harlick, Gabor etc.) at various sizes/scales was extracted from isotropically-resampled volumes containing the contoured nodule. A derived feature vector was then created based on texture-feature changes between the first and second timepoints for each nodule, and volume doubling time (VDT) was calculated.

Area under the ROC curve (AUC) analysis showed that the VDT alone as an indicator of malignancy only performed at AUC=0.70 (95% CI=0.69-0.71). In contrast, the best difference-of-texture feature achieved AUC=0.81 (95% CI=0.80-0.82), with changes in size-linked texture features appearing more reliable than either absolute size at the second timepoint (AUC=0.75, 95% CI=0.74-0.76), or VTD. The mean of the top ten texture AUCs was above 0.76. This shows that quantitative texture measures extracted more information from the CT than size or VDT, and that this information is beneficial in malignancy prediction.

Our results indicate that changes in quantified texture around lung nodules and their surrounding microenvironment may be a more accurate tool for stratifying malignant lung nodules than VDT alone.

Quantitative texture measures may be a useful adjunct in the follow-up of indeterminate lung nodules.

sgpark@lunit.io

We developed a deep learning-based automatic detection (DLAD) algorithm for the detection of malignant pulmonary nodules on chest radiographs, and evaluated its diagnostic performance and nodule detection performance using large-scale chest radiograph (CR) data.

A total of 44,087 CRs comprised of 9,269 abnormal and 34,818 normal CRs were collected. Abnormal CRs were pathologically or clinically diagnosed as having malignant pulmonary nodules in 3,892 individuals (M:F=2,244:1,648; mean age, 63.6 years). Normal CRs were from 30,784 individuals (M:F=16,886:13,798; mean age, 51.3 years). We randomly split the datasets into a training set (33,467 normal and 8,625 abnormal CRs), a validation set (675 normal and 322 abnormal CRs) and a test set (675 normal and 322 abnormal CRs). There were no overlaps among these three datasets. We designed a deep convolutional network with 25 layers and 8 residual connections, and trained this network using a novel semi-supervised learning approach that partially utilized the location of the lesions, i.e., 6 thoracic radiologists manually marked the location of the abnormal lesions (3,213 CRs) in the abnormal CRs of the training set. The same radiologists also tagged the locations of all abnormal CRs in the validation and test sets to evaluate the localization performance of the DLAD algorithm. We then quantitatively verified the performances of DLAD by analyzing receiver-operating characteristics (ROC) curves for classification performance and localization average precision (AP) for detection performance.

In the validation dataset, DLAD showed an area under the ROC curve (AUC) of 0.9793, with an accuracy, sensitivity, and specificity of 93.83%, 85.54% and 98%, respectively. DLAD achieved 0.9257 for localization AP. In the test dataset, the AUC of DLAD was 0.9777 (accuracy, 93.53%; sensitivity, 86.2%; specificity, 96.15%). Localization AP was 0.862.

The DLAD algorithm demonstrated high diagnostic performance in differentiating malignant nodules from normal CR findings. DLAD also showed promising results in automatically detecting the location of nodules.

As a second reader, DLAD has great potential to improve radiologists' detection performance and efficacy of malignant pulmonary nodules on chest radiographs.

giovanni.montana@kcl.ac.uk

The chest x-ray is the most commonly performed radiology study, yet interpretation can be challenging. Deep learning systems for computer-aided detection (CAD) hold great potential but have been limited by the need for large numbers of radiologist-annotated radiographs. Other authors have described some success with report classification or manual annotation based CAD systems.We aimed to develop a novel nodule-detection CAD system, combining manually annotated images with 750,000 reported radiographs.

We investigated image- and nodule-based metrics to compare our CAD system to alternatives. An ethics committee waiver of consent was granted.With natural language processing we analyzed free-text reports to generate labels, indicating which of our 750,000 radiographs contain a nodule or other abnormality. For 1,237 images, we provided radiologist-annotated boxes around 1,861 nodules. We proposed a CAD system using convolutional neural networks trained using hybrid loss on class labels and nodule annotations. The system can predict if there is a nodule and its location.For image-based classification, a test set of 7,850 images, 1,864 containing nodules, was used to assess the CAD system in predicting presence of a nodule. For nodule-based classification a test set of 575 previously unseen radiographs containing 787 nodules was used to investigate co-localization whilst penalizing false positives. For both, we applied other leading methodologies to our test dataset for comparison.

Image-based classification yielded an Accuracy of 0.76 and F1 of 0.67 (cf. 0.64 & 0.55 for annotation-based, and 0.72 & 0.62 for report-based CAD systems, respectively).Nodule-based metrics showed Recall 0.65, Precision 0.15, with 43% overlap between manual and CAD bounding boxes (cf. 0.37, 0.28, 30% for annotation-based, and 0.34, 0.10, 17% for report-based CAD systems).

Our study demonstrates improved nodule detection and localization from combining manual annotations with 'big data' from 750,000 films.With further work, CAD may act as a 'second reader' for chest radiographs, increasing sensitivity in nodule detection without compromising specificity.

Computer-aided detection systems learning from 'big data' can help detect nodules on chest radiographs with potential to act as a cost-effective 'second reader' across healthcare systems worldwide.

marthony.robins@duke.edu

To conduct a public Challenge through Quantitative Imaging Biomarker Alliance (QIBA) to assess the conformance of segmentation algorithms for CT volumetry and to establish interchangeability between real and simulated lesions.

Simulated lung lesion models (based on pathologically confirmed malignant tumors) were virtually inserted into (a) 3 phantom datasets using validated projection and image-domain insertion programs, and (b) 30 clinical chest CT cases containing real lesions "hybrid datasets". The study was designed as a public Challenge to academic researchers and commercial software developers to apply their volume estimation algorithms on simulated and corresponding real lung lesions. Equivalence between real and simulated lesions was analyzed in terms of bias and repeatability coefficient (RC) (phantom only), and algorithm reproducibility (variance between measurements by different algorithms on the same lesion). Comparisons were made relative to insertion method, participant, and lesions (shape, size, location).

8 and 16 groups (industry/academic) participated in the phantom and hybrid sections of the Challenge, respectively. Either fully or semi-automated segmentation algorithms were used. Percent bias and repeatability coefficient based on the defined QIBA compliance criteria showed that 3 of 8 groups were fully compliant with the profile, and one close non-compliance. For compliant groups, %bias (95% confidence intervals) was -0.43% (±5.6%) for real and -0.34 to -2.2% (±5.8%) for simulated lesions; while RC was 13% for real and 2 to 13% for simulated lesions.

Hybrid datasets can overcome phantom lack of realism and patient lack of ground truth limitations. Our results indicate that simulated and real lesions are generally comparable but exhibit differences for certain algorithms. This shows simulated lesions inserted into clinical cases can be used in the assessment of quantitative volumetry methods.

Hybrid datasets help us better assess the performance of lesion segmentation algorithms for QIBA Profile conformance. This enables translation of precision medicine in the context of quantitative CT.

edith.marom@gmail.com

For decades, the CT diagnosis of pulmonary hamartoma (PH) has been based on the presence of popcorn calcifications or fat (-40 to -120 HU) limited to a smooth <2.5cm pulmonary nodule. These rigid criteria were selected to avoid classifying necrotic neoplasms as benign. A minority of the PH fulfill these criteria which results in unnecessary surgical resections. The aim of our study was to assess the spectrum of HU distribution in PH and whether the adoption of a wider threshold for fat content can safely be implemented for non invasively diagnosing PH without misinterpreting malignant lesions as benign.

We retrospectively assessed the CT scans of consecutive histologically confirmed: PH (histPH), pulmonary metastases and primary lung cancers as well as PH diagnosed by CT (CTPH). Their size, volume, average, minimum and maximal HU were assessed using an ROI of at least 8 pixels, placed in the lowest attenuation region of the nodule.

There were 52 histPH, 41 metastases, 49 primary lung cancers, and 34 CTPH. PH average size and volume were 14.5mm and 2270.2mm 3 and that of malignancies 26.5mm and 22,031mm 3. Popcorn calcifications were seen in 2 (4%) histPH, 11 (32%) CTPH and non of the malignant lesions. The average HU for histPH, CTPH, metastases, and primary lung cancers were: 3.99, -10.97, 25.53, 38.61 respectively. Of the malignant lesions, only 4 had an average of <0 HU and comprised of metastses of sarcoma (n=1), colon (n=1), mature cystic teratoma (n=1) and ovary (n=1). Minimum pixel HU value of <0 were seen in 28 (31.1%) of malignant nodules with the lowest minimal pixel value of -42 as compared to 44 (84.6%) of the histPH and 33 (97%) of the CTPH with a minimal pixel value of -168 HU. By raising the average HU threshold value for diagnosing PH from -40 to -20, the sensitivity for diagnosing PH rises from 9.3% to 18.6% while the positive predictive value and the specificity remain 100%.

By increasing the threshold for identification of fat in PH to -20 unnecessary surgical intervention may be prevented without misdiagnosing cancer as benign.

By increasing the threshold for fat identification, a greater number of PH will be safely identified and unnecessary surgery avoided.

yatsushi@belle.shiga-med.ac.jp

To determine the beneficial effect of computer-aided detection (CAD) for pulmonary solid and sub-solid nodules in ultra-low-dose CT with AIDR 3D (ULDCT) and low dose CT with AIDR 3D (LDCT).

This was part of the ACTIve Study, a multi-center research project in Japan. The Institutional Review Board of each institution approved this study, and written informed consent was obtained. In a single visit, 68 subjects underwent chest CT (64-row helical mode) using identical 320-row scanners with different tube currents: 240,120 and 20 mA (2.51, 1.26 and 0.21mSv, respectively). Standard of reference (SOR) was established based on consensus reading of CT images at 240mA by two radiologists. CAD was performed in ULDCT (20mA) and LDCT (120mA) with lung reconstruction kernels. Another 2 observers independently assessed lung nodule presence on both methods. In total and 4 subgroups classified according to the combination of nodular size (<5, >5mm) and characters (solid, sub-solid), nodule detection rate (NDR) for both CAD and 2 observers with CAD was compared between both methods with Fisher's exact probability test, and NDR without CAD and that with were compared with McNemar test for each observer. Scan-based false positive ratio by CAD (FPR) was compared with paired t-test between both methods.

For SOR, 123 solid and 100 sub-solid nodules were identified (mean diameter 6.0 ±4.1 mm, range 2-29 mm). NDR by CAD on ULDCT (28%) was similar to that on LDCT (30%) in total as well as the 4 subgroups (p>0.05). CAD led to a significant increase of NDR for 2 observers on both modalities for smaller solid nodules (<5mm) (p<0.001) and observer 1 on LDCT for larger solid nodules (>5mm) (p=0.031). For larger nodules, NDR for 2 observers with CAD on ULDCT (92% for solid, 77% for sub-solid) was comparable to that on LDCT (95% for solid, 86% for sub-solid) (p>0.05). FPR on LDCT (2.5±4.1) was similar to that on ULDCT (2.4±2.2).

ULDCT showed comparable NDR for larger nodules to LDCT regardless of CAD application. In contrast, adding CAD in ULDCT improved NDR by observers for smaller solid nodules, although similarity in NDR to LDCT was not achieved.

ULDCT could be useful for larger lung nodule detection irrespective of nodular characters and provide improved detection sensitivity of radiologists by CAD for smaller lung solid nodules

dianapenha@gmail.com

The purpose of the study was to determine the intra-scan variability of volumetric measurements of solid pulmonary nodules.

In this retrospective study, 827 consecutive patients that underwent cardiac multi-phase CT scan were evaluated. All the CT exams were performed on a 256-row CT scanner (SIEMENS Somaton Definition Flash) using 0.6mm slice thickness and soft kernel. The image reconstructions were done using 10 phases in 10% of each RR interval. The images were evaluated in the axial plane to identify the lung nodule and after the semi-automatic tool for lung lesions / volumetry was applied. The volume of the nodule was determined two times according with two different phases of the scan for each patient.

For statistical analysis 66 pulmonary nodules with medium volume of 8mm were included. The mean nodule volumetric difference was 513mm3 or 21.6%. Confidence interval of difference observed on measurements taken on different cardiac phases: Percentile 5. 15mm3 or 0.00%; Percentile 50. 62mm3 or 14%, and Percentile 95. 1696mm2 or 58%. The volume measurements showed significant variability for any one given nodule during a single multi-phase scan (p<0.05). There was no correlation between the volume measurement of the nodule and the difference between the volume measurements.

There is significant cardiac phase variability in lung nodule volume measurement.

Lung nodule measurements are important in identifying potential early malignant transformation and relies on identifying a significant increase in size (25%) and change in character between serial scans.In recent years many limiting factors to the measurements' accuracy have been identified, such as nodule size, location, morphology, inspiratory effort and others such as the technical parameters of the scan.Our purpose is to determine the importance of the hemodynamic factors in the lung nodule volumetry, measuring the variation and reliability of semiautomated lung nodule volumetric measurements during the same CT acquisition and in different cardiac phases.

anastasia.oikonomou@sunnybrook.ca

This study aims to determine what timeline should be utilized to follow pulmonary nodules in melanoma patients to confirm metastatic or benign origin and to delineate features that favor metastatic versus benign etiology.

588 patients had surgery for primary melanoma between 2012-15 in our tertiary care centre. Out of these, 148 patients had baseline chest CT and at least one followup (FU) CT and were included in the study. Patients with definitely benign nodules, metastases and other non-melanoma malignancies were excluded. Nodules were volumetrically measured on FU CTs and a cut-off of 15% difference in volume was considered as significant change. Distance from pleura, peripheral versus central and perifissural location, irregularity, solid versus groundglass density were evaluated. Nodules were considered metastases if they increased in size between two FU CTs or if increase was accompanied by multiple new lung nodules or extrapulmonary metastases.

On baseline CT, 70 patients (A) had at least one indeterminate pulmonary nodule (IPN) and 78 (B) had none. In group A all patients had 243 IPN (0.0236 cm3) and only 1 nodule increased 405% in volume in 5 months and was proven metastatic. Out of 243 IPN, 215 were peripheral, 40 perifissural, 218 solid, 25 groundglass, 4 had irregular margins. 18 nodules increased 50% in volume in the first 3-month CT and either resolved or decreased in another 3 months. During FU, overall, 28 patients developed 33 new nodules (0.0814 cm3) out of whom 24 (86%) were metastases. In 4 patients, nodules resolved or decreased in 5.5 months and were presumed benign. All metastases (33) were solid and 4 were perifissural. Median volume increase between 2 CTs (median 3 months) was 260% (p <0.001). All but one metastases had smooth margins.

IPN on baseline CT are most likely benign and monitoring up to 6 months can confirm it. Newly developed nodules have a high possibility of being metastatic and FU in 3 months can confirm it if needed. Volume increase is significantly lower in benign than metastatic nodules. Perifissural location of new nodules does not exclude metastatic origin, however groundglass and irregular margin most likely does.

IPN on baseline CT require monitoring up to 6 months to confirm benign origin. Newly developed nodules have a high risk of being metastatic and follow up in 3 months will confirm increase in size.