Heiko.Peschl@ouh.nhs.uk

To assess the follow-up rule-out accuracy of a convolutional neural network (CNN) in patients with incidentally detected, indeterminate pulmonary nodules in a multi-site, heterogeneous population.

The US National Lung Screening Trial (NLST) dataset was manually curated and used to create a training set: each reported nodule and cancer was located, contoured and diagnostically characterised (9310 benign nodule patients; 1058 cancer patients). All patients with solid and semi-solid nodules of 6mm and above, where benign nodules and cancers could be confidently identified by clinicians (5972 patients, of which 575 were cancer patients), were selected. A CNN was trained using Deep Learning and three thresholds for benign rule-out were calculated at three levels of sensitivity: 100%, 99.5% and 99%. An independent dataset of patients with incidentally detected indeterminate pulmonary nodules was retrospectively collected from a tertiary referral centre and surrounding hospitals in the UK with a heterogeneous mix of scan parameters, manufacturers and clinical indications (610 patients, 698 nodules, 5-15mm). Diagnosis was established according to British Thoracic Society guidelines (2015). The dataset contained 50 cancers from 47 patients (7% of all nodules). Performance was evaluated by measuring the specificity at the three benign rule-out thresholds; i.e. to measure the proportion of benign nodules correctly stratified while missing no or few cancers. Overall Area-Under-the-ROC-Curve analysis (AUC) was also calculated.

The specificity (sensitivity) was 24% (100%), 24% (100%) and 48.6% (100%) at the three thresholds respectively. AUC was 0.93 (95%CI = 0.90-0.96).

On this independent dataset, the CNN was able to correctly classify just under half of the benign nodules whilst not misclassifying any cancers.

Our work shows the potential of CNNs in ruling out benign pulmonary nodules and therefore reducing the need for follow up scans in a large number of patients.