Gregory V. Goldmacher MD, PhD, Presenter: Employee, ICON plc

Ninad Mantri MS, Abstract Co-Author: Employment, ICON plc

James J. Conklin MD, Abstract Co-Author: Employment, ICON plc

David Raunig PhD, Abstract Co-Author: Director, Pfizer Inc Stockholder, Pfizer Inc Employee, ICON plc

Radiological endpoints in oncology trials are often assessed with multiple readers on each case. Reader variability is of interest to regulators and trial designers, who sometimes regard a disagreement rate (DR) over 40% as indicating unreliable data, regardless of indication or cause. Disagreement on progression and response has not been systematically studied across cancer types.

We examined disagreement between independent readers in pooled data from Phase 2 or 3 trials in non-small cell lung cancer (NSCLC), breast cancer, colorectal cancer (CRC), and non-Hodgkin’s Lymphoma (NHL). Each case was assessed by two readers, using RECIST 1.0 or 1.1 in solid tumors and Cheson 1999 in NHL. We calculated DR on whether the subject progressed, date of progression (DOP), best overall response (BOR), duration of response (DOR), or any visit response. DOR was calculated, for cases where readers agreed on a BOR of partial response (PR) or complete response (CR), as the duration from first PR or CR until progression. Where there was disagreement on DOP, we assessed whether it was based on target lesions (TL), non-target lesions (NTL), or new lesions (NL) for solid tumors.

The data included patients from 4 NSCLC trials, 3 breast cancer trials, 3 CRC trials, and 4 NHL trials. The table shows DR on various parameters and combinations. When readers disagreed on DOP in breast cancer (n=377), DR on NL, TL, and NTL, respectively, was 51.5%, 48.8%, and 66.6%. For NSCLC (n=961), it was 45.8%, 76.7%, and 51.3%. For CRC (n=435), it was 43.0%, 8.0%, and 59.1%. For NHL, DR on NL was 24.2%, with TL and NTL data not captured. Causes of disagreement were non-exclusive.

Up to 33% of DOP disagreements are by one visit. A DR above 40% is to be expected for some tumors, especially if multiple parameters are compared. Some have advocated new lesions as the sole reliable progression indicator, but there is considerable variability in new lesion perception. Some cancers show less variability that others on perception of TL progression, possibly due to fewer lesions (less variability in lesion selection), lesions being easier to measure (less variability in measurement), or both. This study establishes baseline rates of disagreement, so that interventions to reduce variability can be assessed.

Radiological tumor assessment in clinical trials is subject to variability which varies across indications.

Goldmacher, G, Mantri, N, Conklin, J, Raunig, D, Rates and Causes of Radiologist Disagreement during Independent Review of Oncology Clinical Trials.  Radiological Society of North America 2014 Scientific Assembly and Annual Meeting, - ,Chicago IL. http://archive.rsna.org/2014/14007940.html