RSNA 2003 

Abstract Archives of the RSNA, 2003


K16-998

Assessing Diagnostic Confidence: Example from a Randomized Controlled Trial of Abdomino-pelvic CT for Acute Abdominal Pain of Uncertain Etiology: Being Confidently Wrong Can Be Dangerous for Health

Scientific Papers

Presented on December 3, 2003
Presented as part of K16: Health Services, Policy and Research (Issues in Research Methodology)

Participants

Chaan Ng MD, PRESENTER: Nothing to Disclose

Abstract: HTML Purpose: Diagnostic confidence has been, and is, used as a measure of diagnostic efficacy, particularly in an evidence-based context, but this measure in isolation fails to account for confident, but wrong, diagnoses. We describe a more robust methodology, that recognizes the interaction of diagnostic confidence and diagnostic accuracy, and compare it to a more conventional evaluation, using an illustrative example from a randomized controlled CT study of acute abdominal pain. Methods and Materials: In a prospective trial, 120 patients admitted to hospital with acute abdominal pain of uncertain etiology were randomized to receive either "early CT" (within 24 hours) or "standard practice". Admitting surgeons rated their diagnostic confidence on admission, and again at 24 hours (1 (=low) to 5 (=high) confidence). The benefits, or otherwise, of the 0-24 hour intervention on diagnostic confidence was scored blindly on a 9-point scale (-4 to +4), taking into account diagnostic confidence, any changes in diagnosis, and its ultimate accuracy. Scoring included both a "best estimate" and "range", the latter recognizing the possibility for pessimistic and optimistic circumstances, and a weighted average computed. The analysis was compared to a more conventional pre-/post-test analysis of confidence (which takes no account of the potential impact of incorrect diagnoses), on both an "intention-to-treat" and "as-treated" basis. Results: Using the conventional (simple pre-/post-test) approach, diagnostic confidence was significantly higher in the "early CT", compared to the "standard practice", group on both an intention-to-treat (95% confidence interval of difference (0.013, 0.932) p=0.044), and as-treated basis ((0.31, 1.20) p=0.001). Using our methodology, the corresponding results for both analyses pointed towards more conservative estimates of efficacy: (-0.53, 0.68) p=0.82, and (0.014, 1.20) p=0.045, respectively. Conclusion: Early CT improves diagnostic confidence in patients admitted with acute abdominal pain, but its efficacy is over-estimated if the interaction of diagnostic confidence and diagnostic accuracy is not taken into account. We suggest that appropriate assessment of diagnostic confidence as a measure of diagnostic efficacy is best undertaken with appropriate incorporation of diagnostic accuracy; there is otherwise the risk of arriving at misleading conclusions.       Questions about this event email: cng@mdanderson.org

Cite This Abstract

Ng MD, C, Assessing Diagnostic Confidence: Example from a Randomized Controlled Trial of Abdomino-pelvic CT for Acute Abdominal Pain of Uncertain Etiology: Being Confidently Wrong Can Be Dangerous for Health.  Radiological Society of North America 2003 Scientific Assembly and Annual Meeting, November 30 - December 5, 2003 ,Chicago IL. http://archive.rsna.org/2003/3102966.html