RSNA 2019

Abstract Archives of the RSNA, 2019


A Real-Time Gaze Tracking System to Analyze Spatial and Temporal Attention Characteristics of Radiologists

Wednesday, Dec. 4 3:50PM - 4:00PM Room: E353B

Oguz Akin, MD, New York, NY (Abstract Co-Author) Research Consultant, Ezra AI
Yingli Tian, New York, NY (Abstract Co-Author) Nothing to Disclose
Longlong Jing, New York, NY (Abstract Co-Author) Nothing to Disclose
Benjamin Reichman, New York, NY (Abstract Co-Author) Nothing to Disclose
Fuat Nurili, New York, NY (Abstract Co-Author) Nothing to Disclose
Cihan Duzgol, MD, New York, NY (Abstract Co-Author) Nothing to Disclose
Turan A. Ilica, New York, NY (Abstract Co-Author) Nothing to Disclose
Omer Aras, MD, Edgewater, NJ (Presenter) Nothing to Disclose

For information about this presentation, contact:


To develop a real-time attention tracking system that can handle dynamic back-and-forth scrolling through a series of images without any hardware or software intrusion while the radiologist interpreting a study.


The system framework was built on the Open Health Imaging Foundation's open-source medical image viewer with integration of a Tobii 4C Eye Tracking camera. Spatial and temporal coordinates of the series of images are automatically recorded and the corresponding attention heapmaps are generated. For attention and search patent analysis, the commonly used eye-tracking metrics are analyzed including area of interest (AOI), time to first fixation, total fixation duration, fixation duration on AOI, number of fixations on AOI, dwell time ratio, and saccade length on each image. Furthermore, 3D dynamic analysis of attention characteristics are studied including total time of the scan, slice of interest (SOI), fixation duration on SOI, number of fixations on SOI, number of scrolling (scrolling back-and-forth while focusing on different areas), number of drilling (scrolling back-and-forth while focusing on one area). The system's performance was tested using the data from 4 radiologists while interpreting 30 CT studies from the LUNA16 dataset.


The system successfully captured the attention data in all interpretation sessions (n=120) without any software or hardware failure. The accuracy of the gaze tracking is 0.4 which is about 3.5-7 mm on the computer screen at a distance range of 0.5-1 m between the observer and the camera. The analysis of the attention heatmaps showed spatial and temporal variations in the 3D dynamic attention characteristics of 4 radiologists, indicating unique search patterns among different observers.


The proposed system can be used as an objective tool to study unique search patterns among human observers. These data can be further used to develop more interactive human-computer interfaces for artificial intelligence applications.


As artificial intelligence applications advance, there is an increasing need to develop seamless human-computer interfaces that can capture the radiologists' attention. Such systems allowing artificial intelligence algorithms to operate more interactively with the human observers in real time could significantly expedite adoption of artificial intelligence in clinical practice.

Printed on: 03/01/22