akino@mskcc.org

To develop a real-time attention tracking system that can handle dynamic back-and-forth scrolling through a series of images without any hardware or software intrusion while the radiologist interpreting a study.

The system framework was built on the Open Health Imaging Foundation's open-source medical image viewer with integration of a Tobii 4C Eye Tracking camera. Spatial and temporal coordinates of the series of images are automatically recorded and the corresponding attention heapmaps are generated. For attention and search patent analysis, the commonly used eye-tracking metrics are analyzed including area of interest (AOI), time to first fixation, total fixation duration, fixation duration on AOI, number of fixations on AOI, dwell time ratio, and saccade length on each image. Furthermore, 3D dynamic analysis of attention characteristics are studied including total time of the scan, slice of interest (SOI), fixation duration on SOI, number of fixations on SOI, number of scrolling (scrolling back-and-forth while focusing on different areas), number of drilling (scrolling back-and-forth while focusing on one area). The system's performance was tested using the data from 4 radiologists while interpreting 30 CT studies from the LUNA16 dataset.

The system successfully captured the attention data in all interpretation sessions (n=120) without any software or hardware failure. The accuracy of the gaze tracking is 0.4º which is about 3.5-7 mm on the computer screen at a distance range of 0.5-1 m between the observer and the camera. The analysis of the attention heatmaps showed spatial and temporal variations in the 3D dynamic attention characteristics of 4 radiologists, indicating unique search patterns among different observers.

The proposed system can be used as an objective tool to study unique search patterns among human observers. These data can be further used to develop more interactive human-computer interfaces for artificial intelligence applications.

As artificial intelligence applications advance, there is an increasing need to develop seamless human-computer interfaces that can capture the radiologists' attention. Such systems allowing artificial intelligence algorithms to operate more interactively with the human observers in real time could significantly expedite adoption of artificial intelligence in clinical practice.