Hee Chun, Presenter: Nothing to Disclose

Bradley Clymer PhD, Abstract Co-Author: Nothing to Disclose

Michael Vinzenz Knopp MD, Abstract Co-Author: Nothing to Disclose

The rapid advances in image acquisition and co-registration lead to multi-parametric/multidimensional data sets. Morphologic information commonly displayed in grey-scale is now supplemented by spatially resolved functional and molecular information. Thus the challenge, how to communicate the additional dimensions of information beyond our capable, but limited visual perception. As our sensory and tactual perception in the hand is excellent, we intended to explore how supplementary information can complement our radiologic visual perception.

In order to introduce an advanced, but readily available tactile interfacing device, we used Microsoft’s sidewinder force feedback2 joy-stick as a hand-based visual-haptic interaction system. We implemented this capability into our IDL (Interactive Data Language, RSI) software windows environment by programming a real-time interface and feedback module. The X, Y, and Z axis on the force feedback joystick device are used to fly/navigate around the 3D imaging volume object, and the different magnitude of force is reflected onto our hands based on voxel aligned intensity values through the device at the pull of the trigger. Several buttons which replace the conventional mouse buttons are used for drawing, rotating, panning, and zooming of the volume of interest (VOI) within the imaging data set. The gain, active duration, and sampling rate of the force are adjustable using slider widgets for different feeling of force feedback.

After technical implementation of the real-time interface, multi-dimensional image sets from functional, dynamic MR and PET studies were used to demonstrate the feasibility of this approach. A pilot evaluation found all data sets navigable in multiple dimensions, and supplementing gray-scale and color–overlay with additional information feedback. In these data sets, force feedback could demonstrate an accurate haptic representation of the supplemental informational image dimensions.

A real-time visual-haptic feedback was shown to be feasible to supplement multi-dimensional medical image analysis and appears to be a potentially powerful and effective addition to visual perception of radiologic information.

Chun, H, Clymer, B, Knopp, M, Medical Image Analysis Interfacing with Real-time Visual-Tactile Sensation.  Radiological Society of North America 2004 Scientific Assembly and Annual Meeting, November 28 - December 3, 2004 ,Chicago IL. http://archive.rsna.org/2004/4415889.html