An Augmented Reality System with Advanced User Interfaces for Image-Guided Intervention Applications

Augmented Reality (AR) is becoming a more common addition to physicians' repertoire for aiding in resident training and patient interactions. However, the use of augmented reality in clinical settings is still beset with many complications, including the lack of physician control over the systems, set modes of interactions within the system, and physician's lack of familiarity with such AR systems. In this paper, we plan to expand on our previous prostate biopsy AR system by adding in improved user interface systems within the virtual world in order to allow the user to more accurately visualize only parts of the system which they consider to be useful at that time. To accomplish this, we have incorporated three-dimensional virtual sliders built from the ground up, using Unity to afford control over each model's RGB values, as well as their transparency. This means that the user would be able to fully edit the color, and transparency of each individual model in real time as they see fit quickly and easily while still being immersed in the augmented space. This would allow users to view internal holograms while not sacrificing the capability to view the external structure. Such leeway could be invaluable when visualizing a tumor within a prostate and would provide the physician with the capability to view as much or as little of the surrounded virtual models as desired, while providing the option to reinstate the surrounding models at will. The AR system can provide a new approach for potential uses in image-guided interventions including targeted biopsy of the prostate.