Abstract：In order to establish a feasible augmented reality(AR) technique for surgical treatment, several proposals for AR have been made in recent years. By means of different pre-and intraoperative information sources like surgical plannings, intraoperative imaging, and tracking devices, AR technique aims at supporting surgeons in localizing anatomical targets, observing critical structures, and sparing healthy tissue. Current research in particular addresses the problem of organ shift and tissue deformation, and obstacles in communication between navigation systems and surgeons. In this review, we examine the potential applications of AR in minimally invasive surgery, with a focus on the use of endoscopic images as a tool to measure, register, and display anatomic structure. As this is a relatively new and rapidly evolving area, our goal is to illustrate potential areas where vision-based navigation can be applied and to describe progress towards realizing these applications.
[1]M?rvik R, Lang T, Tangen G, et al. Image-guided laparoscopic surgery-review and current status. Minerva chirurgica, 2005, 60 (5): 305-325. [2]Gildenberg PL. The birth of stereotactic surgery: a personal retrospective. Neurosurgery, 2004, 54(1):199-207. [3]Birkfellner W, Figl M, Huber K, et al. A head-mounted operating binocular for augmented reality visualization in medicine--design and initial evaluation. IEEE Transactions on Medical Imaging, 2002, 21(8): 991-997. [4]Yip MC, Adebar TK, Rohling RN, et al. 3D ultrasound to stereoscopic camera registration through an air-tissue boundary. Med Image Comput Comput Assist Interv, 2010,13 ( 2): 626-634. [5]Peters TM. Image-guidance for surgical procedures. Phys Med Biol, 2006, 51(14):505-540. [6]Azuma R, Baillot Y, Behringer R, et al. Recent advances in augmented reality. IEEE Comput Graph Appl, 2001,2001 (21): 34-47. [7]Wacker F, Vogt S, Khamene A, et al. An augmented reality system for MR image-guided needle biopsy: initial results in a swine model. Radiology, 2006, 238(2): 497-504. [8]Rassweiler JJ, Müller M, Fangerau M, et al. iPad-assisted percutaneous access to the kidney using marker-based navigation: initial clinical experience. Eur Urol, 2012, 61(3):628-631. [9]Ukimura O. Image-guided surgery in minimally invasive urology. Curr Opin Urol, 2010(20): 136-140. [10]Simon Adler,Zein Salah. Overlay of patient-specific anatomical data for advanced navigation in surgery simulation. International Workshop on Digital Engineering, 2010: 52-58. [11]Mirota DJ, Ishii M, Hager GD. Vision-based navigation in image-guided interventions. Annu Rev Biomed Eng, 2011, 13:297-319. [12]Teber D, Guven S, Simpfend?rfer T, et al. Augmented reality: a new tool to improve surgical accuracy during laparoscopic partial nephrectomy? Preliminary in vitro and in vivo results. Eur Urol, 2009, 56(2):332-338. [13]Su LM, Vagvolgyi BP, Agarwal R, et al. Augmented reality during robot-assisted laparoscopic partial nephrectomy: toward real-time 3D-CT to stereoscopic video registration. Urology, 2009(73):896-900. [14]Sugimoto M, Yasuda H, Koda K, et al. Image overlay navigation by markerless surface registration in gastrointestinal, hepatobiliary and pancreatic surgery. J Hepatobiliary Pancreat Sci, 2010, 17(5):629-636. [15]Fitzpatrick JM, West JB, Maurer CR. Predicting error in rigid-body point-based registration. IEEE Trans Med Imaging, 1998(17): 694-702. [16]Lau WW, Ramey NA, Corso JJ, et al. Stereo-based endoscopic tracking of cardiac surface deformation. Med Image Comput Comput Assist Interv, 2004, 2: 494-501. [17]Richa R, Bo APL, Poignet P. Robust 3D visual tracking for robotic-assisted cardiac interventions. Proc Med Image Comput Comput Assist Interv, 13th ed, Beijing, Springer-Verlag, 2010:267-274. [18]Mountney P, Yang GZ. Motion compensated SLAM for image guided surgery. Proc Med Image Comput Comput Assist Interv, 13th, Beijing, Springer-Verlag, 2010:496-504. [19]Penne J, Holler K, Sturmer M, et al. Time-of-flight 3-D endoscopy. Proc Med Image Comput Comput Assist Interv, 12thed, London, Springer-Verlag, 2009:467-474. [20]Baumhauer M, Simpfendoerfer T, Wolf I, et al. Soft tissue navigation for laparoscopic prostatectomy: Evaluation of camera pose estimation for enhanced visualization in Medical Imaging. J Endourol, 2011, 25(12): 1841-1845. [21]Lee SL, Lerotic M, Vitiello V, et al. From medical images to minimally invasive intervention: Computer assistance for robotic surgery. Comput Med Imaging Graph, 2010,34(1): 33-45. [22]Lemke H. Surgical pacs and the digital operating room. Int J Comput Assist Radiol Surg, 2007(2): 427-444. [23]Peters T. M. Image-guidance for surgical procedures. Phys MedBiol, 2006, 51(14): 505-540. [24]Philipp J Stolka, Matthias Keil. A 3D-Elastography-Guided System for Laparoscopic Partial Nephrectomies. Medical Imaging 2010: Visualization Image-Guided Procedures and Modeling, 2010(7625): 11-12. [25]Pablo Lamata. Augmented Reality for Minimally Invasive Surgery: Overview and Some Recent Advances. Augmented Reality, 5th ed, Croatia, INTECH, 2010: 230.
