1
|
Shabir D, Anjum A, Hamza H, Padhan J, Al-Ansari A, Yaacoub E, Mohammed A, Navkar NV. Development and Evaluation of a Mixed-Reality Tele-ultrasound System. ULTRASOUND IN MEDICINE & BIOLOGY 2023; 49:1867-1874. [PMID: 37263893 DOI: 10.1016/j.ultrasmedbio.2023.04.017] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/15/2022] [Revised: 02/25/2023] [Accepted: 04/28/2023] [Indexed: 06/03/2023]
Abstract
OBJECTIVE The objective of this feasibility study was to develop and assess a tele-ultrasound system that would enable an expert sonographer (situated at the remote site) to provide real-time guidance to an operator (situated at the imaging site) using a mixed-reality environment. METHODS An architecture along with the operational workflow of the system is designed and a prototype is developed that enables guidance in form of audiovisual cues. The visual cues comprise holograms (of the ultrasound images and ultrasound probe) and is rendered to the operator using a head-mounted display device. The position and orientation of the ultrasound probe's hologram are remotely controlled by the expert sonographer and guide the placement of a physical ultrasound probe at the imaging site. The developed prototype was evaluated for its performance on a network. In addition, a user study (with 12 participants) was conducted to assess the operator's ability to align the probe under different guidance modes. RESULTS The network performance revealed the view of the imaging site and ultrasound images were transferred to the remote site in 233 ± 42 and 158 ± 38 ms, respectively. The expert sonographer was able to transfer, to the imaging site, data related to position and orientation of the ultrasound probe's hologram in 78 ± 13 ms. The user study indicated that the audiovisual cues are sufficient for an operator to position and orient a physical probe for accurate depiction of the targeted tissue (p < 0.001). The probe's placement translational and rotational errors were 1.4 ± 0.6 mm and 5.4 ± 2.2º. CONCLUSION The work illustrates the feasibility of using a mixed-reality environment for effective communication between an expert sonographer (ultrasound physician) and an operator. Further studies are required to determine its applicability in a clinical setting during tele-ultrasound.
Collapse
Affiliation(s)
- Dehlela Shabir
- Department of Surgery, Hamad Medical Corporation, Doha, Qatar
| | - Arshak Anjum
- Department of Computer Science and Engineering, Qatar University, Doha, Qatar
| | - Hawa Hamza
- Department of Surgery, Hamad Medical Corporation, Doha, Qatar
| | | | | | - Elias Yaacoub
- Department of Computer Science and Engineering, Qatar University, Doha, Qatar
| | - Amr Mohammed
- Department of Computer Science and Engineering, Qatar University, Doha, Qatar
| | - Nikhil V Navkar
- Department of Surgery, Hamad Medical Corporation, Doha, Qatar.
| |
Collapse
|
2
|
Khorasani M, Abdurahiman N, Padhan J, Zhao H, Al-Ansari A, Becker AT, Navkar N. Preliminary design and evaluation of a generic surgical scope adapter. Int J Med Robot 2023; 19:e2475. [PMID: 36288569 DOI: 10.1002/rcs.2475] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/22/2022] [Revised: 08/21/2022] [Accepted: 10/25/2022] [Indexed: 11/06/2022]
Abstract
BACKGROUND Robotic scope assistant systems are used to visualise and navigate the operative field during a laparoscopic surgery. The objective of this work is to design a surgical scope adapter that enables control of different scope types (zero-degree, angulated, and articulated), and can be connected to any six degree-of-freedom robotic manipulator for usage as a robotic scope assistant system. METHODS A surgical scope adapter compatible with different camera heads and scope types was designed and prototyped. The technical performance of the scope adapter was evaluated and a user study was conducted to assess the human-in-the-loop control. RESULTS All the subjects were able to navigate the simulated operative field. The scope adapter permits continuous motion to explore the operative field as well as intermittent motion to accurately focus on the targeted anatomical landmarks. CONCLUSION The modular and generic nature of the surgical scope adapter may enable its usage across different minimally invasive surgeries.
Collapse
Affiliation(s)
| | | | | | - Haoran Zhao
- Department of Electrical and Computer Engineering, University of Houston, Houston, Texas, USA
| | | | - Aaron T Becker
- Department of Electrical and Computer Engineering, University of Houston, Houston, Texas, USA
| | - Nikhil Navkar
- Department of Surgery, Hamad Medical Corporation, Doha, Qatar
| |
Collapse
|
3
|
Hamza H, Baez VM, Al-Ansari A, Becker AT, Navkar NV. User interfaces for actuated scope maneuvering in surgical systems: a scoping review. Surg Endosc 2023:10.1007/s00464-023-09981-0. [DOI: 10.1007/s00464-023-09981-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/02/2022] [Accepted: 02/25/2023] [Indexed: 03/29/2023]
Abstract
Abstract
Background
A variety of human computer interfaces are used by robotic surgical systems to control and actuate camera scopes during minimally invasive surgery. The purpose of this review is to examine the different user interfaces used in both commercial systems and research prototypes.
Methods
A comprehensive scoping review of scientific literature was conducted using PubMed and IEEE Xplore databases to identify user interfaces used in commercial products and research prototypes of robotic surgical systems and robotic scope holders. Papers related to actuated scopes with human–computer interfaces were included. Several aspects of user interfaces for scope manipulation in commercial and research systems were reviewed.
Results
Scope assistance was classified into robotic surgical systems (for multiple port, single port, and natural orifice) and robotic scope holders (for rigid, articulated, and flexible endoscopes). Benefits and drawbacks of control by different user interfaces such as foot, hand, voice, head, eye, and tool tracking were outlined. In the review, it was observed that hand control, with its familiarity and intuitiveness, is the most used interface in commercially available systems. Control by foot, head tracking, and tool tracking are increasingly used to address limitations, such as interruptions to surgical workflow, caused by using a hand interface.
Conclusion
Integrating a combination of different user interfaces for scope manipulation may provide maximum benefit for the surgeons. However, smooth transition between interfaces might pose a challenge while combining controls.
Collapse
|
4
|
Abdurahiman N, Khorasani M, Padhan J, Baez VM, Al-Ansari A, Tsiamyrtzis P, Becker AT, Navkar NV. Scope actuation system for articulated laparoscopes. Surg Endosc 2023; 37:2404-2413. [PMID: 36750488 PMCID: PMC10017632 DOI: 10.1007/s00464-023-09904-z] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/01/2022] [Accepted: 01/21/2023] [Indexed: 02/09/2023]
Abstract
BACKGROUND An articulated laparoscope comprises a rigid shaft with an articulated distal end to change the viewing direction. The articulation provides improved navigation of the operating field in confined spaces. Furthermore, incorporation of an actuation system tends to enhance the control of an articulated laparoscope. METHODS A preliminary prototype of a scope actuation system to maneuver an off-the-shelf articulated laparoscope (EndoCAMaleon by Karl Storz, Germany) was developed. A user study was conducted to evaluate this prototype for the surgical paradigm of video-assisted thoracic surgery. In the study, the subjects maneuvered an articulated scope under two modes of operation: (a) actuated mode where an operating surgeon maneuvers the scope using the developed prototype and (b) manual mode where a surgical assistant directly maneuvers the scope. The actuated mode was further assessed for multiple configurations based on the orientation of the articulated scope at the incision. RESULTS The data show the actuated mode scored better than the manual mode on all the measured performance parameters including (a) total duration to visualize a marked region, (a) duration for which scope focus shifts outside a predefined visualization region, and (c) number of times for which scope focus shifts outside a predefined visualization region. Among the different configurations tested using the actuated mode, no significant difference was observed. CONCLUSIONS The proposed articulated scope actuation system facilitates better navigation of an operative field as compared to a human assistant. Secondly, irrespective of the orientation in which an articulated scope's shaft is inserted through an incision, the proposed actuation system can navigate and visualize the operative field.
Collapse
Affiliation(s)
| | | | | | - Victor M Baez
- Department of Electrical Engineering, University of Houston, Houston, TX, USA
| | | | | | - Aaron T Becker
- Department of Electrical Engineering, University of Houston, Houston, TX, USA
| | - Nikhil V Navkar
- Department of Surgery, Hamad Medical Corporation, Doha, Qatar.
- Department of Surgery, Surgical Research Section, Hamad General Hospital, Hamad Medical Corporation, PO Box 3050, Doha, Qatar.
| |
Collapse
|
5
|
Shabir D, Anbatawi M, Padhan J, Balakrishnan S, Al‐Ansari A, Abinahed J, Tsiamyrtzis P, Yaacoub E, Mohammed A, Deng Z, Navkar NV. Evaluation of user‐interfaces for controlling movements of virtual minimally invasive surgical instruments. Int J Med Robot 2022; 18:e2414. [DOI: 10.1002/rcs.2414] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/07/2022] [Revised: 04/10/2022] [Accepted: 04/27/2022] [Indexed: 11/10/2022]
Affiliation(s)
- Dehlela Shabir
- Department of Surgery Hamad Medical Corporation Doha Qatar
| | - Malek Anbatawi
- Department of Surgery Hamad Medical Corporation Doha Qatar
| | | | | | | | | | | | - Elias Yaacoub
- Department of Computer Science and Engineering Qatar University Doha Qatar
| | - Amr Mohammed
- Department of Computer Science and Engineering Qatar University Doha Qatar
| | - Zhigang Deng
- Department of Computer Science University of Houston Houston Texas USA
| | | |
Collapse
|
6
|
Shabir D, Abdurahiman N, Padhan J, Anbatawi M, Trinh M, Balakrishnan S, Al-Ansari A, Yaacoub E, Deng Z, Erbad A, Mohammed A, Navkar NV. Preliminary design and evaluation of a remote tele-mentoring system for minimally invasive surgery. Surg Endosc 2022; 36:3663-3674. [PMID: 35246742 PMCID: PMC9001542 DOI: 10.1007/s00464-022-09164-3] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/18/2021] [Accepted: 02/18/2022] [Indexed: 12/26/2022]
Abstract
BACKGROUND Tele-mentoring during surgery facilitates the transfer of surgical knowledge from a mentor (specialist surgeon) to a mentee (operating surgeon). The aim of this work is to develop a tele-mentoring system tailored for minimally invasive surgery (MIS) where the mentor can remotely demonstrate to the mentee the required motion of the surgical instruments. METHODS A remote tele-mentoring system is implemented that generates visual cues in the form of virtual surgical instrument motion overlaid onto the live view of the operative field. The technical performance of the system is evaluated in a simulated environment, where the operating room and the central location of the mentor were physically located in different countries and connected over the internet. In addition, a user study was performed to assess the system as a mentoring tool. RESULTS On average, it took 260 ms to send a view of the operative field of 1920 × 1080 resolution from the operating room to the central location of the mentor and an average of 132 ms to receive the motion of virtual surgical instruments from the central location to the operating room. The user study showed that it is feasible for the mentor to demonstrate and for the mentee to understand and replicate the motion of surgical instruments. CONCLUSION The work demonstrates the feasibility of transferring information over the internet from a mentor to a mentee in the form of virtual surgical instruments. Their motion is overlaid onto the live view of the operative field enabling real-time interactions between both the surgeons.
Collapse
Affiliation(s)
- Dehlela Shabir
- Department of Surgery, Surgical Research Section, Hamad General Hospital, Hamad Medical Corporation, PO Box 3050, Doha, Qatar
| | - Nihal Abdurahiman
- Department of Surgery, Surgical Research Section, Hamad General Hospital, Hamad Medical Corporation, PO Box 3050, Doha, Qatar
| | - Jhasketan Padhan
- Department of Surgery, Surgical Research Section, Hamad General Hospital, Hamad Medical Corporation, PO Box 3050, Doha, Qatar
| | - Malek Anbatawi
- Department of Surgery, Surgical Research Section, Hamad General Hospital, Hamad Medical Corporation, PO Box 3050, Doha, Qatar
| | - May Trinh
- Department of Computer Science, University of Houston, Houston, TX, USA
| | - Shidin Balakrishnan
- Department of Surgery, Surgical Research Section, Hamad General Hospital, Hamad Medical Corporation, PO Box 3050, Doha, Qatar
| | - Abdulla Al-Ansari
- Department of Surgery, Surgical Research Section, Hamad General Hospital, Hamad Medical Corporation, PO Box 3050, Doha, Qatar
| | - Elias Yaacoub
- Department of Computer Science and Engineering, Qatar University, Doha, Qatar
| | - Zhigang Deng
- Department of Computer Science, University of Houston, Houston, TX, USA
| | - Aiman Erbad
- College of Science and Engineering, Hamad Bin Khalifa University, Doha, Qatar
| | - Amr Mohammed
- Department of Computer Science and Engineering, Qatar University, Doha, Qatar
| | - Nikhil V Navkar
- Department of Surgery, Surgical Research Section, Hamad General Hospital, Hamad Medical Corporation, PO Box 3050, Doha, Qatar.
| |
Collapse
|
7
|
3D visualization of perianal fistulas using parametric models. Tech Coloproctol 2022; 26:291-300. [DOI: 10.1007/s10151-022-02573-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/12/2021] [Accepted: 01/13/2022] [Indexed: 10/19/2022]
|
8
|
Velazco-Garcia JD, Navkar NV, Balakrishnan S, Younes G, Abi-Nahed J, Al-Rumaihi K, Darweesh A, Elakkad MSM, Al-Ansari A, Christoforou EG, Karkoub M, Leiss EL, Tsiamyrtzis P, Tsekos NV. Evaluation of how users interface with holographic augmented reality surgical scenes: Interactive planning MR-Guided prostate biopsies. Int J Med Robot 2021; 17:e2290. [PMID: 34060214 DOI: 10.1002/rcs.2290] [Citation(s) in RCA: 12] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/19/2020] [Revised: 05/04/2021] [Accepted: 05/27/2021] [Indexed: 12/15/2022]
Abstract
BACKGROUND User interfaces play a vital role in the planning and execution of an interventional procedure. The objective of this study is to investigate the effect of using different user interfaces for planning transrectal robot-assisted MR-guided prostate biopsy (MRgPBx) in an augmented reality (AR) environment. METHOD End-user studies were conducted by simulating an MRgPBx system with end- and side-firing modes. The information from the system to the operator was rendered on HoloLens as an output interface. Joystick, mouse/keyboard, and holographic menus were used as input interfaces to the system. RESULTS The studies indicated that using a joystick improved the interactive capacity and enabled operator to plan MRgPBx in less time. It efficiently captures the operator's commands to manipulate the augmented environment representing the state of MRgPBx system. CONCLUSIONS The study demonstrates an alternative to conventional input interfaces to interact and manipulate an AR environment within the context of MRgPBx planning.
Collapse
Affiliation(s)
| | - Nikhil V Navkar
- Department of Surgery, Hamad Medical Corporation, Doha, Qatar
| | | | - Georges Younes
- Department of Surgery, Hamad Medical Corporation, Doha, Qatar
| | | | | | - Adham Darweesh
- Department of Clinical Imaging, Hamad Medical Corporation, Doha, Qatar
| | | | | | | | - Mansour Karkoub
- Department of Mechanical Engineering, Texas A&M University-Qatar, Doha, Qatar
| | - Ernst L Leiss
- Department of Computer Science, University of Houston, Houston, Texas, USA
| | | | - Nikolaos V Tsekos
- Department of Computer Science, University of Houston, Houston, Texas, USA
| |
Collapse
|
9
|
Morales Mojica CM, Velazco-Garcia JD, Pappas EP, Birbilis TA, Becker A, Leiss EL, Webb A, Seimenis I, Tsekos NV. A Holographic Augmented Reality Interface for Visualizing of MRI Data and Planning of Neurosurgical Procedures. J Digit Imaging 2021; 34:1014-1025. [PMID: 34027587 DOI: 10.1007/s10278-020-00412-3] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/17/2020] [Revised: 12/06/2020] [Accepted: 12/18/2020] [Indexed: 10/21/2022] Open
Abstract
The recent introduction of wireless head-mounted displays (HMD) promises to enhance 3D image visualization by immersing the user into 3D morphology. This work introduces a prototype holographic augmented reality (HAR) interface for the 3D visualization of magnetic resonance imaging (MRI) data for the purpose of planning neurosurgical procedures. The computational platform generates a HAR scene that fuses pre-operative MRI sets, segmented anatomical structures, and a tubular tool for planning an access path to the targeted pathology. The operator can manipulate the presented images and segmented structures and perform path-planning using voice and gestures. On-the-fly, the software uses defined forbidden-regions to prevent the operator from harming vital structures. In silico studies using the platform with a HoloLens HMD assessed its functionality and the computational load and memory for different tasks. A preliminary qualitative evaluation revealed that holographic visualization of high-resolution 3D MRI data offers an intuitive and interactive perspective of the complex brain vasculature and anatomical structures. This initial work suggests that immersive experiences may be an unparalleled tool for planning neurosurgical procedures.
Collapse
Affiliation(s)
- Cristina M Morales Mojica
- MRI Lab, Department of Computer Science, University of Houston, 4800 Calhoun Road PGH 501, Houston, TX, USA
| | - Jose D Velazco-Garcia
- MRI Lab, Department of Computer Science, University of Houston, 4800 Calhoun Road PGH 501, Houston, TX, USA
| | - Eleftherios P Pappas
- Medical Physics Laboratory, School of Medicine, National and Kapodistrian University of Athens, Athens, Greece
| | | | - Aaron Becker
- Department of Electrical and Computer Engineering, University of Houston, Houston, TX, USA
| | - Ernst L Leiss
- MRI Lab, Department of Computer Science, University of Houston, 4800 Calhoun Road PGH 501, Houston, TX, USA
| | - Andrew Webb
- C.J. Gorter Center for High Field MRI, Leiden University Medical Center, Leiden, Netherlands
| | - Ioannis Seimenis
- Medical Physics Laboratory, School of Medicine, National and Kapodistrian University of Athens, Athens, Greece
| | - Nikolaos V Tsekos
- MRI Lab, Department of Computer Science, University of Houston, 4800 Calhoun Road PGH 501, Houston, TX, USA.
| |
Collapse
|