1
|
Anabtawi M, Shabir D, Padhan J, Al-Ansari A, Aboumarzouk OM, Deng Z, Navkar NV. A holographic telementoring system depicting surgical instrument movements for real-time guidance in open surgeries. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2024; 256:108396. [PMID: 39213900 DOI: 10.1016/j.cmpb.2024.108396] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/16/2024] [Revised: 08/18/2024] [Accepted: 08/23/2024] [Indexed: 09/04/2024]
Abstract
BACKGROUND AND OBJECTIVE During open surgeries, telementoring serves as a valuable tool for transferring surgical knowledge from a specialist surgeon (mentor) to an operating surgeon (mentee). Depicting the intended movements of the surgical instruments over the operative field improves the understanding of the required tool-tissue interaction. The objective of this work is to develop a telementoring system tailored for open surgeries, enabling the mentor to remotely demonstrate the necessary motions of surgical instruments to the mentee. METHODS A remote telementoring system for open surgery was implemented. The system generates visual cues in the form of virtual surgical instrument motion augmented onto the live view of the operative field. These cues can be rendered on both conventional screens in the operating room and as dynamic holograms on a head mounted display device worn by the mentee. The technical performance of the system was evaluated, where the operating room and remote location were geographically separated and connected via the Internet. Additionally, user studies were conducted to assess the effectiveness of the system as a mentoring tool. RESULTS The system took 307 ± 12 ms to transmit an operative field view of 1920 × 1080 resolution, along with depth information spanning 36 cm, from the operating room to the remote location. Conversely, it took 145 ± 14 ms to receive the motion of virtual surgical instruments from the remote location back to the operating room. Furthermore, the user studies demonstrated: (a) mentor's capability to annotate the operative field with an accuracy of 3.92 ± 2.1 mm, (b) mentee's ability to comprehend and replicate the motion of surgical instruments in real-time with an average deviation of 12.8 ± 3 mm, (c) efficacy of the rendered dynamic holograms in conveying information intended for surgical instrument motion. CONCLUSIONS The study demonstrates the feasibility of transmitting information over the Internet from the mentor to the mentee in the form of virtual surgical instruments' motion and projecting it as holograms onto the live view of the operative field. This holds potential to enhance real-time collaborative capabilities between the mentor and the mentee during an open surgery.
Collapse
Affiliation(s)
- Malek Anabtawi
- Department of Surgery, Hamad Medical Corporation, Doha, Qatar
| | - Dehlela Shabir
- Department of Surgery, Hamad Medical Corporation, Doha, Qatar
| | | | | | | | - Zhigang Deng
- Department of Computer Science, University of Houston, Houston, TX, USA
| | - Nikhil V Navkar
- Department of Surgery, Hamad Medical Corporation, Doha, Qatar.
| |
Collapse
|
2
|
Hamza H, Al-Ansari A, Navkar NV. Technologies Used for Telementoring in Open Surgery: A Scoping Review. Telemed J E Health 2024; 30:1810-1824. [PMID: 38546446 DOI: 10.1089/tmj.2023.0669] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 07/20/2024] Open
Abstract
Background: Telementoring technologies enable a remote mentor to guide a mentee in real-time during surgical procedures. This addresses challenges, such as lack of expertise and limited surgical training/education opportunities in remote locations. This review aims to provide a comprehensive account of these technologies tailored for open surgery. Methods: A comprehensive scoping review of the scientific literature was conducted using PubMed, ScienceDirect, ACM Digital Library, and IEEE Xplore databases. Broad and inclusive searches were done to identify articles reporting telementoring or teleguidance technologies in open surgery. Results: Screening of the search results yielded 43 articles describing surgical telementoring for open approach. The studies were categorized based on the type of open surgery (surgical specialty, surgical procedure, and stage of clinical trial), the telementoring technology used (information transferred between mentor and mentee, devices used for rendering the information), and assessment of the technology (experience level of mentor and mentee, study design, and assessment criteria). Majority of the telementoring technologies focused on trauma-related surgeries and mixed reality headsets were commonly used for rendering information (telestrations, surgical tools, or hand gestures) to the mentee. These technologies were primarily assessed on high-fidelity synthetic phantoms. Conclusions: Despite longer operative time, these telementoring technologies demonstrated clinical viability during open surgeries through improved performance and confidence of the mentee. In general, usage of immersive devices and annotations appears to be promising, although further clinical trials will be required to thoroughly assess its benefits.
Collapse
Affiliation(s)
- Hawa Hamza
- Department of Surgery, Hamad Medical Corporation, Doha, Qatar
| | | | - Nikhil V Navkar
- Department of Surgery, Hamad Medical Corporation, Doha, Qatar
| |
Collapse
|
3
|
Takemasa I, Okuya K, Okita K, Akizuki E, Miyo M, Ishii M, Miura R, Ichihara M, Takahiro K, Oki E, Takatsuki M, Eguchi S, Ichikawa D, Kitagawa Y, Sakai Y, Mori M. Tele-proctoring for minimally invasive surgery across Japan: An initial step toward a new approach to improving the disparity of surgical care and supporting surgical education. Ann Gastroenterol Surg 2024; 8:356-364. [PMID: 38455497 PMCID: PMC10914704 DOI: 10.1002/ags3.12750] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/18/2023] [Revised: 09/18/2023] [Accepted: 10/08/2023] [Indexed: 03/09/2024] Open
Abstract
Aim The aim of this study was to verify the clinical feasibility of tele-proctoring using our ultra-low latency communication system with shared internet access. Methods Connections between two multiple remote locations at various distances were established through the TELEPRO® tele-proctoring system. The server records the latency between the two locations for tele-proctoring using the annotations. Questionnaires were administered to the surgeons, assistants, and medical staff. Respondents rated the quickness and quality of communication in terms of latency and disturbances in the audio, video, and usefulness of the live telestrations with annotation. Results Seven hospitals tele-proctored with Sapporo Medical University between January 2021 and September 2022. The median latency of annotation between the two locations ranged from 24.5 to 48.5 ms. No major technological problems occurred, such as streaming interruption, loss of video or audio, poor resolution. The video encoding time was 10 ms, and its decoding time was 0.8 ms. The total latency positively correlated with the distance between two locations (R = 0.55, p < 0.01). The quality of communication regarding latency, disturbance, and surgical education with intraoperative annotative instructions showed similar trends, with perfectly fine being the most common response. No significant differences in surgical quality, educational effect, or social impact were observed between the latency ≥30 and <30 ms groups for whether the size of latency affects surgical education. Conclusion The feasibility of the tele-proctoring system is expected to be a sustainable approach to help education for young surgeons and surgical supports in rural areas, thereby reducing disparities in health care.
Collapse
Affiliation(s)
- Ichiro Takemasa
- Department of Surgery, Surgical Oncology and ScienceSapporo Medical University School of MedicineSapporo CityJapan
| | - Koichi Okuya
- Department of Surgery, Surgical Oncology and ScienceSapporo Medical University School of MedicineSapporo CityJapan
| | - Kenji Okita
- Department of Surgery, Surgical Oncology and ScienceSapporo Medical University School of MedicineSapporo CityJapan
| | - Emi Akizuki
- Department of Surgery, Surgical Oncology and ScienceSapporo Medical University School of MedicineSapporo CityJapan
| | - Masaaki Miyo
- Department of Surgery, Surgical Oncology and ScienceSapporo Medical University School of MedicineSapporo CityJapan
| | - Masayuki Ishii
- Department of Surgery, Surgical Oncology and ScienceSapporo Medical University School of MedicineSapporo CityJapan
| | - Ryo Miura
- Department of Surgery, Surgical Oncology and ScienceSapporo Medical University School of MedicineSapporo CityJapan
| | - Momoko Ichihara
- Department of Surgery, Surgical Oncology and ScienceSapporo Medical University School of MedicineSapporo CityJapan
| | - Korai Takahiro
- Department of Surgery, Surgical Oncology and ScienceSapporo Medical University School of MedicineSapporo CityJapan
| | - Eiji Oki
- Department of Surgery and Science, Graduate School of Medical SciencesKyushu UniversityFukuokaJapan
| | - Mitsuhisa Takatsuki
- Department of Digestive and General Surgery, Graduate School of MedicineUniversity of the RyukyusNishiharaJapan
| | - Susumu Eguchi
- Department of SurgeryNagasaki University Graduate School of Biomedical SciencesNagasakiJapan
| | - Daisuke Ichikawa
- First Department of SurgeryUniversity of Yamanashi, Faculty of Medicine, Graduate School of MedicineYamanashiJapan
| | - Yuko Kitagawa
- Department of SurgeryKeio University School of MedicineTokyoJapan
| | - Yoshiharu Sakai
- Department of SurgeryRed Cross Hospital OsakaOsaka CityJapan
| | - Masaki Mori
- Department of Surgery and Science, Graduate School of Medical SciencesKyushu UniversityFukuokaJapan
| |
Collapse
|
4
|
Shabir D, Anjum A, Hamza H, Padhan J, Al-Ansari A, Yaacoub E, Mohammed A, Navkar NV. Development and Evaluation of a Mixed-Reality Tele-ultrasound System. ULTRASOUND IN MEDICINE & BIOLOGY 2023; 49:1867-1874. [PMID: 37263893 DOI: 10.1016/j.ultrasmedbio.2023.04.017] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/15/2022] [Revised: 02/25/2023] [Accepted: 04/28/2023] [Indexed: 06/03/2023]
Abstract
OBJECTIVE The objective of this feasibility study was to develop and assess a tele-ultrasound system that would enable an expert sonographer (situated at the remote site) to provide real-time guidance to an operator (situated at the imaging site) using a mixed-reality environment. METHODS An architecture along with the operational workflow of the system is designed and a prototype is developed that enables guidance in form of audiovisual cues. The visual cues comprise holograms (of the ultrasound images and ultrasound probe) and is rendered to the operator using a head-mounted display device. The position and orientation of the ultrasound probe's hologram are remotely controlled by the expert sonographer and guide the placement of a physical ultrasound probe at the imaging site. The developed prototype was evaluated for its performance on a network. In addition, a user study (with 12 participants) was conducted to assess the operator's ability to align the probe under different guidance modes. RESULTS The network performance revealed the view of the imaging site and ultrasound images were transferred to the remote site in 233 ± 42 and 158 ± 38 ms, respectively. The expert sonographer was able to transfer, to the imaging site, data related to position and orientation of the ultrasound probe's hologram in 78 ± 13 ms. The user study indicated that the audiovisual cues are sufficient for an operator to position and orient a physical probe for accurate depiction of the targeted tissue (p < 0.001). The probe's placement translational and rotational errors were 1.4 ± 0.6 mm and 5.4 ± 2.2º. CONCLUSION The work illustrates the feasibility of using a mixed-reality environment for effective communication between an expert sonographer (ultrasound physician) and an operator. Further studies are required to determine its applicability in a clinical setting during tele-ultrasound.
Collapse
Affiliation(s)
- Dehlela Shabir
- Department of Surgery, Hamad Medical Corporation, Doha, Qatar
| | - Arshak Anjum
- Department of Computer Science and Engineering, Qatar University, Doha, Qatar
| | - Hawa Hamza
- Department of Surgery, Hamad Medical Corporation, Doha, Qatar
| | | | | | - Elias Yaacoub
- Department of Computer Science and Engineering, Qatar University, Doha, Qatar
| | - Amr Mohammed
- Department of Computer Science and Engineering, Qatar University, Doha, Qatar
| | - Nikhil V Navkar
- Department of Surgery, Hamad Medical Corporation, Doha, Qatar.
| |
Collapse
|
5
|
Long Y, Li C, Dou Q. Robotic surgery remote mentoring via AR with 3D scene streaming and hand interaction. COMPUTER METHODS IN BIOMECHANICS AND BIOMEDICAL ENGINEERING: IMAGING & VISUALIZATION 2022. [DOI: 10.1080/21681163.2022.2145498] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/15/2022]
Affiliation(s)
- Yonghao Long
- Department of Computer Science and Engineering, The Chinese University of Hong Kong, Hong Kong SAR, China
| | - Chengkun Li
- Department of Computer Science and Engineering, The Chinese University of Hong Kong, Hong Kong SAR, China
| | - Qi Dou
- Department of Computer Science and Engineering, The Chinese University of Hong Kong, Hong Kong SAR, China
| |
Collapse
|
6
|
Shabir D, Anbatawi M, Padhan J, Balakrishnan S, Al‐Ansari A, Abinahed J, Tsiamyrtzis P, Yaacoub E, Mohammed A, Deng Z, Navkar NV. Evaluation of user‐interfaces for controlling movements of virtual minimally invasive surgical instruments. Int J Med Robot 2022; 18:e2414. [DOI: 10.1002/rcs.2414] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/07/2022] [Revised: 04/10/2022] [Accepted: 04/27/2022] [Indexed: 11/10/2022]
Affiliation(s)
- Dehlela Shabir
- Department of Surgery Hamad Medical Corporation Doha Qatar
| | - Malek Anbatawi
- Department of Surgery Hamad Medical Corporation Doha Qatar
| | | | | | | | | | | | - Elias Yaacoub
- Department of Computer Science and Engineering Qatar University Doha Qatar
| | - Amr Mohammed
- Department of Computer Science and Engineering Qatar University Doha Qatar
| | - Zhigang Deng
- Department of Computer Science University of Houston Houston Texas USA
| | | |
Collapse
|
7
|
Mulita F, Verras GI, Anagnostopoulos CN, Kotis K. A Smarter Health through the Internet of Surgical Things. SENSORS (BASEL, SWITZERLAND) 2022; 22:s22124577. [PMID: 35746359 PMCID: PMC9231158 DOI: 10.3390/s22124577] [Citation(s) in RCA: 34] [Impact Index Per Article: 17.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/03/2022] [Revised: 06/10/2022] [Accepted: 06/14/2022] [Indexed: 05/14/2023]
Abstract
(1) Background: In the last few years, technological developments in the surgical field have been rapid and are continuously evolving. One of the most revolutionizing breakthroughs was the introduction of the IoT concept within surgical practice. Our systematic review aims to summarize the most important studies evaluating the IoT concept within surgical practice, focusing on Telesurgery and surgical Telementoring. (2) Methods: We conducted a systematic review of the current literature, focusing on the Internet of Surgical Things in Telesurgery and Telementoring. Forty-eight (48) studies were included in this review. As secondary research questions, we also included brief overviews of the use of IoT in image-guided surgery, and patient Telemonitoring, by systematically analyzing fourteen (14) and nineteen (19) studies, respectively. (3) Results: Data from 219 patients and 757 healthcare professionals were quantitively analyzed. Study designs were primarily observational or based on model development. Palpable advantages from the IoT incorporation mainly include less surgical hours, accessibility to high quality treatment, and safer and more effective surgical education. Despite the described technological advances, and proposed benefits of the systems presented, there are still identifiable gaps in the literature that need to be further explored in a systematic manner. (4) Conclusions: The use of the IoT concept within the surgery domain is a widely incorporated but less investigated concept. Advantages have become palpable over the past decade, yet further research is warranted.
Collapse
Affiliation(s)
- Francesk Mulita
- Intelligent Systems Lab, Department of Cultural Technology and Communication, University of the Aegean, 81100 Mytilene, Greece;
- Department of Surgery, General University Hospital of Patras, 26504 Rio, Greece;
- Correspondence: (F.M.); (K.K.); Tel.: +30-6974822712 (K.K.)
| | | | | | - Konstantinos Kotis
- Intelligent Systems Lab, Department of Cultural Technology and Communication, University of the Aegean, 81100 Mytilene, Greece;
- Correspondence: (F.M.); (K.K.); Tel.: +30-6974822712 (K.K.)
| |
Collapse
|
8
|
Shabir D, Abdurahiman N, Padhan J, Anbatawi M, Trinh M, Balakrishnan S, Al-Ansari A, Yaacoub E, Deng Z, Erbad A, Mohammed A, Navkar NV. Preliminary design and evaluation of a remote tele-mentoring system for minimally invasive surgery. Surg Endosc 2022; 36:3663-3674. [PMID: 35246742 PMCID: PMC9001542 DOI: 10.1007/s00464-022-09164-3] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/18/2021] [Accepted: 02/18/2022] [Indexed: 12/26/2022]
Abstract
BACKGROUND Tele-mentoring during surgery facilitates the transfer of surgical knowledge from a mentor (specialist surgeon) to a mentee (operating surgeon). The aim of this work is to develop a tele-mentoring system tailored for minimally invasive surgery (MIS) where the mentor can remotely demonstrate to the mentee the required motion of the surgical instruments. METHODS A remote tele-mentoring system is implemented that generates visual cues in the form of virtual surgical instrument motion overlaid onto the live view of the operative field. The technical performance of the system is evaluated in a simulated environment, where the operating room and the central location of the mentor were physically located in different countries and connected over the internet. In addition, a user study was performed to assess the system as a mentoring tool. RESULTS On average, it took 260 ms to send a view of the operative field of 1920 × 1080 resolution from the operating room to the central location of the mentor and an average of 132 ms to receive the motion of virtual surgical instruments from the central location to the operating room. The user study showed that it is feasible for the mentor to demonstrate and for the mentee to understand and replicate the motion of surgical instruments. CONCLUSION The work demonstrates the feasibility of transferring information over the internet from a mentor to a mentee in the form of virtual surgical instruments. Their motion is overlaid onto the live view of the operative field enabling real-time interactions between both the surgeons.
Collapse
Affiliation(s)
- Dehlela Shabir
- Department of Surgery, Surgical Research Section, Hamad General Hospital, Hamad Medical Corporation, PO Box 3050, Doha, Qatar
| | - Nihal Abdurahiman
- Department of Surgery, Surgical Research Section, Hamad General Hospital, Hamad Medical Corporation, PO Box 3050, Doha, Qatar
| | - Jhasketan Padhan
- Department of Surgery, Surgical Research Section, Hamad General Hospital, Hamad Medical Corporation, PO Box 3050, Doha, Qatar
| | - Malek Anbatawi
- Department of Surgery, Surgical Research Section, Hamad General Hospital, Hamad Medical Corporation, PO Box 3050, Doha, Qatar
| | - May Trinh
- Department of Computer Science, University of Houston, Houston, TX, USA
| | - Shidin Balakrishnan
- Department of Surgery, Surgical Research Section, Hamad General Hospital, Hamad Medical Corporation, PO Box 3050, Doha, Qatar
| | - Abdulla Al-Ansari
- Department of Surgery, Surgical Research Section, Hamad General Hospital, Hamad Medical Corporation, PO Box 3050, Doha, Qatar
| | - Elias Yaacoub
- Department of Computer Science and Engineering, Qatar University, Doha, Qatar
| | - Zhigang Deng
- Department of Computer Science, University of Houston, Houston, TX, USA
| | - Aiman Erbad
- College of Science and Engineering, Hamad Bin Khalifa University, Doha, Qatar
| | - Amr Mohammed
- Department of Computer Science and Engineering, Qatar University, Doha, Qatar
| | - Nikhil V Navkar
- Department of Surgery, Surgical Research Section, Hamad General Hospital, Hamad Medical Corporation, PO Box 3050, Doha, Qatar.
| |
Collapse
|