Technologies Used for Telementoring in Open Surgery: A Scoping Review

Background: Telementoring technologies enable a remote mentor to guide a mentee in real-time during surgical procedures. This addresses challenges, such as lack of expertise and limited surgical training/education opportunities in remote locations. This review aims to provide a comprehensive account of these technologies tailored for open surgery. Methods: A comprehensive scoping review of the scientific literature was conducted using PubMed, ScienceDirect, ACM Digital Library, and IEEE Xplore databases. Broad and inclusive searches were done to identify articles reporting telementoring or teleguidance technologies in open surgery. Results: Screening of the search results yielded 43 articles describing surgical telementoring for open approach. The studies were categorized based on the type of open surgery (surgical specialty, surgical procedure, and stage of clinical trial), the telementoring technology used (information transferred between mentor and mentee, devices used for rendering the information), and assessment of the technology (experience level of mentor and mentee, study design, and assessment criteria). Majority of the telementoring technologies focused on trauma-related surgeries and mixed reality headsets were commonly used for rendering information (telestrations, surgical tools, or hand gestures) to the mentee. These technologies were primarily assessed on high-fidelity synthetic phantoms. Conclusions: Despite longer operative time, these telementoring technologies demonstrated clinical viability during open surgeries through improved performance and confidence of the mentee. In general, usage of immersive devices and annotations appears to be promising, although further clinical trials will be required to thoroughly assess its benefits.

Lessons from the use of telemedicine in the austere military environment and the implications for deployed surgical teams

Over the last 20 years, there have been significant changes in UK surgical training. Civilian surgical training may no longer prepare military surgeons for the range of skills they require on operations. One method to address gaps in knowledge or experience is to use telemedicine to facilitate specialist consultations from UK-based specialists to deployed medical teams. In the UK Defence Medical Services (DMS), this capability is called real-time clinical support (RTCS). RTCS provides a direct audio-visual link from a deployed location anywhere in the world to a supporting medical specialist in the UK. RTCS is currently delivered via a combination of off-the-shelf hardware and commercially available software. This article will outline the current use of RTCS, with emphasis on deployed surgical teams in austere environments, and discuss the advantages and limitations of this capability. However, it must be emphasised that no technology can be a substitute for clinical training and experience. Although several limitations remain, the authors believe that RTCS offers potential benefits for the DMS and could be an important tool aiding deployed clinicians. It can also be argued that by engaging with the concept now, the DMS can shape future developments in this sphere.

Mixed Reality Platforms in Telehealth Delivery: Scoping Review

BACKGROUND

The distinctive features of the digital reality platforms, namely augmented reality (AR), virtual reality (VR), and mixed reality (MR) have extended to medical education, training, simulation, and patient care. Furthermore, this digital reality technology seamlessly merges with information and communication technology creating an enriched telehealth ecosystem. This review provides a composite overview of the prospects of telehealth delivered using the MR platform in clinical settings.

OBJECTIVE

This review identifies various clinical applications of high-fidelity digital display technology, namely AR, VR, and MR, delivered using telehealth capabilities. Next, the review focuses on the technical characteristics, hardware, and software technologies used in the composition of AR, VR, and MR in telehealth.

METHODS

We conducted a scoping review using the methodological framework and reporting design using the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews) guidelines. Full-length articles in English were obtained from the Embase, PubMed, and Web of Science databases. The search protocol was based on the following keywords and Medical Subject Headings to obtain relevant results: "augmented reality," "virtual reality," "mixed-reality," "telemedicine," "telehealth," and "digital health." A predefined inclusion-exclusion criterion was developed in filtering the obtained results and the final selection of the articles, followed by data extraction and construction of the review.

RESULTS

We identified 4407 articles, of which 320 were eligible for full-text screening. A total of 134 full-text articles were included in the review. Telerehabilitation, telementoring, teleconsultation, telemonitoring, telepsychiatry, telesurgery, and telediagnosis were the segments of the telehealth division that explored the use of AR, VR, and MR platforms. Telerehabilitation using VR was the most commonly recurring segment in the included studies. AR and MR has been mainly used for telementoring and teleconsultation. The most important technical features of digital reality technology to emerge with telehealth were virtual environment, exergaming, 3D avatars, telepresence, anchoring annotations, and first-person viewpoint. Different arrangements of technology-3D modeling and viewing tools, communication and streaming platforms, file transfer and sharing platforms, sensors, high-fidelity displays, and controllers-formed the basis of most systems.

CONCLUSIONS

This review constitutes a recent overview of the evolving digital AR and VR in various clinical applications using the telehealth setup. This combination of telehealth with AR, VR, and MR allows for remote facilitation of clinical expertise and further development of home-based treatment. This review explores the rapidly growing suite of technologies available to users within the digital health sector and examines the opportunities and challenges they present.

Towards Wearable Augmented Reality in Healthcare: A Comparative Survey and Analysis of Head-Mounted Displays

Head-mounted displays (HMDs) have the potential to greatly impact the surgical field by maintaining sterile conditions in healthcare environments. Google Glass (GG) and Microsoft HoloLens (MH) are examples of optical HMDs. In this comparative survey related to wearable augmented reality (AR) technology in the medical field, we examine the current developments in wearable AR technology, as well as the medical aspects, with a specific emphasis on smart glasses and HoloLens. The authors searched recent articles (between 2017 and 2022) in the PubMed, Web of Science, Scopus, and ScienceDirect databases and a total of 37 relevant studies were considered for this analysis. The selected studies were divided into two main groups; 15 of the studies (around 41%) focused on smart glasses (e.g., Google Glass) and 22 (59%) focused on Microsoft HoloLens. Google Glass was used in various surgical specialities and preoperative settings, namely dermatology visits and nursing skill training. Moreover, Microsoft HoloLens was used in telepresence applications and holographic navigation of shoulder and gait impairment rehabilitation, among others. However, some limitations were associated with their use, such as low battery life, limited memory size, and possible ocular pain. Promising results were obtained by different studies regarding the feasibility, usability, and acceptability of using both Google Glass and Microsoft HoloLens in patient-centric settings as well as medical education and training. Further work and development of rigorous research designs are required to evaluate the efficacy and cost-effectiveness of wearable AR devices in the future.

Remote Training for Medical Staff in Low-Resource Environments Using Augmented Reality

This work aims to leverage medical augmented reality (AR) technology to counter the shortage of medical experts in low-resource environments. We present a complete and cross-platform proof-of-concept AR system that enables remote users to teach and train medical procedures without expensive medical equipment or external sensors. By seeing the 3D viewpoint and head movements of the teacher, the student can follow the teacher's actions on the real patient. Alternatively, it is possible to stream the 3D view of the patient from the student to the teacher, allowing the teacher to guide the student during the remote session. A pilot study of our system shows that it is easy to transfer detailed instructions through this remote teaching system and that the interface is easily accessible and intuitive for users. We provide a performant pipeline that synchronizes, compresses, and streams sensor data through parallel efficiency.

‘Burr holes in the bush’: Clinician preparedness for undertaking emergency intracranial haematoma evacuation surgery in rural and regional Queensland

OBJECTIVES

Delayed inter-hospital transfers of deteriorating neurotrauma patients from rural and regional hospitals to tertiary centres have seen the need for non-neurosurgeons to undertake emergency intracranial haematoma evacuation surgery locally. In the present study, the authors contributed to the paucity in the literature regarding the widespread availability of cranial access equipment in non-tertiary centres and patient outcomes in Queensland.

METHODS

We surveyed delegates (senior theatre nurses or surgical service directors) from rural and regional Queensland hospitals if they were located outside the local catchment of a tertiary centre and had a CT scanner. Questions regarded availability, location and storage conditions of mechanical cranial access kits, as well as last usage, and associated patient outcomes.

RESULTS

Twenty-six delegates from eligible hospitals responded. Eighteen hospitals offered surgical services. Eleven hospitals housed complete mechanical cranial access kits. Five hospitals housed incomplete kits. Thirteen hospitals housed their equipment sterile in the operating theatre or ED. Eleven hospitals reported using the equipment, with last usage ranging from 4 months to over 30 years. Two hospitals reported using the equipment within 12 months while a further five reported using it within 10 years. Two hospitals reported 'good' outcomes, two 'ok' and one 'poor'.

CONCLUSIONS

The availability of cranial access equipment outside Queensland tertiary centres has been limited. Inter-hospital transfers are likely to persist in Queensland and haematoma evacuation surgery has been a life-saving endeavour, so improving access to cranial access equipment in hospitals where it is currently lacking is highly warranted.

Virtual reality and haptic interfaces for civilian and military open trauma surgery training: A systematic review

OBJECTIVE

Virtual (VR), augmented (AR), mixed reality (MR) and haptic interfaces make additional avenues available for surgeon assessment, guidance and training. We evaluated applications for open trauma and emergency surgery to address the question: Have new computer-supported interface developments occurred that could improve trauma training for civilian and military surgeons performing open, emergency, non-laparoscopic surgery?

DESIGN

Systematic literature review.

SETTING AND PARTICIPANTS

Faculty, University of Maryland School of Medicine, Baltimore., Maryland; Womack Army Medical Center, Fort Bragg, North Carolina; Temple University, Philadelphia, Pennsylvania; Uniformed Services University of Health Sciences, and Walter Reed National Military Medical Center, Bethesda, Maryland.

METHODS

Structured literature searches identified studies using terms for virtual, augmented, mixed reality and haptics, as well as specific procedures in trauma training courses. Reporting bias was assessed. Study quality was evaluated by the Kirkpatrick's Level of evidence and the Machine Learning to Asses Surgical Expertise (MLASE) score.

RESULTS

Of 422 papers identified, 14 met inclusion criteria, included 282 enrolled subjects, 20% were surgeons, the remainder students, medics and non-surgeon physicians. Study design was poor and sample sizes were low. No data analyses were beyond descriptive and the highest outcome types were procedural success, subjective self-reports, except three studies used validated metrics. Among the 14 studies, Kirkpatrick's level of evidence was level zero in five studies, level 1 in 8 and level 2 in one. Only one study had MLASE Score greater than 9/20. There was a high risk of bias in 6 studies, uncertain bias in 5 studies and low risk of bias in 3 studies.

CONCLUSIONS

There was inadequate evidence that VR,MR,AR or haptic interfaces can facilitate training for open trauma surgery or replace cadavers. Because of limited testing in surgeons, deficient study and technology design, risk of reporting bias, no current well-designed studies of computer-supported technologies have shown benefit for open trauma, emergency surgery nor has their use shown improved patient outcomes. Larger more rigorously designed studies and evaluations by experienced surgeons are required for a greater variety of procedures and skills.

COMPETENCIES

Medical Knowledge, Practice Based Learning and Improvement, Patient Care, Systems-Based Practice.

Attitudes Toward Neurosurgery Education for the Nonneurosurgeon: A Survey Study and Critical Analysis of U.S. Military Training Techniques and Future Prospects

BACKGROUND

The U.S. military requires medical readiness to support forward-deployed combat operations. Because time and distance to neurosurgical capabilities vary within the deployed trauma system, nonneurosurgeons are required to perform emergent cranial procedures in select cases. It is unclear whether these surgeons have sufficient training in these procedures.

METHODS

This quality-improvement study involved a voluntary, anonymized specialty-specific survey of active-duty surgeons about their experience and attitudes toward U.S. military emergency neurosurgical training.

RESULTS

Survey responses were received from 104 general surgeons and 26 neurosurgeons. Among general surgeons, 81% have deployed and 53% received training in emergency neurosurgical procedures before deployment. Only 16% of general surgeons reported participating in craniotomy/craniectomy procedures in the last year. Nine general surgeons reported performing an emergency neurosurgical procedure while on deployment/humanitarian mission, and 87% of respondents expressed interest in further predeployment emergency neurosurgery training. Among neurosurgeons, 81% had participated in training nonneurosurgeons and 73% believe that more comprehensive training for nonneurosurgeons before deployment is needed. General surgeons proposed lower procedure minimums for competency for external ventricular drain placement and craniotomy/craniectomy than did neurosurgeons. Only 37% of general surgeons had used mixed/augmented reality in any capacity previously; for combat procedures, most (90%) would prefer using synchronous supervision via high-fidelity video teleconferencing over mixed reality.

CONCLUSIONS

These survey results show a gap in readiness for neurosurgical procedures for forward-deployed general surgeons. Capitalizing on capabilities such as mixed/augmented reality would be a force multiplier and a potential means of improving neurosurgical capabilities in the forward-deployed environments.

Mixed reality surgical mentoring of combat casualty care related procedures in a perfused cadaver model: Initial results of a randomized feasibility study

BACKGROUND

Most telemedicine modalities have limited ability to enhance procedural and operative care. We developed a novel system to provide synchronous bidirectional expert mixed reality-enabled virtual procedural mentoring. In this feasibility study, we evaluated mixed reality mentoring of combat casualty care related procedures in a re-perfused cadaver model.

METHODS

Novices received real-time holographic mentoring from experts using augmented reality via Hololens (Microsoft Inc, Redmond, WA). The experts maintained real-time awareness of the novice's operative environment using virtual reality via HTC-Vive (HTC Corp, Xindian District, Taiwan). Additional cameras (both environments) and novel software created the immersive, shared, 3-dimensional mixed reality environment in which the novice and expert collaborated. The novices were prospectively randomized to either mixed reality or audio-only mentoring. Blinded experts independently evaluated novice procedural videos using a 5-point Likert scale-based questionnaire. Nonparametric variables were evaluated using the Wilcoxon rank-sum test and comparisons using the χ² analysis; significance was defined at P < .05.

RESULTS

Surgeon and nonsurgeon novices (14) performed 69 combat casualty care-related procedures (38 mixed reality, 31 audio), including various vascular exposures, 4-compartment lower leg fasciotomy, and emergency neurosurgical procedures; 85% were performed correctly with no difference in either group. Upon video review, mixed reality-mentored novices showed no difference in procedural flow and forward planning (3.67 vs 3.28, P = .21) or the likelihood of performing individual procedural steps correctly (4.12 vs 3.59, P = .06).

CONCLUSION

In this initial feasibility study, our novel mixed reality-based mentoring system successfully facilitated the performance of a wide variety of combat casualty care relevant procedures using a high fidelity re-perfused cadaver model. The small sample size and limited variety of novice types likely impacted the ability of holographically mentored novices to demonstrate improvement over the audio-only control group. Despite this, using virtual, augmented, and mixed reality technologies for procedural mentoring demonstrated promise, and further study is needed.

AR-AI Tools as a Response to High Employee Turnover and Shortages in Manufacturing during Regular, Pandemic, and War Times

The world faces the continuously increasing issue of a lack of skilled employees, staff migration, and turnover. It is strengthened by unexpected situations such as wars, pandemics, and other civilization crises. Solutions are sought and researched in various branches of industry and academia, including engineering, social sciences, management, and political and computer sciences. From the viewpoint of this paper, this is a side topic of Industry 4.0 and, more specifically, sustainability in working environments, and the issue is related to production employees who perform manual operations. Some of the tasks cannot be carried out under robotization or automation; therefore, novel human-work support tools are expected. This paper presents such highly demanded support tools related to augmented reality (AR) and artificial intelligence (AI). First, a panoramic literature review is given. Secondly, the authors explain the main objective of the presented contribution. Then the authors’ achievements are described—the R&D focus on such solutions and the introduction of the developed tools that are based on AR and AI. Benefits connected to the AR-AI technology applications are presented in terms of both time savings with the tool usage and job simplification, enabling inexperienced, unskilled, or less skilled employees to perform the work in the selected manual production processes.

Augmented-reality telementoring for leg fasciotomy: a proof-of-concept study

BACKGROUND

Prolonged field care is required when casualty evacuation to a surgical facility is delayed by distance, weather or tactical constraints. This situation can occur in both civilian and military environments. In these circumstances, there are no established treatment options for extremity compartment syndrome. Telementoring by a surgeon may enable the local clinician to perform a fasciotomy to decompress the affected compartments.

METHODS

Six military clinicians were asked to perform a two-incision leg fasciotomy in synthetic models under the guidance of an orthopaedic surgeon located 380 km away. Communication occurred through commercially available software and smartglasses, which also allowed the surgeon to send augmented-reality graphics to the operators. Two blinded surgeons evaluated the specimens according to objective criteria. Control specimens were added to ensure the integrity of the evaluation process.

RESULTS

The six study participants were military physician assistants who had extensive clinical experience but had never performed a fasciotomy. The average duration of the procedure was 53 min. All six procedures were completed without major errors: release of all four compartments was achieved through full-length incisions in the skin and fascia. The only surgical complication was a laceration of the saphenous vein. All three control specimens were correctly assessed by the evaluators. None of the participants experienced adverse effects from wearing the smartglasses. Four dropped calls occurred, but the connection was re-established in all cases.

CONCLUSION

All six surgical procedures were completed successfully. We attribute the dropped calls to a mismatch between the size of the graphic files and the available bandwidth. A better technical understanding of the software by the mentoring surgeon would have avoided this problem. Important considerations for future research and practice include protocols for dropped communications, surgical skills training for the operators and communication training for the surgeons.

Augmented reality visualization tool for the future of tactical combat casualty care

ABSTRACT

The objective of this project was to identify and develop software for an augmented reality application that runs on the US Army Integrated Visual Augmentation System (IVAS) to support a medical caregiver during tactical combat casualty care scenarios. In this augmented reality tactical combat casualty care application, human anatomy of individual soldiers obtained predeployment is superimposed on the view of an injured war fighter through the IVAS. This offers insight into the anatomy of the injured war fighter to advance treatment in austere environments.In this article, we describe various software components required for an augmented reality tactical combat casualty care tool. These include a body pose tracking system to track the patient's body pose, a virtual rendering of a human anatomy avatar, speech input to control the application and rendering techniques to visualize the virtual anatomy, and treatment information on the augmented reality display. We then implemented speech commands and visualization for four common medical scenarios including injury of a limb, a blast to the pelvis, cricothyrotomy, and a pneumothorax on the Microsoft HoloLens 1 (Microsoft, Redmond, WA).The software is designed for a forward surgical care tool on the US Army IVAS, with the intention to provide the medical caregiver with a unique ability to quickly assess affected internal anatomy. The current software components still had some limitations with respect to speech recognition reliability during noise and body pose tracking. These will likely be improved with the improved hardware of the IVAS, which is based on a modified HoloLens 2.

Advanced Digital Health Technologies for COVID-19 and Future Emergencies

Background: Coronavirus disease 2019 (COVID-19) has led to a national health care emergency in the United States and exposed resource shortages, particularly of health care providers trained to provide critical or intensive care. This article describes how digital health technologies are being or could be used for COVID-19 mitigation. It then proposes the National Emergency Tele-Critical Care Network (NETCCN), which would combine digital health technologies to address this and future crises. Methods: Subject matter experts from the Society of Critical Care Medicine and the Telemedicine and Advanced Technology Research Center examined the peer-reviewed literature and science/technology news to see what digital health technologies have already been or could be implemented to (1) support patients while limiting COVID-19 transmission, (2) increase health care providers' capability and capacity, and (3) predict/prevent future outbreaks. Results: Major technologies identified included telemedicine and mobile care (for COVID-19 as well as routine care), tiered telementoring, telecritical care, robotics, and artificial intelligence for monitoring. Several of these could be assimilated to form an interoperable scalable NETCCN. NETCCN would assist health care providers, wherever they are located, by obtaining real-time patient and supplies data and disseminating critical care expertise. NETCCN capabilities should be maintained between disasters and regularly tested to ensure continual readiness. Conclusions: COVID-19 has demonstrated the impact of a large-scale health emergency on the existing infrastructures. Short term, an approach to meeting this challenge is to adopt existing digital health technologies. Long term, developing a NETCCN may ensure that the necessary ecosystem is available to respond to future emergencies.