[Surgery 4.0-are we ready?]

Requirement analysis for an AI-based AR assistance system for surgical tools in the operating room: stakeholder requirements and technical perspectives

PURPOSE

We aim to investigate the integration of augmented reality (AR) within the context of increasingly complex surgical procedures and instrument handling toward the transition to smart operating rooms (OR). In contrast to cumbersome paper-based surgical instrument manuals still used in the OR, we wish to provide surgical staff with an AR head-mounted display that provides in-situ visualization and guidance throughout the assembly process of surgical instruments. Our requirement analysis supports the development and provides guidelines for its transfer into surgical practice.

METHODS

A three-phase user-centered design approach was applied with online interviews, an observational study, and a workshop with two focus groups with scrub nurses, circulating nurses, surgeons, manufacturers, clinic IT staff, and members of the sterilization department. The requirement analysis was based on key criteria for usability. The data were analyzed via structured content analysis.

RESULTS

We identified twelve main problems with the current use of paper manuals. Major issues included sterile users' inability to directly handle non-sterile manuals, missing details, and excessive text information, potentially delaying procedure performance. Major requirements for AR-driven guidance fall into the categories of design, practicability, control, and integration into the current workflow. Additionally, further recommendations for technical development could be obtained.

CONCLUSION

In conclusion, our insights have outlined a comprehensive spectrum of requirements that are essential for the successful implementation of an AI- and AR-driven guidance for assembling surgical instruments. The consistently appreciative evaluation by stakeholders underscores the profound potential of AR and AI technology as valuable assistance and guidance.

Hybrid Anomaly Detection in Time Series by Combining Kalman Filters and Machine Learning Models

Due to connectivity and automation trends, the medical device industry is experiencing increased demand for safety and security mechanisms. Anomaly detection has proven to be a valuable approach for ensuring safety and security in other industries, such as automotive or IT. Medical devices must operate across a wide range of values due to variations in patient anthropometric data, making anomaly detection based on a simple threshold for signal deviations impractical. For example, surgical robots directly contacting the patient's tissue require precise sensor data. However, since the deformation of the patient's body during interaction or movement is highly dependent on body mass, it is impossible to define a single threshold for implausible sensor data that applies to all patients. This also involves statistical methods, such as Z-score, that consider standard deviation. Even pure machine learning algorithms cannot be expected to provide the required accuracy simply due to the lack of available training data. This paper proposes using hybrid filters by combining dynamic system models based on expert knowledge and data-based models for anomaly detection in an operating room scenario. This approach can improve detection performance and explainability while reducing the computing resources needed on embedded devices, enabling a distributed approach to anomaly detection.

Threat Assessment and Risk Analysis (TARA) for Interoperable Medical Devices in the Operating Room Inspired by the Automotive Industry

Prevailing trends in the automotive and medical device industry, such as life cycle overarching configurability, connectivity, and automation, require an adaption of development processes, especially regarding the security and safety thereof. The changing requirements imply that interfaces are more exposed to the outside world, making them more vulnerable to cyberattacks or data leaks. Consequently, not only do development processes need to be revised but also cybersecurity countermeasures and a focus on safety, as well as privacy, have become vital. While vehicles are especially exposed to cybersecurity and safety risks, the medical devices industry faces similar issues. In the automotive industry, proposals and draft regulations exist for security-related risk assessment processes. The medical device industry, which has less experience in these topics and is more heterogeneous, may benefit from drawing inspiration from these efforts. We examined and compared current standards, processes, and methods in both the automotive and medical industries. Based on the requirements regarding safety and security for risk analysis in the medical device industry, we propose the adoption of methods already established in the automotive industry. Furthermore, we present an example based on an interoperable Operating Room table (OR table).

What’s new in artificially intelligent joint surgery in China? The minutes of the 2021 IEEE ICRA and literature review

Objective

To outline the main results of the 2021 International Conference on Robotics and Automation (ICRA 2021) of the Institute of Electrical and Electronics Engineers (IEEE) and review the advances in artificially intelligent joint surgery in China.

Methods

The keynote speeches of the 2021 ICRA were summarized in detail, and publications indexed by five core electronic databases (PubMed, Cochrane, Medline, Embase and CNKI) were systematically surveyed (cutoff date: July 30, 2021) in terms of the main topics of the conference. Publications directly related to artificially intelligent joint surgery in China were identified by using the search strategies of (robotic AND arthroplasty OR replacement), (navigation AND arthroplasty OR replacement), (artificial intelligent AND arthroplasty OR replacement), and (mixed reality AND arthroplasty OR replacement) and systemically reviewed.

Results

While robot-assisted arthroplasty in China is mainly performed using robots made in other countries (e.g., Mako from Stryker, USA), China’s domestic R&D of robots and clinical studies of robotic joint surgery have made some achievements. Although reports on the safety, effectiveness and clinical efficacy of China’s domestic robot-assisted joint surgery were presented at conferences, they have rarely been published in journals. Existing data indicate that, after the learning curve is overcome, robot-assisted hip and knee replacement surgery can fully achieve the established goals of precision and individualization and can significantly improve the accuracy of prosthesis placement angle and the recovery of the mechanical axis as compared with conventional surgery. The downside is that the low level of intelligentization and individualization means that existing designs are not conducive to personalization during surgery, resulting in low cost-effectiveness.

Conclusion

The safety and efficacy of domestic robot-assisted arthroplasty in China are well documented, and its accuracy and short-term clinical efficacy have been reported. However, the long-term clinical efficacy and the cost-effectiveness of large-scale clinical application of this technique warrants further study. The inadequacies of robot-assisted surgery should be remedied through the deep integration of medicine, engineering and the network.