Robotic scrub nurse to anticipate surgical instruments based on real-time laparoscopic video analysis

BACKGROUND

Machine learning and robotics technologies are increasingly being used in the healthcare domain to improve the quality and efficiency of surgeries and to address challenges such as staff shortages. Robotic scrub nurses in particular offer great potential to address staff shortages by assuming nursing tasks such as the handover of surgical instruments.

METHODS

We introduce a robotic scrub nurse system designed to enhance the quality of surgeries and efficiency of surgical workflows by predicting and delivering the required surgical instruments based on real-time laparoscopic video analysis. We propose a three-stage deep learning architecture consisting of a single frame-, temporal multi frame-, and informed model to anticipate surgical instruments. The anticipation model was trained on a total of 62 laparoscopic cholecystectomies.

RESULTS

Here, we show that our prediction system can accurately anticipate 71.54% of the surgical instruments required during laparoscopic cholecystectomies in advance, facilitating a smoother surgical workflow and reducing the need for verbal communication. As the instruments in the left working trocar are changed less frequently and according to a standardized procedure, the prediction system works particularly well for this trocar.

CONCLUSIONS

The robotic scrub nurse thus acts as a mind reader and helps to mitigate staff shortages by taking over a great share of the workload during surgeries while additionally enabling an enhanced process standardization.

Versatile end effector for laparoscopic robotic scrub nurse

PURPOSE

Integrating robotic scrub nurses in the operating room has the potential to help overcome staff shortages and limited use of available operating capacities in hospitals. Existing approaches of robotic scrub nurses are mainly focused on open surgical procedures, neglecting laparoscopic procedures. Laparoscopic interventions offer great potential for the context-sensitive integration of robotic systems due to possible standardization. However, the first step is to ensure the safe manipulation of laparoscopic instruments.

METHODS

A robotic platform with a universal gripper system was designed to pick up and place laparoscopic as well as da Vinci[Formula: see text] instruments in an efficient workflow. The robustness of the gripper system was studied using a test protocol, which included a force absorption test to determine the operational safety limits of the design and a grip test to determine the system performance.

RESULTS

The test protocol shows results regarding force and torque absorption capabilities of the end effector, which are essential when transferring an instrument to the surgeon to enable a robust handover. The grip tests show that the laparoscopic instruments can be safely picked up, manipulated and returned independent of unexpected positional deviations. The gripper system also enables the manipulation of da Vinci[Formula: see text] instruments, opening the door for robot-robot interaction.

CONCLUSION

Our evaluation tests have shown that our robotic scrub nurse with the universal gripper system can safely and robustly manipulate laparoscopic and da Vinci[Formula: see text] instruments. The system design will continue with the integration of context-sensitive capabilities.

Automation of Intraoperative Tool Changing for Robot-Assisted Laparoscopic Surgery

Intraoperative tool change is a time-consuming and labor-intensive task for robot-assisted laparoscopic surgery. Serial multi-DOF manipulators are potential devices for realizing automatic intraoperative tool changes because of the layout flexibility and motion range, and multi-DOF makes it feasible for the manipulator to access and fetch the surgical tools by itself. However, the direction of the trocar may change because of the soft abdomen, and the lack of a fixed RCM makes it difficult for manipulators to reinsert a new surgical tool through the trocar. This study proposes a system prototype using a 7-DOF manipulator to automatically conduct the intraoperative tool-changing task. The newly designed surgical tool docking station facilitates surgical tool coupling/decoupling by rotating the manipulator’s end effector once. The proposed trocar recognition method with position error compensation is reliable for aligning a new surgical tool to the trocar port, even when the direction of the trocar is changed. The experimental results confirms that the manipulator can accomplish an intraoperative tool changing task without additional assistance or correction from the human.

Design and evaluation of a slave manipulator with roll-pitch-roll wrist and automatic tool loading mechanism in telerobotic surgery

BACKGROUND

As there is a shortage of scrub nurses in many hospitals, automatic surgical tool exchanging mechanism without human labour has been studied. Minimally invasive robotic surgeries (MIRS) also require scrub nurses. A surgical tool loading mechanism without a scrub nurse's assistance for MIRS is proposed. Many researchers have developed minimally invasive surgical instruments with a wrist joint that can be movable inside the abdomen. However, implementation of a distal rolling joint on a gripper is rare.

METHODS

To implement surgical tool exchanging without a scrub nurse's assistance, a slave manipulator and a tool loader were developed to load and unload a surgical tool unit. A surgical tool unit with a roll-pitch-roll wrist was developed. Several experiments were performed to validate the effectiveness of the slave manipulator and the surgical tool unit.

RESULTS

The slave manipulator and the tool loader were able to successfully unload and load the surgical tool unit without human assistance. The total duration of unloading and loading the surgical tool unit was 97 s. Motion tracking experiments of the distal rolling joint were performed. The maximum positioning error of the step input response was 2°.

CONCLUSIONS

The advantage of the proposed slave manipulator and tool loader is that other robotic systems or human labour are not needed for surgical tool loading. The feasibility of the distal rolling joint in MIS is verified.