A comprehensive operating room information system using the Kinect sensors and RFID

Contagious infection-free medical interaction system with machine vision controlled by remote hand gesture during an operation

Background and objective

Medical image visualization is a requirement in many types of surgery such as orthopaedic, spinal, thoracic procedures or tumour resection to eliminate risk such as "wrong level surgery". However, direct contact with physical devices such as mice or touch screens to control images is a challenge because of the potential risk of infection. To prevent the spread of infection in sterile environments, a contagious infection-free medical interaction system has been developed for manipulating medical images.

Methods

We proposed an integrated system with three key modules: hand landmark detection, hand pointing, and hand gesture recognition. A proposed depth enhancement algorithm is combined with a deep learning hand landmark detector to generate hand landmarks. Based on the designed system, a proposed hand-pointing system combined with projection and ray-pointing techniques allows for reducing fatigue during manipulation. A proposed landmark geometry constraint algorithm and deep learning method were applied to detect six gestures including click, open, close, zoom, drag, and rotation. Additionally, a control menu was developed to effectively activate common functions.

Results

The proposed hand-pointing system allowed for a large control range of up to 1200 mm in both vertical and horizontal direction. The proposed hand gesture recognition method showed high accuracy of over 97% and real-time response.

Conclusion

This paper described the contagious infection-free medical interaction system that enables precise and effective manipulation of medical images within the large control range, while minimizing hand fatigue.

Impact of a Task-Grabbing System for surgical technicians on operating room efficiency

The purpose of this study was to evaluate the effect of the Task-Grabbing System on operating room efficiency. Based on the competition-driven concept of the 'Uber' app, an Task-Grabbing System was designed for task allocation and quality assessment. We implemented the Task-Grabbing System in our hospital operating room and compared the differences in consecutive operation preparation time, turnover time, and task completion time performed by surgical technicians for tasks such as patient pick-up, operating room cleaning, medical equipment recovery, three-piece set delivery, as well as blood gas analysis and intraoperative specimen submission before (October 2019) and after (December 2019) the implementation of the Task-Grabbing System. After the implementation of the Task-Grabbing System, the consecutive operation preparation time was reduced from the average of 43.56-38.55 min (P < 0.05), and the turnover time was decreased from the average of 14.25-12.61 min (P < 0.05). And the respective time consuming of surgical technicians for patients picking up, operating room cleaning, medical facilities recovering, the three-piece set delivering, blood gas analysis sending and intraoperative specimen submitting was significantly shortened (P < 0.05). The Task-Grabbing System could improve the operating room efficiency and effectively mobilize the enthusiasm and initiative of the surgical technicians.

A marker-less human motion analysis system for motion-based biomarker identification and quantification in knee disorders

In recent years the healthcare industry has had increased difficulty seeing all low-risk patients, including but not limited to suspected osteoarthritis (OA) patients. To help address the increased waiting lists and shortages of staff, we propose a novel method of automated biomarker identification and quantification for the monitoring of treatment or disease progression through the analysis of clinical motion data captured from a standard RGB video camera. The proposed method allows for the measurement of biomechanics information and analysis of their clinical significance, in both a cheap and sensitive alternative to the traditional motion capture techniques. These methods and results validate the capabilities of standard RGB cameras in clinical environments to capture clinically relevant motion data. Our method focuses on generating 3D human shape and pose from 2D video data via adversarial training in a deep neural network with a self-attention mechanism to encode both spatial and temporal information. Biomarker identification using Principal Component Analysis (PCA) allows the production of representative features from motion data and uses these to generate a clinical report automatically. These new biomarkers can then be used to assess the success of treatment and track the progress of rehabilitation or to monitor the progression of the disease. These methods have been validated with a small clinical study, by administering a local anaesthetic to a small population with knee pain, this allows these new representative biomarkers to be validated as statistically significant (p -value < 0.05 ). These significant biomarkers include the cumulative acceleration of elbow flexion/extension in a sit-to-stand, as well as the smoothness of the knee and elbow flexion/extension in both a squat and sit-to-stand.

Systematic review of applied usability metrics within usability evaluation methods for hospital electronic healthcare record systems: Metrics and Evaluation Methods for eHealth Systems

BACKGROUND AND OBJECTIVES

Electronic healthcare records have become central to patient care. Evaluation of new systems include a variety of usability evaluation methods or usability metrics (often referred to interchangeably as usability components or usability attributes). This study reviews the breadth of usability evaluation methods, metrics, and associated measurement techniques that have been reported to assess systems designed for hospital staff to assess inpatient clinical condition.

METHODS

Following Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) methodology, we searched Medline, EMBASE, CINAHL, Cochrane Database of Systematic Reviews, and Open Grey from 1986 to 2019. For included studies, we recorded usability evaluation methods or usability metrics as appropriate, and any measurement techniques applied to illustrate these. We classified and described all usability evaluation methods, usability metrics, and measurement techniques. Study quality was evaluated using a modified Downs and Black checklist.

RESULTS

The search identified 1336 studies. After abstract screening, 130 full texts were reviewed. In the 51 included studies 11 distinct usability evaluation methods were identified. Within these usability evaluation methods, seven usability metrics were reported. The most common metrics were ISO9241-11 and Nielsen's components. An additional "usefulness" metric was reported in almost 40% of included studies. We identified 70 measurement techniques used to evaluate systems. Overall study quality was reflected in a mean modified Downs and Black checklist score of 6.8/10 (range 1-9) 33% studies classified as "high-quality" (scoring eight or higher), 51% studies "moderate-quality" (scoring 6-7), and the remaining 16% (scoring below five) were "low-quality."

CONCLUSION

There is little consistency within the field of electronic health record systems evaluation. This review highlights the variability within usability methods, metrics, and reporting. Standardized processes may improve evaluation and comparison electronic health record systems and improve their development and implementation.

The efficiency and effectiveness of surgery information systems in Iran

Background

Despite the prevalent use and advantages of information systems in hospitals, some have failed to meet their predefined objectives. Surgery information system (SIS) is a sub-system of a hospital information system. Its effective and efficient operation could enhance patient care in the busy environment of operating rooms with multiple tasks. The objective of this study was to evaluate the effectiveness and efficiency of SIS in three educational hospitals.

Methods

Data were collected using a questionnaire completed by 82 users of SIS. This questionnaire contains three parts: 1) participants’ demographic information, 2) questions regarding the efficiency of SIS, and 3) questions about its effectiveness. An independent sample t-test was used to compare the efficiency and effectiveness among systems. Chi-squared and Fisher tests were used to determine the relationship between the participants’ demographics and efficiency and effectiveness as well as the relationship between efficiency and effectiveness.

Results

About 23% of the participants rated the system’s efficiency as low, 29% as medium, and 48% as high. Besides, 24% of the participants considered the effectiveness of the system as low, 31% as medium, and 45% as high. There was a significant correlation between the efficiency and effectiveness of SIS (p ≤ 0.0001).

Conclusion

Based on the perspective of most participants (44%)the efficiency and effectiveness of both surgery information systems were acceptable. The results suggest that these systems should be designed in a way that facilitate user’s interaction and reduce the time takes to complete tasks. The results could be useful for developing and designing an efficient and effective system.

Use of Commercial Off-The-Shelf Devices for the Detection of Manual Gestures in Surgery: Systematic Literature Review

Background

The increasingly pervasive presence of technology in the operating room raises the need to study the interaction between the surgeon and computer system. A new generation of tools known as commercial off-the-shelf (COTS) devices enabling touchless gesture–based human-computer interaction is currently being explored as a solution in surgical environments.

Objective

The aim of this systematic literature review was to provide an account of the state of the art of COTS devices in the detection of manual gestures in surgery and to identify their use as a simulation tool for motor skills teaching in minimally invasive surgery (MIS).

Methods

For this systematic literature review, a search was conducted in PubMed, Excerpta Medica dataBASE, ScienceDirect, Espacenet, OpenGrey, and the Institute of Electrical and Electronics Engineers databases. Articles published between January 2000 and December 2017 on the use of COTS devices for gesture detection in surgical environments and in simulation for surgical skills learning in MIS were evaluated and selected.

Results

A total of 3180 studies were identified, 86 of which met the search selection criteria. Microsoft Kinect (Microsoft Corp) and the Leap Motion Controller (Leap Motion Inc) were the most widely used COTS devices. The most common intervention was image manipulation in surgical and interventional radiology environments, followed by interaction with virtual reality environments for educational or interventional purposes. The possibility of using this technology to develop portable low-cost simulators for skills learning in MIS was also examined. As most of the articles identified in this systematic review were proof-of-concept or prototype user testing and feasibility testing studies, we concluded that the field was still in the exploratory phase in areas requiring touchless manipulation within environments and settings that must adhere to asepsis and antisepsis protocols, such as angiography suites and operating rooms.

Conclusions

COTS devices applied to hand and instrument gesture–based interfaces in the field of simulation for skills learning and training in MIS could open up a promising field to achieve ubiquitous training and presurgical warm up.

Surgical data science: The new knowledge domain

Healthcare in general, and surgery/interventional care in particular, is evolving through rapid advances in technology and increasing complexity of care, with the goal of maximizing the quality and value of care. Whereas innovations in diagnostic and therapeutic technologies have driven past improvements in the quality of surgical care, future transformation in care will be enabled by data. Conventional methodologies, such as registry studies, are limited in their scope for discovery and research, extent and complexity of data, breadth of analytical techniques, and translation or integration of research findings into patient care. We foresee the emergence of surgical/interventional data science (SDS) as a key element to addressing these limitations and creating a sustainable path toward evidence-based improvement of interventional healthcare pathways. SDS will create tools to measure, model, and quantify the pathways or processes within the context of patient health states or outcomes and use information gained to inform healthcare decisions, guidelines, best practices, policy, and training, thereby improving the safety and quality of healthcare and its value. Data are pervasive throughout the surgical care pathway; thus, SDS can impact various aspects of care, including prevention, diagnosis, intervention, or postoperative recovery. The existing literature already provides preliminary results, suggesting how a data science approach to surgical decision-making could more accurately predict severe complications using complex data from preoperative, intraoperative, and postoperative contexts, how it could support intraoperative decision-making using both existing knowledge and continuous data streams throughout the surgical care pathway, and how it could enable effective collaboration between human care providers and intelligent technologies. In addition, SDS is poised to play a central role in surgical education, for example, through objective assessments, automated virtual coaching, and robot-assisted active learning of surgical skill. However, the potential for transforming surgical care and training through SDS may only be realized through a cultural shift that not only institutionalizes technology to seamlessly capture data but also assimilates individuals with expertise in data science into clinical research teams. Furthermore, collaboration with industry partners from the inception of the discovery process promotes optimal design of data products as well as their efficient translation and commercialization. As surgery continues to evolve through advances in technology that enhance delivery of care, SDS represents a new knowledge domain to engineer surgical care of the future.

Use of natural user interfaces for image navigation during laparoscopic surgery: initial experience

BACKGROUND

Surgical environments require special aseptic conditions for direct interaction with the preoperative images. We aim to test the feasibility of using a set of gesture control sensors combined with voice control to interact in a sterile manner with preoperative information and an integrated operating room (OR) during laparoscopic surgery.

MATERIAL AND METHODS

Two hepatectomies and two partial nephrectomies were performed by three experienced surgeons in a porcine model. The Kinect, Leap Motion, and MYO armband in combination with voice control were used as natural user interfaces (NUIs). After surgery, surgeons completed a questionnaire about their experience.

RESULTS

Surgeons required <10 min training with each NUI. They stated that NUIs improved the access to preoperative patient information and kept them more focused on the surgical site. The Kinect system was reported as the most physically demanding NUI and the MYO armband in combination with voice commands as the most intuitive and accurate. The need to release one of the laparoscopic instruments in order to use the NUIs was identified as the main limitation.

CONCLUSIONS

The presented NUIs are feasible to directly interact in a more intuitive and sterile manner with the preoperative images and the integrated OR functionalities during laparoscopic surgery.

Evaluation of Kinect 3D Sensor for Healthcare Imaging

Microsoft Kinect is a three-dimensional (3D) sensor originally designed for gaming that has received growing interest as a cost-effective and safe device for healthcare imaging. Recent applications of Kinect in health monitoring, screening, rehabilitation, assistance systems, and intervention support are reviewed here. The suitability of available technologies for healthcare imaging applications is assessed. The performance of Kinect I, based on structured light technology, is compared with that of the more recent Kinect II, which uses time-of-flight measurement, under conditions relevant to healthcare applications. The accuracy, precision, and resolution of 3D images generated with Kinect I and Kinect II are evaluated using flat cardboard models representing different skin colors (pale, medium, and dark) at distances ranging from 0.5 to 1.2 m and measurement angles of up to 75°. Both sensors demonstrated high accuracy (majority of measurements <2 mm) and precision (mean point to plane error <2 mm) at an average resolution of at least 390 points per cm². Kinect I is capable of imaging at shorter measurement distances, but Kinect II enables structures angled at over 60° to be evaluated. Kinect II showed significantly higher precision and Kinect I showed significantly higher resolution (both p < 0.001). The choice of object color can influence measurement range and precision. Although Kinect is not a medical imaging device, both sensor generations show performance adequate for a range of healthcare imaging applications. Kinect I is more appropriate for short-range imaging and Kinect II is more appropriate for imaging highly curved surfaces such as the face or breast.

Usability problems do not heal by themselves: National survey on physicians' experiences with EHRs in Finland

PURPOSE

Survey studies of health information systems use tend to focus on availability of functionalities, adoption and intensity of use. Usability surveys have not been systematically conducted by any healthcare professional groups on a national scale on a repeated basis. This paper presents results from two cross-sectional surveys of physicians' experiences with the usability of currently used EHR systems in Finland. The research questions were: To what extent has the overall situation improved between 2010 and 2014? What differences are there between healthcare sectors?

METHODS

In the spring of 2014, a survey was conducted in Finland using a questionnaire that measures usability and respondents' user experiences with electronic health record (EHR) systems. The survey was targeted to physicians who were actively doing clinical work. Twenty-four usability-related statements, that were identical in 2010 and 2014, were analysed from the survey. The respondents were also asked to give an overall rating of the EHR system they used. The study data comprised responses from 3081 physicians from the year 2014 and from 3223 physicians in the year 2010, who were using the nine most commonly used EHR system brands in Finland.

RESULTS

Physicians' assessments of the usability of their EHR system remain as critical as they were in 2010. On a scale from 1 ('fail') to 7 ('excellent') the average of overall ratings of their principally used EHR systems varied from 3.2 to 4.4 in 2014 (and in 2010 from 2.5 to 4.3). The results show some improvements in the following EHR functionalities and characteristics: summary view of patient's health status, prevention of errors associated with medication ordering, patient's medication list as well as support for collaboration and information exchange between the physician and the nurses. Even so, support for cross-organizational collaboration between physicians and for physician-patient collaboration were still considered inadequate. Satisfaction with technical features had not improved in four years. The results show marked differences between the EHR system brands as well as between healthcare sectors (private sector, public hospitals, primary healthcare). Compared to responses from the public sector, physicians working in the private sector were more satisfied with their EHR systems with regards to statements about user interface characteristics and support for routine tasks. Overall, the study findings are similar to our previous study conducted in 2010.

CONCLUSIONS

Surveys about the usability of EHR systems are needed to monitor their development at regional and national levels. To our knowledge, this study is the first national eHealth observatory questionnaire that focuses on usability and is used to monitor the long-term development of EHRs. The results do not show notable improvements in physician's ratings for their EHRs between the years 2010 and 2014 in Finland. Instead, the results indicate the existence of serious problems and deficiencies which considerably hinder the efficiency of EHR use and physician's routine work. The survey results call for considerable amount of development work in order to achieve the expected benefits of EHR systems and to avoid technology-induced errors which may endanger patient safety. The findings of repeated surveys can be used to inform healthcare providers, decision makers and politicians about the current state of EHR usability and differences between brands as well as for improvements of EHR usability. This survey will be repeated in 2017 and there is a plan to include other healthcare professional groups in future surveys.

Touchless interaction with software in interventional radiology and surgery: a systematic literature review

PURPOSE

In this article, we systematically examine the current state of research of systems that focus on touchless human-computer interaction in operating rooms and interventional radiology suites. We further discuss the drawbacks of current solutions and underline promising technologies for future development.

METHODS

A systematic literature search of scientific papers that deal with touchless control of medical software in the immediate environment of the operation room and interventional radiology suite was performed. This includes methods for touchless gesture interaction, voice control and eye tracking.

RESULTS

Fifty-five research papers were identified and analyzed in detail including 33 journal publications. Most of the identified literature (62 %) deals with the control of medical image viewers. The others present interaction techniques for laparoscopic assistance (13 %), telerobotic assistance and operating room control (9 % each) as well as for robotic operating room assistance and intraoperative registration (3.5 % each). Only 8 systems (14.5 %) were tested in a real clinical environment, and 7 (12.7 %) were not evaluated at all.

CONCLUSION

In the last 10 years, many advancements have led to robust touchless interaction approaches. However, only a few have been systematically evaluated in real operating room settings. Further research is required to cope with current limitations of touchless software interfaces in clinical environments. The main challenges for future research are the improvement and evaluation of usability and intuitiveness of touchless human-computer interaction and the full integration into productive systems as well as the reduction of necessary interaction steps and further development of hands-free interaction.

[A new human machine interface in neurosurgery: The Leap Motion(®). Technical note regarding a new touchless interface]

Currently, cross-sectional imaging viewing is used in routine practice whereas the surgical procedure requires physical contact with an interface (mouse or touch-sensitive screen). This type of contact results in a risk of lack of aseptic control and causes loss of time. The recent appearance of devices such as the Leap Motion(®) (Leap Motion society, San Francisco, USA) a sensor which enables to interact with the computer without any physical contact is of major interest in the field of surgery. However, its configuration and ergonomics produce key challenges in order to adapt to the practitioner's requirements, the imaging software as well as the surgical environment. This article aims to suggest an easy configuration of the Leap Motion(®) in neurosurgery on a PC for an optimized utilization with Carestream(®) Vue PACS v11.3.4 (Carestream Health, Inc., Rochester, USA) using a plug-in (to download at: https://drive.google.com/?usp=chrome_app#folders/0B_F4eBeBQc3ybElEeEhqME5DQkU) and a video tutorial (https://www.youtube.com/watch?v=yVPTgxg-SIk).