Implementation of the Operating Room Black Box Research Program at the Ottawa Hospital Through Patient, Clinical, and Organizational Engagement: Case Study

Systematic review on the frequency and quality of reporting patient and public involvement in patient safety research

BACKGROUND

In recent years, patient and public involvement (PPI) in research has significantly increased; however, the reporting of PPI remains poor. The Guidance for Reporting Involvement of Patients and the Public (GRIPP2) was developed to enhance the quality and consistency of PPI reporting. The objective of this systematic review is to identify the frequency and quality of PPI reporting in patient safety (PS) research using the GRIPP2 checklist.

METHODS

Searches were performed in Ovid MEDLINE, EMBASE, PsycINFO, and CINAHL from 2018 to December, 2023. Studies on PPI in PS research were included. We included empirical qualitative, quantitative, mixed methods, and case studies. Only articles published in peer-reviewed journals in English were included. The quality of PPI reporting was assessed using the short form of the (GRIPP2-SF) checklist.

RESULTS

A total of 8561 studies were retrieved from database searches, updates, and reference checks, of which 82 met the eligibility criteria and were included in this review. Major PS topics were related to medication safety, general PS, and fall prevention. Patient representatives, advocates, patient advisory groups, patients, service users, and health consumers were the most involved. The main involvement across the studies was in commenting on or developing research materials. Only 6.1% (n = 5) of the studies reported PPI as per the GRIPP2 checklist. Regarding the quality of reporting following the GRIPP2-SF criteria, our findings show sub-optimal reporting mainly due to failures in: critically reflecting on PPI in the study; reporting the aim of PPI in the study; and reporting the extent to which PPI influenced the study overall.

CONCLUSIONS

Our review shows a low frequency of PPI reporting in PS research using the GRIPP2 checklist. Furthermore, it reveals a sub-optimal quality in PPI reporting following GRIPP2-SF items. Researchers, funders, publishers, and journals need to promote consistent and transparent PPI reporting following internationally developed reporting guidelines such as the GRIPP2. Evidence-based guidelines for reporting PPI should be encouraged and supported as it helps future researchers to plan and report PPI more effectively.

TRIAL REGISTRATION

The review protocol is registered with PROSPERO (CRD42023450715).

Patient engagement in health implementation research: A logic model

INTRODUCTION

Growing evidence supports patient engagement (PE) in health implementation research to improve the quality, relevance and uptake of research. However, more guidance is needed to plan and operationalize PE before and throughout the research process. The aim of the study was to develop a logic model illustrating the causal links between context, resources, activities, outcomes and impact of PE in an implementation research programme.

METHODS

The Patient Engagement in Health Implementation Research Logic Model (hereafter the Logic Model) was developed using a descriptive qualitative design with a participatory approach, in the context of the PriCARE programme. This programme aims to implement and evaluate case management for individuals who frequently use healthcare services in primary care clinics across five Canadian provinces. Participant observation of team meetings was performed by all team members involved in the programme and in-depth interviews were conducted by two external research assistants with team members (n = 22). A deductive thematic analysis using components of logic models as coding categories was conducted. Data were pooled in the first version of the Logic Model, which was refined in research team meetings with patient partners. The final version was validated by all team members.

RESULTS

The Logic Model highlights the importance of integrating PE into the project before its commencement, with appropriate support in terms of funding and time allocation. The governance structure and leadership of both principal investigators and patient partners have significant effects on PE activities and outcomes. As an empirical and standardized illustration that facilitates a shared understanding, the Logic Model provides guidance for maximizing the impact of patient partnership in various contexts for research, patients, providers and health care.

CONCLUSION

The Logic Model will help academic researchers, decision makers and patient partners plan, operationalize, and assess PE in implementation research for optimal outcomes.

PATIENT OR PUBLIC CONTRIBUTION

Patient partners from the PriCARE research programme contributed to developing the research objectives and designing, developing and validating data collection tools, producing data, developing and validating the Logic Model and reviewing the manuscript.

Analyzing interprofessional teamwork in the operating room: An exploratory observational study using conventional and alternative approaches

Intraoperative teamwork is vital for patient safety. Conventional tools for studying intraoperative teamwork typically rely on behaviorally anchored rating scales applied at the individual or team level, while others capture narrative information across several units of analysis. This prospective observational study characterizes teamwork using two conventional tools (Operating Theatre Team Non-Technical Skills Assessment Tool [NOTECHS]; Team Emergency Assessment Measure [TEAM]), and one alternative approach (modified-Systems Engineering Initiative for Patient Safety [SEIPS] model). We aimed to explore the advantages and disadvantages of each for providing feedback to improve teamwork practice. Fifty consecutive surgical cases at a Canadian academic hospital were recorded with the OR Black Box®, analyzed by trained raters, and summarized descriptively. Teamwork performance was consistently high within and across cases rated with NOTECHS and TEAMS. For cases analyzed with the modified-SEIPS tool, both optimal and suboptimal teamwork behaviors were identified, and team resilience was frequently observed. NOTECHS and TEAM provided summative assessments and overall pattern descriptions, while SEIPS facilitated a deeper understanding of teamwork processes. As healthcare organizations continue to prioritize teamwork improvement, SEIPS may provide valuable insights regarding teamwork behavior and the broader context influencing performance. This may ultimately enhance the development and effectiveness of multi-level teamwork interventions.

Implementation of an Innovative Technology Called the OR Black Box: A Feasibility Study

Introduction. The operating room (OR) Black Box is an innovative technology that captures and compiles extensive real-time data from the OR, allowing identification and analysis of factors that influence intraoperative procedures and performances - ultimately improving patient safety. Implementation of this kind of technology is still an emerging research area and prone to face challenges. Methods. Observational study running from May 2017 to May 2021 conducted at Copenhagen University Hospital - Rigshospitalet, Denmark, involving 152 OR staff and 306 patients. Feasibility of the OR Black Box was assessed in accordance with Bowen's framework with 8 focus areas. Results. The OR Black Box had a high level of acceptability among stakeholders with 100% participation from management, 93% from OR staff, and 98% from patients. The implementation process improved over time, and an average of 80% of the surgeries conducted were captured. The practical aspects such as numerous formal and informal meetings, ethical and legal approval, recruitment of patients were acceptable, albeit time-consuming. The OR Black Box was adopted without any changes in scheduled surgery program, but capturing hours were adjusted to match the surgery program and relocation of OR staff declining to provide consent was possible. Conclusions. Implementation of the OR Black Box was feasible yet challenging. Management, nearly all staff, and patients embraced the initiative; however, ongoing evaluation, information meetings, and commitment from stakeholders are required and crucial to sustain momentum, continue implementation and expansion. Ideas from this study can be useful in the implementation of similar initiatives.

Creating a Practical Transformational Change Management Model for Novel Artificial Intelligence-Enabled Technology Implementation in the Operating Room

Objective

To identify change management (CM) strategies for implementing novel artificial intelligence and similar novel technologies in operating rooms and create a new CM model for future trials and applications inspired by the abovementioned strategies and established models.

Methods

Key phases of technology implementation were defined, and strategies for transformational CM were created and applied in a recent CM experience at our institution between October 15, 2020 and October 15, 2021. We appraised existing CM models and propose the newly created model.

Results

The key phases of the technology implementation were as follows: (1) team assembly; (2) committee approvals; (3) CM; and (4) system installation and go-live. Key strategies were (1) assemble team with necessary expertise; (2) anticipate potential institutional cultural and regulatory hurdles; (3) add agility to project planning and execution; (4) accommodate institutional culture and regulations; (5) early clinical partner buy-in and stakeholder engagement; and (6) consistent communication, all of which contributed to the new CM model creation.

Conclusion

Key CM strategies and a new CM model addressing the unique needs and characteristics of operating room novel technology implementation were identified and created. The new model may be customized and tested for individual institution and project’s needs and characteristics.

Multi-Modal Deep Learning for Assessing Surgeon Technical Skill

This paper introduces a new dataset of a surgical knot-tying task, and a multi-modal deep learning model that achieves comparable performance to expert human raters on this skill assessment task. Seventy-two surgical trainees and faculty were recruited for the knot-tying task, and were recorded using video, kinematic, and image data. Three expert human raters conducted the skills assessment using the Objective Structured Assessment of Technical Skill (OSATS) Global Rating Scale (GRS). We also designed and developed three deep learning models: a ResNet-based image model, a ResNet-LSTM kinematic model, and a multi-modal model leveraging the image and time-series kinematic data. All three models demonstrate performance comparable to the expert human raters on most GRS domains. The multi-modal model demonstrates the best overall performance, as measured using the mean squared error (MSE) and intraclass correlation coefficient (ICC). This work is significant since it demonstrates that multi-modal deep learning has the potential to replicate human raters on a challenging human-performed knot-tying task. The study demonstrates an algorithm with state-of-the-art performance in surgical skill assessment. As objective assessment of technical skill continues to be a growing, but resource-heavy, element of surgical education, this study is an important step towards automated surgical skill assessment, ultimately leading to reduced burden on training faculty and institutes.

Bringing the patient voice into the operating room: engaging patients in surgical safety research with the Operating Room Black Box®

BACKGROUND

Surgery is one of the most common patient experiences in the health care system. Yet, efforts to engage patients in surgical safety research have not matched those of other health care fields. This is a critical issue given the nature of surgery inhibits patients' abilities to advocate for themselves as they are typically under anesthetic when the procedure is performed. We partnered with patients throughout our research program, which uses the Operating Room Black Box^® to enhance surgical patient safety through transparent and proactive analysis of human factors to detect and prevent avoidable errors.

MAIN BODY

In this article, we outline the need for, and our approach to, patient engagement in surgical safety research. Our approach included a series of planned activities and skill development opportunities designed to build capacity and bring together patients, clinicians, and researchers to inform research and practice. We also conducted evaluation surveys during the first year of our program, which have indicated a positive experience by both patient partners and the research team.

CONCLUSION

We believe our approach can serve as an important first step toward building a model for patient engagement in the surgical safety field and could significantly contribute to improved quality of care and outcomes for surgical patients.

Facilitating and supporting the engagement of patients, families and caregivers in research: the "Ottawa model" for patient engagement in research

BACKGROUND

Patient engagement is increasingly being recognized as a critical component of health research; however, institutional models for building infrastructure and capacity for patient engagement in research are limited. There is an opportunity to create reproducible and scalable models of patient engagement in research and share best and promising practices.

MAIN BODY

In this article, we describe the development and features of the framework for the Ottawa Patient Engagement in Research Model at The Ottawa Hospital (TOH) and the Ottawa Hospital Research Institute (OHRI). Key components of the model include: a Patient and Family Engagement Program at TOH, which recruits, educates, and supports patients, families and caregivers to engage in clinical care, governance, and research; the Ottawa Methods Centre within the OHRI, which leads methodological research and provides support to investigators for patient engagement and patient-oriented research at TOH; and the Office of Patient Engagement in Research Activities, also within the OHRI, which facilitates collaborations between patients, researchers, clinicians and other stakeholders. Early success of this model can be attributed to aligned institutional priorities between TOH, OHRI and patients, the establishment of a patient engagement policy, ongoing education and support provided to patient partners and researchers, and innovative recruitment, tracking and evaluation procedures. Ongoing challenges and next steps include promoting diversity among patient partners, implementing an equitable compensation policy, engaging patients across a variety of roles and research areas, and developing resources to expand and sustain this program.

CONCLUSION

This model represents a unique effort of patients, clinicians, researchers, and policymakers across disciplines and institutions to produce a harmonized strategy and infrastructure for meaningful collaboration with patients and families in health research, and capacity building in patient-oriented research.

Optimizing the implementation of a multisite feasibility trial of a mind-body program in acute orthopedic trauma

The Toolkit for Optimal Recovery (TOR) is a mind-body program for patients with acute orthopedic injuries who are at risk for persistent pain/disability. In preparation for a multisite feasibility trial of TOR at three orthopedic trauma centers, we aim to qualitatively identify barriers and facilitators to study implementation and strategies to mitigate the implementation barriers and leverage facilitators.We conducted 18 live video focus groups among providers and three one-on-one interviews with department chiefs at Level 1 trauma centers in three geographically diverse sites (N = 79 participants). Using a content analysis approach, we detected the site-specific barriers and facilitators of implementation of TOR clinical trial. We organized the data according to 26 constructs of the Consolidated Framework for Implementation Research (CFIR), mapped to three Proctor implementation outcomes relevant to the desired study outcomes (acceptability, appropriateness, and feasibility). Across the three sites, we mapped six of the CFIR constructs to acceptability, eight to appropriateness, and three to feasibility. Prominent perceived barriers across all three sites were related to providers' lack of knowledge/comfort addressing psychosocial factors, and organizational cultures of prioritizing workflow efficiency over patients' psychosocial needs (acceptability), poor fit between TOR clinical trial and the fast-paced clinic structure as well as basic needs of some patients (appropriateness), and limited resources (feasibility). Suggestions to maximize the implementation of the TOR trial included provision of knowledge/tools to improve providers' confidence, streamlining study recruitment procedures, creating a learning collaborative, tailoring the study protocol based on local needs assessments, exercising flexibility in conducting research, dedicating research staff, and identifying/promoting champions and using novel incentive structures with regular check-ins, while keeping study procedures as nonobtrusive and language as de-stigmatizing as possible. These data could serve as a blueprint for implementation of clinical research and innovations in orthopedic and other medical settings.