Implementation of the Operating Room Black Box Research Program at the Ottawa Hospital Through Patient, Clinical, and Organizational Engagement: Case Study

Background

A large proportion of surgical patient harm is preventable; yet, our ability to systematically learn from these incidents and improve clinical practice remains limited. The Operating Room Black Box was developed to address the need for comprehensive assessments of clinical performance in the operating room. It captures synchronized audio, video, patient, and environmental clinical data in real time, which are subsequently analyzed by a combination of expert raters and software-based algorithms. Despite its significant potential to facilitate research and practice improvement, there are many potential implementation challenges at the institutional, clinician, and patient level. This paper summarizes our approach to implementation of the Operating Room Black Box at a large academic Canadian center.

Objective

We aimed to contribute to the development of evidence-based best practices for implementing innovative technology in the operating room for direct observation of the clinical performance by using the case of the Operating Room Black Box. Specifically, we outline the systematic approach to the Operating Room Black Box implementation undertaken at our center.

Methods

Our implementation approach included seeking support from hospital leadership; building frontline support and a team of champions among patients, nurses, anesthesiologists, and surgeons; accounting for stakeholder perceptions using theory-informed qualitative interviews; engaging patients; and documenting the implementation process, including barriers and facilitators, using the consolidated framework for implementation research.

Results

During the 12-month implementation period, we conducted 23 stakeholder engagement activities with over 200 participants. We recruited 10 clinician champions representing nursing, anesthesia, and surgery. We formally interviewed 15 patients and 17 perioperative clinicians and identified key themes to include in an information campaign run as part of the implementation process. Two patient partners were engaged and advised on communications as well as grant and protocol development. Many anticipated and unanticipated challenges were encountered at all levels. Implementation was ultimately successful, with the Operating Room Black Box installed in August 2018, and data collection beginning shortly thereafter.

Conclusions

This paper represents the first step toward evidence-guided implementation of technologies for direct observation of performance for research and quality improvement in surgery. With technology increasingly being used in health care settings, the health care community should aim to optimize implementation processes in the best interest of health care professionals and patients.

Impact of physician's sex/gender on processes of care, and clinical outcomes in cardiac operative care: a systematic review

OBJECTIVES

This systematic review aimed to assess the role of physician's sex and gender in relation to processes of care and/or clinical outcomes within the context of cardiac operative care.

DESIGN

A systematic review.

DATA SOURCES

Searches were conducted in PsycINFO, Embase and Medline from inception to 6 September 2018. The reference lists of relevant systematic reviews and included studies were also searched.

ELIGIBILITY CRITERIA FOR SELECTING STUDIES

Quantitative studies of any design were included if they were published in English or French, involved patients of any age undergoing a cardiac surgical procedure and specifically assessed differences in processes of care or clinical patient outcomes by physician's sex or gender. Studies were screened in duplicate by two pairs of independent reviewers.

OUTCOME MEASURES

Processes of care, patient morbidity and patient mortality.

RESULTS

The search yielded 2095 publications after duplicate removal, of which two were ultimately included. These studies involved various types of surgery, including cardiac. One study found that patients treated by female surgeons compared with male surgeons had a lower 30-day mortality. The other study, however, found no differences in patient outcomes by surgeon's sex. There were no studies that investigated anaesthesiologist's sex/gender. There were also no studies investing physician's sex or gender exclusively in the cardiac operating room.

CONCLUSIONS

The limited data surrounding the impact of physician's sex/gender on the outcomes of cardiac surgery inhibits drawing a robust conclusion at this time. Results highlight the need for primary research to determine how these factors may influence cardiac operative practice, in order to optimise provider's performance and improve outcomes in this high-risk patient group.

Association of intraoperative anaesthesia handovers with patient morbidity and mortality: a systematic review and meta-analysis

BACKGROUND

Handover of anaesthesia patient care during surgery is common; however, its association with patient outcome is unclear. This systematic review aimed to assess the impact of anaesthesia handover during surgery on patient outcome.

METHODS

All prospective and retrospective clinical studies specifically investigating the association of intraoperative transfer of anaesthesia care between anaesthesia providers in the operating room with patient morbidity and mortality were included. Searches were conducted from inception to April 24, 2019 in Medline, Medline in Process, CINAHL, and Embase. Reference lists of included studies were searched. Studies were assessed for eligibility and data were extracted by independent reviewers in duplicate with disagreements resolved by consensus or a third reviewer. Risk of bias was assessed in duplicate using the National Institutes of Health Quality Assessment Tool for Observational Cohort and Cross-Sectional Studies. Data were summarised narratively given substantial heterogeneity. An exploratory meta-analysis was conducted using a random-effects model for a subset of comparable studies.

RESULTS

Eight studies met the inclusion criteria. Six studies focused on patients as the unit of analysis (n_patients=605 678) and two focused on anaesthesia providers as the unit of analysis (n_providers=307). Seven studies identified a relationship between anaesthesia handovers and adverse patient outcomes, whereas one suggested that handover may be beneficial to error detection or rectification. Included studies were of fair or good quality. Meta-analysis of four studies found a 40% increased risk of patients experiencing an adverse event when an anaesthesia handover occurs during the procedure (pooled risk ratio=1.40; 95% confidence interval, 1.19 to 1.65; P<0.001; I²=98%).

CONCLUSIONS

Intraoperative anaesthesia handovers generally increase morbidity and mortality for surgical patients but could have the potential to improve safety in certain contexts. Future research should determine the specific handover characteristics that impact safety.

Applying an intersectionality lens to the theoretical domains framework: a tool for thinking about how intersecting social identities and structures of power influence behaviour

Background

A key component of the implementation process is identifying potential barriers and facilitators that need to be addressed. The Theoretical Domains Framework (TDF) is one of the most commonly used frameworks for this purpose. When applying the TDF, it is critical to understand the context in which behaviours occur. Intersectionality, which accounts for the interface between social identity factors (e.g. age, gender) and structures of power (e.g. ageism, sexism), offers a novel approach to understanding how context shapes individual decision-making and behaviour. We aimed to develop a tool to be used alongside applications of the TDF to incorporate an intersectionality lens when identifying implementation barriers and enablers.

Methods

An interdisciplinary Framework Committee (n = 17) prioritized the TDF as one of three models, theories, and frameworks (MTFs) to enhance with an intersectional lens through a modified Delphi approach. In collaboration with the wider Framework Committee, a subgroup considered all 14 TDF domains and iteratively developed recommendations for incorporating intersectionality considerations within the TDF and its domains. An iterative approach aimed at building consensus was used to finalize recommendations.

Results

Consensus on how to apply an intersectionality lens to the TDF was achieved after 12 rounds of revision. Two overarching considerations for using the intersectionality alongside the TDF were developed by the group as well as two to four prompts for each TDF domain to guide interview topic guides. Considerations and prompts were designed to assist users to reflect on how individual identities and structures of power may play a role in barriers and facilitators to behaviour change and subsequent intervention implementation.

Conclusions

Through an expert-consensus approach, we developed a tool for applying an intersectionality lens alongside the TDF. Considering the role of intersecting social factors when identifying barriers and facilitators to implementing research evidence may result in more targeted and effective interventions that better reflect the realities of those involved.

La définition des moments critiques et non critiques en salle d'opération : une étude de consensus Delphi modifiée

BACKGROUND

While the operating room (OR) has significantly benefited from aviation strategies to improve safety, the rate of avoidable human errors remains relatively high. One key aviation strategy that has yet to be formally established in the OR is the "sterile cockpit" rule, which prohibits all non-essential behaviours during critical moments of a flight. Applying this rule to the OR may enhance patient safety, but the critical moments of surgery need to be defined first.

METHODS

This study used a modified Delphi methodology to determine critical moments during surgery according to OR team members across institutions, professions, and specialties. Analysis occurred after each round. The stopping criterion was consensus on 80% of survey items or no change in the mean score for any individual item between two consecutive rounds.

RESULTS

The first round included 304 respondents. Of these, 115 completed the second-round survey, and 75 completed all three rounds (27 nurses, 29 anesthesiologists, 19 surgeons). Critical moments obtained by consensus were: induction of anesthesia; emergence from anesthesia; preoperative briefing; final counts at the end of the procedure; anesthesiologist- or surgeon-relevant intraoperative event; handovers; procedure-specific high-risk surgical moments; crisis resource management situations; medication and equipment preparation; and key medication administration.

CONCLUSIONS

By defining the most critical moments of surgery, future research can determine the relative importance of behaviour and actions at each stage and target interventions to these stages.

Sex/gender and additional equity characteristics of providers and patients in perioperative anesthesia trials: a cross-sectional analysis of the literature

Sex and gender, among other equity-related characteristics, influence the process of care and patients’ outcomes. Currently, the extent to which these characteristics are considered in the anesthesia literature remains unknown. This study assesses their incorporation in randomized controlled trials (RCTs) on anesthesia-related interventions, for both patients and healthcare providers. This is a cross-sectional analysis using an existing dataset derived from the anesthesia literature. The dataset originated from a scoping review searching MEDLINE, Embase, CINAHL, CENTRAL, and the Cochrane Database of Systematic reviews. RCTs investigating the effect of anesthesia-related interventions on mortality for adults undergoing surgery were included. Equity outcome measures were recorded for both patients and providers and assessed for inclusion in the study design, reporting of results, and analysis of intervention effects. Three-hundred sixty-one RCTs (n = 144,674) were included. Most RCTs (91%) reported patient sex/gender, with 58% of patients identified as male. There were 139 studies (39%), where 70% or more of the sample was male, compared to just 14 studies (4%), where 70% or more of the sample was female. Only 10 studies (3%) analyzed results by patient sex/gender, with one reporting a significant effect. There was substantial variation in how age was reported, although nearly all studies (98%) reported some measure of age. For healthcare providers, equity-related information was never available. Better consideration of sex/gender and additional health equity parameters for both patients and providers in RCTs is needed to improve evidence quality, and ultimately, patient care and outcome.

Measuring the teamwork performance of operating room teams: a systematic review of assessment tools and their measurement properties

Teamwork is fundamental to surgical patient safety but is inconsistently measured. While many tools have been developed for elective intraoperative situations, it is unclear which is the most robust. This systematic review aimed to identify tools to measure the teamwork of operating room teams. Studies were included if they examined the measurement properties of these tools. PsycINFO, Embase (via OVID), CINAHL, ERIC, Medline and Medline in Process (via OVID) were searched through to May 3, 2019, as were reference lists of included studies and previously published relevant reviews. Retrieved articles were screened and data extracted in duplicate by two independent reviewers. Quality was assessed using the COSMIN checklist. Of the 2121 references identified, 14 studies of six assessment tools were included. Tools were validated across various specialties, mostly in clinical rather than simulated settings. The Observational Teamwork Assessment for Surgery (OTAS) and Operating Theater Team Non-Technical Skills Assessment Tool (NOTECHS) were the most frequently investigated tools. Though acceptable for assessing teamwork, both NOTECHS and OTAS rely on the questionable assumption that the teamwork of a team is equivalent to the sum of individual performances. Future studies may investigate other assessment tools that assess the whole team as the unit of analysis along with the potential of these tools to provide healthcare providers with meaningful feedback in clinical practice.

Evidence for simulation-based education in hyperbaric medicine: A systematic review

INTRODUCTION

Evidence from many areas of healthcare suggests that skills learned during simulation transfer to clinical settings; however, this has not yet been investigated in hyperbaric medicine. This systematic review aimed to identify, summarize, and assess the impact of simulation-based education in hyperbaric medicine.

METHODS

Eligible studies investigated the effect of simulation-based education for learning in hyperbaric medicine, used any design, and were published in English in a peer-reviewed journal. Learning outcomes across all Kirkpatrick levels were included. MEDLINE, Embase, and the Cochrane Central Register of Controlled Trials were searched. Pairs of independent reviewers assessed references for study eligibility.

RESULTS

We found no article assessing the impact of simulation-based education in hyperbaric medicine published in English. Only one potentially relevant paper published in German was found.

CONCLUSIONS

More research is needed to determine how the hyperbaric medicine community and their patients may benefit from simulation-based education to optimize both practice and patient care.

Exploring stakeholder perceptions around implementation of the Operating Room Black Box for patient safety research: a qualitative study using the theoretical domains framework

Background

Systematically observing clinical performance in the operating room (OR) to support patient safety initiatives faces numerous logistical and methodological challenges. These may be solved by new audio-video recording technologies like the OR Black Box, which is a tool similar to black boxes in aviation. This study aimed to identify barriers and enablers that may influence patients’, clinicians’ and senior leadership team members’ support of the OR Black Box in order to guide its future implementation.

Methods

Patients, clinicians and senior leadership team members were recruited to participate in semistructured interviews informed by the theoretical domains framework (TDF) to identify factors relevant to planning OR Black Box implementation. Deidentified interview transcripts were analysed in duplicate following a TDF coding structure.

Results

Data saturation was achieved at 15 patients, 17 clinicians and 9 senior leadership team members. Seven domains were relevant for patients, nine for clinicians and four for senior leadership. Knowledge and Beliefs about consequences were barriers and enablers for all three groups. Memory, attention and decision processes and Social influences were enablers for both clinicians and senior leadership. Environmental context and resources, Emotion and Behavioural regulation were found to be barriers and enablers for both clinicians and patients. Social/professional role and identity and Reinforcement were enablers for patients only and Optimism and Intentions were barriers and enablers to clinicians.

Conclusions

While most stakeholders were supportive of the OR Black Box, we identified many key areas that need to be addressed during its implementation. It is critical to ensure all stakeholders have adequate and accurate information about the OR Black Box system and research goals, and that the OR Black Box is positioned as a patient safety initiative for learning from and improving practice.

Measuring the teamwork performance of teams in crisis situations: a systematic review of assessment tools and their measurement properties

BACKGROUND

Educational interventions to improve teamwork in crisis situations have proliferated in recent years with substantial variation in teamwork measurement. This systematic review aimed to synthesise available tools and their measurement properties in order to identify the most robust tool for measuring the teamwork performance of teams in crisis situations.

METHODS

Searches were conducted in Embase (via OVID), PsycINFO, Cumulative Index to Nursing and Allied Health Literature, Education Resources Information Center, Medline and Medline In-Process (via OVID) (through 12 January 2017). Studies evaluating the measurement properties of teamwork assessment tools for teams in clinical or simulated crisis situations were included. Two independent reviewers screened studies based on predetermined criteria and completed data extraction. Risk of bias was assessed using the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) checklist.

RESULTS

The search yielded 1822 references. Twenty studies were included, representing 13 assessment tools. Tools were primarily assessed in simulated resuscitation scenarios for emergency department teams. The Team Emergency Assessment Measure (TEAM) had the most validation studies (n=5), which demonstrated three sources of validity (content, construct and concurrent) and three sources of reliability (internal consistency, inter-rater reliability and test-retest reliability). Most studies of TEAM's measurement properties were at no risk of bias.

CONCLUSIONS

A number of tools are available for assessing teamwork performance of teams in crisis situations. Although selection will ultimately depend on the user's context, TEAM may be the most promising tool given its measurement evidence. Currently, there is a lack of tools to assess teamwork performance during intraoperative crisis situations. Additional research is needed in this regard.

Measuring non-technical skills of anaesthesiologists in the operating room: a systematic review of assessment tools and their measurement properties

BACKGROUND

Non-technical skills, such as communication or leadership, are integral to clinical competence in anaesthesia. There is a need for valid and reliable tools to measure anaesthetists' non-technical performance for both initial and continuing professional development. This systematic review aims to summarise the measurement properties of existing assessment tools to determine which tool is most robust.

METHODS

Embase (via OVID), Medline and Medline in Process (via OVID), and reference lists of included studies and previously published relevant systematic reviews were searched (through August 2017). Quantitative studies investigating the measurement properties of tools used to assess anaesthetists' intraoperative non-technical skills, either in a clinical or simulated environment, were included. Pairs of independent reviewers determined eligibility and extracted data. Risk of bias was assessed using the COSMIN checklist.

RESULTS

The search yielded 978 studies, of which 14 studies describing seven tools met the inclusion criteria. Of these, 12 involved simulated crisis settings only. The measurement properties of the Anaesthetists' Non-Technical Skills (ANTS) tool were most commonly assessed (n=9 studies), with studies of two types of validity (content, concurrent) and two types of reliability (internal consistency, interrater). Most of these studies, however, were at serious risk of bias.

CONCLUSIONS

Though there are seven tools for assessing the non-technical skills of anaesthetists, only ANTS has been extensively investigated with regard to its measurement properties. ANTS appears to have acceptable validity and reliability for assessing non-technical skills of anaesthetists in both simulated and clinical settings. Future research should consider additional clinical contexts and types of measurement properties.

Exploring experiences with compulsory psychiatric community treatment: A qualitative multi-perspective pilot study in an urban Canadian context

As medical, ethical and clinical effectiveness debates about the use of compulsory psychiatric treatment continues, it is important to further explore the actual experiences and perspectives of all relevant stakeholders in community treatment orders (CTOs). This qualitative pilot study engaged a total of twenty-seven clients, their family members, and care providers in Toronto, Canada. Semi-structured, one-on-one interviews were conducted between February and July 2013 and analyzed using thematic analysis. Top key themes from all the participants identified include, among others: 1) clients' experiences of coercion while treated under CTO, but a preference for CTOs compared to involuntary hospitalization, nevertheless; 2) limited real opportunities for collaboration in treatment decisions expressed by clients and family members; 3) acceptance of the potential for clinical recovery on CTOs while debating the role of CTO in a broader recovery journey by all stakeholders; 4) general preservation of therapeutic relationships between clients and care providers, while acknowledging the tension of taking on an "enforcer" role by providers; and 5) existence of different avenues for asserting agency by clients. The findings of this research illuminate the nuanced, complex, and adaptive perspectives held by different stakeholders, point to the importance of preserving and enhancing procedural justice in their use, and alert the field to incorporate recovery-based approaches in this controversial practice that is a widely and commonly used clinical tool across many jurisdictions.

Re-evaluating gender differences in self-rated health: The importance of cohort

Research examining gender differences in self-rated health (SRH) has typically not distinguished between age and cohort-related changes in the health of men and women over time. Using longitudinal data from the Panel Study of Income Dynamics, this study finds gender diffegrences in SRH may actually be an artifact of cohort. Prior to examining health across cohorts, women reported worse health than men. With the introduction of cohort to the models, no gender difference was found except in the earliest cohort (born 1924-1933). Historical context is therefore critical to understanding the health trajectories of women and men, which are not uniform across cohorts.

Race, Gender, and the Resources That Matter: An Investigation of Intersectionality and Health

This study examined racial disparities in health among women, in particular, the relationship between social status and both the development of psychosocial resources and good health. These relationships were investigated using a sample of 869 women from the 2007 wave of the U.S. Panel Study of Income Dynamics, Child Development Supplement. While Black women developed the intrapersonal rewards of self-efficacy and self-esteem through socioeconomic status, they did not experience the same degree of health benefits with these advantages as White women. Models relating both self-rated health and chronic conditions suggested that, instead, highly educated Black women were at a persistent health disadvantage relative to Whites, even at the same levels of psychosocial resources. That being said, Black women with higher self-efficacy, and particularly, higher self-esteem, were more likely than Black women with lower levels of these resources to report being in better health. Thus, resources may improve health within a disadvantaged group while still not bringing them up to the level of health experienced by their advantaged counterparts. Overall, the findings demonstrated that research should not treat women as a homogenous group, assuming that mechanisms affecting health operate the same for women regardless of their race.