Assessing team performance – Markers and methods

Analyzing interprofessional teamwork in the operating room: An exploratory observational study using conventional and alternative approaches

Intraoperative teamwork is vital for patient safety. Conventional tools for studying intraoperative teamwork typically rely on behaviorally anchored rating scales applied at the individual or team level, while others capture narrative information across several units of analysis. This prospective observational study characterizes teamwork using two conventional tools (Operating Theatre Team Non-Technical Skills Assessment Tool [NOTECHS]; Team Emergency Assessment Measure [TEAM]), and one alternative approach (modified-Systems Engineering Initiative for Patient Safety [SEIPS] model). We aimed to explore the advantages and disadvantages of each for providing feedback to improve teamwork practice. Fifty consecutive surgical cases at a Canadian academic hospital were recorded with the OR Black Box®, analyzed by trained raters, and summarized descriptively. Teamwork performance was consistently high within and across cases rated with NOTECHS and TEAMS. For cases analyzed with the modified-SEIPS tool, both optimal and suboptimal teamwork behaviors were identified, and team resilience was frequently observed. NOTECHS and TEAM provided summative assessments and overall pattern descriptions, while SEIPS facilitated a deeper understanding of teamwork processes. As healthcare organizations continue to prioritize teamwork improvement, SEIPS may provide valuable insights regarding teamwork behavior and the broader context influencing performance. This may ultimately enhance the development and effectiveness of multi-level teamwork interventions.

Evaluating teamwork development in combat training settings: An exploratory case study utilising the Junior Leaders' Field Gun competition

A behaviourally-anchored observational rating scale (BAORS) of teamwork based upon the 'Big Five' teamwork model (Salas et al., 2005) was selected and adapted for use in a combat training setting - a UK military field gun competition. The teamwork development of 16 newly-formed teams training to master a historic tactical drill was evaluated over the course of a week-long residential programme. Training culminated in a timed field gun competition. Teams were trained and mentored in respects to teamwork and taskwork by experienced military instructors. Teamwork was assessed at the outset and end of training. Significant improvements were evident on all teamwork process dimensions, with the greatest improvement seen in teams' shared understanding of teamwork roles and strategies (shared mental models). The lack of an association between teamwork development and final drill performance is explored, as is the utility of the measurement protocol developed for teamwork assessment in other settings.

Patient and family views of team functioning in primary healthcare teams with nurse practitioners: a survey of patient-reported experience and outcomes

BACKGROUND

Nurse practitioners (NPs) have been added to primary healthcare teams to improve access to care. Team processes, including communication and decision-making, explicate how patients and families view team functioning. Yet, important gaps exist in our understanding of patient-reported experience and outcomes at the level of the healthcare team. We aimed to examine the influence of individual, team, and organizational characteristics, and role clarity on outcomes of care mediated by team processes in primary healthcare teams that include NPs.

METHODS

A cross-sectional survey across six sites representing practices with NPs in Québec, Canada, was conducted between March 2018 and April 2019 as part of a multiple-case study. Patients and families (n = 485; response rate: 53%) completed a validated questionnaire, which included a patient-reported experience measure (PREM) and a patient-reported outcome measure (PROM) of team functioning (Cronbach alpha: 0.771 (PROM) to 0.877 (PREM)). We performed logistic regression and mediation analyses to examine relationships between the individual, team, and organizational characteristics, role clarity, and outcomes of care mediated by team processes.

RESULTS

Patients and families expressed positive perceptions of team functioning (mean 4.97/6 [SD 0.68]) and outcomes of care (5.08/6 [0.74]). Also, high team processes (adjusted odds ratio [AOR] 14.92 [95% CI 8.11 to 27.44]) was a significant predictor of high outcomes of care. Role clarity (indirect effect coefficient ab = 6.48 [95% CI 3.79 to 9.56]), living in an urban area (-1.32 [-2.59 to -0.13]), patient as respondent (-1.43 [-2.80 to -0.14]), and income (1.73 [0.14 to 3.45]) were significant predictors of outcomes of care mediated by team processes.

CONCLUSIONS

This study provides key insights on how primary healthcare teams with NPs contribute to team functioning, using a validated instrument consistent with a conceptual framework. Results highlight that high role clarity, living in a non urban area, family as respondent, and adequate income were significant predictors of high outcomes of care mediated by high team processes. Additional research is needed to compare teams with and without NPs in different settings, to further explicate the relationships identified in our study.

Systematic review of the characteristics of brief team interventions to clarify roles and improve functioning in healthcare teams

AIM

Describe brief (less than half a day) interventions aimed at improving healthcare team functioning.

METHODS

A systematic review on brief team interventions aimed at role clarification and team functioning (PROSPERO Registration Number: CRD42018088922). Experimental or quasi-experimental studies were included. Database searches included CINAHL, Medline, EMBASE, PUBMED, Cochrane, RCT Registry-1990 to April 2020 and grey literature. Articles were screened independently by teams of two reviewers. Risk of bias was assessed. Data from the retained articles were extracted by one reviewer and checked by a second reviewer independently. A narrative synthesis was undertaken.

RESULTS

Searches yielded 1928 unique records. Final sample contained twenty papers describing 19 studies, published between 2009 and 2020. Studies described brief training interventions conducted in acute care in-patient settings and included a total of 6338 participants. Participants' socio-demographic information was not routinely reported. Studies met between two to six of the eight risk of bias criteria. Interventions included simulations for technical skills, structured communications and speaking up for non-technical skills and debriefing. Debriefing sessions generally lasted between five to 10 minutes. Debriefing sessions reflected key content areas but it was not always possible to determine the influence of the debriefing session on participants' learning because of the limited information reported.

DISCUSSION

Interest in short team interventions is recent. Single two-hour sessions appear to improve technical skills. Three to four 30- to 60-minute training sessions spread out over several weeks with structured facilitation and debriefing appear to improve non-technical skills. Monthly meetings appear to sustain change over time.

CONCLUSION

Short team interventions show promise to improve team functioning. Effectiveness of interventions in primary care and the inclusion of patients and families needs to be examined. Primary care teams are structured differently than teams in acute care and they may have different priorities.

Effective Strategies for Teaching Teamwork

Teamwork is essential for health care providers, who must work together to ensure safe and effective patient care. The ability to function effectively as a team is especially important in critical care, where ad hoc teams are brought together for short-term management of crisis situations. Teamwork training has been widely implemented, but ongoing education and practice are needed to maintain and improve competency. This article reviews some of the literature on team science and provides recommendations for enhancing training to promote development of a shared mental model. Strategies such as ensuring multidisciplinary participation, clarifying team resources and goals, and creating practice scenarios can increase the effectiveness of training for critical care teams. Evaluation can provide immediate feedback on learning outcomes and may facilitate subsequent transfer of learning to the clinical setting. Interventions that improve a team's ability to work toward a common goal can improve outcomes for critically ill patients.

Evaluating team decision-making as an emergent phenomenon

Background

The complexity of modern clinical practice has highlighted the fallibility of individual clinicians’ decision-making, with effective teamwork emerging as a key to patient safety. Dual process theory is widely accepted as a framework for individual decision-making, with type 1 processes responsible for fast, intuitive and automatic decisions and type 2 processes for slow, analytical decisions. However, dual process theory does not explain cognition at the group level, when individuals act in teams. Team cognition resulting from dynamic interaction of individuals is said to be more resilient to decision-making error and greater than simply aggregated cognition.

Methods

Clinicians were paired as teams and asked to solve a cognitive puzzle constructed as a drug calculation. The frequency at which the teams made incorrect decisions was compared with that of individual clinicians answering the same question.

Results

When clinicians acted in pairs, 63% answered the cognitive puzzle correctly, compared with 33% of clinicians as individuals, showing a statistically significant difference in performance (χ2(1, n=116)=24.329, P<0.001). Based on the predicted performance of teams made up of the random pairing of individuals who had the same propensity to answer as previously, there was no statistical difference in the actual and predicted teams’ performance.

Conclusions

Teams are less prone to making errors of decision-making than individuals. However, the improved performance is likely to be owing to the effect of aggregated cognition rather than any improved decision-making as a result of the interaction. There is no evidence of team cognition as an emergent and distinct entity.

Alternative Markers of Performance in Simulation: Where We Are and Where We Need To Go

This article on alternative markers of performance in simulation is the product of a session held during the 2017 Academic Emergency Medicine Consensus Conference "Catalyzing System Change Through Health Care Simulation: Systems, Competency, and Outcomes." There is a dearth of research on the use of performance markers other than checklists, holistic ratings, and behaviorally anchored rating scales in the simulation environment. Through literature review, group discussion, and consultation with experts prior to the conference, the working group defined five topics for discussion: 1) establishing a working definition for alternative markers of performance, 2) defining goals for using alternative performance markers, 3) implications for measurement when using alternative markers, identifying practical concerns related to the use of alternative performance markers, and 5) identifying potential for alternative markers of performance to validate simulation scenarios. Five research propositions also emerged and are summarized.