Standardized assessment for evaluation of team skills: validity and feasibility

Assertiveness in physicians' interpersonal professional encounters: A scoping review

PURPOSE

Good relationships between physicians, patients, families and the healthcare team are essential for high-quality care. Medical encounters are sometimes challenging. They may include conflicts, requiring physicians to be assertive: that is to share and protect their needs, rights and values while preserving those of others. Whereas assertiveness has been studied in patients and nursing staff (those with less power in healthcare), physicians' assertiveness, which must be mindful of these power differences, lacks a comprehensive review. Thus, this scoping review focuses on assertive communication in physicians' encounters.

METHODS

A literature search of four online databases: MEDLINE, Embase, PsycINFO and WoS, seeking articles on physicians' assertiveness as a communication style published until February 2022. The Joanna Briggs Institute approach and the Preferred Reporting Items for Systematic Reviews checklist underpinned the review protocol.

RESULTS

We identified 1513 articles relating to assertiveness, reviewed 153 full-text articles and selected 22 for full review, 68% from the last decade. The articles focused mainly on assertive communication and relationships with medical staff, with 54% focusing on bottom-up power relations. In 40% of the articles, no clear definition of assertiveness was included. Definitions included had varied focus: on self, on the other or both. Overall, assertiveness measures varied widely, precluding a methodical comparison.

CONCLUSIONS

Despite the growing interest in physicians' assertiveness, a clearer definition and in-depth exploration of assertiveness are needed alongside development of valid measures of assertiveness appropriate to physicians. Based on the review, we offer a relational definition of assertiveness as the capacity to communicate one's views, concerns, rights and needs while respecting others and preserving therapeutic, collegial and educational professional alliances. This definition may serve to expand research in the field while offering a professional alternative to problematic communication styles-passive and self-denying or paternalistic and aggressive -that obfuscate and thus undermine physician-patient relationships.

Observer-based tools for non-technical skills assessment in simulated and real clinical environments in healthcare: a systematic review

Background

Over the past three decades multiple tools have been developed for the assessment of non-technical skills (NTS) in healthcare. This study was designed primarily to analyse how they have been designed and tested but also to consider guidance on how to select them.

Objectives

To analyse the context of use, method of development, evidence of validity (including reliability) and usability of tools for the observer-based assessment of NTS in healthcare.

Design

Systematic review.

Data sources

Search of electronic resources, including PubMed, Embase, CINAHL, ERIC, PsycNet, Scopus, Google Scholar and Web of Science. Additional records identified through searching grey literature (OpenGrey, ProQuest, AHRQ, King’s Fund, Health Foundation).

Study selection

Studies of observer-based tools for NTS assessment in healthcare professionals (or undergraduates) were included if they: were available in English; published between January 1990 and March 2018; assessed two or more NTS; were designed for simulated or real clinical settings and had provided evidence of validity plus or minus usability. 11,101 articles were identified. After limits were applied, 576 were retrieved for evaluation and 118 articles included in this review.

Results

One hundred and eighteen studies describing 76 tools for assessment of NTS in healthcare met the eligibility criteria. There was substantial variation in the method of design of the tools and the extent of validity, and usability testing. There was considerable overlap in the skills assessed, and the contexts of use of the tools.

Conclusion

This study suggests a need for rationalisation and standardisation of the way we assess NTS in healthcare and greater consistency in how tools are developed and deployed.

Implementation, evaluation, and outcome of TeamSTEPPS in interprofessional education: a scoping review

The ability to effectively work in interprofessional teams is listed as one of the five core competencies in health professions education. Though the importance of interprofessional education (IPE) has been established, results of studies have been difficult to compare due to the high variability of programs. We undertook a scoping review to examine the use of a prescribed curriculum, TeamSTEPPS, in IPE. Articles describing TeamSTEPPS implementations were extracted from Pubmed, Embase, and Scopus. Studies with two or more health professions students reporting on a clear evaluation and published in English were eligible for inclusion. Two researchers independently applied inclusion criteria to studies and reconciled conflicts for a final selection. The reference lists of selected papers were also searched for relevant studies. Data were extracted from each of the articles independently using a standard form. Twenty-four papers describing 23 unique programs were included. Programs used a variety of teaching modalities and included students from two to ten health professions, most commonly medical and nursing students. Programs used a range (n = 11) of validated IPE evaluation surveys, few of which were part of the TeamSTEPPS program. Methods included multimodal evaluations, self-assessment confidence and attitude surveys, pre/post-test models, and external evaluation of simulation performance. There was great variation in the implementation of TeamSTEPPS implying that while a consistent curriculum it can be adapted to meet the needs of different educational contexts. The variation in evaluation methods makes comparing and synthesis of results problematic. Future IPE research can expand on the use of this prescribed curriculum, especially with focus on uniform evaluation methods.

Development of a Simulation-Based Interprofessional Teamwork Assessment Tool

BACKGROUND

The Accreditation Council for Graduate Medical Education (ACGME) Milestone projects required each specialty to identify essential skills and develop means of assessment with supporting validity evidence for trainees. Several specialties rate trainees on a milestone subcompetency related to working in interprofessional teams. A tool to assess trainee competence in any role on an interprofessional team in a variety of scenarios would be valuable and suitable for simulation-based assessment.

OBJECTIVE

We developed a tool for simulation settings that assesses interprofessional teamwork in trainees.

METHODS

In 2015, existing tools that assess teamwork or interprofessionalism using direct observation were systematically reviewed for appropriateness, generalizability, adaptability, ease of use, and resources required. Items from these tools were included in a Delphi method with multidisciplinary pediatrics experts using an iterative process from June 2016 to January 2017 to develop an assessment tool.

RESULTS

Thirty-one unique tools were identified. A 2-stage review narrowed this list to 5 tools, and 81 items were extracted. Twenty-two pediatrics experts participated in 4 rounds of Delphi surveys, with response rates ranging from 82% to 100%. Sixteen items reached consensus for inclusion in the final tool. A global 4-point rating scale from novice to proficient was developed.

CONCLUSIONS

A novel tool to assess interprofessional teamwork for individual trainees in a simulated setting was developed using a systematic review and Delphi methodology. This is the first step to establish the validity evidence necessary to use this tool for competency-based assessment.

Face and content validity of the virtual reality simulator 'ScanTrainer®'

Background

Ultrasonography is a first-line imaging in the investigation of women’s irregular bleeding and other gynaecological pathologies, e.g. ovarian cysts and early pregnancy problems. However, teaching ultrasound, especially transvaginal scanning, remains a challenge for health professionals. New technology such as simulation may potentially facilitate and expedite the process of learning ultrasound. Simulation may prove to be realistic, very close to real patient scanning experience for the sonographer and objectively able to assist the development of basic skills such as image manipulation, hand-eye coordination and examination technique.

Objective

The aim of this study was to determine the face and content validity of a virtual reality simulator (ScanTrainer®, MedaPhor plc, Cardiff, Wales, UK) as reflective of real transvaginal ultrasound (TVUS) scanning.

Method

A questionnaire with 14 simulator-related statements was distributed to a number of participants with differing levels of sonography experience in order to determine the level of agreement between the use of the simulator in training and real practice.

Results

There were 36 participants: novices (n = 25) and experts (n = 11) who rated the simulator. Median scores of face validity statements between experts and non-experts using a 10-point visual analogue scale (VAS) ratings ranged between 7.5 and 9.0 (p > 0.05) indicated a high level of agreement. Experts’ median scores of content validity statements ranged from 8.4 to 9.0.

Conclusions

The findings confirm that the simulator has the feel and look of real-time scanning with high face validity. Similarly, its tutorial structures and learning steps confirm the content validity.

Electronic supplementary material

The online version of this article (10.1186/s10397-017-1020-6) contains supplementary material, which is available to authorized users.

A national evaluation using standardised patient actors to assess STI services in public sector clinical sentinel surveillance facilities in South Africa

OBJECTIVES

Quality concerns in STI service delivery and missed opportunities for integration with HIV testing and prevention services in South Africa have been well documented. This national evaluation aimed to evaluate current utilisation and adherence to national STI guidelines, including partner notification and integration with HIV services, for diagnosis and management of STIs.

METHODS

Facility surveys assessed infrastructure and resource availability, and standardised patient (SP) assessments evaluated quality of STI care in 50 public clinics in nine provinces in South Africa. The primary outcome was the proportion of SPs receiving essential STI care, defined as: offered an HIV test, condoms, partner notification counselling and correct syndromic treatment. Weighted proportions were generated, and SP findings were compared by gender using χ² tests with Rao-Scott correction.

RESULTS

More than 80% of facilities reported medications in stock, with the exceptions of oral cefixime (48.3%), oral erythromycin (75.1%) and paediatric syrups. Among 195 SP encounters, 18.7% (95% CI 10.7% to 30.5%) received all hypothesised essential STI services: offered HIV test (67.1%), offered condoms (31.4%), partner notification counselling (70.2%) and recommended syndromic treatment (60.7%). Men were more likely than women to be offered all services (25.1% vs 12.3%, p=0.023), recommended treatment (70.7% vs 50.9%, p=0.013) and partner notification counselling (79.9% vs 60.6%, p=0.020). Only 6.3% of providers discussed male circumcision with male SPs, and 26.3% discussed family planning with female SPs.

CONCLUSIONS

This evaluation of STI services across South Africa found gaps in the availability of medications, adherence to STI guidelines, condom provision and prevention messaging. Limited integration with HIV services for this high-risk population was a missed opportunity. Quality of STI care should continue to be monitored, and interventions to improve quality should be prioritised as part of national strategic HIV and primary healthcare agendas.

Priority Areas and Potential Solutions for Successful Integration and Sustainment of Health Systems Science in Undergraduate Medical Education

Educators, policy makers, and health systems leaders are calling for significant reform of undergraduate medical education (UME) and graduate medical education (GME) programs to meet the evolving needs of the health care system. Nationally, several schools have initiated innovative curricula in both classroom and workplace learning experiences to promote education in health systems science (HSS), which includes topics such as value-based care, health system improvement, and population and public health. However, the successful implementation of HSS curricula across schools is challenged by issues of curriculum design, assessment, culture, and accreditation, among others. In this report of a working conference using thematic analysis of workshop recommendations and experiences from 11 U.S. medical schools, the authors describe seven priority areas for the successful integration and sustainment of HSS in educational programs, and associated challenges and potential solutions. In 2015, following regular HSS workgroup phone calls and an Accelerating Change in Medical Education consortium-wide meeting, the authors identified the priority areas: partner with licensing, certifying, and accrediting bodies; develop comprehensive, standardized, and integrated curricula; develop, standardize, and align assessments; improve the UME to GME transition; enhance teachers' knowledge and skills, and incentives for teachers; demonstrate value added to the health system; and address the hidden curriculum. These priority areas and their potential solutions can be used by individual schools and HSS education collaboratives to further outline and delineate the steps needed to create, deliver, study, and sustain effective HSS curricula with an eye toward integration with the basic and clinical sciences curricula.

Contextualization and validation of the interprofessional collaborator assessment rubric (ICAR) through simulation: Pilot investigation

BACKGROUND

Simulation can be used for educating, evaluating and assessing psychometric properties of an instrument. The aim of this study was to contextualize and assess the validity and reliability of the Interprofessional Collaborative Assessment tool (ICAR) in an Iranian context using simulation.

METHODS

In this descriptive study, contextualization of the ICAR was assessed through several steps. Firstly, validity assessment was approved through expert panels and Delphi rounds. Secondly, reliability assessment was done by arranging a simulation video and assessing reproducibility, test-retest (ICC), internal consistency (Cronbach's Alpha) and inter-rater reliability (Kappa).The participants included 26 experts, 27 students and 6 staff of the Standardized Simulation Office of Teheran University of Medical Sciences.

RESULTS

Contextualization and validity of the ICAR were approved in an Iranian context. The reliability of the tool was computed to be 0.71 according to Cronbach´s Alpha. The test-retest was calculated to be 0.76.

CONCLUSION

The Iranian ICAR can be a useful tool for evaluating interprofessional collaborative competencies. The development of the instrument through a simulation scenario has been a positive prospect for researchers.

Designing and implementing full immersion simulation as a research tool

Simulation is a valuable research tool used to evaluate the clinical performance of devices, people and systems. The simulated setting may address concerns unique to complex clinical environments such as the Emergency Department, which make the conduct of research challenging. There is limited evidence available to inform the development of simulated clinical scenarios for the purpose of evaluating practice in research studies, with the majority of literature focused on designing simulated clinical scenarios for education and training. Distinct differences exist in scenario design when implemented in education compared with use in clinical research studies. Simulated scenarios used to assess practice in clinical research must not comprise of any purposeful or planned teaching and be developed with a high degree of validity and reliability. A new scenario design template was devised to develop two standardised simulated clinical scenarios for the evaluation of a new assessment framework for emergency nurses. The scenario development and validation processes undertaken are described and provide an evidence-informed guide to scenario development for future clinical research studies.

TEAM ATTITUDE EVALUATION: AN EVALUATION IN HOSPITAL COMMITTEES

INTRODUCTION

Patients' health and safety is not only a function of complex treatments and advanced therapeutic technologies but also a function of a degree based on which health care professionals fulfill their duties effectively as a team. The aim of this study was to determine the attitude of hospital committee members about teamwork in Kerman hospitals.

METHODOLOGY

This study was conducted in 2014 on 171 members of clinical teams and committees of four educational hospitals in Kerman University of Medical Sciences. To collect data, the standard "team attitude evaluation" questionnaire was used. This questionnaire consisted of five domains which evaluated the team attitude in areas related to the team structure, leadership, situation monitoring, mutual support, and communication in the form of a 5-point Likert type scale. To analyze data, descriptive statistical tests, T-test, ANOVA, and linear regression were used.

RESULTS

The average score of team attitude for hospital committee members was 3.9 out of 5. The findings showed that leadership had the highest score among the subscales of team work attitude, while mutual support had the lowest score. We could also observe that responsibility was an important factor in participants' team work attitude (β = -0.184, p = 0.024). Comparing data in different subgroups revealed that employment, marital status, and responsibility were the variables affecting the participants' attitudes in the team structure domain. Marital status played a role in leadership; responsibility had a role in situation monitoring; and work experience played a role in domains of communication and mutual support.

CONCLUSIONS

Hospital committee members had a positive attitude towards teamwork. Training hospital staff and paying particular attention to key elements of effectiveness in a health care team can have a pivotal role in promoting the team culture.