Preliminary evaluation of an educational model for promoting positive team attitudes and functioning among pain medicine fellows

Addressing the Interprofessional Collaboration Competencies of the Association of American Medical Colleges: A Systematic Review of Assessment Instruments in Undergraduate Medical Education

PURPOSE

To summarize characteristics and validity evidence of tools that assess teamwork in undergraduate medical education (UME), and provide recommendations for addressing the interprofessional collaboration competencies of the Association of American Medical Colleges (AAMC).

METHOD

The authors conducted a systematic review, searching MEDLINE, MEDLINE In-process, CINAHL, and PsycINFO from January 1, 1979, through April 1, 2014; they searched reference lists and national meeting abstracts. They included original research reports that described a quantitative tool used to assess teamwork in UME. They abstracted characteristics and validity evidence for the tools, plus study quality, according to established frameworks. Two authors independently abstracted 25% of articles and calculated agreement. Authors then applied predefined criteria to identify tools best suited to address the AAMC's teamwork competencies.

RESULTS

Of 13,549 citations, 70 articles describing 64 teamwork assessment tools were included. Of these 64 tools, 27 (42%) assessed teamwork in classroom, 31 (48%) in simulation, and only 7 (11%) in actual clinical settings. The majority (47; 73%) of tools assessed medical students' teamwork in interprofessional teams. On the basis of content concordance, strength of validity evidence, generalizability of scores, and level of outcomes, four published tools were recommended to assess the AAMC's teamwork competencies: the Collaborative Healthcare Interdisciplinary Relationship Planning Scale, Readiness for Interprofessional Learning Scale, Communication and Teamwork Skills assessment, and Teamwork Mini-Clinical Evaluation Exercise.

CONCLUSIONS

Substantial validity evidence supports the use of several UME teamwork assessments. Four tools have been appropriately designed and sufficiently studied to constitute appropriate assessments of the AAMC's teamwork competencies.

Interprofessional collaboration milestones: advocating for common assessment criteria in graduate medical education

BACKGROUND

Milestone-based assessments of resident physicians inform critical decisions regarding resident competence and advancement. Thus, it is essential that milestone evaluations are based upon strong validity evidence and that consistent evaluation criteria are used across residency programs. A common approach to assessment of interprofessional collaboration milestones is particularly important since standardized measures of individual resident competence in interprofessional collaboration have not been established.

DISCUSSION

We propose that assessments of interprofessional collaboration in graduate medical education meet common criteria, namely, these assessments should: 1) measure competency of an individual resident, 2) occur in the context of an interprofessional team, 3) be ascertained via direct observation of the resident, 4) be performed in a real-world clinical practice setting (such as a hospital ward, outpatient clinic, or operating room). We present the evidence-based rationale for these criteria and cite examples of published assessment instruments that fulfill one or more of the criteria, however further research is needed to ensure fidelity of assessments. The proposed criteria may assist residency educators as they endeavor to provide robust and consistent assessments of interprofessional collaboration milestones.

Teamwork assessment in internal medicine: a systematic review of validity evidence and outcomes

OBJECTIVE

Valid teamwork assessment is imperative to determine physician competency and optimize patient outcomes. We systematically reviewed published instruments assessing teamwork in undergraduate, graduate, and continuing medical education in general internal medicine and all medical subspecialties.

DATA SOURCES

We searched MEDLINE, MEDLINE In-process, CINAHL and PsycINFO from January 1979 through October 2012, references of included articles, and abstracts from four professional meetings. Two content experts were queried for additional studies.

STUDY ELIGIBILITY

Included studies described quantitative tools measuring teamwork among medical students, residents, fellows, and practicing physicians on single or multi-professional (interprofessional) teams.

STUDY APPRAISAL AND SYNTHESIS METHODS

Instrument validity and study quality were extracted using established frameworks with existing validity evidence. Two authors independently abstracted 30 % of articles and agreement was calculated.

RESULTS

Of 12,922 citations, 178 articles describing 73 unique teamwork assessment tools met inclusion criteria. Interrater agreement was intraclass correlation coefficient 0.73 (95 % CI 0.63-0.81). Studies involved practicing physicians (142, 80 %), residents/fellows (70, 39 %), and medical students (11, 6 %). The majority (152, 85 %) assessed interprofessional teams. Studies were conducted in inpatient (77, 43 %), outpatient (42, 24 %), simulation (37, 21 %), and classroom (13, 7 %) settings. Validity evidence for the 73 tools included content (54, 74 %), internal structure (51, 70 %), relationships to other variables (25, 34 %), and response process (12, 16 %). Attitudes and opinions were the most frequently assessed outcomes. Relationships between teamwork scores and patient outcomes were directly examined for 13 (18 %) of tools. Scores from the Safety Attitudes Questionnaire and Team Climate Inventory have substantial validity evidence and have been associated with improved patient outcomes.

LIMITATIONS

Review is limited to quantitative assessments of teamwork in internal medicine.

CONCLUSIONS

There is strong validity evidence for several published tools assessing teamwork in internal medicine. However, few teamwork assessments have been directly linked to patient outcomes.