What indicates competency in systems based practice? An analysis of perspective consistency among healthcare team members

Multi-stakeholder validation of entrustable professional activities for a family medicine care of the elderly residency program: A focus group study

Entrustable Professional Activities (EPAs) have become widely used within Competency-Based Medical Education (CBME) for the training and evaluation of residents. Little is known about the effectiveness of incorporating multiple stakeholder groups in the validation of EPAs. Here, we seek to validate an EPA framework developed for the University of Manitoba Care of the Elderly Enhanced Skills program using online focus groups consisting of five stakeholder groups. Participants were recruited to take part in one of five online focus groups, one for each stakeholder group (physician faculty, residents, non-physician healthcare professionals, administrators/managers, and patients). Each group met one time for 90 minutes over ZOOM®. The themes arising from stakeholder feedback suggest that successful EPAs must neither be too specific nor too expansive in scope, clearly delineate appropriate means of evaluation, and indicate specific clinical settings in which each EPA should be evaluated. Cross-cutting themes included requiring trainees to collaborate with other professionals when it would optimize patient care, and preparing trainees to advocate for their patients' health (Advocacy). The present study demonstrates that multi-stakeholder analysis yields diverse feedback that can help make EPAs more clear, easier to use in evaluation, and more socially accountable.

"Finding My Piece in That Puzzle": A Qualitative Study Exploring How Medical Students at Four U.S. Schools Envision Their Future Professional Identity in Relation to Health Systems

PURPOSE

Health systems science (HSS) curricula equip future physicians to improve patient, population, and health systems outcomes (i.e., to become "systems citizens"), but the degree to which medical students internalize this conception of the physician role remains unclear. This study aimed to explore how students envision their future professional identity in relation to the system and identify experiences relevant to this aspect of identity formation.

METHOD

Between December 2018 and September 2019, authors interviewed 48 students at 4 U.S. medical schools with HSS curricula. Semistructured interviews were audiorecorded, transcribed, and analyzed iteratively using inductive thematic analysis. Interview questions explored how students understood the health system, systems-related activities they envisioned as future physicians, and experiences and considerations shaping their perspectives.

RESULTS

Most students anticipated enacting one or more systems-related roles as a future physician, categorized as "bottom-up" efforts enacted at a patient or community level (humanist, connector, steward) or "top-down" efforts enacted at a system or policy level (system improver, system scholar, policy advocate). Corresponding activities included attending to social determinants of health or serving medically underserved populations, connecting patients with team members to address systems-related barriers, stewarding health care resources, conducting quality improvement projects, researching/teaching systems topics, and advocating for policy change. Students attributed systems-related aspirations to experiences beyond HSS curricula (e.g., low-income background; work or volunteer experience; undergraduate studies; exposure to systems challenges affecting patients; supportive classmates, faculty, and institutional culture). Students also described future-oriented considerations promoting or undermining identification with systems-related roles (responsibility, affinity, ability, efficacy, priority, reality, consequences).

CONCLUSIONS

This study illuminates systems-related roles medical students at 4 schools with HSS curricula envisioned as part of their future physician identity and highlights past/present experiences and future-oriented considerations shaping identification with such roles. These findings inform practical strategies to support professional identity formation inclusive of systems engagement.

The wolf you feed: Challenging intraprofessional workplace-based education norms

CONTEXT

The trajectory towards becoming a medical professional is strongly situated within the clinical workplace. Through participatory engagement, medical trainees learn to address complex health care issues through collaboration with the interprofessional health care team. To help explain learning and teaching dynamics within the clinical workplace, many scholars have relied on socio-cultural learning theories. In the field of medical education, this research has largely adopted a limited interpretation of a crucial dimension within socio-cultural learning theory: the expert who guides the trainee into the community is almost exclusively from the same profession. We contend that this narrow interpretation is not necessary. This limited focus is one we choose to maintain-be that choice intentional or implicit. In this cross-cutting edge paper, we argue that choosing an interprofessional orientation towards workplace learning and guidance may better prepare medical trainees for their future role in health care practice.

METHODS

By applying Communities of Practice and Landscapes of Practice , and supported by empirical examples, we demonstrate how medical trainees are not solely on a trajectory towards the Community of Physician Practice (CoPP) but also on a trajectory towards various Landscapes of Healthcare Practice (LoHCP). We discuss some of the barriers present within health care organisations and professions that have likely inhibited adoption of the broader LoHCP perspective. We suggest three perspectives that might help to deliberately and meaningfully incorporate the interprofessional learning and teaching dynamic within the medical education continuum.

CONCLUSION

Systematically incorporating Landscapes of Competence, Assessment, and Guidance in workplace-based education-in addition to our current intraprofessional approach-can better prepare medical trainees for their roles within the LoHCP. By advocating and researching this interprofessional perspective, we can embark on a journey towards fully harnessing and empowering the health care team within workplace-based education.

Embracing Multiple Stakeholder Perspectives in Defining Trainee Competence

PURPOSE

To explore how multiple stakeholder groups contribute to the understanding of trainee competence.

METHOD

The authors conducted a constructivist qualitative study in 2015 using focus group discussions to explore the perceptions of different stakeholder groups (patients, nurses/nurse practitioners, supervisors/senior physicians, leaders/administrators, trainees) regarding trainee competence in the emergency department. The authors used a conventional content analysis, a comparative analysis of supervisors'/senior physicians' versus other stakeholders' perspectives, and a directed analysis informed by stakeholder theory to analyze the focus group transcripts.

RESULTS

Forty-six individuals participated in nine focus groups. Four categories of competence were identified: Core Clinical Activities, Patient Centeredness, Aligning Resources, and Code of Conduct. Stakeholders generally agreed in their overall expectations regarding trainee competence. Within individual categories, each stakeholder group identified new considerations, details, and conflicts, which were a replication, elaboration, or complication of a previously identified theme. All stakeholders stressed those aspects of trainee competence that were relevant to their work or values. Trainees were less aware of the patient perspective than that of the other stakeholder groups.

CONCLUSIONS

Considering multiple stakeholder perspectives enriched the description and conceptualization of trainee competence. It also can inform the development of curricula and assessment tools and guide learning about inter- and intradisciplinary conflicts. Further research should explore how trainees' perceptions of value are influenced by their organizational context and, in particular, how trainees adapt their learning goals in response to the divergent demands of key stakeholders.

A Communication Training Program to Encourage Speaking-Up Behavior in Surgical Oncology

Patient safety in the OR depends on effective communication. We developed and tested a communication training program for surgical oncology staff members to increase communication about patient safety concerns. In phase one, 34 staff members participated in focus groups to identify and rank factors that affect speaking-up behavior. We compiled ranked items into thematic categories that included role relations and hierarchy, staff rapport, perceived competence, perceived efficacy of speaking up, staff personality, fear of retaliation, institutional regulations, and time pressure. We then developed a communication training program that 42 participants completed during phase two. Participants offered favorable ratings of the usefulness and perceived effect of the training. Participants reported significant improvement in communicating patient safety concerns (t₄₀ = -2.76, P = .009, d = 0.48). Findings offer insight into communication challenges experienced by surgical oncology staff members and suggest that our training demonstrates the potential to improve team communication.

Lessons From Rocket Science: Reframing the Concept of the Physician Health Advocate

Health advocacy is a prominent component of health professionals' training internationally and is frequently discussed in the medical education literature. Despite this, it continues to be a problematic and challenging topic for medical educators, health professionals, and trainees alike. Borrowing from the field of systems engineering, the authors suggest a need to reconceptualize health advocacy using a systems mind-set rather than a physician-centric perspective. Conceptualizing health advocacy as a systemic, collective effort requires educators, practitioners, and trainees to challenge the assumption that the role of a competent physician health advocate can be fully defined without regard to the larger system or collective within which physicians function. Further, this implies a substantially more dynamic understanding of physicians' and other participants' parts in the collective activity.Of course, this new way of conceptualizing physicians' practices is not limited to health advocacy. The current education paradigm trains physicians for individual competency but expects them to practice collectively. Defining physician competen cies, or the competencies of any health care provider, in isolation from the particular system of which that individual is an integral part implicitly places that health care provider as the central focus of that system. Thus, academic medicine needs to move its educational and research efforts forward in a manner that recognizes that a systems engineering approach to health improvement will allow the various players to maximize their individual efforts to more effectively support the collective activity.

Teaching in the clinical workplace: looking beyond the power of 'the one'

None

Dissonant role perception and paradoxical adjustments: an exploratory study on Medical Residents' collaboration with Senior Doctors and Head Nurses

A good collaboration between health professionals is considered to have benefits for patients, healthcare staff, and organizations. Nevertheless, effective interprofessional collaboration is difficult to achieve. This is particularly true for collaboration between Medical Residents (MRs) and the immediate colleagues they interact with, as Senior Doctors (SDs) and Head Nurses (HNs). Role understanding is one of the factors that may explain difficulties in interprofessional collaboration. Based on this hypothesis, this paper focuses on MRs' role, devoting particular attention to differences in role perception between MRs, SDs, and HNs, and to their consequences for interprofessional collaboration. An exploratory qualitative study inspired by Grounded Theory was conducted in April 2009 in a small peripheral and non-university hospital in Switzerland. Data came from two focus groups with MRs (13), one with SDs (8), and one with HNs (7), and were analyzed using the constant comparative method. Findings show that the expected and the enacted role of MR are perceived differently by SDs, HNs and MRs themselves. To face the inconsistencies within MR's role, the three professional groups develop some adjustments that eventually prove to be paradoxical: on one side, they make collaboration possible and preserve the functioning of the ward, while on the other side they lead to mutual misunderstanding and discontent. These findings suggest that there is an urgent need of defining the role of MRs, of delimiting its boundaries and thereby distinguishing it from other health workers, and eventually of promoting a shared representation of it.

ACGME core competency training, mentorship, and research in surgical subspecialty fellowship programs

OBJECTIVE

To determine the perceived effectiveness of surgical subspecialty training programs in teaching and assessing the 6 ACGME core competencies including research.

DESIGN

Cross-sectional survey.

SETTING

ACGME approved training programs in pediatric urology and colorectal surgery.

PARTICIPANTS

Program Directors and recent trainees (2007-2009).

RESULTS

A total of 39 program directors (60%) and 57 trainees (64%) responded. Both program directors and recent trainees reported a higher degree of training and mentorship (75%) in patient care and medical knowledge than the other core competencies (p<0.0001). Practice based learning and improvement, interpersonal and communication, and professionalism training were perceived effective to a lesser degree. Specifically, in the areas of teaching residents and medical students and team building, program directors, compared with recent trainees, perceived training to be more effective, (p = 0.004, p = 0.04). Responses to questions assessing training in systems based practice ubiquitously identified a lack of training, particularly in financial matters of running a practice. Although effective training in research was perceived as lacking by recent trainees, 81% reported mentorship in this area. According to program directors and recent trainees, the most effective method of teaching was faculty supervision and feedback. Only 50% or less of the recent trainees reported mentorship in career planning, work-life balance, and job satisfaction.

CONCLUSIONS

Not all 6 core competencies and research are effectively being taught in surgery subspecialty training programs and mentorship in areas outside of patient care and research is lacking. Emphasis should be placed on faculty supervision and feedback when designing methods to better incorporate all 6 core competencies, research, and mentorship.

Assessing systems-based practice

The conceptual definition of systems-based practice (SBP) does not easily translate into directly observable actions or behaviors that can be easily assessed. At the Academic Emergency Medicine consensus conference on education research in emergency medicine (EM), a breakout group presented a review of the literature on existing assessment tools for SBP, discussed the recommendations for research tool development during breakout sessions, and developed a research agenda based on this discussion.

Systems-based practice: Summary of the 2010 Council of Emergency Medicine Residency Directors Academic Assembly Consensus Workgroup--teaching and evaluating the difficult-to-teach competencies

OBJECTIVES

The development of robust Accreditation Council for Graduate Medical Education (ACGME) systems-based practice (SBP) training and validated evaluation tools has been generally challenging for emergency medicine (EM) residency programs. The purpose of this paper is to report the results of a consensus workgroup session of the 2010 Council of Emergency Medicine Residency Directors (CORD) Academic Assembly with the following objectives: 1) to discuss current and preferred local and regional methods for teaching and assessing SBP and 2) to develop consensus within the CORD community using the modified Delphi method with respect to EM-specific SBP domains and link these domains to specific SBP educational and evaluative methods.

METHODS

Consensus was developed using a modified Delphi method. Previously described taxonomy generation methodology was used to create a SBP taxonomy of EM domain-specific knowledge, skills, and attitudes (KSA). The steps in the process consisted of: 1) an 11-question preconference survey, 2) a vetting process conducted at the 2010 CORD Academic Assembly, and 3) the development and ranking of domain-specific SBP educational activities and evaluation criteria for the specialty of EM.

RESULTS

Rank-order lists were created for preferred SBP education and evaluation methods. Expert modeling, informal small group discussion, and formal small group activities were considered to be the optimal methods to teach SBP. Kruskal-Wallis testing revealed that these top three items were rated significantly higher than self-directed learning projects and lectures (p = 0.0317). Post hoc test via permutation testing revealed that the difference was significant between expert modeling and formal small group activity (adjusted p = 0.028), indicating that expert modeling was rated significantly higher than formal small group activity. Direct observation methods were the preferred methods for evaluation. Multiple barriers to training and evaluation were elucidated. We developed a consensus taxonomy of domains that were felt to be most essential and reflective of the practice of EM: multitasking, disposition, and patient safety. Learning formats linked to the domains were created and specific examples of local best practices collected. Domain-specific anchors of observable actions for the three domains were created.

CONCLUSIONS

This consensus process resulted in the development of a taxonomy of EM-specific domains for teaching and observable tasks for evaluating SBP. The concept of SBP is interlinked with the other general competencies and difficult to separate. Rather than develop specific SBP evaluation tools to measure the competency directly, SBP competency evaluation should be considered one element of a coordinated effort to teach and evaluate the six ACGME general competencies.

Systems-based practice in graduate medical education: systems thinking as the missing foundational construct

BACKGROUND

Since 2001, residencies have struggled with teaching and assessing systems-based practice (SBP). One major obstacle may be that the competency alone is not sufficient to support assessment. We believe the foundational construct underlying SBP is systems thinking, absent from the current Accreditation Council for Graduate Medical Education competency language.

SUMMARY

Systems thinking is defined as the ability to analyze systems as a whole. The purpose of this article is to describe psychometric issues that constrain assessment of SBP and elucidate the role of systems thinking in teaching and assessing SBP.

CONCLUSION

Residency programs should incorporate systems thinking models into their curricula. Trainees should be taught to understand systems at an abstract level, in order to analyze their own healthcare systems, and participate in quality and patient safety activities. We suggest that a developmental trajectory for systems thinking be developed, similar to the model described by Dreyfus and Dreyfus.

Students learn systems-based care and facilitate system change as stakeholders in a free clinic experience

Systems-based practice (SBP) is rarely taught or evaluated during medical school, yet is one of the required competencies once students enter residency. We believe Texas A&M College of Medicine students learn about systems issues informally, as they care for patients at a free clinic in Temple, TX. The mandatory free clinic rotation is part of the Internal Medicine clerkship and does not include formal instruction in SBP. During 2008-2009, a sample of students (n = 31) on the IMED clerkship's free clinic rotation participated in a program evaluation/study regarding their experiences. Focus groups (M = 5 students/group) were held at the end of each outpatient rotation. Students were asked: "Are you aware of any system issues which can affect either the delivery of or access to care at the free clinic?" Data saturation was reached after six focus groups, when investigators noted a repetition of responses. Based upon investigator consensus opinion, data collection was discontinued. Based upon a content analysis, six themes were identified: access to specialists, including OB-GYN, was limited; cost containment; lack of resources affects delivery of care; delays in care due to lack of insurance; understanding of larger healthcare system and free clinic role; and delays in tests due to language barriers. Medical students were able to learn about SBP issues during free clinic rotations. Students experienced how SBP issues affected the health care of uninsured individuals. We believe these findings may be transferable to medical schools with mandatory free clinic rotations.

Development of the objective structured system-interaction examination

STUDY OBJECTIVE

The purpose of this study was to develop an objective method of evaluating resident competency in systems-based practice.

STUDY DESIGN

Faculty developed a 12-station examination, the Objective Structured System-Interaction Examination (OSSIE), patterned after the Objective Structured Clinical Examinations (OSCEs), to evaluate residents' ability to effectively work within the complex medical system of care. Scenarios consisted of multiple situations, such as patient hand-offs, consultations, complicated discharges, and family meetings, in which residents interacted with simulated professionals, simulated patients, and simulated family members to demonstrate the systems-based skills. Twelve second-year residents participated in the OSSIE.

FINDINGS

Along with the standardized professionals, a faculty member provided the resident with immediate feedback and completed an evaluation form designed specifically to assess systems-based practice. Residents, faculty, and staff evaluated the OSSIE and felt it provided a rich learning experience and was a beneficial means of formative assessment. The residents' third-year learning experiences were adapted to meet their needs, and suggestions were offered for curriculum revision.

DISCUSSION

The OSSIE is unique in that it uses standardized professionals, involves scenarios in a variety of settings, and incorporates current technology, including an electronic health record and a state-of-the-art simulation laboratory, into the examination. Challenges to implementation include faculty time, scheduling of residents, and availability of resources.

CONCLUSION

By using the OSSIE, faculty are able to assess, provide constructive feedback, and tailor training opportunities to improve resident competence in systems-based practice. Reliability and validity of an instrument developed for use with the OSSIE are currently being determined.

Systems-based practice defined: taxonomy development and role identification for competency assessment of residents

PURPOSE

To demonstrate a methodology for coding and taxonomy development and to operationally define residents' competence in systems-based practice (SBP) in terms of observable roles, actions, and behaviors.

METHODS

The Accreditation Council for Graduate Medical Education's (ACGME's) full-text definition of SBP and the 6 discrete expectations it contains were content analyzed. Structured interviews of 88 health care professionals using a variant of focus group interviews called nominal group processes were conducted and qualitatively analyzed to identify the key attributes of SBP. Themes obtained from these 2 procedures were conceptually matched and organized to create a taxonomy of observable SPB behaviors and the SBP domain.

RESULTS

Six general resident roles emerged, under which 35 specific behavioral attributes were subsumed. From the SBP domain specified. Sample SBP items categorized by roles were derived that reflected "in-context" representations of ACGME SBP expectations.

CONCLUSION

Our comprehensive analysis created an operational representation of the SBP competency. The taxonomy development model provides a framework for constructing assessment instrument(s) that could be applied to the other ACGME competencies or complex concepts in medical education.