The impact of accreditation on medical schools' processes

What are the perceptions of faculty and academic leaders regarding the impact of accreditation on the continuous quality improvement process of undergraduate medical education programs at Caribbean medical schools?

BACKGROUND

Accreditation and regulation are meant for quality assurance in higher education. However, there is no guarantee that accreditation ensures quality improvement. The accreditation for Caribbean medical schools varies from island to island, and it could be mandatory or voluntary, depending on local government requirements. Caribbean medical schools recently attained accreditation status to meet the Educational Commission for Foreign Medical Graduates (ECFMG) requirements by 2024. Literature suggests that accreditation impacts ECFMG certification rates and medical schools' educational processes. However, no such study has examined accreditation's impact on continuous quality improvement (CQI) in medical schools. This study aims to gather the perceptions and experiences of faculty members and academic leaders regarding the impact of accreditation on CQI across Caribbean medical schools.

METHODS

This qualitative phenomenological study inquiries about the perceptions and experiences of faculty and academic leaders regarding accreditation's impact on CQI. Purposive and snowball sampling techniques were used. Participants were interviewed using a semi-structured interview method. Fifteen participants were interviewed across ten Caribbean medical schools representing accredited medical schools, accreditation denied medical schools, and schools that never applied for accreditation. Interviews were audio recorded, and thematic data analysis was conducted.

RESULTS

Thematic analysis yielded six themes, including accreditation and CQI, CQI irrespective of accreditation, faculty engagement and faculty empowerment in the CQI process, collecting and sharing data, ECFMG 2024 requirements, and organizational structure of CQI.

CONCLUSIONS

There is ongoing quality improvement at Caribbean medical schools, as perceived by faculty members and academic leaders. However, most of the change process is happening because of accreditation, and the quality improvement is due to external push such as accreditation rather than internal motivation. It is recommended that Caribbean medical schools promote internal quality improvement irrespective of accreditation and embrace the culture of CQI.

Did the students' satisfaction rates at Avalon University School of Medicine correlate with the occurrence of accreditation site visits?

INTRODUCTION

Accreditation of medical education programs can be observed from different perspectives. Regulatory/accreditation agencies consider it vital to assure a certain level of quality. Other stakeholders may perceive the accreditation process as a negative experience, draining resources, and efforts. Although accreditation may improve the program's governance and administration, its direct or indirect impact on students must be further investigated. This study explores the relationship between the occurrence of accreditation site visits and student satisfaction rates at Avalon University School of Medicine.

METHODS

A comparison study was conducted with retrospective satisfaction data from two accreditation cycles at AUSOM. We used the Caribbean Accreditation Authority for Education in Medicine and Other Health Professions (CAAM-HP) student surveys for data collection, and data from 2017, 2019, and 2022 were used. The response rate was 70% (n = 71), 72% (n = 47), and 60% (n = 56) for basic science students and 80% (n = 111), 82% (n = 115), and 70% (n = 76) for clinical students in 2017, 2019, and 2022, respectively. The survey for basic sciences students included 37 questions/items, and the survey for clinical students included 39 questions/items. The responses for the questionnaire were on the five-point Likert scale. The retrospective data were evaluated using the unpaired Wilcoxon-rank sum test.

RESULTS

The ratings for the basic science students' survey increased from 2017 to 2019 (first accreditation cycle) only for 11 items/questions and they were increased from 2019 to 2022 for all items/questions. The ratings for clinical science students' surveys increased from 2017 to 2019 (the first accreditation cycle) for all items/questions with a statistically significant p-value. They increased for 28 questions/items from 2019 to 2022, and two items (availability and adequacy of career counseling) showed statistically significant p-values.

CONCLUSIONS

The pre-accreditation preparation and the self-evaluation process while correcting the program's deficiencies are essential triggers for the quality improvement process associated with accreditation.

The impact of accreditation on continuous quality improvement process in undergraduate medical education programs: A scoping review

Background

Accreditation in medical education has existed for more than 100 years, yet the impact of accreditation remains inconclusive. Some studies have shown the effects of accreditation on student outcomes and educational processes at medical schools. However, evidence showing the impact of accreditation on continuous quality improvement of undergraduate medical education programs is still in its infancy. This scoping review explores the impact of accreditation on continuous quality improvement (CQI).

Methods

This scoping review followed the methodology of the Preferred Reporting Items of Systematic Reviews and the Meta-Analysis extension for scoping reviews (PRISMA-ScR) checklist outlined by Arksey and O'Malley (2005). Databases, including PubMed, Medline, ERIC, CINHAL, and Google Scholar, were searched to find articles from 2000 to 2022 related to the accreditation of undergraduate medical education programs and continuous quality improvement.

Results

A total of 35 full-text articles were reviewed, and ten articles met our inclusion criteria. The review of the full-text articles yielded four themes: Accreditation and its standards in general, Accreditation and its impact on student outcomes, Accreditation and its impact on medical school's educational processes, Accreditation and CQI. However, the literature evidence suggesting the impact of accreditation on CQI is minimal. The quality assurance approach is based on meeting the standards of accreditation. The quality improvement approach is based on striving for excellence. Literature suggests a requirement to move from student outcomes to CQI measures. CQI requires everyone in the organization to take responsibility and accountability, considering quality as the result of every single step or process and leaders supporting improvements in data collection and data analysis for quality improvement.

Conclusions

The literature on accreditation and CQI are limited in number. More research studies are required to enhance undergraduate medical education accreditation practices' value to medical students, educators, academic leaders, programs, and the public. It was recommended that medical schools embrace the culture and vision perpetuated by the CQI process.

Pilot study exploring the presence of leadership curricula in undergraduate medical education

INTRODUCTION

The physician's role in the healthcare delivery system extends beyond patient care to include numerous critical leadership roles in healthcare. In addition to treating patients, physicians manage multidisciplinary teams and influence policies to optimise public health. Yet, leadership education is not currently emphasised as part of undergraduate medical education. The purpose of this pilot study was to survey US allopathic medical schools about their current leadership curricula for medical students, including aspects that promote or inhibit effective curricular implementation.

METHODS

A survey was developed and administered to representatives at US allopathic medical schools regarding the presence and nature of leadership curricula for medical students currently in place, as well as barriers to effective implementation.

RESULTS

A total of 41 of 144 medical schools (28%) responded to the survey. Of the schools that responded, 25 schools reported the presence of a leadership curriculum. Primary competencies and goals were similar among these institutions, but the structure of the curriculum and how it is being implemented varied. The study did not identify consistent methods of assessing student leadership competencies. Themes of improved communication and integration within the curriculum emerged as actions promoting successful implementation. Financial restraints and buy-in among stakeholder groups (eg, faculty, staff and students) were mentioned most frequently as barriers to the effective implementation of leadership curricula.

CONCLUSIONS

Although leadership education within undergraduate medical education is emphasised by a variety of groups, including the Association of American Medical Colleges, medical schools do not appear to universally address this topic. The Liaison Committee on Medical Education, the primary accrediting body for medical schools, does not currently require leadership education as a core topic for accreditation. To provide effective education on leadership to medical students, administrative bodies could improve their support for this topic by considering the factors identified that both hinder and promote implementation.

To prove or improve? Examining how paradoxical tensions shape evaluation practices in accreditation contexts

INTRODUCTION

Although programme evaluation is increasingly routinised across the academic health sciences, there is scant research on the factors that shape the scope and quality of evaluation work in health professions education. Our research addresses this gap, by studying how the context in which evaluation is practised influences the type of evaluation that can be conducted. Focusing on the context of accreditation, we critically examine the types of paradoxical tensions that surface as evaluation-leads consider evaluation ideals or best practices in relation to contextual demands associated with accreditation seeking.

METHODS

Our methods were qualitative and situated within a critical realist paradigm. Study participants were 29 individuals with roles requiring responsibility and oversight on evaluation work. They worked across 4 regions, within 26 academic health science institutions. Data were collected using semi-structured interviews and analysed using framework and matrix analyses.

RESULTS

We identified three overarching themes: (i) absence of collective coherence about evaluation practice, (ii) disempowerment of expertise and (iii) tensions as routine practice. Examples of these latter tensions in evaluation work included (i) resourcing accreditation versus resourcing robust evaluation strategy (performing paradox), (ii) evaluation designs to secure accreditation versus design to spur renewal and transformation (performing-learning paradox) and (iii) public dissemination of evaluation findings versus restricted or selective access (publicising paradox). Sub-themes and illustrative data are presented.

DISCUSSION

Our study demonstrates how the high-stakes context of accreditation seeking surfaces tensions that can risk the quality and credibility of evaluation practices. To mitigate these risks, those who commission or execute evaluation work must be able to identify and reconcile these tensions. We propose strategies that may help optimise the quality of evaluation work alongside accreditation-seeking efforts. Critically, our research highlights the limitations of continually positioning evaluation purely as a method versus as a socio-technical practice that is highly vulnerable to contextual influences.

Enabling Implementation of Competency Based Medical Education through an Outcomes-Focused Accreditation System

Competency based medical education is being adopted around the world. Accreditation plays a vital role as an enabler in the adoption and implementation of competency based medical education, but little has been published about how the design of an accreditation system facilitates this transformation. The Canadian postgraduate medical education environment has recently transitioned to an outcomes-based accreditation system in parallel with the adoption of competency based medical education. Using the Canadian example, we characterize four features of an accreditation system that can facilitate the implementation of competency based medical education: theoretical underpinning, quality focus, accreditation standards, and accreditation processes. Alignment of the underlying educational theories within the accreditation system and educational paradigm drives change in a consistent and desired direction. An accreditation system that prioritizes quality improvement over quality assurance promotes educational system development and progressive change. Accreditation standards that achieve the difficult balance of being sufficiently detailed yet flexible foster a high fidelity of implementation without stifling innovation. Finally, accreditation processes that recognize the change process, encourage program development, and are not overly punitive all enable the implementation of competency based medical education. We also discuss the ways in which accreditation can simultaneously hinder the implementation of this approach. As education bodies adopt competency based medical education, particular attention should be paid to the role that accreditation plays in successful implementation.

First-round accreditation and pass rates on licensing examination at different medical schools in China: Closing the performance gap

BACKGROUND

Although the accreditation approach is widely used to ensure the quality of medical education in many countries, there is scant empirical evidence on whether and how it improves actual medical school performance. We focused on conditions in China, which introduced an accreditation system during the 2010s. Specifically, we examined the relationship between first-round accreditation and actual performance based on the results of medical licensing examinations. Referring to organisation theory, we hypothesised that the impacts of accreditation would depend on existing performance gaps.

METHOD

In 2022, we analysed panel data from 105 Chinese medical schools during accreditation (2012 to 2021) and pass rates on medical licensing examinations (2011 to 2019), as matched into 834 school-year records in a window of years before and after accreditation. We employed fixed-effects regression models with a comparison group to exclude factors that may have confounded the impacts of accreditation time. We also demonstrated the heterogeneous effects of accreditation by tier and performance gap of medical schools.

RESULTS

The conservative estimates showed a substantial cumulative improvement (over 15 percentage points) in pass rates during the years before accreditation, with no clear trend indicating performance drops in the years after accreditation. Lower-tiered medical schools gained greater benefits from accreditation. Medical schools with a larger prior performance gap achieved a greater percentage point increase in pass rates with the passage of time in pre-accreditation years.

CONCLUSIONS

This is the first empirical study to investigate whether accreditation has bridged performance gaps among medical schools. The results support the value of accreditation in China, a country that recently established the system, and might work as a substitute for missing information on early accreditation history in countries with long-established accreditation systems. We encourage more studies in countries that have recently introduced accreditation systems.

Assessing the impact of adopting a competency-based medical education framework and ACGME-I accreditation on educational outcomes in a family medicine residency program in Abu Dhabi Emirate, United Arab Emirates

Background

Competency-Based Medical Education (CBME) is now mandated by many graduate and undergraduate accreditation standards. Evaluating CBME is essential for quantifying its impact, finding supporting evidence for the efforts invested in accreditation processes, and determining future steps. The Ambulatory Healthcare Services (AHS) family medicine residency program has been accredited by the Accreditation Council of Graduate Medical Education-International (ACGME-I) since 2013. This study aims to report the Abu Dhabi program's experience in implementing CBME and accreditation.

Objectives

Compare the two residents' cohorts' performance pre-and post-ACGME-I accreditation.Study the bi-annually reported milestones as a graduating residents' performance prognostic tool.

Methods

All residents in the program from 2008 to 2019 were included. They are called Cohort one-the intake from 2008 to 2012, before the ACGME accreditation, and Cohort two-the intake from 2013 to 2019, after the ACGME accreditation, with the milestones used. The mandatory annual in-training exam was used as an indication of the change in competency between the two cohorts. Among Cohort two ACGME-I, the biannual milestones data were studied to find the correlation between residents' early and graduating milestones.

Results

A total of 112 residents were included: 36 in Cohort one and 76 in Cohort two. In Cohort one, before the ACGME accreditation, no significant associations were identified between residents' graduation in-training exam and their early performance indicators, while in Cohort two, there were significant correlations between almost all performance metrics. Early milestones are correlated with the graduation in-training exam score. Linear regression confirmed this relationship after controlling the residents' undergraduate Grade Point Average (GPA). Competency development continues to improve even after residents complete training at Post Graduate Year, PGY4, as residents' achievement in PGY5 continues to improve.

Conclusion

Improved achievement of residents after the introduction of the ACGME-I accreditation is evident. Additionally, the correlation between the graduation in-training exam and graduation milestones, with earlier milestones, suggests a possible use of early milestones in predicting outcomes.

Power/knowledge: A sociomaterial perspective on a new accreditation process during COVID-19

INTRODUCTION

The COVID-19 pandemic had significant impacts on many aspects of health care and education, including the accreditation of medical education programmes. As a community of international educators, it is important that we study changes that resulted from the pandemic to help us understand educational processes more broadly. As COVID-19 unfolded in Canada, a revised format of undergraduate medical accreditation was implemented, including a shift to virtual site visits, a two-stage visit schedule, a focused approach to reviewing standards and the addition of a field secretary to the visit team. Our case study research aimed to evaluate the sociomaterial implications of these changes in format on the process of accreditation at two schools.

METHODS

We interviewed key informants to understand the impacts, strengths and limitations of changes made to the accreditation format. We used an abductive approach to analyse transcripts and applied a sociomaterial lens in looking for interconnections between the material and social changes that were experienced within the accreditation system.

RESULTS

Stakeholders within the accreditation system did not anticipate that changes to the accreditation format would have significant impacts on how accreditation functioned or on its overall outcomes. However, key informants described how the revised format of accreditation reconstructed how power was distributed and how knowledge was produced. The revised format contributed to changes in who held power within each of the programmes, within each of the visiting teams and between site members and visiting team members. As power shifted across stakeholders in response to material changes to the accreditation format, key informants described changes in how knowledge was produced.

CONCLUSIONS

Our findings suggest that the most powerful knowledge about any given programme might best be obtained through individualised tools, technologies and voices that are most meaningful to the unique context of each programme. Deliberate attention to how knowledge and power are influenced by the interactions between material and social processes within accreditation may help educators and leaders see the effects of change.

Mapping nutrition within medical curricula in Australia and New Zealand: a cross-sectional content analysis

Objective

To systematically map nutrition content in medical curricula across all 23 medical schools in Australia and New Zealand accredited by the Australian Medical Council (AMC).

Methods

A cross-sectional content analysis was conducted. Learning outcomes for 20 AMC-accredited medical curricula were extracted from online repositories and directly from universities in February to April 2021. Nutrition relevant learning outcomes or equivalent learning objectives/graduate attributes were identified. Nutrition learning outcomes were analysed according to Bloom's revised taxonomy to determine whether outcomes met cognitive, psychomotor or affective domains and at what level.

Results

Of the total 23 AMC-accredited medical curricula, 20 medical schools had learning outcomes able to be sourced for analysis. A total of 186 nutrition learning outcomes were identified within 11 medical curricula. One medical school curriculum comprised 129 of 186 (69.4%) nutrition learning outcomes. The majority of outcomes (181, 97.3%) were in the cognitive domain of Bloom's revised taxonomy, predominantly at level 3 'applying' (90, 49.7%). The psychomotor domain contained five nutrition learning outcomes (5, 2.7%), while the affective domain contained none. New Zealand medical curricula (153, 82.3%) contained 4.6-fold more nutrition learning outcomes than Australian curricula (33, 17.7%). When comparing clinical and preclinical years across curricula, the proportion of learning outcomes in the psychomotor domain was 3.7-fold higher in clinical years (4.08%) versus preclinical years (1.15%).

Conclusion

There is wide variation across medical curricula regarding the number of nutrition learning outcomes. This may lead to varying competency of medical graduates to provide nutrition care in Australia and New Zealand.

External assessment of medical education quality: indicative model development considering paradox of skill

This study proposes an approach to the external evaluation of medical education programs' quality based on a combination of indicators, including international rankings, external stakeholders' input, and independent agencies' assessments. We modify the success equation with a detailed consideration of the skill component and its decomposition into internal and external quality assurance elements along with authority. We carried out a bibliometric analysis regarding the problem of medical education quality assessment in the context of achieving sustainable development goals. We described the calculation model of external quality assessment indicators through the algorithms of independent education quality assurance agencies' activity and rating indicators shown in the modified Mauboussin's equation. The model considers the economic component (the consequence of achievement) of skill, which is expressed in raising funds from external sources to implement educational and scientific activities. The proposed algorithm for assessing the educational program quality can be applied to benchmark educational program components, complete educational programs within the subject area, and the educational institution for different areas. We propose a "financial" model for educational program quality based on the analysis results. The model makes it possible to determine the need for additional focused funding of the educational program based on the individual analysis of the external evaluation criteria of the achievement level. This study analyzes the accreditation results of more than 110 educational programs in 2020 and 8 months of 2021 within the direction 22 "Medicine" (according to the national classification of fields of knowledge) (state and private Ukrainian medical universities).

Impact of external accreditation on students' performance: Insights from a full accreditation cycle

Background

The process of external academic accreditation involves quality control and auditing measures that focus on the design, delivery, and outcomes of education. It is a demanding and disruptive process in terms of effort, time, money, and human resources. Nevertheless, the extent to which external quality assurance and accreditation procedures affect students' performance at the end of the learning cycle has not been well studied thus far.

Methods

A retrospective quantitative secondary data analysis was conducted in The King Saud University (KSU) undergraduate medical program, with a before-after comparison research design to assess the impact of external accreditation on students' mean grade scores during an accreditation cycle.

Results

Overall, the data pertaining to 1090 students who attended 32,677 examination encounters were included in the analysis. The pre- and post-accreditation analysis revealed a statistically significant improvement in the students' mean scores-80 ± 9 (pre) versus 87 ± 11 (post), with a p-value of (p = 0.003) and a Cohen's d value of 0.591. On the other hand, there was no statistically significant difference in the students' mean passing percentages-96 ± 5 (pre) versus 96 ± 9 (post), with a p-value of (p = 0.815) and a Cohen's d value of 0.043.

Conclusion

The actions involved in the planning phase and the journey through the self-study evaluation not only verify the program's competencies but also functioned as critical boosters for quality improvement processes and, hence, students' learning experiences.

Altruism or nationalism? Exploring global discourses of medical school regulation

BACKGROUND

Although medical school regulation is ubiquitous, the extent to which it should be based on global principles is unclear. In 2010, the Educational Commission for Foreign Medical Graduates (ECFMG) announced that from 2023, overseas doctors would only be eligible for certification to practise in the United States if they had graduated from a medical school that was accredited by a 'recognised' agency. This policy empowered the World Federation for Medical Education (WFME) to create a recognition programme for regulatory agencies around the world, despite a lack of empirical evidence to support medical school regulation.

METHODS

This study employs critical discourse analysis, drawing on the theoretical perspectives of Michel Foucault and Edward Said, to identify discourses that enabled this 'globalising' policy decision to take place. The dataset includes a series of 250 documents gathered around three key events: the Edinburgh declaration by WFME in 1988, the first set of global standards for medical schools by WFME in 2003 and the ECFMG ruling about medical school accreditation in 2010.

FINDINGS

Two discourses, endorsement and modernisation, were dominant throughout this entire period and framed the move to globalise medical school regulation in terms of altruism and improving medical education worldwide. A discourse of resistance was present in the earlier period of this study but faded away as WFME aligned itself with ECFMG after 2010. Two further discourses, protection and control, emerged in the later period of this study and framed the ECFMG ruling in terms of nationalism and protecting American interests.

DISCUSSION

This study proposes a new conceptualisation of the relationship between ECFMG and WFME in light of the apparently contradictory policy motivations of altruism and nationalism. It goes on to consider the implications of this association for the legitimacy of WFME as an organisation that represents all of the world's medical schools.

Students' perception of three-dimensionally printed teeth in endodontic training

INTRODUCTION

In endodontic education, there is a need for thorough training prior to students embarking on clinical treatment. The aim of this study was to use three-dimensional printing technology to create a new model and to compare its suitability for training purposes with resin blocks and extracted teeth.

MATERIALS AND METHODS

Multi-jet-modelling (MJM) produced the 3D model replicating a common difficulty in root-canal morphology. An evaluation study comprising 88 students was run in the sixth semester (summer 2018 and winter 2018/2019). A new questionnaire assessed students' perception of training models and educational environment. Welch's t-test analysed significant differences.

RESULTS

The most pronounced differences between models were noted when rating material hardness, radiopacity, root-canal configuration and suitability for practising. Students estimated their learning outcome as greater with 3D-printed teeth compared to resin blocks. Three-dimensionally printed teeth received significantly lower ratings with regard to enthusiasm, the learning of fine motor skills and spatial awareness, when compared to human teeth (p ≤ .001). However, 3D-printed teeth were appreciated for additional benefits, such as their cleanliness, availability and standardisation of training opportunities with complex root-canal configurations.

CONCLUSION

Students preferred extracted human teeth to 3D-printed teeth with respect to their physical characteristics and training experience. However, educational advantages may compensate for the shortcomings. The new questionnaire proved both adequate and accurate to assess the models and educational environment in endodontic training. The new 3D-printed teeth enhanced the learning opportunities.

Understanding impacts of accreditation on medical teachers and students: A systematic review and meta-ethnography

PURPOSE

Accreditation is widely used by medical schools around the word to evaluate their curricula and educational processes, although its impacts on those involved in the 'frontline' of medical education receive little attention. This study systematically identified and synthesised qualitative studies that have explored medical teachers' and students' experiences of accreditation.

METHODS

Four databases (Pubmed, EMBASE, ERIC, and PsychINFO) were searched for relevant published articles. Synthesis was performed using meta-ethnography.

RESULTS

Eighteen articles were included in the final synthesis with 1017 individual participants from 10 countries. Findings were categorised into four domains, including navigating power differentials, evaluating credibility, influencing medical programmes, and culture and behaviour. The synthesis demonstrates divergent views on the value of accreditation in medical schools from students and staff including both positive and negative impacts on medical education programmes and stakeholders.

CONCLUSIONS

Although accreditation is perceived to have many benefits, it also has a number of unintended consequences, including on staff morale, student-teacher relationships, and teacher workloads. Medical teachers also have a number of concerns about the credibility of accreditation standards, assessors, and processes. Regulators and policymakers should consider the views of teachers and students as they seek to improve current accreditation practices.

The impact of external academic accreditation of undergraduate medical program on students' satisfaction

BACKGROUND

The external academic accreditation is a quality assurance and auditing process that focuses on the structure, process, and outcome of the education. It is an interrupting and highly demanding process in terms of effort, time, financial, and human resources. However, it is unclear in the literature how much of these external quality assurance practices impeded in the accreditation processes would reflect on the other end of the learning pathway, including student satisfaction.

METHODS

A retrospective quantitative secondary data analysis, with a before-after comparison research design, was performed to evaluate external accreditation's impact on students' mean satisfaction score within two accreditation cycles at King Saud University (KSU)-Bachelor of Medicine, Bachelor of Surgery (MBBS) program.

RESULTS

The overall average students' satisfaction scores pre-and-post the first accreditation cycle were 3.46/5 (±0.35), 3.71 (±0.39), respectively, with a P-value of < 0.001. The effect of post first accreditation cycle was sustainable for a couple of years, then maintained above the baseline of the pre-first accreditation cycle until the pre-second accreditation cycle. Similarly, the overall average students' satisfaction scores pre-and-post the second accreditation cycles were 3.57/5 (±0.30) and 3.70 (±0.34), respectively, with a P-value of 0.04. Compared to the first accreditation cycle, the improvement of the mean score of students' satisfaction rates was not sustained beyond the year corresponding to the post-second accreditation cycle.

CONCLUSION

Both accreditation cycles were associated with an increased score in students' satisfaction. The preparatory phase activities and navigation through the self-study assessment while challenging the program's competencies are essential triggers for quality improvement practices associated with accreditation.

Chiropractic program changes facilitated by the European Council on Chiropractic Education Accreditation reports

OBJECTIVE

The European Council on Chiropractic Education (ECCE) currently accredits 10 programs throughout Europe and South Africa. It is assumed that ECCE evaluation activities lead to changes to the chiropractic programs but no systematic evaluation as to whether this is true, and the extent of changes has previously been done. The purpose of this study was to obtain feedback from program heads as to whether ECCE evaluation reports facilitated changes/improvements to their programs and to identify their reported changes.

METHODS

This was a mixed methods audit study using questionnaires with 2 sections. Closed statements requesting the degree of change to each section of the "Standards" based on ECCE evaluation reports (substantial, some, none) were analyzed using frequencies. Written responses identifying the specific changes made based on previous evaluation reports were evaluated independently by 3 researchers using a modified "thematic analysis" approach.

RESULTS

All 10 accredited programs responded. Seven of the 10 programs (70%) reported "some" or "substantial" changes to ≥ 6 sections of the ECCE Standards. The most common section with reported changes was "Educational Program" (8 of 10). "Educational Resources" had the largest number of programs reporting "substantial changes" (4) and was the second most common section to have reported changes. The main themes identified emphasized changes in "infrastructure, equipment and faculty," "increasing evidence-based practice," and "instilling a research culture in faculty and students."

CONCLUSION

ECCE accreditation processes facilitate changes to the chiropractic programs, particularly in the areas of improved infrastructure and faculty, research, and evidence-based practice.

Impact of accreditation on Caribbean medical schools' processes

BACKGROUND

Caribbean graduates contribute significantly to the US healthcare workforce. The accreditation requirements of local governments vary from one Caribbean island to another island. The Educational Commission for Foreign Medical Graduates (ECFMG) requirement that all future applicants be graduates from accredited medical schools drove Caribbean medical schools to seek accreditation. Accreditation has been found to significantly impact the educational processes of Canadian medical schools. Our study aims at investigating Caribbean medical school leaders' perceptions of the impact of accreditation on their school's processes.

METHODS

This qualitative study and data analysis were done using a framework analysis. Academic leaders and faculty members from three different types of Caribbean medical schools (accredited, denied-accreditation schools, never applied for accreditation) were interviewed using semi-structured interviews.

RESULTS

A total of 12 participants from six different Caribbean medical schools participated in the interview process. Themes of processes influenced by accreditation at Caribbean medical schools were similar to those found in the Canadian context and align with best practices of Continuous Quality Improvement (CQI).

CONCLUSIONS

Caribbean medical schools are changing their educational processes as a result of accreditation requirements. Some processes are not maintained in a continuous manner, raising questions about the development of a true CQI culture.

The World Health Organization and the global standardization of medical training, a history

BACKGROUND

This article presents a history of efforts by the World Health Organization and its most important ally, the World Federation for Medical Education, to strengthen and standardize international medical education. This aspect of WHO activity has been largely ignored in recent historical and sociological work on that organization and on global health generally.

METHODS

Historical textual analysis is applied to the digitalized archives and publications of the World Health Organization and the World Federation for Medical Education, as well as to publications in the periodic literature commenting on the standardization of international medical training and the problems associated with it.

RESULTS

Efforts to reform medical training occurred during three distinct chronological periods: the 1950s and 1960s characterized by efforts to disseminate western scientific norms; the 1970s and 1980s dominated by efforts to align medical training with the WHO's Primary Healthcare Policy; and from the late 1980s to the present, the campaign to impose global standards and institutional accreditation on medical schools worldwide. A growing number of publications in the periodic literature comment on the standardization of international medical training and the problems associated with it, notably the difficulty of reconciling global standards with local needs and of demonstrating the effects of curricular change.

Global trends in medical education accreditation

BACKGROUND

Accreditation systems in medical education aim to assure various stakeholders that graduates are ready to further their training or begin practice. The purpose of this paper is to explore the current state of medical education accreditation around the world and describe the incidence and variability of these accreditation agencies worldwide. This paper explores trends in agency age, organization, and scope according to both World Bank region and income group.

METHODS

To find information on accreditation agencies, we searched multiple online accreditation and quality assurance databases as well as the University of Michigan Online Library and the Google search engine. All included agencies were recorded on a spreadsheet along with date of formation or first accreditation activity, name changes, scope, level of government independence, accessibility and type of accreditation standards, and status of WFME recognition. Comparisons by country region and income classification were made based on the World Bank's lists for fiscal year 2021.

RESULTS

As of August 2020, there were 3,323 operating medical schools located in 186 countries or territories listed in the World Directory of Medical Schools. Ninety-two (49%) of these countries currently have access to undergraduate accreditation that uses medical-specific standards. Sixty-four percent (n = 38) of high-income countries have medical-specific accreditation available to their medical schools, compared to only 20% (n = 6) of low-income countries. The majority of World Bank regions experienced the greatest increase in medical education accreditation agency establishment since the year 2000.

CONCLUSIONS

Most smaller countries in Europe, South America, and the Pacific only have access to general undergraduate accreditation, and many countries in Africa have no accreditation available. In countries where medical education accreditation exists, the scope and organization of the agencies varies considerably. Regional cooperation and international agencies seem to be a growing trend. The data described in our study can serve as an important resource for further investigations on the effectiveness of accreditation activities worldwide. Our research also highlights regions and countries that may need focused accreditation development support.

Developing a community of practice for quality assurance within healthcare assessment

BACKGROUND

The Australian Collaboration for Clinical Assessment in Medicine (ACCLAiM) is a voluntary assessment consortium, involving medical schools nationwide. The aims of ACCLAiM are to benchmark student clinical assessment outcomes and to provide quality assurance (QA) of exit-level Objective Structured Clinical Exams (OSCEs). This study aimed to evaluate the impact of the ACCLAiM QA process for optimising OSCE delivery standards at the member schools using a Community of Practice (CoP) framework.

METHODS

A mixed methods sequential explanatory design, involving an online questionnaire and subsequent focus group discussions, was utilised. Questionnaire responses were analysed using descriptive statistics, while thematic analysis was employed for the qualitative data.

RESULTS

Data analysis revealed that school-specific OSCE practices had evolved based on QA feedback, as well as a collaborative sharing of expertise consistent with a CoP model. Extending beyond a QA working group for accountability and demonstration of minimum standards, participation in ACCLAiM QA processes is creating a sustainable socio-academic network focused on quality improvement.

CONCLUSION

Collaborative QA in clinical assessment creates opportunities for optimising standards in OSCE processes and sharing of resources for OSCE assessments. It also allows for professional development and scholarly engagement in assessment research. These benefits contribute to the existence of an emergent CoP model.

The Relationship Between Accreditation Cycle and Licensing Examination Scores: A National Look

PURPOSE

Accreditation aims to ensure all training programs meet agreed-upon standards of quality. The process is complex, resource intensive, and costly. Its benefits are difficult to assess because contextual confounds obscure comparisons between systems that do and do not include accreditation. This study explores accreditation's influence "within system" by investigating the relationship between accreditation cycle and performance on a national licensing examination.

METHOD

Scores on the computer-based portion of the Medical Council of Canada Qualifying Examination Part I, from 1993 to 2017, were examined for all 17 Canadian medical schools. Typically completed upon graduation from medical school, results within each year were transformed for comparability across administrations and linked to timing within each school's accreditation cycle. ANOVAs were used to assess the relationship between accreditation timing and examination scores. Secondary analyses isolated 4-year from 3-year training programs and separated data generated before versus after implementation of a national midcycle informal review program.

RESULTS

Performance on the licensing exam was highest during and shortly after an accreditation site visit, falling significantly until the midpoint in the accreditation cycle (d = 0.47) before rising again. This pattern disappeared after introduction of informal interim review, but too little data have accumulated post implementation to determine if interim review is sufficient to break the influence of accreditation cycle.

CONCLUSIONS

Formal, externally driven, accreditation cycles appear associated with educational processes in ways that translated into student outcomes on a national licensing examination. Whether informal, internal, interim reviews can mediate this effect remains to be seen.

Is accreditation in medical education in Korea an opportunity or a burden?

The accreditation process is both an opportunity and a burden for medical schools in Korea. The line that separates the two is based on how medical schools recognize and utilize the accreditation process. In other words, accreditation is a burden for medical schools if they view the accreditation process as merely a formal procedure or a means to maintain accreditation status for medical education. However, if medical schools acknowledge the positive value of the accreditation process, accreditation can be both an opportunity and a tool for developing medical education. The accreditation process has educational value by catalyzing improvements in the quality, equity, and efficiency of medical education and by increasing the available options. For the accreditation process to contribute to medical education development, accrediting agencies and medical schools must first be recognized as partners of an educational alliance working together towards common goals. Secondly, clear guidelines on accreditation standards should be periodically reviewed and shared. Finally, a formative self-evaluation process must be introduced for institutions to utilize the accreditation process as an opportunity to develop medical education. This evaluation system could be developed through collaboration among medical schools, academic societies for medical education, and the accrediting authority.

The role of accreditation in 21st century health professions education: report of an International Consensus Group

BACKGROUND

Accreditation is considered an essential ingredient for an effective system of health professions education (HPE) globally. While accreditation systems exist in various forms worldwide, there has been little written about the contemporary enterprise of accreditation and even less about its role in improving health care outcomes. We set out to 1) identify a global, contemporary definition of accreditation in the health professions, 2) describe the relationship of educational accreditation to health care outcomes, 3) identify important questions and recurring issues in twenty-first century HPE accreditation, and 4) propose a framework of essential ingredients in present-day HPE accreditation.

METHODS

We identified health professions accreditation leaders via a literature search and a Google search of HPE institutions, as well as by accessing the networks of other leaders. These leaders were invited to join an international consensus consortium to advance the scholarship and thinking about HPE accreditation. We describe the consensus findings from the International Health Professions Accreditation Outcomes Consortium (IHPAOC).

RESULTS

We define accreditation as the process of formal evaluation of an educational program, institution, or system against defined standards by an external body for the purposes of quality assurance and enhancement. In the context of HPE, accreditation is distinct from other forms of program evaluation or research. Accreditation can enhance health care outcomes because of its ability to influence and standardize the quality of training programs, continuously enhance curriculum to align with population needs, and improve learning environments. We describe ten fundamental and recurring elements of accreditation systems commonly found in HPE and provide an overview of five emerging developments in accreditation in the health professions based on the consensus findings.

CONCLUSIONS

Accreditation has taken on greater importance in contemporary HPE. These consensus findings provide frameworks of core elements of accreditation systems and both recurring and emerging design issues. HPE scholars, educators, and leaders can build on these frameworks to advance research, development, and operation of high-quality accreditation systems worldwide.

Differences between medical school and PGY1 learning outcomes: An explanation for new graduates not being "work ready"?

Background: Widespread concerns about new medical graduates' 'work readiness' may reflect, in part, differences in mandatory learning outcomes for medical students and new medical graduates.Purpose: To examine differences between required medical student and PGY1 (first year resident) training program outcomes, and the nature and magnitude of these differences.Method: Comparison, systematic identification and thematic analysis of differences between the graduate outcomes in the Australian Medical Council Standards for the Assessment and Accreditation of Primary Medical Programs and those in the New Zealand Curriculum Framework for Prevocational Training.Results: The relationship between these outcome statements were categorized as: essentially similar; continuity; partial discontinuity; and complete discontinuity of learning trajectory. Areas requiring substantial new learning may reflect medical schools' focus on individual student performance, and on learning and assessments based on single episodes of often uncomplicated illness. This contrasted with a post-graduate focus on integrated health care delivery by teams and management of complex illnesses over the whole patient care journey.Conclusions: Characterizing these marked differences between pre-graduate and postgraduate standards, within a trajectory of learning, explains some of the difficulties in students' preparation for work readiness. These could inform learning interventions to support new graduates' professional development to ensure patient safety. Development and revision of accreditation standards should include formal review against the expectations of the preceding and succeeding phases of learning.

Curriculum Changes and Trends 2010-2020: A Focused National Review Using the AAMC Curriculum Inventory and the LCME Annual Medical School Questionnaire Part II

Medical school curricula have evolved from 2010 to 2020. Numerous pressures and influences affect medical school curricula, including those from external sources, academic medical institutions, clinical teaching faculty, and undergraduate medical students. Using data from the AAMC Curriculum Inventory and the LCME Annual Medical School Questionnaire Part II, the nature of curriculum change is illuminated. Most medical schools are undertaking curriculum change, both in small cycles of continuous quality improvement and through significant change to curricular structure and content. Four topic areas are explored: cost consciousness, guns and firearms, nutrition, and opioids and addiction medicine. The authors examine how these topic areas are taught and assessed, where in the curriculum they are located, and how much time is dedicated to them in relation to the curriculum as a whole. When examining instructional methods overall, notable findings include (1) the decrease of lecture, although lecture remains the most used instructional method, (2) the increase of collaborative instructional methods, (3) the decrease of laboratory, and (4) the prevalence of clinical instructional methods in academic levels 3 and 4. Regarding assessment methods overall, notable findings include (1) the recent change of the USMLE Step 1 examination to a pass/fail reporting system, (2) a modest increase in narrative assessment, (3) the decline of practical labs, and (4) the predominance of institutionally developed written/computer-based examinations and participation. Among instructional and assessment methods, the most used methods tend to cluster by academic level. It is critical that faculty development evolves alongside curricula. Continued diversity in the use of instructional and assessment methods is necessary to adequately prepare tomorrow's physicians. Future research into the life cycle of a curriculum, as well optional curriculum content, is warranted.

Fostering trust, collaboration, and a culture of continuous quality improvement: A call for transparency in medical school accreditation

Medical schools provide the foundation for a physician's growth and lifelong learning. They also require a large share of government resources. As such, they should seek opportunities to maintain trust from the public, their students, faculty, universities, regulatory colleges, and each other. The accreditation of medical schools attempts to assure stakeholders that the educational process conforms to appropriate standards and thus can be trusted. However, accreditation processes are poorly understood and the basis for accrediting authorities' decisions are often opaque. We propose that increasing transparency in accreditation could enhance trust in the institutions that produce society's physicians. While public reporting of accreditation results has been established in other jurisdictions, such as Australia and the United Kingdom, North American accrediting bodies have not yet embraced this more transparent approach. Public reporting can enhance public trust and engagement, hold medical schools accountable for continuous quality improvement, and can catalyze a culture of collaboration within the broader medical education ecosystem. Inviting patients and the public to peer into one of the most formative and fundamental parts of their physicians' professional training is a powerful tool for stakeholder and public engagement that the North American medical education community at large has yet to use.

Accreditation of Canadian Undergraduate Medical Education Programs: A Study of Measures of Effectiveness

PURPOSE

Undergraduate medical education (UME) programs participate in accreditation with the belief that it contributes to improving UME quality and, ultimately, patient care. Linkages between accreditation and UME quality are incomplete. Previous studies focused on student performance on national examinations, medical school processes, medical school's organizational culture types, and degree of implementation of quality improvement activities as markers of the effectiveness of accreditation. The current study sought to identify new indicators of accreditation effectiveness, to better understand the value and impact of accreditation.

METHOD

This qualitative study used an expert-oriented evaluation approach to identify novel markers of accreditation effectiveness. From March 2015 to March 2016, leaders and teachers at 16 of the 17 Canadian UME programs were invited to participate in interviews and focus group discussions aimed at identifying measures of accreditation effectiveness. Themes were extracted using the method of constant comparative analysis.

RESULTS

Sixty-three individuals from 13 (81%) medical schools participated. Eight themes were formulated: Student/graduate performance, UME program processes, quality assurance and continuous quality improvement, stakeholder satisfaction, stakeholder expectations, engagement, research, and UME program quality. The latter 5 themes have not been previously studied as measures of accreditation effectiveness. All themes appear applicable to accreditation of graduate medical education as well. A framework is proposed to guide future research on the impact of accreditation.

CONCLUSIONS

Eight themes were generated, representing direct and indirect indicators of the impact of accreditation. The themes are integrated into a framework proposed to guide future research on the value of accreditation along the continuum of medical education.

Realist evaluation of UK medical education quality assurance

OBJECTIVES

The aim of the study was to explore what components of the General Medical Council's (GMC) Quality Assurance Framework work, for whom, in what circumstances and how?

SETTING

UK undergraduate and postgraduate medical education and training.

PARTICIPANTS

We conducted interviews with a stratified sample of 36 individuals. This included those who had direct experiences, as well as those with external insights, representing local, national and international organisations within and outside medicine.

INTERVENTION

The GMC quality assure education to protect patient and public safety utilising complex intervention components including meeting standards, institutional visits and monitoring performance. However, the context in which these are implemented matters. We undertook an innovative realist evaluation to test an initial programme theory. Data were analysed using framework analysis.

RESULTS

Across components of the intervention, we identified key mechanisms, including transparent reporting to promote quality improvement; dialogical feedback; partnership working facilitating interactions between regulators and providers, and role clarity in conducting proportionate interventions appropriate to risk. The GMC's framework was commended for being comprehensive and enabling a broad understanding of an organisation's performance. Unintended consequences included confusion over roles and boundaries in different contexts which often undermined effectiveness.

CONCLUSIONS

This realist evaluation substantiates the literature and reveals deeper understandings about quality assuring medical education. While standardised approaches are implemented, interventions need to be contextually proportionate. Routine communication is beneficial to verify data, share concerns and check risk; however, ongoing partnership working can foster assurance. The study provides a modified programme theory to explicate how education providers and regulators can work more effectively together to uphold education quality, and ultimately protect public safety. The findings have influenced the GMC's approach to quality assurance which impacts on all medical students and doctors in training.

Advancing the Quality of Medical Education Worldwide: ECFMG's 2023 Medical School Accreditation Requirement

Certification by the Educational Commission for Foreign Medical Graduates (ECFMG®) is required for international medical graduates (IMGs) to enter U.S. graduate medical education (GME). As a gatekeeper to the U.S. health care system, ECFMG has a duty to verify that these individuals have met minimum standards for undergraduate medical education. Historically, ECFMG has focused on evaluating individual graduates, not medical schools. However, in response to the rapid growth of medical schools around the world and increasing physician migration, ECFMG decided in 2010 to institute medical school accreditation as a future requirement for ECFMG certification. More specifically, beginning in 2023, individuals applying for ECFMG certification will be required to be a student or graduate of a medical school that is accredited by an agency recognized by the World Federation for Medical Education (WFME). By requiring accreditation by an agency that has met WFME's standards, ECFMG seeks to improve the quality, consistency and transparency of undergraduate medical education worldwide. The 2023 Medical School Accreditation Requirement is intended to stimulate global accreditation efforts, increase the information publicly available about medical schools, and provide greater assurance to medical students, regulatory authorities, and the public that these future physicians will be appropriately educated.

Describing the Evidence Base for Accreditation in Undergraduate Medical Education Internationally: A Scoping Review

PURPOSE

To summarize the state of evidence related to undergraduate medical education (UME) accreditation internationally, describe from whom and where the evidence has come, and identify opportunities for further investigation.

METHOD

The authors searched Embase, ERIC, PubMed, and Scopus from inception through January 31, 2018, without language restrictions, to identify peer-reviewed articles on UME accreditation. Articles were classified as scholarship if all Glassick's criteria were met and as nonscholarship if not all were met. Author, accrediting agency, and study characteristics were analyzed.

RESULTS

Database searching identified 1,379 nonduplicate citations, resulting in 203 unique, accessible articles for full-text review. Of these and with articles from hand searching added, 36 articles were classified as scholarship (30 as research) and 85 as nonscholarship. Of the 36 scholarship and 85 nonscholarship articles, respectively, 21 (58%) and 44 (52%) had an author from the United States or Canada, 8 (22%) and 11 (13%) had an author from a low- or middle-income country, and 16 (44%) and 43 (51%) had an author affiliated with a regulatory authority. Agencies from high-income countries were featured most often (scholarship: 28/60 [47%]; nonscholarship: 70/101 [69%]). Six (17%) scholarship articles reported receiving funding. All 30 research studies were cross-sectional or retrospective, 12 (40%) reported only analysis of accreditation documents, and 5 (17%) attempted to link accreditation with educational outcomes.

CONCLUSIONS

Limited evidence exists to support current UME accreditation practices or guide accreditation system creation or enhancement. More research is required to optimize UME accreditation systems' value for students, programs, and society.

Quality improvement in medical schools: vision meets culture

CONTEXT

Medical schools face growing pressures to develop quality improvement (QI) strategies to ensure the continuous quality of their education. To best achieve quality, both organisational processes and culture need to be oriented towards quality. Quality processes already exist at medical schools, at least externally driven by accreditation. However, the dominant culture in most medical schools is not typically oriented towards quality.

OBJECTIVES

This study explores whether QI practices are recognised as such in organisations not culturally QI-oriented. Specifically, it examines faculty members' perceptions about the degree of QI implementation in their medical education programmes. Understanding this perception will inform medical school leadership on how best to use resources for a successful execution of the school's QI vision.

METHODS

Leaders, clinical teachers and formal teachers at 16 of the 17 Canadian medical schools were invited to complete the 'Are We Making Progress?' questionnaire of the Malcolm Baldrige National Quality Award framework, the results of which have been broadly validated. The questionnaire measures the perceived level of QI implementation within organisations using 40 statements grouped under the framework's seven domains of performance excellence.

RESULTS

A total of 491 respondents from 11 (69%) schools completed the questionnaire; 173 (35%) identified as clinical teachers, 150 (31%) as formal teachers, and 168 (34%) as leaders. Perceived QI implementation levels were low across programmes (0.70-1.90 in Rasch person measures) and for each category of respondents. This was especially true for the domains of 'Strategy', 'Measurement/analysis/knowledge management' and 'Operations'. Leaders' perceptions of QI implementation were higher than those of teachers.

CONCLUSIONS

Medical schools' existing QI processes are not recognised as QI activities. For QI strategies to succeed, a programme's culture must embrace QI. In the execution of their QI visions, medical schools should spend resources on embedding quality in the organisation culture in addition to strengthening existing QI practices, especially in the domains listed above.

The Positive Effects of Accreditation on Graduate Medical Education Programs in Singapore

BACKGROUND

There is worldwide interest in assessing the impact of accreditation systems to quantify their benefits to medical education and, through this, health care at the local and global levels.

OBJECTIVE

We analyzed ACGME-I Resident Survey data from Singapore for 2011-2018 to assess the impact of accreditation on residents' evaluations of their programs.

METHODS

We focused on 7 questions from the annual Resident Survey, which would be affected by accreditation compliance, along with a single global rating of respondents' overall perception of their program. We assessed for differences among specialty groupings (medical, surgical, and hospital-based) and Singapore's 3 health care systems. Repeated measures analysis of variance procedures was used to assess trends across time for the combined 8 items and each individual item.

RESULTS

Analysis of the combined items showed significant improvement over the 7 years Singaporean programs had accreditation. There were no effects for specialty type or sponsoring institution. Analyses of individual questions showed 6 of 8 were significant for improvement. For the individual question related to duty hour compliance, there was a significant interaction between time and specialty, suggesting medical specialties showed greater improvement across time compared to surgical and hospital-based specialties.

CONCLUSIONS

Implementation of accreditation in Singapore provided educational and clinical learning environment infrastructure not present prior to 2010, with the benefits of this reflected in residents' perceptions of their learning environment. Future assessments of the effects of accreditation might add stakeholder interviews to more fully describe its value and impact.

A Review of Continuous Quality Improvement Processes at Ten Medical Schools

The Liaison Committee on Medical Education now expects all allopathic medical schools to develop and adhere to a documentable continuous quality improvement (CQI) process. Medical schools must consider how to establish a defensible process that monitors compliance with accreditation standards between site visits. The purpose of this descriptive study is to detail how ten schools in the Association of American Medical Colleges' (AAMC) Southern Group on Educational Affairs (SGEA) CQI Special Interest Group (SIG) are tackling practical issues of CQI development including establishing a CQI office, designating faculty and staff, charging a CQI committee, choosing software for data management, if schools are choosing formalized CQI models, and other considerations. The information presented is not meant to certify that any way is the correct way to manage CQI, but simply present some schools' models. Future research should include defining commonalities of CQI models as well as seeking differences. Furthermore, what are components of CQI models that may affect accreditation compliance negatively? Are there "worst practices" to avoid? What LCME elements are most commonly identified for CQI, and what are the successes and struggles for addressing those elements? What are identifiable challenges relating to use of standard spreadsheet software and engaging information technology for support? How can students be more engaged and involved in the CQI process? Finally, how do these major shifts to a formalized CQI process impact the educational experience?