Accreditation of Canadian Undergraduate Medical Education Programs: A Study of Measures of Effectiveness

What are the perceptions of faculty and academic leaders regarding the impact of accreditation on the continuous quality improvement process of undergraduate medical education programs at Caribbean medical schools?

BACKGROUND

Accreditation and regulation are meant for quality assurance in higher education. However, there is no guarantee that accreditation ensures quality improvement. The accreditation for Caribbean medical schools varies from island to island, and it could be mandatory or voluntary, depending on local government requirements. Caribbean medical schools recently attained accreditation status to meet the Educational Commission for Foreign Medical Graduates (ECFMG) requirements by 2024. Literature suggests that accreditation impacts ECFMG certification rates and medical schools' educational processes. However, no such study has examined accreditation's impact on continuous quality improvement (CQI) in medical schools. This study aims to gather the perceptions and experiences of faculty members and academic leaders regarding the impact of accreditation on CQI across Caribbean medical schools.

METHODS

This qualitative phenomenological study inquiries about the perceptions and experiences of faculty and academic leaders regarding accreditation's impact on CQI. Purposive and snowball sampling techniques were used. Participants were interviewed using a semi-structured interview method. Fifteen participants were interviewed across ten Caribbean medical schools representing accredited medical schools, accreditation denied medical schools, and schools that never applied for accreditation. Interviews were audio recorded, and thematic data analysis was conducted.

RESULTS

Thematic analysis yielded six themes, including accreditation and CQI, CQI irrespective of accreditation, faculty engagement and faculty empowerment in the CQI process, collecting and sharing data, ECFMG 2024 requirements, and organizational structure of CQI.

CONCLUSIONS

There is ongoing quality improvement at Caribbean medical schools, as perceived by faculty members and academic leaders. However, most of the change process is happening because of accreditation, and the quality improvement is due to external push such as accreditation rather than internal motivation. It is recommended that Caribbean medical schools promote internal quality improvement irrespective of accreditation and embrace the culture of CQI.

Did the students' satisfaction rates at Avalon University School of Medicine correlate with the occurrence of accreditation site visits?

INTRODUCTION

Accreditation of medical education programs can be observed from different perspectives. Regulatory/accreditation agencies consider it vital to assure a certain level of quality. Other stakeholders may perceive the accreditation process as a negative experience, draining resources, and efforts. Although accreditation may improve the program's governance and administration, its direct or indirect impact on students must be further investigated. This study explores the relationship between the occurrence of accreditation site visits and student satisfaction rates at Avalon University School of Medicine.

METHODS

A comparison study was conducted with retrospective satisfaction data from two accreditation cycles at AUSOM. We used the Caribbean Accreditation Authority for Education in Medicine and Other Health Professions (CAAM-HP) student surveys for data collection, and data from 2017, 2019, and 2022 were used. The response rate was 70% (n = 71), 72% (n = 47), and 60% (n = 56) for basic science students and 80% (n = 111), 82% (n = 115), and 70% (n = 76) for clinical students in 2017, 2019, and 2022, respectively. The survey for basic sciences students included 37 questions/items, and the survey for clinical students included 39 questions/items. The responses for the questionnaire were on the five-point Likert scale. The retrospective data were evaluated using the unpaired Wilcoxon-rank sum test.

RESULTS

The ratings for the basic science students' survey increased from 2017 to 2019 (first accreditation cycle) only for 11 items/questions and they were increased from 2019 to 2022 for all items/questions. The ratings for clinical science students' surveys increased from 2017 to 2019 (the first accreditation cycle) for all items/questions with a statistically significant p-value. They increased for 28 questions/items from 2019 to 2022, and two items (availability and adequacy of career counseling) showed statistically significant p-values.

CONCLUSIONS

The pre-accreditation preparation and the self-evaluation process while correcting the program's deficiencies are essential triggers for the quality improvement process associated with accreditation.

The impact of accreditation on continuous quality improvement process in undergraduate medical education programs: A scoping review

Background

Accreditation in medical education has existed for more than 100 years, yet the impact of accreditation remains inconclusive. Some studies have shown the effects of accreditation on student outcomes and educational processes at medical schools. However, evidence showing the impact of accreditation on continuous quality improvement of undergraduate medical education programs is still in its infancy. This scoping review explores the impact of accreditation on continuous quality improvement (CQI).

Methods

This scoping review followed the methodology of the Preferred Reporting Items of Systematic Reviews and the Meta-Analysis extension for scoping reviews (PRISMA-ScR) checklist outlined by Arksey and O'Malley (2005). Databases, including PubMed, Medline, ERIC, CINHAL, and Google Scholar, were searched to find articles from 2000 to 2022 related to the accreditation of undergraduate medical education programs and continuous quality improvement.

Results

A total of 35 full-text articles were reviewed, and ten articles met our inclusion criteria. The review of the full-text articles yielded four themes: Accreditation and its standards in general, Accreditation and its impact on student outcomes, Accreditation and its impact on medical school's educational processes, Accreditation and CQI. However, the literature evidence suggesting the impact of accreditation on CQI is minimal. The quality assurance approach is based on meeting the standards of accreditation. The quality improvement approach is based on striving for excellence. Literature suggests a requirement to move from student outcomes to CQI measures. CQI requires everyone in the organization to take responsibility and accountability, considering quality as the result of every single step or process and leaders supporting improvements in data collection and data analysis for quality improvement.

Conclusions

The literature on accreditation and CQI are limited in number. More research studies are required to enhance undergraduate medical education accreditation practices' value to medical students, educators, academic leaders, programs, and the public. It was recommended that medical schools embrace the culture and vision perpetuated by the CQI process.

Pilot study exploring the presence of leadership curricula in undergraduate medical education

INTRODUCTION

The physician's role in the healthcare delivery system extends beyond patient care to include numerous critical leadership roles in healthcare. In addition to treating patients, physicians manage multidisciplinary teams and influence policies to optimise public health. Yet, leadership education is not currently emphasised as part of undergraduate medical education. The purpose of this pilot study was to survey US allopathic medical schools about their current leadership curricula for medical students, including aspects that promote or inhibit effective curricular implementation.

METHODS

A survey was developed and administered to representatives at US allopathic medical schools regarding the presence and nature of leadership curricula for medical students currently in place, as well as barriers to effective implementation.

RESULTS

A total of 41 of 144 medical schools (28%) responded to the survey. Of the schools that responded, 25 schools reported the presence of a leadership curriculum. Primary competencies and goals were similar among these institutions, but the structure of the curriculum and how it is being implemented varied. The study did not identify consistent methods of assessing student leadership competencies. Themes of improved communication and integration within the curriculum emerged as actions promoting successful implementation. Financial restraints and buy-in among stakeholder groups (eg, faculty, staff and students) were mentioned most frequently as barriers to the effective implementation of leadership curricula.

CONCLUSIONS

Although leadership education within undergraduate medical education is emphasised by a variety of groups, including the Association of American Medical Colleges, medical schools do not appear to universally address this topic. The Liaison Committee on Medical Education, the primary accrediting body for medical schools, does not currently require leadership education as a core topic for accreditation. To provide effective education on leadership to medical students, administrative bodies could improve their support for this topic by considering the factors identified that both hinder and promote implementation.

First-round accreditation and pass rates on licensing examination at different medical schools in China: Closing the performance gap

BACKGROUND

Although the accreditation approach is widely used to ensure the quality of medical education in many countries, there is scant empirical evidence on whether and how it improves actual medical school performance. We focused on conditions in China, which introduced an accreditation system during the 2010s. Specifically, we examined the relationship between first-round accreditation and actual performance based on the results of medical licensing examinations. Referring to organisation theory, we hypothesised that the impacts of accreditation would depend on existing performance gaps.

METHOD

In 2022, we analysed panel data from 105 Chinese medical schools during accreditation (2012 to 2021) and pass rates on medical licensing examinations (2011 to 2019), as matched into 834 school-year records in a window of years before and after accreditation. We employed fixed-effects regression models with a comparison group to exclude factors that may have confounded the impacts of accreditation time. We also demonstrated the heterogeneous effects of accreditation by tier and performance gap of medical schools.

RESULTS

The conservative estimates showed a substantial cumulative improvement (over 15 percentage points) in pass rates during the years before accreditation, with no clear trend indicating performance drops in the years after accreditation. Lower-tiered medical schools gained greater benefits from accreditation. Medical schools with a larger prior performance gap achieved a greater percentage point increase in pass rates with the passage of time in pre-accreditation years.

CONCLUSIONS

This is the first empirical study to investigate whether accreditation has bridged performance gaps among medical schools. The results support the value of accreditation in China, a country that recently established the system, and might work as a substitute for missing information on early accreditation history in countries with long-established accreditation systems. We encourage more studies in countries that have recently introduced accreditation systems.

Power/knowledge: A sociomaterial perspective on a new accreditation process during COVID-19

INTRODUCTION

The COVID-19 pandemic had significant impacts on many aspects of health care and education, including the accreditation of medical education programmes. As a community of international educators, it is important that we study changes that resulted from the pandemic to help us understand educational processes more broadly. As COVID-19 unfolded in Canada, a revised format of undergraduate medical accreditation was implemented, including a shift to virtual site visits, a two-stage visit schedule, a focused approach to reviewing standards and the addition of a field secretary to the visit team. Our case study research aimed to evaluate the sociomaterial implications of these changes in format on the process of accreditation at two schools.

METHODS

We interviewed key informants to understand the impacts, strengths and limitations of changes made to the accreditation format. We used an abductive approach to analyse transcripts and applied a sociomaterial lens in looking for interconnections between the material and social changes that were experienced within the accreditation system.

RESULTS

Stakeholders within the accreditation system did not anticipate that changes to the accreditation format would have significant impacts on how accreditation functioned or on its overall outcomes. However, key informants described how the revised format of accreditation reconstructed how power was distributed and how knowledge was produced. The revised format contributed to changes in who held power within each of the programmes, within each of the visiting teams and between site members and visiting team members. As power shifted across stakeholders in response to material changes to the accreditation format, key informants described changes in how knowledge was produced.

CONCLUSIONS

Our findings suggest that the most powerful knowledge about any given programme might best be obtained through individualised tools, technologies and voices that are most meaningful to the unique context of each programme. Deliberate attention to how knowledge and power are influenced by the interactions between material and social processes within accreditation may help educators and leaders see the effects of change.

Is West Really Best? The Discourse of Modernisation in Global Medical School Regulation Policy

Phenomenon: In 2012, the World Federation for Medical Education (WFME) established a recognition programme to evaluate medical school regulatory agencies across the world, in response to a new U.S. accreditation policy. Given the predominantly Western origins and Eastern impacts of the WFME programme, this article deconstructs tensions in the programme using postcolonial theory. Approach: Critical discourse analysis examines the intersections of language, knowledge, and power relations to highlight what can or cannot be said about a topic. We employed it to delineate the dominant discourse underpinning the WFME recognition programme. We drew on the theoretical devices of Edward Said, whose work is foundational in postcolonial thinking but has not been widely used in medical education scholarship to date. An archive of literature about the WFME recognition programme dating back to 2003, when WFME first released global standards for medical education, was analyzed. Findings: In the globalization of medical school regulation, the discourse of modernization can be conceptualized as a means of holding knowledge and power in the West, and enacting this power on those in the East, playing on fears of marginalization in the event of non-engagement. The discourse allows these practices to be presented in an honorable and heroic way. Insights: By uncovering the representation of the WFME recognition programme as being modern and modernizing, this article explores how such conceptualisations can close off debate and scrutiny, and proposes further examination of this programme through a lens that recognizes the inherent inequities and geopolitical power differentials that it operates within.

Accreditation Approaches for Professional Education Programs: Towards Best Practice

Accreditation has a central role in the quality assurance of professional education programs, but research on the effectiveness of different models of accreditation is limited. The purpose of this study was to rapidly appraise the evidence for the effectiveness, impact, and feasibility of different accreditation approaches, in order to inform best practices for the accreditation of professional education programs. The study focused on accreditation for programs which produce practice-ready graduates, including for veterinary programs. The authors searched several databases for articles published from 2000 to 2020, using search terms identified during a scoping phase, and applied a "rapid review" methodology in line with contextual, time, and resource requirements. Relevant articles which were classed as empirical or conceptual were included in the study, while papers appraised as solely commentaries or descriptive were excluded from the evidence base. The full-text review included 32 articles. We identified a clear transition in the literature from input- and process-based models (pre- and early 2000s) to outcomes-based models (in the 2000s and early 2010s). Continuous quality improvement and targeted models (including risk-based and thematic) represent more recent approaches in accreditation practice. However, as noted by other scholars, we identified limited empirical evidence for the relative effectiveness of different accreditation approaches in professional education, although evidence for the more recent accreditation approaches is emerging. In terms of best practice in view of the current lack of definitive evidence for the adoption of any specific model of accreditation, we argue that accrediting authorities adopt a contextual approach to accreditation which includes clearly articulating the purpose and focus of their regulatory activities, and selecting and implementing accreditation methods that are consistent with their underlying principles.

The impact of external academic accreditation of undergraduate medical program on students' satisfaction

BACKGROUND

The external academic accreditation is a quality assurance and auditing process that focuses on the structure, process, and outcome of the education. It is an interrupting and highly demanding process in terms of effort, time, financial, and human resources. However, it is unclear in the literature how much of these external quality assurance practices impeded in the accreditation processes would reflect on the other end of the learning pathway, including student satisfaction.

METHODS

A retrospective quantitative secondary data analysis, with a before-after comparison research design, was performed to evaluate external accreditation's impact on students' mean satisfaction score within two accreditation cycles at King Saud University (KSU)-Bachelor of Medicine, Bachelor of Surgery (MBBS) program.

RESULTS

The overall average students' satisfaction scores pre-and-post the first accreditation cycle were 3.46/5 (±0.35), 3.71 (±0.39), respectively, with a P-value of < 0.001. The effect of post first accreditation cycle was sustainable for a couple of years, then maintained above the baseline of the pre-first accreditation cycle until the pre-second accreditation cycle. Similarly, the overall average students' satisfaction scores pre-and-post the second accreditation cycles were 3.57/5 (±0.30) and 3.70 (±0.34), respectively, with a P-value of 0.04. Compared to the first accreditation cycle, the improvement of the mean score of students' satisfaction rates was not sustained beyond the year corresponding to the post-second accreditation cycle.

CONCLUSION

Both accreditation cycles were associated with an increased score in students' satisfaction. The preparatory phase activities and navigation through the self-study assessment while challenging the program's competencies are essential triggers for quality improvement practices associated with accreditation.

Chiropractic program changes facilitated by the European Council on Chiropractic Education Accreditation reports

OBJECTIVE

The European Council on Chiropractic Education (ECCE) currently accredits 10 programs throughout Europe and South Africa. It is assumed that ECCE evaluation activities lead to changes to the chiropractic programs but no systematic evaluation as to whether this is true, and the extent of changes has previously been done. The purpose of this study was to obtain feedback from program heads as to whether ECCE evaluation reports facilitated changes/improvements to their programs and to identify their reported changes.

METHODS

This was a mixed methods audit study using questionnaires with 2 sections. Closed statements requesting the degree of change to each section of the "Standards" based on ECCE evaluation reports (substantial, some, none) were analyzed using frequencies. Written responses identifying the specific changes made based on previous evaluation reports were evaluated independently by 3 researchers using a modified "thematic analysis" approach.

RESULTS

All 10 accredited programs responded. Seven of the 10 programs (70%) reported "some" or "substantial" changes to ≥ 6 sections of the ECCE Standards. The most common section with reported changes was "Educational Program" (8 of 10). "Educational Resources" had the largest number of programs reporting "substantial changes" (4) and was the second most common section to have reported changes. The main themes identified emphasized changes in "infrastructure, equipment and faculty," "increasing evidence-based practice," and "instilling a research culture in faculty and students."

CONCLUSION

ECCE accreditation processes facilitate changes to the chiropractic programs, particularly in the areas of improved infrastructure and faculty, research, and evidence-based practice.

Interprofessional education-relevant accreditation standards in Canada: a comparative document analysis

BACKGROUND

Increasing evidence suggests that sustainable delivery of interprofessional education (IPE) has the potential to lead to interprofessional collaborative practice (IPCP), which in turn has the potential to lead to enhanced healthcare systems and improved patient-centered care health outcomes. To enhance IPE in Canada, the Accreditation of Interprofessional Health Education (AIPHE) project initiated collaborative efforts among accrediting organizations of six health professions to embed IPE language into their respective accreditation standards. To further understand the impact of the AIPHE project, this study evaluated the accountability of the IPE language currently embedded in Canadian health professions' accreditation standards documents and examined whether such language spanned the five accreditation standards domains identified in the AIPHE project.

METHODS

We conducted a comparative content analysis to identify and examine IPE language within the "accountable" statements in the current accreditation standards for 11 Canadian health professions that met our eligibility criteria.

RESULTS AND DISCUSSION

A total of 77 IPE-relevant accountable statements were identified across 13 accreditation standards documents for the 11 health professions. The chiropractic, pharmacy, and physiotherapy documents represented nearly 50% (38/77) of all accountable statements. The accountable statements for pharmacy, dentistry, dietetics, and nursing (registered) spanned across three-to-four accreditation standards domains. The remaining nine professions' statements referred mostly to "Students" and "Educational program." Furthermore, the majority of accreditation standards documents failed to provide a definition of IPE, and those that did, were inconsistent across health professions.

CONCLUSIONS

It was encouraging to see frequent reference to IPE within the accreditation standards of the health professions involved in this study. The qualitative findings, however, suggest that the emphasis of these accountable statements is mainly on the students and educational program, potentially compromising the sustainability and development, implementation, and evaluation of this frequently misunderstood pedagogical approach. The findings and exemplary IPE-relevant accountable statements identified in this paper should be of interest to all relevant stakeholders including those countries, where IPE accreditation is still emerging, as a means to accelerate and strengthen achieving desired educational and health outcomes.