The link between quality and accreditation of residency programs: the surveyors' perceptions

To prove or improve? Examining how paradoxical tensions shape evaluation practices in accreditation contexts

INTRODUCTION

Although programme evaluation is increasingly routinised across the academic health sciences, there is scant research on the factors that shape the scope and quality of evaluation work in health professions education. Our research addresses this gap, by studying how the context in which evaluation is practised influences the type of evaluation that can be conducted. Focusing on the context of accreditation, we critically examine the types of paradoxical tensions that surface as evaluation-leads consider evaluation ideals or best practices in relation to contextual demands associated with accreditation seeking.

METHODS

Our methods were qualitative and situated within a critical realist paradigm. Study participants were 29 individuals with roles requiring responsibility and oversight on evaluation work. They worked across 4 regions, within 26 academic health science institutions. Data were collected using semi-structured interviews and analysed using framework and matrix analyses.

RESULTS

We identified three overarching themes: (i) absence of collective coherence about evaluation practice, (ii) disempowerment of expertise and (iii) tensions as routine practice. Examples of these latter tensions in evaluation work included (i) resourcing accreditation versus resourcing robust evaluation strategy (performing paradox), (ii) evaluation designs to secure accreditation versus design to spur renewal and transformation (performing-learning paradox) and (iii) public dissemination of evaluation findings versus restricted or selective access (publicising paradox). Sub-themes and illustrative data are presented.

DISCUSSION

Our study demonstrates how the high-stakes context of accreditation seeking surfaces tensions that can risk the quality and credibility of evaluation practices. To mitigate these risks, those who commission or execute evaluation work must be able to identify and reconcile these tensions. We propose strategies that may help optimise the quality of evaluation work alongside accreditation-seeking efforts. Critically, our research highlights the limitations of continually positioning evaluation purely as a method versus as a socio-technical practice that is highly vulnerable to contextual influences.

Residents' perspective on the quality of postgraduate training programs in Pakistan - the good, the bad and the ugly

Objectives:

To assess the satisfaction of trainees towards different attributes of their training programs.

Methods:

This cross-sectional survey was carried out by enrolling trainee doctors currently working in Medical, Surgical, Dental and Allied specialties of the country by sending a validated and piloted questionnaire through email. Data collection was done from 1^st to 31^st January 2021 after taking ethical approval from the concerned authorities. Data was analysed using SPSS v. 19.0.

Results:

A total of 516 completed responses were received from 15 major cities of the country. The overall perceived satisfaction towards clinical skills (42%), teaching skills (31.4%), personal growth and development (23.6%), research (21%) and supervisor’s role (44.2%) were considerably low with the most common causes for non-satisfaction being poor work-life balance (59%), financial instability (54.5%), poor research facilities (53%), poor career guidance (44%) and poor skill development (42.4%) in descending order. Senior years of residency, government and private set-ups, less than four and greater than 13 residents on average with less than three supervisors per department, excessive duty hours and financial instability in-lieu of not doing locums were statistically related to poor satisfaction across majority of the facets of residency as well the overall satisfaction towards training programs.

Conclusion:

There is a tremendous scope for improvement in the recognized and partially acknowledged attributes of our training programs. Yearly feedback surveys involving residents is essential for enlightening the authorities and mitigating the trainees’ grievances.

The impact of external academic accreditation of undergraduate medical program on students' satisfaction

BACKGROUND

The external academic accreditation is a quality assurance and auditing process that focuses on the structure, process, and outcome of the education. It is an interrupting and highly demanding process in terms of effort, time, financial, and human resources. However, it is unclear in the literature how much of these external quality assurance practices impeded in the accreditation processes would reflect on the other end of the learning pathway, including student satisfaction.

METHODS

A retrospective quantitative secondary data analysis, with a before-after comparison research design, was performed to evaluate external accreditation's impact on students' mean satisfaction score within two accreditation cycles at King Saud University (KSU)-Bachelor of Medicine, Bachelor of Surgery (MBBS) program.

RESULTS

The overall average students' satisfaction scores pre-and-post the first accreditation cycle were 3.46/5 (±0.35), 3.71 (±0.39), respectively, with a P-value of < 0.001. The effect of post first accreditation cycle was sustainable for a couple of years, then maintained above the baseline of the pre-first accreditation cycle until the pre-second accreditation cycle. Similarly, the overall average students' satisfaction scores pre-and-post the second accreditation cycles were 3.57/5 (±0.30) and 3.70 (±0.34), respectively, with a P-value of 0.04. Compared to the first accreditation cycle, the improvement of the mean score of students' satisfaction rates was not sustained beyond the year corresponding to the post-second accreditation cycle.

CONCLUSION

Both accreditation cycles were associated with an increased score in students' satisfaction. The preparatory phase activities and navigation through the self-study assessment while challenging the program's competencies are essential triggers for quality improvement practices associated with accreditation.

Clinicians' perspectives on quality: do they match accreditation standards?

BACKGROUND

Quality of training is determined through programs' compliance with accreditation standards, often set for a number of years. However, perspectives on quality of training within these standards may differ from the clinicians' perspectives on quality of training. Knowledge on how standards relate to clinicians' perspectives on quality of training is currently lacking yet is expected to lead to improved accreditation design.

METHODS

This qualitative study design was based on a case-study research approach. We analyzed accreditation standards and conducted 29 interviews with accreditors, clinical supervisors and trainees across Australia and the Netherlands about the quality and accreditation of specialist medical training programs. The perspectives were coded and either if applicable compared to national accreditation standards of both jurisdictions, or thematized to the way stakeholders encounter accreditation standards in practice.

RESULTS

There were two evident matches and four mismatches between the perspectives of clinicians and the accreditation standards. The matches are: (1) accreditation is necessary (2) trainees are the best source for quality measures. The mismatches are: (3) fundamental training aspects that accreditation standards do not capture: the balance between training and service provision, and trainee empowerment (4) using standards lack dynamism and (5) quality improvement; driven by standards or intrinsic motivation of healthcare professionals.

CONCLUSION

In our Australian and Dutch health education cases accreditation is an accepted phenomenon which may be improved by trainee empowerment, a dynamic updating process of standards and by flexibility in its use.

Development and validation of The Breaking Bad News Attitudes Scale

BACKGROUND

Communication of bad news plays a critical role in the physician-patient relationship, and a variety of consensus guidelines have been developed to this purpose, including the SPIKES protocol. However, little is known about physicians' attitudes towards breaking bad news and to be trained to deliver it. This study aimed to develop and validate a self-report questionnaire to assess physicians' attitudes towards principles of the SPIKES protocol and training on them.

METHODS

The Breaking Bad News Attitudes Scale (BBNAS) was administered to 484 pediatricians and 79 medical students, recruited at two scientific conferences and two medical schools in Brazil. The questionnaire structural validity, reliability, and associations with other variables were tested.

RESULTS

The BBNAS showed adequate validity and good reliability, with two factors measuring attitudes towards the SPIKES strategy for braking bad news (α = 0.81) and the possibility to be trained on it (α = 0.77), respectively.

CONCLUSION

The novel questionnaire is a psychometrically sound measure that provides information on physicians' agreement with the SPIKES protocol. The BBNAS can provide useful information for planning training and continuing education programs for clinicians on communication of bad news using the SPIKES as a framework.

The European Council on Chiropractic Education identification of critical standards to accredit chiropractic programs: a qualitative study and thematic analysis

OBJECTIVE

The objectives for this project were to: (1) identify and agree upon "critical standards" that must be "fully" or "substantially compliant" for a maximum 8-year reaccreditation, (2) compare recent compliance for each critical standard for all accredited programs, and (3) identify which standards have lower compliance levels compared to others.

METHODS

This qualitative study uses thematic analysis. The 37 European Council on Chiropractic Education (ECCE) standards were assessed by the Quality Assurance Committee (QAC) to identify "critical standards" requiring "fully" or "substantially compliant" ratings for the maximum 8-year accreditation time period. These were approved by the entire ECCE. A table identifying specific criteria for each compliance level then was created. The most recent evaluation reports for all accredited programs were reviewed to record compliance levels for each critical standard and identify the number falling below the status of "substantially compliant". Specific standards with the highest proportion falling below "substantially compliant" also were identified.

RESULTS

Eighteen of 37 standards were deemed critical. Two of 10 accredited programs had 0 "critical standards" below "substantially compliant" and two programs had three below this level. The most common standard to fall below "substantially compliant" was "faculty recruitment" with three programs (30%) at "partially compliant."

CONCLUSION

Identification and approval of "critical standards" requiring at least substantial compliance and the compliance criteria table facilitate implementation of the flexible 8-year reaccreditation period, providing the flexibility needed to work collaboratively with national accrediting agencies. "Faculty recruitment" standard had the highest percentage of programs rated as "partially" compliant.

Accreditation systems for Postgraduate Medical Education: a comparison of five countries

There is a widespread consensus about the need for accreditation systems for evaluating post-graduate medical education programs, but accreditation systems differ substantially across countries. A cross-country comparison of accreditation systems could provide valuable input into policy development processes. We reviewed the accreditation systems of five countries: The United States, Canada, The United Kingdom, Germany and Israel. We used three information sources: a literature review, an online search for published information and applications to some accreditation authorities. We used template analysis for coding and identification of major themes. All five systems accredit according to standards, and basically apply the same accreditation tools: site-visits, annual data collection and self-evaluations. Differences were found in format of standards and specifications, the application of tools and accreditation consequences. Over a 20-year period, the review identified a three-phased process of evolution-from a process-based accreditation system, through an adaptation phase, until the employment of an outcome-based accreditation system. Based on the five-system comparison, we recommend that accrediting authorities: broaden the consequences scale; reconsider the site-visit policy; use multiple data sources; learn from other countries' experiences with the move to an outcome-based system and take the division of roles into account.