Cost evaluations in health professions education: a systematic review of methods and reporting quality

Cost identification in health professions education: A scoping review

BACKGROUND

Resource scarcity in health professions education (HPE) demands rigorous consideration of costs. Yet, thus far, we have been unable to completely and consistently identify the costs of HPE. To address this knowledge gap and enable use of economic evidence in decision making, a detailed overview of all existing costs and approaches to cost identification in HPE is needed.

OBJECTIVES

This review summarises the diversity of costs, cost perspectives and cost identification methods used in economic research of HPE to answer the following questions: which educational topics, costs, cost perspectives and cost identification methods are being investigated in HPE literature?

METHODS

This investigation followed the Joanna Briggs Institute guidelines for scoping reviews. PubMed, ERIC, CINAHL and PsycINFO were iteratively searched for English language publications between 2012 and 2022 that reported costs of HPE. Data was extracted for study characteristics, educational context and economic methodology.

RESULTS

A total of 136 articles of original research on the costs of educating all major licence healthcare professionals were included. A diffuse interest in costs is reflected in publications from 93 distinct journals and by the diversity of educational topics and cost types explored. However, the majority of investigations failed to define the cost perspective (86, 63%) and cost identification methods (117, 86%) and did not demonstrate common reporting structure or cost terminology, all of which contribute to persistent inconsistencies in cost identification.

CONCLUSION

The source of disharmonious cost identification in HPE is fuelled by diverse interests and divergent concepts of costs and costing methodology. Left in its current state, disharmonious cost identification will continue to limit transparency, comparison and synthesis of evidence, hamper objectivity in implementation and policy decisions and impede the efficient and sustainable allocation of resources. A research agenda must be developed, prioritised, and validated by the broader community to develop a theoretical framework for HPE cost research, define stakeholders, elicit values and preferences and chart a path toward harmonised costing.

Where do we invest money to implement active learning on caries detection? An economic evaluation

This trial-based economic evaluation aimed to estimate the incremental cost of implementing an active learning strategy (theoretical-practical workshop) to substitute the didactic lecture as the sole method for students training in caries detection. We also provided a budget impact analysis and explored the composition of costs related to the activity. Data from the coordinating centre of a multicentre randomized and controlled study (IuSTC01) was analyzed as the first part of our main economic analysis plan. The perspective of the educational provider (the institution implementing the activity) was considered, and an immediate time horizon was adopted. All used resources were valued in Brazilian Real by adopting a microcosting strategy. Costs for each strategy were estimated and converted into international dollars. The incremental Cost per student and the total cost of implementing the complete teaching strategy for 80 students were calculated. Monte Carlo simulations were used to estimate the uncertainties. The incremental Cost estimated for the workshop implementation would be $7.93 per student (interquartile range (IQR): $7.8-8.1), and the total cost of the teaching activity would be $684 (IQR:672-696). The laboratory training comprised more than 50% of the total amount spent, and a higher percentage of this value was related to human resources costs (72%). Saving 40% of the costs could be expected for the next rounds of activities in the institution, assuming no need for additional preparation of didactic materials and tutor training. A modest incremental cost per student and an acceptable organizational budget impact should be expected for the institution when including active learning training in caries detection for undergraduate students, mainly related to the human resources involved.

Assessing methodological quality in dental education research using MERSQI: Analysis of publications from two journals

PURPOSE

The Medical Education Research Study Quality Instrument (MERSQI) has been used frequently to assess the methodological quality of medical education but not for dental education. The present study aimed to assess the methodological quality using MERSQI scores of articles published in the Journal of Dental Education (JDE) and the European Journal of Dental Education (EJDE).

METHODS

A cross-sectional assessment of the quality of manuscripts published in 2012, 2017, and 2022 JDE and EJDE was conducted. MERSQI data, numbers of authors, first and corresponding author degrees, geographic origins, and funding information were also extracted for each included study. Descriptive and analytical statistics were conducted, and significance level was set at α < 0.05.

RESULTS

Four hundred ninety-five articles met the inclusion criteria. The most common study design was a single-group cross-sectional or single-group posttest and conducted in one institution for all studied years. In all journals and years, studies were assessed mainly by participants. The study outcome was mostly satisfaction, attitudes, perceptions, opinions, and general facts. The total mean MERSQI score for each journal and year varied. Year and geographic origin significantly affected the total MERSQI score. Papers originating from Asia had the highest score, followed by South America, Europe, North America, Oceania, and Africa.

CONCLUSION

MERSQI score is applicable to the assessment of the methodological quality of dental educational research. The MERSQI score for most of the domains was similar for both journals. The MERSQI score was affected by publication years and geographic origins.

Effective factors in planning, implementation, and management of educational program evaluation in medical sciences: A practical guide

BACKGROUND

Educational program evaluation is a complex issue, and it is essential to have knowledge of the potential challenges and solutions during the whole process. The present study aimed to identify the influential components in planning, implementation, and evaluation management of educational programs in medical sciences and then provide an applied guide to guarantee the best possible evaluation by evaluators of educational programs.

MATERIALS AND METHODS

This descriptive study was conducted in three steps. First, the effective components in planning, implementation, and evaluation management of educational programs in medical sciences were reviewed. Second, experts' opinion was asked through a focus group discussion regarding the mentioned components. Third, regarding the complied applied guide, the opinions of 40 medical educationist and program evaluation experts were investigated using a checklist.

RESULTS

An applied guide for planning, implementation, and evaluation management of educational programs in medical sciences consists of eight stages: determining the evaluation questions and standards, determining the type of information required, determining resources to collect information, determining methods and tools to collect information, determining data analysis methods, determining the timing and frequency of reporting, determining the appropriate ways of reporting, and determining strategies to maintain the cooperation of data sources.

CONCLUSION

The spread of educational programs in medical sciences universities leads to an increasing need for program evaluation to provide evidence of their effectiveness and improvement. The present research provided an applied guide to make the evaluation of educational programs feasible by using a set of concepts, principles, methods, theories, and models.

Developing discipline-specific online pedagogies for nursing education

BACKGROUND

Many nurse educators have pivoted their teaching to online formats since the COVID-19 pandemic began. Nurse educators face the dilemma that person-centred approaches are particularly challenging to replicate online. Current research provides general recommendations for designing and delivering online learning, but less is known about the usefulness of discipline-specific pedagogies for nursing education.

AIM

This study explores the value of creating discipline-specific pedagogies for online learning in baccalaureate nursing education.

METHOD

Using an action research approach, the authors document their lived experience of designing and delivering a course in two different formats - blended and entirely online. Drawing on existing and new educational models related to online learning, we explore how practice can inform the development of emergent frameworks to guide online education.

RESULTS

Using reflective practice, the authors developed an emergent framework that draws on embodied learning theory to enhance the online delivery of a disability and enablement course for nursing students.

CONCLUSIONS

The concept of embodied pedagogies may offer a starting point for developing guidelines for person-centred and student-centred nursing education online.

Educational and Patient Care Impacts of In Situ Simulation in Healthcare: A Systematic Review

ABSTRACT

This systematic review was performed to assess the effectiveness of in situ simulation education. We searched databases including MEDLINE and Embase for studies comparing in situ simulation with other educational approaches. Two reviewers screened articles and extracted information. Sixty-two articles met inclusion criteria, of which 24 were synthesized quantitatively using random effects meta-analysis. When compared with current educational practices alone, the addition of in situ simulation to these practices was associated with small improvements in clinical outcomes, including mortality [odds ratio, 0.66; 95% confidence interval (CI), 0.55 to 0.78], care metrics (standardized mean difference, -0.34; 95% CI, -0.45 to -0.21), and nontechnical skills (standardized mean difference, -0.52; 95% CI, -0.99 to -0.05). Comparisons between in situ and traditional simulation showed mixed learner preference and knowledge improvement between groups, while technical skills showed improvement attributable to in situ simulation. In summary, available evidence suggests that adding in situ simulation to current educational practices may improve patient mortality and morbidity.

Identifying costs in health professions education: a scoping review protocol

INTRODUCTION

Profound changes in the healthcare sector and the growing, global shortage of healthcare workers are driving the demand to update and scale-up health professions education (HPE). At the same time, educational institutions are faced with increasing resource scarcity and decisions of where and how to allocate resources must be made conscious of costs. However, costs are known to be inaccurately and incompletely identified in HPE research. Therefore, to improve rigour in cost evidence and enable its use in decision-making, it is necessary to have a better understanding of the costs, stakeholders and cost identification methods being used in HPE research. This protocol has been developed to outline the rationale and methods for a scoping review of the literature intended to map the current state of cost and cost identification evidence in HPE.

METHODS AND ANALYSIS

This protocol is developed in accordance with the Joanna Briggs Institute Manual for Evidence Synthesis. PubMed, ERIC, CINAHL and PsycInfo will be scoped for all types of English language publications from 2012 to 2022 that investigate costs associated with the education of health professionals. Two independent reviewers will assess publications for eligibility. Data will be extracted on the educational topic and context, the costs, cost types, perspectives and methods of cost identification as represented in the literature.

ETHICS AND DISSEMINATION

No human participants are involved in this scoping review. All evidence for this study are sourced from public databases; therefore, ethical approval was not required. Final results will be disseminated through peer-reviewed publications and presented at conferences.

TRIAL REGISTRATION NUMBER

A preliminary version of this protocol was prospectively registered on Open Science Framework (OSF) on 13 June 2022 and is available at https://doi.org/10.17605/OSF.IO/ET4XB.

A cost analysis of a 5-day simulation-based learning program for speech-language pathology student training

Purpose: There is poor reporting of the cost of simulation and greater transparency is needed. The primary study aim was to conduct a financial analysis of the university/training institution costs associated with a 5-day simulation-based learning program for speech-language pathology students. The secondary aim was to consider the economic costs of the model.Method: Costs associated with the delivery of a 5-day simulation-based learning program for speech-language pathology students from six Australian universities were collected regarding: (a) pre-program training, (b) personnel, (c) room hire, (d) equipment, and (e) consumables. Both financial costs and economic costs (Australian dollar, at June 2017) were calculated per university site, and per student.Result: The simulation program was run 21 times involving 176 students. Average total financial cost per program ranged from $4717 to $11 425, with cost variation primarily attributed to local labour costs and various use of in-kind support. Average financial cost per student was $859 (range $683-$1087), however this was almost double ($1461 per student, range $857-$2019) in the economic cost calculation. Personnel was the largest contributing cost component accounting for 76.6% of financial costs. Personnel was also the highest contributing cost in the economic analysis, followed by room hire.Conclusion: This study provides clarity regarding financial and economic costing for a 5-day simulation-based learning program. These data can help universities consider potential up-front financial costs, and well as strategies for financial cost minimisation, when implementing simulation-based learning within the university context.

Technology-enhanced simulation in emergency medicine: Updated systematic review and meta-analysis 1991-2021

Background

Over the past decade, the use of technology-enhanced simulation in emergency medicine (EM) education has grown, yet we still lack a clear understanding of its effectiveness. This systematic review aims to identify and synthesize studies evaluating the comparative effectiveness of technology-enhanced simulation in EM.

Methods

We searched MEDLINE, EMBASE, PsycINFO, CINAHL, ERIC, Web of Science, and Scopus to identify EM simulation research that compares technology-enhanced simulation with other instructional modalities. Two reviewers screened articles for inclusion and abstracted information on learners, clinical topics, instructional design features, outcomes, cost, and study quality. Standardized mean difference (SMD) effect sizes were pooled using random effects.

Results

We identified 60 studies, enrolling at least 5279 learners. Of these, 23 compared technology-enhanced simulation with another instructional modality (e.g., living humans, lecture, small group), and 37 compared two forms of technology-enhanced simulation. Compared to lecture or small groups, we found simulation to have nonsignificant differences for time skills (SMD 0.33, 95% confidence interval [CI] -0.23 to 0.89, n = 3), but a large, significant effect for non-time skills (SMD 0.82, 95% CI 0.18 to 1.46, n = 8). Comparison of alternative types of technology-enhanced simulation found favorable associations with skills acquisition, of moderate magnitude, for computer-assisted guidance (compared to no computer-assisted guidance), for time skills (SMD 0.50, 95% CI -1.66 to 2.65, n = 2) and non-time skills (SMD 0.57, 95% CI 0.33 to 0.80, n = 6), and for more task repetitions (time skills SMD 1.01, 95% CI 0.16 to 1.86, n = 2) and active participation (compared to observation) for time skills (SMD 0.85, 95% CI 0.25 to 1.45, n = 2) and non-time skills (SMD 0.33 95% CI 0.08 to 0.58, n = 3).

Conclusions

Technology-enhanced simulation is effective for EM learners for skills acquisition. Features such as computer-assisted guidance, repetition, and active learning are associated with greater effectiveness.

A randomized trial comparing cadaveric dissection and examination of prosections as applied surgical anatomy teaching pedagogies

Anatomy is an important component in the vertical integration of basic science and clinical practice. Two common pedagogies are cadaveric dissection and examination of prosected specimens. Comparative studies mostly evaluate their immediate effectiveness. A randomized controlled trial design was employed to compare both the immediate and long-term effectiveness of dissection and prosection. Eighty third-year medical students undergoing their surgical rotation from the Yong Loo Lin School of Medicine were randomized into two groups: dissection and prosection. Each participated in a one-day hands-on course following a similar outline that demonstrated surgical anatomy in the context of its clinical relevance. A pre-course test was conducted to establish baseline knowledge. A post-course test was conducted immediately after and at a one-year interval to evaluate learner outcome and knowledge retention. A post-course survey was conducted to assess participant perception. Thirty-nine and thirty-eight participants for the dissection and prosection groups, respectively, were included for analysis. There was no significant difference between mean pre-course test scores between the dissection and prosection groups [12.6 (3.47) vs. 12.7 (3.16), P > 0.05]. Both the mean immediate [27.9 (4.30) vs. 24.9 (4.25), P < 0.05] and 1 year [23.9 (4.15) vs. 19.9 (4.05), P < 0.05] post-course test scores were significantly higher in the dissection group. However, when adjusted for course duration [dissection group took longer than prosection group (mean 411 vs. 265 min)], these findings were negated. There is no conclusive evidence of either pedagogy being superior in teaching surgical anatomy. Based on learner surveys, dissection provides a greater learner experience.

Methods of teaching evidence-based practice: a systematic review

BACKGROUND

To identify the effectiveness of different teaching modalities on student evidence-based practice (EBP) competency.

METHODS

Electronic searches were conducted in MEDLINE, Cochrane central register of controlled trials, PsycINFO, CINAHL, ERIC, A + Education and AEI through to November 2021. We included randomised-controlled trials comparing EBP teaching modes on EBP knowledge, skills, attitudes or behaviour in undergraduate and post-graduate health professions education. Risk of bias was determined using the Cochrane risk of bias tool.

RESULTS

Twenty-one studies were included in the review. Overall, no single teaching modality was identified as being superior to others at significantly increasing learner competency in EBP. Changes in learner knowledge, skills, attitudes and behaviour were conflicting, with studies either reporting no change, or a moderate increase in EBP behavioural outcomes when directly compared to another intervention.

CONCLUSION

Current evidence highlights the lack of a single teaching modality that is superior than others regarding learner competency in EBP, regardless of health professions discipline or graduate status. The poor quality, heterogeneity of interventions and outcome measures limited conclusions. Further research should focus on the development of high-quality studies and use of psychometrically validated tools to further explore the impact of different EBP teaching modalities.

Economic evaluation of CPD activities for healthcare professionals: A scoping review

CONTEXT

Continuing professional development (CPD) activities for healthcare professionals are central to the optimisation of patient safety and person-centred care. Although there is some evidence on the economics of healthcare professionals training, very little is known about the costs and benefits of CPD.

METHODS

This study aimed to review the research evidence on economic evaluations of CPD activities for healthcare professionals. CINAHL, MEDLINE/PubMed, Scopus, Econlit and Web of Science databases were used to identify articles published between 2010 and 2021.

RESULTS

Of the 6791 titles identified, 119 articles met the inclusion criteria and were included in this scoping review. The majority of articles were partial economic evaluations of CPD programmes (n = 70); half were from the USA. Studies that included multiple professions were most prevalent (n = 54), followed by nurses (n = 34) and doctors (n = 23). Patient outcomes were the most commonly reported outcome (n = 51), followed by change in clinical practice (n = 38) and healthcare professionals' knowledge gain (n = 19).

CONCLUSIONS

There is an urgent call for more evidence regarding the economic evaluations of CPD. This is particularly important in view of the rising costs of healthcare globally. The majority of studies included in this review did not provide detailed information on the evaluations and many focused exclusively on the cost of CPD activities rather than outcomes.

Costs of Physician Continuous Professional Development: A Systematic Review

PURPOSE

An essential yet oft-neglected step in cost evaluations is the selection of resources (ingredients) to include in cost estimates. The ingredients that most influence the cost of physician continuous professional development (CPD) are unknown, as are the relative costs of instructional modalities. This study's purpose was to estimate the costs of cost ingredients and instructional modalities in physician CPD.

METHOD

The authors conducted a systematic review in April 2020, searching MEDLINE, Embase, PsycInfo, and the Cochrane Library for comparative cost evaluations of CPD for practicing physicians. Two reviewers, working independently, screened articles for inclusion and extracted information on costs (converted to 2021 U.S. dollars) for each intervention overall, each ingredient, and each modality.

RESULTS

Of 3,338 eligible studies, 62 were included, enumerating costs for 86 discrete training interventions or instructional modalities. The most frequently reported ingredients were faculty time (25 of 86 interventions), materials (24), administrator/staff time (23), and travel (20). Ingredient costs varied widely, ranging from a per-physician median of $4 for postage (10 interventions) to $525 for learner time (13); equipment (9) and faculty time were also relatively expensive (median &gt; $170). Among instructional modalities (≤ 11 interventions per modality), audit and feedback performed by physician learners, computer-based modules, computer-based virtual patients, in-person lectures, and experiences with real patients were relatively expensive (median &gt; $1,000 per physician). Mailed paper materials, video clips, and audit and feedback performed by others were relatively inexpensive (median ≤ $62 per physician). Details regarding ingredient selection (10 of 62 studies), quantitation (10), and pricing (26) were reported infrequently.

CONCLUSIONS

Some ingredients, including time, are more important (i.e., contribute more to total costs) than others and should be prioritized in cost evaluations. Data on the relative costs of instructional modalities are insightful but limited. The methods and reporting of cost valuations merit improvement.

Wrestling with the bottom line in medical education

Wiese and Bennett advance the conversation about economic evaluation in health professions education by outlining its potential applications in pratice and research.

Quality of cost evaluations of physician continuous professional development: Systematic review of reporting and methods

INTRODUCTION

We sought to evaluate the reporting and methodological quality of cost evaluations of physician continuing professional development (CPD).

METHODS

We conducted a systematic review, searching MEDLINE, Embase, PsycInfo, and the Cochrane Database for studies comparing the cost of physician CPD (last update 23 April 2020). Two reviewers, working independently, screened all articles for inclusion. Two reviewers extracted information on reporting quality using the Consolidated Health Economic Evaluation Reporting Standards (CHEERS), and on methodological quality using the Medical Education Research Study Quality Instrument (MERSQI) and a published reference case.

RESULTS

Of 3338 potentially eligible studies, 62 were included. Operational definitions of methodological and reporting quality elements were iteratively revised. Articles reported mean (SD) 43% (20%) of CHEERS elements for the Title/Abstract, 56% (34%) for Introduction, 66% (19%) for Methods, 61% (17%) for Results, and 66% (30%) for Discussion, with overall reporting index 292 (83) (maximum 500). Valuation methods were reported infrequently (resource selection 10 of 62 [16%], resource quantitation 10 [16%], pricing 26 [42%]), as were descriptions/discussion of the physicians trained (42 [68%]), training setting (42 [68%]), training intervention (40 [65%]), sensitivity analyses of uncertainty (9 [15%]), and generalizability (30 [48%]). MERSQI scores ranged from 6.0 to 16.0 (mean 11.2 [2.4]). Changes over time in reporting index (initial 241 [105], final 321 [52]) and MERSQI scores (initial 9.8 [2.7], final 11.9 [1.9]) were not statistically significant (p ≥ 0.08).

DISCUSSION

Methods and reporting of HPE cost evaluations fall short of current standards. Gaps exist in the valuation, analysis, and contextualization of cost outcomes.

Unpacking economic programme theory for supervision training: Preliminary steps towards realist economic evaluation

INTRODUCTION

Cost studies are increasingly popular given resource constraints. While scholars stress the context-dependent nature of cost, and the importance of theory, cost studies remain context-blind and atheoretical. However, realist economic evaluation (REE) privileges context and the testing/refinement of economic programme theory. This preliminary REE serves to test and refine economic programme theory for supervision training programmes of different durations to better inform future programme design/implementation.

METHODS

Our preliminary REE unpacked how short (half-day) and extended (12 week) supervision training programmes in Victoria, Australia, produced costs and outcomes. We employed mixed methods: qualitative realist and quantitative cost methods. Economically optimised programme models were developed guided by identified cost-sensitive mechanisms and contexts.

RESULTS

As part of identified context-mechanism-outcome configurations (CMOCs) for both training programmes, we found a wider diversity of positive outcomes but greater costs for the extended programme (11 outcomes; AU$3069/learner) compared with the short programme (7 outcomes; $385/learner). We identified four shared cost-sensitive mechanisms for both programmes (training duration, learner protected time, learner engagement, and facilitator competence) and one shared cost-sensitive context (learners' supervisory experience). We developed two economically optimised programme models: (1) short programme for experienced supervisors facilitated by senior educators ($406/learner); and (2) extended programme for inexperienced supervisors facilitated by junior educators ($2104/learner).

DISCUSSION

Our initial economic programme theory was somewhat supported, refuted and refined. Results were partly consistent with previous research, but also extended it through unpacking cost-sensitive mechanisms and contexts. Although our preliminary REE fills a pressing gap in the methodology literature, conducting REE was challenging given our desire to integrate economic and realist analyses fully, and remain faithful to realist principles. Attention to training duration and experience levels of the facilitator-learner dyad may help to balance the cost and outcomes of training programmes.

Stratification of Health Professional Education and Its Funding Disparities: Evidence From China During the Period of 1998–2017

Background: The finance of health professional education (HPE) is of immense importance for effective and sustainable health systems, yet relevant empirical research was scarce due to the lack of financial data. The study aimed to bridge the gap by presenting the scenario of finance for health professional institutions (HPIs) of different tiers in China and exploring how the stratification of institutions affected their funding disparities.

Methods: The study employed data collected from the Ministry of Education in China, and selected the HPIs mainly based on the World Directory of Medical Schools. The funding levels and disparities of China's HPIs during the period (1998–2017) were analyzed with descriptive statistics, and the indicators of funding per institution and funding per student were both considered. The average funding in HPIs was presented by tiers, and the Gini coefficient and Theil index were employed to describe the differences in financing among HPIs over the span.

Results: The study found that the number of HPIs has kept growing over the past two decades, with both the funding per institution and the funding per student increasing steadily. Specifically, the average funding per institution of the three tiers increased by 31.5 times, 13.4 times, and 10.5 times separately, with the first-tier universities having an absolute advantage compared to lower tiers. As for the financing disparities among HPIs, the Gini coefficient of the funding per institution maintained to be over 0.5, with the third-tier institutions scoring the highest, while the Gini coefficient of the funding per student all ranged approximately from 0.2 to 0.3. Through the decomposition of the inequalities measured by the Theil index, the share of the between-tier difference in per-institution funding grew from 29.7 in 1998 to 77.9% in 2017.

Conclusions: The funding disparities between tiers of HPIs in China gradually became more accentuated, with the top-tier institutions taking up the largest share. Although the stratified development in HPE has posed a challenge to the unified quality assurance of medical personnel training, it may also be regarded as an effective pathway for developing countries like China to achieve stable development in health professional education.

Costs and Economic Impacts of Physician Continuous Professional Development: A Systematic Scoping Review

PURPOSE

Nearly all health care professionals engage in continuous professional development (CPD), yet little is known about the cost and cost-effectiveness of physician CPD. Clarification of key concepts, comprehensive identification of published work, and determination of research gaps would facilitate application of existing evidence and planning for future investigations. The authors sought to systematically map study themes, methods, and outcomes in peer-reviewed literature on the cost and value of physician CPD.

METHOD

The authors conducted a scoping review, systematically searching MEDLINE, Embase, PsycInfo, and Cochrane Library databases for comparative economic evaluations of CPD for practicing physicians through April 2020. Two reviewers, working independently, screened all articles for inclusion. Three reviewers iteratively reviewed all included articles to inductively identify key features including participants, educational interventions, study designs, cost ingredients, and cost analyses. Two reviewers then independently reexamined all included articles to code these features.

RESULTS

Of 3,338 potentially eligible studies, 111 were included. Physician specialties included internal, family, or general medicine (80 studies [72%]), surgery (14 studies [13%]), and medicine subspecialties (7 studies [6%]). Topics most often addressed general medicine (45 studies [41%]) or appropriate drug use (37 studies [33%]). Eighty-seven studies (78%) compared CPD with no intervention. Sixty-three studies (57%) reported the cost of training, and 79 (71%) evaluated the economic impact (money saved/lost following CPD). Training cost ingredients (median 3 itemized per study) and economic impact ingredients (median 1 per study) were infrequently and incompletely identified, quantified, or priced. Twenty-seven studies (24%) reported cost-impact expressions such as cost-effectiveness ratio or net value. Nineteen studies (17%) reported sensitivity analyses.

CONCLUSIONS

Studies evaluating the costs and economic impact of physician CPD are few. Gaps exist in identification, quantification, pricing, and analysis of cost outcomes. The authors propose a comprehensive framework for appraising ingredients and a preliminary reference case for economic evaluations.

Funding of health professional education: China's 20-year process and a global comparison

CONTEXT

Funding is an essential requirement for ensuring the quality of health professional education worldwide. Adequate funding is of immense significance in training health professionals. Due to the difficulty of accessing relevant data, quantitative research of the kind is scarce.

OBJECTIVES

This study aims at analysing the trends of funding levels and funding sources for health professional institutions in China spanning the past 20 years and making a global comparison.

METHODS

We used data from Ministry of Education (MOE) of the People's Republic of China to analyse its funding level and structure of funding sources of health professional education in China during 1998-2017. When analysing funding level, we used two indicators: total funding and funding per student. We chose the United States, and analysed its funding level and structure to allow easy comparison to the situation in China. The data from a Lancet report (Lancet, 376, 2010, 1923) were also used to analyse global funding status to make an international comparison.

RESULTS

Funding levels of health professional institutions in China has increased significantly in the past 20 years, while the average annual growth rate of funding per student (4.5%) is lower than that of total funding (19.9%). In terms of the structure of funding sources, fiscal appropriation accounts for 51.1% on average, and tuition, fees and scientific research income explains 37.0%.

CONCLUSION

From 1998 to 2017, the total funding and funding per student of HPE in China increased continuously, and the total funding increased at a faster rate. The increase in funding of HPE in China is closely related to the efforts of the Chinese government and the implementation of relevant policies. Even so, funding of HPE in China is likely to remain relatively low compared to other countries.

How to conduct cost and value analyses in health professions education: AMEE Guide No. 139

Growing demand for accountability, transparency, and efficiency in health professions education is expected to drive increased demand for, and use of, cost and value analyses. In this AMEE Guide, we introduce key concepts, methods, and literature that will enable novices in economics to conduct simple cost and value analyses, hold informed discussions with economic specialists, and undertake further learning on more advanced economic topics. The practical structure for conducting analyses provided in this guide will enable researchers to produce robust results that are meaningful and useful for improving educational practice. Key steps include defining the economic research question, identifying an appropriate economic study design, carefully identifying cost ingredients, quantifying, and pricing the ingredients consumed, and conducting sensitivity analyses to explore uncertainties in the results.

Opening up the black box of a Gateway to Medicine programme: a realist evaluation

OBJECTIVES

A Gateway to Medicine programme, developed in partnership between a further and higher education setting and implemented to increase the socioeconomic diversity of medicine, was examined to identify precisely what works within the programme and why.

DESIGN

This study employed realist evaluation principles and was undertaken in three phases: document analysis and qualitative focus groups with widening access (WA) programme architects; focus groups and interviews with staff and students; generation of an idea of what works.

SETTING

Participants were recruited from a further/higher education setting and were either enrolled or involved in the delivery of a Gateway to Medicine programme.

PARTICIPANTS

Twelve staff were interviewed either individually (n=3) or in one of three group interviews. Nine focus groups (ranging from 5 to 18 participants in each focus group) were carried out with Gateway students from three consecutive cohorts at 2-3 points in their Gateway programme year.

RESULTS

Data were generated to determine what 'works' in the Gateway programme. Turning a realist lens on the data identified six inter-relating mechanisms which helped students see medicine as attainable and achievable and prepared them for the transition to medical school. These were academic confidence (M1); developing professional identity (M2); financial support/security (M3); supportive relationships with staff (M4) and peers (M5); and establishing a sense of belonging as a university student (M6).

CONCLUSIONS

By unpacking the 'black box' of a Gateway programme through realist evaluation, we have shown that such programmes are not solely about providing knowledge and skills but are rather much more complex in respect to how they work. Further work is needed to further test the mechanisms identified in our study in other contexts for theory development and to identify predictors of effectiveness in terms of students' preparedness to transition.

An Approach to Economic Evaluation in Undergraduate Anatomy Education

Medical education research is becoming increasingly concerned with the value (defined as "educational outcomes per dollar spent") of different teaching approaches. However, the financial costs of various approaches to teaching anatomy are under-researched, making evidence-based comparisons of the value of different teaching approaches impossible. Therefore, the aims of this study were to report the cost of six popular anatomy teaching methods through a specific, yet generalizable approach, and to demonstrate a process in which these results can be used in conjunction with existing effectiveness data to undertake an economic evaluation. A cost analysis was conducted to report the direct and indirect costs of six anatomy teaching methods, using an established approach to cost-reporting. The financial information was then combined with previously published information about the effectiveness of these six teaching methods in increasing anatomy knowledge, thereby demonstrating how estimations of value can be made. Dissection was reported as the most expensive teaching approach and computer aided instruction/learning (CAI/L) was the least, based on an estimation of total cost per student per year and assuming a student cohort size of just over 1,000 (the United Kingdom average). The demonstrated approach to economic evaluation suggested computer aided instruction/learning as the approach that provided the most value, in terms of education outcomes per dollar spent. The study concludes by suggesting that future medical education research should incorporate substantially greater consideration of cost, in order to draw important conclusions about value for learners.

Spending Wisely: The Role of Cost and Value Research in the Pursuit of Advancing Anatomical Sciences Education

Studies of "cost and value" in anatomical sciences education examine not only what works, but at what cost, thus evaluating the inputs and outputs of education. This research provides insights into how to use available resources (e.g., academic time, budgets, infrastructure) as a mechanism to obtaining the maximum outcomes available. The purpose of this viewpoint article is to expand on the application of cost and value concepts to anatomical sciences education, contextualizing these concepts through a deeper dive into the more costly educational approaches of human donor dissection. In doing so, both questions and opportunities are raised for the discipline of anatomical sciences going forward. Educational decisions, inclusive of cost and value appraisals, consider the range of outcomes for which the activity is designed to achieve, and the activity's integration with the philosophy of the educational program it is contributing to; these decisions, thus, evaluate more than just cost alone. Healthcare students' engagement with human donor dissection pedagogy offers an array of reported non-economic benefits, including non-traditional discipline-independent skill (NDIS) development (e.g., professionalism, teamwork skills). These skills are often harder to measure, but are no less important to the final pedagogical decision-making process. The goal of cost and value research is to create an evidence-base toward education that delivers maximum value for a given spend. Anatomy educators, researchers, and decision makers who embrace cost and value dialogue, and interpret and apply findings from studies of educational costs, are best positioned to improve the educational value for their learners and provide effective outputs for all stakeholders.

The Financial Cost of Interprofessional Ambulatory Training: What's the Bottom Line?

BACKGROUND

Team-based care is recommended as a building block of high-performing primary care but has not been widely adapted in training sites. Cost may be one barrier to a team-based approach.

OBJECTIVE

We quantified incremental annual faculty and staff costs as well as potential cost savings associated with an interprofessional (IP) ambulatory training program compared to a traditional residency clinic at the same site.

METHODS

Cost calculations for the 2017-2018 academic year were made using US Department of Labor median salaries by profession and divided by the number of residents trained per year. Cost implications of lower no-show rates were calculated by multiplying the difference in no-show rate by the number of scheduled appointments, and then by the weighted average of the reimbursement rate.

RESULTS

A total of 1572 arrived appointments were seen by the 10 residents in the IP program compared with 8689 arrived appointments seen by 57 residents in the traditional clinic. The no-show rate was 11.5% (265 of 2311) in the IP program and 19.2% (2532 of 13 154) in the traditional clinic (P < .001). Total cost to the health system through higher staffing needs was $113,897, or $11,390 per trained resident.

CONCLUSIONS

Total costs of the IP model due to higher faculty and staff to resident ratios totaled $11,390 per resident per year. Understanding the faculty and staff costs and potential cost-saving opportunities associated with transformation to an IP model may assist in sustainability.

"You can't always get what you want…": economic thinking, constrained optimization and health professions education

Every choice we make in health professions education has a cost, whether it be financial or otherwise; by choosing one action (e.g., integrating more simulation, studying more for a summative examination) we lose the opportunity to take an alternative action (e.g., freeing up time for other teaching, leisure time). Economics significantly shapes the way we behave and think as educators and learners and so there is increasing interest in using economic ways of thinking and approaches to examine and understand how choices are made, the influence of constraints and boundaries in educational decision making, and how costs are felt. Thus, in this article, we provide a brief historical overview of modern economics, to illustrate how the core concepts of economics-scarcity (and desirability), rationality, and optimization-developed over time. We explain the important concept of bounded rationality, which explains how individual, meso-factors and contextual factors influence decision making. We then consider the opportunities that these concepts afford for health professions education and research. We conclude by proposing that embracing economic thinking opens up new questions and new ways of approaching old questions which can add knowledge about how choice is enacted in contemporary health professions education.

Managing the tension: From innovation to application in health professions education

Innovations in education are essential for solving problems and introducing new ways of thinking. However, implementation of these innovations must take several factors into consideration, including the context, the environment, the stakeholders, the technology needed, the cost, the pace of implementation, appropriateness, and available resources. When these factors are not balanced and considered, tensions arise. This paper describes tensions in five major educational domains, namely curriculum, instruction, assessment, accreditation/regulation, and faculty development. For each of these domains, the typical problems are described, and solutions are presented to manage the tension.

Increasing value in research: cost evaluations in health professions education

The authors use Foo et al.'s discussion of the value of economic evaluations to consider how such techniques might advance the practice of selection for medical school.