Quality of reporting of experimental studies in medical education: a systematic review

Quality assessment of paediatric education and research in southwest China: A cross-sectional study

Currently, there is a limited analysis of medical education research both domestically and internationally. To enhance, improve the quality of medical education, this study conducted a quantitative analysis of teaching project data from an affiliated hospital during the period 2016-2022. The results indicated that a total of 133 teaching projects were initiated during this period, with an average age of project leaders being 42.73±6.45 years. Regarding professional ranks and titles, municipal-level project leaders had a high concentration of seniors (48.15%), while at the university-level, most project leaders held the title of deputy seniors (58.82%). At the university-level, project leaders were mainly distributed between deputy senior titles (37.08%) and intermediate titles (38.20%). In terms of research content, nearly half of the studies (46.62%) focused on teaching methods and models. Further regression analysis revealed that professional ranks and titles were an independent factor influencing the project level (P<0. 05). These findings suggest the need for improvement in medical education research, including addressing the uneven distribution of research topics, enhancing the research capacity of junior and mid-career medical education teachers, and improving the dissemination of research results.

Assessing the Landscape in Medical Education Literature in Medical Oncology: A Scoping Review

PURPOSE

Medical oncology and medical education (ME) have both expanded exponentially over the past 50 years; thus, it is important to understand the current status of postgraduate medical oncology education and develop ways to advance this field. This study undertakes a scoping review of ME literature in medical oncology to inform future scholarship in this area.

METHODS

MEDLINE, Embase, ERIC, and Web of Science were searched to find peer-reviewed English language articles on postgraduate ME in medical oncology published from 2009 to 2020. Established scoping review methodologies were used in study design; articles were classified by specialty, learner training level, region of authorship, institution type, year of publication, journal type, study methodology, and research topic. Curriculum intervention, scholarship, and domain(s) of physician competency were also assessed. The results were interpreted using descriptive statistics and collated using predetermined conceptual frameworks.

RESULTS

A total of 2,959 references were initially found across four databases. After title and abstract screening, 305 articles remained; after full-text review, 144 articles were included in final analysis. Postgraduate medical oncology education research is increasing, with the majority of articles published in North America. Quantitative studies were most common, primarily survey approaches. For physician competencies, professionalism and medical expertise comprised the large majority of article focuses, whereas very few articles addressed leadership or health advocacy. Curriculum development, professional development, and communication skills were dominant research themes while no articles discussed teacher training.

CONCLUSION

Although areas such as professionalism and communication skills are well-studied, medical oncology ME research is lacking in leadership, health advocacy, and teacher training. This study provides valuable guidance for future ME research in medical oncology and establishes a benchmark to examine changes in educational scholarship over time.

Improving Graduate Medical Education by Aggregating Data Across the Medical Education Continuum

ABSTRACT

Meaningful improvements to graduate medical education (GME) have been achieved in recent decades, yet many GME improvement pilots have been small trials without rigorous outcome measures and with limited generalizability. Thus, lack of access to large-scale data is a key barrier to generating empiric evidence to improve GME. In this article, the authors examine the potential of a national GME data infrastructure to improve GME, review the output of 2 national workshops on this topic, and propose a path toward achieving this goal.The authors envision a future where medical education is shaped by evidence from rigorous research powered by comprehensive, multi-institutional data. To achieve this goal, premedical education, undergraduate medical education, GME, and practicing physician data must be collected using a common data dictionary and standards and longitudinally linked using unique individual identifiers. The envisioned data infrastructure could provide a foundation for evidence-based decisions across all aspects of GME and help optimize the education of individual residents.Two workshops hosted by the National Academies of Sciences, Engineering, and Medicine Board on Health Care Services explored the prospect of better using GME data to improve education and its outcomes. There was broad consensus about the potential value of a longitudinal data infrastructure to improve GME. Significant obstacles were also noted.Suggested next steps outlined by the authors include producing a more complete inventory of data already being collected and managed by key medical education leadership organizations, pursuing a grass-roots data sharing pilot among GME-sponsoring institutions, and formulating the technical and governance frameworks needed to aggregate data across organizations.The power and potential of big data is evident across many disciplines, and the authors believe that harnessing the power of big data in GME is the best next step toward advancing evidence-based physician education.

Expanding Interdisciplinarity: A Bibliometric Study of Medical Education Using the Medical Education Journal List-24 (MEJ-24)

Introduction

Interdisciplinary research, which integrates input (e.g., data, techniques, theories) from two or more disciplines, is critical for solving wicked problems. Medical education research is assumed to be interdisciplinary. However, researchers have questioned this assumption. The present study, a conceptual replication, clarifies the nature of medical education interdisciplinarity by analyzing the citations of medical education journal articles.

Method

The authors retrieved the cited references of all articles in 22 medical education journals between 2001-2020 from Web of Science (WoS). We then identified the WoS classifications for the journals of each cited reference.

Results

We analyzed 31,283 articles referencing 723,683 publications. We identified 493,973 (68.3%) of those cited references in 6,618 journals representing 242 categories, which represents 94% of all WoS categories. Close to half of all citations were categorized as "education, scientific disciplines" and "healthcare sciences and services". Over the study period, the number of references consistently increased as did the representation of categories to include a diversity of topics such as business, management, and linguistics.

Discussion

Our study aligns with previous research, suggesting that medical education research could be described as inwardly focused. However, the observed growth of categories and their increasing diversity over time indicates that medical education displays increasing interdisciplinarity. Now visible, the field can raise awareness of and promote interdisciplinarity, if desired, by seeking and highlighting opportunities for future growth.

An analysis of vascular surgery education publications reveals an educational shortage

BACKGROUND

As vascular surgery training continues to evolve with the growth of integrated vascular surgery residency (0+5) programs and the consolidation of fellowship programs, optimizing all aspects of the education for vascular surgery trainees, both fellows and 0+5 residents, has become increasingly important. In the present study, we aimed to determine the prevalence, quality, and content of vascular surgery education publications across journals.

METHODS

Journal websites (n = 26) and PubMed were queried to identify vascular surgery education publications from 2012 to 2021. The publications were organized into 11 content categories: (1) curriculum, (2) simulation, (3) trainee assessment, (4) program evaluation, (5) wellness/burnout, (6) diversity/inclusion, (7) mentorship/career, (8) case outcomes, (9) perceptions of training, (10) social media, and (11) other. Publication interactivity and quality were measured via PlumX data and Medical Education Research Study Quality Instrument scores. The data were analyzed via univariate analysis and linear regression.

RESULTS

A total of 115 vascular surgery education publications (0.2% [interquartile range (IQR), 0.04%-0.5%] of total publications) were identified from the selected journals. The Journal of Vascular Surgery had the highest proportion (0.8%) of vascular surgery education publications, followed by the Journal of Surgical Education (0.7%) and Annals of Vascular Surgery (0.6%). Vascular surgery journals constituted most (79%) of the publications. Of the authors, 15% (IQR, 0%-25%) had a master's or doctorate degree in education. Senior authors were more often identified as male gender (77%), and more first authors (41%) were identified as female gender. An interactivity analysis showed that there were 10.3 citations (IQR, 12), 33.1 captures (IQR, 34), and 8.4 social media interactions (IQR 14) per publication. The educational quality had a median Medical Education Research Study Quality Instrument score of 11 (IQR, 9-12.5), with 49% of publications having a score greater than the median. Publications on training (44% curriculum and 20% simulation) were significantly more frequent than other topics (P < .001), with no change in the publication content over 10 years (P = .29). The volume of vascular surgery education publications did not change during the study period (P = .13) despite the ongoing changes in the educational environment.

CONCLUSIONS

Despite the increasing importance placed on vascular surgery education by national vascular societies, publications on vascular surgery education have remained sparse among all journals. Also, the vascular surgery educational content has not changed during the past 10 years, with a primary focus on curriculum and simulation training. Further promotion of vascular surgery educational research is required to increase the quality, volume, and diversity of education publications.

Theoretical Frameworks in Medical Education: Using a Systematic Review of Ophthalmology Education Research to Create a Theory of Change Model

BACKGROUND

Theoretical frameworks provide a lens to examine questions and interpret results; however, they are underutilized in medical education.

OBJECTIVE

To systematically evaluate the use of theoretical frameworks in ophthalmic medical education and present a theory of change model to guide educational initiatives.

METHODS

Six electronic databases were searched for peer-reviewed, English-language studies published between 2016 and 2021 on ophthalmic educational initiatives employing a theoretical framework. Quality of studies was assessed using the Grading of Recommendations, Assessment, Development, and Evaluations (GRADE) approach; risk of bias was evaluated using the Medical Education Research Study Quality Instrument (MERSQI) and the Accreditation Council for Graduate Medical Education (ACGME) guidelines for evaluation of assessment methods. Abstracted components of the included studies were used to develop a theory of change model.

RESULTS

The literature search yielded 1661 studies: 666 were duplicates, 834 studies were excluded after abstract review, and 132 after full-text review; 29 studies (19.2%) employing a theoretical framework were included. The theories used most frequently were the Dreyfus model of skill acquisition and Messick's contemporary validity framework. GRADE ratings were predominantly "low," the average MERSQI score was 10.04, and the ACGME recommendation for all assessment development studies was the lowest recommendation. The theory of change model outlined how educators can select, apply, and evaluate theory-based interventions.

CONCLUSIONS

Few ophthalmic medical education studies employed a theoretical framework; their overall rigor was low as assessed by GRADE, MERSQI, and ACGME guidelines. A theory of change model can guide integration of theoretical frameworks into educational initiatives.

Taking Aim at the Injury Prevention Curriculum: Educating Residents on Talking to Patients About Firearm Injury

The medical community recognizes that firearm injury is a public health problem. Yet we lack both the tools for and the implementation of evidence-based firearm injury screening and counseling techniques. One reason for these deficits is the lack of clinical training related to engaging patients in firearm injury risk reduction. In this issue, Rickert et al describe a pre-post evaluation of a 2-part firearm injury prevention training curriculum for first-year medical residents at a single academic medical center. Their manuscript serves an important, but still preliminary, step forward for the field of postgraduate medical education on firearm injury and its prevention. Important elements of this project and paper consist of the inclusion of multiple medical disciplines and the use of standardized patients to evaluate participants' learning. This project also points to the need for further growth. We must commit to consistently and conscientiously framing injuries from guns not as "gun violence" but rather as "firearm injuries" to ensure that they are considered squarely in both the public health space and the clinical space. An ideal curriculum would also highlight the importance of trauma-informed care, cultural competency, and antiracist medical practice while countering implicit biases (e.g., toward gun owners, victims of firearm violence, perpetrators of violence). It should address barriers, as well as facilitators, to change. And most importantly, future educational work must evaluate the effect of these trainings on actual clinical practice-and, even better, the efficacy of education in changing behavior and patient-level outcomes.

The Status of SBIRT Training in Health Professions Education: A Cross-Discipline Review and Evaluation of SBIRT Curricula and Educational Research

PURPOSE

To assess the quality of curricular research on the Screening Brief Intervention and Referral to Treatment (SBIRT) approach and determine the presence of useful training modalities, particularly motivational interviewing (MI) training, across health care training curricula.

METHOD

The authors conducted a systematic review of published, peer-reviewed studies in PubMed, ERIC, CINAHL, Ovid HealthSTAR, and PsycINFO databases through March 2021 for English-language studies describing SBIRT, a curriculum for health care trainees, and curricular intervention outcomes. After the records were independently assessed, data were extracted and 20% of the studies were double-coded for interrater reliability.

RESULTS

Of 1,856 studies, 95 were included in the review; 22 had overlapping samples and were consolidated into 10 nested studies, leaving 83 total. Interrater reliability ranged from moderate (κ = .74, P < .001) to strong (κ = .91, P < .001) agreement. SBIRT training was delivered to trainees across many professions, including nursing (n = 34, 41%), medical residency (n = 28, 34%), and social work (n = 24, 29%). Nearly every study described SBIRT training methods (n = 80, 96%), and most reported training in MI (n = 54, 65%). On average, studies reported 4.06 (SD = 1.64) different SBIRT training methods and 3.31 (SD = 1.59) MI training methods. Their mean design score was 1.92 (SD = 0.84) and mean measurement score was 1.89 (SD = 1.05). A minority of studies measured SBIRT/MI skill (n = 23, 28%), and 4 studies (5%) set a priori benchmarks for their curricula.

CONCLUSIONS

SBIRT training has been delivered to a wide range of health care trainees and often includes MI. Rigor scores for the studies were generally low due to limited research designs and infrequent use of objective skill measurement. Future work should include predefined training benchmarks and validated skills measurement.

Quality of cost evaluations of physician continuous professional development: Systematic review of reporting and methods

INTRODUCTION

We sought to evaluate the reporting and methodological quality of cost evaluations of physician continuing professional development (CPD).

METHODS

We conducted a systematic review, searching MEDLINE, Embase, PsycInfo, and the Cochrane Database for studies comparing the cost of physician CPD (last update 23 April 2020). Two reviewers, working independently, screened all articles for inclusion. Two reviewers extracted information on reporting quality using the Consolidated Health Economic Evaluation Reporting Standards (CHEERS), and on methodological quality using the Medical Education Research Study Quality Instrument (MERSQI) and a published reference case.

RESULTS

Of 3338 potentially eligible studies, 62 were included. Operational definitions of methodological and reporting quality elements were iteratively revised. Articles reported mean (SD) 43% (20%) of CHEERS elements for the Title/Abstract, 56% (34%) for Introduction, 66% (19%) for Methods, 61% (17%) for Results, and 66% (30%) for Discussion, with overall reporting index 292 (83) (maximum 500). Valuation methods were reported infrequently (resource selection 10 of 62 [16%], resource quantitation 10 [16%], pricing 26 [42%]), as were descriptions/discussion of the physicians trained (42 [68%]), training setting (42 [68%]), training intervention (40 [65%]), sensitivity analyses of uncertainty (9 [15%]), and generalizability (30 [48%]). MERSQI scores ranged from 6.0 to 16.0 (mean 11.2 [2.4]). Changes over time in reporting index (initial 241 [105], final 321 [52]) and MERSQI scores (initial 9.8 [2.7], final 11.9 [1.9]) were not statistically significant (p ≥ 0.08).

DISCUSSION

Methods and reporting of HPE cost evaluations fall short of current standards. Gaps exist in the valuation, analysis, and contextualization of cost outcomes.

Elevating Radiology Education Research Through a Dedicated Research Fellowship: Adding Professional Identity as Essential for Success

RATIONALE AND OBJECTIVES

Education research has been undervalued in radiology, with few radiology journals having dedicated space for such research and barriers to production including dedicated time, resources, grant funding, and mentorship. As radiologists, we have an opportunity to study education in our unique learning environments and create more effective ways to teach that are grounded in established education theory, solid assessment methodologies, and a focus on merit as well as worth.

MATERIALS AND METHODS

The growth of education research in our field requires a two-pronged approach; We must cultivate leaders in education research from within our specialty, and continue to submit high quality work to radiology journals to increase exposure of the reviewers and readership to education methodologies.

RESULTS

We have created the first radiology education research fellowship to this end. This manuscript details the fellowship creation process, the first education research fellow experience, and in-depth analysis of the inaugural year via a qualitative program review.

CONCLUSION

We were successful in supporting an inaugural fellow in attaining the knowledge and skills to become a productive education researcher. Beyond this, the fellowship experience was a catalyst in developing her unique professional identity as an education master which will further raise the status of education research in pediatric radiology.

Strategies for Measuring Advances in Pharmacy Student Learning

Objective. To provide guidance to authors and reviewers on how to design and evaluate educational research studies to better capture evidence of pharmacy student learning.Findings. A wide variety of assessment tools are available to measure student learning associated with educational innovations. Each assessment tool is associated with different advantages and disadvantages that must be weighed to determine the appropriateness of the tool for each situation. Additionally, the educational research design must be aligned with the intent of the study to strengthen its impact.Summary. By selecting research methods aligned with Kirkpatrick's levels of training evaluation, researchers can create stronger evidence of student learning when evaluating the effectiveness of teaching innovations.

Conducting qualitative research through time: how might theory be useful in longitudinal qualitative research?

This paper explores the use of theory in longitudinal qualitative research, an approach to research which explores lived experiences as they unfold. The authors illustrate how the complexity of conducting qualitative research through time drives an understanding and use of theory that differs from other research approaches. Longitudinal qualitative research considers time as fluid, subjective, and unbounded-in contrast to the more common taken-for-granted understanding of time as fixed, objective, and linear. Furthermore, longitudinal qualitative research is predicated on a premise of trust in the context of enduring research relationships. Therefore, while subject-matter theories used to investigate topics of interest to health professions educators may be useful frameworks for other types of research, longitudinal qualitative research needs theories that accommodate the myriad of changes in lived experiences through time. The authors share their decade-long, longitudinal qualitative research story, highlighting their decision points and insights. In doing so, they foreground issues such as time as fluid as an important contribution to health professions education literature.

The characteristics of effective technology-enabled dementia education: a systematic review and mixed research synthesis

BACKGROUND

Dementia education is required to address gaps in dementia-specific knowledge among health and social care practitioners amidst increasing dementia prevalence. Harnessing technology for dementia education may remove obstacles to traditional education and empower large communities of learners. This systematic review aimed to establish the technological and pedagogical characteristics associated with effective technology-enabled dementia education for health and social care practitioners.

METHODS

MEDLINE, PubMed, Web of Science, CINAHL, Scopus, PsycINFO, ERIC and OVID Nursing Database were searched from January 2005 until February 2020. Quantitative, qualitative and mixed methods studies were eligible for inclusion. Study quality was assessed with the Mixed Methods Appraisal Tool. Quantitative evidence was categorised based on Kirkpatrick's Model. Qualitative data was synthesised thematically and integrated with quantitative findings before conclusions were drawn.

RESULTS

Twenty-one published papers were identified. Participants were acute, primary and long-term care practitioners, or were students in higher education. Most training was internet-based; CD-ROMs, simulations and tele-mentoring were also described. Technology-enabled dementia education was predominantly associated with positive effects on learning outcomes. Case-based instruction was the most frequently described instructional strategy and videos were common modes of information delivery. Qualitative themes emerged as existing strengths and experience; knowledge gaps and uncertainty; developing core competence and expertise; involving relevant others; and optimising feasibility.

DISCUSSION

Technology-enabled dementia education is likely to improve dementia knowledge, skills and attitudes among health and social care practitioners from multiple practice contexts. Confidence in the results from quantitative studies was undermined by multiple confounding factors that may be difficult to control in the educational research context. Convenience and flexibility are key benefits of technology-enabled instructive and simulated pedagogy that can support the application of theory into practice. More research is required to understand the role of online learning networks and provisions for equitable engagement. A future emphasis on organisational and environmental factors may elucidate the role of technology in ameliorating obstacles to traditional dementia education.

SYSTEMATIC REVIEW REGISTRATION

PROSPERO ( CRD42018115378 ).

Online learning developments in undergraduate medical education in response to the COVID-19 pandemic: A BEME systematic review: BEME Guide No. 69

BACKGROUND

The COVID-19 pandemic spurred an abrupt transition away from in-person educational activities. This systematic review investigated the pivot to online learning for nonclinical undergraduate medical education (UGME) activities and explored descriptions of educational offerings deployed, their impact, and lessons learned.

METHODS

The authors systematically searched four online databases and conducted a manual electronic search of MedEdPublish up to December 21, 2020. Two authors independently screened titles, abstracts and full texts, performed data extraction and assessed risk of bias. A third author resolved discrepancies. Findings were reported in accordance with the STORIES (STructured apprOach to the Reporting in healthcare education of Evidence Synthesis) statement and BEME guidance.

RESULTS

Fifty-six articles were included. The majority (n = 41) described the rapid transition of existing offerings to online formats, whereas fewer (n = 15) described novel activities. The majority (n = 27) included a combination of synchronous and asynchronous components. Didactics (n = 40) and small groups (n = 26) were the most common instructional methods. Teachers largely integrated technology to replace and amplify rather than transform learning, though learner engagement was often interactive. Thematic analysis revealed unique challenges of online learning, as well as exemplary practices. The quality of study designs and reporting was modest, with underpinning theory at highest risk of bias. Virtually all studies (n = 54) assessed reaction/satisfaction, fewer than half (n = 23) assessed changes in attitudes, knowledge or skills, and none assessed behavioral, organizational or patient outcomes.

CONCLUSIONS

UGME educators successfully transitioned face-to-face instructional methods online and implemented novel solutions during the COVID-19 pandemic. Although technology's potential to transform teaching is not yet fully realized, the use of synchronous and asynchronous formats encouraged virtual engagement, while offering flexible, self-directed learning. As we transition from emergency remote learning to a post-pandemic world, educators must underpin new developments with theory, report additional outcomes and provide details that support replication.

Collaborative healthcare education programmes for continuing professional education in low and middle-income countries: A Best Evidence Medical Education (BEME) systematic review. BEME Guide No. 65

BACKGROUND

Large discrepancies exist between standards of healthcare provision in high-income (HICs) and low and middle-income countries (LMICs). The root cause is often financial, resulting in poor infrastructure and under-resourced education and healthcare systems. Continuing professional education (CPE) programmes improve staff knowledge, skills, retention, and practice, but remain costly and rare in low-resource settings. One potential solution involves healthcare education collaborations between institutions in HICs and LMICs to provide culturally appropriate CPE in LMICs. To be effective, educational partnerships must address the challenges arising from differences in cultural norms, language, available technology and organisational structures within collaborating countries.

METHODS

Seven databases and other sources were systematically searched on 7 July 2020 for relevant studies. Citations, abstracts, and studies were screened and consensus was reached on which to include within the review. 54 studies were assessed regarding the type of educational programme involved, the nature of HIC/LMIC collaboration and quality of the study design.

RESULTS

Studies varied greatly regarding the types and numbers of healthcare professionals involved, pedagogical and delivery methods, and the ways in which collaboration was undertaken. Barriers and enablers of collaboration were identified and discussed. The key findings were: 1. The methodological quality of reporting in the studies was generally poor. 2. The way in which HIC/LMIC healthcare education collaboration is undertaken varies according to many factors, including what is to be delivered, the learner group, the context, and the resources available. 3. Western bias was a major barrier. 4. The key to developing successful collaborations was the quality, nature, and duration of the relationships between those involved.

CONCLUSION

This review provides insights into factors that underpin successful HIC/LMIC healthcare CPE collaborations and outlines inequities and quality issues in reporting.

Transesophageal Echocardiography Simulator Training: A Systematic Review and Meta-analysis of Randomized Controlled Trials

SUMMARY STATEMENT

We aimed to assess the learning effects of novice transesophageal echocardiography (TEE) simulator training and to identify gaps in existing studies. We performed a systematic review and meta-analysis of randomized controlled trials (RCTs) comparing the learning effects of novice TEE training with versus without simulators, searching published articles and proceedings in 6 major databases in June 2019. We included 9 RCTs (268 participants). Compared with nonsimulator training, TEE simulator training resulted in higher skill and knowledge posttraining test scores with large effect sizes (standardized mean difference = 0.81 for skill, 1.61 for knowledge; low-certainty evidence) and higher training satisfaction with a small effect size (standardized mean difference = 0.36; very low-certainty evidence). No RCTs reported training budget or patient outcomes. Additional well-designed studies with low risk of bias and large sample sizes are needed to provide reliable and robust findings and develop more effective TEE simulation-based training curricula.

Establishing the effectiveness of technology-enabled dementia education for health and social care practitioners: a systematic review

BACKGROUND

Dementia prevalence is increasing globally and yet evidence suggest that gaps exist in dementia-specific knowledge among health and social care practitioners. Technological modes of educational delivery may be as effective as traditional education and can provide practitioners with increased accessibility to dementia training. Benefits of digitally based dementia education have been established including pedagogical strategies that influence dementia knowledge and care attitudes. This review aimed to appraise and synthesise contemporary experimental evidence that evaluated technology-enabled dementia education for health and social care practitioners. Outcomes based on Kirkpatrick's Model were learner satisfaction; knowledge, skills, and attitudes; behaviours; and results.

METHODS

MEDLINE, CINAHL, and Web of Science were among 8 bibliographic databases searched from January 2005 until February 2020. Keywords included dementia and education (and terms for technological modes of education, learning, or training). We included experimental and quasi-experimental studies. Medical Education Research Study Quality Instrument established the overall quality of included studies and pragmatic application of Mixed Methods Appraisal Tool established individual study quality and highlighted methodological features of educational research. Narrative synthesis was conducted as heterogeneous outcome data precluded meta-analysis.

RESULTS

We identified 21 relevant studies: 16 evaluated online dementia education and 5 evaluated computer-based approaches. Most studies used before-after designs and study quality was moderate overall. Most studies reported knowledge-based outcomes with statistically significant findings favouring the training interventions. Positive effects were also observed in studies measuring skills and attitudinal change. Fewer studies reported significant findings for behavioural change and results due to training. Case-based instruction was a frequently described instructional strategy in online dementia education and videos were common information delivery modes. CD-ROM training and simulation activities were described in computer-based dementia education.

DISCUSSION

Future emphasis must be placed on teaching and learning methods within technology-enabled dementia education which should be role relevant and incorporate active and interactive learning strategies. Future evaluations will require contextually relevant research methodologies with capacity to address challenges presented by these complex educational programmes and multi-component characteristics.

SYSTEMATIC REVIEW REGISTRATION

This systematic review is based on a protocol registered with PROSPERO ( CRD42018115378 ).

Best Practices Using Ex Vivo Animal Brain Models in Neurosurgical Education to Assess Surgical Expertise

BACKGROUND

Ex vivo animal brain simulation models are being increasingly used for neurosurgical training because these models can replicate human brain conditions. The goal of the present report is to provide the neurosurgical community interested in using ex vivo animal brain simulation models with guidelines for comprehensively and rigorously conducting, documenting, and assessing this type of research.

METHODS

In consultation with an interdisciplinary group of physicians and researchers involved in ex vivo models and a review of the literature on the best practices guidelines for simulation research, we developed the "ex vivo brain model to assess surgical expertise" (EVBMASE) checklist. The EVBMASE checklist provides a comprehensive quantitative framework for analyzing and reporting studies involving these models. We applied The EVBMASE checklist to the studies reported of ex vivo animal brain models to document how current ex vivo brain simulation models are used to train surgical expertise.

RESULTS

The EVBMASE checklist includes defined subsections and a total score of 20, which can help investigators improve studies and provide readers with techniques to better assess the quality and any deficiencies of the research. We classified 18 published ex vivo brain models into modified (group 1) and nonmodified (group 2) models. The mean total EVBMASE score was 11 (55%) for group 1 and 4.8 (24.2%) for group 2, a statistically significant difference (P = 0.006) mainly attributed to differences in the simulation study design section (P = 0.003).

CONCLUSIONS

The present findings should help contribute to more rigorous application, documentation, and assessment of ex vivo brain simulation research.

Content analysis of metadata, titles, and abstracts (CAMTA): application of the method to business and management research

Purpose

This paper offers a “content analysis of metadata, titles, and abstracts” (CAMTA) method underpinned by a newly evolved metadata, title, abstract, introduction, methodology, results, analysis, and discussion (M-TAIMRAD) Framework.

Design/methodology/approach

Draws on innovations of content analysis from the field of health- care to offer a pragmatic and transparent method for conducting rigorous and valid research within the field of business and management.

Findings

Replicable and valid guidelines for conducting the CAMTA method are offered, including an illustration. This is followed by a critical examination of the potential applications and benefits of the method to the field of business and management research.

Originality/value

The CAMTA method enables researchers to assimilate and synthesise metadata, titles and abstracts as a means of identifying grounds for future research and theory development. This will help to advance the field and subsequently benefit the wider readership including fellow academics, practitioners and policymakers. The flexibility of the CAMTA method means that it can be used as a stand-alone method or combined as part of a mixed-methods approach.

The Impact of a Medical Education Research Faculty Development Program on Career Development, Through the Lens of Social Cognitive Career Theory

OBJECTIVES

The Medical Education Research Certificate at the Council of Residency Directors in Emergency Medicine (MERC at CORD), a specialized adaptation of the Association of American Medical Colleges MERC program, provides faculty development in education research in emergency medicine. However, its long-term influence on career development remains unknown. Our study explored the impact of MERC at CORD on career development through the lens of social cognitive career (SCC) theory.

METHODS

This was a prospective qualitative study using a constructivist/interpretivist paradigm to assess long-term career development outcomes. A purposeful randomized stratified sampling strategy of MERC at CORD graduates (2011-2014) ensured diversity of representation (sex, region, number of research publications, and project group leadership). Subjects were invited by e-mail to participate in semistructured phone interviews. Thematic analysis by two independent reviewers followed an iterative process until saturation was reached.

RESULTS

Twelve graduates were interviewed. All engaged with MERC at CORD early in their careers with minimal previous education research experience. Currently, all hold medical education leadership positions. Graduates had a mean of 19.3 publications (range = 9-43). Themes explaining reasons for participating in MERC at CORD include: desire for education research skills, recommendation of mentors/colleagues, and accessibility. Themes citing the program's value to career development include networking/collaboration, mentorship, informational framework to build upon, and the application of theoretical knowledge through experiential learning. MERC at CORD impacted career development aligning with the core domains of SCC theory including self-efficacy, outcome expectations, and goals.

CONCLUSION

MERC at CORD enhanced the long-term career development of participants by providing a core knowledge framework in a mentored, experiential learning environment. Participants identified themes aligned with SCC theory as influential in their long-term career advancement in medical education including the development of education research skills, successful completion of education research, career acceleration, promotion, niche development, and formulation of professional goals.

The myth of ivory tower versus practice-oriented research: A systematic review of randomised studies in medical education

CONTEXT

A long-standing myth in medical education research is a divide between two different poles: research aiming to advance theory with little focus on practical applications ('ivory tower' research) and practically oriented research aiming to serve educators and decision-makers with little focus on advancing theory ('in-the-trenches' practice). We explored this myth in a sample of randomised medical education studies using Stokes' four-quadrant framework for the classification of research perspective.

METHODS

We searched MEDLINE, Embase, CINAHL, PsychINFO, ERIC, Web of Science and Scopus for studies in medical education using a randomised design that were published between 1 January 2018 and 31 December 2018. We used Stokes' four-quadrant framework to categorise the studies according to their use of theory, concepts and their justification for practical use. We compared medical education research published in medical education journals and clinical journals.

RESULTS

A total of 150 randomised studies were included in the analysis. The largest segment of studies (46.7%) was categorised as use-inspired basic research (Pasteur's Quadrant), closely followed by pure applied research (40.7%, Edison's Quadrant). Only a few studies were categorised as aiming to advance knowledge with no thought for practical educational application (2.0%, Bohr's Quadrant). The proportion of studies that included educational concepts and theory differed according to publication in clinical journals or medical education journals: 40.5% vs 71.8%, respectively, P < .001. There were no differences between journals with regard to the proportion of studies that included a practical educational or clinical rationale (P = .99).

CONCLUSION

In a large sample of studies using randomised designs, we found no evidence to support the myth that medical education research divides between two singular poles represented by 'ivory tower research' and 'in-the-trenches practice'. We did confirm prevailing assumptions regarding an emphasis on non-theoretical medical education research in clinical journals.

Improving hospital falls screening and mitigation using a health professional education framework

OBJECTIVE

Although health professional education has the potential to mitigate hospital falls risk, the best methods to develop, deliver and evaluate health professional education remain unclear. This study applied evidence-based approaches to education design to improve falls risk mitigation.

DESIGN

Mixed methods using questionnaires to evaluate health professionals knowledge of evidence-based falls risk assessment and mitigation, followed by semi-structured interviews with individual health professionals.

SETTING

Five large Australian hospitals.

PARTICIPANTS

For each hospital, 10 clinical leaders from nursing and allied health professions were invited to participate in falls workshops.

METHODS

46 participants received a three-hour education program on the latest evidence in hospital falls risk assessment and how to implement evidence-based falls screening and management. This was based on the "4P" education model (Presage, Planning, Process and Product). They were taught practical skills to enable them to educate other health professionals.

RESULTS

The education workshop significantly changed participants' views about best practice guidelines for falls screening and prevention. Participants felt more confident in assessing falls risk and judging and implementing the best mitigation strategies. They were prepared and motivated to educate others about falls prevention and satisfied with the skills gained.

CONCLUSIONS

A high-quality education program grounded in a rigorous quality framework improved health professionals knowledge regarding evidence-based falls prevention. Use of evidence-based rationales for behaviour change promotes effective learning.

Education interventions for health professionals on falls prevention in health care settings: a 10-year scoping review

Background

Falls in hospitals are a major risk to patient safety. Health professional education has the potential to be an important aspect of falls prevention interventions. This scoping review was designed to investigate the extent of falls prevention education interventions available for health professionals, and to determine the quality of reporting.

Method

A five stage scoping review process was followed based on Arksey and O’Malley’s framework and refined by the Joanna Briggs Institute Methodology for JBI Scoping Reviews. Five online databases identified papers published from January 2008 until May 2019. Papers were independently screened by two reviewers, and data extracted and analysed using a quality reporting framework.

Results

Thirty-nine publications were included. Interventions included formal methods of educational delivery (for example, didactic lectures, video presentations), interactive learning activities, experiential learning, supported learning such as coaching, and written learning material. Few studies employed comprehensive education design principles. None used a reporting framework to plan, evaluate, and document the outcomes of educational interventions.

Conclusions

Although health professional education is recognised as important for falls prevention, no uniform education design principles have been utilised in research published to date, despite commonly reported program objectives. Standardised reporting of education programs has the potential to improve the quality of clinical practice and allow studies to be compared and evaluated for effectiveness across healthcare settings.

The Hiker's Guide to the RIME Supplement: Choosing Directions in Research

In this Commentary, the authors seek to build on prior RIME commentaries by considering how researchers transition from worldviews, focal lengths, and research goals to research directions and methodological choices. The authors use the analogy of a hiker to illustrate how different researchers studying a similar phenomenon can choose among different research directions, which lead down different paths and offer different perspectives on a problem. Following the hiker analogy, the authors use the "Research Compass" to categorize the 15 research papers included in the 2020 Research in Medical Education supplement according to their research aim and corresponding methodological approach. The authors then discuss implications of the relative balance of these study types within this supplement and within health professions education research at large, emphasizing the critical importance of studying a topic from multiple vantage points to construct a richer and more nuanced understanding of health professions education challenges. The authors conclude by recognizing the challenges we face in the current era of COVID-19 and by calling health professions education researchers and practitioners to continue our collective efforts to improve learner education and patient care, as we together navigate the unfamiliar terrain of the present day.

Evaluation of the reporting quality of observational studies in master of public health dissertations in China

Backgrounds

Master of public health (MPH) plays an important role in Chinese medical education, and the dissertations is an important part of MPH education. In MPH dissertations, most are observational studies. Compared with randomized controlled trial (RCT), observational studies are more prone to information bias. So, the reporting of the observational studies should be transparent and standard. But, no research on evaluating the reporting quality of the MPH dissertation has been found.

Methods

A systematic literature search was performed in the Wanfang database from January 1, 2014 to May 31, 2019. The Strengthening the Reporting of Observation Studies in Epidemiology (STROBE) statement was adopted to evaluate the reporting quality of the selected studies. Articles that met the following criteria were selected: (1) observational studies, including cross-sectional studies, case-control studies, and cohort studies; (2) original articles; (3) studies on humans, including both adults and children.

Results

The Median of compliance to individual STROBE items was 74.79%. The mean (standard deviation) of STROBE score was 14.29 (1.84). Five items/sub-items were 100% reported (“reported” and “partly reported” were combined): background, objectives, study design, report numbers of individuals at each stage, and key result. Fifteen items/sub-items were reported by 75% or more. Reporting of methods and results was often omitted: missing data (6.67%), sensitivity analyses (3.63%), flow diagram (15.15%), and absolute risk (0%). Logistic regression analysis indicated that cohort studies (OR = 3.41, 95% CI = 1.27–9.16), funding support (OR = 4.37, 95% CI = 1.27–9.16) and more published papers during postgraduate period (OR = 3.46, 95% CI = 1.40–8.60) were related to high reporting quality.

Conclusion

In short, the reporting quality of observational studies in MPH’s dissertations in China is suboptimal. However, it’s necessary to improve the reporting of method and results sections. We recommend that authors should be stricter to adhere STROBE statement when conducting observational studies.

Performing Medical Education Research in Urology: Challenges and Opportunities

PURPOSE OF REVIEW

The Accreditation Council for Graduate Medical Education (ACGME) mandates educating resident physicians in evidence-based medicine (EBM) as a core program requirement. However, despite the significant emphasis placed on EBM, graduate medical education is far from evidence-based, and urology is a specialty where medical education research (MER) is particularly sparse. We want to articulate the challenges and opportunities with performing meaningful medical education research in urology training programs.

RECENT FINDINGS

Some studies suggest that the rigor of MER could be much stronger. The nature of GME requires researchers to use alternative study designs. Further, the unique role of residents as both learner and study subject and the dual role of faculty as researcher and educator pose challenges to carrying out research. There is a tremendous opportunity for improvement and innovation in both quality and efficiency of urology resident education. Rigorous MER is required to advance this opportunity, and the fundamental key is development of mentors and collaboration.

Twelve tips for conducting educational design research in medical education

Despite a steady growth in educational innovations and studies investigating the acceptance and effectiveness of these innovations, medical education has not realized sufficient improvement in practice and outcomes from these investments. In light of this lack of impact, there has been a growing call for studies that more effectively bridge the gap between research and practice. This paper introduces Educational Design Research (EDR) as a promising approach to address this challenge. Twelve tips are provided to inspire and guide medical educators to conduct EDR to achieve the dual goals of tackling a significant educational problem in a specific context while at the same time advancing the theoretical knowledge that may be used to improve practice elsewhere.

The Distinctions Between Theory, Theoretical Framework, and Conceptual Framework

Health professions education (HPE) researchers are regularly asked to articulate their use of theory, theoretical frameworks, and conceptual frameworks in their research. However, all too often, these words are used interchangeably or without a clear understanding of the differences between these concepts. Further problematizing this situation is the fact that theory, theoretical framework, and conceptual framework are terms that are used in different ways in different research approaches. In this article, the authors set out to clarify the meaning of these terms and to describe how they are used in 2 approaches to research commonly used in HPE: the objectivist deductive approach (from theory to data) and the subjectivist inductive approach (from data to theory). In addition to this, given that within subjectivist inductive research theory, theoretical framework, and conceptual framework can be used in different ways, they describe 3 uses that HPE researchers frequently rely on: fully inductive theory development, fully theory-informed inductive, and theory-informing inductive data analysis.

Accuracy of acupuncture point location among first-year acupuncture students: A pilot study evaluating the impact of training and use of an adjustable ruler

BACKGROUND

The proportional method of acupuncture point location (APL) currently taught at Endeavor College of Natural Health and advocated by the World Health Organization Regional Office for the Western Pacific (WPRO) was found to be imprecise and/or inaccurate in previous student studies. The ruler and elastic methods of APL were identified as more accurate or precise than the proportional method of APL but were not well received by student participants. Use of an adjustable ruler may overcome barriers to uptake of the more accurate APL methods. This pilot study was the first to evaluate the comparative accuracy of the adjustable ruler and the proportional methods of APL in first-year students at a major Australian acupuncture training college.

METHODS

After 10 weeks of in-class instruction in both proportional and adjustable ruler methods of APL, student participants (n = 14) attempted location of three acupuncture points (LI10, SP6 and ST38) on a volunteer using both APL methods of interest. A self-administered questionnaire and lecturer field notes elucidated attitudes to implementation of both APL methods.

RESULTS

Points marked using the adjustable ruler were closer to the correct location than those marked using the proportional method across all three acupuncture points. Students and lecturers rated the adjustable ruler more highly than the proportional method for ease of learning and ease of use.

CONCLUSION

Encouraging results with the adjustable ruler method warrant further larger scale studies. Use of the adjustable ruler method of APL should be considered for use in point location training at educational institutions teaching traditional acupuncture.

Tools for Medical Education Scholarship: From Curricular Development to Educational Research

The development of successful educational scholarship, either curricula or medical education research, is vital in ensuring that the field of medical education continues to evolve. Fostering the skills of medical educators in conducting high-quality educational research is essential to this process because publishing such research helps to disseminate best educational practices to the medical community at large. Unfortunately, developing rigorous medical education research can be challenging for pediatric hospitalists within busy clinical settings. In this article, we aim to discuss key principles and frameworks for curricular development as well as offer guidance in transforming a curriculum into a scholarly medical education research product for pediatric hospital medicine providers.

Augmented reality in medical education: a systematic review

INTRODUCTION

The field of augmented reality (AR) is rapidly growing with many new potential applications in medical education. This systematic review investigated the current state of augmented reality applications (ARAs) and developed an analytical model to guide future research in assessing ARAs as teaching tools in medical education.

METHODS

A literature search was conducted using PubMed, Embase, Web of Science, Cochrane Library, and Google Scholar. This review followed PRISMA guidelines and included publications from January 1, 2000 to June 18, 2018. Inclusion criteria were experimental studies evaluating ARAs implemented in healthcare education published in English. Our review evaluated study quality and determined whether studies assessed ARA validity using criteria established by the GRADE Working Group and Gallagher et al., respectively. These findings were used to formulate an analytical model to assess the readiness of ARAs for implementation in medical education.

RESULTS

We identified 100,807 articles in the initial literature search; 36 met inclusion criteria for final review and were categorized into three categories: Surgery (23), Anatomy (9), and Other (4). The overall quality of the studies was poor and no ARA was tested for all five stages of validity. Our analytical model evaluates the importance of research quality, application content, outcomes, and feasibility of an ARA to gauge its readiness for implementation.

CONCLUSION

While AR technology is growing at a rapid rate, the current quality and breadth of AR research in medical training is insufficient to recommend the adoption into educational curricula. We hope our analytical model will help standardize AR assessment methods and define the role of AR technology in medical education.

Evaluation of e-learning for medical education in low- and middle-income countries: A systematic review

In low- and middle-income countries (LMICs), e-learning for medical education may alleviate the burden of severe health worker shortages and deliver affordable access to high quality medical education. However, diverse challenges in infrastructure and adoption are encountered when implementing e-learning within medical education in particular. Understanding what constitutes successful e-learning is an important first step for determining its effectiveness. The objective of this study was to systematically review e-learning interventions for medical education in LMICs, focusing on their evaluation and assessment methods. Nine databases were searched for publications from January 2007 to June 2017. We included 52 studies with a total of 12,294 participants. Most e-learning interventions were pilot studies (73%), which mainly employed summative assessments of study participants (83%) and evaluated the e-learning intervention with questionnaires (45%). Study designs, evaluation and assessment methods showed considerable variation, as did the study quality, evaluation periods, outcome and effectiveness measures. Included studies mainly utilized subjective measures and custom-built evaluation frameworks, which resulted in both low comparability and poor validity. The majority of studies self-concluded that they had had an effective e-learning intervention, thus indicating potential benefits of e-learning for LMICs. However, MERSQI and NOS ratings revealed the low quality of the studies' evidence for comparability, evaluation instrument validity, study outcomes and participant blinding. Many e-learning interventions were small-scale and conducted as short-termed pilots. More rigorous evaluation methods for e-learning implementations in LMICs are needed to understand the strengths and shortcomings of e-learning for medical education in low-resource contexts. Valid and reliable evaluations are the foundation to guide and improve e-learning interventions, increase their sustainability, alleviate shortages in health care workers and improve the quality of medical care in LMICs.

Artificial Intelligence in Medical Education: Best Practices Using Machine Learning to Assess Surgical Expertise in Virtual Reality Simulation

OBJECTIVE

Virtual reality simulators track all movements and forces of simulated instruments, generating enormous datasets which can be further analyzed with machine learning algorithms. These advancements may increase the understanding, assessment and training of psychomotor performance. Consequently, the application of machine learning techniques to evaluate performance on virtual reality simulators has led to an increase in the volume and complexity of publications which bridge the fields of computer science, medicine, and education. Although all disciplines stand to gain from research in this field, important differences in reporting exist, limiting interdisciplinary communication and knowledge transfer. Thus, our objective was to develop a checklist to provide a general framework when reporting or analyzing studies involving virtual reality surgical simulation and machine learning algorithms. By including a total score as well as clear subsections of the checklist, authors and reviewers can both easily assess the overall quality and specific deficiencies of a manuscript.

DESIGN

The Machine Learning to Assess Surgical Expertise (MLASE) checklist was developed to help computer science, medicine, and education researchers ensure quality when producing and reviewing virtual reality manuscripts involving machine learning to assess surgical expertise.

SETTING

This study was carried out at the McGill Neurosurgical Simulation and Artificial Intelligence Learning Centre.

PARTICIPANTS

The authors applied the checklist to 12 articles using machine learning to assess surgical expertise in virtual reality simulation, obtained through a systematic literature review.

RESULTS

Important differences in reporting were found between medical and computer science journals. The medical journals proved stronger in discussion quality and weaker in areas related to study design. The opposite trends were observed in computer science journals.

CONCLUSIONS

This checklist will aid in narrowing the knowledge divide between computer science, medicine, and education: helping facilitate the burgeoning field of machine learning assisted surgical education.

Use of the "Step-back" Method for Education Research Consultation at the National Level: A Pilot Study

BACKGROUND

There are a limited number of emergency medicine (EM) physicians with expertise in education research. The Harvard Macy "step-back" method is an emerging model utilized to gather group feedback. Despite its use in multiple educational settings, there are little published data demonstrating effectiveness.

OBJECTIVES

Our objective was to create and evaluate a national faculty development session providing consultation in education research utilizing the step-back method.

METHODS

This was a pilot study. EM experts in education research from across the country served as facilitators for a faculty development session held at the 2018 Council of Emergency Medicine Residency Directors Academic Assembly. Small groups consisting of two or three facilitators and one or two participants were formed and each participant underwent a step-back consultation for their education research study. Participants wrote their study question before and after the session. After the session, facilitators and participants completed an evaluative survey consisting of multiple-choice, Likert-type, and free-response items. Descriptive statistics were reported. Qualitative analysis using a thematic approach was performed on free-response data. Participant study questions were assessed by the PICO (population, intervention, comparison, outcome) and FINER (feasible, interesting, novel, ethical, relevant) criteria. Both scales were evaluated using a two-way random-consistency intraclass correlation. Before and after scores were evaluated with a paired t-test.

RESULTS

Twenty-four facilitators and 13 participants completed the step-back session. Evaluations from 20 facilitators and nine participants were submitted and analyzed. Sixteen of 20 facilitators felt that the step-back method "greatly facilitated" their ability to share their education research expertise. All facilitators and participants recommended that the session be provided at a future academic assembly. Regarding suggestions for improvement, qualitative analysis revealed three major themes: praise for the session, desire for additional time, and a room set up more conducive to small group work. Seven of nine responding participants felt that the session was "very valuable" for improving the strength of their study methods. Qualitative analysis regarding change in study as a result of the step-back session yielded four major themes: refinement of study question, more specific outcomes and measurements, improvement in study design, and greater understanding of study limitations. Both FINER and PICO scale comparisons showed improvement pre- and postintervention (PICO 60% relative increase; FINER 16% relative increase). Neither achieved statistical significance (PICO t(5) = -1.835, p = 0.126; and FINER t(5) = -1.305, p = 0.249).

CONCLUSION

A national-level education research consultation utilizing the step-back method was feasible to implement and highly valued by facilitators and participants. Potential positive outcomes include refinement of study question, more specific outcomes and measurements, improvement in study design, and greater understanding of limitations. These results may inform others who want to utilize this method.

Simulation Training to Improve the Ability of First-Year Doctors to Assess and Manage Deteriorating Patients: a Systematic Review and Meta-analysis

Many simulation courses now exist which aim to prepare first-year doctors for the task of assessing and managing potentially deteriorating patients. Despite the substantial resources required, the degree to which participants benefit from such courses, and which aspects of the simulation training are optimal for learning, remains unclear. A systematic literature search was undertaken across seven electronic databases. Inclusion criteria were that the intervention must be a simulation of a deteriorating patient scenario that would likely be experienced by first-year doctors, and that participants being first-year doctors or in their final year of medical school. Studies reporting quantitative benefits of simulation on participants' knowledge and simulator performance underwent meta-analyses. The search returned 1444 articles, of which 48 met inclusion criteria. All studies showed a benefit of simulation training, but outcomes were largely limited to self-rated or objective tests of knowledge, or simulator performance. The meta-analysis demonstrated that simulation improved participant performance by 16% as assessed by structured observation of a simulated scenario, and participant knowledge by 7% as assessed by written assessments. A mixed-methods analysis found conflicting evidence about which aspects of simulation were optimal for learning. The results of the review indicate that simulation is an important tool to improve first-year doctors' confidence, knowledge and simulator performance with regard to assessment and management of a potentially deteriorating patient. Future research should now seek to clarify the extent to which these improvements translate into clinical practice, and which aspects of simulation are best suited to achieve this.

Theory-based strategies for teaching evidence-based practice to undergraduate health students: a systematic review

BACKGROUND

Undergraduate students across health professions are required to be capable users of evidence in their clinical practice after graduation. Gaining the essential knowledge and clinical behaviors for evidence-based practice can be enhanced by theory-based strategies. Limited evidence exists on the effect of underpinning undergraduate EBP curricula with a theoretical framework to support EBP competence. A systematic review was conducted to determine the effectiveness of EBP teaching strategies for undergraduate students, with specific focus on efficacy of theory-based strategies.

METHODS

This review critically appraised and synthesized evidence on the effectiveness of EBP theory-based teaching strategies specifically for undergraduate health students on long or short-term change in multiple outcomes, including but not limited to, EBP knowledge and attitudes. PubMed, CINAHL, Scopus, ProQuest Health, ERIC, The Campbell Collaboration, PsycINFO were searched for published studies and The New York Academy of Medicine, ProQuest Dissertations and Mednar were searched for unpublished studies. Two independent reviewers assessed studies using the Joanna Briggs Institute Meta-Analysis of Statistics Assessment and Review Instrument.

RESULTS

Twenty-eight studies reporting EBP teaching strategies were initially selected for review with methodological quality ranging from low to high. Studies varied in course duration, timing of delivery, population and course content. Only five included papers reported alignment with, and detail of, one or more theoretical frameworks. Theories reported included Social Cognitive Theory (one study), Roger's Diffusion of Innovation Theory (two studies) and Cognitive Apprenticeship Theory (one study). Cognitive Flexibility Theory and Cognitive Load Theory were discussed in two separate papers by the same authors. All but one study measured EBP knowledge. Mixed results were reported on EBP knowledge, attitudes and skills across the five studies.

CONCLUSIONS

EBP programs for undergraduate health students require consideration of multiple domains, including clinical behaviors, attitudes and cognitive learning processes; Interventions grounded in theory were found to have a small but positive effect on EBP attitudes. The most effective theory for developing and supporting EBP capability is not able to be determined by this review therefore additional rigorous research is required.

Qualitative Research Methods in Medical Education

Qualitative research was originally developed within the social sciences. Medical education is a field that comprises multiple disciplines, including the social sciences, and utilizes qualitative research to gain a broader understanding of key phenomena within the field. Many clinician educators are unfamiliar with qualitative research. This article provides a primer for clinician educators who want to appraise or conduct qualitative research in medical education. This article discusses a definition and the philosophical underpinnings for qualitative research. Using the Standards for Reporting Qualitative Research as a guide, this article provides a step-wise approach for conducting and evaluating qualitative research in medical education. This review will enable the reader to understand when to utilize qualitative research in medical education and how to interpret reports using qualitative approaches.

Quantitative Research Methods in Medical Education

There has been a dramatic growth of scholarly articles in medical education in recent years. Evaluating medical education research requires specific orientation to issues related to format and content. Our goal is to review the quantitative aspects of research in medical education so that clinicians may understand these articles with respect to framing the study, recognizing methodologic issues, and utilizing instruments for evaluating the quality of medical education research. This review can be used both as a tool when appraising medical education research articles and as a primer for clinicians interested in pursuing scholarship in medical education.

An Evidence-Based Checklist for Improving Scoping Review Quality

A scoping review aims to systematically explore and map the research available from a wide range of sources. The objective of this study was to produce a scoping review checklist to guide future scoping studies to enable rigorous review and critique of phenomena of interest. The methods used included a review of literature, expert consensus group meetings, a modified Delphi survey and, finally, verification against recent scoping study examples. Results showed that the checklist was able to identify key elements of scoping reviews. The 22-item Scoping Review Checklist (SRC), which includes two optional stakeholder consultation items, has been developed using rigorous recommended approaches. The checklist can be used to guide the conduct and critique of scoping studies.

Downstream Processing Technologies/Capturing and Final Purification : Opportunities for Innovation, Change, and Improvement. A Review of Downstream Processing Developments in Protein Purification

Increased pressure on upstream processes to maximize productivity has been crowned with great success, although at the cost of shifting the bottleneck to purification. As drivers were economical, focus is on now on debottlenecking downstream processes as the main drivers of high manufacturing cost. Devising a holistically efficient and economical process remains a key challenge. Traditional and emerging protein purification strategies with particular emphasis on methodologies implemented for the production of recombinant proteins of biopharmaceutical importance are reviewed. The breadth of innovation is addressed, as well as the challenges the industry faces today, with an eye to remaining impartial, fair, and balanced. In addition, the scope encompasses both chromatographic and non-chromatographic separations directed at the purification of proteins, with a strong emphasis on antibodies. Complete solutions such as integrated USP/DSP strategies (i.e., continuous processing) are discussed as well as gains in data quantity and quality arising from automation and high-throughput screening (HTS). Best practices and advantages through design of experiments (DOE) to access a complex design space such as multi-modal chromatography are reviewed with an outlook on potential future trends. A discussion of single-use technology, its impact and opportunities for further growth, and the exciting developments in modeling and simulation of DSP rounds out the overview. Lastly, emerging trends such as 3D printing and nanotechnology are covered. Graphical Abstract Workflow of high-throughput screening, design of experiments, and high-throughput analytics to understand design space and design space boundaries quickly. (Reproduced with permission from Gregory Barker, Process Development, Bristol-Myers Squibb).

Theory, a lost character? As presented in general practice education research papers

OBJECTIVES

The use of theory in research is reflected in its presence in research writing. Theory is often an ineffective presence in medical education research papers. To progress the effective use of theory in medical education, we need to understand how theory is presented in research papers. This study aims to elicit how theory is being written into general practice (GP) vocational education research papers in order to elucidate how theory might be more effectively used. This has relevance for the field of GP and for medical education more broadly.

METHODS

This is a scoping review of the presentation of theory in GP vocational education research published between 2013 and 2017. An interpretive approach is taken. We frame research papers as a form of narrative and draw on the theories of Aristotle's poetics and Campbell's monomyth. We seek parallels between the roles of theory in a research story and theories of characterisation.

RESULTS

A total of 23 papers were selected. Theories of 'reflective learning', 'communities of practice' and 'adult learning' were most used. Six tasks were assigned to theory: to align with a position; to identify a research problem; to serve as a vehicle for an idea; to provide a methodological tool; to interpret findings, and to represent an object of examination. The prominence of theory in the papers ranged from cameo to major roles. Depending on the way theory was used and the audience, theory had different impacts. There were parallels between the tasks assigned to theory and the roles of four of Campbell's archetypal characters. Campbell's typology offers guidance on how theory can be used in research paper 'stories'.

CONCLUSIONS

Theory can be meaningfully present in the story of a research paper if it is assigned a role in a deliberate way and this is articulated. Attention to the character development of theory and its positioning in the research story is important.

A Scoping Review of Medical Education Research in Neurosurgery

BACKGROUND

Little is known about the state of medical education (ME) research in neurosurgery. As ME started to develop in neurosurgery in recent years, it is important to understand the current status and develop the theory for advancement in neurosurgery. The aim of this study was to undertake a scoping review of neurosurgery literature on ME research.

METHODS

MEDLINE, SCOPUS, and PubMed databases were searched. Inclusion criteria were full-text articles in English published from January 2006 to December 2017. Research aspects included country of publication, annual number of publications, journal types, type of participants, frequently researched topics, and research design. Search terms included neurosurgery, medical education, teaching, training, learning, and curriculum.

RESULTS

A total of 9863 references were found across 3 databases. After duplicate removal and further screening, 533 references remained for coding analysis. ME research activity in neurosurgery is increasing and commonly observed in Western countries. Identified articles were mostly quantitative, with curriculum, assessment (especially simulation), and teaching and learning being the most dominant research themes.

CONCLUSIONS

This study highlights the need for enhanced quantity and quality of ME research in neurosurgery. It identifies areas of highest priority and aspects to be improved and provides us with a rationale for future development in ME in neurosurgery. These findings reveal future education research direction and programmatic research areas, while also establishing a benchmark to assess changes in educational scholarship over time.

Health Professions' Digital Education: Review of Learning Theories in Randomized Controlled Trials by the Digital Health Education Collaboration

Background

Learning theory is an essential component for designing an effective educational curriculum. Reviews of existing literature consistently lack sufficient evidence to support the effectiveness of digital interventions for health professions’ education, which may reflect disconnections among learning theories, curriculum design, use of technology, and outcome evaluation.

Objective

The aim of this review was to identify, map, and evaluate the use of learning theories in designing and implementing intervention trials of health professions’ digital education, as well as highlight areas for future research on technology-enhanced education via the establishment of a development framework for practice and research.

Methods

We performed a systematic search of Medical Literature Analysis and Retrieval System Online, Excerpta Medica database, Cochrane Central Register of Controlled Trials (Cochrane Library), PsycINFO, Cumulative Index to Nursing and Allied Health Literature, Education Resources Information Center, and Web of Science for randomized controlled trials (RCTs) published between 2007 and 2016.

Results

A total of 874 RCTs on digital health education were identified and categorized into online-offline, mobile digital education, and simulation-based modalities for pre and postregistration health professions’ education. Of these, 242 studies were randomly selected for methodological review and thematic analysis. Data were extracted by one author using a standardized form, with a (48/242, 20%) random sample extracted by a second author, in duplicate. One-third (81/242, 33.4%) of the studies reported single or multiple learning theories in design, assessment, conceptualization, or interpretation of outcomes of the digital education interventions. Commonly reported learning theories were problem-based learning (16/81, 20%), social learning theory (11/81, 14%), and cognitive theory of multimedia learning (10/81, 12%). Most of these studies assessed knowledge (118/242, 48.8%), skills (62/242, 25.6%), and performance (59/242, 24.3%) as primary outcomes with nonvalidated assessment tools (151/242, 62.4%). Studies with reported learning theories (χ²₁=8.2; P=.002) and validated instruments (χ²₁=12.6; P=.006) have shown effective acquisition of learning outcomes.

Conclusions

We proposed a Theory-Technology Alignment Framework to safeguard the robustness and integrity of the design and implementation of future digital education programs for the training of health professionals.

Enhancing Educational Scholarship Through Conceptual Frameworks: A Challenge and Roadmap for Medical Educators

Historically, health sciences education has been guided by tradition and teacher preferences rather than by the application of practices supported by rigorous evidence of effectiveness. Although often underutilized, conceptual frameworks-theories that describe the complexities of educational and social phenomenon-are essential foundations for scholarly work in education. Conceptual frameworks provide a lens through which educators can develop research questions, design research studies and educational interventions, assess outcomes, and evaluate the impact of their work. Given this vital role, conceptual frameworks should be considered at the onset of an educational initiative. Use of different conceptual frameworks to address the same topic in medical education may provide distinctive approaches. Exploration of educational issues by employing differing, theory-based approaches advances the field through the identification of the most effective educational methods. Dissemination of sound educational research based on theory is similarly essential to spark future innovation. Ultimately, this rigorous approach to medical education scholarship is necessary to allow us to establish how our educational interventions impact the health and well-being of our patients.

Intervention Descriptions in Medical Education: What Can Be Improved? A Systematic Review and Checklist

PURPOSE

Many medical education studies focus on the effectiveness of educational interventions. However, these studies often lack clear, thorough descriptions of interventions that would make the interventions replicable. This systematic review aimed to identify gaps and limitations in the descriptions of educational interventions, using a comprehensive checklist.

METHOD

Based on the literature, the authors developed a checklist of 17 criteria for thorough descriptions of educational interventions in medical education. They searched the Ovid MEDLINE, Embase, and ERIC databases for eligible English-language studies published January 2014-March 2016 that evaluated the effects of educational interventions during classroom teaching in postgraduate medical education. Subsequently, they used this checklist to systematically review the included studies. Descriptions were scored 0 (no information), 1 (unclear/partial information), or 2 (detailed description) for each of the 16 scorable criteria (possible range 0-32).

RESULTS

Among the 105 included studies, the criteria most frequently reported in detail were learning needs (78.1%), content/subject (77.1%), and educational strategies (79.0%). The criteria least frequently reported in detail were incentives (9.5%), environment (5.7%), and planned and unplanned changes (12.4%). No article described all criteria. The mean score was 15.9 (SD 4.1), with a range from 8 (5 studies) to 25 (1 study). The majority (76.2%) of articles scored 11-20.

CONCLUSIONS

Descriptions were frequently missing key information and lacked uniformity. The results suggest a need for a common standard. The authors encourage others to validate, complement, and use their checklist, which could lead to more complete, comparable, and replicable descriptions of educational interventions.