A bibliometric analysis of evaluative medical education studies: characteristics and indexing accuracy

Hotspots and frontiers of autophagy and chemotherapy in lung cancer: a bibliometric and visualization analysis from 2003 to 2023

Autophagy was considered to induce resistance in chemotherapy, which was significantly associated with proliferation of cancer; however, few bibliometric studies on the relation between autophagy and chemotherapy in lung cancer are available. The aim of the present study was to provide a comprehensive overview of the knowledge structure and research hotspots of autophagy and chemotherapy in lung cancer by bibliometric analysis. Publications related to autophagy and chemotherapy in lung cancer from 2003 to 2023 were searched on the Web of Science Core Collection (WoSCC) database. The bibliometric analysis was conducted by using VOSviewers, CiteSpace, and the R package "bibliometrix." A total of 675 articles from 70 countries, led by China and the United States, were included in the analysis. The number of publications related to autophagy and chemotherapy in lung cancer is increasing year by year. Nanjing Medical University, Zhejiang University, China Medical University, and Sichuan University are among the main research institutions contributing to this field. The journal Cancers is the most popular publication in this area, with Autophagy being the most co-cited journal. These publications involve 4481 authors, with Chiu Chien-chih and Gewirtz David having published the most papers, and Noboru Mizushima being the most frequently co-cited author. Studying the relation between autophagy and chemotherapy in the occurrence and development of lung cancer, and exploring therapeutic strategies involving autophagy and chemotherapy in lung cancer, are the primary topics in this research field. "Tumor stem cells," "microRNA," and "EGFR" emerge as the primary keywords in the emerging research hotspots. Indeed, this bibliometric study provides valuable insights into the research trends and developments concerning autophagy and chemotherapy in lung cancer. By identifying recent research frontiers and highlighting hot directions, this study serves as a valuable reference for scholars interested in understanding the relationship between autophagy and chemotherapy in lung cancer. The comprehensive summary of findings offers a foundation for further exploration and advancement in this critical area of cancer research.

Knowledge atlas analysis of virtual vascular interventional studies

OBJECTIVE

To examine the research status, hot spots, and development trend of virtual vascular intervention simulation training in order to organize the knowledge and support its practice.

METHODS

Publications on virtual vascular intervention and respective teaching evaluation (2001-2021) were retrieved from Web of Science. Citespace 5.2.R2 and Carrot2 software was used to perform the bibliometric and visual analysis on virtual vascular intervention research networks.

RESULTS

Of the 1543 articles retrieved, 1520 were relevant to virtual vascular intervention research. Studies on vascular intervention emerged from 2002 with an upward trend being observed from 2010. The trend peaked between 2019 and 2020 with ∼140 published articles (9.21%) during the period. Spain published most studies (n = 146) followed by USA (n = 99) and England (n = 49). COMPUT EDUC (n = 177; 14.47%) journal publishing the highest number of articles. Among 705 authors, GARRISON D R, MAYER RE, DEDE C, COOK DA had highest publications. The frequent keywords used were "Virtual reality" (n = 105) and "education" (n = 105). Low cooperation between countries, authors' networks, and relatively stable academic team was noted making clinical application of virtual vascular intervention weak.

CONCLUSION

Considerable growth was observed in vascular intervention therapy using VR technology in the last 20 years. With consistent efforts like strengthening cross-institutional and multidisciplinary collaboration networks, VR based simulation training capability can be increased that in turn can improve the outcomes of complex vascular interventions.

The cardiovascular system at high altitude: A bibliometric and visualization analysis

BACKGROUND

When exposed to high-altitude environments, the cardiovascular system undergoes various changes, the performance and mechanisms of which remain controversial.

AIM

To summarize the latest research advancements and hot research points in the cardiovascular system at high altitude by conducting a bibliometric and visualization analysis.

METHODS

The literature was systematically retrieved and filtered using the Web of Science Core Collection of Science Citation Index Expanded. A visualization analysis of the identified publications was conducted employing CiteSpace and VOSviewer.

RESULTS

A total of 1674 publications were included in the study, with an observed annual increase in the number of publications spanning from 1990 to 2022. The United States of America emerged as the predominant contributor, while Universidad Peruana Cayetano Heredia stood out as the institution with the highest publication output. Notably, Jean-Paul Richalet demonstrated the highest productivity among researchers focusing on the cardiovascular system at high altitude. Furthermore, Peter Bärtsch emerged as the author with the highest number of cited articles. Keyword analysis identified hypoxia, exercise, acclimatization, acute and chronic mountain sickness, pulmonary hypertension, metabolism, and echocardiography as the primary research hot research points and emerging directions in the study of the cardiovascular system at high altitude.

CONCLUSION

Over the past 32 years, research on the cardiovascular system in high-altitude regions has been steadily increasing. Future research in this field may focus on areas such as hypoxia adaptation, metabolism, and cardiopulmonary exercise. Strengthening interdisciplinary and multi-team collaborations will facilitate further exploration of the pathophysiological mechanisms underlying cardiovascular changes in high-altitude environments and provide a theoretical basis for standardized disease diagnosis and treatment.

A Bibliometric Analysis of the 100 Most Cited Articles on Problem-Based Learning in Medical Education

Problem-based learning (PBL) is an instructional approach used in medical education that is characterized by solving problems in small groups with tutor guidance. More than 50 years since PBL's inception, many questions remain to be addressed about its processes and learning outcomes. The purpose of the study was to examine the bibliometric characteristics of the 100 most cited articles on PBL in medical education and to identify landmark papers that have made significant contributions to PBL research. Results were systematically reviewed for citation frequency, publication year, journal, article type, article focus, authors, author collaboration, and country collaboration. The number of citations ranged from 81 to 3531 times cited with 31,041 total citations. The articles were contributed by 211 authors in 23 journals and most articles (68%) were published in Medical Education, Academic Medicine, and Medical Teacher. The majority of the articles (71%) originated from Netherlands, Canada, and the United States and six prolific authors were identified. Almost half of the articles are classified as empirical research. Article foci included theoretical foundations of PBL, curriculum design, learning outcomes and processes, tutors, assessment, guides to PBL implementation, commentaries, and student well-being. The strong author and country collaborations indicate continued global interest in the PBL instructional method, which is likely to remain an active topic of research as the evidence of its effectiveness over traditional instructional methods as well as its most impactful components is inconclusive.

Patient involvement in assessment of postgraduate medical learners: A scoping review

CONTEXT

Competency-based assessment of learners may benefit from a more holistic, inclusive, approach for determining readiness for unsupervised practice. However, despite movements towards greater patient partnership in health care generally, inclusion of patients in postgraduate medical learners' assessment is largely absent.

METHODS

We conducted a scoping review to map the nature, extent and range of literature examining the inclusion (or exclusion) of patients within the assessment of postgraduate medical learners. Guided by Arskey and O'Malley's framework and informed by Levac et al. and Thomas et al., we searched two databases (MEDLINE® and Embase®) from inception until February 2021 using subheadings related to assessment, patients and postgraduate learners. Data analysis examined characteristics regarding the nature and factor influencing patient involvement in assessment.

RESULTS

We identified 41 papers spanning four decades. Some literature suggests patients are willing to be engaged in assessment, however choose not to engage when, for example, language barriers may exist. When stratified by specialty or clinical setting, the influence of factors such as gender, race, ethnicity or medical condition seems to remain consistent. Patients may participate in assessment as a stand-alone group or part of a multi-source feedback process. Patients generally provided high ratings but commented on the observed professional behaviours and communication skills in comparison with physicians who focused on medical expertise.

CONCLUSION

Factors that influence patient involvement in assessment are multifactorial including patients' willingness themselves, language and reading-comprehension challenges and available resources for training programmes to facilitate the integration of patient assessments. These barriers however are not insurmountable. While understudied, research examining patient involvement in assessment is increasing; however, our review suggests that the extent which the unique insights will be taken up in postgraduate medical education may be dependent on assessment systems readiness and, in particular, physician readiness to partner with patients in this way.

Barriers and facilitators to program directors' use of the medical education literature: a qualitative study

BACKGROUND

It is unclear how often frontline clinical teachers are using this literature and its evidence base in teaching and assessment. Our study purpose was to examine postgraduate program director perspectives on the utilization and integration of evidence-based medical education literature in their teaching and assessment practices.

METHODS

The authors conducted semi-structured telephone interviews with a convenience sample of current and former program directors from across Canada. Interviews were transcribed and analyzed inductively to distil pertinent themes.

RESULTS

In 2017, 11 former and current program directors participated in interviews. Major themes uncovered included the desire for time-efficient and easily adaptable teaching and assessment tools. Participants reported insufficient time to examine the medical education literature, and preferred that it be 'synthesized for them'. (i.e., Best evidence guidelines). Participants recognised continuing professional development and peer to peer sharing as useful means of education about evidence-based tools. Barriers to the integration of the literature in practice included inadequate time, lack of financial compensation for teaching and assessment, and the perception that teaching and assessment of trainees was not valued in academic promotion.

DISCUSSION

Faculty development offices should consider the time constraints of clinical teachers when planning programming on teaching and assessment. To enhance uptake, medical education publications need to consider approaches that best meet the needs of a targeted audiences, including frontline clinical teachers. This may involve novel methods and formats that render evidence and findings from their studies more easily 'digestible' by clinical teachers to narrow the knowledge to practice gap.

A Bibliometric Analysis of Publication Patterns in Pediatric Neurology

PURPOSE

To examine the publication patterns of pediatric neurology articles in general pediatric, general neurology, and neurology subspecialty journals using a bibliometric approach.

METHODS

The top 5 journals in general pediatrics, general neurology and neurology subspecialties were identified using the 2017 Journal Citations Report (JCR). For general pediatric journals, we selected 4 pediatric subspecialties for comparison of publication patterns with neurology: immunology, endocrinology, gastroenterology, and respirology. For general neurology and neurology subspecialty journals, we searched both the top 5 neurology and neurology subspecialty journals for pediatric articles. Using Ovid Medline, we identified articles published between 2009-2017.

RESULTS

With regards to child neurology-based articles, 1501 were published in general pediatrics journals, 643 in general neurology journals and 685 in neurology subspecialty journals. Examination of the top pediatric journals revealed that Pediatrics published the most neurology-based articles. Neurology-based studies were published more frequently than other pediatric subspecialty studies. Of the top general neurology Neurology published the most child neurology-based articles, while Epilepsia published the most child neurology-based articles out of neurology subspecialty journals. Cohort studies were the most frequent study type across all journals.

CONCLUSION

Our study revealed that child neurology articles are published more often in pediatric journals as opposed to general neurology and neurology subspecialty journals. We also found that in general pediatric journals, neurology-based articles are published more frequently compared to other specialties. Our results provide guidance to authors when considering submission of their pediatric neurology research.

Bibliometric Analysis of Neurology Articles Published in General Medicine Journals

IMPORTANCE

A significant portion of neurology literature is published in general medicine journals. Despite this, a detailed examination of publication patterns of neurology articles in these journals has not yet been carried out.

OBJECTIVE

To examine the publication patterns of neurology articles in general medicine journals during a 10-year period using a bibliometric approach.

DESIGN, SETTING, AND PARTICIPANTS

This cross-sectional bibliometric analysis identified the top 5 general medicine journals using the 2017 Journal Citations Report. Four other medical subspecialties (ie, immunology, endocrinology, gastroenterology, and pulmonology) were selected for comparison of publication patterns with neurology. Using MEDLINE, the 5 journals were searched for articles published between 2009 and 2018 that were indexed with the following MeSH terms: nervous system diseases, immune system diseases, endocrine system diseases, gastrointestinal diseases, and respiratory tract diseases. Data analysis was conducted from February 2019 to December 2020.

MAIN OUTCOMES AND MEASURES

Publications were characterized by journal, specialty, and study design. These variables were used for comparison of publication numbers.

RESULTS

The general medicine journals with the 5 highest journal impact factors (JIF) were New England Journal of Medicine (NEJM; JIF 79.3), Lancet (JIF 53.3), JAMA (JIF 47.7), BMJ (JIF 23.6), and PLOS Medicine (JIF 11.7). Our bibliometric search yielded 3719 publications, of which 1098 (29.5%) were in neurology. Of these 1098 neurology publications, 317 (28.9%) were published in NEJM, 205 (18.7%) in Lancet, 284 (25.9%) in JAMA, 214 (19.5%) in BMJ, and 78 (7.1%) in PLOS Medicine. Randomized clinical trials were the most frequent neurology study type in general medicine journals (519 [47.3%]). The number of publications in each of the other specialties were as follows: immunology, 817; endocrinology, 633; gastroenterology, 353; and pulmonology, 818.

CONCLUSIONS AND RELEVANCE

The results of this study provide some guidance to authors regarding where they may wish to consider submitting their neurology research. Compared with other specialties, neurology-based articles are published more frequently in general medicine journals.

Knowledge syntheses in medical education: A bibliometric analysis

PURPOSE

This bibliometric analysis maps the landscape of knowledge syntheses in medical education. It provides scholars with a roadmap for understanding where the field has been and where it might go in the future, thereby informing research and educational practice. In particular, this analysis details the venues in which knowledge syntheses are published, the types of syntheses conducted, citation rates they produce, and altmetric attention they garner.

METHOD

In 2020, the authors conducted a bibliometric analysis of knowledge syntheses published in 14 core medical education journals from 1999 to 2019. To characterize the studies, metadata were extracted from PubMed, Web of Science, Altmetrics Explorer, and Unpaywall.

RESULTS

The authors analyzed 963 knowledge syntheses representing 3.1% of the total articles published (n = 30,597). On average, 45.9 knowledge syntheses were published annually (SD = 35.85, median = 33), and there was an overall 2620% increase in the number of knowledge syntheses published from 1999 to 2019. The journals each published, on average, a total of 68.8 knowledge syntheses (SD = 67.2, median = 41) with Medical Education publishing the most (n = 189; 19%). Twenty-one types of knowledge synthesis were identified, the most prevalent being systematic reviews (n = 341; 35.4%) and scoping reviews (n = 88; 9.1%). Knowledge syntheses were cited an average of 53.80 times (SD = 107.12, median = 19) and received a mean Altmetric Attention Score of 14.12 (SD = 37.59, median = 6).

CONCLUSIONS

There has been considerable growth in knowledge syntheses in medical education over the past 20 years, contributing to medical education's evidence base. Beyond this increase in volume, researchers have introduced methodological diversity in these publications, and the community has taken to social media to share knowledge syntheses. Implications for the field, including the impact of synthesis types and their relationship to knowledge translation, are discussed.

The use of BEME reviews in the medical education literature

INTRODUCTION

Knowledge syntheses in medical education are intended to promote the translation to, and mobilization of, research knowledge into practice. Despite the effort invested in conducting them, how these knowledge syntheses are used is unclear. This study aimed to explore how knowledge syntheses published by the Best Evidence Medical Education Collaboration (BEME) have been used in a cross-section of published literature.

METHODS

Citation patterns for BEME reviews were explored using data drawn from Web of Science and Scopus, and a sub-sample of citing papers.

RESULTS

Bibliometric data on 3419 papers citing 29 BEME reviews were analysed. More detailed data were extracted from a random sample of 629 full-text papers.

DISCUSSION

BEME reviews were most often positioned to consolidate and summarize the current state of knowledge on a particular topic and to identify gaps in the literature; they were also used to justify current research, and less frequently to contextualize and explain results, or direct future areas of research. Their use to identify instruments or methodological approaches was relatively absent.

CONCLUSION

While BEME reviews are primarily used to justify and support other studies, the current literature does not demonstrate their translation to educational practice.

Publication Trends in Psychotherapy: Bibliometric Analysis of the Past 5 Decades

OBJECTIVE

Bibliometric analyses are commonly used to document publication trends over time; however, this methodology has not been used to investigate possible trends concerning publishing about psychotherapy brands. In this study, the authors sought to identify the publication trends of peer-reviewed articles about 30 psychotherapy brands.

METHODS

Analyses were focused on the past 50 years and on each decade from 1970 to 2019. All searches were performed between October 2018 and January 2019 on the EbscoHost platform. Two databases were selected for the searches: PsycINFO and PubMed.

RESULTS

In the 28,594 articles reviewed, most published articles concerned cognitive-behavioral therapy (CBT), and five brands accounted for almost 78% of all publications: CBT, psychoanalysis, family systems therapy, behavioral therapy, and cognitive therapy. Three trends were identified across decades: five therapies consistently yielded the largest number of publications, the number of publications focused on therapies with less research support declined from the 1970s to the 1990s, and publications about therapies with more of a research basis increased in the 1990s through the 2010s. Publications on meditation and mindfulness presented the most salient growth area for all psychotherapies across the 5 decades. A few psychotherapy brands have dominated the publishing realm during the past 50 years and across each decade.

CONCLUSIONS

Possible explanations for these publication trends were considered, including the emergence of the evidence-based therapy movement and various sociohistorical changes. Potential psychotherapy publications trends in the future are discussed.

Characterizing the literature on validity and assessment in medical education: a bibliometric study

INTRODUCTION

Assessment in Medical Education fills many roles and is under constant scrutiny. Assessments must be of good quality, and supported by validity evidence. Given the high-stakes consequences of assessment, and the many audiences within medical education (e. g., training level, specialty-specific), we set out to document the breadth, scope, and characteristics of the literature reporting on validation of assessments within medical education.

METHOD

Searches in Medline (Ovid), Web of Science, ERIC, EMBASE (Ovid), and PsycINFO (Ovid) identified articles reporting on assessment of learners in medical education published since 1999. Included articles were coded for geographic origin, journal, journal category, targeted assessment, and authors. A map of collaborations between prolific authors was generated.

RESULTS

A total of 2,863 articles were included. The majority of articles were from the United States, with Canada producing the most articles per medical school. Most articles were published in journals with medical categorizations (73.1% of articles), but Medical Education was the most represented journal (7.4% of articles). Articles reported on a variety of assessment tools and approaches, and 89 prolific authors were identified, with a total of 228 collaborative links.

DISCUSSION

Literature reporting on validation of assessments in medical education is heterogeneous. Literature is produced by a broad array of authors and collaborative networks, reported to a broad audience, and is primarily generated in North American and European contexts. Our findings speak to the heterogeneity of the medical education literature on assessment validation, and suggest that this heterogeneity may stem, at least in part, from differences in constructs measured, assessment purposes, or conceptualizations of validity.

Blurring the boundaries: using institutional ethnography to inquire into health professions education and practice

CONTEXT

Qualitative, social science approaches to research have surged in popularity within health professions education (HPE) over the past decade. Institutional ethnography (IE) offers the field another sociological approach to inquiry. Although widely used in nursing and health care research, IE remains relatively uncommon in the HPE research community. This article provides a brief introduction to IE and suggests why HPE researchers may wish to consider it for future studies.

METHODS

Part 1 of this paper presents IE's conceptual grounding in: (i) the entry point to inquiry ('materiality'), (ii) a generous definition of 'work' and (iii) a focus on how 'texts' such as policies, forms and written protocols influence activity. Part 2 of this paper outlines the method's key features through exemplars from our own research. Part 3 discusses the ways in which research that blurs the lines between educational and clinical practice can be both generative for HPE and accomplished using IE.

RESULTS

The authors demonstrate the usefulness of IE for studying complex social issues in HPE. It is posited that a key added value of IE is that it goes beyond individual-level explanations of problems and phenomena, yet also closely studies individuals' activities, rather than remaining at an abstract or distant level of analysis. Thereby, IE can result in feasible and meaningful social change at the nexus of health professions education and other social systems such as clinical practice.

CONCLUSIONS

IE adds to the growing qualitative research toolkit for HPE researchers. It is worth considering because it may enable change through the study of HPE in relation to other social processes, structures and systems, including the clinical practice world. A particular benefit may be found in blending HPE research with research on clinical practice, toward changing practice and policy through IE, given the interrelated nature of these fields.

Reporting quality and risk of bias in randomised trials in health professions education

CONTEXT

Complete reporting of research is essential to enable consumers to accurately appraise, interpret and apply findings. Quality appraisal checklists are giving way to tools that judge the risk for bias.

OBJECTIVES

We sought to determine the prevalence of these complementary aspects of research reports (completeness of reporting and perceived risk for bias) of randomised studies in health professions education.

METHODS

We searched bibliographic databases for randomised studies of health professions education. We appraised two cohorts representing different time periods (2008-2010 and 2014, respectively) and worked in duplicate to apply the CONSORT guidelines and Cochrane Risk of Bias tool. We explored differences between time periods using independent-samples t-tests or the chi-squared test, as appropriate.

RESULTS

We systematically identified 180 randomised studies (2008-2010, n = 150; 2014, n = 30). Frequencies of reporting of CONSORT elements within full-text reports were highly variable and most elements were reported in fewer than 50% of studies. We found a statistically significant difference in the CONSORT reporting index (maximum score: 500) between the 2008-2010 (mean ± standard deviation [SD]: 242.7 ± 55.6) and 2014 (mean ± SD: 311.6 ± 53.2) cohorts (p < 0.001). High or unclear risk for bias was most common for allocation concealment (157, 87%) and blinding of participants (147, 82%), personnel (152, 84%) and outcome assessors (112, 62%). Most risk for bias elements were judged to be unclear (range: 51-84%). Risk for bias elements significantly improved over time for blinding of participants (p = 0.007), incomplete data (p < 0.001) and the presence of other sources of bias (p < 0.001).

CONCLUSIONS

Reports of randomised studies in health professions education frequently omit elements recommended by the CONSORT statement. Most reports were assessed as having a high or unclear risk for bias. Greater attention to how studies are reported at study outset and in manuscript preparation could improve levels of complete transparent reporting.

Eight years' experience with a Medical Education Journal Club in Mexico: a quasi-experimental one-group study

BACKGROUND

A time-honored strategy for keeping up to date in medicine and improving critical appraisal skills is the Journal Club (JC). There are several reports of its use in medicine and allied health sciences but almost no reports of JC focused on medical education. The purpose of the study is to describe and evaluate an eight years' experience with a medical education Journal Club (MEJC).

METHODS

We started a monthly medical education JC in 2006 at UNAM Faculty of Medicine in Mexico City. Its goal is to provide faculty with continuing professional development in medical education. A discussion guide and a published paper were sent 2 weeks before sessions. We reviewed the themes and publication types of the papers used in the sessions, and in June-July 2014 administered a retrospective post-then-pre evaluation questionnaire to current participants that had been regular attendees to the JC for more than 2 years. The retrospective post-then-pre comparisons were analyzed with Wilcoxon signed-rank test. Effect sizes were calculated for the pre-post comparisons with Cohen's r.

RESULTS

There have been 94 MEJC sessions until July 2014. Average attendance is 20 persons, a mix of clinicians, educators, psychologists and a sociologist. The articles were published in 32 different journals, and covered several medical education themes (curriculum, faculty development, educational research methodology, learning methods, assessment, residency education). 22 Attendees answered the evaluation instrument. The MEJC had a positive evaluation from good to excellent, and there was an improvement in self-reported competencies in medical education literature critical appraisal and behaviors related to the use of evidence in educational practice, with a median effect size higher than 0.5. The evaluation instrument had a Cronbach's alpha of 0.96.

CONCLUSIONS

A periodic Medical Education Journal Club can improve critical appraisal of the literature, and be maintained long-term using evidence-based strategies. This activity is a useful adjunct to the scholarship of teaching.

The top-cited articles in medical education: a bibliometric analysis

PURPOSE

To identify and examine the characteristics of the 50 top-cited articles in medical education.

METHOD

Two searches were conducted in the Web of Knowledge database in March 2014: a search of medical education journals in the category "Education, Scientific Discipline" (List A) and a keyword search across all journals (List B). Articles identified were reviewed for citation count, country of origin, article type, journal, authors, and publication year.

RESULTS

Both lists included 56 articles, not 50, because articles with the same absolute number of citations shared the same rank. The majority of List A articles were published in Academic Medicine (34; 60.7%) and Medical Education (16; 28.6%). In List B, 27 articles (48.2%) were published in medical education journals, 19 (33.9%) in general medicine and surgery journals, and 10 (17.9%) in higher education and educational psychology journals. Twenty-six articles were included in both lists, with different rankings. Reviews and articles constituted the majority of articles; there were only 8 research papers in List A and 13 in List B. Articles mainly originated from the United States, Canada, the Netherlands, and the United Kingdom. The majority were published from 1979 to 2007. There was no correlation between year and citation count.

CONCLUSIONS

The finding that over half of List B articles were published in nonmedical education journals is consistent with medical education's integrated nature and subspecialty breadth. Twenty of these articles were among their respective non-medical-education journals' 50 top-cited papers, showing that medical education articles can compete with subject-based articles.

A BEME (Best Evidence in Medical Education) systematic review of the use of workplace-based assessment in identifying and remediating poor performance among postgraduate medical trainees

BACKGROUND

Workplace-based assessments were designed to facilitate observation and structure feedback on the performance of trainees in real-time clinical settings and scenarios. Research in workplace-based assessments has primarily centred on understanding psychometric qualities and performance improvement impacts of trainees generally. An area that is far less understood is the use of workplace-based assessments for trainees who may not be performing at expected or desired standards, referred to within the literature as trainees 'in difficulty' or 'underperforming'. In healthcare systems that increasingly depend on service provided by junior doctors, early detection (and remediation) of poor performance is essential. However, barriers to successful implementation of workplace-based assessments (WBAs) in this context include a misunderstanding of the use and purpose of these formative assessment tools. This review aims to explore the impact - or effectiveness - of workplace-based assessment on the identification of poor performance and to determine those conditions that support and enable detection, i.e. whether by routine or targeted use where poor performance is suspected. The review also aims to explore what effect (if any) the use of WBA may have on remediation or on changing clinical practice. The personal impact of the detection of poor performance on trainees and/or trainers may also be explored.

METHODS/DESIGN

Using BEME (Best Evidence in Medical Education) Collaboration review guidelines, nine databases will be searched for English-language records. Studies examining interventions for workplace-based assessment either routinely or in relation to poor performance will be included. Independent agreement (kappa .80) will be achieved using a randomly selected set of records prior to commencement of screening and data extraction using a BEME coding sheet modified as applicable (Buckley et al., Med Teach 31:282-98, 2009) as this has been used in previous WBA systematic reviews (Miller and Archer, BMJ doi:10.1136/bmj.c5064, 2010) allowing for more rigorous comparisons with the published literature. Educational outcomes will be evaluated using Kirkpatrick's framework of educational outcomes using Barr's adaptations (Barr et al., Evaluations of interprofessional education; a United Kingdom review of health and social care, 2000) for medical education research.

DISCUSSION

Our study will contribute to an ongoing international debate regarding the applicability of workplace-based assessments as a meaningful formative assessment approach within the context of postgraduate medical education.

SYSTEMATIC REVIEW REGISTRATION

The review has been registered by the BEME Collaboration www.bemecollaboration.org .

The h-index in medical education: an analysis of medical education journal editorial boards

BACKGROUND

Disciplines differ in their authorship and citation practices, thus discipline-specific h-index norms are desirable. Thus the goal of this study was to examine the relationship between the h-index and academic rank in the field of medical education, and the differences in the h-index between MD's and PhD's in this field.

METHODS

Due to the absence of a formalized registry of medical educators, we sampled available editorial board membership (considered a proxy for identifying 'career' medical educators) to establish h-index values. These were determined using Web of Science (WoS) and Google Scholar (GS), and internet searching was used to determine their academic rank. The correlation between authors' h-indices derived from WoS and GS was also determined.

RESULTS

130 editors were identified (95 full professors, 21 associate professors, 14 assistant professors). A significant difference was noted between the h-indices of full professors and associate/assistant professors (p < .001). Median h-indices equaled 14 for full professors (Interquartile range [IQR] =11); 7 for associate professors (IQR =7) and 6.5 for assistant professors (IQR = 8). h-indices of MD's and PhD's did not differ significantly. Moderate correlation between GS and WOS h-indices was noted R = 0.46, p < .001.

CONCLUSIONS

The results provide some guidance as to the expected h-indices of a select group of medical educators. No differences appear to exist between assistant professor and associate professor ranks or between MD's and PhD's.

Productivity in medical education research: an examination of countries of origin

BACKGROUND

Productivity and countries of origin of publications within the field of medical education research have not been explored. Using bibliometric techniques we conducted an analysis of studies evaluating medical education interventions, examining the country where research originated as well as networks of authors within countries identified as 'most productive'.

METHODS

PubMed was used to search for evaluative studies of medical education. We then examined relative productivity of countries with >100 publications in our sample (number of publications/number of medical schools in country). Author networks from the top 2 countries with the highest relative productivity were constructed.

RESULTS

6874 publications from 18,883 different authors were included. The countries with the highest relative publication productivity were Canada (37.1), Netherlands (28.3), New Zealand (27), the UK (23), and the U.S.A (17.1). Author collaboration networks differed in both numbers of authors and intensity of collaborations in the countries with highest relative productivity.

CONCLUSIONS

In terms of the number of publications of evaluative studies in medical education, Canada.