Content Evaluation of Physical Medicine and Rehabilitation Residency Websites

Opportunities to Improve Communication With Residency Applicants: Cross-Sectional Study of Obstetrics and Gynecology Residency Program Websites

Background

As part of the residency application process in the United States, many medical specialties now offer applicants the opportunity to send program signals that indicate high interest to a limited number of residency programs. To determine which residency programs to apply to, and which programs to send signals to, applicants need accurate information to determine which programs align with their future training goals. Most applicants use a program's website to review program characteristics and criteria, so describing the current state of residency program websites can inform programs of best practices.

Objective

This study aims to characterize information available on obstetrics and gynecology residency program websites and to determine whether there are differences in information available between different types of residency programs.

Methods

This was a cross-sectional observational study of all US obstetrics and gynecology residency program website content. The authorship group identified factors that would be useful for residency applicants around program demographics and learner trajectories; application criteria including standardized testing metrics, residency statistics, and benefits; and diversity, equity, and inclusion mission statements and values. Two authors examined all available websites from November 2011 through March 2022. Data analysis consisted of descriptive statistics and one-way ANOVA, with P<.05 considered significant.

Results

Among 290 programs, 283 (97.6%) had websites; 238 (82.1%) listed medical schools of current residents; 158 (54.5%) described residency alumni trajectories; 107 (36.9%) included guidance related to the preferred United States Medical Licensing Examination Step 1 scores; 53 (18.3%) included guidance related to the Comprehensive Osteopathic Medical Licensing Examination Level 1 scores; 185 (63.8%) included international applicant guidance; 132 (45.5%) included a program-specific mission statement; 84 (29%) included a diversity, equity, and inclusion statement; and 167 (57.6%) included program-specific media or links to program social media on their websites. University-based programs were more likely to include a variety of information compared to community-based university-affiliated and community-based programs, including medical schools of current residents (113/123, 91.9%, university-based; 85/111, 76.6%, community-based university-affiliated; 40/56, 71.4%, community-based; P<.001); alumni trajectories (90/123, 73.2%, university-based; 51/111, 45.9%, community-based university-affiliated; 17/56, 30.4%, community-based; P<.001); the United States Medical Licensing Examination Step 1 score guidance (58/123, 47.2%, university-based; 36/111, 32.4%, community-based university-affiliated; 13/56, 23.2%, community-based; P=.004); and diversity, equity, and inclusion statements (57/123, 46.3%, university-based; 19/111, 17.1%, community-based university-affiliated; 8/56, 14.3%, community-based; P<.001).

Conclusions

There are opportunities to improve the quantity and quality of data on residency websites. From this work, we propose best practices for what information should be included on residency websites that will enable applicants to make informed decisions.

Orthopaedic Surgery Residency Program Websites: Assessing Information Gaps and Opportunities for Attracting Applicants

INTRODUCTION

The quality and volume of information on orthopaedic surgery residency program websites are helpful for informed decision making of prospective applicants. The purpose of this study was to evaluate the content on orthopaedic surgery residency program websites and identify areas for improvement.

METHODS

In November 2023, 203 orthopaedic surgery residency programs were reviewed using the Association of American Medical Colleges Residency Explorer tool. Fourteen nonfunctional websites were excluded, leaving n = 189. Websites were assessed for 11 variables: resources for assisting resident research (eg, biomechanics laboratory, research assistant, and advisors), residents' past institutional affiliations, current resident profiles (including hometown and interests), alumni data, fellowship matches, American Board of Orthopedic Surgery I/II pass rates, Alpha Omega Alpha Society affiliation, target United States Medical Licensing Examination/ Comprehensive Osteopathic Medical Learning Exam Step 2 scores, educational activities (journal club, grand rounds, didactics), evaluation methods, and mentorship.

RESULTS

Website variables differed markedly among identified program types, which included community programs, university programs (UPs), and affiliated programs (APs). Disparities were noted in the disclosure of residents' previous institutions ( P = 0.001), the availability of alumni names ( P = 0.001), research resources ( P = 0.001), academic activities ( P = 0.007), assigned mentorship programs ( P = 0.006), and fellowship match information ( P = 0.027). Notably, community programs shared statistically significantly less information in these areas compared with UPs and APs ( P = 0.001). While there was a proportionate sharing of information between UPs and APs, UPs exhibited a slightly higher overall percentage of websites sharing more detailed information.

CONCLUSION

The quality and comprehensiveness of information on orthopaedic surgery residency program websites vary markedly. Most of the orthopaedic surgery residency programs provided information on residents' past institutions and academic activities. However, very few provided information on designated mentorship programs or objective measures such as target Step 2 examination scores and ABOS pass rates. Improving the quality of information available on program websites will enhance transparency and consistency, enabling prospective applicants to make more informed decisions about where to apply and interview.

An evaluation of physical medicine and rehabilitation accredited sports medicine fellowship websites

BACKGROUND

Due to the virtual format of fellowship applications, prospective physical medicine and rehabilitation (PM&R) sports medicine fellowship applicants commonly rely upon fellowship website content to make decisions about applications. Studies have shown that information available to fellowship applicants on program websites is inadequate, and there is no standard for important components of sports medicine fellowship websites.

OBJECTIVE

To determine what sports medicine fellowship program features PM&R residents, fellows, and attending physicians consider the most important to be listed on fellowship websites, as well as to assess the scope of information available on fellowship websites.

DESIGN

We developed a 30-item list of criteria that fell into categories of general information, fellowship education, recruitment, experience, and academic research. Survey participants were asked to rank the importance of those various items. We then assessed the 21 PM&R accredited sports medicine fellowship websites for the presence of these criteria.

PARTICIPANTS

Ninety-six survey respondents composed of PM&R residents, current sports medicine fellows, and attendings.

INTERVENTIONS

Not applicable.

MAIN OUTCOME MEASURE(S)

Not applicable.

RESULTS

Items ranked as most important were those in the categories of general information, fellowship experience, and fellowship recruitment. The most valued items were program location, program coordinator contact information, program length, specific requirements for applying, number of positions, types of procedures taught, and specific sites covered. Academic research items were ranked as least important.

CONCLUSIONS

Survey respondents identified many fellowship website items as important factors when applying to programs. Many of these items were not listed on fellowship websites, suggesting that adding these criteria to fellowship sites might benefit applicants. Our findings can be used to develop standardized criteria for important components of sports medicine fellowship websites to improve the application and recruitment process.

Association of Academic Physiatrists Residency and Fellowship Program Directors' Resident Recruitment Subcouncil Position Paper on Residency Recruitment

ABSTRACT

Residency recruitment practices have undergone significant changes in the last several years. Interviews are now conducted fully virtually leading to both positive and negative downstream effects, including decreased cost to applicants and programs, decreased time away from clinical activities, flexibility in scheduling, and increased applications for applicants and program directors. In response to these changes, the Association of Academic Physiatrists Residency and Fellowship Program Directors Council convened a workgroup consisting of program directors, program coordinators, residents, and medical students who reviewed the available literature to provide an evidence-based set of best practices for program leaders and applicants. Available data from the Association of American Medical Colleges and its relevance to future recruitment cycles are also discussed.

Assessment and Recommendations for Website Platforms of Neuroanesthesiology Fellowship Programs

BACKGROUND

Websites serve as recruitment and educational tools for many fellowship programs, including neuroanesthesiology. Since the COVID-19 pandemic, when interviews, conferences, and institutional visits were moved online, websites have become more important for applicants when deciding on their preferred fellowship program. This study evaluated the content of the websites of neuroanesthesiology fellowship programs.

METHODS

Neuroanesthesiology fellowship program websites were identified from the websites of the International Council on Perioperative Neuroscience Training and the Society for Neuroscience in Anesthesiology and Critical Care. The content was assessed against 24 predefined criteria.

RESULTS

Fifty-three fellowship programs were identified, of which 42 websites were accessible through a Google search and available for evaluation. The mean number of criteria met by the 42 fellowship websites was 12/24 (50%), with a range of 6 to 18 criteria. None of the evaluated fellowship websites met all 24 predefined criteria; 20 included more than 50% of the criteria, whereas 7 included fewer than 30% of the criteria. Having a functional website, accessibility through a single click from Google, and a detailed description of the fellowship program were the features of most websites. Information about salary and life in the area, concise program summaries, and biographical information of past and current fellows were missing from a majority of websites.

CONCLUSION

Important information was missing from most of the 42 evaluated neuroanesthesiology fellowship program websites, potentially hindering applicants from making informed choices about their career plans.

What Can You Learn About Psychiatry Programs from Their Website? A Content Evaluation for Potential Applicants

OBJECTIVE

This project aimed to assess the information contained on general psychiatry program websites and identify common themes that may be useful and informative for residency applicants.

METHODS

A survey study design was used to evaluate all US general psychiatry program websites as listed in the FREIDA database. The evaluation form included 44 binary (yes or no) items. Two reviewers rated each item on all program websites between September 2021 and January 2022. Item discrepancies were settled by a third reviewer. Fisher's exact tests evaluated differences between geographic regions and program types. Multidimensional scaling and Rasch modeling were conducted to examine clustering and the probability of items reported on program websites.

RESULTS

A total of 285 websites were identified; 13 were excluded. Internal consistency was high among reviewers, Cronbach's Alpha = 0.927; κ = 0.863. Websites varied considerably in quality. Significant inconsistent reporting was observed by region for current residents' photos and alumni careers (fellowship/jobs); p<0.001. Program types varied regarding information about program faculty, which included significant differences for faculty photo, faculty research interest, and faculty research publications; p<0.001.

CONCLUSIONS

While inter-rater reliability was high, considerable variation among websites was observed. Residency programs could be improved by consistently reporting resident and faculty information. Results show that applicants may encounter issues finding pertinent information, as programs' FREIDA link did not direct the user to the residency program website two-thirds of the time.

Will ChatGPT Match to Your Program?

ABSTRACT

ChatGPT and other artificial intelligence word prediction large database models are now readily available to the public. Program directors should be aware of the general features of this technology and consider its effect in graduate medical education, including the preparation of materials such as personal statements. The authors provide a sample ChatGPT-generated personal statement and general considerations for program directors and other graduate medical education stakeholders. The authors advocate that programs and applicants will be best served by transparent expectations about how/if programs will accept application materials created using artificial intelligence, starting with this application cycle. Graduate medical education will have many additional factors to consider for the innovative use and safeguards for the ethical application of artificial intelligence in clinical care and educational processes. However, the exponential increase in the application of this technology requires an urgent review for appropriate management of program procedures, iteration of policies, and a meaningful national discussion.

Evaluation of sleep medicine fellowship program websites

STUDY OBJECTIVES

Sleep fellowship program websites likely serve as a preliminary source of information for prospective fellows. Arguably, applicants have likely become even more reliant on program websites during the COVID-19 pandemic due to travel restrictions and social-distancing measures limiting in-person interviews. In this study, we evaluated the content and comprehensiveness of sleep medicine fellowship websites to identify areas of improvement.

METHODS

A list of sleep medicine fellowship programs in the United States participating in the 2021 match cycle was compiled using the Electronic Residency Application Service (ERAS) and Fellowship and Residency Electronic Interactive database (FREIDA) websites. Twenty-two prespecified content criteria related to education, recruitment, and compensation were used to evaluate each program website. Sleep programs' website comprehensiveness were compared based on US location, type, matching status, core specialty, and size of programs.

RESULTS

Seventy-eight US sleep fellowship program websites were evaluated. Most program websites had a working hyperlink on ERAS or FREIDA. There was considerable variability in content reported across program websites, with a mean of 56.8% of content items reported per program. There was a greater educational website content comprehensiveness for internal medicine compared with other specialty-based sleep programs. There was no difference in sleep programs' website comprehensiveness based on US location, type, matching status, or size of programs.

CONCLUSIONS

Website content comprehensiveness among sleep fellowship programs is variable. There is opportunity for all sleep fellowship programs to improve their websites to better inform prospective trainees.

CITATION

Shenoy S, Akberzie W, Landeo-Gutierrez JS, Leon Guerrero CR, Karroum EG. Evaluation of sleep medicine fellowship program websites. J Clin Sleep Med. 2023;19(6):1083-1088.

Analyzing Diversity Elements on Otolaryngology Residency Program Websites

OBJECTIVE

To investigate how the internet presence of Otolaryngology residency programs influences recruitment of diverse applicants.

STUDY DESIGN

Retrospective cohort study.

SETTING

Website, Instagram, and AAMC Residency Explorer data from all US, non-military, allopathic otolaryngology residency programs.

METHODS

We identified 10 common ways in which residency programs communicate their commitment to diversity through web-based platforms. We then analyzed program websites and Instagram pages for the presence of these 10 diversity elements. Univariate and multivariate linear regression were used to evaluate the association between presence of a diversity elements and the proportion of underrepresented minority residents in the program.

RESULTS

Review of 106 Otolaryngology residency program websites and Instagram pages was completed from February to May 2021. Most programs (69.8%) satisfied at least one diversity element. Of the programs reviewed, 83 had demographic information available for comparison. After adjusting for Doximity ranking, multivariate linear regression demonstrated that several elements were positive predictors of program diversity. Diversity and inclusion message (p < 0.0001), statement encouraging URM applicants (p < 0.0001), dedicated diversity chair/committee (p = 0.005), and diversity related articles/blog posts (p = 0.006) were independently associated with a greater proportion of URM residents in a given program.

CONCLUSION

These data demonstrate that providing diversity related information on residency program websites may play a role in improving program diversity. The large percentage of programs that lack the presence of any diversity element (30.2%) demonstrates that there is significant room for improvement. This study presents a promising strategy through which programs can improve recruitment of diverse residents.

Website Usability Analysis of U.S. Military Residency Programs

INTRODUCTION

The Military Match is the residency matching system for medical students attending the Uniformed Services University of Health Sciences, and the students were funded by the Health Professions Scholarship Program through the U.S. Army, Air Force, and Navy. To evaluate and compare military residency programs, students use residency program websites. Often, the residency program's website serves as a key source, or the only point of reference, when considering residency options, especially during times when face-to-face interactions are limited.This report aims to provide a systematic evaluation of military residency programs and their websites.

MATERIALS AND METHODS

Utilizing a previously published website usability scoring system, military residency programs were categorized to objectively and quantitatively analyze their websites. Usability was divided into four categories for quantifiable analysis: accessibility, marketing, content quality, and technology. The methodology for this analysis was replicated from published reports that have examined healthcare website usability. Each website was analyzed and scored in four categories: accessibility, content quality, marketing, and technology. A "General Usability" score was calculated for each website using a composite of the key factors within the four categories. An overall score was generated utilizing the weighted percentage across all four categories. To address deficiencies of the original methodology, a secondary analysis was performed on the listed websites utilizing an automated methodology for website usability.

RESULTS

A comprehensive list of 125 Accreditation Council for Graduate Medical Education U.S. Military residency program websites was compiled. Of these, 96 programs and 106 websites were evaluated. The primary analysis employing usability methodology identified technology as the highest ranked category with a score of 0.749 (SD ± 0.039) (SE 0.005) (P < .05). Marketing and content quality were the lowest scoring categories with mean scores of 0.414 (SD ± 0.054) (SE 0.006) and 0.428 (SD ± 0.229) (SE 0.027), respectively (P < .05). There was no significant difference in overall usability rankings or scores among the 96 residency program websites across the three branches (P < .05).Secondary analysis with the new usability methodology demonstrated military residency websites to exhibit more external backlinking compared to internal backlinking (P < 0.05) and no social media backlinking to any of the 106 analyzed websites. When comparing the three services, the Army had significantly lower external backlinking ranking 43.4 (P < .05) and overall backlinking ranking 56.4 (P < 0.05) when compared to the Navy (mean 48.8 and 71.7, and 43.4). There were no other differences in backlinking rankings across the three branches.

CONCLUSIONS

Residency websites have become a primary way to communicate information to applicants. By assessing the overall usability of the various military residency websites, we determined the effectiveness of these websites to relay information to prospective students interested in applying for military residency. We predict that by improving website accessibility, residency programs increase their effectiveness at communicating information to potential applicants and increase interest in military residency programs.

Analysis of Website Accessibility and Content for All 92 Accredited Hand Surgery Fellowship Programs in the United States: An Update

Purpose

The internet is an important information source for hand surgery fellowship applicants. A previous analysis of hand fellowship websites in 2014 demonstrated they were often inaccessible and incomplete. Given the increased importance of virtual information, we performed an updated assessment of the accessibility and content of hand fellowship program websites.

Methods

Websites of 92 accredited hand surgery fellowship programs were evaluated for the following: (1) accessibility; and (2) the presence of 13 fellow recruitment and 13 fellow education criteria, as defined in prior studies. We used Mann-Whitney U and Kruskal-Wallis tests to assess whether the geographic region, number of fellows, or affiliation with a top orthopedic hospital or medical school were associated with website content.

Results

Functional website links that redirect to the appropriate fellowship program website are provided for 47 (51.1%) of 92 programs on the American Society for Surgery of the Hand fellowship directory. All missing websites were accessible via independent Google searches. Fellowship program websites contained an average of 13.9 ± 4.4 total criteria (range, 3–23). Of the 15 criteria examined in both 2014 and 2021, there were significant (P < .05) increases in the prevalences of 4: current fellow(s), salary, social media links, and operative experience.

Conclusions

Despite a slight increase in accessibility since 2014, nearly half of hand surgery fellowship program websites remain inaccessible from the American Society for Surgery of the Hand directory. Program websites averaged approximately half of the criteria analyzed, with many websites failing to provide information deemed important by applicants.

Clinical relevance

Our study provides an impetus for improving the accessibility and content of hand surgery fellowship websites. A website that incorporates criteria examined in this study can serve as an effective recruitment tool by providing consistent baseline information that may help applicants decide which programs align with personal values and future career goals.

In the Mind of the Ophthalmology Residency Applicant: Deciding Where to Apply, Interview, and Rank

Objective This article characterizes the resources used by ophthalmology residency applicants when deciding where to apply, interview, and rank.

Design Cross-sectional, online survey.

Participants All applicants to the University of California–San Francisco ophthalmology residency program during the 2019 to 2020 and 2020 to 2021 application cycles.

Methods A secure, anonymous, 19-item post-match questionnaire was distributed to participants inquiring about demographic information, match outcomes, and resources used to learn and make decisions about residency programs. Results were analyzed using qualitative and quantitative methods.

Main Outcome Measures Qualitative ranking of resources used to decide where to apply, interview, and rank.

Results One hundred thirty-six of 870 solicited applicants responded to the questionnaire, for a response rate of 15.6%. Digital platforms were ranked as more important resources than people (i.e., faculty, career advisors, residents, and program directors) when applicants were deciding where to apply and interview. Digital platforms became far less important when applicants were formulating their rank lists, at which time the program's academic reputation, perceived happiness of residents and faculty, interview experience, and geographic location were more important. When learning about residency programs, 100% of respondents engaged with program Web sites, and the majority engaged with program emails (n=88 [85.4%]), Doximity (n=82 [79.6%]), Reddit (n=64 [62.1%]), Instagram (n=59 [57.3%]), the FREIDA residency program database (n=55 [53.4%]), and YouTube (n=53 [51.5%]). All 13 digital platforms included in the survey were utilized by at least 25% of respondents, largely passively (i.e., reading rather than producing content). Respondents indicated that the most important topics to include on program Web sites were the number of residents accepted per year, current resident profiles, and resident alumni job/fellowship placement.

Conclusion Applicants engage heavily with digital media in deciding where to apply and interview but rely heavily on their personal experiences with the program in deciding where to rank. Ophthalmology programs may facilitate recruitment of applicants by optimizing their digital media platforms.

Website usability analysis of American anesthesiology residency programs: Original Paper (Preprint)

Background

The Association of American Medical Colleges has recently issued recommendations for the upcoming 2022-2023 application cycle that residency programs should conduct all interviews for this upcoming application cycle over the web. In light of these recommendations, many students will have limited exposure to anesthesiology programs and will rely on information gleaned digitally. This change means that the aspects of program websites used to provide information, such as size, structure, location, requirements, and contact information, will be crucial in helping prospective residents decide where and how to apply in the future. An evaluation of website usability, which includes initial appearance along with factors that influence its ease of navigation and convenience of use, can thus be applied to anesthesiology residency websites. Areas of need can be targeted to increase web presence and provide effective pathways to exhibit the different attributes of their programs to future applicants.

Objective

This study aimed to compile a list of US anesthesiology residency programs and their websites while objectively analyzing the websites using a formally published usability scoring system, as well as to identify positive and negative trends to offer areas of improvement among anesthesiology residency websites.

Methods

We included only 114 US anesthesiology residency program websites in our sample set, since some websites we analyzed showed errors or inconclusive. Website usability was separated into 4 distinct categories for analysis based on methodology outlined in previous literature on both health care website usability and residency website usability. The 4 categories were Accessibility, Marketing, Content Quality, and Technology. Each website was then analyzed and scored based on key components highlighted within the 4 categories. The multiple factors were then graded using a percentage system to create a comprehensive score for each program.

Results

The highest scoring category was Content Quality (mean 4.7, SD 2.48, SE 0.23). The lowest scoring category was Technology (mean 0.9, SD 0.38, SE 0.04).

Conclusions

Through the application of a health care website usability framework, multiple anesthesiology residency programs were analyzed and scored in the areas of Accessibility, Marketing, Content Quality, and Technology, which allowed us to determine the effectiveness of the usability of these websites to convey information to their end user. Websites must communicate vital information, with usability at the forefront, to continue to grow, especially as the United States faces challenges due to the COVID-19 pandemic. Our recommendation is that anesthesiology programs should strive to improve website usability to increase the ease by which applicants can collect vital information about anesthesiology programs. A few proposed solutions include making changes such as decreasing error pages on websites, migrating away from using in-line cascading style sheets, and improving web page loading speeds to improve the Technology category.

Evaluation of Diversity and Inclusion Presence Among US Physical Medicine and Rehabilitation Residency Program Websites

ABSTRACT

The primary aim of this study was to evaluate the presence of diversity and inclusion promotion among US physical medicine and rehabilitation residency program Websites. A list of physical medicine and rehabilitation residency programs was obtained from the Fellowship and Residency Electronic Interactive Database. The program Website links were obtained from the Fellowship and Residency Electronic Interactive Database or Google. Two reviewers evaluated the presence of 11 predetermined diversity and inclusion metrics. The presence of individual resident and faculty photos and biographies was also evaluated. Cross-sectional and descriptive analyses were performed. A total of 90 physical medicine and rehabilitation residency program Websites were reviewed. Only 15 (17%) Websites met at least 1 of the 11 predetermined criteria. Nine (10%) Websites mentioned diversity in their program mission statement, whereas 7 (8%) included a separate diversity statement. Eight (9%) Websites had a dedicated diversity page/section. Of Websites that included information on their residents and faculty, 62 (85%) had completed 100% of resident photos and 41 (47%) Websites had completed 100% of faculty photos. A total of 30 (41%) Websites had completed 100% of resident biographies, and 15 (17%) Websites had completed 100% of faculty biographies. Less than 20% of all residency program Websites promoted diversity and inclusion. Residency Websites lack key information that prospective applicants use to evaluate diversity.

Cosmetic dermatologic surgery fellowship websites and social media presence: Opportunities for improved applicant recruitment

INTRODUCTION

The American Society for Dermatologic Surgery (ASDS) established a cosmetic dermatologic surgery fellowship in 2013. Programs often outline details of fellowships on their websites to help prospective applicants make informed decisions. Our primary goal was to evaluate the content quality of online information for all ASDS-accredited cosmetic dermatologic surgery fellowships on program websites and the ASDS website. Our secondary goal was to describe program activity on social media platforms as another avenue for applicant recruitment.

METHODS

Program websites were assessed using an aggregate score from twenty-one standardized content quality variables. Social media activity on Facebook and Instagram from January 6 2021, to March 6, 2021, was categorized.

RESULTS

Among 24 cosmetic dermatologic surgery fellowship programs, 23 had websites. Basic information was provided across most websites or the ASDS website (eg, address, 95.8%), but more qualitative variables like research opportunities or didactic schedule were not consistently reported. Most programs had highly active social media accounts (91.7% on Facebook and 79.2% on Instagram).

CONCLUSION

There is a gap of information availability between the ASDS website and individual cosmetic dermatologic surgery fellowship websites. Increasing information availability may enhance the applicant recruitment process and serve as a low-cost intervention to ensure optimal fit.

Nationwide Analysis of Integrated and Independent Interventional Radiology Residency Websites During the COVID-19 Pandemic

OBJECTIVES

To analyze current interventional radiology residency program websites based on validated criteria and highlight areas for improvement during the COVID-19 pandemic.

MATERIALS AND METHODS

ACGME-accredited interventional radiology residency programs were identified from the Society of Interventional Radiology (SIR) public database, including 91 independent and 89 integrated programs. Program Webpages were then evaluated during September and October 2020 based on the presence of 48 criteria, organized into seven main categories including visibility & communication, program information, curriculum information, faculty description, research, recruitment, and salary and benefits. Programs were also evaluated based on region and research ranking. Additionally, 166 programs with accreditation for Early Specialization in Interventional Radiology (ESIR) were assessed for the presence or absence of ESIR pathway acknowledgement on program webpages.

RESULTS

The online search yielded information on all integrated programs (89/89, 100%) and 74 independent programs (74/91, 80.3%). For the ESIR programs, the online search for accreditation acknowledgement yielded 108 programs (108/166, 65%) approved for this pathway. Only seven of the 89 integrated programs met at least 75% of the criteria. Of the 91 independent programs, only one met at least 75% of the criteria. On average, integrated programs met more criteria (25, 52%) than independent programs (17, 36%). When comparing programs based on national rank, the visibility & communication category met more criteria on average than the lower ranked programs (integrated =73% vs. 64%, p = 0.01), (independent = 73% vs. 45%, p = 0.01). When comparing programs regionally, statistical significance was found only in the research category (p = 0.01). When comparing the integrated programs with the independent programs for averages in the 7 categories and the total criteria, statistical significance was found in all categories except facility description: visibility & communication (67.5% vs. 53. 7%, p = 0.01), program information (75.7% vs. 58%, p = 0.01), curriculum information (54.8% vs. 31.4, p = 0.01), research (42.5% vs. 27.5%, p = 0.01), recruitment (42.6% vs. 26.8%, p = 0.01), salary & benefits (47.8% vs. 26.8%, p = 0.01), and total criteria (52% vs. 35.8%, p = 0.01.

CONCLUSION

IR residency programs across the country are proficient in providing curricular, and logistical information online. However, improvement is needed in providing nonacademic highlights unique to programs that can aid in maximizing applicant match and compatibility. The information provided by online resources has the potential to influence residency applicant's program ranking and chosen pathway, particularly during the COVID19 pandemic.

Website usability analysis of United States emergency medicine residencies

OBJECTIVES

The Council of Residency Directors (CORD) in Emergency Medicine (EM) has recommended that all residency programs should conduct virtual interviews for the 2020 to 2021 application cycle due to the COVID-19 pandemic. While factors such as geographical region, city, program size, or hospital affiliation are not modifiable, EM residencies can bridge the information gap created by a lack of face-to-face interaction by representing themselves digitally. Measuring usability provides an objective method for EM residencies to improve their Web presence and effectively represent themselves to applicants.

METHODS

Our sample set included 55 U.S. EM residency program websites. Using methodology replicated from previous literature on health care website usability, we divided usability into four categories for quantifiable analysis: accessibility, marketing, content quality, and technology. Analysis was performed on each website and scored in all four categories. A "general usability" score was calculated for each website using a composite of the key factors within the four categories. Using a weighted percentage across all of the factors, an overall score was calculated.

RESULTS

Content quality was the overall highest scoring category (mean ± SD = 5.4, SE = 0.33). The overall lowest performing category was technology (mean ± SD = 0.8 ± 0.09, SE = 0.01).

CONCLUSIONS

Measuring usability can help EM residency programs identify ways to improve their Web presence. To effectively promote their programs, residencies need quality content that communicates their key features. Our recommendation is for all residency programs to periodically perform website audits and apply the usability measures outlined to improve their digital presence, especially during times when face-to-face interactions will be limited.

Evaluation of Adult Neurology Residency Program Websites

OBJECTIVE

Neurology residency program websites often serve as the initial face of a program for prospective residents early in the application process. We evaluated adult neurology residency program websites to determine their comprehensiveness to identify areas for improvement.

METHODS

A list of adult neurology residency programs in the United States was compiled using information on the Electronic Residency Application Service (ERAS) and Fellowship and Residency Electronic Interactive Database (FREIDA) websites. A total of 24 website criteria covering educational, recruitment, and compensation content were assessed for comprehensiveness. Programs' website comprehensiveness was compared based on geographic location, program affiliation (community and/or academic), program size, and program/hospital Doximity and U.S. News & World Report rankings.

RESULTS

A total of 153 US adult neurology residency program websites were evaluated. Fewer than one-half of program websites were accessible with a direct link from either FREIDA or ERAS. The number of residency program websites reporting each content criterion varied greatly. Mean percentage of overall website comprehensiveness among neurology residency programs was 65.9%. Northeast location, academic affiliation, larger programs, and top-ranked programs on Doximity were associated with greater program website comprehensiveness.

INTERPRETATION

There is opportunity for all neurology residency programs to improve their websites to provide prospective applicants with a more informed and comprehensive perspective of programs during the application process. ANN NEUROL 2021;89:637-642.

Improving the online presence of residency programs to ameliorate COVID-19's impact on residency applications

Objectives: The coronavirus disease 2019 (COVID-19) pandemic has created several challenges for residency programs and prospective interns alike during the upcoming application cycle, including the cancellation of away sub-internships and in-person interviews. Given prior research documenting that applicants' application and ranking decisions are significantly influenced by residency webpages, a potential solution to the loss of in-person experiences during the pandemic is the expansion of residency programs' online presence through their program websites, provision of virtual grand rounds and pseudo-away rotations, and enhancement of virtual interviews. This study seeks to summarize the existing literature on these areas and provide concrete suggestions for improving programs' virtual presence.Methods: The authors summarize earlier literature querying the content of program websites across 14 medical specialties, which documented significant gaps in the content of interest to applicants.Results: Among 14 analyzed specialties, the majority of programs had a functional website (>90%), with the exception of interventional radiology (73.9%). However, significant gaps in content were documented, with the percentage of content variables contained on websites ranging from 33.3% to 70.5% (median = 47.0%, interquartile range = 37.8-52.6%). Program websites were also limited by underrepresentation of content most valued by applicants as well as potential areas of inaccurate or outdated information.Conclusions: There are several interventions programs can undertake to address existing gaps in online presence. During an application cycle facing unprecedented resource strain, bolstering the online presence of programs may facilitate an improved fit between programs and future residents.

A Novel Ticket System for Capping Residency Interview Numbers: Reimagining Interviews in the COVID-19 Era

The 2019 novel coronavirus (COVID-19) pandemic has led to dramatic changes in the 2020 residency application cycle, including halting away rotations and delaying the application timeline. These stressors are laid on top of a resident selection process already under duress with exploding application and interview numbers-the latter likely to be exacerbated with the widespread shift to virtual interviewing. Leveraging their trainee perspective, the authors propose enforcing a cap on the number of interviews that applicants may attend through a novel interview ticket system (ITS). Specialties electing to participate in the ITS would select an evidence-based, specialty-specific interview cap. Applicants would then receive unique electronic tickets-equal in number to the cap-that would be given to participating programs at the time of an interview, when the tickets would be marked as used. The system would be self-enforcing and would ensure each interview represents genuine interest between applicant and program, while potentially increasing the number of interviews-and thus match rate-for less competitive applicants. Limitations of the ITS and alternative approaches for interview capping, including an honor code system, are also discussed. Finally, in the context of capped interview numbers, the authors emphasize the need for transparent preinterview data from programs to inform applicants and their advisors on which interviews to attend, learning from prior experiences and studies on virtual interviewing, adherence to best practices for interviewing, and careful consideration of how virtual interviews may shift inequities in the resident selection process.

Comprehensive Evaluation of Primary Care Sports Medicine Fellowship Websites

OBJECTIVE

To evaluate the comprehensiveness of primary care sports medicine fellowship websites and identify potential areas of improvement.

DESIGN

Cross-sectional analysis of fellowship program websites using quantitative and descriptive statistics.

SETTING

Internet.

PARTICIPANTS

A total of 192 primary care sports medicine fellowship websites listed on the Electronic Residency Application Service (ERAS) website.

INDEPENDENT VARIABLES

Program Specialty and Program Region.

MAIN OUTCOME MEASURES

The presence or absence of 19 predetermined criteria on primary care sports medicine fellowship websites.

RESULTS

The average number of criteria that was included on each website was 9.8 (SD 3.5) (51.6%) of the possible 19. Programs had as few as 2 of the 19 (10.5%) criteria included on their website, and others had as many as 17 of the 19 (89.4%) criteria. Of the 192 primary care sports medicine fellowships, only 5 (2.6%) addressed at least 80% of the 19 different criteria. No primary care sports medicine fellowship website included all 19 criteria.

CONCLUSIONS

Most primary care sports medicine fellowship websites do not offer comprehensive information about their programs for prospective applicants.

The Content and Accessibility of Orthopaedic Residency Program Websites

BACKGROUND

Applicants to orthopaedic surgery residency programs face a competitive match. Internet resources such as program websites allow prospective applicants to gauge interest in particular programs. This study evaluated the content and accessibility of orthopaedic surgery residency program websites.

METHODS

Existing orthopaedic surgery residency programs for the 2020 application cycle were identified on the Electronic Residency Application Service (ERAS) website. Individual program websites were accessed through links directly from the ERAS website, and a Google search for each program was performed to corroborate accessibility. Programs websites were then reviewed and evaluated on the presence of 20 criteria selected for their potential to influence resident recruitment (10) and education (10), respectively. The results were compared with the lone 2001 study and with orthopaedic fellowship website analyses.

RESULTS

One hundred eighty-nine orthopaedic surgery residency programs were accredited at the time of the study. Only 6 programs (3.2%) did not have a website identifiable through ERAS or Google searches, leaving a final sample size of 183 websites. Approximately 73.3% of all recruitment content and 44.9% of education content were present on the websites available. There was a significant increase in all available recruitment and education content (p < 0.05) when compared with the lone 2001 study. Orthopaedic residency program websites provide comparable recruitment content at a higher rate (71.1%) than orthopaedic fellowship websites (59.6%) but fall slightly below average in presentation of education content (44.9% vs 45.9%).

CONCLUSION

This is the first study in nearly 20 years to assess the content and accessibility of orthopaedic residency program websites. There is noticeable variability in the presentation of website content, but approximately 73.3% of recruitment content and 44.9% of the educational content were easily accessible through internet search. Orthopaedic surgery residency programs and their applicants may benefit from standardization of program websites and an increase in recruitment and education content.