A Modified Delphi Process to Define Competencies for Assessment Leads Supporting a Doctor of Pharmacy Program

Creating Competencies for Providing Care in a Diverse, Equitable, and Inclusive Manner: A Delphi Technique

OBJECTIVE

Competencies related to diversity could be essential to training and assessing student pharmacists in their readiness to provide equitable and inclusive care. Such competencies are lacking in pharmacy education; therefore, this study aimed to explore the development of diversity competencies and supporting factors needed to prepare students to meet the competencies and provide patient care in a diverse, equitable, and inclusive manner.

METHODS

Pharmacy diversity thought leaders were invited to participate in a 3-round modified Delphi survey. Survey items were created using a Backward Design. Qualitative data were analyzed using the Constant Comparative Method. Draft competency statements were created based on the collective comments from Round 1 along with themes related to the supporting factors needed to achieve the competencies. Consensus on themes, competency statements, and edits were identified in Round 2. Additional comments and feedback on edits were gathered in Round 3. Consensus was preset at 85.7%.

RESULTS

Seven competency statements were created. Knowledge, skills, attitudes, behaviors, values, curricular content/pedagogy, and resources needed to prepare students to meet the competencies and support faculty would need were identified.

CONCLUSION

Preliminary diversity competencies and supporting factors needed were developed using the perspective of thought leaders. Further evaluation, including the development of student-appropriate competencies, testing and validation of content and assessment tools, and determining place in the pharmacy curriculum, are future steps that should be considered in the competency development process.

Best practices for PGY1 letters of intent: Recommendations from a modified Delphi process

PURPOSE

The aim of this study was to provide consensus recommendations from residency program leaders on letters of intent (LOIs) written by postgraduate year 1 (PGY1) pharmacy residency candidates.

METHODS

A 3-round modified Delphi process was used to determine consensus among PGY1 residency program leaders across the country. A screening and demographic survey was utilized to ensure representation of panelists. The initial items for round 1 of the study were developed using existing published literature, with pilot testing by 2 residency program directors. For each round, respondents rated items on a 7-point Likert scale, with opportunities to provide qualitative feedback and modifications for lower-rated items. For future rounds, items were adjusted based on respondent feedback. Only items meeting predefined consensus were included in the final recommendations.

RESULTS

A total of 254 pharmacists were invited to participate in the panel, with 41 completing the demographic and study consent survey. There were 35 participants in round 1, 34 of whom remained for rounds 2 and 3. The panel created 18 LOI recommendations for PGY1 residency candidates. Most recommendations were focused on the content of the LOI, while others were related to formatting.

CONCLUSION

The recommendations from this study can be employed by PGY1 pharmacy residency candidates to enhance their likelihood of success in the residency application process.

Developing public health competency statements and frameworks: a scoping review and thematic analysis of approaches

Competencies ensure public health students and professionals have the necessary knowledge, skills, values, and behaviours to do their jobs effectively. Public health is a dynamic and complex field requiring robust competency statements and frameworks that are regularly renewed. Many countries have public health competencies, but there has been no evidence synthesis on how these are developed. Our research aim was to synthesize the extent and nature of the literature on approaches and best practices for competencies statement and framework development in the context of public health, including identifying the relevant literature on approaches for developing competency statements and frameworks for public health students and professionals using a scoping review; and, synthesizing and describing approaches and best practices for developing public health competency statements and frameworks using a thematic analysis of the literature identified by the scoping review. We conducted a scoping review and thematic analysis of the academic and grey literature to synthesize and describe approaches and best practices for developing public health competency statements and frameworks. A systematic search of six databases uncovered 13 articles for inclusion. To scope the literature, articles were assessed for characteristics including study aim, design, methods, key results, gaps, and future research recommendations. Most included articles were peer-reviewed journal articles, used qualitative or mixed method design, and were focused on general, rather than specialist, public health practitioners. Thematic analysis resulted in the generation of six analytical themes that describe the multi-method approaches utilized in developing competency statements and frameworks including literature reviews, expert consultation, and consensus-building. There was variability in the transparency of competency framework development, with challenges balancing foundational and discipline-specific competencies. Governance, and intersectoral and interdisciplinary competency, are needed to address complex public health issues. Understanding approaches and best practices for competency statement and framework development will support future evidence-informed iterations of public health competencies.

Consensus recommendations for the treatment of migraine prevention

OBJECTIVE

The objectives are to know the opinion of neurologists and hospital pharmacists on those aspects still under debate regarding the role of anti-CGRP monoclonal antibodies in the preventive treatment of migraine. To identify those controversies that still exist. To propose agreed recommendations for improvement of care. And to promote access of clinicians and patients to these new treatments in the prevention of migraine with biological drugs, in order to improve patient care and follow-up.

METHODOLOGY

Recommendations for the use of biological drugs in the prevention of migraine were identified and evaluated through the Delphi consensus methodology, proposing 88 statements grouped into three themes: a clinical module that deals with the management of biological treatments in migraine; a patient module that discusses patient education and adherence improvement strategies; and a coordination module that includes statements related to strategies to improve joint work between the two groups. The 9-point Likert ordinal scale was used to score these recommendations and, subsequently, the data was statistically analyzed through different metrics.

RESULTS

After both rounds of voting, consensus was reached in agreement on 71 of the 88 statements (80.7%), leaving one statement (1.1%) with consensus in disagreement and 16 remaining as indeterminate (18.2%).

CONCLUSIONS

The high degree of consensus indicates that the opinion of neurologists and hospital pharmacists on the role of anti-CGRP monoclonal antibodies in the treatment of migraine is very similar and allows identifying those controversies that still exist, to improve the care and follow-up of patients with migraine.

[Translated article] Consensus recommendations on the preventive treatment of migraine

OBJECTIVE

The objectives are to know the opinion of neurologists and hospital pharmacists on those aspects still under debate regarding the role of anti-Calcitonin Gene-related Peptide monoclonal antibodies in the preventive treatment of migraine. To identify those controversies that still exist. To propose agreed recommendations for improvement of care. And to promote access of clinicians and patients to these new treatments in the prevention of migraine with biological drugs, in order to improve patient care and follow-up.

METHODOLOGY

Recommendations for the use of biological drugs in the prevention of migraine were identified and evaluated through the Delphi consensus methodology, proposing 88 statements grouped into 3 themes: a clinical module that deals with the management of biological treatments in migraine; a patient module that discusses patient education and adherence improvement strategies; and a coordination module that includes statements related to strategies to improve joint work between the two groups. The 9-point Likert ordinal scale was used to score these recommendations and, subsequently, the data was statistically analysed through different metrics.

RESULTS

After both rounds of voting, consensus was reached in agreement on 71 of the 88 statements (80.7%), leaving 1 statement (1.1%) with consensus in disagreement and 16 remaining as indeterminate (18.2%).

CONCLUSIONS

The high degree of consensus indicates that the opinion of neurologists and hospital pharmacists on the role of anti-Calcitonin Gene-related Peptide monoclonal antibodies in the preventive treatment of migraine is very similar and allows identifying those controversies that still exist, to improve the care and follow-up of patients with migraine.

Guidance for Qualitative Research Manuscripts in Pharmacy Education

The goal of this Best Practice Review is to support researchers in successfully preparing and publishing qualitative research in pharmacy education. Standard practice from the literature and journals' guidance from related fields were reviewed, and recommendations and resources applicable to qualitative research in pharmacy education were compiled for researchers planning to conduct and publish qualitative research. This review provides recommendations, not requirements, for publication in the Journal and is intended to be a guide, especially for authors and reviewers relatively new to the field of qualitative research. Additionally, researchers planning to publish their qualitative research are advised to review available best practices and standards, such as the Consolidated Criteria for Reporting Qualitative Research checklist and the Standards for Reporting Qualitative Research. Given the diverse methodology of qualitative research, it is important for authors to provide sufficient details and justifications of selected methods for transparency and to report collected results in a manner that allows reviewers and readers to adequately assess the validity of their study and the applicability of the findings.

Trends in the Number of Authors and Institutions in Papers Published in AJPE 2015-2019

Objective. To investigate the number of authors and unique institutions per paper published in the American Journal of Pharmaceutical Education (AJPE) in 2015 through 2019, and to examine the number of authors and unique institutions for papers that were nominated for the Rufus A. Lyman Award in the same period.Methods. Articles published in AJPE from 2015 through 2019 were reviewed. Data collected for each article included article type, number of authors, and number of institutions.Results. Of the 811 articles published in AJPE during this period, the number of authors increased significantly from a mean (SD) of 3.5 (1.8) to 4.5 (2.2). The number of unique institutions also increased significantly from 1.7 (1.1) to 2.4 (1.8).Conclusion. There is a trend toward a greater number of authors and unique institutions for the publications in one pharmacy education journal. Explanations for this trend may include pressure to publish, increased research complexity, and expanded interprofessional collaboration.

Establishing the Content Validity of a Student Pharmacist Patient Counseling Competency Assessment in Oncology

Objective. The goal of this project was to establish content validity and describe internal consistency of a patient counseling competency assessment instrument used to evaluate student pharmacists practicing in an oncology setting.Methods. The study involved a modified e-Delphi panel of oncology clinical pharmacy specialists, clinical pharmacy generalists, and oncology pharmacy residents. Iterative rounds of the e-Delphi process were conducted until consensus was reached on most instrument items. Consensus was defined as agreement by at least 75% of participants that an item was or was not important.Results. The modified e-Delphi process included three rounds of responses from 13 panelists and resulted in a 35-item instrument with consensus reached on 33/35 (94%) of the items. All participants indicated that the assessment result options allowed them to indicate the student's level of competency either extremely well or very well.Conclusion. A modified e-Delphi method was used to validate a reliable instrument for the assessment of student pharmacist counseling abilities in an oncology setting. Similar methodology should be considered during the development of student assessment tools, especially for high-impact student pharmacist activities such as chemotherapeutic medication counseling.

Dentists as Primary Care Providers: Expert Opinion on Predoctoral Competencies

Dentistry and medicine traditionally practice as separate professions despite sharing goals for optimal patient health. Many US residents experience both poor oral and general health, with difficulty accessing care. More efficient collaboration between these professions could enhance health. The COVID-19 pandemic disclosed further disparities while underscoring concerns that physician supply is inadequate for population needs. Hence, enhancing healthcare provider education to better meet the public's health needs is critical. The proposed titles “Oral Physician” or “Oral Health Primary Care Provider” (OP-PCP) acknowledge dentist's capacity to diagnose and manage diseases of the orofacial complex and provide some basic primary healthcare. The US Surgeon General's National Prevention Council and others recommend such models. Medical and dental education already overlap considerably, thus it is plausible that dental graduates could be trained as OP-PCPs to provide primary healthcare such as basic screening and preventive services within existing dental education standards. In 2018, 23 dental and medical educators participated in an expert-opinion elicitation process to review educational competencies for this model. They demonstrated consensus on educational expansion and agreed that the proposed OP-PCP model could work within existing US Commission on Dental Accreditation (CODA) standards for predoctoral education. However, there were broader opinions on scope of practice details. Existing CODA standards could allow interested dental programs to educate OP-PCPs as a highly-skilled workforce assisting with care of medically-complex patients and to helping to reduce health disparities. Next steps include broader stakeholder discussion of OC-PCP competencies and applied studies including patient outcome assessments.

A Modified Delphi Involving Laboratory Faculty to Define Essential Skills for Pharmacy Graduates

Objective. To define essential skills for Doctor of Pharmacy (PharmD) graduates that are needed in the four most common sectors of pharmacy practice as determined by expert faculty who instruct within pharmacy skills laboratories.Methods. A three-round Delphi method was used to establish consensus. In the first round, participants were asked what skills were needed by students at entry to practice in community, health-system, ambulatory care, and managed care pharmacy settings. In rounds two and three, participants were asked to rate each skill with a level of importance using a 10-point Likert scale (1=not important to 10=very important).Results. In round one, participants produced a collective list of 289 essential skills. These skill statements were sent to participants in rounds two and three to assign a level of importance. After the third round, participants reached consensus using a mean level of importance for a final list of 69 community pharmacy skills, 47 health-system, 60 ambulatory care, and 15 managed care skills. These skills were then mapped to entrustable professional activities domains for schools and colleges pharmacy to use as a resource when assessing core competency development in the curriculum.Conclusion. The Delphi technique was used successfully with expert pharmacy skills laboratory faculty to identify laboratory-focused essential skills that recent PharmD graduates should have prior to entering community, health-system, ambulatory care, or managed care pharmacy practice. These essential skills can be used to guide curriculum development, develop milestone markers, and help ensure students are practice ready.

A key performance indicators redefinition initiative at a school of pharmacy using a modified Delphi consensus technique

Objective:

The Outcomes and Assessment Committee at the Virginia Commonwealth University School of Pharmacy was tasked with refining the school’s key performance indicators (KPIs) to improve programmatic assessment by focusing on the most important measures.

Methods:

Initially, 56 KPIs were tracked, nine of which were university mandated, divided into 10 modules: admissions, community outreach, continuing education, diversity, faculty experience and success, fundraising, graduate program, research and scholarship, staff experience and success, and student experience and success. Using a three-round Delphi consensus technique, KPIs were reviewed by faculty and staff. Each participant responded whether they considered each KPI to be essential or not essential for school quality assessment and improvement. Consensus for the first, second, and third rounds was defined as ≥90%, ≥80%, and ≥75% agreement, respectively.

Results:

Of 109 faculty and staff invited, 49 participated in the first round, 51 in the second, and 42 in the third. At the end of the third round, accumulated consensus was achieved for 35 out of 88 (39.8%) KPIs that were considered essential and 3 out of 88 (3.4%) that were considered non-essential. Consensus percentage per module was: 15.4% (2/13) admissions, 28.6% (2/7) community outreach, 33.3% (3/9) continuing education, 27.3% (3/11) diversity, 62.5% (5/8) faculty experience and success, 55.6% (5/9) fundraising, 40% (4/10) graduate program, 33.3% (3/9) research and scholarship, 57.1% (4/7) staff experience and success, and 66.7% (4/6) student experience and success.

Conclusions:

Ultimately, 35 KPIs achieved consensus as essential to measure achievement of benchmarks for the school, which totals 44 KPIs, including nine university mandated KPIs. The process facilitated faculty and staff involvement in KPI selection and achieved improved focus for programmatic assessment.

Development of an evidence-based residency preparation checklist using a Delphi process

PURPOSE

To develop an evidence-based tool that will provide concise guidance to pharmacy students who want to become competitive postgraduate year 1 (PGY1) residency applicants.

METHODS

A systematic literature search was conducted to identify articles describing student or school factors and specific interventions or activities associated with improved or decreased residency match rates, as well as studies describing residency program directors' (RPDs') or preceptors' perceptions of qualified applicants. An initial checklist was developed, with an item for each relevant factor. A consensus on checklist items was built through a 2-round Delphi process with a panel of RPDs. Ultimately, items that received a median score of at least 5 on a 7-point scale with less than one-third of the ratings being a 1, 2, or 3 were included.

RESULTS

The initial checklist of 34 items, primarily related to grade point average, professional involvement, work experience, or professional development, was evaluated by a panel of 25 RPD participants. Six of 34 items (18%) were reevaluated in round 2, along with 1 added item and 4 items substantively modified based on comments; 2 items were merged. Ultimately, 33 items met the criteria for consensus and were included in the final checklist.

CONCLUSION

A checklist of items to guide prospective pharmacy residency applicants was developed through a systematic literature search and verified by program directors using a Delphi process.

Identifying High-Impact and Managing Low-Impact Assessment Practices

Those in pharmacy education who are tasked with assessment may be overwhelmed by deadlines, data collection, and reporting, leaving little time to pause and examine the effectiveness of their efforts. However, assessment practices must be evaluated for their impact, including their ability to answer important questions, use resources effectively, and contribute to meaningful educational change. Often assessments are implemented, but then attention is diverted to another assessment before the data from the former assessment can be fully interpreted or used. To maximize the impact of assessment practices, tough and uncomfortable decisions may need to be made. In this paper, we suggest an approach for examining and making decisions about assessment activities and provide guidance on building high-impact assessment practices, evolving or "sunsetting" low-impact assessment practices, and managing mandated assessment.

Surveying the current landscape of assessment structures and resources in US schools and colleges of pharmacy

INTRODUCTION

Expectations for assessment in higher education have increased in recent decades, prompting institutions to invest additional resources in this area. This study aimed to determine the resources, structure, and perception of assessment resources in United States schools and colleges of pharmacy (S/COPs).

METHODS

Assessment personnel in S/COPs were surveyed electronically. Information collected included S/COP demographics, composition of assessment positions, experience and training of assessment personnel, and structure and responsibilities of committees engaged in assessment. Respondents' perception of their S/COPs having sufficient assessment personnel, recent changes in assessment, and the factors that prompted assessment changes were also surveyed.

RESULTS

Respondents included individuals from 113 S/COPs (84% response rate). Most S/COPs had 1-2 assessment positions and 1-2 assessment-related committees. The most common assessment position titles were assistant/associate dean, director, coordinator/specialist, and administrative assistant. Dean-level administrators typically had worked in assessment the longest, whereas directors were more likely to have formal assessment training. Most respondents (75%) agreed they had sufficient assessment personnel to meet the 2007 Accreditation Council for Pharmacy Education Standards. Nearly two-thirds of respondents agreed they had sufficient personnel to meet the 2016 Standards and support their current assessment plan/process. Most S/COPs had a formal assessment committee (93%) and an average of two committees overseeing assessment.

CONCLUSION

Most S/COPs reported having sufficient resources to support assessment activities. Although there were some consistent themes, there does not appear to be a single model for structuring assessment resources or committees. Effectiveness of various assessment structures represents an area for future research.

Use of a Modified Delphi Process to Determine Course Objectives for a Model Global Health Course in a Pharmacy Curriculum

Objective. To define course objectives for a model global health course in a pharmacy curriculum. Methods. A modified Delphi process was used to determine a consensus among proposed course objectives. A three-round email panel was sent to members of three special interest groups (SIGs) within the American Association of Colleges of Pharmacy (Public Health, Global Pharmacy Education, and Health Disparities and Cultural Competence) to recruit educators broadly interested or engaged in this area of education. An initial list of 80 potential course objectives across 11 domains was proposed for inclusion. Objectives that were cumulatively rated as either "extremely important" or "very important" by at least 75%, 80%, and 85% of respondents in each of the three rounds, respectively, were moved forward (first and second rounds) or accepted (third round). Results. Responses were received from 87, 73, and 70 faculty panel members in the three consecutive rounds. The initial list of proposed objectives was narrowed to 65 objectives (19% reduction), and 38 objectives (53%) after the first and second rounds, respectively. The final list was composed of 20 objectives from seven domains. Global burden of disease and social/environmental determinants of health contained the most objectives selected by consensus. Conclusion. The process identified a consensus for course objectives for a model global health education course. These objectives can be used by pharmacy faculty to align global health education in the profession.